Welcome to ALPypeRL documentation!

ALPypeRL or AnyLogic Python Pipe for Reinforcement Learning is an open source library for connecting AnyLogic simulation models with reinforcement learning frameworks that are compatible with OpenAI Gymnasium interface (single agent).

Important

No license is required for single instance experimentation. AnyLogic PLE is free! Download it from here.

Note

ALPypeRL has been developed using ray rllib as the base RL framework. ray rllib is an industry leading open source package for Reinforcement Learning that offers lots of interesting features. Because of that, ALPypeRL has certain dependencies to it (e.g. trained policy deployment and evaluation).

General:

• ALPypeRL
  • Features
  • Environments
  • Installation
  • Requirements
  • General architecture
  • Reinforcement learning basics
• The AnyLogic Connector
  • Add ALPypeRLConnector library to your AnyLogic Palette
  • Drag and drop an instance of ALPypeRLConnector
  • Implement ALPypeRLClientController
  • Call requestAction when the RL agent in the simulation requires a new action
• How to train your first policy. The CartPole-v0 example.
  • Create the Action and Observation spaces
  • [Optional] Wrap your CustomEnv around the BaseAnyLogicEnv
  • Policy configuration and training execution
  • Track your training progress using tensorboard
  • The CartPole-v0 implementation
• How to define a space
  • 1. Using the ALPypeRLConnector in AnyLogic
  • 2. Using the BaseAnyLogicEnv in python
• How to set continuous actions. The CartPole-v1 example.
• How to set an array of continuous actions. The CarPole-v2 example.
• How to set an array of mixed actions. The CarPole-v3 example.
• Evaluating your trained policy
  • Launch your trained policy server
  • Set your ALPypeRLConnector mode to EVALUATE
• Simulation randomness and how to handle failed runs
• How to set simulation parameter values from your training script

Advanced:

• Running your model directly from AnyLogic
• Running alpyperl in a docker container
• Scaling your training
• Running your simulation model as an AnyLogic CustomExperiment
• Most common issues and how to troubleshoot them
• API

Indices and tables

• Index

• Module Index

• Search Page