Scaling your training
Warning
To scale your training or run more than 1 simulation model at a time (in parallel) you will need an exported version of your AnyLogic model. In order for you to export the model, you MUST have a valid AnyLogic Professional license.
If you are using the rllib
package, you can define how many simulation instances you want to execute in parallel by tunning the rollouts parameters:
from alpyperl.examples.cartpole_v0.cartpole_env import CartPoleEnv
from ray.rllib.algorithms.ppo import PPOConfig
# Initialise policy configuration (e.g. PPOConfig), rollouts and environment
policy = (
PPOConfig()
.rollouts(
num_rollout_workers=2,
num_envs_per_worker=5,
)
.environment(
CartPoleEnv,
env_config={
'run_exported_model': False,
'exported_model_loc': './resources/exported_models/cartpole_v0'
'show_terminals': False
}
)
.build()
)
# Create training loop
for _ in range(10):
result = policy.train()
# Save policy at known location
checkpoint_dir = policy.save("./resources/trained_policies/cartpole_v0")
print(f"Checkpoint saved in directory '{checkpoint_dir}'")
# Close all enviornments (otherwise AnyLogic model will be hanging)
policy.stop()
The total number of simulation instances will be defined by num_rollout_workers x num_envs_per_worker
.
In addition to that, you must enable 'run_exported_model': True
in the environment and point to the exported model folder 'exported_model_loc': './resources/exported_models/cartpole_v0'
in order for the parallel execution to take place.