You will find all available examples at /experiments/examples/. Although the algorithms are complex, the way to execute them is really simple. For instance, if you want to train MARA robot using ppo2_mlp, you should execute the following command:
cd ~/ros2learn/experiments/examples/MARA python3 train_ppo2_mlp.py
Note that you can add the command line arguments provided by the environment, which in this case are provided by the gym-gazebo2 Env. Use
-h to get all the available commands.
If you want to test your own trained neural networks, or train with different environment form gym-gazebo2, or play with the hyperparametes, you must update the values of the dictionary directly in the corresponding algorithm itself. For this example, we are using ppo2_mlp from baselines submodule, so you can edit the
mara_mpl() function inside baselines/ppo2/defaults.py.
Once you are done with the training, or if you want to test some specific checkpoint of it, you can run that using one of the running-scripts available. This time, to follow with the example, we are going to run a saved ppo2_mlp policy.
First, we will edit the already mentioned
mara_mpl() dictionary, in particular the
trained_path value, in baselines/ppo2/defaults.py to the checkpoint we want (checkpoints placed by default in /tmp/ros2learn). Now we are ready to launch the script.
Since we want to visualize it in real conditions, we are also going to set some flags:
cd ~/ros2learn/experiments/examples/MARA python3 run_ppo2_mlp.py -g -r -v 0.3
This will launch the simulation with the visual interface, real time physics (no speed up) and 0.3 rad/sec velocity in each servomotor.
The logdir path will change according to the used environment ID and the used algorithm in training. Now you just have to execute Tensorboard and open the link it will provide (or localhost:port_number) in your web browser. You will find many useful graphs like the reward (eprewmean) plotted there.
You can also set a specific port number in case you want to visualize more than one tensorboard file from different paths.
tensorboard --logdir=/tmp/ros2learn/MARACollision-v0/ppo2_mlp --port 8008