# ROS2Learn Usage

### Train an agent

You will find all available examples at /experiments/examples/. Although the algorithms are complex, the way to execute them is really simple. For instance, if you want to train MARA robotic arm using ppo2_mlp, you should execute the following command:

cd ~/ros2learn/experiments/examples/MARA
python3 train_ppo2_mlp.py

Note that you can add the command line arguments provided by the environment, which in this case are provided by the gym-gazebo2 Env. Use -h to get all the available commands.

If you want to test your own trained neural networks, or train with different environment form gym-gazebo2, or play with the hyperparametes, you must update the values of the dictionary directly in the corresponding algorithm itself. For this example, we are using ppo2_mlp from baselines submodule, so you can edit the mara_mpl() function inside baselines/ppo2/defaults.py.

### Run a trained policy

Once you are done with the training, or if you want to test some specific checkpoint of it, you can run that using one of the running-scripts available. This time, to follow with the example, we are going to run a saved ppo2_mlp policy.

First, we will edit the already mentioned mara_mpl() dictionary, in particular the trained_path value, in baselines/ppo2/defaults.py to the checkpoint we want (checkpoints placed by default in /tmp/ros2learn). Now we are ready to launch the script.

Since we want to visualize it in real conditions, we are also going to set some flags:

cd ~/ros2learn/experiments/examples/MARA
python3 run_ppo2_mlp.py -g -r -v 0.3

This will launch the simulation with the visual interface, real time physics (no speed up) and 0.3 rad/sec velocity in each servomotor.

### Visualize training data on tensorboard

The logdir path will change according to the used environment ID and the used algorithm in training. Now you just have to execute Tensorboard and open the link it will provide (or localhost:port_number) in your web browser. You will find many useful graphs like the reward (eprewmean) plotted there.

You can also set a specific port number in case you want to visualize more than one tensorboard file from different paths.

tensorboard --logdir=/tmp/ros2learn/MARACollision-v0/ppo2_mlp --port 8008