Add official Clover simulation config (#254)

* clover_description: Add preliminary configs/models

* clover_description: Use proper models for the drone

* clover_description: Be more specific about spawn arguments

* clover_description: Tweak parameters a bit, add collision boxes

* travis: Add .dae files to the list of ignored by eclint

* Add clover_simulation package

* clover: Add Gazebo plugin sources

* builder: Ignore clover_gazebo_plugins for actual drone

* clover_gazebo_plugins: Expose include directories for plugins

This should fix building the unit tests

* clover_gazebo_plugins: Remove dependency on gazebo_ros

This should prevent RPi image failing to build.

* travis, gitattributes: Mark clover_gazebo_plugins as vendored, stop checks

* clover_simulation: Minor package.xml fix

* clover_description: Add IMU joint preservation

Oh, Gazebo, you are ever so very helpful, it's hard to put my appreciation into words! If not for your helpful model simplification, I wouldn't have spent two hours looking through the plugin sources, the urdf sources, trying lots of
different options for the joints and links, and finally getting an answer from GazeboOverflow or however you've named your Q&A site. How wonderful it is to have an issue that makes you tear your hair out just because you know
what's better for me!

* clover_simulation: Add the bare necessities to run a simulation

* clover_gazebo_plugins: Prevent gazebo from trying to download models

For some reason the models are no longer available, so Gazebo just spends some time waiting for a timeout.

* clover_gazebo_plugins: Update Gazebo model database URI

* clover_simulation: Add script to find and launch PX4

* clover_simulation: Fix launch file

* clover_description: Add missing plugins

* simulation: Re-enable gazebo_ros dependencies

This will force rosdep to try to install gazebo_ros on the drone,
but this can be counteracted by --skip-keys rosdep option.
This does not look reliable, but I could not come up with a better
solution.

* builder: Be more resilient about apt-get errors

* builder: Remove reference to resolve_rosdep

* clover_description: Update Clover model, change xacro description

Previous xacro description file was not performing too well, so I went with
a more sensible route and started changing iris.xacro to use our Clover model.

* clover_description: Bring back constants.xacro

* clover_description: Prevent lumping for camera link/joint

* clover_description: Move near clipping plane further away

* clover_description: Allow setting width/height for rpi_cam

* clover_description: Add a Clover model with a camera

* clover_description: Remove whitespaces

* clover_description: Add drone+camera spawning .launch file

* clover_simulation: Add gazebo_ros here as well

* clover_simulation: Spawn drone with camera by default

* clover_simulation: Allow specifying data path for px4

* clover_simulation: Add startup scripts from px4

Big TODO: Clean them up eventually

* clover_simulation: Use local data files

* clover_simulation: Launch clover services by default

* clover_description: Depend on gazebo_plugins as well

libgazebo_ros_camera is in gazebo_plugins, so we need that package.

* clover_description: Fix camera_sensor description

* clover_description: Fix typo in package.xml

* clover_simulation: rename sim_gazebo.launch to simulator.launch

* clover_simulation: Don't look for ROMFS in px4_source_path

We provide our own, no reason to fail if we can't find the originals.

* clover_simulation: Remove extra CMakeLists.txt

* clover_description: Use xacro: namespace for xacro macros

* clover_description: Fix package.xml formatting

* clover_description: Better camera defaults

* clover_description: Add distance sensor

* clover_description: Add leg colliders

* clover_simulation: Actually forward vehicle name

* clover_description: Revert adding additional colliders

Unfortunately, this breaks physics too much

* clover_description: Tweak drone physics, make it more bouncy

* clover_description: Don't spawn the drone inside the floor

* clover_description: Set rangefinder min range outside drone collider

* clover_simulation: Set default flow parameters for Clover

* clover_description: Update Clover 4 model

* clover_simulation: rename sim_gazebo.launch to simulator.launch (#233)

* clover_simulation: Add workaround for Gazebo crashes in VMware

* clover_simulation: Ignore .git for now

* clover: Add "simulated" argument

* clover_simulation: Start Gazebo early

* clover_gazebo_plugins: Remove unused files

* clover_description: Allow turning sensors on and off

* clover_description: Fix rangefinder creation

* Remove unneeded stuff and use PX4 from catkin workspace

* Remove clover_gazebo_plugins

* Rename arg simulated to simulator

* clover: Change target names to avoid clashing with PX4

* Fix

* clover_simulation: Re-add deleted comments

* Add loop model

* loop.material: use tabs instead of spaces

* loop model: don’t rotate by yaw

* loop.material: turn on alpha_blend

* Rename model loop to loop_line

* Add parquet plane model

* loop_line: fix description

* Set alpha_blend for loop_solid material

* Add square line model

* Add CATKIN_IGNORE to models directory

* Add LED strip Gazebo model

* Add hardcoded URDF LED strip

* clover_description: Add LED xarco model

clover_simulation: Implement LED visual plugin and controller

* clover_simulation: Make led plugin less chatty

* clover_simulation: Depend on led_msgs

This should allow the packages to be built in the proper order.

* clover_simulation: Support building against Kinetic

* clover_simulation: Don't build plugins if Gazebo is not installed

* clover_description: Get rid of "constants" file

* clover_description: Add README

* clover_simulation: Add README

* clover_simulation: Make parquet thicker

Otherwise the rangefinder beam goes right through it.

* docs: Start working on simulation articles

* docs: Start working on the simulation overview (en)

* Add launch-file for PX4 v1.8.2

* clover_simulation: Disable GPS, use EKF2 by default

Ideally we should be using LPE, but it is broken in PX4 1.10, and our need for a somewhat working simulator is higher than for a completely correct one.

* clover_simulation: Add experimental throttling camera

* clover_simulation: Add note about throttling camera

* clover_description: Remove unused file

* clover_simulation: Link against CameraPlugin

* clover_description: Add option to use throttling_camera

* Add clover.world

* clover_description: Add calculated inertial parameters

* simulator: change default world to clover.world

* clover_simulation: Start working on ArUco generation script

Port over aruco_gazebo_gen, add more options.
Does not modify the world right now.

* clover_simulation: Make LED plugin less chatty

* clover_description: Be more ROS-like in script naming

* clover_simulation: Implement model insertion to the world

* clover_simulation: Allow specifying output model dir

* clover_description: Don't use throttling camera by default

throttling camera is still a work-in-progress, there's no reason to
enable it by default.

* clover_simulation: Use proper script name in CMakeLists

This is what typically happens when I'm rushed.

* docs: Add instructions for VM setup (en)

* clover_simulation: Remove extra spaces

* docs: Describe simulation usage (en)

* clover_simulation: Remove led_strip

* docs/assets: Crunch sim image a bit

* clover: Bump VL53L1X version

For some reason, 0.0.2 is not installable on x86.

* docs/simulation: Fix capitalization

* Remove remnants of clover_gazebo_plugins

* Remove unneeded Clover 3 model

* Remove empty.world and asphalt_plane model

* Remove unused LED strip model

* Reduce images size

* Shortened simulator related urls

Co-authored-by: Oleg Kalachev <okalachev@gmail.com>

docs/en/SUMMARY.md

@@ -47,6 +47,11 @@
   * [Using JavaScript](javascript.md)
   * [ROS](ros.md)
   * [MAVROS](mavros.md)
+  * Simulation
+    * [Overview](simulation.md)
+    * [Native setup](simulation_native.md)
+    * [VM setup](simulation_vm.md)
+    * [Usage](simulation_usage.md)
 * Supplementary materials
   * [COEX Pix](coex_pix.md)
   * [Guide on autonomous flight](auto_setup.md)

docs/en/simulation.md

@@ -0,0 +1,32 @@
+# Simulation overview
+
+The Clover simulation environment allows users to run and debug their code within a simulator while using most of the features available on the real drone. The simulation utilizes [PX4 SITL mode](sitl.md) and uses the same ROS code as the real drone. Most hardware is simulated as well.
+
+## Features
+
+Basic, user-installable environment includes:
+
+* high-quality Clover 4 visual model;
+* Gazebo plugins for Clover-specific hardware (e.g. LED strip);
+* modification-friendly [`xacro`](https://wiki.ros.org/xacro) drone descriptions;
+* sample models and worlds;
+* [`roslaunch`](https://wiki.ros.org/roslaunch) files for quick simulation launching and configuration.
+
+Additionally, a [virtual machine image](simulation_setup_vm.md) that mimics the real drone as closely as possible is provided with the following features:
+
+* easy access to the simulation environment;
+* Visual Studio Code editor, preconfigured to work with ROS packages;
+* Monkey web server for web-based Clover plugins;
+* always-running `roscore` service;
+* ROS GUI tools (`rviz`, `rqt`).
+
+## Architecture
+
+The simulation environment is based on the following components:
+
+* [Gazebo](http://gazebosim.org/), a state-of-the-art robotics simulator;
+* [PX4](https://px4.io/), specifically its SITL (software-in-the-loop) components;
+* [`sitl_gazebo`](https://github.com/PX4/sitl_gazebo) package containing Gazebo plugins for PX4;
+* ROS packages and Gazebo plugins.
+
+<!-- TODO: Write more, add a diagram, etc -->

docs/en/simulation_native.md

@@ -0,0 +1,84 @@
+# Native setup
+
+Setting up the simulation environment from scratch requires some effort, but results in the most performant setup, with less chance of driver issues.
+
+Prerequisites: Ubuntu 18.04, [native ROS installation](ros-install.md).
+
+## Create a workspace for the simulation
+
+Throughout this guide we will be using the `catkin_ws` as the workspace name. Feel free to change it in your setup. We will be creating it in the home directory of the current user (`~`).
+
+Create the workspace and clone Clover sources:
+
+```bash
+mkdir -p ~/catkin_ws/src
+cd ~/catkin_ws/src
+git clone https://github.com/CopterExpress/clover -b clover_description
+git clone https://github.com/CopterExpress/ros_led
+```
+
+Install all prerequisites using `rosdep`:
+
+```bash
+cd ~/catkin_ws
+rosdep update
+rosdep install --from-paths src --ignore-src -y
+```
+
+## Get PX4 sources
+
+PX4 will be built along with the other packages in our workspace. You may clone it directly into the workspace or put it somewhere and symlink to `~/catkin_ws/src`. We will need to put its `sitl_gazebo` submodule in `~/catkin_ws/src` as well. For simplicity's sake we will clone the firmware directly to the workspace:
+
+```bash
+cd ~/catkin_ws/src
+git clone --recursive https://github.com/CopterExpress/Firmware -b v1.10.1-clever
+ln -s Firmware/Tools/sitl_gazebo ./sitl_gazebo
+```
+
+## Install PX4 prerequisites
+
+PX4 comes with its own script for dependency installation. We may as well leverage it:
+
+```bash
+cd ~/catkin_ws/src/Firmware/Tools/setup
+sudo ./ubuntu.sh
+```
+
+This will install everything required to build PX4 and its SITL environment.
+
+You may want to skip installing the ARM toolchain if you're not planning on compiling PX4 for your flight controller. To do this, use the `--no-nuttx` flag:
+
+```
+sudo ./ubuntu.sh --no-nuttx
+```
+
+## Install geographiclib datasets
+
+`mavros` requires geographiclib datasets to be present:
+
+```bash
+cd ~
+wget https://raw.githubusercontent.com/mavlink/mavros/6f5bd5a1a67c19c2e605f33de296b1b1be9d02fc/mavros/scripts/install_geographiclib_datasets.sh
+sudo ./install_geographiclib_datasets.sh
+rm ./install_geographiclib_datasets.sh
+```
+
+## Build the simulator
+
+With all dependencies installed, you can build your workspace:
+
+```bash
+cd ~/catkin_ws
+catkin_make
+```
+
+> **Note** Some of the files - particularly Gazebo plugins - require large amounts of RAM to be built. You may wish to reduce the number of parallel jobs; the number of parallel jobs should be equal to the amount of RAM in gigabytes divided by 2 - so a 16GB machine should use no more than 8 jobs. You can specify the number of jobs using the `-j` flag: `catkin_make -j8`
+
+## Run the simulator
+
+In order to be sure that everything was built correctly, try running the simulator for the first time:
+
+```bash
+source ~/catkin_ws/devel/setup.bash
+roslaunch clover_simulation simulator.launch
+```

docs/en/simulation_usage.md

@@ -0,0 +1,49 @@
+# Using the simulator
+
+The Clover simulation environment allows the user to test their code without any risk of equipment damage. Additionally, the [virtual machine](simulation_setup_vm.md)-based environment has additional (non-ROS) services that are present on a real drone, like Monkey web server.
+
+## Running the simulation
+
+After [setting up the simulation packages](simulation_setup_native.md) or [importing and running the VM](simulation_setup_vm.md), you can use `roslaunch` to start Gazebo simulation:
+
+```bash
+# Be sure to activate your workspace first
+source ~/catkin_ws/devel/setup.bash
+roslaunch clover_simulation simulator.launch
+```
+
+> **Note** Alternatively, if you are using the VM, just double-click on the `Gazebo PX4` icon on the desktop.
+
+This will launch Gazebo server and client, the PX4 SITL binary and Clover nodes. The terminal in which the command was run will display diagnostic messages from the nodes and PX4, and will accept input for the PX4 command interpreter:
+
+![Gazebo simulation screenshot](../assets/simulation_usage/01_running_gazebo.jpg)
+
+You can use QGroundControl to configure the simulated drone parameters, plan missions (if GPS is simulated) and control the drone using a joystick:
+
+![Gazebo and QGC](../assets/simulation_usage/02_gazebo_qgc.jpg)
+
+You can also use [our simplified OFFBOARD control](simple_offboard.md) to control the drone, and traditional ROS GUI utilities like [rviz and rqt](rviz.md) to analyze the drone state:
+
+![Gazebo and RQT](../assets/simulation_usage/03_gazebo_rqt.jpg)
+
+## Configuring the simulation
+
+The simulation can be configured by passing additional arguments to the `roslaunch` command or by changing the `~/catkin_ws/src/clover/clover_simulation/launch/simulator.launch` file. Nodes that provide [ArUco detection](aruco.md), [optical flow calculation](optical_flow.md) and other services can be configured by changing their respective `.launch` files, just like on a real drone.
+
+### Changing the drone parameters
+
+![vscode with simulator.launch open](../assets/simulation_usage/04_vscode_config.jpg)
+
+You can enable or disable some of the drone sensors by changing parameters in the `simulator.launch` file. For example, in order to enable GPS, set the `gps` argument to `true`:
+
+```xml
+    <arg name="gps" value="true"/>
+```
+
+Note that this will simply enable the sensor, you will have to change the PX4 estimator parameters to enable GPS fusion.
+
+If you wish to add additional sensors or change their placement, you will have to change the drone description. The description file is located in `~/catkin_ws/src/clover/clover_description/urdf/clover/clover4.xacro`, and uses the [xacro](http://wiki.ros.org/xacro) format to build URDF description.
+
+### Changing the default world
+
+Gazebo plugins for the drone currently require the `real_time_update_rate` world parameter to be 250, and `max_step_size` to be 0.004. Using other values will not work. Consider using `~/catkin_ws/src/clover/clover_simulation/resources/worlds/clover.world` as a base.

docs/en/simulation_vm.md

@@ -0,0 +1,64 @@
+# Simulation VM setup
+
+In addition to [native installation instructions](simulation_setup_native.md), we provide a [preconfigured developer virtual machine image](https://github.com/CopterExpress/clover_vm/releases). The image contains:
+
+* Ubuntu 18.04 with XFCE lightweight desktop environment;
+* ROS packages required to develop for the Clover platform;
+* QGroundControl;
+* preconfigured Gazebo simulation environment;
+* Visual Studio Code with C++ and Python plugins.
+
+The VM is an easy way to set up a simulation environment, but can be used as a development environment for a real drone as well.
+
+You can download the latest VM image [in the VM releases repository](https://github.com/CopterExpress/clover_vm/releases).
+
+> **Note** The virtual machine should be used when native installation is not feasible or possible. You may experience reduced performance in programs that use 3D rendering, like rviz and Gazebo.
+
+## Setting up the VM
+
+You need to use a VM manager that supports OVF format, like [Virtualbox](https://www.virtualbox.org/wiki/Downloads), [VMware Player](https://www.vmware.com/products/workstation-player.html) or [VMware Workstation](https://www.vmware.com/products/workstation-pro.html).
+
+> **Note** At the time of writing Virtualbox had issues running the VM, particularly with 3D applications. We recommend using VMware Player or VMware Workstation if possible. The following steps assume you're using VMware Player.
+
+Make sure that you have hardware virtualization enabled in your BIOS/UEFI (it may be supported by your hardware but turned off by default). The steps to enable virtualization differ from manufacturer to manufacturer, but should be described in your system manual. Consult your system's manufacturer if you're having trouble turning virtualization on.
+
+1. Import the OVA archive into your virtualization environment. Use the **Open a Virtual Machine** option in VMware Player:
+
+    ![Open dialog with clever-devel.ova selected](../assets/simulation_setup_vm/01_import_vm.png)
+
+    > **Note** You may see a dialog box with a warning about the VM format:  
+    ![Import failure dialog](../assets/simulation_setup_vm/02_import_failure.png)  
+    You can safely ignore the warning and press **Retry**.
+
+2. Right-click on the VM name and select **Virtual Machine Settings**. In the new window, set the following parameters:
+
+    * increase the amount of memory available to the virtual machine (a good rule of thumb is 2048 MB per CPU core, but no less than 4 GB):
+    ![Increasing avaliable memory](../assets/simulation_setup_vm/03_max_memory.png)
+    * increase the amount available CPU cores:
+    ![Increasing cpu cores](../assets/simulation_setup_vm/04_core_count.png)
+    * enable 3D acceleration:
+    ![Enabling 3D acceleration](../assets/simulation_setup_vm/05_3d_acceleration.png)
+    * enable USB 2.0/3.0 controller  (if you plan to connect external devices to the VM):
+    ![USB 3.0 controller](../assets/simulation_setup_vm/06_usb_3_0.png)
+    * optionally enable the "bridged" network connection (if you plan to connect to a real drone):
+    ![Enabling bridge networking](../assets/simulation_setup_vm/07_bridge_networking.png)
+
+    > **Note** Some host network adapters may not work well with the bridged network. Consider using external USB Wi-Fi adapters managed by your VM to connect to a real drone.
+
+3. "Power on" the virtual machine. You may see a warning message about your host system not providing 3D acceleration:
+
+    ![No 3D support from host](../assets/simulation_setup_vm/08_no_3d_acceleration.png)
+
+    Make sure you have the latest GPU drivers for your host system. If the warnings persist, add the following line to `clover-devel.vmx` (actual file name may differ based on the VM name):
+
+    ```
+    mks.gl.allowBlacklistedDrivers = "TRUE"
+    ```
+
+    You can find this file in a folder where the VM is imported to.
+
+4. (Bridged networking only) Set up network bridge configuration in VM settings or using `vmware-netcfg` utility (in Linux):
+
+    ![vmware-netcfg interface](../assets/simulation_setup_vm/09_netcfg.png)
+
+    Select `vmnet0` in the networks list, set it to *Bridged*, and choose the adapter you are planning to use to connect to drone in the drop-down menu.