diff --git a/docs/en/aruco.md b/docs/en/aruco.md index afb7c116..cdb3630a 100644 --- a/docs/en/aruco.md +++ b/docs/en/aruco.md @@ -1,9 +1,8 @@ -# Navigation using ArUco markers +# ArUco markers -> **Note** Documentation for the versions [of image](microsd_images.md), starting with **0.15**. For earlier versions, see [documentation for version **0.14**](https://github.com/CopterExpress/clever/blob/v0.14/docs/ru/aruco.md). +> **Note** The following applies to [image versions](microsd_images.md) **0.16** and up. Older documentation is still available [for version **0.15.1**](https://github.com/CopterExpress/clever/blob/v0.15.1/docs/ru/aruco.md). -[ArUco-Markers](https://docs.opencv.org/3.2.0/d5/dae/tutorial_aruco_detection.html) is a popular technology for positioning -robotic systems using computer vision. +[ArUco markers](https://docs.opencv.org/3.2.0/d5/dae/tutorial_aruco_detection.html) are commonly used for vision-based position estimation. Examples of ArUco markers: @@ -13,159 +12,13 @@ Examples of ArUco markers: For rapid generation of markers for printing, you may use an online tool: http://chev.me/arucogen/. -## aruco\_pose +[Clever Raspberry Pi image](microsd_images.md) contains a pre-installed `aruco_pose` ROS package, which can be used for marker detection. -The `aruco_pose` module allows restoring the position of the copter relative to the map of ArUco markers and communicating it to the flight controller using the [Vision Position Estimation](https://dev.px4.io/en/ros/external_position_estimation.html) mechanism. +## Modes of operation -If the source of the copter position by the markers is available, the option appears for precise autonomous indoor navigation by the positions using the [simple_offboard](simple_offboard.md) module. +There are several preconfigured modes of operation for ArUco markers on the Clever drone: -### Turning on +* [single marker detection and navigation](aruco_marker.md); +* [map-based navigation](aruco_map.md). -Make sure that in the clever launch file \(`~/catkin_ws/src/clever/clever/launch/clever.launch`\), the start of aruco\_pose and [computer vision cameras](camera.md) is turned on: - -```xml - -``` - -```xml - -``` - -After the launch-file is edited, restart package `clever`: - -```(bash) -sudo systemctl restart clever -``` - -### Calibrating the ArUco marker map - -An automatically generated [ArUco-board](https://docs.opencv.org/trunk/db/da9/tutorial_aruco_board_detection.html) may be used as a map of marks. - -The map of marks is adjusted using file `~/catkin_ws/src/clever/clever/launch/aruco.launch`. To use ArUco-board, enter its parameters: - -```xml - - - - - - - - - - - - - - - - - - - - -``` - -The vertical and horizontal distance between the markers may be set separately: - -```xml - - - - - -``` - -If a map with a custom order of marker IDs is used, parameter `marker_ids` may be used: - -```xml -[5, 7, 9, 11, 13, 15] -``` - -The markers are numbered from the top left corner of the field. - -For monitoring the map that is currently used by the copter for navigation, one can watch the content of topic `aruco_pose/map_image`. In a browser, it may be viewed with [web\_video\_server](web_video_server.md) by following the link [http://192.168.11.1:8080/snapshot?topic=/aruco\_pose/map\_image](http://192.168.11.1:8080/snapshot?topic=/aruco_pose/map_image): - -![](../../assets/Снимок экрана 2017-11-27 в 23.20.49.png) - -When flying, make sure that the markers glued to the floor correspond to the map. - -In topic `aruco_pose/debug` \([http://192.168.11.1:8080/snapshot?topic=/aruco\_pose/debug](http://192.168.11.1:8080/snapshot?topic=/aruco_pose/debug)\) the current result of markers recognitions is available: - -TODO - -### The system of coordinates - -According to [agreement](http://www.ros.org/reps/rep-0103.html), the standard ENU system of coordinates is used in the marker field: - -* x — rightward \(conditional East\); -* y — forward \(conditional North\); -* z — upward. - -_Note_: the definition above is provided for a situation when the marker field is on the floor. - -First, the zero is the bottom left point of the marker field. The yaw angle is considered zero when the copter is faced rightward\(along the x-axis\). - -![The system of markers coordinates](../assets/aruco-frame.png) - -### Configuring the flight controller - -Correct Vision Position Estimation requires making sure \(via [QGroundControl](gcs_bridge.md)\) that: - -* **For Pixhawk**: Firmware with LPE \(local position estimator\) is installed. For Pixhawk [download firmware `px4fmu-v2_lpe.px4`](https://github.com/PX4/Firmware/releases). - - **For Pixracer**: parameter `SYS_MC_EST_GROUP` should be set to`local_position_estimator, attitude_estimator_q`. - - > **Note** After changing the value of parameter `SYS_MC_EST_GROUP` restart the flight controller. -* In parameter `LPE_FUSION` **only** flags `vision position`, `land detector` are enabled. The final value _20_. -* Compass disabled: `ATT_W_MAG` = 0 -* Complimentary filter external heading weight: `ATT_W_EXT_HDG` = 0.5 -* Orientation by yaw by vision enabled: `ATT_EXT_HDG_M` = 2 `MOCAP`. -* VPE settings: `LPE_VIS_DELAY` = 0 sec, `LPE_VIS_XY` = 0.1 m, `LPE_VIS_Z` = 0.15 m. -* Recommended settings for land detector: - * `COM_DISARM_LAND` = 1 s - * `LNDMC_ROT_MAX` = 45 deg - * `LNDMC_THR_RANGE` = 0.5 - * `LNDMC_Z_VEL_MAX` = 1 m/s - - - -### Flight - -A properly configured copter starts holding position by VPE \(in [modes](modes.md) `POSCTL` or `OFFBOARD`\) automatically. - -For [autonomous flights](simple_offboard.md) do you will be able to use functions `navigate`, `set_position`, `set_velocity`. For flying to specific coordinates of the marker field, use frame `aruco_map`: - -```python -# First, the copter has to take off to see the map of marks -# and for frame aruco_map to appear: -navigate(0, 0, 2, frame_id='body', speed=0.5, auto_arm=True) # take off to the altitude of 2 meters - -time.sleep(5) - -# Flying to coordinate 2:2 of the marker field at the altitude of 2 meters -navigate(2, 2, 2, speed=1, frame_id='aruco_map') # flying to coordinate 2:2 at the altitude of 3 meters -``` - -See [other functions](simple_offboard.md) simple_offboard. - -### Location of markers on the ceiling - -![Markers on the ceiling](../assets/IMG_4175.JPG) - -To navigate by the markers placed on the ceiling, you need to set the main camera facing up and [set the corresponding frame of the camera](camera_frame.md). - -To set the map of markers in a "turned over" system of coordinates, change parameter `aruco_orientation` in file `~/catkin_ws/src/clever/clever/aruco.launch`: - -```xml - -``` - -When this parameter is set, frame aruco\_map will also be "turned over". Thus, to fly at the altitude of 2 meters below the ceiling, argument `z` should be set to 2: - -```python -navigate(x=1, y=2, z=1.1, speed=0.5, frame_id='aruco_map') +> **Info** Additional documentation for the `aruco_pose` ROS package is available [on GitHub](https://github.com/CopterExpress/clever/blob/master/aruco_pose/README.md). diff --git a/docs/en/aruco_map.md b/docs/en/aruco_map.md new file mode 100644 index 00000000..cf3ce769 --- /dev/null +++ b/docs/en/aruco_map.md @@ -0,0 +1,84 @@ +# Map-based navigation with ArUco markers + +> **Info** Marker detection requires the camera module to be correctly plugged in and [configured](camera.md). + + + +> **Hint** We recommend using our [custom PX4 firmware](firmware.md). + +`aruco_map` module detects whole ArUco-based maps. Map-based navigation is possible using vision position estimate (VPE). + +## Configuration + +In order to enable map detection set `aruco_map` and `aruco_detect` arguments to `true` in `~/catkin_ws/src/clever/clever/launch/aruco.launch`: + +```xml + + +``` + +Set `aruco_vpe` to `true` to publish detected camera position to the flight controller as VPE data: + +```xml + +``` + +## Marker map definition + +Map is defined in a text file; each line has the following format: + +``` +marker_id marker_size x y z z_angle y_angle x_angle +``` + +`N_angle` is the angle of rotation along the `N` axis in radians. + +Map path is defined in the `map` parameter: + +```xml + +``` + +Some map examples are provided in [`~/catkin_ws/src/clever/aruco_pose/map`](https://github.com/CopterExpress/clever/tree/master/aruco_pose/map). + +Grid maps may be generated using the `genmap.py` script: + +```bash +rosrun aruco_pose genmap.py length x y dist_x dist_y first > ~/catkin_ws/src/clever/aruco_pose/map/test_map.txt +``` + +`length` is the size of each marker, `x` is the marker count along the *x* axis, `y` is the marker count along the *y* axis, `dist_x` is the distance between the centers of adjacent markers along the *x* axis, `dist_y` is the distance between the centers of the *y* axis, `first` is the ID of the first marker (bottom left marker, unless `--top-left` is specified), `test_map.txt` is the name of the generated map file. The optional `--top-left` parameter changes the numbering of markers, making the top left marker the first one. + +Usage example: + +```bash +rosrun aruco_pose genmap.py 0.33 2 4 1 1 0 > ~/catkin_ws/src/clever/aruco_pose/map/test_map.txt +``` + +### Checking the map + +The currently active map is posted in the `/aruco_map/image` ROS topic. It can be viewed using [web_video_server](web_video_server.md) by opening the following link: http://192.168.11.1:8080/snapshot?topic=/aruco_map/image + + + +Current estimated pose (relative to the detected map) is published in the `aruco_map/pose` ROS topic. If the VPE is disabled, the `aruco_map` [TF frame](frames.md) is created; otherwise, the `aruco_map_detected` frame is created instead. Visualization for the current map is also posted in the `aruco_map/visualization` ROS topic; it may be visualized in [rviz](rviz.md). + +An easy to understand detected map visualization is posted in the `aruco_map/debug` ROS topic (live view is available on http://192.168.11.1:8080/stream_viewer?topic=/aruco_map/debug): + + + +## Coordinate system + +The marker map adheres to the [ROS coordinate system convention](http://www.ros.org/reps/rep-0103.html), using the ENU coordinate system: + +* the **x** axis points to the right side of the map; +* the **y** axis points to the top side of the map; +* the **z** axis points outwards from the plane of the marker + + + +## VPE setup + +In order to enable vision position estimation you should use the following [PX4 parameters](px4_parameters.md). + + \ No newline at end of file diff --git a/docs/en/aruco_marker.md b/docs/en/aruco_marker.md new file mode 100644 index 00000000..de8ce646 --- /dev/null +++ b/docs/en/aruco_marker.md @@ -0,0 +1,111 @@ +# ArUco marker detection + +> **Info** Marker detection requires the camera module to be correctly plugged in and [configured](camera.md). + +`aruco_detect` module detects ArUco markers and publishes their positions in ROS topics and as [TF frames](frames.md). + +This is useful in conjunction with other positioning systems, such as [GPS](gps.md), [Optical Flow](optical_flow.md), PX4Flow, visual odometry, ultrasonic ([Marvelmind](https://marvelmind.com)) or UWB-based ([Pozyx](https://www.pozyx.io)) localization. + +Using this module along with [map-based navigation](aruco_map.md) is also possible. + +## Setup + +Set the `aruco_detect` argument in `~/catkin_ws/src/clever/clever/launch/aruco.launch` to `true` to automatically launch the module: + +```xml + +``` + +For the module to work correctly the following parameters should be set: + +```xml + + + + +``` + +`known_tilt` should be set to: + +* `map` if *all* markers are on the ground; +* `map_flipped` if *all* markers are on the ceiling; +* an empty string otherwise. + +You may specify length for each marker individually by using the `length_override` parameter: + +```xml + + +``` + +## Coordinate system + +Each marker has its own coordinate systems. It is aligned as follows: + +* the **x** axis points to the right side of the marker; +* the **y** axis points to the top side of the marker; +* the **z** axis points outwards from the plane of the marker + + + +## Working with detected markers + +Navigation within the marker-based TF frames is possible with `simple_offboard` node. + +Sample code to fly to a point 1 metre above marker with id 5: + +```python +navigate(frame_id='aruco_5', x=0, y=0, z=1) +``` + +Sample code to fly to a point 1 metre to the left and 2 metres above marker with id 7: + +```python +navigate(frame_id='aruco_7', x=-1, y=0, z=2) +``` + +Sample code to rotate counterclockwise while hovering 1.5 metres above marker id 10: + +```python +navigate(frame_id='aruco_10', x=0, y=0, z=1.5, yaw_rate=0.5) +``` + +Note that if the required marker isn't detected for 0.5 seconds after the `navigate` command, the command will be ignored. + +These frames may also be used in other services that accept TF frames (like `get_telemetry`). The following code will get the drone's position relative to the marker with id 3: + +```python +telem = get_telemetry(frame_id='aruco_3') +``` + +Note that if the required marker isn't detected for 0.5 seconds, the `telem.x`, `telem.y`, `telem.z`, `telem.yaw` fields will contain `NaN`. + +## Handling marker detection in Python + +The following snippet shows how to read the `aruco_detect/markers` topic in Python: + +```python +import rospy +from aruco_pose.msg import MarkerArray +rospy.init_node('my_node') + +# ... + +def markers_callback(msg): + print 'Detected markers:': + for marker in msg.markers: + print 'Marker: %s' % marker + +# Create a Subscription object. Each time a message is posted in aruco_detect/markers, the markers_callback function is called with this message as its argument. +rospy.Subscriber('aruco_detect/markers', MarkerArray, markers_callback) + +# ... + +rospy.spin() +``` + +Each message contains the marker ID, its corner points on the image and its position relative to the camera. + +--- + +Suggested reading: [map-based navigation](aruco_map.md)