Move ROS Noetic (#327)

* builder: Use 64-bit Raspberry Pi OS

* travis: Use 64-bit builder

* builder: Don't try to install Melodic packages on Noetic

* clover: Use package version 3, update dependencies

* travis: Enable Noetic build

* standalone_install: Auto-select Python, ROS distro

* builder: Use variable substitution for ROS_DISTRO

* builder: Add Noetic package definitions

* builder: Use variable substitution for validation

* aruco_pose, clover: Allow compiling against OpenCV 3 and 4

* builder: Add proper Noetic repository

* builder: Don't force Tornado version

Assume rosbridge_suite depends on the right one.

* builder: Install packages for Python 3

* builder/test: Use Python3 interpreter for ROS tests

TODO (?): add tests for Python2?

* builder: Use Python 3 syntax for Python 3 tests

* builder: Install rpi_ws281x for Python3

* standalone_install: Use proper Python for pytest

* builder: Install espeak for python3

* builder: Use proper path for roscore

* builder: Install rosdep, etc. for python3

* builder: Run Clever/Clover test with Python3

* builder: Use Python3 for Clever compat layer

* builder: Enable OpenCV 4.2 repository

* builder: Force versions for ROS packages that use OpenCV

Also, hold their versions so that they don't get updated for no reason.

* aruco_pose/draw: Replace OpenCV projection code with a rewrite

* builder: Don't try to install compressed_transport twice

* clover: Fix importing urllib for Python3

* aruco_pose, clover: Expose Python scripts through CMake

* clover/selfcheck: Be more python3-compatible

This is basically commit a01d199890 from buster-python3, not sure if it aged well.

* roswww_static: Add python script installation

* clover_blocks: Use Python3 syntax for exec

* aruco_pose: Remove unused code

* Melodic => Noetic in some docs

* docs: add 0.22 migration article

* docs: remove unneeded comment

* docs: python 3 updates

* docs: python 3 update in auto_setup article

* docs: add ROS Noetic transition note

* aruco.launch: add placement, length and map arguments

* genmap.py: add -o argument for output file name

* docs: use -o argument of genmap.py

* simple_offboard: correctly check manual control timeout, separate it from kill switch check

* blocks: force led_leds index to int

* docs: update and fix 0.22 migration articles

* blocks: fix set_leds with color-typed argument

* aruco_gen: Open file in binary mode for Python3 compatibility

* clover: Use proper variable in aruco.launch

* led: change default number of leds to 72

* aruco_pose: Make sure there are no undefined symbols

Also, compile in apriltag_quad_thresh.cpp - it contains some of the functions referenced
in aruco.cpp, which would otherwise be undefined.

* aruco_pose: Make vendored library compatible with older OpenCVs

* aruco_pose, clover: Reduce the amount of OpenCV libs requested

* aruco_pose, clover: Move subscriptions to the end of init

* aruco_pose: Don't expose vendored library symbols

* aruco_pose: Simplify dynamic parameter callback setting

* builder: Build with debug symbols

* clover: Attempt to respawn dying nodelets

* Change Raspberry Pi OS to latest armhf, use packages.coex.tech as a source

* Add CRYPTOGRAPHY_DONT_BUILD_RUST=1

* Fix Node.js installation

* image: use older CMake (3.13.4-1)
Fixing https://travis-ci.org/github/CopterExpress/clover/jobs/764367665#L6984

* image: update Raspberry Pi OS to 2021-03-04

* image: bring back moving ld.so.preload out of the way while building

* Fix pthreads ld error

* Try to fix pthreads ld error

* Another attempt to fix pthreads ld error

* Yet another attempt to fix pthreads ld error

* Try to fix

* Be verbose

* Temporarily disable rc and camera_markers building

* Fix standalone-install

* Revert "Temporarily disable rc and camera_markers building"

This reverts commit e119220e91.

* Try to fix

* Try to fix

* Revert "image: use older CMake (3.13.4-1)"

This reverts commit df28da0060.

* Revert "Revert "image: use older CMake (3.13.4-1)""

This reverts commit a28c774e8f.

* Verbosity

* Debugging

* More debugging

* Display all CMake variables

* Try to fix

* Another try to fix

* Revert "Another try to fix"

This reverts commit 5a4c3a0da7.

* Another try to fix

* And another

* And yet another

* Continue...

* Cleanup

* Sources lists cleanup

* More cleanup

* Restore .git directory in clover repo

* Fix building documentation

* Fix documentation building in image

* Trigger build to update ws281x package

* Test

* Disable unneeded hack

* Disable hack

* image: add cmake-modules package

* www: add viewing clover.err file from web interface

* Remove hacks

* Show nodelet version

* docs: add packages article

* image: add image-view package for recording video from topics

* Minor fix

* CI: add Docker authentication on image build

* CI: fix Bash syntax

* CI: fix authentication in Docker

* CI: move Melodic build and editorconfig-lint to GitHub Actions (#331)

* Create main.yml

* Update main.yml

* Disable native Melodic build in Travis

* Run editorconfig-lint in Actions

* Let wget be less verbose

* Test

* Test ok

* Disable editorconfig-lint in Travis

* docs: add links to hardware sources

* CI: move image building to GitHub actions (#335)

* Start working on building image in GitHub actions

* Trigger GitHub on push to any branch

* Fix TRAVIS_TAG

* Add compress image step

* Disable image build in Travis

* Add upload image step

* Fix compress image

* Fix

* Fix

* Minor fix

* Trigger build on tag

* Show images sizes not in human format

* Upload only built image

* Make prerelease

* Upload assets on release not on tags

* readme: change build badge to GitHub Actions

* readme: add support chat badge

* CI: move documentation building to GitHub Actions (#337)

* CI: change docs target branch to actions

* CI: change docs target branch to master

* CI: use gh-pages target branch for docs

* CI: split up to several workflows

* CI: remove .travis.yml

* CI: change apt to apt-get

* CI: push documentation site to the main repo

* builder: less verbosity

* CI: add new key for apt
Fixing https://github.com/CopterExpress/clover/runs/2700356960#step:3:74

* Add Noetic building to CI

* Add test for QR recognition

* Fix

* Move QR recognition test to a separate file

* Fix QR recognition code for Python 3

* Import SetLEDs, LEDStateArray, LEDState in tests

* Add more imports to tests
(from documentation)

* Fix permissions

* Fix standalone-install for Python 2

* Fix QR recognition test

* Don’t use ROS for QR recognition test

* docs: remove non-working example

* Make v4l2 device file an argument in main_camera.launch

* Wait for v4l2 device before launching the camera driver

* Use exec in waitfile

* Transfer main camera nodelet manager to main_camera.launch

* Update cv_camera version to 0.5.1

* docs: minor fix

* Revert cv_camera to 0.5.0

* Update Raspberry Pi OS to 2021-05-07

* docs: add link to the last ROS Melodic version.

Co-authored-by: Alexey Rogachevskiy <sfalexrog@gmail.com>
This commit is contained in:
Oleg Kalachev
2021-06-08 20:13:46 +03:00
committed by GitHub
parent 5f62a8639a
commit f77843f4a5
68 changed files with 750 additions and 944 deletions

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env python
#!/usr/bin/env python3
from distutils.core import setup

View File

@@ -0,0 +1,18 @@
async_web_server_cpp:
debian:
buster: [ros-noetic-async-web-server-cpp]
led_msgs:
debian:
buster: [ros-noetic-led-msgs]
ros_pytest:
debian:
buster: [ros-noetic-ros-pytest]
tf2_web_republisher:
debian:
buster: [ros-noetic-tf2-web-republisher]
web_video_server:
debian:
buster: [ros-noetic-web-video-server]
ws281x:
debian:
buster: [ros-noetic-ws281x]

View File

@@ -3,7 +3,7 @@ Description=Launcher for the ROS master, parameter server and rosout logging nod
[Service]
User=pi
ExecStart=/bin/sh -c ". /opt/ros/melodic/setup.sh; ROS_HOSTNAME=`hostname`.local exec roscore"
ExecStart=/bin/sh -c ". /opt/ros/noetic/setup.sh; ROS_HOSTNAME=`hostname`.local exec roscore"
Restart=on-failure
RestartSec=3

View File

@@ -117,7 +117,7 @@ ${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/avahi-s
# Clover
${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/clover.service' '/lib/systemd/system/'
${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/roscore.service' '/lib/systemd/system/'
${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/melodic-rosdep-clover.yaml' '/etc/ros/rosdep/'
${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/noetic-rosdep-clover.yaml' '/etc/ros/rosdep/'
${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/ros_python_paths' '/etc/sudoers.d/'
${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/pigpiod.service' '/lib/systemd/system/'
${BUILDER_DIR}/image-chroot.sh ${IMAGE_PATH} copy ${SCRIPTS_DIR}'/assets/launch.nanorc' '/usr/share/nano/'

View File

@@ -21,6 +21,9 @@ INSTALL_ROS_PACK_SOURCES=$3
DISCOVER_ROS_PACK=$4
NUMBER_THREADS=$5
# Current ROS distribution
ROS_DISTRO=noetic
echo_stamp() {
# TEMPLATE: echo_stamp <TEXT> <TYPE>
# TYPE: SUCCESS, ERROR, INFO
@@ -68,7 +71,8 @@ my_travis_retry() {
# TODO: 'kinetic-rosdep-clover.yaml' should add only if we use our repo?
echo_stamp "Init rosdep"
my_travis_retry rosdep init
echo "yaml file:///etc/ros/rosdep/melodic-rosdep-clover.yaml" >> /etc/ros/rosdep/sources.list.d/20-default.list
# FIXME: Re-add this after missing packages are built
echo "yaml file:///etc/ros/rosdep/${ROS_DISTRO}-rosdep-clover.yaml" >> /etc/ros/rosdep/sources.list.d/20-default.list
my_travis_retry rosdep update
echo_stamp "Populate rosdep for ROS user"
@@ -76,22 +80,39 @@ my_travis_retry sudo -u pi rosdep update
export ROS_IP='127.0.0.1' # needed for running tests
# echo_stamp "Reconfiguring Clover repository for simplier unshallowing" # TODO: bring back
# cd /home/pi/catkin_ws/src/clover
# git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
echo_stamp "Remove .git from Clover to reduce the size"
rm -rf /home/pi/catkin_ws/src/clover/.git # TODO: remove
# echo_stamp "Reconfiguring Clover repository for simplier unshallowing"
cd /home/pi/catkin_ws/src/clover
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
# This is sort of a hack to force "custom" packages to be installed - the ones built by COEX, linked against OpenCV 4.2
# I **wish** OpenCV would not be such a mess, but, well, here we are.
echo_stamp "Installing OpenCV 4.2-compatible ROS packages"
apt install -y --no-install-recommends \
ros-${ROS_DISTRO}-compressed-image-transport=1.14.0-0buster \
ros-${ROS_DISTRO}-cv-bridge=1.15.0-0buster \
ros-${ROS_DISTRO}-cv-camera=0.5.0-0buster \
ros-${ROS_DISTRO}-image-publisher=1.15.3-0buster \
ros-${ROS_DISTRO}-web-video-server=0.2.1-0buster
apt-mark hold \
ros-${ROS_DISTRO}-compressed-image-transport \
ros-${ROS_DISTRO}-cv-bridge \
ros-${ROS_DISTRO}-cv-camera \
ros-${ROS_DISTRO}-image-publisher \
ros-${ROS_DISTRO}-web-video-server
echo_stamp "Installing libboost-dev" # https://travis-ci.org/github/CopterExpress/clover/jobs/766318908#L6536
my_travis_retry apt-get install -y --no-install-recommends libboost-dev libboost-all-dev
echo_stamp "Build and install Clover"
cd /home/pi/catkin_ws
# Don't try to install gazebo_ros
my_travis_retry rosdep install -y --from-paths src --ignore-src --rosdistro melodic --os=debian:buster \
my_travis_retry rosdep install -y --from-paths src --ignore-src --rosdistro ${ROS_DISTRO} --os=debian:buster \
--skip-keys=gazebo_ros --skip-keys=gazebo_plugins
my_travis_retry pip install wheel
my_travis_retry pip install -r /home/pi/catkin_ws/src/clover/clover/requirements.txt
source /opt/ros/melodic/setup.bash
my_travis_retry pip3 install wheel
my_travis_retry pip3 install -r /home/pi/catkin_ws/src/clover/clover/requirements.txt
source /opt/ros/${ROS_DISTRO}/setup.bash
# Don't build simulation plugins for actual drone
catkin_make -j2 -DCMAKE_BUILD_TYPE=Release -DCATKIN_BLACKLIST_PACKAGES=clover_gazebo_plugins
catkin_make -j2 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCATKIN_BLACKLIST_PACKAGES=clover_gazebo_plugins
echo_stamp "Install clever package (for backwards compatibility)"
cd /home/pi/catkin_ws/src/clover/builder/assets/clever
@@ -108,23 +129,20 @@ touch node_modules/CATKIN_IGNORE docs/CATKIN_IGNORE _book/CATKIN_IGNORE clover/w
echo_stamp "Installing additional ROS packages"
my_travis_retry apt-get install -y --no-install-recommends \
ros-melodic-dynamic-reconfigure \
ros-melodic-compressed-image-transport \
ros-melodic-rosbridge-suite \
ros-melodic-rosserial \
ros-melodic-usb-cam \
ros-melodic-vl53l1x \
ros-melodic-ws281x \
ros-melodic-rosshow
ros-${ROS_DISTRO}-dynamic-reconfigure \
ros-${ROS_DISTRO}-rosbridge-suite \
ros-${ROS_DISTRO}-rosserial \
ros-${ROS_DISTRO}-usb-cam \
ros-${ROS_DISTRO}-vl53l1x \
ros-${ROS_DISTRO}-ws281x \
ros-${ROS_DISTRO}-rosshow \
ros-${ROS_DISTRO}-cmake-modules \
ros-${ROS_DISTRO}-image-view
# TODO move GeographicLib datasets to Mavros debian package
echo_stamp "Install GeographicLib datasets (needed for mavros)" \
&& wget -qO- https://raw.githubusercontent.com/mavlink/mavros/master/mavros/scripts/install_geographiclib_datasets.sh | bash
# FIXME: Buster comes with tornado==5.1.1 but we need tornado==4.2.1 for rosbridge_suite
# (note that Python 3 will still have a more recent version)
pip install tornado==4.2.1
echo_stamp "Running tests"
cd /home/pi/catkin_ws
# FIXME: Investigate failing tests
@@ -141,7 +159,7 @@ cat << EOF >> /home/pi/.bashrc
LANG='C.UTF-8'
LC_ALL='C.UTF-8'
export ROS_HOSTNAME=\`hostname\`.local
source /opt/ros/melodic/setup.bash
source /opt/ros/${ROS_DISTRO}/setup.bash
source /home/pi/catkin_ws/devel/setup.bash
EOF

View File

@@ -64,15 +64,14 @@ echo "APT::Acquire::Retries \"3\";" > /etc/apt/apt.conf.d/80-retries
echo_stamp "Install apt keys & repos"
# TODO: This STDOUT consist 'OK'
curl http://deb.coex.tech/aptly_repo_signing.key 2> /dev/null | apt-key add -
apt-get update \
&& apt-get install --no-install-recommends -y dirmngr > /dev/null \
&& apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
echo "deb http://packages.ros.org/ros/ubuntu buster main" > /etc/apt/sources.list.d/ros-latest.list
echo "deb http://deb.coex.tech/opencv3 buster main" > /etc/apt/sources.list.d/opencv3.list
echo "deb http://deb.coex.tech/rpi-ros-melodic buster main" > /etc/apt/sources.list.d/rpi-ros-melodic.list
echo "deb http://deb.coex.tech/clover buster main" > /etc/apt/sources.list.d/clover.list
wget -O - 'http://packages.coex.tech/key.asc' | apt-key add -
echo 'deb http://packages.coex.tech buster main' >> /etc/apt/sources.list
echo_stamp "Update apt cache"
@@ -99,18 +98,18 @@ tree \
vim \
libjpeg8 \
tcpdump \
ltrace \
libpoco-dev \
libzbar0 \
python-rosdep \
python-rosinstall-generator \
python-wstool \
python-rosinstall \
python3-rosdep \
python3-rosinstall-generator \
python3-wstool \
python3-rosinstall \
build-essential \
libffi-dev \
monkey \
pigpio python-pigpio python3-pigpio \
i2c-tools \
espeak espeak-data python-espeak python3-espeak \
ntpdate \
python-dev \
python3-dev \
@@ -144,7 +143,7 @@ my_travis_retry pip3 install butterfly[systemd]
systemctl enable butterfly.socket
echo_stamp "Install ws281x library"
my_travis_retry pip install --prefer-binary rpi_ws281x
my_travis_retry pip3 install --prefer-binary rpi_ws281x
echo_stamp "Setup Monkey"
mv /etc/monkey/sites/default /etc/monkey/sites/default.orig

View File

@@ -16,16 +16,20 @@ set -ex
echo "Run image tests"
export ROS_DISTRO='melodic'
export ROS_DISTRO='noetic'
export ROS_IP='127.0.0.1'
source /opt/ros/melodic/setup.bash
source /opt/ros/${ROS_DISTRO}/setup.bash
source /home/pi/catkin_ws/devel/setup.bash
systemctl start roscore
cd /home/pi/catkin_ws/src/clover/builder/test/
./tests.sh
./tests.py
./tests_py3.py
[[ $(./test_qr.py) == "Found QRCODE with data Проверка Unicode with center at x=66.0, y=66.0" ]]
[[ $(./tests_clever.py) == "Warning: clever package is renamed to clover" ]] # test backwards compatibility
systemctl stop roscore
echo "Move /etc/ld.so.preload back to its original position"
mv /etc/ld.so.preload.disabled-for-build /etc/ld.so.preload

View File

@@ -6,21 +6,40 @@ set -e
apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654 # https://github.com/osrf/docker_images/issues/535
apt-get update
apt-get install -y curl
curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
python ./get-pip.py
if [ "x${ROS_PYTHON_VERSION}" = "x3" ]; then
PYTHON=python3
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
else
PYTHON=python
curl https://bootstrap.pypa.io/pip/2.7/get-pip.py -o get-pip.py
fi
${PYTHON} ./get-pip.py
# Step 1.5: Add deb.coex.tech to apt
curl http://deb.coex.tech/aptly_repo_signing.key 2> /dev/null | apt-key add -
echo "deb http://deb.coex.tech/ros xenial main" > /etc/apt/sources.list.d/coex.tech.list
echo "yaml file:///etc/ros/rosdep/coex.yaml" > /etc/ros/rosdep/sources.list.d/99-coex.list
CODENAME=$(lsb_release -sc)
cat <<EOF > /etc/ros/rosdep/coex.yaml
led_msgs:
ubuntu:
xenial: ros-kinetic-led-msgs
bionic: ros-melodic-led-msgs
debian:
stretch: ros-kinetic-led-msgs
buster: ros-melodic-led-msgs
${CODENAME}: [ros-${ROS_DISTRO}-led-msgs]
async_web_server_cpp:
ubuntu:
${CODENAME}: [ros-${ROS_DISTRO}-async-web-server-cpp]
ros_pytest:
ubuntu:
${CODENAME}: [ros-${ROS_DISTRO}-ros-pytest]
tf2_web_republisher:
ubuntu:
${CODENAME}: [ros-${ROS_DISTRO}-tf2-web-republisher]
web_video_server:
ubuntu:
${CODENAME}: [ros-${ROS_DISTRO}-web-video-server]
ws281x:
ubuntu:
${CODENAME}: [ros-${ROS_DISTRO}-ws281x]
EOF
apt-get update
rosdep update
@@ -38,7 +57,7 @@ cd /root/catkin_ws
catkin_make
# Step 4: Run tests
pip install --upgrade pytest
${PYTHON} -m pip install --upgrade pytest
cd /root/catkin_ws
source devel/setup.bash
catkin_make run_tests && catkin_test_results

BIN
builder/test/qr.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

42
builder/test/test_qr.py Executable file
View File

@@ -0,0 +1,42 @@
#!/usr/bin/env python3
# Test QG recognition example
# Should be synced with the documentation: /docs/en/camera.md, /docs/ru/camera.md
# TODO: use real ROS topics
import rospy
from pyzbar import pyzbar
from cv_bridge import CvBridge
from sensor_msgs.msg import Image
bridge = CvBridge()
# rospy.init_node('barcode_test')
# Image subscriber callback function
def image_callback(data):
cv_image = bridge.imgmsg_to_cv2(data, 'bgr8') # OpenCV image
barcodes = pyzbar.decode(cv_image)
for barcode in barcodes:
b_data = barcode.data.decode("utf-8")
b_type = barcode.type
(x, y, w, h) = barcode.rect
xc = x + w/2
yc = y + h/2
print("Found {} with data {} with center at x={}, y={}".format(b_type, b_data, xc, yc))
# rospy.signal_shutdown('done')
# image_sub = rospy.Subscriber('main_camera/image_raw', Image, image_callback, queue_size=1)
# ==============================================================================
# Publish test image
# rospy.sleep(2)
import cv2
img = cv2.imread('qr.png')
image_callback(bridge.cv2_to_imgmsg(img, 'bgr8'))
# image_pub = rospy.Publisher('/main_camera/image_raw', Image, queue_size=1, latch=True)
# image_pub.publish(bridge.cv2_to_imgmsg(img, 'bgr8'))
# rospy.spin()

View File

@@ -1,21 +1,29 @@
#!/usr/bin/env python
#!/usr/bin/env python3
# validate all required modules installed
import rospy
from geometry_msgs.msg import PoseStamped
from sensor_msgs.msg import Range, BatteryState
import cv2
import cv2.aruco
from sensor_msgs.msg import Image
from cv_bridge import CvBridge
import numpy
import mavros
from mavros_msgs.msg import State, StatusText, ExtendedState
from mavros_msgs.msg import State, StatusText, ExtendedState, RCIn, Mavlink
from mavros_msgs.srv import CommandBool, CommandLong, SetMode
from std_srvs.srv import Trigger
from clover.srv import GetTelemetry, Navigate, NavigateGlobal, SetPosition, SetVelocity, \
SetAttitude, SetRates, SetLEDEffect
from led_msgs.srv import SetLEDs
from led_msgs.msg import LEDStateArray, LEDState
from aruco_pose.msg import Marker, MarkerArray, Point2D
import dynamic_reconfigure.client
import tf2_ros
import tf2_geometry_msgs
@@ -28,4 +36,4 @@ import pigpio
# from espeak import espeak
from pyzbar import pyzbar
print cv2.getBuildInformation()
print(cv2.getBuildInformation())

View File

@@ -54,6 +54,8 @@ rosversion usb_cam
rosversion cv_camera
rosversion web_video_server
rosversion rosshow
rosversion nodelet
rosversion image_view
# validate examples are present
[[ $(ls /home/pi/examples/*) ]]

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env python
#!/usr/bin/env python3
# test backwards compatibility