Autoware.Auto with SVL Simulator #

Table of Contents

Overview top#

This guide describes setting up and using Autoware.Auto with the SVL Simulator. As Autoware.Auto is still under-development, full self-driving is not yet possible. This guide will focus on running individual modules which have been implemented.

Setup top#

Requirements top#

• Linux operating system
• NVIDIA graphics card

Install Docker CE top#

To install Docker CE please refer to the official documentation.

We also suggest following through with the post installation steps to run docker as a non-root user.

Verify that you can run docker commands without sudo:

# In a desktop shell

docker run hello-world

This command downloads a test image and runs it in a container. When the container runs, it prints a message and exits.

Install NVIDIA Container Toolkit top#

Before installing the NVIDIA Container Toolkit, make sure that you have the appropriate NVIDIA drivers installed. To test if the NVIDIA drivers are properly installed, enter nvidia-smi in a terminal. If the drivers are installed properly an output similar to the following should appear:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.59       Driver Version: 440.59       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:65:00.0  On |                  N/A |
|  0%   59C    P5    22W / 250W |   1490MiB / 11175MiB |      4%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1187      G   /usr/lib/xorg/Xorg                           863MiB |
|    0      3816      G   /usr/bin/gnome-shell                         305MiB |
|    0      4161      G   ...-token=7171B24E50C2F2C595566F55F1E4D257    68MiB |
|    0      4480      G   ...quest-channel-token=3330599186510203656   147MiB |
|    0     17936      G   ...-token=5299D28BAAD9F3087B25687A764851BB   103MiB |
+-----------------------------------------------------------------------------+

The installation steps for the NVIDIA Container Toolkit are available from the official documentation.

Verify that NVIDIA Container Toolkit is working by running nvidia-smi inside a base CUDA container:

# In a desktop shell

docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

This will verify that Docker CE and NVIDIA Container Toolkit are installed and working properly.

Simulator installation top#

• Download and extract the latest simulator release.
• (Optional) Download the latest PythonAPI release (make sure the release version matches the simulator) and install it using pip:

# In a desktop shell

cd PythonAPI
pip3 install --user .

Install Autoware.auto top#

Install ADE top#

# In a desktop shell
# (from https://ade-cli.readthedocs.io/en/latest/install.html)

cd ~/.local/bin
wget https://gitlab.com/ApexAI/ade-cli/uploads/f6c47dc34cffbe90ca197e00098bdd3f/ade+x86_64
mv ade+x86_64 ade
chmod +x ade
./ade --version
# 4.0.0

./ade update-cli
./ade --version
# <latest-version>

PATH=$PATH:~/.local/bin
mkdir -p ~/adehome
cd ~/adehome
touch .adehome

Download Autoware.Auto top#

Download Autoware.Auto under the ~/adehome folder.

# In a desktop shell

cd ~/adehome
git clone https://gitlab.com/autowarefoundation/autoware.auto/AutowareAuto.git

Installation and Development: Setup guide guide for Autoware.auto.

Each of the external dependency repos (ROS2 bridge, messages, etc.) must be cloned to the external folder under the source directory:

mkdir -p ~/adehome/AutowareAuto/src/external/
cd ~/adehome/AutowareAuto/src/external/

Downloading the AutowareAuto "messages" module#

This module is necessary for the further build progress.

# in ~/adehome/AutowareAuto/src/external/
git clone https://gitlab.com/autowarefoundation/autoware.auto/autoware_auto_msgs.git

Run ADE container#

# in ~/adehome/AutowareAuto/
ade start --update

To stop the ADE container run

ade stop

NOTE All the next build steps assume an ADE container is already running.

Install ROS 2 LGSVL Bridge top#

There are two ways of installing ros2-lgsvl-bridge:

• Using the Package Manager
• Building source code

1. Using the Package Manager (preferred)#

# In the ade container

sudo apt update
sudo apt install ros-foxy-lgsvl-bridge

# Test the bridge (then ctrl-c to stop the bridge):
lgsvl_bridge

• Note: If sudo apt update returns apt update: signatures were invalid in ADE, try updating the repo key with curl http://repo.ros2.org/repos.key | sudo apt-key add - as noted in answers.ros.org.

2. Building from source code#

External modules folder#

Downloading the ROS2 bridge#

# in ~/adehome/AutowareAuto/src/external/
git clone https://github.com/lgsvl/ros2-lgsvl-bridge.git
cd ros2-lgsvl-bridge
git checkout foxy-devel

Building the ROS2 bridge#

Refer to README.md in the repo.

ade enter
# In the ade container

cd ~/AutowareAuto
colcon build --packages-select lgsvl_bridge --cmake-args '-DCMAKE_BUILD_TYPE=Release'

Running the ROS2 bridge#

Refer to README.md in the repo.

ade enter
# In the ade container
source ~/AutowareAuto/install/setup.bash
lgsvl_bridge

Install ROS 2 LGSVL Messages top#

Downloading#

# in ~/adehome/AutowareAuto/src/external/
git clone https://github.com/lgsvl/lgsvl_msgs.git

Building#

ade enter
# In the ade container

source ~/AutowareAuto/src/external/ros2-lgsvl-bridge/install/setup.bash
lgsvl_bridge

cd ~/AutowareAuto
colcon build --cmake-args '-DCMAKE_BUILD_TYPE=Release'

As an example of a successful build, observe in the console output something like:

...
Summary: 115 packages finished [8min 5s]
...

Note: You may want to build only lgsvl_msgs package with the following command:

colcon build --packages-select lgsvl_msgs --cmake-args '-DCMAKE_BUILD_TYPE=Release'

Testing#

Note: the ROS2 Foxy has deprecated the ros2 msg list command, use the ros2 interface list instead.

ade enter
# In the ade container
cd ~/AutowareAuto
source install/setup.bash
ros2 interface list | grep lgsvl_msgs
# If you can see the list of lgsvl_msgs, they're ready to be used.
# Example:
...
lgsvl_msgs/CanBusData
lgsvl_msgs/VehicleControlData
lgsvl_msgs/VehicleStateData
...

Run Simulator alongside Autoware.Auto top#

The ROS 2 web bridge allows the simulator and Autoware.auto to communicate. To test this connection we can visualize sensor data from the simulator in rviz2 (running in the Autoware.auto container).

Start the Autoware.Auto containers without NVIDIA setup:#

NOTE Stop any ADE container if it is already running: $ ade stop

# In a desktop shell

cd ~/adehome/AutowareAuto
source .aderc-amd64-foxy-lgsvl
ade start

Start the Autoware.Auto containers with NVIDIA setup:#

NOTE Stop any ADE container if it is already running: $ ade stop

Create a aderc file which has nvidia setup:

# In a desktop shell

vim  ~/adehome/AutowareAuto/.aderc-amd64-foxy-lgsvl-nvidia

Paste the next block into the .aderc-amd64-foxy-lgsvl-nvidia file:

export ADE_DOCKER_RUN_ARGS="--cap-add=SYS_PTRACE --net=host --privileged --add-host ade:127.0.0.1 -e RMW_IMPLEMENTATION=rmw_cyclonedds_cpp --runtime=nvidia -ti --rm -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix \
-e NVIDIA_VISIBLE_DEVICES=all \
-e NVIDIA_DRIVER_CAPABILITIES=compute,utility,display"
export ADE_GITLAB=gitlab.com
export ADE_REGISTRY=registry.gitlab.com
export ADE_DISABLE_NVIDIA_DOCKER=false
export ADE_IMAGES="
  registry.gitlab.com/autowarefoundation/autoware.auto/autowareauto/amd64/ade-foxy:master
  registry.gitlab.com/autowarefoundation/autoware.auto/autowareauto/amd64/binary-foxy:master
  registry.gitlab.com/autowarefoundation/autoware.auto/ade-lgsvl/foxy:2020.06
  nvidia/cuda:11.0-base
"

(Stop and re-) start the ADE container:

# In a desktop shell, first stop ADE container if it was previously running
ade stop

cd ~/adehome/AutowareAuto
source .aderc-amd64-foxy-lgsvl-nvidia
ade start
ade enter

Build and start RViz in the running ADE container#

ade enter
# In the ADE container
cd ~/AutowareAuto
colcon build --cmake-args '-DCMAKE_BUILD_TYPE=Release'
source ~/AutowareAuto/install/setup.bash

• Start rviz2:

# In the ade container

rviz2 -d /home/"${USER}"/AutowareAuto/install/autoware_auto_examples/share/autoware_auto_examples/rviz2/autoware.rviz

Start the SVL Simulator#

• Outside of the ADE container launch the executable and click on the OPEN BROWSER button to open the web UI.

bash $ (path\to\downloaded\simulator)/svlsimulator-linux64-2021.3/simulator

• In the Vehicles tab under Library look for Lexus2016RXHybrid. If not available, see the Library page to add it.

  • Make sure that Autoware.Auto sensor configuration has the ROS2 bridge and all of sensors are added.
  • Click Vehicles under Library in the left side and click Lexus2016RXHybrid and click Autoware.Autoin Sensor Configurations.
  • If you can see i mark next to sensor name, click Add to Library button to add sensor plugins into library.

• Switch to the Simulations tab and click the Add new button:

  • Enter a Simulation Name and click Next.
  • Select Random Traffic in Runtime Template.
  • Select a map from the drop down menu. If none are available follow this guide to get a map.
  • Select the Lexus2016RXHybrid from the drop down menu of Vehicle.
  • Select the Autoware.Auto in Sensor Configuration and click Next.
• Select Autoware.Auto (Apex.AI) in Autopilot and enter the bridge address (default:localhost:9090) in Bridge IP box and click Next.
  • Click Publish.
  • Press Run Simulation button.

Launch ROS 2 LGSVL bridge#

(in a new terminal window):

ade enter
# In the ADE container
source ~/AutowareAuto/install/setup.bash
lgsvl_bridge

Configure RViz to see the LIDAR point cloud#

In the RViz Displays panel (top left on the screenshot below):

• Set the Fixed Frame (under Global Options) to lidar_front.
• Verify either a PointCloud2 or a Transformed Points message is added.
• Set the message's Topic field to the /lidar_front/points_raw topic.

Run Simulator alongside Autoware.Auto in ADE docker container top#

Those who wish to run SVL Simulator inside ADE might find the following instructions helpful, but it is recommended that users instead run SVL Simulator on the host (not from inside ADE).

1. Build ADE docker container#

Copy the following Dockerfile into ~/adehome/AutowareAuto/tools/ade_image folder.

FROM ubuntu:20.04
CMD ["bash"]

ARG ROS_DISTRO=foxy

# https://docs.ros.org/en/foxy/Installation/Ubuntu-Development-Setup.html
RUN apt update && apt install locales
RUN locale-gen en_US en_US.UTF-8
RUN update-locale LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8
RUN export LANG=en_US.UTF-8

# tz America/Los_Angeles
ENV TZ=America/Los_Angeles
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get install -y tzdata

RUN apt update && apt install -y curl gnupg2 lsb-release python3-pip gettext
RUN curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key  -o /usr/share/keyrings/ros-archive-keyring.gpg

RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/ros2.list > /dev/null
# /https://docs.ros.org/en/foxy/Installation/Ubuntu-Development-Setup.html

RUN apt-get update && \
    apt-get install -y \
      python3-vcstool \
      ros-$ROS_DISTRO-cyclonedds \
      ros-$ROS_DISTRO-rmw-cyclonedds-cpp && \
    rm -rf /var/lib/apt/lists/* #/tmp/ros-deps

COPY apt-packages /tmp/
RUN apt-get update && \
    apt-get install -y \
        $(cat /tmp/apt-packages | cut -d# -f1 | envsubst) \
    && rm -rf /var/lib/apt/lists/* /tmp/apt-packages

RUN echo 'ALL ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
RUN echo 'Defaults env_keep += "DEBUG ROS_DISTRO"' >> /etc/sudoers


COPY pip3-packages /tmp/
RUN pip3 install -U \
        $(cut -d# -f1 </tmp/pip3-packages) \
    && rm -rf /root/.cache /tmp/pip-* /tmp/pip3-packages


RUN git clone https://github.com/rigtorp/udpreplay && mkdir -p udpreplay/build \
      && cd udpreplay/build && cmake .. && make && make install \
      && cd - && rm -rf udpreplay/


COPY bashrc-git-prompt /
RUN cat /bashrc-git-prompt >> /etc/skel/.bashrc && \
    rm /bashrc-git-prompt
COPY gdbinit /etc/gdb/


# ===================== CLEAN UP ZONE ===================== #
# Commands in the cleanup zone will be cleaned up before every release
# and put into the correct place.
RUN apt-get update \
  && apt-get install -y \
    unzip \
    ca-certificates \
    libx11-6 \
    libxau6 \
    libxcb1 \
    libxdmcp6 \
    libxext6 \
    libvulkan1 \
    libgl1 \
    libgtk2.0-0 \
    vulkan-utils \
    docker.io \
    xdg-utils \
  && apt-get clean

ADD "https://gitlab.com/nvidia/container-images/vulkan/raw/master/nvidia_icd.json" /etc/vulkan/icd.d/nvidia_icd.json
RUN chmod 644 /etc/vulkan/icd.d/nvidia_icd.json

# ===================== END OF CLEAN UP ZONE ===================== #

# Add print url into tty part.
COPY print-tty.desktop /usr/share/applications/print-tty.desktop
COPY print-tty.sh /usr/bin/print-tty.sh
RUN chmod 755 /usr/bin/print-tty.sh
RUN apt-get update \
  && apt-get install -y \
    ros-$ROS_DISTRO-lgsvl-bridge \
  && apt-get clean
# /Add print url into tty part.

# Do full package upgrade as last step
# to avoid disrupting layer caching
RUN apt-get update && \
    apt-get -y dist-upgrade && \
    rm -rf /var/lib/apt/lists/*

COPY env.sh /etc/profile.d/ade_env.sh
COPY gitconfig /etc/gitconfig
COPY entrypoint /ade_entrypoint
COPY colcon-defaults.yaml /usr/local/etc/colcon-defaults.yaml
RUN echo "export COLCON_DEFAULTS_FILE=/usr/local/etc/colcon-defaults.yaml" >> \
    /etc/skel/.bashrc
ENTRYPOINT ["/ade_entrypoint"]
CMD ["/bin/sh", "-c", "trap 'exit 147' TERM; tail -f /dev/null & wait ${!}"]

Copy the following print-tty.desktop file into ~/adehome/AutowareAuto/tools/ade_image folder.

[Desktop Entry]
Name=PrintTty
GenericName=Print On Tty
Comment=Prints text on first tty
TryExec=echo
Exec=print-tty.sh %F
Terminal=true
Type=Application
MimeType=text/english;text/plain;text/x-makefile;text/x-c++hdr;text/x-c++src;text/x-chdr;text/x-csrc;text/x-java;text/x-moc;text/x-pascal;text/x-tcl;text/x-tex;application/x-shellscript;text/x-c;text/x-c++;application/pdf;application/rdf+xml;application/rss+xml;application/xhtml+xml;application/xhtml_xml;application/xml;image/gif;image/jpeg;image/png;image/webp;text/html;text/xml;x-scheme-handler/ftp;x-scheme-handler/http;x-scheme-handler/https;

Copy the following print-tty.sh file into ~/adehome/AutowareAuto/tools/ade_image folder.

#!/bin/sh

echo "$@" >/dev/tty

Build docker container and tag it:

cd ~/adehome/AutowareAuto/tools/ade_image
docker build -t registry.gitlab.com/autowarefoundation/autoware.auto/autowareauto/amd64/ade-foxy:local .

2. Build and start ADE docker container#

Copy the following .aderc-amd64-foxy-lgsvl-nvidia into ~/adehome/AutowareAuto folder:

export ADE_DOCKER_RUN_ARGS="--cap-add=SYS_PTRACE --net=host --privileged --add-host ade:127.0.0.1 -e RMW_IMPLEMENTATION=rmw_cyclonedds_cpp --runtime=nvidia -ti --rm -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix \
-e NVIDIA_VISIBLE_DEVICES=all \
-e NVIDIA_DRIVER_CAPABILITIES=compute,utility,display"
export ADE_GITLAB=gitlab.com
export ADE_REGISTRY=registry.gitlab.com
export ADE_DISABLE_NVIDIA_DOCKER=false
export ADE_IMAGES="
  registry.gitlab.com/autowarefoundation/autoware.auto/autowareauto/amd64/ade-foxy:local
  registry.gitlab.com/autowarefoundation/autoware.auto/autowareauto/amd64/binary-foxy:master
  registry.gitlab.com/apexai/ade-lgsvl:2021.2.1
  nvidia/cuda:11.0-base
"

Note: If you have registry.gitlab.com/apexai/ade-lgsvl:2021.2.1 simulator docker container, you may include it in .aderc-amd64-foxy-lgsvl-nvidia file (as in the example above); but if you don't you can mount local simulator into /opt/lgsvl in ade docker container when you start ade container.

Source the .aderc-amd64-foxy-lgsvl-nvidia:

source .aderc-amd64-foxy-lgsvl-nvidia

Unzip Simulator in desktop shell:

unzip svlsimulator-linux64-2021.2.1.zip -d ~

Start ade:

ade start -- -e DISPLAY -e XAUTHORITY=/tmp/.Xauthority \
-v ${XAUTHORITY}:/tmp/.Xauthority -v /tmp/.X11-unix:/tmp/.X11-unix \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp/LGElectronics/:/tmp/LGElectronics/ \
-v ~/svlsimulator-linux64-2021.2.1:/opt/lgsvl

3. Launch Simulator in ADE docker container#

ade enter
lgsvl_bridge &
/opt/lgsvl/simulator

Click link to cluster and URL is printed on the terminal.

Copy and paste the URL on your web browser to open the SVL Simulator user interface.

Run AVP Parking Demo in Autoware.Auto#

This explains running the pre-built Autoware.Auto from the ADE docker container.

If you want to run Autoware.Auto built from source code, refer to How to build the code.

1. Run ROS 2 LGSVL Bridge#

Check if you installed ROS 2 LGSVL Bridge.

If not, Install the ROS 2 LGSVL Bridge.

# In the ade container
source /opt/AutowareAuto/setup.bash
lgsvl_bridge

2. Run AVP Sim launch file#

# In the ade container
source /opt/AutowareAuto/setup.bash
ros2 launch /opt/AutowareAuto/src/launch/autoware_demos/launch/avp_sim.launch.py

3. Run RViz2#

# In the ade container
source /opt/AutowareAuto/setup.bash
rviz2 -d /opt/AutowareAuto/share/autoware_auto_launch/config/avp.rviz

rviz2 will load the AutonomouStuff map for the parking demo.

4. Run the Simulator#

# In a desktop shell, launch SVL Simulator
(path\to\downloaded\simulator)/svlsimulator-linux64-2021.3/simulator

Simulation setup should be as follows:

• Runtime Template: Random Traffic
• Maps: AutonomouStuff
• Vehicle Asset: Lexus2016RXHybrid
• Sensor configuration: Autoware.Auto
• Bridge: ROS2
• Autopilot: Autoware.Auto (Apex.AI)
• Connection: localhost:9090 (In the case of the Simulator and Autoware.Auto running on the same machine.)

5. Setting Pose Estimate and Goal Pose#

• Click 2D Pose Estimate Click to the initial position and drag downwards to set ego's direction.

• Click 2D Goal Pose Click on the desired parking spot and drag perpendicular to the lane.

Depending on the parking spot, the planner may or may not succeed in finding a route.