< >
Home » TX2入门教程软件篇 » TX2入门教程软件篇-安装apollo自动驾驶软件包(JetPack 3.0)

TX2入门教程软件篇-安装apollo自动驾驶软件包(JetPack 3.0)

TX2入门教程软件篇-安装apollo自动驾驶软件包

说明：

  • 介绍如何在tx2上安装apollo自动驾驶软件包
  • 环境：jetson3.0 full 版
  • 扩展256G的固态硬盘作为启动盘

背景

  • NVIDIA Jetson TX2是CPU架构为ARM的平台，在该平台安装Apollo有两个方案:
  • 1.使用apollo原生的安装方式，需要重新编译默认内核，以增加docker支持。
  • 2.因为网络等其他因素限制，不适用docker环境，可以按照Dockerfile安装环境依赖类库，该文件中的依赖还可能不完整，需要安装过程中提示的依赖才可以编译Apollo。
  • 以下的本篇指南以第一点为主
  • Note:最好在移动硬盘中安装Apollo，因为在刷入内核的过程中，有可能出现因为显卡驱动的问题，无法出现图形界面的情况。另外一个方面，TX2内置容量30G，容量也很有限，所以放在移动硬盘中安装很有必要。

所需设备列表

  • Samsung 850 EVO (250GB)
  • Nvdia Jetson TX2开发者套件

安装固态硬盘

  • 1.安装固态硬盘：
  • 在搜索中”Disk”磁盘工具，选择对应的磁盘。点击设置按钮，选择“Format Partition”，为磁盘命名，点击格式化。
  • 2.磁盘分区：
$ sudo parted /dev/sda/ mklabel gpt
  • 3.格式化分区
  • 4.挂载：
  • 在TX2挂载该硬盘之后，我们默认该硬盘的位置为：/media/nvidia/{ssd}/，后续的描述中，凡是引用该部分内容的，都是从该硬盘中拷贝或其他操作。
  • 5.固态硬盘的安装与挂载可参考视频:https://www.youtube.com/watch?v=KxZ-e6G7INg

重新编译内核

  • TX2安装docker的问题等于自己执行了安装Docker的步骤过程。
Here is the config I used.
https://github.com/frankjoshua/buildJetsonTX2Kernel/blob/master/docker_config/config

I rebuilt the kernel using instructions from https://github.com/jetsonhacks/buildJetsonTX2Kernel

Run getKernelSources.sh then close out the kernel editor gui after it starts. Then copy the file ./docker_conf/config to /usr/src/kernel/kernel-4.4/.config

Then run makeKernel.sh and copyImage.sh.

After you reboot “uname -r” should display 4.4.15-docker

4.4.38-container

Finally

sudo apt install docker.io

No aufs support yet but so far everything is working. I am running docker containers the were originally created for the Raspberry Pi 3 and they work great so far.

Install cuda

  • 流程：
$ cd ~ 
 
$ wget http://developer.download.nvidia.com/devzone/devcenter/mobile/jetpack_l4t/013/linux-x64/cuda-repo-l4t-8-0-local_8.0.84-1_arm64.deb
 
$ sudo dpkg -i cuda-repo-l4t-8-0-local_8.0.84-1_arm64.deb
$ sudo apt-get update
$ sudo apt-get install cuda-toolkit-8-0
  • install done ,check directory /usr/local/cuda and /usr/local/cuda-8.0,this two directories will use later.

编译Apollo的准备工作

  • Arm平台的第三方类库兼容准备。
  • 增加同步tegra Lib: /usr/lib/aarch64-linux-gnu/tegrascripts/apollo_base.sh line 120
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/apollo/lib:/apollo/bazel-genfiles/external/caffe/lib:/home/caros/secure_upgrade/depend_lib:/usr/lib/aarch64-linux-gnu/tegra
  • why ?
  • docker/scripts/dev_start.sh line 129 and 130: 增加devices nvhost* 、nvmap
devices="${devides} $(find_device nvhost*)"
devices="${devides} $(find_device nvmap)"
  • 增加共享库目录cuda目录，在docker/scripts/dev_start.sh start line 161，增加如下内容：
-v /usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra:ro  \
-v /usr/local/cuda:/usr/local/cuda:ro  \
-v /usr/local/cuda-8.0:/usr/local/cuda-8.0:ro  \
  • aarch64版本的apollo需要自己配置所需的一部分依赖包，以及编译aarch64平台所需的几个第三方动态库。
  • 有兴趣的可以直接做成脚本，一键安装。

安装ROS

  • 在arm架构环境下，ros 使用的动态类库。
  • solution :
tar zxvf ros-indigo-apollo-1.5.1-aarch64.tar.gz
rsync -av ros/ /apollo/third_party/ros_aarch64
source /apollo/third_party/ros_aarch64/setup.bash

  • 需要自己准备aarch64版本的ros

  • 1.准备include

    cp -a /media/nvidia/{ssd}/ros/include/* /usr/include/.

  • 将ros目录拷贝至 /home/tmp/ros

安装Caffe

  • aarch64版本需要自己准备caffe的依赖包，为了方便直接模仿x86版本，在external目录下手工添加@caffe and caffe dir。
  • 拷贝x86下面的/usr/include/caffe 目录，并打包到移动硬盘；
  • 将其从移动硬盘拷贝到TX2中的运行容器：
cp -a /media/nvidia/{ssd}/caffe /usr/include/.

安装ipopt

  • 编译Apollo需要的coin类库。
# 下载最新版本
wget http://www.coin-or.org/download/source/Ipopt/Ipopt-3.12.9.tgz
# 解压
tar xvzf Ipopt-3.12.9.tgz
  • 按照官方给出的编译过程编译：
  • Get IPOPT third-party packages:
cd ~/Ipopt-3.12.9/ThirdParty/Blas
./get.Blas
cd ../Lapack
./get.Lapack
cd ../Mumps
./get.Mumps
cd ../Metis
./get.Metis
cd ../../                        #Go back to the IPOPT base dir
  • Compile IPOPT:
cd ~/Ipopt-3.12.9/
mkdir build
cd build
../configure --build=arm-linux   #  
make -j 4                        #Compile using 4 cores (if you have them)
make install
  • Apollo需要的部分：
mkdir /usr/local/ipopt
mkdir /usr/local/ipopt/include

# include 
cp -a /media/nvidia/{ssd}/install_apollo/IPopt/include/coin/*  /usr/include/.

# lib
cp -a /media/nvidia/{ssd}/install_apollo/IPopt/lib/*  /usr/include/.
  • install reference :
https://medium.com/@notus.li/install-ipopt-on-ubuntu-16-04-e2644fa93545
http://richbits.rbarnes.org/install-jmodelica-on-lubuntu-1504.html

安装local_integ

  • 融合定位的模块，目前是以so的形式开放，所以可以将x86中已经编译好的部分拿过来使用。
cp -r /media/nvidia/{ssd}/local_integ /usr/local/

  • 继续编译。

  • 安装qpOASES

  • 这是个数值求解库，apollo的x86版是以依赖包的形式，aarch64是改为直接使用动态库，但是需要自己在平台下编译，否则链接报错。

# get release package 
wget http://www.qpoases.org/go/qpOASES-3.2.1.zip 
# unzip 
unzio release 
# compilation of the qpOASES library libqpOASES.a (or libqpOASES.so) and test examples:
make 

# include 类库包含
cp -a /apollo/third_party/qpOASES-3.2.1/include/*  /usr/include/.
cp -a /apollo/third_party/apOASES-3.2.1/bin/libqp* /usr/lib/

  • This library libqpOASES.a provides the complete functionality of the qpOASES software package. It can be used by, e.g., linking it against a main function from the examples folder. The make also compiles a couple of test examples; executables are stored within the directory /bin.

  • debug

# 如果报错，使用指令`ld -lpq_oases --verbose`查看编译所需动态库名称
mv /usr/lib/libqpOASES.so /usr/lib/libqp_oases.so
mv /usr/lib/libqpOASES.a /usr/lib/libqp_oases.a

安装gflags

  • GFlags是Google开源的一个命令行flag（区别于参数）库。在容器环境中安装：
sudo apt-get install libgflags-dev

安装glog

  • glog是一个C++轻量日志库，支持常用的日志功能：例如日志级别，自定义级别，条件输出，DEBUG日志等，同时支持gflag的参数配置方式。
  • 适配过程同上，在容器环境中直接安装
apt-get install libgoogle-glog-dev
  • 运行Demo

问题集

  • 问题：openssldev库未安装：
scripts/extract-cert.c:21:25: fatal error: openssl/bio.h: No such file or directory
  • 解决方案：重新安装openssldev库
sudo apt-get install libssl-dev
  • 问题:arm架构下的头文件缺失：
ERROR: /apollo/modules/perception/traffic_light/util/BUILD:5:1: C++ compilation of rule '//modules/perception/traffic_light/util:perception_traffic_light_util' failed (Exit 1).
In file included from modules/perception/traffic_light/util/color_space.cc:36:0:
./modules/perception/traffic_light/util/color_space.h:39:23: fatal error: immintrin.h: No such file or directory
compilation terminated.
INFO: Elapsed time: 223.112s, Critical Path: 38.69s
============================
[ERROR] Build failed!hu
[INFO] Took 227 seconds
============================
  • 解决方案：
  • 因为目前Apollo对于Arm架构的支持还不够完整，尤其是交通灯识别模块，存在以下两方面的问题：
是色彩转换部分的问题
是caffe库的问题
  • 这个适配内部团队正在进行中，当前的编译将跳过该模块。
  • 在apollo.sh中做如下修改，不编译traffic_light模块：
function generate_build_targets() {
  if [ -z $NOT_BUILD_PERCEPTION ] ; then
    BUILD_TARGETS=`bazel query //... except //modules/perception/traffic_light/...`
  • 问题：qp类库问题
[INFO] Building on aarch64...
INFO: Reading 'startup' options from /apollo/tools/bazel.rc: --batch_cpu_scheduling
INFO: Found 1172 targets...
ERROR: /apollo/modules/calibration/lidar_ex_checker/BUILD:23:1: Linking of rule '//modules/calibration/lidar_ex_checker:lidar_ex_checker' failed (Exit 1).
/usr/bin/ld: cannot find -lqp_oases
collect2: error: ld returned 1 exit status
INFO: Elapsed time: 8.190s, Critical Path: 0.51s
============================
[ERROR] Build failed!
[INFO] Took 12 seconds
============================
  • 解决方案
ld -lqp_oases --verbose
==================================================
attempt to open //usr/local/lib/aarch64-linux-gnu/libqp_oases.so failed
attempt to open //usr/local/lib/aarch64-linux-gnu/libqp_oases.a failed
attempt to open //lib/aarch64-linux-gnu/libqp_oases.so failed
attempt to open //lib/aarch64-linux-gnu/libqp_oases.a failed
attempt to open //usr/lib/aarch64-linux-gnu/libqp_oases.so failed
attempt to open //usr/lib/aarch64-linux-gnu/libqp_oases.a failed
attempt to open //usr/local/lib/libqp_oases.so failed
attempt to open //usr/local/lib/libqp_oases.a failed
attempt to open //lib/libqp_oases.so failed
attempt to open //lib/libqp_oases.a failed
attempt to open //usr/lib/libqp_oases.so failed
attempt to open //usr/lib/libqp_oases.a failed
attempt to open //usr/aarch64-linux-gnu/lib/libqp_oases.so failed
attempt to open //usr/aarch64-linux-gnu/lib/libqp_oases.a failed
ld: cannot find -lqp_oases
root@in_dev_docker:/apollo# cp /usr/local/lib/libqpOASES.so /usr/local/lib/libqp_oases.so 
root@in_dev_docker:/apollo# cp /usr/local/lib/libqpOASES.a /usr/local/lib/libqp_oases.a
  • 错误
ERROR: /apollo/modules/calibration/lidar_ex_checker/BUILD:23:1: Linking of rule '//modules/calibration/lidar_ex_checker:lidar_ex_checker' failed (Exit 1).
bazel-out/local-dbg/bin/_solib_arm/libmodules_Scommon_Smath_Sqp_Usolver_Slibactive_Uset_Uqp_Usolver.so: undefined reference to `qpOASES::QProblem::init(double const*, double const*, double const*, double const*, double const*, double const*, double const*, int&, double*, double const*, double const*, qpOASES::Bounds const*, qpOASES::Constraints const*, double const*)'
bazel-out/local-dbg/bin/_solib_arm/libmodules_Scommon_Smath_Sqp_Usolver_Slibactive_Uset_Uqp_Usolver.so: undefined reference to `qpOASES::QProblem::QProblem(int, int, qpOASES::HessianType, qpOASES::BooleanType)'
collect2: error: ld returned 1 exit status
INFO: Elapsed time: 325.592s, Critical Path: 45.69s
============================
[ERROR] Build failed!
[INFO] Took 334 seconds
============================
  • 错误：
\cc_test(
    name = "active_set_qp_solver_test",
    size = "small",
    srcs = [ 
        "active_set_qp_solver_test.cc",
    ],  
    deps = [ 
        ":active_set_qp_solver",
        "@gtest//:main",
    ],  
)

cpplint()

ERROR: /apollo/modules/common/math/qp_solver/BUILD:50:1: Linking of rule '//modules/common/math/qp_solver:active_set_qp_solver_test' failed (Exit 1).
  • 解决方案：
  • 方案一：安装上面的过程，重新编译qpOASES，将生成的动态类库复制到/usr/lib/目录。
  • 方案二：尝试重新安装ipopt ，没有效果。https://github.com/startcode/qp-oases/
  • https://github.com/startcode/qp-oases/archive/v3.2.1-1.tar.gz
ERROR: /apollo/modules/dreamview/backend/handlers/BUILD:23:1: C++ compilation of rule '//modules/dreamview/backend/handlers:image' failed (Exit 1).
In file included from modules/dreamview/backend/handlers/image.cc:24:0:
./modules/perception/traffic_light/util/color_space.h:39:23: fatal error: immintrin.h: No such file or directory
compilation terminated.
INFO: Elapsed time: 411.108s, Critical Path: 45.76s
============================
[ERROR] Build failed!
[INFO] Took 415 seconds
============================
  • 参考：https://devtalk.nvidia.com/default/topic/1000224/jetson-tx2/docker-on-the-tx2/
  • 错误：

ERROR: /apollo/modules/perception/obstacle/onboard/BUILD:53:1: C++ compilation of rule '//modules/perception/obstacle/onboard:perception_obstacle_lidar_subnode' failed (Exit 1).
modules/perception/obstacle/onboard/lidar_process_subnode.cc: In member function 'void apollo::perception::LidarProcessSubnode::TransPointCloudToPCL(const PointCloud2&, apollo::perception::pcl_util::PointCloudPtr*)':
modules/perception/obstacle/onboard/lidar_process_subnode.cc:370:20: error: 'isnan' was not declared in this scope
if (!isnan(pt.x) && !isnan(pt.y) && !isnan(pt.z) && !isnan(pt.intensity)) {

ERROR: /apollo/modules/common/math/qp_solver/BUILD:31:1: Linking of rule '//modules/common/math/qp_solver:active_set_qp_solver' failed (Exit 1).
//usr/local/lib/libqpOASES.so: file not recognized: File format not recognized

ERROR: /apollo/modules/calibration/lidar_ex_checker/BUILD:23:1: Linking of rule '//modules/calibration/lidar_ex_checker:lidar_ex_checker' failed (Exit 1).
bazel-out/local-dbg/bin/solib_arm/libmodules_Scommon_Smath_Sqp_Usolver_Slibactive_Uset_Uqp_Usolver.so: undefined reference to qp_oases_dpotrf_' bazel-out/local-dbg/bin/_solib_arm/libmodules_Scommon_Smath_Sqp_Usolver_Slibactive_Uset_Uqp_Usolver.so: undefined reference toqp_oases_dgemm'
collect2: error: ld returned 1 exit status

  • try start module dreamview :
bazel-bin/modules/dreamview/dreamview: error while loading shared libraries: libqpOASES.so.3.2: cannot open shared object file: No such file or directory
  • bazel问题：
# 屏蔽某些文件、目录的编译
bazel query '//... except kind(.*test, //...)' | xargs bazel build
  • 编译选项：
bazel query //... | xargs bazel build --jobs=3
  • 更改
bazel query //... | xargs bazel build --jobs=3 --define ARCH=aarch64 --define CAN_CARD=fake_can --cxxopt=-DUSE_ESD_CAN=false --cxxopt=-DCPU_ONLY
``` 

build result 
```bash 
root@in_dev_docker:/apollo/data/log# tail -f * 
==> control.out <==
/apollo/bazel-bin/modules/control/control: error while loading shared libraries: libqpOASES.so.3.2: cannot open shared object file: No such file or directory

==> dreamview.out <==
/apollo/bazel-bin/modules/dreamview/dreamview: error while loading shared libraries: libqpOASES.so.3.2: cannot open shared object file: No such file or directory

==> monitor.out <==
/apollo/bazel-bin/modules/monitor/monitor: error while loading shared libraries: libqpOASES.so.3.2: cannot open shared object file: No such file or directory

==> planning.out <==
/apollo/bazel-bin/modules/planning/planning: error while loading shared libraries: libipopt.so.1: cannot open shared object file: No such file or directory

==> roscore.out <==

注意：

  • 如果前面第一个操作有问题，导致屏幕蓝屏，或者黑屏只显示鼠标等。需要重置 dpkg 后再试，总之按提示操作就好了。
  • 解决：
sudo dpkg --configure -a

参考：

  • http://www.fzb.me/apollo/install_apollo_on_jetson_tx2.html

纠错,疑问,交流: 请进入讨论区或 点击加入Q群

获取最新文章: 扫一扫右上角的二维码加入“创客智造”公众号


标签: tx2入门教程软件篇, apollo, apollo自动驾驶软件包