Cross Compile TensorFlow C++ app for the Jetson TK1

Last time I’ve posted about cross compiling TF for the TK1. That however was a canned sample example from TF, based on the bazel build system.
Let’s say we want to make our own TF C++ app and just link vs. TF for inference on the TK1.
Now that’s a huge mess.
First we need to cross-compile TF with everything built in.
Then we need protobuf cross-compiled for the TK1.
Bundle everything together, cross(-compile) our fingers and pray.
The prayer doesn’t help. But let’s see what does…

I’ve learned a lot from:

Build TF

To make TF run meaningfully, we need to make two libraries: libtensorflow_framework.so and libtensorflow_cc.so.
Then bazel build it with the CROSSTOOL we created last time:

$ bazel build --crosstool_top=//arm-compiler:toolchain --cpu=armeabi-v7a --config=opt -s //tensorflow:libtensorflow_framework.so
$ bazel build --crosstool_top=//arm-compiler:toolchain --cpu=armeabi-v7a --config=opt -s //tensorflow:libtensorflow_cc.so

If you haven’t bazel cleaned since last time this step should only take a minute or so.

Build Protobuf

First we need to compile PB for the host. Trust me on this.
TF team has done an excellent job to facilitate this: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/makefile

$ sudo apt-get install autoconf automake libtool curl make g++ unzip zlib1g-dev git python
$ cd ${TF_ROOT}
$ bash tensorflow/contrib/makefile/download_dependencies.sh
$ bash tensorflow/contrib/makefile/compile_linux_protobuf.sh

Then we move to compiling PB for arm-linux

$ cd ${TF_ROOT}/tensorflow/contrib/makefile/downloads/protobuf/
$ ./autogen
$ ./configure --with-protoc=$(pwd)/../../gen/protobuf/bin/protoc --host=arm-linux --target=arm-linux --disable-shared --enable-cross-compile --prefix=$(pwd)/build --exec-prefix=$(pwd)/build --with-sysroot=${GCC_LINARO_DIR}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/ CXX=${GCC_LINARO_DIR}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++
$ make
$ make install

See what I did there? we needed a working protoc, and we can’t use the cross-compiled one since it doesn’t run on the host, it’s made for arm…

Gather everything for app cross-compilation

I’ve created a script that takes everything from the TF directory and places it where you need it:

Example app

We’ll set up the example app with CMake:

cmake_minimum_required(VERSION 2.8)
project(tf_cpp_example)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${PROJECT_SOURCE_DIR}/bin)
set(SOURCE_FILES main.cpp)
set(EXECUTABLE tf_cpp_example)
include_directories(tensorflow/include
            tensorflow/include/google/tensorflow
            tensorflow/include/nsync
            tensorflow/include/eigen3
            )
link_directories(${PROJECT_SOURCE_DIR}/tensorflow/lib)
add_executable(${EXECUTABLE} ${SOURCE_FILES})
target_link_libraries(${EXECUTABLE} tensorflow_cc tensorflow_framework)

With a very basic TF C++ code from here: https://www.tensorflow.org/api_guides/cc/guide

#include "tensorflow/cc/client/client_session.h"
#include "tensorflow/cc/ops/standard_ops.h"
#include "tensorflow/core/framework/tensor.h"
int main() {
  using namespace tensorflow;
  using namespace tensorflow::ops;
  Scope root = Scope::NewRootScope();
  // Matrix A = [3 2; -1 0]
  auto A = Const(root, { {3.f, 2.f}, {-1.f, 0.f} });
  // Vector b = [3 5]
  auto b = Const(root, { {3.f, 5.f} });
  // v = Ab^T
  auto v = MatMul(root.WithOpName("v"), A, b, MatMul::TransposeB(true));
  std::vector outputs;
  ClientSession session(root);
  // Run and fetch v
  TF_CHECK_OK(session.Run({v}, &outputs));
  // Expect outputs[0] == [19; -3]
  LOG(INFO) << outputs[0].matrix();
  return 0;
}

We also need a CMake cross-compilation toolchain file, which I’ve called arm-linux-gnueabihf.cmake:

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSROOT ${GCC_ARM_ROOT}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/arm-linux-gnueabihf/libc/)
set(CMAKE_C_COMPILER ${GCC_ARM_ROOT}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER ${GCC_ARM_ROOT}/gcc-linaro-4.9.4-2017.01-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-g++)
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

Then compile with cmake as per usual, specifying the cross compilation toolchain file

$ mkdir build
$ cd build
$ cmake -DCMAKE_TOOLCHAIN_FILE=../arm-linux-gnueabihf.cmake ..
$ make VERBOSE=1 # just for kicks

Once it builds, push the executable, libraries, etc. to the TK1

$ scp ${TF_ROOT}/bazel-bin/tensorflow/lib*.so ubuntu@***.***.***.***:/home/ubuntu/Documents/tfexample

You can also execute it remotely on the TK1, to save a terminal window:

$ ssh ubuntu@***.***.***.*** 'cd /home/ubuntu/Documents/tfexample && LD_LIBRARY_PATH=/home/ubuntu/Documents/tfexample ./tf_cpp_example'
2018-03-16 18:09:32.078588: I /home/****/Documents/tf_cpp_example/main.cpp:20] 19
-3

And that’s about it – you can now build standalone Tf 1.5+ apps in C++ for the Jetson TK1!
Enjoy!
Roy.

Build TF

Build Protobuf

Gather everything for app cross-compilation

Example app

Leave a Reply