[ Update: IoT Edge has introduced official support of ARM-64 as of version 1.0.8. The below steps are no longer necessary. For more information see the release notes. ]
JetPack 4.2 includes an Ubuntu 18.04 environment and updates to CUDA, Tensorflow, and Open CV. One of the best changes is support for Python 3 in the version of Open CV provided. In JetPack 3.3 a build of Open CV was necessary to support Python 3, and this was not a trivial undertaking.
The following are the steps required to install IoTEdge on JetPack 4.2 (tested and known to work on the Jetson TX-2 development board). I have also included a number of tips on working with IoT Edge in this environment.
Note: there are several ways to install IoT Edge on the TX-2. These are workarounds as IoT Edge is currently unsupported. The method outlined below uses the ARM-32 version of IoT Edge running on ARM-64. This allows the Microsoft compiled version of the IoT Edge daemon to run without recompilation. For the native ARM-64 workaround see this article.
After flashing from a 16.04 machine using nvidia sdk –
Update numpy in python 3 for CV2:
OpenCV works with Python 3 in Jetpack 4.2 so that is good, but it requires that numpy be updated for Python 3 (at the time this was written). Follow these steps:
- sudo apt install python3-pip
- pip3 install -U numpy
Install docker following directions from here:
[ Note that this assumes you are setting up a development environment. If you are setting up for production, install Moby by following the instructions at https://docs.microsoft.com/en-us/azure/iot-edge/how-to-install-iot-edge-linux-arm just before the installation of IoT Edge ]
Create docker group, etc so you don’t have to always use ‘sudo’:
Add the docker group if it doesn’t already exist:
sudo groupadd docker
Add the connected user “$USER” to the docker group. Change the user name to match your preferred user if you do not want to use your current user:
sudo gpasswd -a $USER docker
Either do a
newgrp docker or log out/in to activate the changes to groups.
Using a separate hard drive as the docker store:
The TX2 does not have a lot of storage and it can easily get used during the development process. You can add another drive and choose to set it up as the storage for all docker related data. After installing your new drive, you need to configure it to be used as the main storage location for docker.
After installing docker:
- sudo service docker stop
- sudo mv /var/lib/docker <location on your new drive where you want docker files, e.g. /media/usr/data>
- rm -rf /var/lib/docker && ln -s /data/docker /var/lib/
- service docker start
Change to rotation for iotedge logs:
To avoid the docker logs filling up and causing resource issues on production machines, it is recommended to set the logs to overwrite old data once they reach a max size. The logs are located in the docker containers folder (usually //var/lib/docker but see secondary docker storage above).
You can set this up these policies for your log files and it will apply to all new containers. Because of this, it is important to set this up after docker install and before IoT Edge install. To setup the policies, create a daemon.json file in /etc/docker.
Here is an example of the necessary json:
Details can be found here:
More details in this blog: https://sandervandevelde.wordpress.com/2019/01/20/add-rolling-logging-to-your-moby-containers
Install IoT Edge:
To Install IoT Edge on the device…
- $ sudo dpkg –add-architecture armhf
- $ sudo apt-get update
- $ sudo apt-get install libc-bin libc-dev-bin libc6 libc6:armhf libc6-dev libgcc1 libgcc1:armhf locales
- $ wget http://ports.ubuntu.com/ubuntu-ports/pool/main/h/hostname/hostname_3.20_armhf.deb
- $ sudo dpkg -i ./hostname_3.20_armhf.deb
- $ sudo apt-get install -f
- $ sudo apt install libssl1.0:armhf libssl-dev:armhf
- Check the /usr/lib/arm-linux-gnueabihf directory. You should see libssl.so and libcrypto.so files with version number 1.0.0. You may see version 1.1 installed along with 1.0.0.
- You will need to create links to the 1.0.2 versions of libssl and libcrypto that iotedge is expecting. from /usr/lib/arm-linux-gnueabihf. Change to this directory, then issue an ‘ls’ command. You should see libssl.so and libcrypto.so files but with different file version 1.0.0. Setup soft links from these to version 1.0.2 as follows:
$ sudo ln -s libssl.so.1.0.0 libssl.so.1.0.2
$ sudo ln -s libcrypto.so.1.0.0 libcrypto.so.1.0.2
- $ wget -O libiothsm-std.deb https://aka.ms/libiothsm-std-linux-armhf-latest
- sudo dpkg -i ./libiothsm-std.deb
- wget -O iotedge.deb https://aka.ms/iotedged-linux-armhf-latest
- sudo dpkg -i ./iotedge.deb
- $ sudo apt-get install -f
- $ sudo nano /etc/iotedge/config.yaml
If you are a GUI person, you can also use gedit in place of nano
- Configure Azure IoTHub if you have not already, and create your edge device. Retrieve the connection string of your Edge device and add it in the indicated location in the config.yaml file. For step-by-step instructions on creating the device and retrieving the connection string, see https://docs.microsoft.com/en-us/azure/iot-edge/how-to-register-device-portal
- Change the edgeAgent config/image to reflect the architecture. For example “mcr.microsoft.com/azureiotedge-agent:1.0.0-linux-arm32v7“
Note: this may not be necessary if using Moby instead of docker
- Save and close the file
- $ sudo systemctl daemon-reload
- $ sudo systemctl restart iotedge
Run IoT Edge daemon from the command line:
If you run edge as a daemon, you may end up getting 127 errors. This just says something went wrong in the process started by the system. If you want better error info, you can try running iotedged from the command line using:
- $ sudo /usr/bin/iotedged -c /etc/iotedge/config.yaml
Note, if you run iotedged from the command line, and it configures hsm, you will need to clear that configuration before starting iotedged as a service. If not, the service will fail repeatedly at the configuring hsm stage. You can clear it by removing the libiothsm, and deleting the /var/lib/iotedge/hsm directory. Then reinstall libiothsm as before.
Drive runs out of space during development:
To avoid issues with running out of storage during development:
You can delete your local containers (non-running) and images as follows:
- Delete all containers
- docker rm $(docker ps -a -q)
- Delete all images
- docker rmi $(docker images -q)