Traverse OpenWRT SDK for cross-compilation

Ten64 Forum
1

I would like to create a custom application using the OpenWRT SDK.
The steps I have done are:

sudo apt update
sudo apt install git-core build-essential libssl-dev libncurses5-dev unzip gawk zlib1g-dev
sudo apt install subversion mercurial
  • Prepared the SDK
./scripts/feeds update -a
./scripts/feeds install -a
sed -i 's/CONFIG_SIGNED_PACKAGES=y/CONFIG_SIGNED_PACKAGES=n/g' .config
make defconfig
make -j$(nproc)

Unfortunately make fails with:

 make[1] world
 make[2] package/compile
 make[3] -C package/toolchain compile
 make[3] -C feeds/ten64/ten64-controller-util clean-build
 make[3] -C feeds/ten64/ten64-controller-util compile
 make[3] -C package/kernel/linux compile
 make[3] -C feeds/ten64/trousers clean-build
 make[3] -C feeds/ten64/hwmon-traverse-sensors clean-build
 make[3] -C feeds/ten64/trousers compile
 make[3] -C feeds/ten64/hwmon-traverse-sensors compile
    ERROR: package/feeds/ten64/trousers failed to build.
make -r world: build failed. Please re-run make with -j1 V=s or V=sc for a higher verbosity level to see what's going on
make: *** [/home/chris/openwrt-sdk-arm64-efi_gcc-8.4.0_musl.Linux-x86_64/include/toplevel.mk:223: world] Error 1

Trying make -j1 V=s gives

...
configure: error: openssl is currently the only supported crypto library for trousers. Please install openssl from http://www.openssl.org or the -devel package from your distro
make[3]: *** [Makefile:76: /home/chris/openwrt-sdk-arm64-efi_gcc-8.4.0_musl.Linux-x86_64/build_dir/target-aarch64_generic_musl/trousers-0.3.14/.configured_68b329da9893e34099c7d8ad5cb9c940] Error 1
make[3]: Leaving directory '/home/chris/openwrt-sdk-arm64-efi_gcc-8.4.0_musl.Linux-x86_64/feeds/ten64/trousers'
time: package/feeds/ten64/trousers/compile#0.89#0.44#2.36
    ERROR: package/feeds/ten64/trousers failed to build.
...
2

I’ve just checked the SDK package, it looks like there is an error in the package feed configuration which I will need to track down.

So do this first:

sed 's/arm64-2102/arm64_2102/g' feeds.conf.default > feeds.conf

Try without this step:

./scripts/feeds install -a

This installs ALL packages from OpenWrt’s various collections (openwrt itself, packages, LuCI etc) into the SDK system, which would take quite a long time to compile.

If you are trying to compile a certain OpenWrt package which we don’t have, you can do:

$ ./scripts/feeds update -a
$ ./scripts/feeds search p190nd
Search results in feed 'packages':
p910nd                          A small non-spooling printer server
...
$ ./scripts/feeds install p910nd
$ make defconfig
$ sed -i 's/CONFIG_SIGNED_PACKAGES=y/CONFIG_SIGNED_PACKAGES=n/g' .config
$ make -j$(nproc)
 make[1] world
 make[2] package/compile
 make[3] -C package/toolchain clean-build
 make[3] -C package/toolchain compile
 make[3] -C feeds/packages/net/p910nd clean-build
 make[3] -C feeds/packages/net/p910nd compile
 make[3] -C package/kernel/linux compile
 make[2] package/index

After this, the compiled OpenWrt package (.ipk) will be in bin/packages/aarch64_generic:

$ find bin/packages/aarch64_generic/
...
bin/packages/aarch64_generic/packages/p910nd_0.97-9_aarch64_generic.ipk

If you are trying to add your own package, you can follow the guide at OpenWrt.org: “Hello, world!” for OpenWrt /Creating a package from your application

3

I am trying to compile with openssl:

[ 16%] Building CXX object libs/platformlinux/CMakeFiles/platformlinux.dir/src/IPCCommsSSLClientLinux.cpp.o
In file included from /home/chris/openwrt-sdk-arm64-efi_gcc-8.4.0_musl.Linux-x86_64/build_dir/target-aarch64_generic_musl/dpx596-1.0/libs/platformlinux/src/IPCCommsSSLClientLinux.cpp:19:
/home/chris/openwrt-sdk-arm64-efi_gcc-8.4.0_musl.Linux-x86_64/build_dir/target-aarch64_generic_musl/dpx596-1.0/libs/platformlinux/inc/platformlinux/IPCCommsSSLClientLinux.hpp:16:10: fatal error: openssl/ssl.h: No such file or directory
#include <openssl/ssl.h>
^~~~~~~~~~~~~~~
compilation terminated.

4

Two things I can think of:

Is your package pulling in openssl as a dependency, e.g:

define Package/example
  DEPENDS:=+libopenssl 
...
endef

and is libopenssl installed into the SDK? (it may not be if you are developing directly inside the SDK)

./scripts/feeds install libopenssl
make oldconfig # to ensure openssl is selected
5

Reports:

chris@chris-VirtualBox:~/openwrt-sdk-arm64-efi_gcc-8.4.0_musl.Linux-x86_64$ ./scripts/feeds install libopenssl
WARNING: Makefile 'package/feeds/dpx596packages/dpx596/Makefile' has a dependency on 'libopenssl', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'eip197-mini-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'r8169-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'e100-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'bnx2-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'bnx2x-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'ar3k-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'mwifiex-sdio-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'kmod-phy-bcm-ns-usb2', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'edgeport-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'kmod-phy-bcm-ns-usb3', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'amdgpu-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'radeon-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'prism54-firmware', which does not exist
WARNING: Makefile 'package/kernel/linux/Makefile' has a dependency on 'rtl8192su-firmware', which does not exist
WARNING: No feed for package 'libopenssl' found
6

It sounds like your SDK doesn’t have the package feeds in it. Running ./scripts/feeds update -a should fix that.

You will need to alter the feeds.conf.default if you haven’t already - see up thread or just

sed 's/arm64-2102/arm64_2102/g' feeds.conf.default > feeds.conf
7

Thanks.

For others, also ensure that all required feeds are enabled in feeds.conf. For my SDK (openwrt-sdk-arm64-efi_gcc-8.4.0_musl.Linux-x86_64), feeds.conf looks like

src-git packages https://git.openwrt.org/feed/packages.git;openwrt-21.02
src-git luci https://git.openwrt.org/project/luci.git;openwrt-21.02
src-git routing https://git.openwrt.org/feed/routing.git;openwrt-21.02
src-git telephony https://git.openwrt.org/feed/telephony.git;openwrt-21.02
#src-link custom /usr/src/openwrt/custom-feed
src-git ten64 https://gitlab.com/traversetech/ls1088firmware/ten64-openwrt-feed.git
8

For a packages set called PACKAGES, and a package within the packages called PACKAGE:

When compiling with the SDK, the parts required seem to be:

  • The source code
  • The PACKAGES feeds directory (usually sym-link in SDKROOT/feeds) with Makefile
  • Add PACKAGES feeds directory to SDKROOT/feeds.conf

ASIDE: If at this point I want to “clean” build, I do:

  • rm -rf SDKROOT/feeds/PACKAGES*
  • rm -rf SDKROOT/build-dir/target-aarch64_generic_musl/PACKAGE-VERSION

To compile I do:

  • cd SDKROOT
  • ./scripts/feeds update PACKAGES
  • ./scripts/feeds install -a -p PACKAGES
  • export PATH=SDKROOT/staging_dir/host/bin:$PATH
  • make package/PACKAGE/compile

My questions are:

(1) If I change a source file, what do I have to do for the change to be recompiled (hopefully without recompiling everything)?

(2) If I change the feeds Makefile, what do I have to do for the build system to recognise the changes and re-build the required sources?

9

Firstly, The SDK, by default, will remove source files after it’s done building them. It will make your life easier if you turn it off.

make menuconfig → “Advanced Configuration Options” → deselect “Automatic removal of build directories”

After you do that, you should be able to go into build-dir/target-aarch64_generic_musl/PACKAGE-VERSION and modify a source file.

(Careful! This will get deleted if you clean the SDK, perhaps a better way is to regularly copy from your working source tree into here)

After that, you should be able to rebuild the package with your latest changes (make package/PACKAGE/compile)

Incrementing the package version field (PKG_VERSION) will cause the SDK to extract the package sources from new.

This might be a better way to force a clean rebuild:

make package/PACKAGE/clean
make package/PACKAGE/compile

Sometimes (often when making major changes to the package Makefile), the package information cache gets out of sync. It can help to remove it and force the SDK to read all the package makefiles again:

rm -rf tmp
10

The examples you provide all do a clean build - like “make clean; make”.

How do I do a build without a complete copy and clean build when a source file changes?
Is there dependency checking?

11

If you change files under build_dir/target-aarch64_generic_musl/PACKAGENAME-PACKAGEVER it should do a recompile in place without extracting/copying the source again. It will do this for all dependencies in the package chain as well.

If it doesn’t recognize any source changes you do, I would check the makefile of the source code itself to ensure the correct files are flagged as source.