BeagleBone Black, Mac OS X, cross-compile toolchain notes

BeagleBone Black - Mac OS X, 

Cross-compile toolchain setup, crosstool-ng

Version tracking:

  • Mac OS X 10.6 (Lion - yes, I hate upgrading)
  • crosstool-ng v1.20
  • eglibc 2.13 (BBB ships with debian) - compatible with glibc 2.13
  • gcc 4.6.3 (BBB ships with debian)
  • linux kernel 3.8.13 (BBB ships with debian)


Crosstool Install:

There is always more than one way to skin a cat. You can either build crosstool-ng or you can have MacPorts or HomeBrew set up a compiler toolchain builder. If you chose to build crosstool-ng yourself a good outline is here. I used brew and still had to build some of external tools. 

The first step in setting up crosstool-ng is to install it via MacPorts or HomeBrew. Since, I used brew, rather than macports. Step zero is to set up $PATH correctly so that they play well together if you prefer to use one over the other occasionally. In my case, I added the repository for brew to $PATH before anything else, so that when I use brew via terminal it does not conflict with the apple versions of tools.

Once brew is properly configured, you can then install crosstool-ng directly with a few tweaks to the repo file. This is mostly because the newer versions of xcode toolchain have started do not include the gnu versions of tools. You will be missing the gnu grep and gettext (which may be needed for Maverick or Yosemite, I just installed it and had no probs, with below). 

Most dependency conflicts can be resolved by using homebrew/dupes. The exception being grep... there's the darwin version and the gnu version... for that you should edit the crosstool-ng.rb file as follows rather than install a parallel version as some have suggested. Note that when you do brew update your changes will be lost to the crosstool-ng.rb file.

$: cd /
$: brew update
$: brew tap homebrew/dupes
$: brew install gettext
$: brew edit crosstool-ng

Add to bottom of the depends_on section of the file:
depends_on 'grep' => '--default-names'

After saving:
$: brew install crosstool-ng

This should download, build and install crosstool and it's dependencies without any hiccups.
Note: you should get a caveat at the end about 'installing a modern gcc compiler' to use crosstool-ng.


Create a case-sensitive sparse image disk using Disk Utility:

Next you need to create a 'spare image' that is 'case-sensitive' to build the toolchain for BBB's ARM processor. I used disk utility and there were a few gotchas I ran into while attempting to do this. The first is that you cannot create a sparse image on the 'top-level' of your Mac... it must be created in your user space. So, I just created it on ~/Desktop initially. But keep in mind that where ever you create the spare image, your Mac will mount a path to it in /Volumes/$Name.
  1. Open Disk Utility
  2. Select the top-level hard drive (not the disk... the mac formatted drive space)
  3. File > New > Burn Disk Image...
  4. Allocate 5GB, case-sensitive, sparse image disk

Now you have somewhere to compile the toolchain. Keep going, you're nearly started...


Configure crosstool-ng environment tools:

After creating the spare image disk, navigate to it and continue.

$: cd /Volumes/ct-ng

Even if you configure ct-ng properly, you will likely get some warnings after the tarballs are downloaded because there are references to autoconf that are not resolved. These will be warning aclocal and automake not installed, and it will say the version that crosstool-ng was built against.

Note: If you're not sure about the autoconf or you don't trust me... just run ct-ng menuconfig and press <esc><esc> and <enter> to save a default .config file on the sparse image disk location. Then you can run ct-ng build to build the toolchain and you will see the warnings in the terminal just after all the tarballs are downloaded, extracted, patched and the first pass over the C Library starts (about 8-15min into the process) - you will see the warnings. It will continue, and probably build, but my feel was "it ain't right".

Here's what I did to resolve the autoconf warnings. These are the numbers for crosstool-ng v1.20, below (the warning I got had v2.65):

$: cd /Volumes/ct-ng
$: curl -OL http://ftpmirror.gnu.org/autoconf/autoconf-2.65.tar.gz
$: tar xzf autoconf-2.65.tar.gz
$: cd autoconf-2.65
$: ./configure --prefix=/Volumes/ct-ng/autotools-bin
$: make
$: make install
$: export PATH=$PATH:/Volumes/ct-ng/autotools-bin/bin

$: cd /Volumes/ct-ng
$: curl -OL http://ftpmirror.gnu.org/automake/automake-1.11.tar.gz
$: tar xzf automake-1.11.tar.gz
$: cd automake-1.11
$: ./configure --prefix=/Volumes/ct-ng/autotools-bin
$: make
$: make install

$: cd /Volumes/ct-ng
$: curl -OL http://ftpmirror.gnu.org/libtool/libtool-2.4.tar.gz
$: tar xzf libtool-2.4.tar.gz
$: cd libtool-2.4
$: ./configure --prefix=/Volumes/ct-ng/autotools-bin
$: make
$: make install

$: cd /Volumes/ct-ng
$: rm -f autoconf-2.65.tar.gz
$: rm -f automake-1.11.tar.gz
$: rm -f libtool-2.4.tar.gz
$: rm -rf autoconf-2.65
$: rm -rf automake-1.11
$: rm -rf libtool-2.4

Now you should have autoconf installed and added to your $PATH for ct-ng to use.


Verify version info on BBB:

Next, you have to configure crosstool-ng with the menuconfig to target the architecture that you are trying to compile against to build the toolchain. In my case, I had to dig around on the BBB to find the version of gcc and the kernel numbers to make sure they were all correct.

At this point you should be able to ssh into the BBB and get this info:
$: ssh root@192.168.7.2
root@beaglebone:~# ldd --version
root@beaglebone:~# cat /proc/version
root@beaglebone:~# gdb --version

The first command will tell you the version of glibc and the second command will give you the version of the kernel and gcc. After you've verified that it is correct for the board you simply navigate the config editor and tweak all the settings until it can compile properly.


Configure crosstool-ng '.config' file for build:

I had to modify the menu configuration a lot before I actually got something working. My first few attempts were quite futile for various configuration settings that I had missed. It takes about 30-1hr to compile and install everything so a little upfront research is a good time saver. My suggestions are are to first briefly review the Cortex-A8 documentation, the gcc compiler options for ARM, and the gcc installation options at the links below before you get started building.
Understanding the above options are the key to navigating some of the info in the menu configuration setup in crosstool-ng. You can enter the tool by using:
$: cd /Volumes/ct-ng
$: ct-ng menuconfig

The specific options I would recommend tweaking are the following:
  • Paths and misc options -> Warning from tools' builds = y
  • Paths and misc options -> Prefix Dir = (/Volumes/ct-ng/$your_choice)
  • Target options -> Build multilib toolchain = y
  • Target options -> Architecture level = (armv7-a)
  • Target options -> Tune for cpu = cortex-a8
  • Target options -> Floating point = (auto (let gcc decide))
  • Target options -> Target CFLAGS = remove (--with-float=hard)
    • Note: see Toolchain option below on Tuple's alias
  • Target options -> Use EABI = y
  • Toolchain options -> Tuples vendor string = ($your_choice)
  • Toolchain options -> Tuple's alias = (arm-linux-gnueabi)
    • Note: this comes by default as 'arm-linux-gnueabihf' which specifies 'hard-float' or that the lib is a hardware floating point library... I noticed that the 'hard' config caused VFP errors when compiling gcc against 'armv7-a' and 'cortex-a8'. Also, I wanted a multilib toolchain for more compiler optimization later.
  • Operating system -> Linux kernel version = (3.8.13)
  • C Library -> C Library = (glibc)
  • C Library -> glibc version = (2.13)
  • C Library -> extra target CFLAGS = (armv7-a)
  • C Compiler -> gcc version = (4.3.6)
  • C Compiler -> C++ = y
  • C Compiler -> Enable LTO = y
    • I wasn't sure on this, but Mac uses LTO and it seemed to be only for the compile side... so, I said yes to the file bloat... may change this later?
  • C Compiler -> Decimal floats = (auto)
  • Debug facitiies -> dmalloc = n
    • Note: dmalloc will give you errors building on Mac, unfortunately you cannot use it. See this post.
  • Debug facilities -> duma = y
  • Debug facilities -> gdb = y
  • Debug facilities -> ltrace = y
  • Debug facilities -> strace = y
    • Note: obviously all of the above in 'Debug facilities' are optional...
That's it. The only other gotcha is... The terminal process will need you to set the ulimit to higher value so you don't randomly get some unknown errors because the process has exited from the memory growing too large. At this point you should be able to build the toolchain:
$: cd /Volumes/ct-ng
$: ulimit -n 4096
$: ct-ng build

Note: If you run into errors, I highly recommend:
$: cd /Volumes/ct-ng
$: ct-ng clean
$: ct-ng build

The clean command will also remove the tarballs, so... you can always copy the tarball directory before you clean from .build/tarballs to save you the trouble of downloading everything over and over. If you change something in the .config though, you may want to disregard this statement? It depends on if you know what you're doing... 

  • I noticed that when you run ct-ng clean, it removes all the extracted and patched files from .build/src
  • Restarting ct-ng build after you've copied the tarballs back into the directory will skip the download process and move directly into extracting and patching in src.


Configuring Eclipse:

Great now you've got a toolchain, but how do you use it? In the end you will have a folder called $your_choice above.  In my case, I called it 'hh' for me. All of the following documentation that follows is for /Volumes/ct-ng/hh/.
  1. Download the latest version of eclipse+CDT
  2. This will require JDK 1.7 or greater - not true for Lion... grr, updating... security, grumble grumble...
If all goes well you can just get started with eclipse directly - meaning you've updated debian on the BBB side to include all your shared libraries already...

Next, create a new C++ project in eclipse:




Now that you've built the project, it's time to deploy it. 

  1. Window >> Open Perspective >> Other...
  2. Remote System Explorer
  3. Right-click on the treeview white space under 'Local' >> New Connection...
  4. Choose 'Linux' >> Next
  5. Replace 'LOCALHOST' with '192.168.7.2', maybe name the connection and give description >> Next
  6. Click 'ssh.files' >> Next
  7. Click 'processes.shell.linux' >> Next
  8. Click 'ssh.shells' >> Next
  9. Click Finish
  10. Under the new connection 'Sftp Files', drop down 'My Home' to be prompted with login
  11. Type 'root' for the user and no password, and connect
  12. Drop down 'Root' to see the entire filesystem under '/'
  13. Drag and drop your application onto the BBB
  14. ssh to the file in a terminal (check permissions) and run ./hello_world
Now the real debug work begins...

4 comments:

  1. This is pretty good stuff Hans, thanks!

    ReplyDelete
  2. Thanks Hans. I don't know if this is of any help for others but if you type ct-ng list-samples one of the options is:

    arm-cortex_a8-linux-gnueabi

    So if you then enter:

    ct-ng arm-cortex_a8-linux-gnueabi

    ... it will get you started with most of those options (I think). Some other problems I ran into (I'm trying to build under El Capitan/10.11) that I don't see mentioned...

    The first is that it complains it can't find ginstall. The solution seems to be to create a symbolic link in /Volumes/YourCaseSensitiveVolume/.build/tools/bin to ginstall. Unfortunately if you run ct-ng clean, it will erase the .build directory and you'll have to recreate the link.

    Speaking of Case Sensitive directories... there appears to be a bug in the Disk Utility GUI in that if you create a new image and then assign it to be case sensitive... if you go and do a "get info" on it after it's created, you'll find that it's not case sensitive. Fortunately you can change this after the fact and it will take.

    Finally, it throws errors with static linking so...

    nano /Volumes/YourCaseSensitiveVolume/.config

    ... and then change the following from y to n:

    CT_WANTS_STATIC_LINK=n
    CT_CC_STATIC_LIBSTDCXX=n

    After all of that if I run ct-ng build... well... it does something, and it seems to be doing a lot of something as it has one of the cores fully pegged. I left it for over 12 hours but whatever it was doing... it never finished as it was still doing it 12 hours later so I figure something is wrong but I'm not quite sure how to diagnose what it is. The point where it gets "stuck" is here:

    [EXTRA] Installing user-supplied crosstool-NG configuration
    [DEBUG] ==> Executing: 'mkdir' '-p' '/Volumes/CaSe/arm-cortex_a8-linux-gnueabi/bin'
    [DEBUG] ==> Executing: 'install' '-m' '0755' '/usr/local/Cellar/crosstool-ng/1.21.0/lib/ct-ng.1.21.0/scripts/toolchain-config.in' '/Volumes/CaSe/arm-cortex_a8-linux-gnueabi/bin/arm-cortex_a8-linux-gnueabi-ct-ng.config'

    If anyone else it trying to create a crosstool in El Capitan and you've gotten further than this, I'd love to hear from you. :)

    Cheers, Joe

    ReplyDelete
    Replies
    1. Thanks for the feedback Joe. I'm not sure about the many problems you'll face on El Capitan. I have specifically not upgraded one machine from Lion for ease of use. They did a whole number on things at Maverick too, so I've heard of many such problems there also.

      All I can say is best of luck. I'll probably have to rebuild the toolchain for the new OS X at some point soon anyway. So, I'll just update it when I get to it.

      A quick 'which ginstall' on my system (Lion) came back with:
      /usr/local/bin/ginstall

      I'm not entirely sure, but I don't know if 'usr/local' exists past Yosemite in the base install. (You might have to create it and change permission for Homebrew?) This conversation is getting into experimental land for me.

      In my experience it was a lot of trial and error and waiting for the toolchain throw a build error - hence a slow process. I would recommend building all the tools you can from source (such as autobuild, automake etc...)

      Delete
  3. "The first is that it complains it can't find ginstall. The solution seems to be to create a symbolic link in /Volumes/YourCaseSensitiveVolume/.build/tools/bin to install."

    More specifically...

    ln -s /Volumes/YourCaseSensitiveVolume/.build/tools/bin/install /Volumes/YourCaseSensitiveVolume/.build/tools/bin/ginstall

    ReplyDelete