Using Buildroot For Real Projects: Thomas Petazzoni Free Electrons Thomas - Petazzoni@free

Embedded Linux Conference Europe 2011
Using Buildroot
for real projects
Thomas Petazzoni
Free Electrons
thomas.petazzoni@free-
electrons.com
Free Electrons. Kernel, drivers and embedded Linux development, consulting, training and support. http://free-electrons.com
Thomas Petazzoni
I Embedded Linux engineer and trainer at Free Electrons since

2008
I Embedded Linux development: kernel and driver development,
system integration, boot time and power consumption
optimization, consulting, etc.
I Embedded Linux and driver development training, with
materials freely available under a Creative Commons license.
I http://www.free-electrons.com
I Major contributor to Buildroot, an open-source, simple and
fast embedded Linux build system
I Living in Toulouse, south west of France
Agenda
I What is Buildroot ?
I How does it work ?
I Example systems generated by Buildroot
I Recommendations for real projects
I Toolchain recommendations
I Project specific configuration and files
I Project specific packages
I Enabling application developers
I Misc best practices
I Features in 2011.11 making things better
I Conclusion
What is Buildroot ? (1/2)
I Buildroot is an embedded Linux build system

I Its goal is to build
I a cross-compiling toolchain
I a root filesystem with multiple cross-compiled libraries and
applications
I a kernel image and bootloader images
or any combination of these
I It has a kconfig configuration mechanism, identical to the one
used in the kernel
I It is completely written in make
I It builds only what’s necessary. A base system, containing
just Busybox, takes less than 2 minutes to be built from
scratch.
What is Buildroot ? (2/2)
I It is designed with simplicity in mind

I Standard languages used, relatively lightweight infrastructure.
I Very easy to add packages or customize the build system
behaviour.
I It is best-suited for small to medium-sized embedded
systems
I In the generated root filesystem, Buildroot doesn’t track which
source package installed what. You need to do complete
rebuilds for a clean root filesystem after configuration changes.
I Generates systems with no package management mechanism
on the target (no ipkg, or apt)
I 700+ packages
I Stable releases published every three months
I Active user/developer community
Basic usage (1/2)
$ make menuconfig
Basic usage (2/2)
$ make
...
...
...
$ ls output/images/
dataflash_at91sam9m10g45ek.bin rootfs.tar
rootfs.ubi rootfs.ubifs
u-boot.bin u-boot-env.bin
uImage
Ready to use !
How does it work ?
I Configuration options defined in Config.in files, and stored

in a .config file
I Buildroot starts with the toolchain: either it generates it, or
imports an existing toolchain, or uses crosstool-NG to
generate one
I It creates a basic root filesystem from a skeleton (just a few
configuration files)
I Once the toolchain is ready, Buildroot goes through the list of
selected packages. It simply fetches, configures, builds and
installs all packages, respecting their dependencies.
I It builds the kernel image and/or bootloader images, if
requested
I It creates one or more root filesystem images, using fakeroot
Top-level source code organization (1/2)
I board/
contains hardware-specific and project-specific files. Covered
later.
I boot/
contains config options and recipes for various bootloaders
I configs/
the default configurations. Covered later.
I docs/
Yes, we have some documentation.
I fs/
contains config options and makefiles to generate the various
filesystem images (jffs2, ubifs, ext2, iso9660, initramfs, tar
and more). Also contains the root filesystem skeleton
Top-level source code organization (2/2)
I linux/
contains the config options and makefile to generate the Linux
kernel, and also the kernel part of real-time extensions
(Xenomai, RTAI, etc.)
I package/
contains the config options and makefiles for all userspace
packages
I support/
various misc stuff needed for the build (kconfig code, etc.)
I target/
mostly historic directory. No longer contains anything useful
besides the default device table.
I toolchain/
config options and makefiles to build or import the toolchain
What does it generate?
In the output directory (output by default, but out-of-tree build

with O= is supported):
I build/, a directory with one sub-directory per component
built. The directory contains the source of that component,
and this is where it is built.
I host/, a directory that contains the utilities built for the host
machine, including the toolchain
I A subdirectory, host/usr/<tuple>/sysroot/ contains the
toolchain sysroot, with all libraries and headers needed to build
applications for the target.
I staging/, a historical symbolic link to
host/usr/<tuple>/sysroot
I target/, the target root filesystem (without device files)
I images/, where final images are stored
Example 1: Multi-function device
I The device is an ARM AT91-based platform with GPS, RFID

readers, GSM modem, Ethernet and USB.
I The Buildroot configuration:
I CodeSourcery ARM glibc toolchain
I Linux kernel
I Busybox for the basic system
I Dropbear for SSH access (debugging)
I Qt with only QtCore, QtNetwork and QtXml, no GUI
I QExtSerialPort
I zlib, libxml2, logrotate, pppd, strace, a special RFID library,
popt library
I The Qt application
I JFFS2 root filesystem
I Filesystem size: 11 MB. Could be reduced by using uClibc.
I Build time: 10 minutes on a fast build server (quad-core i7,
12 GB of RAM)
Example 2: vehicle navigation system
I An x86-based system, with an OpenGL application for vehicle

navigation system.
I External glibc toolchain generated with crosstool-NG
I The Grub bootloader
I Linux kernel, of course
I Busybox
I A part of the X.org stack (the server, a few drivers, and some
client libraries), including libdrm, Mesa
I The fglrx ATI proprietary OpenGL driver
I ALSA utils, ALSA library, V4L library, Flashrom, LM Sensors,
Lua, Dropbear, Ethtool
I The OpenGL application and its data
I Filesystem size: 95 MB, with 10 MB of application (binary +
data) and 45 MB (!) of fglrx driver.
I Build time: 27 minutes on a fast build server (quad-core i7,
12 GB of RAM)
Toolchain
Three choices:
I Buildroot builds a toolchain. This is limited to uClibc based
toolchains, and the toolchain is rebuilt completely at every
complete Buildroot rebuild.
I Buildroot uses crosstool-NG as a back-end to generate the
toolchain. More choices available (glibc, eglibc, uClibc
supported, more gcc versions, etc.). However, requires
toolchain rebuild at every complete Buildroot rebuild.
I Buildroot can import external toolchains. This is definitely
the mechanism I recommend.
I Allows to re-use CodeSourcery toolchains, or custom
toolchains built with crosstool-NG or Buildroot
I Importing the toolchain into Buildroot takes just a few
seconds, which saves the toolchain build time at every
Buildroot rebuild
I The toolchain rarely needs to be changed compared to the
root filesystem, so this mechanism makes sense
External toolchains: toolchain profile
Buildroot contains presets for well-known binary toolchains such as

the CodeSourcery ones, for which Buildroot knows the properties
(C library used, etc.) and the download location.
External toolchains: custom toolchain
Buildroot can also use custom, locally installed toolchains, in

which case one must tell Buildroot a few details about the
toolchain (C library being used, location, prefix, etc.)
External toolchains HOWTO
I You can either:

I Use an existing pre-built toolchain, available publicly from the
Web
I Generate a custom toolchain, and store it pre-built next to the
Buildroot directory on the local filesystem, or in some internal
FTP/HTTP server
I And then point your Buildroot configuration to this toolchain:
I As a custom toolchain
I Or by adding a new profile so that the toolchain gets
downloaded automatically and all its configuration is already
known to Buildroot
I This way, users of the Buildroot environment don’t have to
worry about the toolchain and don’t have to wait for it to
build.
Project specific files
For each project, the recommendation is to create a directory
board/<company>/<project>/
which will be used to store:

I Configuration files
I Root filesystem additions
I Project-specific kernel/bootloader patches
I All other project specific files
Kernel and bootloader changes
I The kernel and bootloaders often require modifications for a

particular hardware platform.
I My typical workflow is the following :
I Have a branch in a kernel Git tree in which I commit the
necessary changes
I Generate patches with git format-patch
I Import them into Buildroot in the
board/<company>/<project>/linux-patches/ directory
I Configure Buildroot so that it applies the patches before
building the kernel (option BR2 LINUX KERNEL PATCH)
I The advantage is that the Buildroot environment is easy to
use for others (no external dependency on a kernel Git tree)
I The drawback is that Buildroot does not directly use your
kernel Git tree. This has been improved recently, covered later.
Kernel and bootloaders example (1/2)
$ ls board/<company>/<project>/
linux-2.6.39.config samba-script.tcl
linux-2.6.39-patches/ u-boot-1.3.4-patches/
at91bootstrap-1.16-patches/
$ ls board/<company>/<project>/linux-2.6.39-patches/
linux-0001-foobar.patch linux-0002-barfoo.patch
$ ls board/<company>/<project>/u-boot-1.3.4-patches/
u-boot-0001-barfoo.patch u-boot-0002-foobar.patch
$ ls board/<company>/<project>/at91bootstrap-1.16-patches/
at91bootstrap-0001-bleh.patch at91bootstrap-0002-blah.patch
Kernel and bootloaders example (2/2)
BR2_TARGET_UBOOT=y
BR2_TARGET_UBOOT_BOARDNAME="company_project"
BR2_TARGET_UBOOT_1_3_4=y
BR2_TARGET_UBOOT_CUSTOM_PATCH_DIR=
"board/company/project/u-boot-1.3.4-patches/"
# BR2_TARGET_UBOOT_NETWORK is not set
BR2_TARGET_AT91BOOTSTRAP=y
BR2_TARGET_AT91BOOTSTRAP_BOARD="at91sam9m10g45ek"
BR2_TARGET_AT91BOOTSTRAP_CUSTOM_PATCH_DIR=
"board/company/project/at91bootstrap-1.16-patches/"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="2.6.39"
BR2_LINUX_KERNEL_PATCH=
"board/company/project/linux-2.6.39-patches/"
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE=
"board/company/project/linux-2.6.39.config"
Root filesystem customization (1/3)
The Buildroot process to create the root filesystem is:

I Copy a skeleton to the output/target directory. The default
one is in fs/skeleton, but a different one can be used.
I Copy base libraries (C libraries and al.) from the toolchain
sysroot into the target root filesystem
I Install all packages
I Execute a configuration-specified post build script
I Generate the filesystem images in the selected formats (jffs2,
ubifs, etc.)
Depending on which modifications are necessary, you might need

to:
I Use a different filesystem skeleton if the default skeleton is
really not appropriate. If only tiny modifications are needed,
do them in the post build script. I don’t recommend this
solution as it consists in duplicating the default skeleton,
which would prevent from taking advantage of future
improvements of the default skeleton.
I Create packages to install applications, data files, etc. The
skeleton is really only for basic config files.
I Create a post build script that adjusts the root filesystem
as needed.
I Create a post-build script in

board/<company>/<project>/post-build.sh and tell
Buildroot to run it with the BR2 ROOTFS POST BUILD SCRIPT.
I This script is executed from the Buildroot main directory and
receives the target filesystem directory as argument.
I Typically, my script does:
I Tune the /etc/inittab, /etc/fstab or /etc/securetty
files
I Copy the contents of
board/<company>/<project>/rootfs-additions/ into the
target root filesystem.
In this directory, I put additional init scripts, configuration files
for network services, or override existing files.
Root filesystem customization: example script
TARGETDIR=$1
# Set root password to ’root’. Password generated with

# mkpasswd, from the ’whois’ package in Debian/Ubuntu.
sed -i ’s%^root::%root:8kfIfYHmcyQEE:%’ $TARGETDIR/etc/shadow
# Application/log file mount point

mkdir -p $TARGETDIR/applog
grep -q "^/dev/mtdblock7" $TARGETDIR/etc/fstab || \
echo "/dev/mtdblock7\t\t/applog\tjffs2\tdefaults\t\t0\t0" \
>> $TARGETDIR/etc/fstab
# Copy the rootfs additions

cp -a $BOARDDIR/rootfs-additions/* $TARGETDIR/
/dev managament
Buildroot offers four solutions for /dev management:

I static, where device files are created at build time. They are
listed in a configurable device table.
I devtmpfs, the kernel filesystem which creates dynmically
device nodes. Buildroot makes sure that your kernel has the
right options selected. Recommended solution.
I devtmpfs+mdev, which allows to trigger scripts or
applications on device insertion/removal.
I devtmpfs+udev, the full-fledge solution.
Whichever solution is choosen, there is still a device table used,
which is used to set specific permissions and ownership on certain
files or directories. The list of device tables is configured through
BR2 ROOTFS DEVICE TABLE.
Project-specific packages (1/5)
I A project typically requires one or more specific packages,

containing project-specific applications or libraries.
I In order to make the build process fully integrated, those can
be integrated in Buildroot.
I In order to isolate your packages from all other packages,
create a sub-directory for all your packages:
I Create a package/<company>/ directory
This allows to isolate your specific packages from all other
packages.
I Put a package/<company>/<company>.mk files into it, with
just:
include package/<company>/*/*.mk
This tells Buildroot to parse the makefiles of your packages.
I Obviously, if you need to package an existing open-source
component, package it under package/ and submit the patch
to the Buildroot community.
For each package:

I Create a package/<company>/<yourpkg>/ directory
I Create a package/<company>/<yourpkg>/<yourpkg>.mk
makefile that contains the recipe to build your package
I Create a package/<company>/<yourpkg>/Config.in file
that details the configuration options of your package. At
least one is mandatory, in order to enable/disable your
package.
I From package/Config.in, source your
package/<company>/<yourpkg>/Config.in, preferably in a
company-specific menu to make merges with future Buildroot
versions easier.
To create the .mk file, three infrastructures are available:
I The AUTOTARGETS infrastructure, for autotools-based
packages.
You describe the tarball URL and version, the dependencies
and configuration options, and Buildroot does all the rest.
I The CMAKETARGETS infrastructure, for CMake-based
packages.
Here as well, Buildroot does most of the work.
I The GENTARGETS infrastructure, for other packages not
using a well-known build system.
Here, Buildroot has no knowledge of the build system, so you
have to specify what needs to be done to configure, build and
install the component.
See the Buildroot documentation for more details on using these
package infrastructures.
I Typically, the package source code is downloaded as a tarball

from HTTP/FTP or from a Git/Mercurial/Subversion
repository.
I For project-specific applications or libraries, it might be useful
to store them locally.
I For small utilities, it can be directly in the package directory, in
an src/ subdirectory.
I For larger programs or libraries having their own version
control, it is possible to override the package extraction
commands to copy their source code in the build directory
The application source code is stored in

../company-application/ relative to the Buildroot source code.
MY_APPLICATION_VERSION = 1.0
MY_APPLICATION_DEPENDENCIES = \
qextserialport host-pkg-config
define MY_APPLICATION_EXTRACT_CMDS
cp -a $(TOPDIR)/../company-application/* $(@D)/
endef
define MY_APPLICATION_INSTALL_TARGET_CMDS
cp $(@D)/myapp $(BINARIES_DIR)
endef
$(eval $(call CMAKETARGETS))
During application development
I Buildroot is really an integration utility. Once a package has

been built, it is not rebuilt, even if its source code changes.
I When working on the development of a component, it is
usually more convenient to build it outside of Buildroot, for a
quicker compile/test/debug cycle.
I The upcoming source directory override feature, covered later,
makes it easier to use Buildroot during development.
Using Buildroot to build external components
I The output/host directory is the Buildroot SDK.

I output/host/usr/bin/ARCH-linux-* are the
cross-compilation utilities. A wrapper is used so that the
compiler is passed the appropriate --sysroot option. From
the user perspective, the compiler therefore automagically
finds the libraries and headers.
I output/host/usr/bin also contains other useful utilities for
the build process
I pkg-config, which is configured to look for libraries in the
right location by default.
I qmake, if Qt is used, also configured with the correct paths
I output/toolchainfile.cmake is a CMake description of
the toolchain. Can be passed to cmake using
-DCMAKE TOOLCHAIN FILE=....
Using Buildroot to build external components
A simple application
BRPATH/output/host/usr/bin/arm-unknown-linux-gcc -o prog prog.c
$(BRPATH/output/host/usr/bin/pkg-config --libs --cflags glib-2.0)
Autotools component
export PATH=$PATH:BRPATH/output/host/usr/bin/
./configure --host=arm-unknown-linux
CMake component
cmake -DCMAKE_TOOLCHAIN_FILE=BRPATH/output/toolchainfile.cmake
make
Forcing Buildroot to rebuild a package
I Buildroot keeps stamp files in the package build directory

output/build/pkg-version/. They are named
.stamp extracted, .stamp configured, .stamp built,
.stamp target installed, etc.
I Removing a stamp file and re-executing make will force
Buildroot to do the corresponding step again.
I Removing the complete package build directory will force
Buildroot to rebuild the package from scratch.
I This will be improved in the upcoming Buildroot version.
Using NFS during application development
I Mounting the root filesystem over NFS is very practical during

development
I The output/target directory created by Buildroot cannot
directly be used for NFS mount, because it does not contain
any device file and the permissions may not be correct. This is
because Buildroot runs as non-root.
I The recommended way is:
I Ask Buildroot to generate a tarball image of the root filesystem
I Uncompress this tarball image, as root, in some directory
exported by NFS.
I make && tar -xf -C /nfsroot/
output/images/rootfs.tar
I /nfsroot should be exported with the NFS options rw and
no root squash in /etc/exports
Storing the project configuration
I Once you have defined a Buildroot configuration for your

project, you want to save it somewhere and allow others to
use it.
I Just do:
I make savedefconfig
Creates a defconfig file in the top Buildroot source directory.
I mv defconfig configs/company project defconfig
I Others can then load this configuration very easily:
I make company project defconfig
Summarizing the project-specific bits
I In the end, your project-specific bits are in:

I board/<company>/<project>/ for the kernel and bootloader
patches and configuration, post-build script and root filesystem
overlay
I configs/company project defconfig for the Buildroot
configuration.
I package/company/ for your specific packages
I Your modifications are cleanly isolated from the Buildroot
core, making it easy to upgrade Buildroot when needed.
Be prepared for offline builds
I When working on a project, you’ll want to make sure that

your build can be done offline in order to not depend on
resources that may disappear from the Net.
I Here are some useful Buildroot features to do so:
I make source will download into the Buildroot download
cache all the files required to do the build
I make external-deps will list the name of all the files
(essentially tarballs) that are required to do the build.
Identifies the useful tarballs in the Buildroot download cache.
I With the BR2 PRIMARY SITE configuration option, Buildroot
will download files from a specified HTTP/FTP server before
trying to reach the official site for each package. Useful to
create an internal server.
I The location of the local download cache can be configured
with BR2 DL DIR or with the BUILDROOT DL DIR environment
variable.
In 2011.11: local method
I The upcoming 2011.11 release will natively support packages

whose source code is in a local directory.
I Simply need to use:
MYPKG SITE = /some/local/directory
MYPKG SITE METHOD = local
I Buildroot will rsync the source code from the specified
location into its build directory in
output/build/pkg-version.
In 2011.11: source directory override
I A package typically specifies a HTTP or FTP location for its

tarball
I You might want to use a local source code for this package,
instead of the official tarball
I The upcoming 2011.11 allows to specify an override file
I This file is simply a makefile with variable assignments
I Those variable assignments allows you to tell Buildroot: “for
package foo, use the source code in this directory rather than
the normal tarball location”
I ZLIB OVERRIDE SRCDIR = /home/foo/my-zlib/
I LINUX OVERRIDE SRCDIR = /home/foo/linux-2.6/
I The source code is rsync’ed from the given source directory
into the Buildroot build directory.
In 2011.11: other features
I Addition of make <pkg>-reconfigure and make

<pkg>-rebuild
I Instead of manually manipulating the stamp files, those
commands restart the package build process at the
configuration stage or at the compilation stage.
I Makes using Buildroot during kernel or application
development a lot easier.
I Support for fetching from Mercurial repository
I Support for fetching using scp, both for packages and for the
primary download site
Conclusion
I Buildroot has multiple features making it easy to customize

the generated system
I Addition of kernel and bootloader patches and configuration
I Addition of project specific configuration files and scripts
I Addition of project specific packages
I These features makes Buildroot suitable to generate
embedded Linux systems for a wide range of projects, with a
preference on moderately large systems.
I From our experience, Buildroot also remains simple enough to
be used and understood by non-Linux experts.
Questions?
Thomas Petazzoni
[email protected]
Special thanks to the Buildroot community, including Peter Korsgaard, Baruch Siach,
Thomas de Schampheleire, Arnout Vandecappelle and Will Moore for their comments
and suggestions on this presentation.
Slides under CC-BY-SA 3.0.

Using Buildroot For Real Projects: Thomas Petazzoni Free Electrons Thomas - Petazzoni@free

Uploaded by

Copyright:

Available Formats

Using Buildroot For Real Projects: Thomas Petazzoni Free Electrons Thomas - Petazzoni@free

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Using Buildroot For Real Projects: Thomas Petazzoni Free Electrons Thomas - Petazzoni@free

Uploaded by

Copyright:

Available Formats

Embedded Linux Conference Europe 2011

I Embedded Linux engineer and trainer at Free Electrons since

I Buildroot is an embedded Linux build system

I It is designed with simplicity in mind

I Configuration options defined in Config.in files, and stored

In the output directory (output by default, but out-of-tree build

I The device is an ARM AT91-based platform with GPS, RFID

I An x86-based system, with an OpenGL application for vehicle

Buildroot contains presets for well-known binary toolchains such as

Buildroot can also use custom, locally installed toolchains, in

I You can either:

For each project, the recommendation is to create a directory

which will be used to store:

I The kernel and bootloaders often require modifications for a

The Buildroot process to create the root filesystem is:

Depending on which modifications are necessary, you might need

I Create a post-build script in

# Set root password to ’root’. Password generated with

# Application/log file mount point

# Copy the rootfs additions

Buildroot offers four solutions for /dev management:

I A project typically requires one or more specific packages,

For each package:

I Typically, the package source code is downloaded as a tarball

The application source code is stored in

$(eval $(call CMAKETARGETS))

I Buildroot is really an integration utility. Once a package has

I The output/host directory is the Buildroot SDK.

I Buildroot keeps stamp files in the package build directory

I Mounting the root filesystem over NFS is very practical during

I Once you have defined a Buildroot configuration for your

I In the end, your project-specific bits are in:

I When working on a project, you’ll want to make sure that

I The upcoming 2011.11 release will natively support packages

I A package typically specifies a HTTP or FTP location for its

I Addition of make <pkg>-reconfigure and make

I Buildroot has multiple features making it easy to customize

You might also like