Darbat Darwin L4kernel
Darbat Darwin L4kernel
Darbat Darwin L4kernel
This publication is distributed by the Embedded, Real-Time and Operating Systems Program of the National
ICT Australia.
THIS DOCUMENT IS PROVIDED “AS IS” WITHOUT ANY WARRANTIES, INCLUDING ANY WARRANTY
OF MERCHANTABILITY, NON-INFIRNGEMENT, FITNESS FOR ANY PARTICULAR PURPOSE, OR ANY
WARRANTY OTHERWISE ARISING OF ANY PROPOSAL, SPECIFICATION OR SAMPLE.
Permission to make digital or hard copies of this work for personal or commercial use, including redistribution,
is granted without fee, provided that the copies are distributed in tact, without any deletions, alterations or
additions. In particular, this copyright notice and the authorship must be preserved on all copies. To copy
otherwise, or to modify, requires prior specific permission.
Contact Details:
email: [email protected]
web: http://ertos.nicta.com.au/
3
Contents
1 Release Information 4
1.1 Features 4
1.2 Limitations 5
3 Booting 9
3.1 Configuring the Victim 9
3.2 Configuring the Host 9
3.3 A boot 9
4 System Architecture 12
4.1 Components 12
4.1.1 L4 12
4.1.2 Iguana 12
4.1.3 Init 13
4.1.4 Naming 13
4.1.5 Timer 13
4.1.6 I/O Kit 13
4.1.7 Darbat 13
4.1.8 Darbat Applications 13
4.2 Bootstrap 14
4.3 Darbat Server Internals 14
4.4 Darbat user-land 15
4.5 I/O Kit Internals 15
4.5.1 KEXT Interface 15
4.6 I/O Kit/Darbat Interface 16
4.7 Source-code layout 16
Bibliography 18
1 Release Information
Darbat, the L4/Darwin project is an experimental port of Darwin to the L4 microkernel to study the characteristics
of a large-scale microkernel-based system. It includes a port of I/O Kit to L4, a modified libc to communicate
with the Darbat server, and of course XNU. Many of the machine-dependent parts of XNU have been heavily
modified (pmap, thread/task creation, etc) but much left unchanged (most of BSD, and large parts of OSFMK
work without modification).
This release is a binary & hardware compatibility and performance demonstrator. This version of Darbat boots
natively on Apple Intel EFI hardware. It can also load binary I/O Kit drivers from Intel Mac OS X 10.4.6 and
user binary applications unmodified. Performance of the system call path is comparable to that of 10.4.6.
Darbat is based on publicly available Darwin and I/O Kit source code. Where incompatibilities were found
between the public source code and release binaries, we have reverse-engineered a compatibility solution using
available headers, documentation and lots of hard work. :)
Specifically, Darbat is based on the Darwin 8.2 source code (8C46), available from opendarwin.org. XNU is
based on version 792.2.4. For compatibility, the dyld version used is 44.17, from the Darwin 8.6 release.
This release is intended to be just a demonstrator. There are still many bugs and stability problems. Don’t
expect to use Darbat for any real work. Be surprised if Darbat has more than a few minutes of up-time.
1.1 Features
The summary of interesting features for this release is as follows:
Table 1.1: Cycle counts of various syscalls on a 1.83GHz iMac Core Duo (17”)
1.2 Limitations
This release has a number of limitations which prevent it from being used for serious work yet.
Text console
Darbat only supports a keyboard and text console interface for now. There is no graphics support.
ACPI
Directly related to the ICH7 chipset is that we currently only have limited support for ACPI in L4 and I/O
Kit. Because we do not have a full ACPI driver we directly program the ICH7 interrupt routing registers to
suit our static assignment. Apple has a functional ACPI I/O Kit driver, and in theory, it should be possible to
make use of it, however, such a configuration is not yet supported. We are currently working on support for
this.
Network
Networking support in Darbat currently does not work. There is experimental infrastructure for it, and we
have previously had a driver ported from Linux. We cannot release the driver due to GPL licensing problems
and we have not yet updated the networking support code to work with proper drivers.
Stability
This release is definitely more about features and performance than stability. Sorry! :)
SMP
This release of L4 and Darbat are currently uni-processor only. Booting on a dual-processor machine works,
however only one CPU is used.
Signals
BSD Signal support is very limited to some very specific cases of signal delivery. For example, Control-C
does not work.
This section how to build an image you can boot on real hardware.
2.1 Requirements
So far we have successfully booted darbat on the following machines:
The 17” MacBook Pro video configuration is known not to work, however. Currently we have no ability to boot
on either a simulated machine or inside a virtualised environment. This is because we have found no simulators
or virtualisation tools that support EFI or the full CPU feature set used by Darwin and OS X. If you have one
or know of one, please let us know.
To boot Darbat you need an Intel Mac machine to boot on, called the victim. You also need another machine,
the host. The host can be either another Intel Mac or a x86 Linux machine with a FireWire port. The host is
used to download Darbat onto the victim machine at boot time and to interact with the L4 kernel debugger.
Assembling the binary modules is relatively straight-forward. The steps are as follows and should be done on
an Intel 10.4.x Apple.
http://www.ertos.nicta.com.au/downloads/darbat/darbat-0.2-bin.tar.bz2
˜/darbat-0.2-bin % ./tools/dbg1394
Building darbat from source will requires OS X 10.4.x and Xcode 2.3. Make sure you have ‘gcc version 4.0.1
(Apple Computer, Inc. build 5341)’ or later. Anything earlier will cause a reboot early in the boot process. You
will also need the SCons build system [8]. We suggest that you use the latest pre-release, as we have discovered
that the latest stable version is rather slow.
The steps to build from the source are as follows. Binary drivers are copied in during the build, however you
must first configure a disk with applications. This must be done on an Intel OS X machine.
http://www.ertos.nicta.com.au/downloads/darbat/darbat-0.2-bin.tar.bz2
http://www.ertos.nicta.com.au/downloads/darbat/darbat-0.2-src.tar.bz2
˜/darbat-0.2-src % cp ../darbat-0.2-bin/disk.img .
˜/darbat-0.2-src % ../darbat-0.2-bin/tools/dbg1394
2.3 Tools
2.3.1 dite
The dite [2] utility is used to bundle multiple files into a single ELF binary. The resulting dite file can be loaded
to the correct addresses by a bootloader, which can jump to the registered entry point. It should be pointed out
that Dite is merely a tool for us, a convenience that helps us merge the Mach-O binary executables and support
files into a single file that can be conveniently downloaded over FireWire.
Darbat is loaded over FireWire from the bootimg.dite. This file contains the L4 microkernel, iguana, init and
timer servers, I/O Kit Darbat, an HFS+ rootdisk image and a number of plists and binary files for I/O Kit.
The dite utility comes pre-built in the binary distribution. It is automatically built from source in the source
code distribution.
2.3.2 dbg1394
The dbg1394 program is an application for booting and remote debugging of a darbat machine over a FireWire
bus. This program can be compiled on both Linux and OS X. There are pre-built binaries in the binary
distribution. Dbg1394 also supports a gdb remote interface via a TCP socket. This can be used to inspect
the victim machine’s physical memory using gdb or some graphical utility like ddd or Xcode.
2.3.3 fwload
The fwload program is a bootloader component which works with dbg1394. Fwload acts as a bridge between
the elilo loader and the L4 microkernel. Fwload is compiled from source on a Linux machine. The fwload.dmg
disk image contains the files necessary to load fwload on an Intel Mac. Simply burn this image to CD or copy
the contents to a USB thumb drive. Together dbg1394 and fwload provide a bi-directional serial communication
over the FireWire bus.
3 Booting
As mentioned in Section 2.1, booting Darbat requires two machines, linked via a FireWire cable. Darbat
bootstrap is started using elilo, the EFI version of the Linux loader lilo.
Elilo is used to load the fwload program. The fwload program parses some of the information provided by
elilo, sets up information for the L4 boot and communicates with dbg1394.
Dbg1394 on the host machine loads the resulting ELF boot image, named bootimg.dite into physical memory
on the victim machine. It then hands the entry point to fwload to jump to L4’s startup code.
3.3 A boot
Booting darbat is relatively straightforward. Connect the host and victim machines via a FireWire cable. Insert
your boot CD or plug in your USB key to the victim. Reboot the victim and hold the ‘Option’ key down from
early boot. This should enter a boot option menu allowing you to select the boot device. Depending on your
firmware version you may have to wait a few sections for the CD option to appear. Select your boot device and
press Enter. The screen should turn grey and stay there while fwload waits for a connection.
On the host machine, execute the dbg1394 program. This should perform a FireWire bus scan and locate the
victim machine. If the victim is not found, make sure they are connected correctly via FireWire, and both your
machines are 32-bit little endian.
Once fwload and dbg1394 has established a connection, fwload outputs some text to dbg1394 and then requests
a boot. At the boot request dbg1394 loads the bootimg.dite file into the remote physical memory, also writing
out information about what segments it is loading. Dbg1394 then tells fwload the entry point for L4 from the
bootimg.dite file. Fwload then jumps to this address and starts L4.
When L4 starts, it once again initialises the FireWire controller and performs a bus reset, forcing dbg1394 to
re-scan the bus. Once they have once again established a connection L4’s kernel debugger input and output is
over the FireWire connection.
You should see a lot of debug output from various OS components. Early during Darbat’s startup it will prompt
for the video output of the victim machine. Type the number of the type of victim machine into the dbg1394
program on the host machine and the boot should continue.
After much more output the display on the victim machine will clear and you should be greeted with single-user
mode launchd startup and a bash prompt.
Some early sample output is as follows:
˜/darbat-0.2-bin % ./tools/dbg1394
dbg1394 test starting
==========> dbg1394 running <==========
...
If you wish to boot the system, but stay in single user mode:
sh /etc/rc
-sh-2.05b#
The boot disk has a script, setup.sh which you can source in order to set some environment variables and mount
the local hard disk (read only). This might look as follows:
-sh-2.05b# ls
.DS_Store Desktop DF dev sbin usr
.Trashes bin etc setup.sh var
Desktop DB build mnt tmp
The path is now set so you can execute binaries off the hard disk. Some programs may or may not work
depending on your exact OS X revision.
If you are having problems booting, there may be a number of reasons why the system does not start correctly.
In some instances the dbg1394 client will freeze. This may require a FireWire bus reset (i.e. unplug & replug
the cable). Otherwise, something may have gone wrong during initialisation. Reboot the client and try again!
Anecdotally it seems the faster machines (MacBook Pro, iMac) are more reliable booting darbat than the
MacBook and Mac minis. Occasionally the hard-disk or USB keyboard do not initialise fully, even though you
get to a shell prompt. For the keyboard, this can be fixed by unplugging and re-plugging the USB keyboard.
For a laptop is is possible to attach an external keyboard. If in doubt, reboot.
4 System Architecture
Protected Mode
L4
Hardware
The name Darbat is actually used to refer to both the darbat system as a whole, as well as the L4 port of the
XNU component of Darwin.
Figure 4.1 illustrates the component layout in the Darbat system. Each component is isolated from one another
using hardware memory protection for improved security. Components typically communicate using L4 IPC
and shared memory.
The system can be logically divided into two sections. The first section is the Iguana OS [3], including the
Iguana server, the Iguana init program, the naming server and the timer driver. The other section is Darbat and
I/O Kit.
4.1 Components
A brief description of each component is as follows:
4.1.1 L4
L4 [6] is the secure, second-generation microkernel at the core of Darbat’s performance and security. A full
discussion of L4 is far beyond the scope of this document, and there are already documents which discuss it.
Our version of L4 is a variant of L4 Pistachio.
In Darbat, L4 provides fast messaging between applications and secure sharing of memory.
4.1.2 Iguana
Iguana is an OS personality for L4. Iguana runs as the privileged task and manages the policy in the system for
resource allocation. A full discussion of Iguana is also beyond the scope of this document.
Iguana provides a single address space for native L4 applications. While these applications share a single
address space, that is, a single mapping from virtual to physical addresses, it does not mean there is no
protection. Each Iguana application executes in its own hardware address space. Therefore, each Iguana
application can have different, or no permissions, on each page of virtual memory.
4.1.3 Init
4.1.4 Naming
The naming program provides a shared namespace through which Iguana applications can publish and discover
services published by other applications.
4.1.5 Timer
The timer program is a hardware timer driver for the system. Iguana programs needing timers and wake-ups
communicate with the timer application via L4 IPC. The Darbat server and I/O Kit both use the timer driver.
On x86 the timer driver uses the 8254 timer chip and can provide periodic and one-shot timers.
I/O Kit is the device driver subsystem for the Darbat system. It is, however, independent of Darbat and can
operate without it. For example, it has been used with Wombat [5], our L4/Linux port. I/O Kit is predominantly
C++ and runs purely in user-mode in Darbat.
I/O Kit is a framework suited for device drivers. It supports dynamic loading of binary Apple OS X device
drivers. As well as device drivers and device families, I/O Kit provides enough kernel support code to manage
threads, memory, interrupts and other abstractions device drivers need.
4.1.7 Darbat
The Darbat server is the XNU component of Darwin running de-privileged as an L4 task. Darbat manages
external address spaces for darbat user-land, and communicates with I/O Kit for device driver access. Operation
of I/O Kit and Darbat server is completely isolated. A failure in either should not extensively affect the other.
Darbat, being XNU, includes both the Mach and BSD components of Darwin. Darbat handles system call
requests from user-land applications and manages their address spaces.
Darbat applications are regular Darwin applications. Applications such as launchd, bash, ls, etc. are binary
compatible, however they rely on a modified libc (libSystem) and dynamic linker to support using L4 for
system calls.
4.2 Bootstrap
There are a lot of steps involved in the bringing up of the whole Darbat system. This section provides a brief
overview.
When L4 starts it is in a 32-bit direct-mapped mode inherited from EFI. L4 configures some bootstrap page-
tables (also direct-mapped). L4 then performs a PCI bus scan in order to try and find a FireWire device for the
kernel debugger. Once the FireWire device has been found and initialised, L4 can set itself up.
The first user application to start is Iguana, the privileged task. The details of the Iguana binary are stored in
L4’s kernel interface page when the boot image is created.
Iguana starts and configures itself and discovers resources such as physical memory. Iguana then starts the init
program based on information in the bootinfo records. Init executes a compile-time generated script to startup
the timer, naming, Darbat and I/O Kit.
The I/O Kitstartup sequence is no different from how it would be started in the traditional Mach-based I/O Kit.
The I/O Kitis initialised by calling the StartIOKit() function in the IOStartIOKit.cpp. This is normally done
by the Platform Expert. In the L4-based version of the I/O Kit, this is done from within the XNU emulation
library, which contains a stripped-down version of the Platform Expert.
Darbat startup is similar to that of normal XNU. Mach is bootstrapped and started, then BSD is initialised.
During BSD initialisation the rootdisk built into the dite image is mounted. Darbat then locates and begins
talking to I/O Kit to locate disk and network resources.
Darbat finishes startup by starting a user-land process, launchd. launchd is run as any other process and
dynamically linked to libSystem. launchd ultimately forks and execs a bash prompt for user interaction.
ramdisk
IOKit Shared
Text
User Address−Spaces
Virtual
"Physical"
applications requesting system calls and handling replies. I/O Kit interaction is also handled through the darbat
main thread.
For the most part the XNU kernel is unmodified. Mach IPC is still used on top of the L4 IPC mechanisms.
Because Darbat implements the pmap interface, all Mach VM operations are supported.
Driver support is provided by the I/O Kit. Darbat currently only makes use of the ICH7 disk controller (for
mounting the internal hard disk) and the USB controller (for keyboard input). Others will probably load, and
have loaded before, but in general will not function yet without further work. The current implementation does
not resolve KPI dependencies properly and these must be removed from the plists.
Darbat Process
pistachio
The pistachio directory contains the source code for the L4 microkernel.
iguana/iokit
This directory contains the entry point and main loop code for the I/O Kit server. The bulk of the code is in
libraries, however.
libs/xnuglue
Xnuglue is the support code for running the necessary Mach infrastructure for I/O Kit on L4.
darbat
This directory contains the the XNU component of darwin with darbat changes. Notable sub-directories
include igcompat and osfmk/l4 for darbat support code.
Bibliography
[4] Geoffrey Lee. I/O kit drivers for L4. BE thesis, University of New South Wales, November 2005.
[5] Ben Leslie, Carl van Schaik, and Gernot Heiser. Wombat: A portable user-mode Linux for embedded
systems. In Linux.conf.au, Canberra, April 2005.
[6] Jochen Liedtke. On µ-kernel construction. In Symposium on Operating Systems, pages 237–250, Copper
Mountain, CO, USA, December 1995.