PinePhone boots Apache NuttX RTOS

📝 28 Aug 2022

Apache NuttX RTOS booting on Pine64 PinePhone

Apache NuttX RTOS booting on Pine64 PinePhone

UPDATE: PinePhone is now officially supported by Apache NuttX RTOS (See this)

Suppose we’re creating our own Operating System (non-Linux) for Pine64 PinePhone

This article explains how we ported Apache NuttX RTOS to PinePhone. And we’ll answer the questions along the way!

Let’s walk through the steps to create our own PinePhone Operating System

Allwinner A64 SoC User Manual

Allwinner A64 SoC User Manual

§1 Allwinner A64 SoC

What’s inside PinePhone?

At the heart of PinePhone is the Allwinner A64 SoC (System-on-a-Chip) with 4 Cores of 64-bit Arm Cortex-A53

The A64 SoC in PinePhone comes with 2GB RAM (or 3GB RAM via a mainboard upgrade)…

A64’s Memory Map says that the RAM starts at address 0x4000 0000.

So our OS will run at 0x4000 0000?

Not quite! Our OS will actually be loaded at 0x4008 0000

We’ll see why in a while, but first we talk about a Very Important Cable…

PinePhone connected to USB Serial Debug Cable

PinePhone connected to USB Serial Debug Cable

§2 USB Serial Debug Cable

Can we watch what happens when PinePhone boots?

There’s a magical cable for that: USB Serial Debug Cable (pic above)…

It connects to PinePhone’s Headphone Port (pic below) and exposes PinePhone’s hidden UART Port. Genius!

(Remember to flip the Headphone Switch to OFF)

I highly recommend the USB Serial Debug Cable for PinePhone Hacking.

What secrets will the Debug Cable reveal?

We’ll find out shortly! First we need to prep our microSD Card for hacking…

PinePhone UART Port in disguise

PinePhone UART Port in disguise

§3 PinePhone Jumpdrive

Let’s watch and learn how a Linux Kernel boots on PinePhone.

We pick a small, simple Linux Kernel: PinePhone Jumpdrive

And prepare a microSD Card for Jumpdrive…

  1. Download pine64-pinephone.img.xz

  2. Write the downloaded file to a microSD Card with Balena Etcher

  3. Insert the microSD Card into PinePhone

Don’t power up PinePhone yet! We need to talk about PinePhone’s Bootloader…

PinePhone Jumpdrive on microSD

PinePhone Jumpdrive on microSD

§4 U-Boot Bootloader

What happens when we power up PinePhone?

The U-Boot Bootloader runs, searching for a Linux Kernel to boot (on microSD or eMMC)…

Whoa! These docs look so superdry…

There’s an easier way to grok U-Boot. Let’s watch PinePhone boot Jumpdrive!

U-Boot Bootloader on PinePhone

U-Boot Bootloader on PinePhone

§5 Boot Log

Now we’re ready to watch what happens when PinePhone boots…

  1. Insert the Jumpdrive microSD into PinePhone

  2. Flip PinePhone’s Headphone Switch to OFF

  3. Connect our computer to PinePhone via the USB Serial Debug Cable

  4. Launch a Serial Terminal (115.2 kbps) on our computer and connect to PinePhone…

    For Linux:

    ## Change ttyUSB0 to the USB Serial Device
    sudo screen /dev/ttyUSB0 115200

    For macOS: Use screen or CoolTerm

    For Windows: Use PuTTY

  5. Power up PinePhone

This is the Boot Log that we’ll see…

$ sudo screen /dev/ttyUSB0 115200

DRAM: 2048 MiB
Trying to boot from MMC1
NOTICE:  BL31: v2.2(release):v2.2-904-gf9ea3a629
NOTICE:  BL31: Built : 15:32:12, Apr  9 2020
NOTICE:  BL31: Detected Allwinner A64/H64/R18 SoC (1689)
NOTICE:  BL31: Found U-Boot DTB at 0x4064410, model: PinePhone
NOTICE:  PSCI: System suspend is unavailable

BL31 refers to Arm Trusted Firmware, the very first thing that runs on PinePhone.

BL31 finds the Device Tree (DTB) for the U-Boot Bootloader, and starts U-Boot…

U-Boot 2020.07 (Nov 08 2020 - 00:15:12 +0100)
DRAM:  2 GiB
MMC:   Device 'mmc@1c11000': seq 1 is in use by 'mmc@1c10000'
mmc@1c0f000: 0, mmc@1c10000: 2, mmc@1c11000: 1
Loading Environment from FAT... *** Warning - bad CRC, using default environment

starting USB...
No working controllers found
Hit any key to stop autoboot:  0 

Yep U-Boot can be stopped! Later we’ll hit some keys to stop U-Boot and run some commands.

But for now we let U-Boot do its booting thing…

switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
653 bytes read in 3 ms (211.9 KiB/s)

U-Boot scans our microSD and discovers a U-Boot Script boot.scr

(We’ll talk more about the script)

The U-Boot Script loads the Linux Kernel into RAM and starts it…

## Executing script at 4fc00000
gpio: pin 114 (gpio 114) value is 1
4275261 bytes read in 192 ms (21.2 MiB/s)
Uncompressed size: 10170376 = 0x9B3008
36162 bytes read in 4 ms (8.6 MiB/s)
1078500 bytes read in 50 ms (20.6 MiB/s)
## Flattened Device Tree blob at 4fa00000
   Booting using the fdt blob at 0x4fa00000
   Loading Ramdisk to 49ef8000, end 49fff4e4 ... OK
   Loading Device Tree to 0000000049eec000, end 0000000049ef7d41 ... OK

Starting kernel ...
/ #

The Linux Kernel is running! It works like we expect…

/ # uname -a
Linux (none) 5.9.1jumpdrive #3 SMP Sun Nov 8 00:41:50 CET 2020 aarch64 GNU/Linux

/ # ls
bin                info.sh            root               telnet_connect.sh
config             init               sbin               usr
dev                init_functions.sh  splash.ppm
error.ppm.gz       linuxrc            splash.ppm.gz
etc                proc               sys

And that’s how PinePhone boots a Linux Kernel, thanks to the U-Boot Bootloader!

§6 Boot Script

What’s boot.scr?

Found U-Boot script /boot.scr

According to the log above, the U-Boot Bootloader runs the U-Boot Script boot.scr to…

Here’s the Source File: Jumpdrive/src/pine64-pinephone.txt

setenv kernel_addr_z 0x44080000

setenv bootargs loglevel=0 silent console=tty0 vt.global_cursor_default=0

gpio set 114

if load ${devtype} ${devnum}:${distro_bootpart} ${kernel_addr_z} /Image.gz; then
  unzip ${kernel_addr_z} ${kernel_addr_r}
  if load ${devtype} ${devnum}:${distro_bootpart} ${fdt_addr_r} /sun50i-a64-pinephone-1.2.dtb; then
    if load ${devtype} ${devnum}:${distro_bootpart} ${ramdisk_addr_r} /initramfs.gz; then
      booti ${kernel_addr_r} ${ramdisk_addr_r}:${filesize} ${fdt_addr_r};
    else
      booti ${kernel_addr_r} - ${fdt_addr_r};
    fi;
  fi;
fi

(We’ll explain fdt_addr_r, kernel_addr_r and ramdisk_addr_r)

The above U-Boot Script pine64-pinephone.txt is compiled to boot.scr by this Makefile: Jumpdrive/Makefile

%.scr: src/%.txt
	@echo "MKIMG $@"
	@mkimage -A arm -O linux -T script -C none -n "U-Boot boot script" -d $< $@

(mkimage is documented here)

§7 Boot Address

What are fdt_addr_r, kernel_addr_r and ramdisk_addr_r?

They are Environment Variables defined in U-Boot. To see the variables in U-Boot…

  1. Power off PinePhone

  2. On our computer’s Serial Terminal, keep hitting Enter, don’t stop…

  3. Power up PinePhone

U-Boot should stop and reveal the U-Boot Prompt

U-Boot 2020.07 (Nov 08 2020 - 00:15:12 +0100)
Hit any key to stop autoboot:
=>

Enter printenv to print the Environment Variables

=> printenv
kernel_addr_r=0x40080000
fdt_addr_r=0x4FA00000
ramdisk_addr_r=0x4FE00000

(See the Complete Log)

When we match these addresses with our U-Boot Script, we discover…

Aha that’s why our kernel must start at 0x4008 0000!

Yep! Thus we can…

And PinePhone will boot our own OS! (Theoretically)

But there’s a catch: U-Boot expects to find a Linux Kernel Header in our OS…

§8 Linux Kernel Header

What! A Linux Kernel Header in our non-Linux OS?

Yep it’s totally strange, but U-Boot Bootloader expects our OS to begin with an Arm64 Linux Kernel Header as defined here…

The doc says that a Linux Kernel Image (for Arm64) should begin with this 64-byte header

u32 code0;              /* Executable code */
u32 code1;              /* Executable code */
u64 text_offset;        /* Image load offset, little endian */
u64 image_size;         /* Effective Image size, little endian */
u64 flags;              /* kernel flags, little endian */
u64 res2  = 0;          /* reserved */
u64 res3  = 0;          /* reserved */
u64 res4  = 0;          /* reserved */
u32 magic = 0x644d5241; /* Magic number, little endian, "ARM\x64" */
u32 res5;               /* reserved (used for PE COFF offset) */

(Source)

Let’s make a Linux Kernel Header to appease U-Boot…

§9 NuttX Header

How do we make a Linux Kernel Header in our non-Linux OS?

Apache NuttX RTOS can help!

This is how we created the Arm64 Linux Kernel Header in NuttX: arch/arm64/src/common/arm64_head.S

  /* Kernel startup entry point.
   * ---------------------------
   *
   * This must be the very first address in the loaded image.
   * It should be loaded at any 4K-aligned address.
   * __start will be set to 0x4008 0000 in the Linker Script
   */
  .globl __start;
__start:

  /* DO NOT MODIFY. Image header expected by Linux boot-loaders.
   *
   * This add instruction has no meaningful effect except that
   * its opcode forms the magic "MZ" signature of a PE/COFF file
   * that is required for UEFI applications.
   */
  add     x13, x18, #0x16  /* the magic "MZ" signature */
  b       real_start       /* branch to kernel start */

(“MZ” refers to Mark Zbikowski)

The header begins at Kernel Start Address 0x4008 0000.

At the top of the header we jump to real_start to skip the header.

(NuttX code begins at real_start after the header)

Then comes the rest of the header…

  /* PinePhone Image load offset from start of RAM */
  .quad   0x0000  

  /* Effective size of kernel image, little-endian */
  .quad   _e_initstack - __start

  /* Informative flags, little-endian */
  .quad   __HEAD_FLAGS

  .quad   0          /* reserved */
  .quad   0          /* reserved */
  .quad   0          /* reserved */
  .ascii  "ARM\x64"  /* Magic number, "ARM\x64" */
  .long   0          /* reserved */

/* NuttX OS Code begins here, after the header */
real_start: ... 

(_e_initstack is End of Stack Space)

(__HEAD_FLAGS is defined here)

(UPDATE: We don’t need to change the Image Load Offset)

What’s the value of __start?

Remember kernel_addr_r, the Kernel Start Address from U-Boot?

In NuttX, we define the Kernel Start Address __start as 0x4008 0000 in our Linker Script: boards/arm64/qemu/qemu-a53/scripts/dramboot

SECTIONS
{
  /* PinePhone uboot load address (kernel_addr_r) */
  . = 0x40080000;
  _start = .;

We also updated the Kernel Start Address in the NuttX Memory Map

We’re almost ready to boot NuttX on PinePhone!

Will we see anything when NuttX boots on PinePhone?

Not yet. We need to implement the UART Driver…

Allwinner A64 UART Controller Registers

Allwinner A64 UART Controller Registers

§10 UART Output

Our operating system will show some output on PinePhone’s Serial Debug Console as it boots.

To do that, we’ll talk to the UART Controller on the Allwinner A64 SoC…

Flip the A64 User Manual to page 562 (“UART”) and we’ll see the UART Registers. (Pic above)

PinePhone’s Serial Console is connected to UART0 at Base Address 0x01C2 8000

The First Register of UART0 is what we need: UART_THR at 0x01C2 8000

A64 UART Register UART_THR

What’s UART_THR?

UART_THR is the Transmit Holding Register.

We’ll write our output data to 0x01C2 8000, byte by byte, and the data will appear in the Serial Console. Let’s do that!

Did we forget something?

Rightfully we should wait for THR Empty (Transmit Buffer Empty) before sending our data.

And we should initialise the UART Baud Rate. We’ll come back to this.

§11 NuttX UART Macros

NuttX writes to the UART Port with some clever Arm Assembly Macros.

This Assembly Code in NuttX…

cpu_boot:
  PRINT(cpu_boot, "- Ready to Boot CPU\r\n")

(Source)

Calls our PRINT Macro to print a string at startup.

Which is super convenient because our Startup Code has plenty of Assembly Code!

What’s inside the macro?

Our PRINT Macro calls…

Which loads our UART Base Address

/* PinePhone Allwinner A64 UART0 Base Address: */
#define UART0_BASE_ADDRESS 0x1C28000

/* Print a character on the UART - this function is called by C
 * x0: character to print
 */
GTEXT(up_lowputc)
SECTION_FUNC(text, up_lowputc)
  ldr   x15, =UART0_BASE_ADDRESS  /* Load UART Base Address */
  early_uart_ready    x15, w2     /* Wait for UART ready    */
  early_uart_transmit x15, w0     /* Transmit to UART       */
  ret

(Source)

And calls early_uart_transmit Macro to transmit a character…

/* UART transmit character
 * xb: register which contains the UART base address
 * wt: register which contains the character to transmit
 */
.macro early_uart_transmit xb, wt
  /* Write to UART_THR (Transmit Holding Register) */
  strb  \wt, [\xb]
.endm

(Source)

That’s how we print a string to the console at startup!

What’s early_uart_ready?

early_uart_ready Macro waits for the UART Port to be ready to transmit, as explained here…

How do we initialise the UART Port?

Right now we don’t initialise the UART Port because U-Boot has kindly done it for us. Eventually this needs to be fixed: qemu_lowputc.S

/* UART initialization
 * xb: register which contains the UART base address
 * c: scratch register number
 */
GTEXT(up_earlyserialinit)
SECTION_FUNC(text, up_earlyserialinit)
  ## TODO: Set PinePhone Allwinner A64 Baud Rate Divisor:
  ## Write to UART_LCR (DLAB), UART_DLL and UART_DLH
  ...

(up_earlyserialinit is called by our Startup Code)

More about this in the Appendix…

We’re finally ready to boot our own PinePhone Operating System!

Apache NuttX RTOS booting on Pine64 PinePhone

Apache NuttX RTOS booting on Pine64 PinePhone

§12 PinePhone Boots NuttX

Can we boot our own OS on PinePhone… By replacing a single file on Jumpdrive microSD?

Earlier we said that we’ll overwrite Image.gz on Jumpdrive microSD to boot our own OS…

Let’s do it!

  1. Prepare a microSD Card with PinePhone Jumpdrive

    “PinePhone Jumpdrive”

  2. Follow these steps to build Apache NuttX RTOS

    “Build NuttX for PinePhone”

    Or download nuttx.bin from here (includes BASIC Interpreter)…

    NuttX Binary Image for PinePhone: nuttx.bin

  3. Compress the NuttX Binary Image

    ## Compress the NuttX Binary Image
    cp nuttx.bin Image
    rm -f Image.gz
    gzip Image

    Or download Image.gz from here (includes BASIC Interpreter)…

    Compressed NuttX Binary Image: Image.gz

  4. Overwrite Image.gz on Jumpdrive microSD

    ## Copy compressed NuttX Binary Image to Jumpdrive microSD
    ## TODO: Change the microSD Path
    cp Image.gz "/Volumes/NO NAME"
  5. Connect PinePhone to our computer with a USB Serial Debug Cable

    “Boot Log”

  6. Insert Jumpdrive microSD into PinePhone and power up

On our computer’s Serial Terminal, we see PinePhone’s U-Boot Bootloader loading NuttX RTOS into RAM…

U-Boot 2020.07 (Nov 08 2020 - 00:15:12 +0100)
Found U-Boot script /boot.scr
653 bytes read in 3 ms (211.9 KiB/s)
## Executing script at 4fc00000
gpio: pin 114 (gpio 114) value is 1
99784 bytes read in 8 ms (11.9 MiB/s)
Uncompressed size: 278528 = 0x44000
36162 bytes read in 4 ms (8.6 MiB/s)
1078500 bytes read in 51 ms (20.2 MiB/s)
## Flattened Device Tree blob at 4fa00000
   Booting using the fdt blob at 0x4fa00000
   Loading Ramdisk to 49ef8000, end 49fff4e4 ... OK
   Loading Device Tree to 0000000049eec000, end 0000000049ef7d41 ... OK

(See the Complete Log)

Then NuttX runs…

Starting kernel ...

HELLO NUTTX ON PINEPHONE!
- Ready to Boot CPU
- Boot from EL2
- Boot from EL1
- Boot to C runtime for OS Initialize

nx_start: Entry
up_allocate_heap: heap_start=0x0x400c4000, heap_size=0x7f3c000

arm64_gic_initialize: TODO: Init GIC for PinePhone
arm64_gic_initialize: CONFIG_GICD_BASE=0x1c81000
arm64_gic_initialize: CONFIG_GICR_BASE=0x1c82000
arm64_gic_initialize: GIC Version is 2

up_timer_initialize: up_timer_initialize: cp15 timer(s) running at 24.00MHz, cycle 24000
up_timer_initialize: _vector_table=0x400a7000
up_timer_initialize: Before writing: vbar_el1=0x40227000
up_timer_initialize: After writing: vbar_el1=0x400a7000

uart_register: Registering /dev/console
uart_register: Registering /dev/ttyS0

work_start_highpri: Starting high-priority kernel worker thread(s)
nx_start_application: Starting init thread
lib_cxx_initialize: _sinit: 0x400a7000 _einit: 0x400a7000 _stext: 0x40080000 _etext: 0x400a8000
nsh: sysinit: fopen failed: 2

nshn:x _msktfaarttf:s :C PcUo0m:m aBnedg innonti nfgo uInddle  L oNouptt
 Shell (NSH) NuttX-10.3.0-RC2

(Yeah the output is slightly garbled, here’s the workaround)

NuttX Shell works perfectly OK on PinePhone…

nsh> uname -a
NuttX 10.3.0-RC2 fc909c6-dirty Sep  1 2022 17:05:44 arm64 qemu-armv8a

nsh> help
help usage:  help [-v] [<cmd>]

  .         cd        dmesg     help      mount     rmdir     true      xd        
  [         cp        echo      hexdump   mv        set       truncate  
  ?         cmp       exec      kill      printf    sleep     uname     
  basename  dirname   exit      ls        ps        source    umount    
  break     dd        false     mkdir     pwd       test      unset     
  cat       df        free      mkrd      rm        time      usleep    

Builtin Apps:
  getprime  hello     nsh       ostest    sh        

nsh> hello
task_spawn: name=hello entry=0x4009b1a0 file_actions=0x400c9580 attr=0x400c9588 argv=0x400c96d0
spawn_execattrs: Setting policy=2 priority=100 for pid=3
Hello, World!!

nsh> ls /dev
/dev:
 console
 null
 ram0
 ram2
 ttyS0
 zero

Watch the Demo on YouTube

Another Demo Video

Yep NuttX boots on PinePhone… After replacing a single Image.gz file!

§13 Upcoming Fixes

Right now we’re running NuttX on a Single Arm64 CPU. In future we might run on all 4 Arm64 CPUs of PinePhone…

We fixed some issues with Arm64 Interrupts on PinePhone…

And we fixed UART Input in our UART Driver

Now we’re ready to build the Missing Drivers for PinePhone! Like MIPI DSI Display, I2C Touch Panel, LTE Modem, …

Below are tips for debugging the NuttX Boot Sequence on PinePhone…

  1. “PinePhone Boots NuttX”

  2. “Boot Sequence”

  3. “Boot Debugging”

  4. “Memory Map”

  5. “Handling Interrupts”

  6. “Dump Interrupt Vector Table”

  7. “Interrupt Debugging”

Arm64 Source Files in NuttX

Arm64 Source Files in NuttX

§14 NuttX Source Code

Apache NuttX RTOS has plenty of Arm64 Code that will be helpful to creators of PinePhone Operating Systems.

The Arm64 Architecture Functions (pic above) are defined here…

These functions implement all kinds of Arm64 Features: FPU, Interrupts, MMU, Tasks, Timers, …

The Arm64 Startup Code (including Linux Kernel Header) is at…

Previously NuttX supports only one Arm64 Target Board: QEMU Emulator.

Below are the Source Files and Build Configuration for QEMU Emulator

We clone this to create a Target Board for PinePhone…

And we start the Board-Specific Drivers for PinePhone in pinephone_bringup.c

Our Board calls the Architecture-Specific Drivers at…

The UART Driver is located at a64_serial.c and a64_lowputc.S

(More about UART Driver)

The QEMU Target for NuttX is described in this article…

§15 What’s Next

It’s indeed possible to boot our own OS on PinePhone… By replacing a single file on Jumpdrive microSD!

We’ve done that with Apache NuttX RTOS, which has plenty of code that will be helpful for PinePhone OS Developers.

Will NuttX work with all PinePhone features?

NuttX on PinePhone might take a while to become a Daily Driver

But today NuttX is ready to turn PinePhone into a valuable Learning Resource!

There’s plenty to be done for NuttX on PinePhone, please lemme know if you would like to join me 🙏

Please check out the other articles on NuttX for PinePhone…

Many Thanks to my GitHub Sponsors for supporting my work! This article wouldn’t have been possible without your support.

Got a question, comment or suggestion? Create an Issue or submit a Pull Request here…

lupyuen.github.io/src/uboot.md

§16 Notes

  1. This article is the expanded version of this Twitter Thread

  2. Check out this detailed doc on porting Genode OS to PinePhone…

    “Genode Operating System Framework 22.05”

    PinePhone’s Touch Display is explained in pages 171 to 197.

    PinePhone’s LTE Modem is covered in pages 198 to 204.

§17 Appendix: PinePhone is now supported by Apache NuttX RTOS

PinePhone is now officially supported by Apache NuttX Mainline!

Follow these steps to build and boot the master branch of NuttX

Or download the Build Outputs from…

We’ll see this on the Serial Console

We have updated these articles to point to the PinePhone code in NuttX Mainline…

§17.1 Upcoming Features

Upcoming Features for NuttX on PinePhone…

And we’ll be able to run LVGL Touchscreen Apps on PinePhone. Stay Tuned!

§17.2 Upcoming Fixes

These are the Upcoming Fixes for NuttX on PinePhone…

  1. RAM Size will be increased to 2 GB: chip.h

    // Allwinner A64 Memory Map
    // TODO: Increase RAM to 2 GB
    #define CONFIG_RAMBANK1_ADDR      0x40000000
    #define CONFIG_RAMBANK1_SIZE      MB(128)
  2. Only Single Core CPU has been tested on PinePhone: pinephone/defconfig

    We shall test Quad Core CPU on PinePhone: qemu-armv8a/nsh_smp/defconfig

    ## TODO: Enable Symmetric Multiprocessing (SMP) for PinePhone
    CONFIG_ARCH_INTERRUPTSTACK=8192
    CONFIG_DEFAULT_TASK_STACKSIZE=16384
    CONFIG_IDLETHREAD_STACKSIZE=16384
    CONFIG_PTHREAD_STACK_MIN=16384
    CONFIG_SMP=y
    CONFIG_SYSTEM_TASKSET=y
    CONFIG_TESTING_OSTEST_STACKSIZE=16384
    CONFIG_TESTING_SMP=y
  3. Enable Memory Protection so that NuttX Apps can’t access NuttX Kernel Memory and Hardware Registers.

§17.3 Porting Notes

Porting Notes for NuttX on PinePhone…

  1. Image Load Offset in the Linux Kernel Header isn’t used: arm64_head.S

    .quad 0x480000 /* Image load offset from start of RAM */

    NuttX boots OK without changing the Image Load Offset.

    (Seems the Image Load Offset is not used by the U-Boot Bootloader. It’s probably used by the Linux Kernel only)

  2. Previously the Vector Base Address Register for EL1 was set incorrectly. (See this)

    The new code doesn’t have this problem.

    (Is it due to the Image Load Offset?)

Build NuttX

§18 Appendix: Build NuttX for PinePhone

UPDATE: PinePhone is now officially supported by Apache NuttX RTOS (See this)

Follow these steps to build Apache NuttX RTOS for PinePhone…

§18.1 Download NuttX

Download the Source Code for NuttX…

## Create NuttX Directory
mkdir nuttx
cd nuttx

## Download NuttX OS
git clone \
  https://github.com/apache/nuttx \
  nuttx

## Download NuttX Apps
git clone \
  https://github.com/apache/nuttx-apps \
  apps

## We'll build NuttX inside nuttx/nuttx
cd nuttx

§18.2 Install Prerequisites

Install the Build Prerequisites below, but skip the RISC-V Toolchain…

§18.3 Download Toolchain

Download the Arm Toolchain for AArch64 Bare-Metal Target aarch64-none-elf

For Linux x64 and WSL:

For macOS:

(I don’t recommend building NuttX on Plain Old Windows CMD, please use WSL instead)

Add the downloaded Arm Toolchain to the PATH

## For Linux x64 and WSL:
export PATH="$PATH:$HOME/gcc-arm-11.2-2022.02-x86_64-aarch64-none-elf/bin"

## For macOS:
export PATH="$PATH:/Applications/ArmGNUToolchain/11.3.rel1/aarch64-none-elf/bin"

Check the Arm Toolchain…

$ aarch64-none-elf-gcc -v
gcc version 11.3.1 20220712 (Arm GNU Toolchain 11.3.Rel1)

§18.4 Build NuttX

Finally we configure and build NuttX…

## For QEMU: Configure NuttX for Arm Cortex-A53 Single Core
tools/configure.sh qemu-armv8a:nsh

## For PinePhone: Configure NuttX for PinePhone
tools/configure.sh pinephone:lcd

## Build NuttX
make

## Dump the disassembly to nuttx.S
aarch64-none-elf-objdump \
  -t -S --demangle --line-numbers --wide \
  nuttx \
  >nuttx.S \
  2>&1

(See the Build Log)

(See our Build Script)

On an old MacBook Pro 2012, NuttX builds in 2 minutes.

If we wish to use the BASIC Interpreter, follow these steps to enable it…

Then run make to rebuild NuttX.

If the build fails with this error…

token "@" is not valid in preprocessor

Look for this file in the Arm64 Toolchain…

aarch64-none-elf/include/_newlib_version.h

And apply this patch, so that it looks like this…

// Near the end of _newlib_version.h, insert this...
#define _NEWLIB_VERSION "4.2.0"
#define __NEWLIB__ 4
#define __NEWLIB_MINOR__ 2

#endif /* !_NEWLIB_VERSION_H__ */

§18.5 Output Files

The NuttX Output Files may be found here…

The NuttX Binary Image nuttx.bin will be gzipped and copied to Jumpdrive microSD as Image.gz

For Troubleshooting: Refer to these files…

This article explains how we may load the NuttX ELF Image nuttx into Ghidra for inspection…

§19 Appendix: Allwinner A64 UART

Earlier we talked about our implementation of Allwinner A64 UART

Let’s talk about these changes…

Allwinner A64 UART Register UART_THR

Allwinner A64 UART Register UART_THR

§19.1 Wait for UART Ready

How do we wait for the UART Port to be ready before we transmit data?

See the pic above. According to the Allwinner A64 UART doc (page 563, “UART”)…

We should write data to the UART Port…

ONLY WHEN the THRE Bit is set.

(THRE means Transmit Holding Register Empty)

Where’s the THRE Bit?

THRE Bit is Bit 5 (0x20) of UART_LSR.

UART_LSR (Line Status Register) is at Offset 0x14 from the UART Base Address.

In Arm64 Assembly, this is how we wait for the UART to be ready: a64_lowputc.S

/* PinePhone Allwinner A64: 
 * Wait for UART to be ready to transmit
 * xb: register which contains the UART base address
 * wt: scratch register number
 */

.macro early_uart_ready xb, wt
1:
  /* Load the Line Status Register at Offset 0x14 from UART Base Address */
  ldrh  \wt, [\xb, #0x14]

  /* Check the THRE Bit (Tx Holding Register Empty) */
  tst   \wt, #0x20

  /* If UART is not ready (THRE=0), jump back to label `1:` */
  b.eq  1b                     
.endm

§19.2 Initialise UART

How will we initialise the UART Port?

UPDATE: See the UART Configuration steps here…

According to the Allwinner A64 UART doc (page 562, “UART”)…

We might initialise the UART Port in up_earlyserialinit like so…

  1. Set DLAB Flag to allow update of UART Divisor…

    ldr  x15, =UART1_BASE_ADDRESS
    mov  x0,  #0x80
    strb w0,  [x15, #0x0C]
  2. Write the UART Divisor (Least Significant Byte) to UART_DLL…

    mov  x0, #(divisor % 256)
    strb w0, [x15, #0x00]
  3. Write the UART Divisor (Most Significant Byte) to UART_DLH…

    mov  x0, #(divisor / 256)
    strb w0, [x15, #0x04]
  4. Clear DLAB Flag to disallow update of UART Divisor…

    mov  x0, #0x00
    strb w0, [x15, #0x0C]

(Confused? x0 and w0 are actually the same register, 64-bit vs 32-bit)

Where…

TODO: What is the Serial Clock Frequency (SCLK)?

§19.3 UART Driver

We have implemented the UART Driver for PinePhone’s Allwinner A64 UART Port…

Check out the details in this article…