Yaghmour Embedded Android Workshop
Yaghmour Embedded Android Workshop
Yaghmour Embedded Android Workshop
Workshop
ELCE 2014
Karim Yaghmour
@karimyaghmour
1
These slides are made available to you under a Creative Commons Delivered and/or customized by
Share-Alike 3.0 license. The full terms of this license are here:
https://creativecommons.org/licenses/by-sa/3.0/
4
Introduction to Embedded Android
● Basics
● History
● Ecosystem
● Legal framework
● Platform and hardware requirements
● Development tools
5
1. Basics
● Features
● UX Concepts
● App Concepts
6
1.1. Features (old snapshot)
● Application framework enabling reuse and replacement of components
● Dalvik virtual machine optimized for mobile devices
● Integrated browser based on the open source WebKit engine
● Optimized graphics powered by a custom 2D graphics library; 3D graphics based
on the OpenGL ES 1.0 specification (hardware acceleration optional)
● SQLite for structured data storage
● Media support for common audio, video, and still image formats (MPEG4, H.264,
MP3, AAC, AMR, JPG, PNG, GIF)
● GSM Telephony (hardware dependent)
● Bluetooth, EDGE, 3G, and WiFi (hardware dependent)
● Camera, GPS, compass, and accelerometer (hardware dependent)
● Rich development environment including a device emulator, tools for debugging,
memory and performance profiling, and a plugin for the Eclipse IDE
7
1.2. UX Concepts
● Browser-like
● iPhone-ish
● No user-concept of “task”
● Main keys:
● HOME
● SEARCH
● BACK
● MENU
● App-model allows users to safely install/test almost
anything
8
“Click” Activity #1
Activity
“Back” “Click”
Activity #2
Activity #1
“Click”
“Home”
“Back”
Activity #3
Activity #2
Activity #1
Home
Menu
Back 9
1.3. App Concepts
● No single entry point (No main() !?!?)
● Unlike Windows or Unix API/semantics in many
ways
● Processes and apps will be killed at random:
developer must code accordingly
● UI disintermediated from app “brains”
● Apps are isolated, very
● Behavior predicated on low-memory conditions
10
2. History
● 2002:
● Sergey Brin and Larry Page started using Sidekick smartphone
● Sidekick one of 1st smartphones integrating web, IM, mail, etc.
● Sidekick was made by Danger inc., co-founded by Andy Rubin (CEO)
● Brin/Page met Rubin at Stanford talk he gave on Sidekick’s development
● Google was default search engine on Sidekick
● 2004:
● Despite cult following, Sidekick wasn’t making $
● Danger inc. board decided to replace Rubin
● Rubin left. Got seed $. Started Android inc. Started looking for VCs.
● Goal: Open mobile hand-set platform
● 2005 - July:
● Got bought by Google for undisclosed sum :)
● 2007 - November:
● Open Handset Alliance announced along with Android
11
● 2008 - Sept: Android 1.0 is released
● 2009 - Feb.: Android 1.1
● 2009 - Apr: Android 1.5 / Cupcake
● 2009 - Sept.: Android 1.6 / Donut
● 2009 - Oct: Android 2.0/2.1 / Eclair
● 2010 - May: Android 2.2 / Froyo
● 2010 - Dec: Android 2.3 / Gingerbread
● 2011 - Jan : Android 3.0 / Honeycomb – Tablet-optimized
● 2011 – May: Android 3.1 – USB host support
● 2011 – Nov: Android 4.0 / Ice-Cream Sandwich – merge Gingerbread and
Honeycomb
● 2012 – Jun: Android 4.1 / Jelly Bean – Platform Optimization
● 2012 – Nov: Android 4.2 / Jelly Bean - Multi-user support
● 2013 -- July: Android 4.3 / Jelly Bean - GL ES 3.0
● 2013 -- Oct: Android 4.4 / Kit Kat - Low-memory support
● 2014 -- ???: Android ?.? / L* - ART and other pixie dust
12
3. Ecosystem
● More than a Billion devices activated
● 1.3M apps (vs. 1.2M for Apple's app store)
● 85% global smartphone marketshare (iOS is 12%)
13
3.1. Who's playing?
● Leading IP:
Y
● Google
D
● Semiconductor manufacturers:
ARM, Intel, Freescale, TI, Qualcomm, NVIDIA, ...
O
●
● Handset manufacturers:
B
● Motorola, Samsung, HTC, LG, Sony-Ericsson, ...
● Y
Tablet manufacturers:
Motorola, Samsung, Archos, DELL, ASUS, ...
R
●
● Special-purpose devices:
E
● App stores:
Android Market, Amazon App Store, V CAST Apps, B&N NOOK Apps, ...
E
14
3.2. Open Handset Alliance
● “... a group of 80 technology and mobile companies who have come together to
accelerate innovation in mobile and offer consumers a richer, less expensive, and
better mobile experience. Together we have developed Android™, the first
complete, open, and free mobile platform.”
● Unclear what OHA does or what benefits, if any, members derive
● Not an organization with board members, staff, etc. ... just an “Alliance”
● Google's Android team are the lead on all bleeding edge dev, all else tag along
● OHA is largely inactive / absent
● Comprised of:
● Mobile Operators: Sprint, T-Mobile, Vodafone, NTT Docomo, ...
● Handset Manufacturers: HTC, Motorola, LG, Samsung, Sony Ericsson, ...
● Semiconductor Companies: ARM, Freescale, Intel, NVIDIA, Qualcomm, TI, ...
● Software Companies: Google, ...
● Commercialization Companies: ...
15
4. Legal Framework
● Code access
● Code licenses
● Branding use
● Google's own Android Apps
● Alternative App stores
● Oracle v. Google
16
4.1. Code Access
● Parts:
● Kernel
● Android Open Source Project (AOSP)
● Kernel:
● Should have access to latest shipped version => GPL requirement
● Google-maintained forks at android.googlesource.com
● AOSP:
● Usually Code-drops every 6 months
● Official AOSP branches at android.googlesource.com
● Managed by “repo” tool, an overlay to “git”
● Honeycomb (3.0) code required talking to Google
● ... which hasn't precluded moders from lifting binaries off the SDK and putting
Honeycomb on all sorts of devices, including B&N's Nook ...
17
4.2. Code Licenses
● Kernel:
● GNU General Public License (a.k.a. GPL)
● AOSP:
● Mostly Apache License 2.0 (a.k.a. ASL)
● Having GPL-free user-space was a design goal
● A few GPL and LGPL parts: mainly BlueZ and DBUS
● Some key components in BSD: Bionic and Toolbox
● “external/” directory contains a mixed bag of licenses, incl. lots of GPL
● May be desirable to add GPL/LGPL components:
● BusyBox
● uClibc / eglibc / glibc
18
4.3. Branding Use
● Android Robot:
● Very much like the Linux penguin
● Android Logo (A-N-D-R-O-I-D w/ typeface):
● Cannot be used
● Android Custom Typeface:
● Cannot be used
● Android in Official Names:
● As descriptor only: “for Android”
● Most other uses require approval
● Android in Messaging:
● Allowed if followed by a generic: “Android Application”
● Compliance through CDD/CTS involved in “approval”
19
4.4. Google's own Android Apps
● The non-AOSP apps:
● Android Market
● YouTube
● Maps and Navigation
● Gmail
● Voice
● SkyMap
● ...
● Require:
● CTS/CDD Compliance
● Signed agreement w/ Google
● Inquiries: [email protected]
20
4.5. Alternative “App Stores”
● A couple of stores are already public:
● Android Market
● Amazon App Store
● V CAST Apps
● B&N NOOK Apps
● ...
● Nothing precluding you from having your own
21
4.6. Oracle v. Google
● Filed August 2010
● Patent infringement:
● 6,125,447; 6,192,476; 5,966,702; 7,426,720; RE38,104; 6,910,205; and 6,061,520
● Copyright infringement:
● Android does not use any Oracle Java libraries or JVM in the final product.
● Android relies on Apache Harmony and Dalvik instead.
● In October 2010, IBM left Apache Harmony to join work on Oracle's
OpenJDK, leaving the project practically orphaned.
● ...
● In Spring of 2012 Oracle lost both on Copyright and Patent fronts
22
5. Platform and Hardware requirements
● In principle:
● Android runs on top of Linux
● Therefore: if it runs Linux, it can run Android
● Known to have been made to work on:
● ARM
● x86
● MIPS
● SuperH
● Put in all sort of devices:
● Washers, micro-wave ovens, car systems, etc.
23
5.1. Compliance Definition Document
● Software: MUST conform to AOSP
● Application Packaging Compatibility: support “.apk” files
● Multimedia Compatibility: decoders, encoders, recording, ...
● Developer Tool Compatibility: adb, ddms, Monkey
● Hardware compatibility:
●
Display and Graphics
●
Input Devices
● Data Connectivity
●
Cameras
●
Memory and Storage
● USB
● Performance Compatibility
● Security Model Compatibility
● Software Compatibility Testing
● Updatable Software: MUST include mechanism to update
24
5.2. Compatibility Test Suite
25
6. Development tools
● Requirements
● App dev tools and resources
● App debugging
26
6.1. Requirements
● App development and debugging:
● Windows / Mac / Linux workstation
● JDK
● Eclipse w/ ADT plugin
● Highly recommended: real device(S)
● Platform development:
● GNU cross-dev toolchain
● JTAG debugger
● ... more on this later
27
6.2. App dev tools and resources
● SDK:
● android – manage AVDs and SDK components
● apkbuilder – creating .apk packages
● dx – converting .jar to .dex
● adb – debug bridge
● ...
● Emulator – QEMU-based ARM emulator
● Use KVM for x86 instead
● NDK: GNU toolchain for native binaries
● Documentation: developer.android.com
28
29
30
6.3. App debugging
● adb
● ddms
● monkeyrunner
● traceview
● logcat
● Eclipse integration (ADT plugin)
31
Concepts and Internals
1. Android Concepts 9. JNI
2. Framework Intro 10.System Server
3. Overall Architecture 11.Calling on Services
4. System startup 12.Activity Manager
5. Linux Kernel 13.Binder
6. Hardware Support 14.HAL
7. Native User-Space 15.Stock AOSP Apps
8. Dalvik
32
1. Android Concepts
● Components
● Intents
● Component lifecycle
● Manifest file
● Processes and threads
● Remote procedure calls
33
1.1. Components
● 1 App = N Components
● Apps can use components of other applications
● App processes are automagically started whenever any
part is needed
● Ergo: N entry points, !1, and !main()
● Components:
● Activities
● Services
● Broadcast Receivers
● Content Providers
34
1.2. Intents
● Intent = asynchronous message w/ or w/o
designated target
● Like a polymorphic Unix signal, but w/o
required target
● Intents “payload” held in Intent Object
● Intent Filters specified in Manifest file
35
1.3. Component lifecycle
● System automagically starts/stops/kills
processes:
● Entire system behaviour predicated on low memory
● System triggers Lifecycle callbacks when
relevant
● Ergo: Must manage Component Lifecycle
● Some Components are more complex to
manage than others
36
37
1.4. Manifest file
● Informs system about app’s components
● XML format
● Always called AndroidManifest.xml
● Activity = <activity> ... static
● Service = <service> ... static
● Broadcast Receiver:
● Static = <receiver>
● Dynamic = Context.registerReceiver()
● Content Provider = <provider> ... static
38
1.5. Processes and threads
● Processes
● Default: all callbacks to any app Component are issued to the main process thread
● <activity>—<service>—<recipient>—<provider> have process attribute to override
default
● Do NOT perform blocking/long operations in main process thread:
– Spawn threads instead
● Process termination/restart is at system’s discretion
● Therefore:
– Must manage Component Lifecycle
● Threads:
● Create using the regular Java Thread Object
● Android API provides thread helper classes:
– Looper: for running a message loop with a thread
– Handler: for processing messages
– HandlerThread: for setting up a thread with a message loop
39
1.6. Remote procedure calls
● Android RPCs = Binder mechanism
● No Sys V IPC due to in-kernel resource leakage
● Binder is a low-level functionality, not used as-is
● Instead: must define interface using Interface
Definition Language (IDL)
● IDL fed to aidl Tool to generate Java interface
definitions
40
2. Framework Introduction
● UI
● Data storage
● Security/Permissions
● ... and much more ... :
● Graphics
● Audio and Video
● Location and Maps
● Bluetooth
● NFC
41
2.1. UI
● Everything based on hierarchy of Views and
ViewGroups (layouts)
● Declared in XML or dynamically through Java
● UI components:
● Widgets
● Event handlers
● Menus
● Dialogs
● Notifications
● ...
42
2.2. Data storage
● Shared preferences
● Private primitive key-pair values
● Internal storage
● Private data on device memory
● External storage
● Public data on shared external device (SD)
● SQLite DB
● Private DB
● Network connection
● Web-based storage (REST)
43
2.3. Security/Permissions
● Most security enforced at process level: UID, GID
● Permissions enforce restrictions on:
● Per-process operations
● Per-URI access
● Applications are sandboxed
● Specific permissions required to “exit” sandbox
● Decision to grant access based on:
● Certificates
● User prompts
● All permissions must be declared statically
44
2.4. Native development
● Useful for:
● Porting existing body of code to Android
● Developing optimized native apps, especially for gaming
● Provides:
● Tools and build files to generate native code libraries from C/C++
● Way to embed native libs into .apk
● Set of stable (forward-compatible) native libs
● Documentation, samples and tutorials
● Enables:
● Calling native code from Java using JNI
● Implementing fully native apps (since 2.3)
● Doesn't allow you to:
● Compile traditional Linux/Unix apps as-is
45
3. Architecture
● Embedded Linux
● Modern phone/tablet
● System-on-Chip (SoC)
● Android
46
47
48
49
50
4. System Startup
● Bootloader
● Kernel
● Init
● Zygote
● System Server
● Activity Manager
● Launcher (Home)
51
52
4.1. Bootloader
● aosp/bootable/bootloader
● Custom bootloader for Android
● USB-based
● Implements the “fastboot” protocol
● Controlled via “fastboot” cli tool on host
● aosp/bootable/recovery
● UI-based recovery boot program
● Accessed through magic key sequence at boot
● Usually manufacturer specific variant
53
● Flash layout:
0x0000038600000x000003900000 : "misc"
0x0000039000000x000003e00000 : "recovery"
0x000003e000000x000004300000 : "boot" Kernel
0x0000043000000x00000c300000 : "system" /system
0x00000c3000000x0000183c0000 : "userdata" /data
0x0000183c00000x00001dd20000 : "cache" /cache
0x00001dd200000x00001df20000 : "kpanic"
0x00001df200000x00001df60000 : "dinfo"
0x00001df600000x00001dfc0000 : "setupdata"
0x00001dfc00000x00001e040000 : "splash1"
0x0000003000000x000001680000 : "modem"
From Acer Liquid-E
54
4.2. Kernel
● Early startup code is very hardware dependent
● Initializes environment for the running of C code
● Jumps to the architecture-independent
start_kernel() function.
● Initializes high-level kernel subsystems
● Mounts root filesystem
● Starts the init process
55
4.3. Android Init
● Open, parses, and runs /init.rc:
● Create mountpoints and mount filesystems
● Set up filesystem permissions
● Set OOM adjustments properties
● Start daemons:
– adbd
– servicemanager (binder context manager)
– vold
– netd
– rild
– app_process -Xzygote (Zygote)
– mediaserver
– ...
56
4.4. Zygote, etc.
● Init:
● app_process -Xzygote (Zygote)
● frameworks/base/cmds/app_process/app_main.cpp:
● runtime.start(“com.android.internal.os.Zygote”, ...
● frameworks/base/core/jni/AndroidRuntime.cpp:
● startVM()
● Call Zygote's main()
● frameworks/base/core/java/com/android/internal/os/Zy
goteInit.java:
● ...
57
● preloadClasses()
● startSystemServer()
● ... magic ...
● Call SystemServer's run()
● frameworks/base/services/java/com/android/server
/SystemServer.java:
● Start all system services/managers
● Start ActivityManager:
– Send Intent.CATEGORY_HOME
– Launcher2 kicks in
58
5. Linux Kernel
59
5.1. Androidisms
● Wakelocks
● lowmem handler
● Binder
● ashmem – Anonymous Shared Memory
● RAM console
● Logger
● ...
60
6. Hardware support
Bluetooth BlueZ through D-BUS IPC (to avoid GPL contamination it seems)
GPS Manufacturer-provided libgps.so
Wifi wpa_supplicant
Display Std framebuffer driver (/dev/fb0)
Keymaps and Keyboards Std input event (/dev/event0)
Lights Manufacturer-provided liblights.so
Backlight
Keyboard
Buttons
Battery
Notifications
Attention
Audio Manufacturer-provided libaudio.so (could use ALSA underneath ... at least as illustrated in their porting guide)
Camera Manufacturer-provided libcamera.so (could use V4L2 kernel driver underneath ... as illustrated in porting guide)
Power Management “Wakelocks” kernel patch
Sensors Manufacturer-provided libsensors.so
Accelerometer
Magnetic Field
Orientation
Gyroscope
Light
Pressure
Temperature
Proximity
Radio Layer Interface Manufacturer-provided libril-<companyname>-<RIL version>.so
61
7. Native User-Space
● Mainly
● /data => User data
● /system => System components
● Also found:
● /dev
● /proc
● /sys
● /sbin
● /mnt
● /cache
● Etc.
62
● Libs:
Bionic, SQLite, SSL, OpenGL|ES,
Non-Posix: limited Pthreads support, no SysV IPC
● Toolbox
● Daemons:
servicemanager, vold, rild, netd, adbd, ...
63
8. Dalvik
● Sun-Java =
Java language + JVM + JDK libs
● Android Java =
Java language + Dalvik + Apache Harmony
● Target:
● Slow CPU
● Relatively low RAM
● OS without swap space
● Battery powered
● Has had JIT since 2010
64
8.1. ART
● “Android RunTime”
● 64 bit
● Multi-core
● AOT instead of JIT
● Better debugging capabilities
(profiling/stack/crashes)
● Introduced as alternative to Dalvik
● Has now replaced Dalvik
65
9. JNI – Java Native Interface
● Call gate for other languages, such as C, C++
● Equivalent to .NET's pinvoke
● Usage: include and call native code from App
● Tools = NDK ... samples included
● Check out “JNI Programmer's Guide and
Specification” - freely available PDF
66
10. System Services
Entropy Service Device Policy Audio Service
Power Manager Status Bar Headset Observer
Activity Manager Clipboard Service Dock Observer
Telephone Registry Input Method Service UI Mode Manager Service
Package Manager NetStat Service Backup Service
Account Manager NetworkManagement Service AppWidget Service
Content Manager Connectivity Service Recognition Service
System Content Providers Throttle Service Status Bar Icons
Battery Service Accessibility Manager DiskStats Service
Lights Service Mount Service ADB Settings Observer
Vibrator Service Notification Manager
Alarm Manager Device Storage Monitor
Init Watchdog Location Manager
Sensor Service Search Service
Window Manager DropBox Service
Bluetooth Service Wallpaper Service
67
68
11. Calling on System Services
● Use getSystemService
● Ex: NotificationManager Object reference:
String ns = Context.NOTIFICATION_SERVICE;
NotificationManager mNotificationManager = (NotificationManager) \
getSystemService(ns);
69
12. ActivityManager
● Start new Activities, Services
● Fetch Content Providers
● Intent broadcasting
● OOM adj. maintenance
● Application Not Responding
● Permissions
● Task management
● Lifecycle management
70
● Ex. starting new app from Launcher:
● onClick(Launcher)
● startActivity(Activity.java)
● <Binder>
● ActivityManagerService
● startViaZygote(Process.java)
● <Socket>
● Zygote
71
13. Binder
● CORBA/COM-like IPC
● Data sent through “parcels” in “transactions”
● Kernel-supported mechanism
● /dev/binder
● Check /proc/binder/*
● android.* API connected to System Server
through binder.
72
73
14. Hardware Abstraction Layer
/frameworks/base/core/...
/frameworks/base/services/java/...
AOSP-provided
ASL
/frameworks/base/services/jni/
/hardware/libhardware/
/device/[MANUF.]/[DEVICE]
Manuf.-provided /sdk/emulator/
Manuf. license
Kernel or module
Manuf.-provided
GPL-license
74
15. Stock AOSP Apps
75
Working with the
Android Open Source Project
● Tools and location
● Content
● Building
● Build tricks
● Build system architecture
● Output images
● Using adb
● Custom toolchains and dev kits
● Compatibility Test Suite
● Basic hacks
76
1. Tools and location
● Location:
● http://android.googlesource.com/
● Get “repo”:
$ curl https://dl-ssl.google.com/dl/googlesource/git-repo/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
● Fetch the AOSP:
● Make sure you fetch a tagged release
● Gingerbread:
$ repo init -u https://android.googlesource.com/platform/manifest
-b android-2.3.7_r1
$ repo sync
77
2. Content
bionic C library replacement
bootable Reference bootloader
build Build system
cts Compatibility Test Suite
dalvik Dalvik VM
development Development tools
device Device-specific files and components
external Copy of external projects used by AOSP
frameworks System services, android.*, Android-related cmds, etc.
hardware Hardware support libs
libcore Apache Harmony
ndk The NDK
packages Stock Android apps, providers, etc.
prebuilt Prebuilt binaries
sdk The SDK
system pieces of the world that are the core of the embedded linux platform at
the heart of Android.
78
79
3. Building
● Requires 64-bit Ubuntu 10.04
● Packages required:
$ sudo⌴ apt-get⌴ install⌴ build-essential⌴ libc6-dev⌴ \
> ia32-libs⌴ lib32z1⌴ bison⌴ flex⌴ gperf⌴ git-core⌴ \
> g++⌴ libc6-dev-i386⌴ libz-dev⌴ libx11-dev⌴ \
> libstdc++6⌴ lib32ncurses5⌴ lib32ncurses5-dev⌴ \
> g++-multilib
● Possibly fix a few symbolic links:
$ sudo ln⌴ -s⌴ /usr/lib32/libstdc++.so.6⌴ /usr/lib32/libstdc++.so
81
4. Build Tricks
● Commands (from build/envsetup.sh):
● godir
● croot
● mm
● m
● Speeding up the Build:
● CPU
● RAM
● SSD
● CCACHE
– $ export USE_CCACHE=1
82
5. Build System Architecture
● Non-Recursive
● “Modules” build predicated on Android.mk
83
6. Output Images
● All output and build in [aosp]/out/
● Images at [aosp]/out/target/product/generic/:
● ramdisk.img
● system.img
● userdata-qemu.img
● Kernel is in:
● prebuilt/android-arm/kernel/kernel-qemu
● Emulator overrides:
● -kernel
● -initrd
84
7. Using adb
● Can use to control/interface w/ running AOSP,
including emulator.
● Shell: Host
$ adb shell
Target
#
● Dumping the log:
$ adb logcat
● Copying files to/from target:
$ adb push foo /data/local
$ adb pull /proc/config.gz
85
8. Custom Toolchains and Dev Kits
● Rationale
● SDK generation
● NDK generation
86
8.1. Rationale
● SDK:
● Providing other internal teams or external developers
access to your modified/custom Android APIs.
● NDK:
● Same as SDK rationale
● Custom cross-dev toolchain:
●
To avoid having to use a binary toolchain from 3rd party.
● To control the build parameters used to create the
toolchain. Ex.: use uClibc instead of glibc.
87
8.2. SDK generation
● Building the SDK:
$ . build/envsetup.sh
$ lunch sdk-eng
$ make sdk
● If API modified, do this before make:
$ make update-api
● Location: [aosp]/out/host/linux-x86/sdk/
● Using a custom SDK:
● Eclipse->Window->Preferences->Android->”SDK Location”
● Eclipse->Window->"Android SDK and AVD Manager"-
>"Installed Packages"->"Update All..."
88
8.3. NDK generation
● Build
$ cd ndk/build/tools
$ export ANDROID_NDK_ROOT=[aosp]/ndk
$ ./make-release --help
$ ./make-release
IMPORTANT WARNING !!
This process is EXTREMELY LONG and may take SEVERAL HOURS on a dual-core
machine. If you plan to do that often, please read docs/DEVELOPMENT.TXT
that provides instructions on how to do that more easily.
89
9. Compatibility Test Suite
● Android Compatibility Program:
● Source code to Android stack
● Compatibility Definition Document (CDD) – Policy
● Compatibility Test Suite (CTS) – Mechanism
● Each Android version has own CDD & CTS
● CTS:
● Part of AOSP
● Run from host using USB over to attached device
● Based on JUnit
● Runs various test apps on target
● Relies on ADB
● Provides report to be be analyzed and/or sent back to Google
90
91
● Report:
● .zip file containing XML files and screen-shots
● Sent to: [email protected]
● Building the CTS:
$ . build/envsetup.sh
$ make cts
● Launching the CTS:
$ cd out/host/linux-x86/bin/
$ ./cts
$ cts_host >
$ cts_host > help
...
92
● Using the CTS:
$ cts_host > ls --plan
List of plans (8 in total):
Signature
RefApp
VM
Performance
AppSecurity
Android
Java
CTS
$ ./cts start --plan CTS
93
● Areas covered:
● Signature tests
● Platform API tests
● Dalvik VM tests
● Platform Data Model
● Platform Intents
● Platform Permissions
● Platform Resources
94
10. Basic Hacks
● Add device
● Add app
● Add app overlay
● Add native tool or daemon
● Add native library
95
10.1. Adding a new device
● Create directory in device/:
● device/acme/coyotepad/
● Makefile checklist:
● AndroidProducts.mk
● full_coyotepad.mk
● BoardConfig.mk
● Android.mk
● Menu integration:
● vendorsetup.sh
96
10.1.1. AndroidProducts.mk
PRODUCT_MAKEFILES := \
$(LOCAL_DIR)/full_coyotepad.mk
97
10.1.2. full_coyotepad.mk
$(call inheritproduct, $
(SRC_TARGET_DIR)/product/languages_full.mk)
# If you're using 4.2/Jelly Bean, use full_base.mk instead of
full.mk
$(call inheritproduct, $(SRC_TARGET_DIR)/product/full.mk)
DEVICE_PACKAGE_OVERLAYS :=
PRODUCT_PACKAGES +=
PRODUCT_COPY_FILES +=
PRODUCT_NAME := full_coyotepad
PRODUCT_DEVICE := coyotepad
PRODUCT_MODEL := Full Android on CoyotePad, meepmeep
98
10.1.3. BoardConfig.mk
TARGET_NO_KERNEL := true
TARGET_NO_BOOTLOADER := true
TARGET_CPU_ABI := armeabi
BOARD_USES_GENERIC_AUDIO := true
USE_CAMERA_STUB := true
99
10.1.4. Android.mk
LOCAL_PATH := $(call mydir)
include $(CLEAR_VARS)
ifneq ($(filter coyotepad,$(TARGET_DEVICE)),)
include $(call allmakefilesunder,$(LOCAL_PATH))
endif
100
10.1.5. vendorsetup.sh
add_lunch_combo full_coyotepadeng
101
10.1.4. Android.mk
LOCAL_PATH := $(call mydir)
include $(CLEAR_VARS)
ifneq ($(filter coyotepad,$(TARGET_DEVICE)),)
include $(call allmakefilesunder,$(LOCAL_PATH))
endif
102
10.2. Adding new applications
● Can use Eclipse to create initial version
● Copy Eclipse project to packages/apps
● Local:
● Add app to device/acme/coyotepad/
● Add Android.mk to app
● Add to PRODUCT_PACKAGES in full_coyotepad.mk
● Global:
● Add application in [aosp]/packages/apps
● Add an appropriate Android.mk file to project
● Add project to PRODUCT_PACKAGES in
[aosp]/build/target/product/core.mk
103
LOCAL_PATH:= $(call mydir)
include $(CLEAR_VARS)
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := $(call alljavafilesunder, src)
LOCAL_PACKAGE_NAME := HelloWorld
include $(BUILD_PACKAGE)
104
10.3. Adding an app overlay
● device/acme/coyotepad/overlay/
● full_coyotepad.mk:
● DEVICE_PACKAGE_OVERLAYS :=
device/acme/coyotepad/overlay
105
10.4. Adding a native tool or daemon
● Local:
device/acme/coyotepad/
● Global:
● system/core/
● system/
● frameworks/base/cmds/
● frameworks/native/cmds/
● external/
106
LOCAL_PATH:= $(call mydir)
include $(CLEAR_VARS)
LOCAL_MODULE := helloworld
LOCAL_MODULE_TAGS := optional
LOCAL_SRC_FILES := helloworld.cpp
LOCAL_SHARED_LIBRARIES := liblog
include $(BUILD_EXECUTABLE)
107
10.5. Add a native library
● Local:
device/acme/coyotepad/
● Global:
● system/core/
● frameworks/base/libs/
● frameworks/native/libs/
● external/
● In <= 2.3:
● See build/core/prelink-linux-arm.map
108
LOCAL_PATH:= $(call mydir)
include $(CLEAR_VARS)
LOCAL_MODULE := libmylib
LOCAL_MODULE_TAGS := optional
LOCAL_PRELINK_MODULE := false
LOCAL_SRC_FILES := $(call allcfilesunder,.)
include $(BUILD_SHARED_LIBRARY)
109
Kernel Selection
● Google:
● http://android.googlesource.com
● Vanilla:
● http://www.kernel.org
● Either way ... you're screwed:
● Android kernel is a fork
● No resolution in sight
● Cannot use vanilla kernel as-is ... androidisms
● Recent kernel have most Androidisms
● Android team continues to add stuff in their fork
● Talk to your SoC vendor
110
Native Android User-Space
● Filesystem layout
● Architecture vs. filesystem
● Build system and filesystem
● adb
● Command line
● Init
● ueventd
111
1. Filesystem layout
112
1.1. Root directory
115
2. Architecture vs. filesystem
116
3. Build system and filesystem
● Build templates
● BUILD_EXECUTABLE = /system/bin
● BUILD_SHARED_LIBRARY = /system/lib
● BUILD_PACKAGE = /system/app
● Default rights and ownership
● system/core/include/private/android_filesystem_config.h
● User IDs / names
● android_dirs struct
● android_files struct
117
4. adb
118
4.1. What adb can do
● See online help for full details
● Managing device connections
● Remote commands:
● shell
● log dumping
● bug reports
● port forwarding
● dalvik debugging
119
● Filesystem commands
● push
● pull
● sync
● install / uninstall
● State-altering commands
● reboot
● run as root
● switching connection type
● controlling the emulator
● Tunneling PPP
120
5. Command line
● Shell:
● NetBSD shell up to 2.3/GB
● MirBSD Korn shell since 4.0/ICS
● Toolbox:
● Traditional Linux commands
● Global properties
● Input events
● Services control
● Logging
● ioctl
● Device wiping
● etc.
121
● In aosp:
● /system/core/toolbox
● In filesystem:
●
/system/bin/toolbox
● Provides
alarm date getevent insmod ls mv powerd renice schedtop
smd top dd getprop ioctl lsmod nandread printenv rm
sendevent start umount cat hd ionice lsof netstat ps rmdir
setconsole stop uptime chmod df id kill mkdir newfs_msdos r
rmmod setkey sync vmstat chown dmesg ifconfig ln readtty
rotatefb setprop syren watchprops cmp exists iftop log mount
notify reboot route sleep wipe
122
● logcat
● netcfg
● debuggerd
123
6. Init
124
6.1. Config files
● Location:
● /init.rc
● /init.[board].rc
● Semantics:
● actions
– Property triggers
– Commands
● services (not related to system services or apps)
– Parameters
125
6.2. Global properties
● /dev/socket/property_service
● /dev/__properties__
● foo.bar.property
● ro.*
● persist.* => /data/property
● ctl.*
● net.change
126
● Std files:
● /system/build.prop
● /default.prop
● Other files:
● /system/default.prop
● /data/local.prop
● Code:
– property_get("dalvik.vm.heapsize", heapsizeOptsBuf+4, "16m");
127
7. ueventd
128
7.1. /ueventd.rc
/dev/null 0666 root root
/dev/zero 0666 root root
/dev/full 0666 root root
/dev/ptmx 0666 root root
/dev/tty 0666 root root
129
System Services and Framework
Internals
1. Kickstarting the Framework
2. Utilities and Commands
3. Native Daemons
4. System Services Internals
5. Hardware Abstraction Layer
6. Android Framework
130
1. Kickstarting the Framework
● Core building blocks
● System services
● Boot animation
● Dex optimization
● Apps startup
131
1.1. Core building blocks
● Service manager
● Dalvik
● Android runtime
● Zygote
132
1.2. System services
133
1.3. Boot animation
● Default boot animation
● bootanimation.zip
● setprop debug.sf.nobootanimation 1
134
1.4. Dex optimization
●
Files:
● .jar
● .apk
●
Directories:
● BOOTCLASSPATH
● /system/etc/permission/platform.xml
● /system/framework
● /system/app
● /vendor/app
● /data/app
● /data/app-private
135
1.5. Apps startup
● Input methods
● Persistent apps
● Home screen
● BOOT_COMPLETED intent
● APPWIDGET_UPDATE intent
136
2. Utilities and Commands
● General-purpose commands
● service
● dumpsys
● dumpstate
● rawbu
● Service-specific utilities
● am
● pm
● svc
137
● ime
● input
● monkey
● bmgr
● stagefright
● Dalvik utilities
● dalvikvm
● dvz
● dexdump
138
3. Native daemons
● servicemanager
● installd
● vold
● netd
● rild
● keystore
139
3.1. servicemanager
● In aosp:
● /frameworks/base/cmds/servicemanager/
● In filesystem:
● /system/bin/
● Provides:
● Context management for binder
● Service index for entire system
140
3.2. installd
● In aosp:
● /frameworks/base/cmds/installd
● In filesystem:
● /system/bin/
● Provides:
● Package install/uninstall
● Sanity checks and verifications
● Interfaces with Package Manager service
141
3.3. vold
● In aosp:
● /system/vold/
● In filesystem:
● /system/bin/
● Provides:
● Volume mounter
● Auto-mount
● Auto-format mounted devices
142
3.4. netd
● In aosp:
● /system/netd/
● In filesystem:
● /system/bin/
● Provides:
● Management of aspects of networking
● Interfaces with Network Management service
143
3.5. rild
● In aosp:
● /hardware/ril/mock-ril/
● In filesystem:
● /system/bin/
● Provides:
● “Radio Interface Layer” to phone hardware
144
3.6. keystore
● In aosp:
● frameworks/base/cmds/keystore/
● In filesystem:
● /system/bin
● Provides:
● Store key-value pairs
● Security keys for connecting to network infrastructure
145
4. System Services Internals
● Services run by System Server
● Observing the System Server
● Calling on system services
● Inside a few system services
● Creating your own system service
146
4.1. Services run by the System
Server
Entropy Service Device Policy Audio Service
Power Manager Status Bar Headset Observer
Activity Manager Clipboard Service Dock Observer
Telephone Registry Input Method Service UI Mode Manager Service
Package Manager NetStat Service Backup Service
Account Manager NetworkManagement Service AppWidget Service
Content Manager Connectivity Service Recognition Service
System Content Providers Throttle Service Status Bar Icons
Battery Service Accessibility Manager DiskStats Service
Lights Service Mount Service ADB Settings Observer
Vibrator Service Notification Manager
Alarm Manager Device Storage Monitor
Init Watchdog Location Manager
Sensor Service Search Service
Window Manager DropBox Service
Bluetooth Service Wallpaper Service
147
4.1.1. Some stats
● frameworks/base/services/java/com/android/ser
ver:
● 8.1 M
● ~300 files
● 190 kloc
● Activity manager:
● 1.6M
● 45+ files
● 35+ kloc
148
4.2. Observing the System Server
● Logcat
● dumpsys
149
4.2.1. logcat
●
Find the System Server's PID
$ adb shell ps | grep system_server
system 63 32 120160 35408 ffffffff afd0c738 S system_server
● Look for its output:
$ adb logcat | grep “63)”
...
D/PowerManagerService( 63): bootCompleted
I/TelephonyRegistry( 63): notifyServiceState: 0 home Android Android 310260 UMTS CSS not supp...
I/TelephonyRegistry( 63): notifyDataConnection: state=0 isDataConnectivityPossible=false reason=null
interfaceName=null networkType=3
I/SearchManagerService( 63): Building list of searchable activities
I/WifiService( 63): WifiService trying to setNumAllowed to 11 with persist set to true
I/ActivityManager( 63): Config changed: { scale=1.0 imsi=310/260 loc=en_US touch=3 keys=2/1/2 nav=3/1 ...
I/TelephonyRegistry( 63): notifyMessageWaitingChanged: false
I/TelephonyRegistry( 63): notifyCallForwardingChanged: false
I/TelephonyRegistry( 63): notifyDataConnection: state=1 isDataConnectivityPossible=true reason=simL...
I/TelephonyRegistry( 63): notifyDataConnection: state=2 isDataConnectivityPossible=true reason=simL...
D/Tethering( 63): MasterInitialState.processMessage what=3
I/ActivityManager( 63): Start proc android.process.media for broadcast
com.android.providers.downloads/.DownloadReceiver: pid=223 uid=10002 gids={1015, 2001, 3003}
I/RecoverySystem( 63): No recovery log file
W/WindowManager( 63): App freeze timeout expired.
...
150
4.2.2. dumpsys
Currently running services:
SurfaceFlinger
accessibility
account
activity
alarm
appwidget
audio
backup
...
wifi
window
-------------------------------------------------------------------------------
DUMP OF SERVICE SurfaceFlinger:
+ Layer 0x396b90
z= 21000, pos=( 0, 0), size=( 480, 800), needsBlending=1, needsDithering=1, invalidat ...
0]
name=com.android.launcher/com.android.launcher2.Launcher
client=0x391e48, identity=6
[ head= 1, available= 2, queued= 0 ] reallocMask=00000000, inUse=-1, identity=6, status=0
format= 1, [480x800:480] [480x800:480], freezeLock=0x0, dq-q-time=53756 us
...
151
4.3. Calling on System Services
● Use getSystemService
● Ex: NotificationManager Object reference:
String ns = Context.NOTIFICATION_SERVICE;
NotificationManager mNotificationManager = (NotificationManager) \
getSystemService(ns);
152
4.4. Inside a few System Services
● Get the AOSP ... repo, etc.
● Tricks:
● Import into Eclipse and collapse methods
● Use reverse-engineering tools:
– Imagix
– Rationale
– Lattix
– Scitools
– ...
● Be patient, this isn't documented anywhere ...
153
4.4.1. ActivityManager
● Start new Activities, Services
● Fetch Content Providers
● Intent broadcasting
● OOM adj. maintenance
● Application Not Responding
● Permissions
● Task management
● Lifecycle management
154
● Ex. starting new app from Launcher:
● onClick(Launcher)
● startActivity(Activity.java)
● <Binder>
● ActivityManagerService
● startViaZygote(Process.java)
● <Socket>
● Zygote
155
4.4.2. Package Manager
● 19 kloc
● 830 K
● Installation / removal
● Permissions
● Intent resolution (also IntentResolver.java)
● Called by Activity Manager
156
4.4.3. Window Manager
● Main thread
● Window manipulation
● Wallpaper handling
● Orientation
● Focus
● Layering
● Input event management
157
4.4.4. Notification Manager
● Toasts
● Notifications
● Sound playback (see NotificationPlayer.java)
158
4.4.5. Power Manager
● Wakelocks
● Sleep
● Brightness
● Lock
159
4.4.6. Network Management Service
● Talks to “netd” /system/netd
● Interface configuration
● Tethering
● DNS
160
4.4.7. Mount Service
● Mount / Unmount
● Format
● USB mass storage
● OBB
161
4.4.8. Location Manager
● Manage location providers
● getBestProvider()
● Proximity alerts
● Last known location
162
4.4.9. Status Bar Manager
● Expand / collapse
● Icon visibility
● Reveal callbacks
● Callbacks for notification manager
163
4.4.10. Backup Manager
● Enable / disable
● Transport management
● backupNow()
● ...
164
4.5. Creating your own System
Service
● Add your code to:
frameworks/base/services/java/com/android/server/
● Have the SystemServer.java init+reg. your service
● Define hardware API for apps
● Expose through:
● frameworks/base/core/java/android/os/[server].aidl
● Call on native “driver” code through JNI
● Implement or connect to appropriate driver
● Create an app that calls on service
● May need to create new SDK ...
165
166
4.5.1. OpersysService.java
package com.android.server;
import android.content.Context;
import android.os.Handler;
import android.os.IOpersysService;
import android.os.Looper;
import android.os.Message;
import android.os.Process;
import android.util.Log;
public class OpersysService extends IOpersysService.Stub {
private static final String TAG = "OpersysService";
private OpersysWorkerThread mWorker;
private OpersysWorkerHandler mHandler;
private Context mContext;
public OpersysService(Context context) {
super();
mContext = context;
mWorker = new OpersysWorkerThread("OpersysServiceWorker");
mWorker.start();
Log.i(TAG, "Spawned worker thread");
}
public void setValue(int val) {
Log.i(TAG, "setValue " + val);
Message msg = Message.obtain();
msg.what = OpersysWorkerHandler.MESSAGE_SET;
msg.arg1 = val;
mHandler.sendMessage(msg); 167
}
private class OpersysWorkerThread extends Thread{
public OpersysWorkerThread(String name) {
super(name);
}
public void run() {
Looper.prepare();
mHandler = new OpersysWorkerHandler();
Looper.loop();
}
}
private class OpersysWorkerHandler extends Handler {
private static final int MESSAGE_SET = 0;
@Override
public void handleMessage(Message msg) {
try {
if (msg.what == MESSAGE_SET) {
Log.i(TAG, "set message received: " + msg.arg1);
}
}
catch (Exception e) {
// Log, don't crash!
Log.e(TAG, "Exception in OpersysWorkerHandler.handleMessage:", e);
}
}
}
} 168
4.5.2. IOpersysService.aidl
package android.os;
/**
* {@hide}
*/
interface IOpersysService {
void setValue(int val);
}
169
4.5.3. frameworks/base/Android.mk
...
core/java/android/os/IPowerManager.aidl \
core/java/android/os/IOpersysService.aidl \
core/java/android/os/IRemoteCallback.aidl \
...
170
4.5.4. SystemServer.java
...
try {
Slog.i(TAG, "Opersys Service");
ServiceManager.addService(“opersys”, new OpersysService(context));
} catch (Throwable e) {
Slog.e(TAG, "Failure starting OpersysService Service", e);
}
...
171
4.5.5. HelloServer.java
package com.opersys.helloserver;
import android.app.Activity;
import android.os.Bundle;
import android.os.ServiceManager;
import android.os.IOpersysService;
import android.util.Log;
public class HelloServer extends Activity {
private static final String DTAG = "HelloServer";
/** Called when the activity is first created. */
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
IOpersysService om =
IOpersysService.Stub.asInterface(ServiceManager.getService("opersys"));
try {
Log.d(DTAG, "Going to call service");
om.setValue(20);
Log.d(DTAG, "Service called succesfully");
}
catch (Exception e) {
Log.d(DTAG, "FAILED to call service");
e.printStackTrace();
}
} 172
}
5. Hardware Abstraction Layer
/frameworks/base/core/...
/frameworks/base/services/java/...
AOSP-provided
ASL
/frameworks/base/services/jni/
/hardware/libhardware/
/device/[MANUF.]/[DEVICE]
Manuf.-provided /sdk/emulator/
Manuf. license
Kernel or module
Manuf.-provided
GPL-license
173
● [aosp]/hardware/libhardware/include/hardware
● gps.h
● lights.h
● sensors.h
● [aosp]/hardware/ril/include/telephony/
● ril.h
● Examples in [aosp]/device/samsung/crespo/
● libaudio
● libcamera
● liblight
● libsensors
● Using JNI to call C functions
174
6. Android Framework
● Location and components
● android.*
● Customization
175
6.1. Location and components
● [aosp]/frameworks/base
●
/cmds => native cmds and daemons
● /core => android.* and com.android.*
● /data => Fonts and sounds
● /graphics => 2D & Renderscript
●
/include => “C” includes
●
/keystore => security key store
● /libs => “C” libraries
● /location => Location provider
● /media => Stagefright, codecs, etc.
● /native => Native code for some frameworks components
● /obex => Bluetooth obex
●
/opengl => GL library and java code
●
/packages => A few core packages (Status Bar)
● /services => System server
● /telephony => Phone related functionality
● /tools => A few core tools (aapt, aidl, ...)
●
/voip => RTP & SIP interfaces
●
/vpn => VPN functionality
● /wifi => Wifi manager, monitor, etc.
176
6.2. android.*
accessibilityservice content hardware pim speech
accounts database inputmethodservice preference test
annotation ddm net provider text
app debug nfc security util
appwidget emoji os server view
bluetooth gesture service webkit widget
177
6.3. Customization
● Extending API
● Boot screen
● Status bar
● Network
● Preloaded apps
● Browser bookmarks
● Email provider customization
● Themes
178
6.3.1. Extending API – System service
● frameworks/base/core/java/android/
● app/ContextImpl.java
● content/Context.java
● os/OpersysManager.java
179
6.3.1.1. app/ContextImpl.java
...
import android.os.IOpersysService;
import android.os.OpersysManager;
...
registerService(OPERSYS_SERVICE, new ServiceFetcher() {
public Object createService(ContextImpl ctx) {
IBinder b =
ServiceManager.getService(OPERSYS_SERVICE);
IOpersysService service =
IOpersysService.Stub.asInterface(b);
return new OpersysManager(service);
}});
...
180
6.3.1.2. content/Context.java
...
/**
* Use with {@link #getSystemService} to retrieve a
* {@link android.nfc.NfcManager} for using NFC.
*
* @see #getSystemService
*/
public static final String NFC_SERVICE = "nfc";
/** The Opersys service **/
public static final String OPERSYS_SERVICE = "opersys";
...
181
6.3.1.3. os/OpersysManager.java
package android.os;
import android.os.IOpersysService
public class OpersysManager
{
public void setValue(int value)
{
try {
mService.setValue(value);
} catch (RemoteException e) {
}
}
public OpersysManager(IOpersysService service)
{
mService = service;
}
IOpersysService mService;
}
182
6.3.2. Status bar
● Location:
● frameworks/base/packages/SystemUI/src/com/android/systemui/statusbar
● Disable phone signal - SignalClusterView.java:
if (mMobileVisible && !mIsAirplaneMode) {
/* DISABLE
mMobileGroup.setVisibility(View.VISIBLE);
mMobile.setImageResource(mMobileStrengthId);
mMobileActivity.setImageResource(mMobileActivityId);
mMobileType.setImageResource(mMobileTypeId);
mMobileGroup.setContentDescription(mMobileTypeDescription + ...
*/
183
● Disable battery meter -
policy/BatteryController.java:
/* DISABLE
for (int i=0; i<N; i++) {
ImageView v = mIconViews.get(i);
v.setImageResource(icon);
v.setImageLevel(level);
v.setContentDescription(mContext.getString(R.string.acces...
}
*/
184
6.3.3. Network
● Locations:
● Global static:
– frameworks/base/core/res/res/xml/apns.xml
● Device static:
– PRODUCT_COPY_FILES := vendor/acme/etc/apns-conf-us.xml:system/etc/apns-conf.xml
● Dynamic:
– system/etc/apns-conf.xml
● Format:
<apn carrier="T-Mobile US"
mcc="310"
mnc="260"
apn=" wap.voicestream.com"
user="none"
server="*"
password="none"
proxy=" 216.155.165.50"
port="8080"
mmsc="http://216.155.174.84/servlets/mms"
/>
185
6.3.4. Browser bookmarks
● See packages/apps/Browser/res/values/strings.xml
<!-- Bookmarks -->
<string-array name="bookmarks">
<item>Google</item>
<item>http://www.google.com/</item>
<item>Yahoo!</item>
<item>http://www.yahoo.com/</item>
<item>MSN</item>
<item>http://www.msn.com/</item>
<item>MySpace</item>
<item>http://www.myspace.com/</item>
...
186
6.3.5. Email provider customization
● See packages/apps/Email/res/xml/providers.xml
<!-- Gmail variants -->
<provider id="gmail" label="Gmail" domain="gmail.com">
<incoming uri="imap+ssl+://imap.gmail.com" username="$email"/>
<outgoing uri="smtp+ssl+://smtp.gmail.com" username="$email"/>
</provider>
<provider id="googlemail" label="Google Mail" domain="googlemail.com">
<incoming uri="imap+ssl+://imap.googlemail.com" username="$email"/>
<outgoing uri="smtp+ssl+://smtp.googlemail.com" username="$email"/>
</provider>
...
<!-- Common US providers -->
187
6.3.6. Themes
● See
framework/base/core/res/res/values/styles.xml
188
Thank you ...
189
Acknowledgements:
● Some figures and snippets taken from Google's
Android “Dev Guide” at developer.android.com
distributed under the Apache 2.0 license.
190