EdgeX Explorations

Tiejun Chen, VMware China R&D, ATC

Huaqiao Zhang, VMware China R&D, ATC

@Tiejun_Chen @Huaqiao_Zhang
Agenda
• EdgeX Foundry introduction
• EdgeX Foundry practices and explorations
– EdgeX Foundry solution
• Preempt-RT Linux over minimal Linux distribution
• Standard | Preempt-RT Linuxkit
• HW-assisted Virtualization solution
• Todo
– EdgeX Foundry Web UI
• Demo
• Todo
• Liota & EdgeX Foundry
EdgeX Foundry Overview
• An open source, vendor neutral project (and ecosystem)
• A microservice, loosely coupled software framework for
IoT edge computing
• Hardware and OS agnostic
• Goal: enable and encourage growth in IoT solutions
– The community builds and maintains common building blocks and
APIs
– Plenty of room for adding value and getting a return on investment
– Allowing best-of-breed solutions
EdgeX Micro Service Architecture
• Contextually, EdgeX micro services are divided into 4 layers
• Device Services (device services for various protocols like Modbus, BACnet, …
and SDK to create new device services)
– Communicate in native sensor/device protocol to the physical – that is to the IoT
“things”
– Transform sensor data to common format
– Translate command requests to actuate devices (in native protocol/format)
• Core Services (core data, metadata, command & configuration/registration)
– Offers temporary persistence of edge data and facilitates actuation of things
through common API
– Collect sensor data
– Understand what sensors/devices are connected how to communicate with them
(metadata)
– Provision facility for new sensors/devices (and device services)
– Manage device actuation requests to device services/devices
– Provide micro service registry
– Provide micro service configuration
• Supporting Services (logging, notifications, scheduler, rules engine)
– Normal software application duties plus “edge intelligence”
– Logging
– Notifications and alerting
– Scheduling and clean up
– Rules engine
• Export Services (client, distribution)
– On or off box client registration of data
– Distribution center of sensor data to clients
– Transport edge data to the enterprise and cloud systems in a manner they request
EdgeX Foundry Practice and Exploration

• IoT devices
– Resource-constrained
– Real time requirement
• Practice and Exploration
– Using minimal Linux distributions
– Integrating preemp-rt Linux kernel patches
EdgeX Foundry Practice and Exploration

• Minimal Linux distributions

– A variety of server distribution
• e.g. ubuntu server based
• Optimization and customization
– Linuxkit
• A toolkit for building secure, portable and lean operating
systems for containers.
• Everything is run in a container
• lightweight, fully customizable
• All core LinuxKit packages are build from linuxkit/alpine
EdgeX Foundry Practice and Exploration

• Preempt-RT Linux kernel

– Code
• The current development -RT tree
» http://git.kernel.org/pub/scm/linux/kernel/git/r
t/linux-rt-devel.git
• The current stable -RT tree
» https://git.kernel.org/pub/scm/linux/kernel/git/
rt/linux-stable-rt.git
• Released stable -RT patchset
» https://www.kernel.org/pub/linux/kernel/proje
cts/rt/4.14/
EdgeX Foundry Practice and Exploration

• Enable preempt-rt Linux kernel over Ubuntu server

– ubuntu-18.04.1-live-server-amd64.iso
–
$ cat /boot/config-4.15.0-32-generic | grep PREEMPT
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
# CONFIG_PREEMPTIRQ_EVENTS is not set

– Enable preempt-rt Linux kernel

$ git clone git://kernel.ubuntu.com/ubuntu/ubuntu-bionic.git
$ apt-get build-dep linux-image-$(uname -r)
$ cd ubuntu-bionic
$ git am {-rt patches}
– Fourth level
$ cp /boot/config-4.15.0-32-generic .config & enable CONFIG_PREEMPT_FULL
$ make & make modules_install & make install
$ reboot
» Fifth level
EdgeX Foundry Practice and Exploration

• Linuxkit was originally standard kernel based.

• How to integrate preempt-rt Linux kernel
patches
– My goal is to construct Preempt-RT Linuxkit
– Code
• https://github.com/linuxkit/linuxkit/tree/master/kernel/patc
hes-4.14.x-rt
– New project to EdgeX Foundry
• https://github.com/linuxkit/linuxkit/tree/master/projects
• Support docker compose
EdgeX Foundry Practice and Exploration

• Linuxkit-EdgeX.yml
kernel:
image: linuxkit/kernel:4.14.62 or 4.14.59-rt
cmdline: "console=tty0"
init:
…
onboot:
…
services:
…
- name: compose
image: tiejunworld/edgex-compose:0.1
binds:
- /var/run:/var/run
- /etc/compose:/compose
rootfsPropagation: shared
capabilities:
- CAP_SYS_ADMIN
files:
- path: var/lib/docker
directory: true
- path: etc/docker/daemon.json
contents: '{"debug": true, "hosts": ["unix:///var/run/docker.sock"]}'
mode: "0644"
EdgeX Foundry Practice and Exploration

• Linuxkit-EdgeX.yml
…
- path: etc/compose/docker-compose-barcelona-0.2.1.yml
source: docker-compose-barcelona-0.2.1.yml
- path: etc/compose/images/edgexfoundry/docker-core-command:0.2.1.tar
source: image-cache/edgexfoundry/docker-core-command:0.2.1.tar
- path: etc/compose/images/edgexfoundry/docker-core-config-seed:0.2.1.tar
source: image-cache/edgexfoundry/docker-core-config-seed:0.2.1.tar
- path: etc/compose/images/edgexfoundry/docker-core-data:0.2.1.tar
source: image-cache/edgexfoundry/docker-core-data:0.2.1.tar
…
- path: etc/compose/images/edgexfoundry/docker-edgex-volume:0.2.tar
source: image-cache/edgexfoundry/docker-edgex-volume:0.2.tar
- path: etc/compose/images/edgexfoundry/docker-export-client:0.2.1.tar
source: image-cache/edgexfoundry/docker-export-client:0.2.1.tar
- path: etc/compose/images/edgexfoundry/docker-export-distro:0.2.1.tar
source: image-cache/edgexfoundry/docker-export-distro:0.2.1.tar
- path: etc/compose/images/edgexfoundry/docker-support-logging:0.2.1.tar
source: image-cache/edgexfoundry/docker-support-logging:0.2.1.tar
- path: etc/compose/images/edgexfoundry/docker-support-notifications:0.2.tar
source: image-cache/edgexfoundry/docker-support-notifications:0.2.tar
- path: etc/compose/images/edgexfoundry/docker-support-rulesengine:0.2.1.tar
source: image-cache/edgexfoundry/docker-support-rulesengine:0.2.1.tar
- path: etc/compose/images/edgexfoundry/docker-support-scheduler:0.2.1.tar
source: image-cache/edgexfoundry/docker-support-scheduler:0.2.1.tar
EdgeX Foundry Practice and Exploration

• Virtualization in IoT edge system

• Use case
– Telecommunication systems
– Avionic systems
– Industrial automation
– Medical technology
– Automotive
EdgeX Foundry Practice and Exploration
Automotive evolution Virtualization in automotive

Autosar
RTOS AGL GENIVI Android
OS

Hypervisor
CPU GPU/FPGA Memory Devices

Bosch

Benefits of virtualization in automotive

• Costs
Hardware (ECU) consolidation
Support legacy technology Hypervisor solution
• Security
• Embedded System hypervisor
Temporal and spatial isolation • Automotive Edge Computing Platform
• Mixed criticality system
Certifications -- ASIL
• Multi-tenancy
• Flexibility and Interoperability
• Software defined architecture
EdgeX Foundry Practice and Exploration

• EdgeX Foundry in virtualized IoT system

– Each EdgeX Foundry per VM/tenant
– Multiple EdgeX Foundry
Guest OS Guest OS Guest OS

EdgeX Foundry EdgeX Foundry EdgeX Foundry

Hypervisor
Hardware platform
EdgeX Foundry Practice and Exploration

• EdgeX Foundry in virtualized IoT system

– Minimalize each guest OS with Linuxkit
– Enable preempt-rt Linux
Preempt-rt minimal GOS Preempt-rt Linxkit GOS Guest OS

EdgeX Foundry EdgeX Foundry EdgeX Foundry

Hypervisor
Hardware platform
EdgeX Foundry Practice and Exploration

• EdgeX Foundry in virtualized IoT system

– One privileged Linuxkit VM deployed just with
edgex-device-service
EdgeX GOS EdgeX GOS EdgeX GOS

edgex-export-services edgex-export-services edgex-export-services

edgex-supporting-services edgex-supporting-services edgex-supporting-services

edgex-core-services edgex-core-services edgex-core-services

Preempt-RT Linuxkit GOS

edgex-device-service

Hypervisor
Hardware platform
Todo 1/4
• EdgeX Foundry into virtualized IoT system
– Protect data
• EdgeX Foundry with AMP
• EdgeXFaas (Ongoing)
• Integrate Edgex Foundry ROS (Ongoing)
– Who takes control of devices
Todo 2/4
• EdgeX Foundry into virtualized IoT system
– Keep data private

EdgeX GOS
edgex-export-services Guest OS Guest OS

edgex-supporting-services docker-edgex-mongo docker-edgex-mongo

edgex-core-services
edgex-device-service

Hypervisor
Hardware platform
Todo 3/4
• EdgeX Foundry with AMP
– Minimal Linuxkit including EdgeX

EdgeX Linux GOS

edgex-export-services

edgex-supporting-services

edgex-core-services IoT OS
edgex-device-service

Hardware platform
Core
Core A
A Core
Core B
B
Todo 4/4
• EdgexFaas
– EdgeX with Function As A Service
EdgeX GOS

Event
Event engine
engine || API
API Gateway
Gateway
edgex-supporting-services Preempt-rt GOS
Unikernel
container container

edgex-core-services Function Function Function

edgex-device-services

Hypervisor
Hardware platform
EdgeX Foundry UI
• Motivation
– What if we have no web console?
• Master all kinds of knowledge like docker, yml, profiles, services, …
• Complexity
– Too much operation steps
– Different data structure
– But what if we have a web UI console specific to EdgeX
Foundry? 
• Using EdgeX Foundry naturally
• Debugging Edgex Foundry easily
EdgeX Foundry UI Landscape
EdgeX Foundry UI framework
EdgeX Foundry UI workflow

Data
A: EdgeX
Foundry
Services

B: EdgeX
One Single User Select One Access Other Dynamic
User Login Foundry
Console Given Gateway Page Proxy
Services

C: EdgeX
Foundry
Store Gateway Services
Metadata Get Gateway Instance IP Selected
By The Current User
EdgeX Foundry UI Demo
EdgeX Foundry UI
https://www.youtube.com/watch?v=2EOHR_gUeic

Enable Modbus/RS485
https://www.youtube.com/watch?v=_2AHiUcMzew&t=399s
EdgeX Foundry UI Todo
• Already merged into
https://github.com/edgexfoundry-holding/edgex-ui-go
• Next
– Start/Stop/Update EdgeX Microservice
– Support log | notification | dashboard
– Integrate map service

Any questions to EdgeX Foundry UI

• Huaqiao Zhang (c)
– [email protected]
– Twitter: @Huaqiao_Zhang
EdgeX Foundry Integration
• Liota & EdgeX Foundry
– Liota – Little IoT Agent
• https://github.com/vmware/liota
• Liota applications are comprised of six basic abstractions that
represent a complete data flow from a device attached to the
edge system to an application in a data-center.
– How
• Option 1
– Set up local mqtt broker
– Listen in all mqtt topics
• Option 2
– Export service
Thank you!

I love everything 
[email protected]
EdgeX Foundry Summary
• A collection of a dozen+ microservices
–Written in multiple languages (Java, Go, C, … we are polyglot
believers!!)
–Several commonly used library projects (common domain objects,
client libraries, etc.)
• Microservices are deployed via Docker and Docker
Compose
• REST communications between the service
–Some services exchange data via message bus (core data to export
services and rules engine)
• EdgeX data flow:
–It can trigger device actuation through Command service
–Sensor data is collected by a Device Service from a thing
–Data is passed to the Core Services for local persistence
–Data is then passed to Export Services for transformation, formatting,
filtering and can then be sent “north” to enterprise/cloud systems
–Data is then available for edge analysis and can trigger device
actuation through Command service