Gluing the IoT world with Java and

LoRaWAN
The SkopjePulse.mk case

Panche Chavkovski
2018-02-06 / Jfokus
 Who am I?

Panche Chavkovski
Senior Software Engineer @ netcetera
jug.mk Leader
Codefu.mk administrator
Hardware & IoT enthusiast
TTN SK Initiator

http://pance.mk/ and @hsilomedus

2
3
 SkopjePulse

Problems Solution
§ extreme air pollution § Leverage technology
§ Crowd-sourced sensor network
§ Excessive urban noise
§ Data analysis and availability
§ Flash floods § Warnings
§ Clearer insights
No clear means for improvement § Data-driven basis for action

https://skopjepulse.mk/faq

4
 System overview

LoRaWAN Analytics

Mobile
Devices Data Storage
apps

Front-end &
Data Fetching
! API

Warnings
5
LoRaWAN

6
 LoRa & LoRaWAN

Long Range Long Range Wide Area Network

§ Based on FSK & CSS, multiple channels § MAC layer on top of LoRa
§ Low Power § IoT communication standard
§ Low Cost
§ Long range § Standardized stack
§ Free ISM bands: 433, 868, 915 MHz § Driven by LoRa Alliance
§ Proprietary by Semtech

7
 Features

Fully bi-directional Easy commisioning Network scalability

End-to-end security Built-in positioning

8
 Limitations
Low speed (default BW)
§ SF12: 250bps with the farthest distance and the longest air-time
§ SF7: 5470bps with the shortest distance and the shortest air-time

Small package sizes: ~1-20bytes

Less frequent
§ 2-200 uplinks per day
§ 1-10 downlinks per day

9
 LoRaWAN architecture
End nodes Gateway / concentrator Network server(s) Application server(s)

LoRa RF TCP/IP SSL TCP/IP SSL

LoRaWAN LoRaWAN Secure payload

AES secured payload

10
 LoRaWAN devices
Different classes
§ Class A, rare async devices (current): one uplink with two downlink windows
§ Class B, always on, timed: beacon downlink windows
§ Class C, always on, always listening.

Network sign on:

§ Over The Air Authorization (OTAA): App EUI and Key
§ Authorization By Personalization (ABP): App / Network keys and Device address

11
 TheThingsNetwork

Global, crowdsourced Internet Of Things data network

§ Community driven with local initiators and teams
§ Provided network and back-end
§ Free for fair use
§ Can be elevated to SLA (private – public partnership)
§ Multiple integration options
§ LoRaWAN as base technology

https://www.thethingsnetwork.org/

12
 How to use TTN

Registration Integrations
https://console.thethingsnetwork.org/ § MQTT
§ Applications § HTTP
§ Devices § Data storage
§ Keys, UIDs, Credentials § Amazon AWS
§ Cayenne
§ EVRYTHNG

13
Devices

14
 The start up kit

Microchip RN2483 (LoRaWAN stack) RFM95W (LoRa transceiver)

Arduino (MCU) El. Design & PCB CAD Software

15
 KiCAD usage
Scheme design 3D render and gerbers

PCB design

16
 Assembly
Solder and connect

PCB fabrication 3D printed enclosure

17
 Make it work

Code Beware
§ Arduino or other embedded code § Select and inspect used sensors
§ Restricted environment § Test and / or calibrate
– 32KB flash § Check for factory defects
– 2KB RAM
§ Plan for hardware wear-out and lifespan
– 16MHz CPU
§ Don’t assume anything
§ Perpetual non-observed execution

18
System design

19
 Rules to go by

Decouple Aim for and produce

§ Data transceiver § Robustness
§ Data Store § Scaling
§ Web and API § Fallbacks
§ Analytics & processing § Data sanity

20
 Our approach

SpringBoot framework
§ Self contained web server / application
§ Modular
§ Starter packs for needed spring pieces
§ Less configuration, more conventions
§ Light(er)weight
§ Container and orchestrator friendly (Cloud native)

21
Data fetching

22
 Communication first!
(almost) No overhead
§ Use binary protocols if possible
§ Always on connection
– reconnect fast
– plan for fallback

React first, store later!

Get closer to the data source.

23
 Our approach

TheThingsNetwork MQTT Integration

Spring Boot + Eclipse Paho client + Gson

§ @Component client implementing MqttCallback
§ Connect in @PostConstruct
§ @Scheduled(fixedrate = …) watchdog @Component

Details on MQTT URL, credentials and message formats:

https://www.thethingsnetwork.org/docs/applications/mqtt/api.html

24
Data storage

25
 IoT data
IoT data is almost always impartial and time-series based

Use data store that is designed for sorted time-series

§ Utilize recent data caches or TTLs
§ Don’t hold on redundancies
§ Always just append data
§ Never aggregate, but process and store
§ Design to live with eventual consistency

26
 Our approach
Cassandra DB 3.7

Spring Boot + Cassandra Driver Core + Extras

§ Cassandra Cluster wrapped in @Service
§ InstantCodec for java.time interoperability
§ QueryBuilder
§ No data filtering.

* spring-data-cassandra seemed like a poor choice.

27
Analytics

28
 Averaging data

Situation Our approach

§ A lot of (impartial) data § Side-running SpringBoot microservice
§ Observed system load § Runs a few times a day
§ Possible downtime § Checks first
§ Retroactivity § Calculates last two days, week and month
§ Uses intermediary data
§ Uses simple heuristic

29
Warnings

!
30
 Early warning system

Keep a track on what’s happening

§ Periodically fetch the recent data
§ Analyze, produce and save current state
§ Notify if the state has changed significantly
§ Don’t spam
§ Periodically cleanup

Our approach: SpringBoot + SpringSocial + Firebase

31
Front-end & API

32
 Provide and visualize the data
Interactive cockpit and data
explorers
§ D3js client side visualizations
§ Leaflet.js + OpenStreetMap
(+ Stamen designs)

Provide open to use API

§ SpringBoot + SpringMVC +
SpringSecurity
§ Set rules to avoid easy DOS
§ Log usage
§ Circuit breakers
https://skopjepulse.mk/new
33
 Control the system

SPA for device and user

management
§ Angular2
§ ng2-admin

System health dashboard

34
Mobile apps

35
Native Android and iOS apps

36
Extras

37
 WiFi devices
Where you *really* can’t do LoRaWAN
§ ESP8266 powered device
§ HTTPS communication
§ provision device address securely
§ implement own address -> key mapping

More: http://pance.mk/index.php/securing-esp8266-communication/

38
 AWS This is my architecture

http://tiny.cc/awsttn
39
Discussion

40
Thank you

41