This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2020 1

A Millimetre-wave Radar Based Fall Detection

Method using Line Kernel Convolutional Neural
Network
Bo Wang, Student Member, IEEE, Liang Guo, Hao Zhang, Student Member, IEEE and Yong-Xin Guo,
Fellow, IEEE

Abstract— Fall accidents are significant threats to the health and

life of older people. When a millimetre-wave (mmWave) frequency
modulated continuous wave (FMCW) radar is used for fall detection,
the selected features for further classification can determine the
detection performance. In this paper, a line kernel convolutional
neural network (LKCNN) is proposed to process the baseband data
directly to detect fall motions. This method utilizes the character-
istic of a convolutional neural network (CNN) that it can learn to
extract useful features during the training process. A data sample
generation method is also proposed to generate multiple samples
for the training process by utilizing the multiple receiving channels
and sufficiently small pulse repetition time (PRT). The experiment
results show that the proposed method can detect fall motions with high accuracy, sensitivity and specificity with fewer
network parameters and less computation cost, which is meaningful in realizing an all-time indoor fall detection system.
Index Terms— Convolutional neural network, line convolution kernel, millimetre-wave radar, data sample generation, fall
detection

I. I NTRODUCTION time (DT) maps [1]–[4], range-time (RT) maps [1] [2], Mel-
frequency cepstrum coefficients (MFCC) [5], radar data cube

F ALL accidents are one of the leading causes of the non-

natural deaths of older people who live alone. Thus, it is
crucial to design and implement a 24/7 in-door fall detection
[6], etc. The selection of extracted features and corresponding
radar parameters can determine the detection performance.
However, manual engineering of features cannot guarantee an
and timely alarming system for critical environments. The
optimal feature selection, which may be the limitation to the
radar sensor is an ideal choice considering its outstanding
methods.
performance in sensing human motions and capability in
Assuming that the baseband data captured by the radar
privacy protection. The frequency modulated continuous wave
carry enough information, there should exist an effective fall
(FMCW) radar, which can monitor both the range and the
detection method based on feature extraction from the base-
Doppler frequency shift at the same time, is demonstrated to
band data directly. The power of convolutional neural networks
perform satisfactorily in detecting fall actions [1] [2].
(CNN) is extracting useful features automatically when being
In previous literature [1]–[6], after the radar baseband data trained. Motivated by this point, a fall detection method using
is obtained from the radar sensors, signal processing methods line kernel CNN (LKCNN) to process baseband data directly
are usually used to manually generate certain radar maps as is proposed in this paper. In this method, feature extraction and
extracted features for further classification, such as Doppler- classification operations are merged into one single network
model. In other words, being trained on the baseband data
Manuscript received May 27, 2020; revised June 18, 2020; and
accepted July 1, 2020. This work was supported in part by NRF through directly, the LKCNN can approximate the unknown feature
the AI Singapore Programme under grant Award No. AISG-100E-2019- extraction method which is effective for further classification
042. (Corresponding author, Y.X. Guo). operations to detect fall motions.
B. Wang and Y.X. Guo are with the Department of Electrical and
Computer Engineering, National University of Singapore, Singapore To train a robust CNN, a large number of representative data
117576, and also with the National University of Singapore Suzhou samples is critical. A data sample generation method that can
Research Institute, Suzhou, Jiangsu Province, China, 215123 (e-mail: generate a large number of data samples from one record of
[email protected]; [email protected]).
L. Guo is with Harbin Institute of Technology, Harbin, Heilongjiang baseband data for the training process is proposed in this paper.
Province, China, 150001 (e-mail: [email protected]). H. Zhang This method is feasible when two prerequisites are fulfilled.
is with Northwestern Polytechnical University, Xi’an, Shaanxi Province, The first one is a small pulse repetition time (PRT) and a
China, 710072 ([email protected]). L. Guo and H. Zhang were
with the Department of Electrical and Computer Engineering, National wide bandwidth (BW) of the deposited radar sensor, whose
University of Singapore, Singapore 117576. characteristics are analyzed in Section IV. The second one is

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
2 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2020

Radar
RX1 LNA IF ADC

2m
RX2 LNA IF ADC
Desk

45° 75cm 90cm

RX3 LNA IF ADC

Floor
RX4 LNA IF ADC Mattress

I/Q Data Outputs

Position 3

Position 1
Position 2
TX PA Signal Generator
Placement
p
in experiment

Fig. 1. The simplified block diagram of the deployed FMCW radar

Fig. 2. Experimental environment setup

that the adopted radar has multiple receiving channels, as is

TABLE I
shown in Fig. 1.
PARAMETERS FOR EXPERIMENTS
The remainder of this paper is organized as follows. Sec-
Parameters Quantity
tion II introduces the adopted FMCW radar and the experimen-
Starting frequency 77 GHz
tal setup. In Section III, the signal model and signal processing Stop frequency 78.9882 GHz
algorithms used in this paper, including the proposed sample Bandwidth 1.9882 GHz
generation method, are presented. In Section IV, the selection Frequency slope 53.735 MHz/µs
Chirp duration 37 µs
of the critical parameters, PRT and BW, is analyzed. In Idle time between chirps 48 µs
Section V, the proposed fall detection method using baseband Pulse repetition time 85 µs
data and LKCNN is introduced, and the experiment results Number of receiving antennas 4
are presented. Section VI gives some discussion, and finally, Sampling frequency 2.4 MHz
Number of frames 400
Section VII concludes this paper. Number of chirps per frame 200

II. E XPERIMENTALAL S ET- UP AND M EASUREMENT TABLE II

D ESIGNED M OTIONS
Fig. 1 shows a simplified block diagram of the millimetre-
wave (mmWave) FMCW radar system used in the experiment, Motions Labels
Fall forward towards radar Falling down
which is a part of the IWR1642BOOST FMCW radar module Fall backward off radar Falling down
[7] operating at the frequency band from 76 GHz to 81 Fall forward towards radar without arm motions Falling down
GHz developed by Texas Instruments for data collection. The Fall forward off radar without arm motions Falling down
evaluation board includes two transmitters and four receivers. Bend Not falling down
Squat (and stand) Not falling down
In our experiment, only one transmitter and four receivers are Sit down Not falling down
utilized. The antenna peak gain is larger than 9 dBi across the
whole operating frequency band. The starting frequency is set
at 77 GHz. The BW is set at 1.9882 GHz, and thus the range are designed, as are listed in Table II. Eleven volunteers aged
resolution is around 7.5 cm. The PRT is 85 µs, denoted as Tc . 22-28 years old took part in the experiment. The heights of
The detailed adopted radar parameters for the experiments are the volunteers range from 1.65 m to 1.8 m, and the weights
specified in Table I. range from 55 kg to 75 kg. For each motion performed by one
The radar is deposited on a 75 cm-high desk, and thus the volunteer, a record of data was captured by the radar from a
antennas are around 90 cm high from the floor. There are some certain angle. The duration of each record is around 7 seconds.
desks, chairs and computers placed around the experimental 231 records of baseband data in total were captured.
area. The subject falls into a mattress placed about 2 m away
from the centre of the mattress to the radar. The size of the
mattress is around 1.37 m × 1.91 m × 0.22 m. Three cases III. S IGNAL P ROCESSING
are considered where the mattress is placed parallel to the line A. Baseband Data
of sight (LOS) of the radar, with an angular difference of 45 For an FMCW radar [8], the transmitted signal can be
degrees and 90 degrees since in real life, people may fall in expressed as
different angles related to the LOS of the radar. Seven motions,
including daily activities and different falling-down motions, xT (t) = AT cos(2πfc t + πβt2 ), (1)

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
AUTHOR et al.: PREPARATION OF PAPERS FOR IEEE SENSORS JOURNAL 3

where AT is the transmitted power, fc is the starting frequency is cancelled, and the data indicating information of the moving
of the chirp and β is the chirp slope. β can be computed by targets are left for further operations. Assuming that the output
B sequence of the single delay line canceller of a discrete-time
β= , (2) sequence x[m] is z[m], Fig. 4 shows the flow chart of the
Tchirp
single delay line canceller and the transfer function can be
where B is the chirp bandwidth and Tchirp is the chirp written as
duration.
When a point object is detected, for each receiving channel, H(z) = 1 − z −1 . (6)
after a short time delay td , the reflected wave will be received
by the receiving antennas. The time delay is dependent to the
radial range d of the object given by
td = 2d/c, (3) 𝒙[𝒎] + ෍ 𝒛𝒎
-
where c is the speed of microwave travelling in the air.
The received signal is a scaled and shifted version of the 𝒛−𝟏
transmitted signal given by
xR (t) = αAT cos[2πfc (t − td ) + πβ(t − td )2 ], (4) Fig. 4. Flow chart of the MTI method

where α is an amplitude scaling coefficient that accounts for

factors such as atmospheric attenuation and target scattering.
The received signal is down-converted to baseband by a local C. Data Sample Generation Method
copy of the transmitted signal, and the frequency shift between Data used for machine learning algorithms, such as images
the transmitted signal and the received signal is known as the and sound files, are the representations or descriptions of the
beat frequency fbeat given by target objects. In some cases, data describing the same target
can be regarded as different samples for the machine learning
fbeat = β · td . (5)
algorithm since they are different representations from the
After a series of processing, the baseband I/Q (in- algorithm’s point of view.
phase/quadrature-phase) signals are generated and then dig- A radar with multiple receiving channels can be used for
itized as I/Q data by using A/D converters. I/Q data in one angular measurement, while in the proposed method, the angu-
chirp form a column, and by sequencing all chirps in time, a lar information is ignored to reduce computational cost. Data
baseband data matrix can be generated, as is shown in Fig. 3. from different channels are not identical. Different channels
The y-axis is known as the fast time axis, and the x-axis is can be regarded as different observers and generate individual
the slow time axis. radar signal representations of the motions.

chirp Frame 1 Frame 2

fast
time
fast ··· ··· ···
time ···
··· ···
···
···
···

···
···
···

··· ··· ···

I data
··· ··· ···
Q data
··· ··· ···
slow time slow time
Sample 1 Sample 2 ··· Sample Nchirp
Fig. 3. Sketch map of a baseband data matrix
Fig. 5. Demonstration of the data sample generation method

In a baseband data matrix with a PRT of Tc , the total number

B. Single Delay Line Canceller of chirps is Ntotal . If the nth chirp of every N (n ≤ N ) chirps
The task is to detect fall motions, and thus only moving is selected to form a new baseband data matrix, this data matrix
targets are of our interests. In this paper, a primary moving can be regarded as data collected by radar with a PRT of
target indication (MTI) method, single delay line canceller, N × Tc , as is shown in Fig. 5. To simplify the expression,
is used for background subtraction. When the single delay every Nchirp chirps are grouped as a frame. The number of
line canceller is used to process baseband data, for each chirp frames can be computed as Nf rame = bNtotal /Nchirp c. The
except the first one, by subtracting the corresponding data number of chirps in one generated sample is the same as the
value of the previous chirp, the static background information number of frames. When different n values are picked, the

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
4 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2020

generated matrixes are not the same and can be regarded as The characteristic velocities of human motions are within
different samples. In a baseband data matrix for one channel, a certain range, and vf all,max can be decided by experience.
the maximum number of matrixes that can be generated is A proper PRT value makes the velocity estimation cover the
Nsamples . Assuming the radar has Nchannel channels, the total range. A smaller PRT value is usually preferred since high
number of generated baseband data samples using one record velocity is an important characteristic of human fall-down
of baseband data can be calculated as motion compared with other motions. However, a smaller
PRT can also lead to a larger amount of data and higher
Nsamples = Nchannel × Nchirp . (7) computation cost, and thus a properly small PRT needs to be
selected considering the trade-off between velocity estimation
The generated samples can be used as augmented samples coverage and computation cost.
when a neural network is trained. The sample generation
method is similar to data augmentation [9] used in image
processing. However, the samples generated by the data aug- B. FMCW Radar Range Estimation
mentation method are often used only for the training process. The accuracy of distance measurement is essentially depen-
While for radar data, samples are generated from different dent on the SNR of the system. A radar system with high
observation time and angles, and they are ‘real’ samples, and SNR can usually give an accurate range estimation. A high
thus they can also be used as individual validation samples. range resolution results in good separation of different parts
A precondition for this data sample generation method is of the moving target and helps to demonstrate the distance
that the data with a PRT of multiple times of Tc have carried distribution of different motions. The range measurement error
enough information for fall detection. For an FMCW radar δR can be used to demonstrate the accuracy of distance
with a wide bandwidth, the data still carry enough information estimation, and it can be computed as
for fall detection when PRT is large. The detailed analysis is
dres c
in the following section. δR ∼
=√ = √ , (12)
2SN R 2B 2SN R
c
IV. P ULSE R EPETITION T IME AND B ANDWIDTH S ETTING where dres = 2B is the range resolution [10]. According to
A NALYSIS (12), a wide BW results in high distance resolution and small
range measurement error. Hence, an FMCW radar with high
Two radar parameters, PRT and BW, are critical in the
SNR and a properly wide BW is potential to gain pleasing
proposed method. PRT and BW influence the velocity and
performance in fall detection.
range resolutions, respectively. Proper selection of the two pa-
rameters guarantees the detection performance of the proposed
method and feasibility of the proposed data sample generation C. Selection of PRT and BW
method.
When the BW is wide enough, the range measurement
error is small. The range information by the baseband data is
A. FMCW Radar Velocity Estimation accurate enough to detect different motions. Thus, when the
An FMCW radar senses the velocity of a moving object PRT is set at a large value, the amount of information carried
with the help of the phase shift between the corresponding by the captured data can guarantee accurate fall detection.
points of the same object in two chirps. The velocity of an Then when the BW is wide, a large PRT can be used to reduce
object can be estimated as the computation cost. In our experiment, the BW of around 2
GHz is wide enough, while the selected PRT is still a small
λ∆φ value. The small PRT ensures when the proposed data sample
vestimation = , (8)
4πTc generation method is used, the generated samples still carry
where λ is the wavelength of the radar wave, and ∆φ is enough information for fall detection.
the phase difference. The unambiguous phase shift can be
achieved when V. P ROPOSED CNN BASED F EATURE E XTRACTION AND
|∆φ| < π. (9) C LASSIFICATION M ETHOD
The maximum velocity the radar can estimate can be A. Line Kernel CNN
calculated as Compared to classical fully-connected neural networks, with
λπ λ the help of convolution layers, CNNs take advantage of
vestimation < = . (10) the hierarchical pattern in data and assemble more complex
4πTc 4Tc
patterns using smaller and simpler patterns, making the scale
For fall detection, the system is supposed to be able to sense
of connectedness and complexity lower. Appropriate zero-
and describe all velocity components and their distributions. In
padding is used to keep the shape after convolution layers.
order to sense the maximum velocity vf all,max when a person
Max-pooling layers can find out the most prominent features.
falls onto the ground, the maximum PRT should be set at
A stack of convolution layers and max-pooling layers are
λ used for automatic feature extraction. The unknown feature
Tc,max < . (11)
4vf all,max extractor, which is effective for fall detection, is approximated

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
AUTHOR et al.: PREPARATION OF PAPERS FOR IEEE SENSORS JOURNAL 5

TABLE III
D ETAILED S TRUCTURE OF CNN S
LKCNN-1 LKCNN-2 LKCNN-3 LKCNN-4 LKCNN-5 SKCNN-1 SKCNN-2
Layer type Filter shape Filter shape Filter shape Filter shape Filter shape Filter shape Filter shape
0 Input - - - - - - -
1 Convolution 256×(1,9) 256×(9,1) 256×(1,9) 256×(1,9) 256×(1,9) 32×(3,3) 32×(3,3)
2 Max Pooling (1,4) (4,1) (1,4) (1,4) (1,4) (2,2) (2,2)
3 Convolution 128×(1,7) 128×(7,1) 128×(1,7) 128×(1,7) 128×(1,7) 64×(3,3) 64×(3,3)
4 Max Pooling (1,4) (4,1) (1,4) (1,4) (1,4) (2,2) (2,2)
5 Convolution 64×(1,5) 64×(5,1) 64×(5,1) - 64×(1,5) 128×(3,3) -
6 Max Pooling (1,25) (8,1) (4,1) - (1,5) (2,2) -
7 Convolution - - - - 32×(1,3) - -
8 Max Pooling - - - - (1,5) - -
9 Flatten - - - - - - -
10 Fully-connected (8192,2) (25600,2) (51200,2) (409600,2) (4096,2) (102400,2) (204800,2)
11 Softmax - - - - - - -

0 0 0
100
20 20
20 75
40 40
50
Sample points

40 60 60
25
60 0 80 80
80 -25 100 100
-50 120 120
100
-75 0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
120
-100
(a)
0 50 100 150 200 (a) 250 300 350 Sample Points 0 0 0 0 0 0
0 100 20 20 20 20 20 20
20 75 40 40 40 40 40 40
Sample points

50 60 60 60 60 60 60
40
25 80 80 80 80 80 80
60 0
100 100 100 100 100 100
80 -25
-50
120 120 120 120 120 120
100 0 20 40 60 80 0 20 40 60 80 0 20 0 20 (c) 0 20 0 20
-75 (b)
120 -100
0 50 100 150 200 250 300 350 Sample Points 0 0
(b) 20
20
40 40
60 60
Fig. 6. Examples of two generated samples: (a) falling forward towards 80 80
100 100
the radar motion (b) squatting motion. 120 120
0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
(d)
0 0 0 0 0 0
by these layers during the training process. The output of each 20 20 20 20 20 20
40 40 40 40 40 40
convolution layer can be written as 60 60 60 60 60 60
80 80 80 80 80 80
Z ci = ReLU(Z ci−1 ∗ K ci + bci ), (13) 100
120
100
120
100
120
100
120
100
120
100
120
0 20 40 60 80 0 20 40 60 80 0 20 0 20 0 20 0 20
where ci denotes the ith convolution layer, Z ci is the feature (e) (f)
The numbers in the figures are sample points
map computed by layer ci , K ci is the convolution kernel of
layer ci and bci is the corresponding bias. ReLU function is a Fig. 7. Examples of output of different convolution layers of LKCNN-1
of falling forward towards the radar motion ((a)-(c)) and squatting motion
non-linear activation function. ((d)-(f)): (a)(d) Outputs of the first convolution layer (b)(e) Outputs of the
For image recognition, convolution kernels are usually second convolution layer (c)(f) Outputs of the third convolution layer.
square with a size of N ×N , which is helpful to learn the local
features [11] [12]. According to the radar characteristics, the
data points in the same row or column of the baseband data a probability distribution consisting of two probabilities. By
matrix have the closest correlations. Different from traditional comparing the computed probabilities, the category of fall or
convolution neural network using square kernels, the line non-fall of the inferenced motion can be decided.
convolution kernels with the size of N × 1 or 1 × N are
used for convolution layers, which is similar to the manner
of classical signal processing methods like short-time Fourier B. Experiment Results
transform (STFT) and discrete Fourier transform (DFT). Phys- According to Table I and Section III. C, 200 chirps form
ical meanings and relationship among the data are considered 1 frame (Nchirp =200) and there are 4 receiving channels
when line convolution kernels are used. In other words, by (Nchannel =4). Thus, the generated samples are with a PRT of
imitating manners of the classical signal processing methods, 200 × Tc , and according to (7), 4 channels×200 chirps = 800
the network can better understand the baseband data and samples are generated using one record of captured baseband
extract useful features. data. 134,400 samples of 8 volunteers are used as the training
A fully-connected layer is used as a classifier to use the set, and 50,400 samples of 3 volunteers are for testing. Fall
extracted feature maps to detect fall from other motions. motions are set as positive samples, and non-fall motions are
Adam optimizer is used under cross-entropy loss function. The labelled as negative ones. After the data samples are generated,
softmax function is used to normalize the output result into the MTI method described in Section III. B is implemented for

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
6 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2020

TABLE IV LKCNN-1, leading to heavier storage space and memory cost.

C LASSIFICATION M ATRICS ON THE VALIDATION S ET The networks are used to predict 200 samples continuously,
Method Accuracy Sensitivity Specificity and the average prediction time is computed. LKCNN-1 can
1 LKCNN-1 98.74% 99.63% 97.55% make the prediction faster than InceptionV3. The average
2 LKCNN-2 94.67% 94.78% 94.52%
3 LKCNN-3 92.54% 93.78% 90.89% processing time that one generated sample is used to produce
4 LKCNN-4 78.84% 79.28% 78.25% a DT map on the deployed laptop is around 50.2 ms, and
5 LKCNN-5 96.27% 97.26% 94.95% thus the proposed LKCNN is potential to overperform other
6 SKCNN-1 82.04% 83.86% 79.61%
methods which utilize Doppler signatures in computing speed.
7 SKCNN-2 72.78% 74.78% 70.11%
8 InceptionV3 97.36% 99.48% 94.54% In our experiment, a GPU is used, while in a real application,
the fall detection methods can be realized on a mini-computer
TABLE V
C OMPARISON OF T HREE N ETWORKS
with limited computation power. The proposed method with
fewer parameters and faster computing speed can be more
Average Prediction
Method Accuracy HDF5 file size
Time suitable in real applications.
1 LKCNN-1 98.74% 3.35 MB 51.4 ms To verify the feasibility of the trained models, another three
2 InceptionV3 97.36% 250 MB 164.8 ms volunteers helped to collect data as the test set. The radar
3 SKCNN-1 82.04% 3.44 MB 43.2 ms parameters remain the same except that the PRT is set as 200Tc
TABLE VI and only one receiving channel is used. This is supposed to be
C LASSIFICATION M ATRICS ON THE T EST S ET the setting for a real application. The radar is set at the same
Method Accuracy Sensitivity Specificity three angles, as is described in Section II, and the volunteers
1 LKCNN-1 95.24% 94.44% 96.30% performed the seven motions. The total number of samples in
2 SKCNN-1 69.84% 72.22% 66.67%
the test set is 63, with 36 positive ones and 27 negative ones.
3 InceptionV3 93.65% 94.44% 92.59%
The classification matrics in Table VI verify the feasibility of
the proposed method.
The above results show that the proposed LKCNN is more
background subtraction. Fig. 6 shows two generated samples suitable in approximating the optimum feature extractor and
of a fall and a non-fall motions after the MTI operation. During make the prediction for fall detection. The proposed LKCNN
the training process, 20% of the training samples are randomly utilizes small storage space and memory, and make the pre-
picked from the training set to run in an augmentation mode, diction at a fast speed. Thus the computation cost in a real
and the number of fall and non-fall training samples is application can be reduced, which is meaningful for realizing
also balanced during the process. A laptop with an Intel(R) a 24/7 monitoring system. The experiment results show that
Core(TM) i7-7700HQ CPU and an NVIDIA GTX1070 GPU is LKCNN can extract useful features for further classification,
used for training and testing. The trained network structure and and it is a better choice for indoor falling-down detection using
parameters are stored in an HDF5 file. Different LKCNNs and the baseband data captured by an FMCW radar.
CNN with square convolution kernels (SKCNN) are trained,
using different convolution kernels and number of convolution VI. D ISCUSSION
layers. The detailed structures of the CNNs are specified in A. Experiment Set-up
Table III. The target of our research is detecting fall accidents indoors
Table IV gives classification metrics obtained using net- for older people who live alone. The experiments were mainly
works described in Table III. The accuracy, sensitivity (true designed to validate the feasibility of the proposed method.
positive rate) and the specificity (true negative rate) are used However, before the method is deployed in a real application,
to show the performance of the models. It can be observed the following factors need to be further considered:
that LKCNN performs better than SKCNN. LKCNN-1 with 1. It is actually difficult to acquire data from older people. In
convolution kernels in the shape of 1 × N shows the best our experiment, young volunteers imitated the older people’s
performance with high accuracy, sensitivity and specificity. motions, and thus the performance of the young people can
Fig. 7 shows some examples of the outputs of different still show the feasibility of the proposed idea. It is reasonable
convolution layers of LKCNN-1 in inferencing the samples to believe that by capturing data and updating the model suc-
shown in Fig. 6. It is potential to further improve the detection cessively from real applications for older people, the detection
accuracy with more convolution layers, while the trade-off performance will be further enhanced.
between the performance improvement and computation cost 2. In our experiment, 4 fall motions and 3 non-fall motions
has to be considered in real application. are performed. Arm motions change the shape of the whole
By transfer learning from a trained model on ImageNet body, resulting in different velocity and distance distributions.
[13], an InceptionV3 [12] network is also trained on the same Hence, the arm motion is the key difference for different fall
dataset described above as a comparison. Convolutional layers motions. The designed non-fall motions are also the most
and full-connected layers are all updated during the training confusing motions to fall for the radar sensor. Thus, the
process. Table V shows the comparison among LKCNN-1, designed motions can show the feasibility of the proposed
SKCNN-1 and InceptionV3. The tuned InceptionV3 network method.
can achieve similar detection performance to LKCNN-1. How- 3. In an indoor environment, the distance between the
ever, the HDF5 file size of InceptionV3 is much larger than subject and the radar sensor is usually within 5 m. For the

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
AUTHOR et al.: PREPARATION OF PAPERS FOR IEEE SENSORS JOURNAL 7

adopted radar, within the range, there is no worry about the and lower cost, which is meaningful for realizing an all-time
penetration loss against the environment noise, and thus, the indoor fall detection system.
radar can sense the whole range. In our experiment, only data
at a distance of 2 m are captured as an instance to simplify R EFERENCES
the experiment. [1] B. Jokanović and M. Amin, “Fall detection using deep learning in
4. The experiment environment in this paper is complicated range-Doppler radars,” IEEE Transactions on Aerospace and Electronic
with many computers, desks, chairs, etc. The deployed MTI Systems, vol. 54, no. 1, pp. 180–189, 2017.
[2] T. Yang, J. Cao, and Y. Guo, “Placement selection of millimeter
method minimizes the influence of stationary objects in an wave FMCW radar for indoor fall detection,” in 2018 IEEE MTT-S
indoor environment. However, the influence of moving objects, International Wireless Symposium (IWS). IEEE, 2018, pp. 1–3.
such as fans, sweeping robots or pets, can hardly be ignored. [3] R. Zhang and S. Cao, “Real-time human motion behavior detection via
CNN using mmWave radar,” IEEE Sensors Letters, vol. 3, no. 2, pp.
The robustness of the proposed method to the moving objects 1–4, 2018.
needs further validation by experiments. [4] F. Jin, R. Zhang, A. Sengupta, S. Cao, S. Hariri, N. K. Agarwal, and
S. K. Agarwal, “Multiple patients behavior detection in real-time using
mmWave radar and deep CNNs,” in 2019 IEEE Radar Conference
B. Idea of the Proposed Method (RadarConf). IEEE, 2019, pp. 1–6.
[5] B. Erol, M. Amin, F. Ahmad, and B. Boashash, “Radar fall detectors: A
To our knowledge, in previous literature about fall detection comparison,” in Radar Sensor Technology XX, vol. 9829. International
[1]–[6], before neural networks are implemented, certain sig- Society for Optics and Photonics, 2016, p. 982918.
nal processing methods are usually utilized to generate certain [6] B. Erol and M. G. Amin, “Radar data cube analysis for fall detection,”
in 2018 IEEE International Conference on Acoustics, Speech and Signal
radar maps for further classification. In these methods, the Processing (ICASSP). IEEE, 2018, pp. 2446–2450.
following classifiers process the radar maps as common images [7] “IWR1642 Single-Chip 76- to 81-GHz mmWave Sensor,” Texas
rather than radar data. As a result, the physical characteristic Instruments, 2018. [Online]. Available: https://www.ti.com/lit/ds/
symlink/iwr1642.pdf?ts=1590495205734
of radar data may be ignored, and the selection of signal [8] A. Ahmad, J. C. Roh, D. Wang, and A. Dubey, “Vital signs monitoring
processing methods may decide the classification results. It of multiple people using a FMCW millimeter-wave sensor,” in 2018
is reasonable to believe that if the physical characteristic of IEEE Radar Conference (RadarConf18). IEEE, 2018, pp. 1450–1455.
[9] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le, “Au-
the radar data is well utilized and proper features are extracted, toaugment: Learning augmentation policies from data,” arXiv preprint
the classification results can be more satisfactory. The power arXiv:1805.09501, 2018.
of CNN is that it can learn to extract useful features. As [10] G. R. Curry, Radar system performance modeling. Artech House
Norwood, 2005, vol. 1.
a result, a CNN that can mimic the manner of classical [11] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner et al., “Gradient-based
signal processing method or merge the processing operations learning applied to document recognition,” Proceedings of the IEEE,
inside the network is supposed to achieve better performance, vol. 86, no. 11, pp. 2278–2324, 1998.
[12] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna,
since the physical characteristic of the data is well utilized “Rethinking the Inception Architecture for Computer Vision,” 2015.
during the process. The proposed LKCNN utilizes the physical [Online]. Available: http://arxiv.org/abs/1512.00567
characteristic by using line convolution kernels. Some other [13] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet:
A large-scale hierarchical image database,” in 2009 IEEE conference on
models which merge the classical signal processing method computer vision and pattern recognition. IEEE, 2009, pp. 248–255.
inside the network structure, such as wavelet networks [14] [14] Q. Zhang and A. Benveniste, “Wavelet networks,” IEEE transactions on
and STFNets [15], are also possible to achieve similar classi- Neural Networks, vol. 3, no. 6, pp. 889–898, 1992.
[15] S. Yao, A. Piao, W. Jiang, Y. Zhao, H. Shao, S. Liu, D. Liu, J. Li,
fication accuracy. However, their complex network structures T. Wang, S. Hu et al., “STFNets: Learning Sensing Signals from the
result in heavy computation cost, and thus the proposed Time-Frequency Perspective with Short-Time Fourier Neural Networks,”
LKCNN can be more feasible in a real application. Concisely, in The World Wide Web Conference, 2019, pp. 2192–2202.
adjusting CNN structure to utilize the characteristics of the
raw data can be a choice rather than just transforming the
data into images.

VII. C ONCLUSION
Any signal processing algorithm can be regarded as a certain
feature extraction method, and feature selection is important
for further classification after the baseband data are obtained. Bo Wang (S’19) received the B.Sc. degree in
telecommunication engineering from Beijing In-
CNN can learn to extract useful features and approximate stitute of Technology and the M.Sc. degree in
the effective feature extractor when being trained. Inspired electrical engineering from National University of
by this point, an indoor fall detection method is proposed Singapore in 2016 and 2018, respectively. He is
currently pursuing the PhD degree from National
using LKCNN to process baseband data sample directly. University of Singapore. His research interests
Experiment results demonstrate that the proposed network can include radar signal processing and AI-assisted
detect falling-down actions with high accuracy, sensitivity, RF sensors for biomedical applications.
and specificity on the radar baseband data after background
subtraction. A data sample generation method utilizing the
characteristic of the deployed radar is also proposed. Direct use
of baseband data without post-processing also results in fewer
operations and calculations, leading to less power consumption

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JSEN.2020.3006918, IEEE Sensors
Journal
8 IEEE SENSORS JOURNAL, VOL. XX, NO. XX, XXXX 2020

Liang Guo received the B.Sc. degree and the

M.Sc. degree in information and communication
engineering from Harbin Institute of Technology,
Harbin, China in 2014 and 2016, respectively.
He is currently pursuing the PhD degree in in-
formation and communication engineering from
Harbin Institute of Technology. His research in-
terests include array signal processing, clutter
suppression and radar signal processing.

Hao Zhang (S’17) was born in Xuzhou, China.

He received the B.E. and Ph.D. degrees in
electronic engineering from Nanjing University
of Science and Technology, Nanjing, China, in
2014 and 2019, respectively. He is currently an
Associate Professor with the School of Micro-
electronics, Northwestern Polytechnical Univer-
sity, Xi’an, China. From 2015 to 2018, he was
a joint Ph.D. candidate with the Department of
Electrical and Computer Engineering, National
University of Singapore. His current research
interests include WPT and RF energy harvesting, MMIC, flexible micro-
electronics and sensors for biomedical and healthcare applications. Dr.
Zhang was a receipt of the MTT-S Travel Grant Award of Asian Pacific
Microwave Conference (APMC2019). He also serves as a Reviewer
for many international journals, such as the IEEE Transactions on
Microwave Theory and Techniques, IEEE Microwave Magazine, IEEE
Microwave and Wireless Components Letters, and IEEE Journal of
Electromagnetics, RF, and Microwaves in Medicine and Biology.

Yong-Xin Guo (M’03-SM’05-F’18)received the

B.Eng. and M. Eng. degrees from Nanjing Uni-
versity of Science and Technology, Nanjing,
China, and the Ph.D. degree from City University
of Hong Kong, all in electronic engineering, in
1992, 1995 and 2001, respectively.
He is currently a Full Professor at the Depart-
ment of Electrical and Computer Engineering,
National University of Singapore (NUS). Con-
currently, He is Director, Center for Peak of Ex-
cellence on Smart Medical Technology at NUS
Suzhou Research Institute. He has authored or co-authored over 460
international journal and conference papers and 4 book chapters. He
holds over 30 granted/filed patents in USA, China and Singapore. His
current research interests include antennas, wireless power, RF and
millimeter-wave sensing, and MMIC modeling and design for wireless
communications, IoTs, and biomedical applications. He has graduated
15 PhD students at NUS.
Dr. Guo is the Chair for IEEE AP-S Technical Committee on Antenna
Measurement in 2018-2020. He has served as General Chair/Co-Chair
for IEEE MTT-S IMWS-AMP 2020, APMC 2019, AWPT 2017, ACES-
China 2017, IEEE MTT-S IMWS-AMP 2015 and IEEE MTT-S IMWS-Bio
2013. He has served as Technical Program Committee (TPC) co-chair
for IEEE MTT-S IMBioC 2020, IMWS-AMP 2019/2017 and RFIT2009.
He is serving as Associate Editor for IEEE Antennas and Propaga-
tion Magazine. Dr Guo is elected as Editor-in-Chief, IEEE Journal of
Electromagnetics, RF and Microwave in Medicine and Biology (2020-
2023). He has served as the IEEE Fellow Evaluation Committee for
IEEE Engineering in Medicine and Biology Society (2019-2020). He is
a recipient of 2020 IEEE Microwave and Wireless Components Letters
Tatsuo Itoh Prize of the IEEE Microwave Theory and Techniques Society
(MTT-S). He is an IEEE Fellow.

1558-1748 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: University of Exeter. Downloaded on July 17,2020 at 06:28:53 UTC from IEEE Xplore. Restrictions apply.