CROP STATUS SENSING BASED ON MACHINE VISION

FOR PRECISION FARMING

Noburo Noguchi*, John F. Reid**, Kazunobu Ishii***, and Hideo Terao****

*Contact author: Associate Professor, Bio-Production Engineering, Graduate School of Agriculture,

Hokkaido University, Sapporo 060-8589, JAPAN
**Manager, Technical Center, Deere and Companies, Moline, IL 61265, USA
***Assistant Professor, Bio-Production Engineering, Graduate School ofAgriculture, Hokkaido University,
Sapporo 060-8589, JAPAN
*** *Professor, Bio-Production Engineering, Graduate School ofAgriculture, Hokkaido University, Sapporo
060-8589, JAPAN

Abstract: Sensors are an essential part of intelligent agricultural machinery. Machine vision,
in particular, can supply information about current crop status, including maturity and weed
infestations. The information gathered through machine vision and other sensors such as
GPS can be used to create field management schedules for chemical application, cultivation
and harvest. The purpose of the study is to develop an intelligent machine vision system for
an agricultural mobile robot A mUlti-spectral imaging system was developed to remotely
obtain crop status on a field. The developed multi-spectral imaging system (MSIS) consists
of an imaging sensor, an ilumination sensor, a differential GPS, and a portable computer.
The imaging sensor was a custom-developed 3-CCD camera, which contains t~lfee separate
optical paths and CCD image plane. Special optical filters were installed over the sensors
providing three video channels of Green (G), Red (R), and near infrared (NfR) . The field
experiment of MSIS was conducted using cornfield. The? for estimation of the crop height
showed 0.73. And, the r for estimation of the SPAD value showed 0.92 by using both
reflectance and leaf area infonnation. As a result, the developed vision system enables a
robot to recognize the crop status and efficiently conduct field operation through the timely
information. The outputs include crop stress maps with nitrogen deficiency indexes on the
field which is available on building a database for precise field management.

Keywords; vision intelligence, remote sensing, GIS, machine vision, multi-spectral

imaging system

1. INTRODUCTION is not the solution. A new mode of thought. a new

agricultural technology is required for the future.
Agriculture in developed countries after the Intelligent robotic tractors are one potential solution
Industrial Revolution has tended to favor increases (Noguchi, et al., 1997, 1998). Sensors are an
in energy input through the use of larger tractors essential part of intelligent agricultural machinery.
and increased chemical and fertilizer application. Machine VISIOn, in particular, can supply
AltllOUgh this agricultural technology has negative information about current crop status, including
societal and environmental implications, it has maturity and weed infestations. The infonnation
supported food for rapidly increasing human gathered through machine vision and other
population. In western countries, "sustainable sensors such as GPS can be used to create field
agriculture", was developed to reduce the management schedules for chemical application,
environmental impact of production agriculture. cultivation and harvest. The purpose of Ule study
A.i the same time. the global agricultural workforce is to develop an intelligent machine vision system
continues to shrink; each worker is responsible for for an agricultural mobiles robot as shown in
greater areas of land. Simply continuing the Fig. 1 (Noguchi, et aI., 1999). The vision system
current trend toward larger and heavier equipment developed is able to simutaneously detect crop

34
rows and gather field infonnation. In particular, U.S.A.). The imaging sensor was a custom-
nitrogen stress of the crop is important factor for developed 3-CCD camera, which contains three
crop growth and yield. Abmad, et al. (1999) and separate optical paths and CCD image plane.
Iida, et af (2000) con finned that greelUless index Special optical filters were installed over the
could express nitrogen stress status of crops. In sensors providing three video channels of Green
fact, a SPAD meter which was developed and (G), Red (R), and near infrared (NIR) shown in
commercialized by Minolta Co Ltd. detects a kind Fig. 2. These three channels have center
of Greenness Index (SPAD value) based on a ratio wavelengths of 55Oom, 65Onm, and 800nm and
of transmittance of two LEDs (65Onm and bandwidth of approximately 1000m for each
94Oom). A SPAD meter has been widely used as channel. In addition, the aperture of lens and
a sensor for crop nitrogen status. (Schepers, et al. CCD gain in each channel can be control1ed
1992 and Kim, et al. 2000) But, since the SPAD through RS232C to ensure a wide dynamic range
meter can measure the nitrogen stress with of the sensor. Images were captured by an image
pinching a leave by a detector that is contact-base sensor with a resolution of 640H X 480V at 8
method, it isn't useful for the sensor attached on bit/pixe!. A flame grabber (FlashBus MV, Integral
the robot. The sensor for detecting the crop status Technologies, U.S.A.) was used to acquire images
must adopt non-contact base method. Besides, in through a high speed PCI interface. On the other
respect of crop status parameters, the height is hands, an ambient illumination sensor was
also important as the crop status index as well. employed for compensating light power change.
Therefore, the intelligent vision system based on The ambient illumination sensor had three filters
multi-spectral imaging system (MS IS) was of G, R, and NIR, which are almost same
developed for detecting nitrogen stress and crop bandwidth with those of MS IS. The MS IS system
height. The vision system enables a robot to was attached on the front of the mobile robot.
recognize the crop status and efficiently conduct Because the mobile robot has an RTK-GPS and a
field operation through the timely infonnation. Fiber optic gyroscope for navigation, the
The outputs include crop stress maps with infonnation from the MSIS can be applied for the
nitrogen deficiency indexes on the field. GIS mapping as well as for real-time fertilizer or
chemical application. In test platform, the another
machine vision (Kodak Megaplus 1.6i) was also
2. MULTI-SPECRAL IMAGING SYSTEM (MSIS) installed on the system to investigate the
hardware perfonnance of the MSIS. Two machine
2.1 MSIS hardware visions on the robot were set at- same field-of-
The developed MSIS consists of an imaging view.
sensor, an illumination sensor (SKRI850A, 4-
channel, Skye Instruments Ltd., U.K.), an RTK- 2.2 Image Processing
GPS (MS750, Trimble, U.S.A.), and a portable One of the important issues of the image
computer (Field Pac, Do1ch Computer Systems, processing is to segment out various unnecessary

Vision Intelligence for Precision RISP _____ eco Sensor

l. .
Agriculture
V"'rl~.I • ..(Iri.
~1lC1t.-·
Il1IpWttm-1
VlaIon I....
• ReoognIdoiI. ..01 !:lOP iIlId_
.. . 'F rilZllr IIppilc::8ikiO
• GtowIII 0I1:n!P& .~ .~
. "...~ • YWd MI&Miion
INFRARI!D

GREEN

Fig. 1 Concept of VlSlon intelligence for Fig. 2 Construction of MS IS.

agricultural mobile robot.

35
 regions in the image. For instance, soil and weed 3. FIELD EXPERIMENT
regions have to be segmented from the image
before decision-making of the crop status. And, 3.1 Hardware performance based on outdoor
even after segmenting the image, regions of environment
shadow and directly reflected from the crop To confirm validity of the MSIS concept, the
leaves have to be considered on image processing. precise experiment of MSIS was conducted in a
In the paper, the NIR image was utilized for field. Most important is whether the MSIS can
segmenting the vegetation and soil because these precisely measure the chlorophyll like a SPAD
regions can be easily segmented using an NIR meter. In this research, corn was chosen as a test
image. Nitrogen stress of crop is usually detected crop and MSIS measured leaf samples with
by reflectance of green band from leaves because various SPAD values in outdoor environment.
the chlorophyll is well correlated with nitrogen Chlorophyll obtained by chemical analyses was
contained in leaves. On the other hand, the employed as reference data for the MS IS. Fig. 5
chlorophyll absorbs red band light. As described shows the experiment procedure for evaluating
above, a SPAD meter uses red LED transmittance MSIS. A field-of-interest (FOl) completely
to detect the nitrogen stress of crop. Besides, the corresponded with a leaf area for chemical
crop height can also tell the crop status, which is analysis. (approximately 10 cm x 3 cm). In the
available for precise field management. Therefore, experiment, MSIS and MegaPlus simultaneously
the goal of the crop parameters detected by the obtained an image of same field-of-view (FOV),
MSIS in the paper was to remotely estimate crop and the images were acquired under two different
height, chlorophyll content and SPAD value. light conditions. Fig. 6 shows the accuracy of the
Because the MSIS can control the CCD gains as MSIS estimating chlorophyll content in a leave.
well as the aperture, the MS IS response relating The value of (G-responseINIR-response) was
to light reflectance can be calculated as follows, adopted to estimate the chlorophyll content. High
re!. correlation between MSIS response and
i - responce =__ 1-
chlorophyll-a content, which was R2 "of 0.8246,
refNlR was obtained in the experiment. From this result,
XI it was confirmed that the MSIS could remotely
re! I = ----'- detect the chlorophyll content. Moreover, high
EIA/I
correlation between the SPAD value and
Hence, X is gray level. E is aperture, AI is output
chlorophyll-a content was already proved by
from the illumination sensor. i is G, R, NIR
many other researches. It was concluded that the
channel. As you can see, the AI compensates
MSIS could predict SPAD value as well as
outdoor illumination change. MSIS response can
chlorophyll-a content.
be individually acquired for three channels; Red,
Green and NIR. Basically, an estimator of the
3.2 GIS mapping by field experiment
chlorophyll content and SPAD value using MSIS
In above session, it was clear that the MSIS
response was developed. The leaf area index was
concept was valid to detect the" crop nitrogen
defined as ratio of leaf area to whole image area
status. The field experiment for the MSIS was
as shown in Fig. 4. The estimator for crop height
conducted at a cornfield in Hokkaido University,
was developed based on a defined leaf area index.
The prior knowledge that was a leaf area had high
correlation with leaf area in growing period, was
utilized to develop the estimator.

nlroi

NIR inage histogral'll

leaf area index (L,u)
Leaf area
)(100
Computer
LAI= Whole
11lumination arM. c.pturo
Sensor &N O boordl

Fig. 3 Experiment platform. Fig. 4 Segmentation of vegetation and definition of

leaf area index.

36
 100 geographic posItIOn was continuously updated
,.....,
E 95 _ R2 =0.7275 from the GPS and associated with each image.
Corn stresses detected by MSIS were clearly
..3 90 corresponded with SPAD map. Fig. 11 shows the
~
+'
..t:
boO • • estimation accuracy of the yield calculated from
'Qi
..t:
C.
85

80
:-- MSIS. The results were investigated about
relationship between LAI and actual yield. Jf was
....0 0.94. And, R.M.S. error was 0.47 kg. It was
0 75
concluded that MSIS can provide the information
70 of yield before the harvest.
150 170 190 210 230 250 270

Leaf area index [%] 4. CONCLUSIONS

Visi on Intelligence for an agricultural mobile

Fig. 9 Accuracy of estimating crop height by robot was presented. Since agriCUltural systems
MSIS on the field . are dramatically affected by changing information
from time-valiant environment. The robot must
have a sensory device for collecting useful
information from the field. Real-time MSIS
response information can provide the crop health
and growth to the robot and farmers. In next step,
the decision-making by the robot itself regarding
fertilizer treatment is required to real-time
nitrogen application. The tendency of future
technology will not be limited to robot area. The
MSIS can contribute to Precision Farming and
Actual measured SPAD

,
.....
~
. . · ... 1;
",U . • • I
identification of crop growth model as complex
system .
. ·,....-.. 1.
, tt.a:1'! , • •

4 1. · ....ct!

-....
....a .... l"f
4.1"' I .L6Ii!:
LOoIiI·""
....
," · 1.41 ACKNOWLEDGEMENT
i' , a •
. ..- This research was supported by Japan Ministry of
t. ._.....'"
....
. . . . 4. 1.

...... ... Agriculture, Forestry and Fisheries, and the

equipment was provided by Case Corporation,
Estimated SPAD by MSIS
V.S .. All of the mentioned supports and
assistance are gratefully acknowledged.

Fig. 10 GIS map of estimated crop SPAD in

the field.
REFERENCE

Ahmad, I.S., Reid, J.F., Noguchi, N. and Hansen,

15 R2:O.94 A.C. : Nitrogen sensing for precision
R.M.S.=O.47(kg) agriculture using chlorophyll maps, 1999,
ASAE Paper 993035.
Iida, T. , Noguchi , N., Ishii, K., Terao, H.:
Nitrogen Stress Sensing. System using
Machine Vision for Precision Farming (part
5 L-__ ~ ____L -_ _ ~ _ _ _ _L -_ _ ~ _ __ _
I), 2000, Journal of the Japanese Society of
10 12 14 16 18 20 Agricultural Machinery, 62(2), 87-93.
LA! [%]
Kim, Y., Reid, J.F. , Zhang, Q., Hansen, A.C. :
Infotoronic Decision-making for a Field Crop
Sensing System in Precision Agriculture,
Fig.ll Corn yield estimation by LAI calculated 2000, Proceedings of IFAC BIO-Robotics Il,
from MSIS. 289-294.

38
Noguchi, N., Ishii, K., and Terao, H.; Noguchi, N., J.F. Reid, Q. Zhang, L. Tian and
Development of an Agricultural Mobile A.C. Hansen: Vision Intelligence for Mobile
Robot using a GeomagneticDirection Sensor Agro-Robotic System, 1999, Journal of
and Image Sensors, 1997, Journal of Robotics and Mechatronics, Vol.ll No.3,
Agricultural Engineering Research, 67(1), l- 193-199.
IS.
Sheppers, J.S., Francis, 0.0., Vigil, M., Below,
Noguchi, N., Reid, J.F.,Zhang, Q., and Tian, L.F.; F.E.: Comparison of corn leaf nitrogen
Vision Intelligence for Precision Fanning concentration and chlorophyll meter reading,
using Fuzzy Logic Optimized Genetic 1992, Communications in Soil Science and
Algorithm and Artificial Neural Plant Analysis, 23(17-20), 2174-2187.
Network, 1998, ASAE Paper No.983034.

39