V O L U M E 2 N O . 1 • S P R I N G 2 0 1 2 • A U V S I • 2 7 0 0 S o u t h Q u i n c y S t r e e t , S u i t e 4 0 0 , A r l i n g t o n , VA 2 2 2 0 6 , U S A
Agriculture
Inside this issue:
John Deere’s autonomous tech Detecting microbial invasions with UAVs Japan’s aging farmers turn to robotics Mission Critical
•
Spring 2012
1
Promoting and Supporting Unmanned Systems and Robotics Across the Globe
NOW symposium.auvsi.org
LAS VEGAS
6 – 9 AUGUST auvsishow.org
Conference from 6 - 9 August Tradeshow from 7 - 9 August 500+ Exhibiting Companies 40+ Countries Represented 8,000+ Attendees
REGISTER TODAY! ®
CONTENTS V O L U M E 2 N O . 1 • S P R I N G 2 0 1 2
8
Green giant
Farming mainstay John Deere adds intelligence to many of its vehicles, spanning every area from orchards to the military
4 Essential
16 State of the art
A map of where the seeds have been sown in robotic farming
components
The latest in agriculture robotics
26 Q & A
28 Timeline
A history of technology at John Deere
An U.S. Department of Ag expert talks on the National Robotics Initiative
On the cover: A quadrotor with a multispectral imaging sensor from the University of Florida looks for citrus greening, a disease that ravages local plants. Read more on Page 36. Photo courtesy Reza Ehsani.
Mission Critical
•
Spring 2012
1
30 Market report Potential game changers for the slow but steady growth in high-tech ag
36 Uncanny valley For aerial crop imaging to jump, the FAA has to say how high to fly
38 Testing, testing Carnegie Mellon looks to automate specialty crop farming
18 A blighty wind Virginia Tech researchers are using UAVs to track blankets of microbial invaders that affect everything from plants to animals to even human health
41 Technology gap Needed: better sensors, planning
42 Spotlight Mack Robotics’ BinBot could save lives and litigation
44 End users Keenan Amundsen of the University of Nebraska - Lincoln
31 Far East of Eden When it comes to pairing robotics and agriculture, Japan is the cream of the crop
Mission Critical is published four times a year as an official publication of the Association for Unmanned Vehicle Systems International. Contents of the articles are the sole opinions of the authors and do not necessarily express the policies or opinion of the publisher, editor, AUVSI or any entity of the U.S. government. Materials may not be reproduced without written permission. All advertising will be subject to publisher’s approval and advertisers will agree to indemnify and relieve publisher of loss or claims resulting from advertising contents. Annual subscription and back issue/reprint requests may be addressed to AUVSI.
2
Mission Critical
•
Spring 2012
Editor’s message Editorial Vice President of Communications and Publications, Editor Brett Davis davis@auvsi.org Managing Editor Danielle Lucey lucey@auvsi.org
W
elcome to the second year of Mission Critical magazine, our now not-so-new quarterly publication that
embraces a different aspect of robotics in fields that have spanned driverless cars to
Associate Editor Stephanie Levy slevy@auvsi.org
home and health care.
Contributing Writer Liz Ruskin
Advertising Senior Advertising and Marketing Manager Lisa Fick fick@auvsi.org +1 571 255 7779
botic agriculture. Many of us are raised be-
in robotics, beginning with their Machine
lieving that farming is a generations-old art.
Sync technology, which turns tractors into
While this isn’t false, the farms of the 21st
autonomous machines. The company is
century are full of high-tech gear, like so-
field-testing its future intelligent farming
phisticated irrigation systems or genetically
exploits through the Autonomous Orchard
engineered disease-resistant plants.
Project. The company aims to automate
While automating the farm is still a niche market in the United States, Japan has fully embraced the concept, becoming a giant in robotic farming.
using ground robots to farm the country’s prized strawberry harvest and how more forgiving airspace access laws allow farmers to crop dust over rice fields. In a country where the average farmer is nearly 66 years old, robotics has become more of a necessity than a trendy tech movement for
Executive Vice President Gretchen West AUVSI Headquarters 2700 South Quincy Street, Suite 400 Arlington, VA 22206 USA +1 703 845 9671 info@auvsi.org www.auvsi.org
Mission Critical Associate Editor Stephanie Levy explores the company’s endeavors
Liz Ruskin looks at how the Asian nation is
President and CEO Michael Toscano
ricultural giant John Deere. On Page 31,
This edition takes us to a literal field — ro-
On Page 8, Mission Critical contributor
A publication of
Danielle Lucey
the Land of the Rising Sun.
labor-intensive tasks in all kinds of weather and lighting conditions, so work can still persist all the way up to the next rooster’s crow. In a feature found on Page 18, I spoke with Virginia Tech crop biosecurity guru David Schmale, whose interdisciplinary work uses autonomous planes to track the movement of a vicious bacteria that affects plant and animal life on farms, causing blights in the fields and even possibly death to livestock. By tracking this genus of bacteria, which travels in layers in the atmosphere, sometimes from as far away as other countries, farmers could one day potentially prevent
Also, for Japan, a robot-filled story of the
diseases to their livelihood by doing pre-
heartbreaking aftermath of the 2011 earth-
ventative fungicide spraying.
quake and tsunami is coming full circle. A year ago, the country used unmanned systems to monitor the aftermath and search for victims of the disaster, but now, through a new program from the government, Japan intends to use the ravaged shores of Sendai and transform them in into the proverbial Elysian Fields for robotic farming. Flying the flag for the United States is ag-
The health and productivity of agriculture affects us all, from the price we pay for milk at the grocery store to our own personal health. Delving into agriculture has shed a lot of light on how important it is to ensure this sector is thriving, and securing that future safer, cheaper and easier with robotics is a real possibility. We hope you enjoy learning about it as much as we have.
Mission Critical
•
Spring 2012
3
Essential Components Bovine bonanza: Lely sells 12,500th cow milking robot Lely, a Dutch farm technology company, recently delivered its 12,500th cow-milking robot, which equals hundreds upon thousands of cows that produce millions of gallons of milk a day — numbers increased because of the technology.
SCAN IT
or
Click IT:
To see the winning video, click here or scan this barcode with your smartphone
The Astronaut A4 milking system is essentially a station that cows walk into, where then 3-D cameras target a cow’s udders and then the machine proceeds to milk it. Afterward, brushes clean the cow and a feeding trough emerges to coax the cow out of the stall. The Astronaut A4 also treats the milk in the system, ensuring a quality product through a Milk Quality Control sensor located in the arm of the robot, which performs the milking, says Lely. It measures things like protein, fat, lactose and color. The company says that since the machine uses a “dynamic herd approach,” which predicts how each cow will contribute to the total milk yield, this method increases total yield by 10 to 15 percent versus farms that milk twice a day. Lely says the robotic system is affordable to any dairy
A cow feeds while getting milked by the Lely Astronaut A4. Photo courtesy Lely.
farm that produces more than 300,000 liters, or 79,252 gallons, of milk per year. To promote the sale of the Astronaut A4 to its 12,500 customer, the company created a video competition for users to show how the system has impacted their farm.
misread what is actually present on a farm because of the time lapse it takes to create an image. What from the satellite’s point of view may look like an unauthorized building might actually be an innocent haystack. Testing of the UAVs is occurring in vineyards in the south of France. Wine
Spying the farm: UAVs may soon check up on EU farm subsidy claims
growers can claim about €10,000 per hectare in subsidies. The tests are try-
As a way to check up on who is, and more importantly isn’t, following up on the European Union’s Common Agricultural Policy rules, the EU is turning to satellites and unmanned systems to view farmland from above. The use of satellites is a few years old, but unmanned aerial systems are currently undergoing trials for the EU to use. The use of UAS would help in countries like Austria, which currently does not use satellites to scan farms because its mountainous terrain casts shadows that engulf a lot of farmland. Satellites can also often
4
Mission Critical
•
Spring 2012
SCAN IT
or
Click IT:
To read a paper from a February European Commission workshop on the matter, scan this barcode with your smartphone. http://www.eda.europa.eu/Libraries/
Essential Components
Strawberry fields for robots: UK’s NPL studies autonomous berry picking Scientists at the U.K.’s National Physical Laboratory have developed an imaging technology that can identify the ripeness of strawberries before they are picked. This leader-follower shopping cart completes the robotic agriculture cycle in the grocery store. Photo courtesy Chaotic Moon.
Researchers hope to use the technology to develop a new crop-picking robot. NPL’s technology uses radio frequencies,
ing to determine if the UAVs can have up to accuracy required.
Farm to table: Whole Foods tests follow-me shopping cart
Other tests are occurring in Italy and in
Mobile application studio company Chaotic
Spain’s Catalonia region.
Moon Labs has developed an Xbox Kinect-
10 centimeters’ resolution to determine the
Scanning a farm with a satellite currently costs about one-third of the price of using an inspector on a field visit, according to
powered shopping cart that is designed to
Scientists at NPL say the efficiency of this technology can bring down the cost and waste of harvesting fragile produce. First, the microwave measurement system maps the structure of the strawberry. This
Agency (RPA), which checks on the status
from an outgrowth of the company’s Board
of subsidy claims. The effort is to ensure
of Awesomeness product — a motion con-
farmers aren’t claiming any subsidy fraud.
trol motorized skateboard — and the larger Board of Imagination.
where there is some doubt about accura-
The similarity with the products lies in the
cy, and then only at the specific fields for
Xbox Kinect sensor they use, which is
which the doubt exists,” an RPA spokesper-
synced with a Windows 8 tablet for motion
son said in an interview with the BBC. “This
control.
reduces cost to the taxpayer.”
can also learn based on past experiences.
they’re shopping. Called the Smarter Cart, the creation came
saves time, lifts the burden on farmers and
tion to determine ripeness. The software
play follow-the-leader with a person while
numbers from the U.K.’s Rural Payments
“The RPA follows up only on those claims
microwaves, terahertz and infrared detec-
The cart can also track items by scanning barcodes, ensuring the product matches a
data then plugs into an algorithm that allows the technology to determine information about ripeness from a single measurement. Researchers tested the new technology on strawberries because of the demand for the fruit. They had previously run tests of the equipment on cauliflower, but had to call it off when demand for cauliflower dropped. “The focus now is strawberries. This is a fairly easy fruit to measure as it has high water content and dry leaves, and micro-
Each EU country inspects about 5 percent
shopper’s list, and it can detect if, for in-
of its farms per year, with many more
stance, an item is not gluten free when that
checking more than that.
has been specified as a parameter.
All this UAV research could play well for the
Whole Foods used its flagship location in
“Strawberries are also a high-value fruit that
future of the systems in Europe, since the EU
Austin, Texas, to test a cart this past Feb-
are very time-consuming to pick, so there is
is currently strategizing if it is possible to
ruary using about 30 shoppers. Chaotic
a stronger business case to implement auto-
loosen the restrictions of UAVs in civil ap-
Moon aims to test more than one cart at a
mated picking technology for strawberries
plications.
time beginning in April.
than with some other crops.”
wave imaging is particularly useful for identifying water levels,” lead scientist Dr. Richard Dudley said in a press release.
Mission Critical
•
Spring 2012
5
Essential Components — continued from Page 5
Pint-size propellers: The making of lightweight agricultural robots The robotics team at Queensland University of Technology in Australia is working on ways to make small, lightweight robots that cooperate in teams, or swarms, to survey large swaths of fields and apply herbicide. Their goal is to make a swarm of robots with cameras and advanced navigation capabilities that can cover large areas and resupply themselves when necessary. Dr. Ben Upcroft, faculty member at QUT, says the camera on the robot would have image recognition software to “pick out features of the weed which make it different from the rest of the crop.” The three-year project, which recently received nearly $400,000 in funding from the Australian Research Council, also includes the University of Sydney and Queenslandbased Advanced Agricultural Systems, run by Andrew Bate. Bate, who has a grain and cattle farm at Bendee, southwest of Emerald in central Queensland, said the automation of agriculture was a new frontier. “We’ve already reached peak farmland, so we have to figure out smarter farming systems which increase yield in a more cost-effective and environmentally sustainable way,” Bate said. “Every other industry is already enjoying the benefits of robotics. This is the revolution farming has to have.”
Harvest Automation secures $7.8 million in funding for plant-carrying robots
don and Tel Aviv, along with backing from existing investors Founder Collective, Life Sciences Partners, Cultivian Ventures and the Massachusetts Technology Development Corp.
Leptron’s Avenger available for ag use Recently receiving a lot of press because of a purchase by Arlington, Texas, police, Utah company Leptron Industrial Robotic
“I’m extremely excited to have the opportunity to work with … Entrée, who I know will bring significant value to Harvest. The strong support in this round from our existing investors is also a great indicator of the positive direction we have set and the collective enthusiasm for the future of the company,” says Charles Grinnell, CEO of Harvest Automation.
Helicopters’ Avenger platform is being promoted for use in agricultural applications. Available with commercial, off the shelf camera turrets, HD video, infrared and night-vision cameras from FLIR, synthetic aperture radar, and LIDAR packages, the 2-mile-range system could aid in mapping erosion, floods and other land management necessities, says the company.
The earliest prototypes of the robot were developed in 2008 with the company’s initial funding sponsors.
The electric platform was made in conjunction with Guided Systems Technologies, and it can either fly autonomously or
The small robots, about knee height, have a gripper for holding onto a potted plant and place the plants on a deck to carry the pot. Sensors on the vehicle let the robot know its surroundings, though they don’t use GPS or
through operator control with a joystick. The platform can fly for about 30 minutes and can sustain in 40 mph winds at up to 12,000 feet.
map their surroundings. While the robots currently do not have a price, Harvest Automation is aiming for the machines to run between $25,000 and $50,000. In the summer of 2011, the company tested three of the robots at 11 nurseries and greenhouses across the United States. “All 11 growers gave us deposits on their first purchases, which was really encouraging,” said Grinnell, chief executive of Harvest Automation, in an interview with The Boston Globe.
North Billerica, Mass., company Harvest Automation recently secured $7.8 million in funding to continue their work on prototype robotic farm equipment that would maneuver plants around nurseries and greenhouses. The company’s latest round of funding comes from Entrée Capital, based in Lon-
6
Mission Critical
•
Spring 2012
SCAN IT
or
Click IT:
Click this photo or scan this barcode with your smartphone to see Leptron’s Avenger perform grass reclamation work.
Driverless Car
sUMMiT
RegisteR Now! auvsi.org/dcs
12-13 June 2012
MotorCity Casino Hotel • Detroit
Driverless Cars in 2022‌
Making the Vision a Reality
Driverless Car Summit 2012 will bring together leaders from the
FACT
llion uld remove 112 mi Driverless cars co ar ye ch ea ad ro m the drunk drivers fro
FACT
The average car is imm obile for 96% of its lifetime
robotics and automotive communities with a common goal of making driverless cars a reality by 2022. The event will feature a mixture of inspiring presentations and interactive discussions designed to develop the path toward making driverless vehicles
FACT
a mainstream mode of transportation.
in In 2010, 35,000 people died in car accidents If car 2009. in 0 36,00 from the United States, down m deaths were put on a wall the size of the Vietna . years two every lt rebui be to have Memorial, it would
FACT
Maximum capacity of a highway today: 2,400 vehicles per hour. Driverless cars could more than double that using existing infrastructure.
Supported by:
John Deere answers customer demand for more autonomous agriculture By Stephanie Levy
I
f you harvest your crops using ra-
“Operators can see the location of all grain
dealership as they set up the new equip-
dio-operated combines, automated
carts in the network and, from the combine,
ment. Moorehead says the learning curve
tractors and the latest GPS technology, you
they can send a ready-to-unload request,”
for the new equipment is not steep. End us-
just might be a robot.
John Deere’s Agricultural Management
ers of Machine Sync, and other automated
Solutions manager Holli Brokaw said in
John Deere products already on the market,
an August 2011 press release highlight-
range from large industrial farms to owner/
ing the technology. “This information en-
operator farms.
And you may not realize it, but those technologies came to the farm through research across platforms in the agricultural, commercial and military sectors.
ables operators and managers to make the best decisions to maximize the harvesting
John Deere has provided technology solu-
fleet, while reducing fuel consumption and
tions in agriculture for 175 years (for more
ground compaction.”
information, see this issue’s Timeline on Page 28). Recently, those solutions have included bringing more automation to the farming process. The company’s transition has mirrored that of the auto industry: John Deere started adding driver assistance technologies to its tractors, but now a boom in automation development and demand may plant the seed for fully autonomous robots
“Since the [farmer] who’s buying this is the guy who’s sitting in the tractor, he has a lot of interest in improving his comfort level, whereas in a factory farm, perhaps the guy
“You can put someone like myself in
buying it isn’t the guy driving the tractor or
there and it will still do well,” says Stew-
the truck,” Moorehead says.
art Moorehead, robotics engineer at John Deere’s Intelligent Mobile Equipment division. “With this coordination by the radio allowing the vehicles to talk to each other and know where the other vehicles are, you can really reduce your downtime.”
Orange you glad it’s autonomous? To reach the next level of autonomy in agriculture, John Deere launched the Autonomous Orchard Project with the University
Reduced downtime leads to increased ef-
of Florida, Carnegie Mellon University and
ficiency, which Moorehead says accom-
Cornell University. The project currently
Machine Sync goes to market
plishes the end goal of lower costs.
boasts two tractors operating in Florida, as
In the summer of 2011, John Deere rolled
stop the combine and wait in the field for
out its Machine Sync technology. This sys-
a tractor to come so it can unload grain on
tem will drive a tractor autonomously while
them,” Moorehead says. “When it’s time to
a combine unloads grain, or other harvest
harvest this crop you’d like to get it har-
yield, into the wagon. Radios in the com-
vested as quickly as possible, otherwise the
“Tractors are the workhorses of the mod-
bine — which stores the crop during har-
weather might change. Once conditions
ern farm,” writes Anthony Stentz, Cristian
vesting — allow the combine to “talk” to the
are right for the harvest they really want to
Dima, Carl Wellington, Herman Herman
tractor during this operation. All the while,
get it done and get it done now.”
and David Stager in their 2002 paper
on and off the farm.
farmers know where all the combines are located in the field and how full they are.
8
Mission Critical
•
Spring 2012
“It’s very expensive for these farmers to
Buyers of Machine Sync will receive training from workers at their local John Deere
well as 10- and 15-foot mowers. Moorehead showcased the Autonomous Orchard Project as part of his presentation at the AUVSI-NDIA
Transportation
Technology
Transfer Initiative in May 2011.
about the Autonomous Orchard Project, published in Autonomous Robots. “By automating these machines, we can increase
the productivity, improve safety and reduce
and ditches in the field. If one of these ob-
The advantages of orchard automation are
costs for many agricultural operations.”
stacles is detected, the vehicle computes a
logistical and environmental. Improved lo-
safe speed to gradually slow to a stop. In
gistics lead to increased productivity; for
a sprayer capacity alone, the autonomous
example, an autonomous tractor can con-
tractor can follow a path of seven kilome-
duct day and night operations while driving
ters at speeds ranging from 5 to 8 kilome-
at a constant speed. This automation also
ters per hour.
removes people from chemical spraying
The purpose of the Autonomous Orchard Project is to increase the productivity of specialty crop producers by automating labor-intensive tasks in the field. Initially, the project just covered mowing and spraying
and reduces the reliance on fuel and harm-
tasks in the orange groves. By using a force
Moorehead said in his presentation that
multiplication approach, autonomous trac-
it’s key the autonomous farm vehicles de-
tors are capable of driving in a structured
tect and interact with “real obstacles.” That
“That would be good to remove people
orchard environment for most of the time. If
means the tractor can’t stop every time a
from those activities,” Moorehead says.
necessary, they can call a remote supervi-
branch, weed or stray orange falls in its
sor for help via wireless radio.
path. When the tractor stops for a detected
Using multiple sensors — a SICK laser, color and thermal cameras, and a MA/COM radar — the tractors in the Autonomous Orchard Project can detect people, vehicles
obstacle, it informs the remote supervisor who can then examine the data to determine if a real obstacle is actually there. Before motion resumes, the operator verifies that the way forward is clear.
ful chemicals.
The Autonomous Orchard Project continues to work on improved path planning and perception for the vehicles and all involved hope to get more customer use feedback. Better perception and path planning means making the tractors adaptable to weather
John Deere’s new Machine Sync brings higher levels of autonomy to the harvesting process. Moorehead says more automation leads to “better performance, less spilled grain because of poor station keeping and greater safety.” All photos courtesy of John Deere.
Mission Critical
•
Spring 2012
9
Video cameras on the tractors from the Autonomous Orchard project show obstacles at all angles. And while the tractor may move slowly, it gives the operator more time to work with the machine.
a traditional manned tractor to get the best automated efficiency. “It may be hard to imagine, but as you’re driving this tractor you’re trying to drive in this very straight line,” Moorehead says. “You’re trying to drive very close to where you drove the previous path, but you also have to turn around to make sure the impleThe Autonomous Orchard Project takes place at the University of Florida’s Institute of Food and Agricultural Sciences Extension.
ment is okay. It’s actually quite a stressful activity.” Now stretch that process out over the course of a 10-14 hour day. It’s enough to make anyone tired. Fatigue in the field can
variants like dust, fog, rain and snow. Moorehead says these next steps should take about three years.
hind the wheel, but a GPS map of the field
lead to manual errors and headland incon-
helps steer the vehicle.
sistencies, especially on irregularly shaped
John Deere’s Intelligent Total Equipment Control, or iTEC Pro, complements AutoTrac. Available on two different tractor
fields of crops. AutoTrac and iTEC Pro reduce or eliminate these risks to efficiency, safety and cost.
In Europe, John Deere is merging its work in
models, iTEC Pro coordinates the turns and
the agriculture and commercial sectors with
“You reduce your fuel usage, chemical us-
subsequent functions of the vehicle. It also
a new autonomous lawn mower. Tango,
age, seed usage, and at the end of the day
positions the tractor and implement cor-
debuted in 2011, is a robotic lawn mower
this all translates into money, but it has oth-
rectly for the next pass through the field.
designed specifically for homeowner use.
er advantages as well,” Moorehead says.
Moorehead recommends installing both on
“The company farms are in tune to this.”
Mark Bodwell, manager of military utility vehicle business at John Deere, says Tango has not yet been introduced in the U.S. because there is already a large commercial
The tractors in the Autonomous Orchard Project have been used in research and development for 10 years.
market for manned lawn care that doesn’t exist in Europe. “There are a lot of people, a lot of companies you can call in the United States that will mow your lawn for you,” Bodwell explains. “They don’t have that in Europe. Plus, they have smaller lawns.” These projects aren’t John Deere’s first foray into autonomous farming. In 2001, the company released its AutoTrac system for agriculture. AutoTrac automates tractor driving in the field and controls the implements on board. There is still a person be-
10
Mission Critical
•
Spring 2012
SCAN IT
or
Click IT:
Click or scan this barcode with your smartphone to read Stentz, Dima, Wellington, Herman and Stager’s Autonomous Orchard Project paper.
A soldier uses a video game console to teleoperate the R-Gator.
SCAN IT
or
Click IT:
Click or scan this barcode with your smartphone to see the R-Gator in action
Gator’s growl
The Gator has a 14-pound adjustable pay-
robotics, its initial offerings were the kinds
Off the farm, John Deere has developed
load capacity and four different levels of
of tools engineers and nerds dream of —
autonomy. At its most basic, the warfighter
“a laptop with all kinds of demands and
can drive the Gator like a regular car. Next,
screens full of numbers,” as Moorehead de-
the driver in the Gator can record a path for
scribes it. John Deere had to strip down the
the vehicle while driving it. At the third level
interface to make it more user friendly.
unmanned ground vehicles for military use. The company’s Gator vehicle, unveiled in 2004, has already become a popular UGV thanks to its simple user interface and the military’s reputation as an early adopter. John Deere unveiled the A3 version of the R-Gator at the Association of the U.S. Army
of control, Gator is teleoperational. Finally, the vehicle can be operated with total autonomy. To do all this, the Gator interface
convention in October 2011.
resembles that of a video game. John Deere
“The military has some really challeng-
warfighter straps it to an existing combat
ing problems,” Moorehead says. “Robotics may provide solutions for keeping the soldier out of harm’s way, resupplying the soldiers and helping with reconnaissance.”
also made it portable and wearable. The vest. Bodwell says these capabilities make
“It was a great way for us to take some of the autonomous components we’ve been working on in agriculture and in turf care and apply them to a domain that was, in some ways, more challenging and more rugged,” Moorehead says.
the Gator especially useful for obstacle de-
The Gator has been used in the U.S. Army
tection and avoidance.
as a test vehicle or surrogate for other un-
But when John Deere got its start in military
manned programs, and the Navy has used
Mission Critical
•
Spring 2012
11
John Deere — continued from Page 11 Gator for tasks like perimeter security.
example, lessons learned in fuel saving
start to integrate path planning into find-
Bodwell says NASA also got its hands on
techniques for autonomous farm vehicles
ing more optimal ways to work the field,”
a Gator to use as a surrogate for the Mars
have been applied to John Deere’s work on
Bodwell says.
rover program.
newer models of the Gator.
In 2011, John Deere also partnered with
“Many of these technologies that are very
Boeing to work on the assisted carriage
necessary in robotics — things like path
system program with the British military.
planning that can directly translate into
Bodwell says U.K. soldiers used Gator for
some of these guidance programs — we
a demonstration of their version of the U.S.
“Robotics is about optimization and about finding the shortest path.”
Stephanie Levy is associate editor of Mission Critical.
military’s lighten-the-load tasks. “The biggest surprise I’ve seen working inside the military is the surprise on people’s faces when they understand the technology that John Deere actually provides,” Bodwell says.
For More Information: http://www.deere.com http://www.deere.co.uk/wps/dcom/en_GB/products/ equipment/autonomous_mower/autonomous_mower.page
Farm to (drawing) table “We don’t view it in our company as: We do this robotics for agriculture and we do
http://www.rec.ri.cmu.edu/projects/auto_spraying/ http://edis.ifas.ufl.edu/ae467
this robotics for the military,” Moorehead says. “For us it’s all robotics, and there’s a very rich transfer of knowledge, lessons learned and technology between our tractor program and the military program.” Starting small, Bodwell says the planning and algorithms that go into John Deere software apply to both military and agricultural robotics. But in putting together autonomous vehicles, equal examples of technology transfer have happened from agriculture to military and vice versa. “The motor controller we use for steering [R-Gator] is the same motor controller we use for turf,” Bodwell says. “The vehicle control unit comes off our hay baler. A lot of the actuators we use on the products come off other products within the John Deere enterprise or inventory.” John
Deere’s
technology
transfer between its military and agricultural robotic platforms can solve common problems across the industries. For
12
Mission Critical
•
Spring 2012
John Deere’s R-Gator.
Putting the cart before the horse: John Deere’s technological evolution
nate human operators in the farming equipment itself.
A
s John Deere celebrates its 175th anniversary this year, the company is “an old company on the edge of technology trying to lead this industry,” according to Darren Havens, division manager of marketing for John Deere. Speaking to an agriculture-oriented conference last fall sponsored by AUVSI’s Atlanta Chapter, Havens pointed out that wasn’t always the case. “We have internal communications where we said, ‘Yeah, we’re going to offer the tractor, but still the horse and plow is the way to go,’” Havens said to big laughs from the audience. “It’s hard for people to get their heads around.” Now, Havens said the same thing is happening with the introduction of unmanned systems into the agriculture industry. John Deere
“The first ones are really tangible machines that, logistically, people can get their heads around,” Havens said. John Deere’s biggest challenges involve the use of data, and what it can mean for an end user just bringing automation to the field. John Deere is already working to manage ergonomic data. As a leader in technological development in farming, John Deere has taken on the responsibility of getting customers to understand data, simplifying the data process and standardizing data in the agriculture industry. “The ability to make that seamless, easy and executable as they go into the next planning season is what they need and desire,” Havens said. For example, there is currently very little consistency in data standards for autonomous systems in agriculture. Havens said the easiest way to fix that — so farmers can work from system to system or company to
We have internal communications where we said, ‘Yeah, we’re going to offer the tractor, but still the horse and plow is the way to go.’ — Darren Harris, John Deere marketing manager kick-started this movement in 1997 when it purchased the company NavCom. NavCom offers global navigation satellite systems and sensors for agriculture, as well as for undersea exploration.
company — is to make standard prescriptions for the machine’s data. The goal is to create a software development kit that is as user-friendly as an app on an iPhone.
These sensor capabilities have translated to more efficiency in the field. Havens said that while the technology may still be confusing for the end user, this efficiency, and the results it produces, is something they can get behind.
Havens also discussed the ways in which unmanned aerial vehicles could be used in an agriculture setting. Specifically, in order to effectively use UAVs in the field, Havens said John Deere, and other agbased companies, have to think about who the customer is, what the cost/value benefit would be for using UAVs in the field and the data insights that can be gleaned from UAVs that are not already available through traditional methods. Havens said convincing customers of the cost/value benefit of UAVs will probably prove to be the most difficult challenge.
“This was the ‘aha’ moment for a lot of customers, when they could jump into a vehicle or tractor and it goes on its own,” he said. “Customers are getting very attuned to that precise placement of whatever they’re doing.” As a next step, John Deere is looking to coordination of technology through telematics, or the integrated use of telecommunications and informatics, for their next phase efforts. Havens explained it as a piece of technology’s ability to communicate two ways. In 2011, every John Deere tractor, combine and sprayer that went out on the market was equipped with this telematics technology. Havens said this allowed farmers to “monitor that machine’s performance and logistics and manage the data flow from system to system easily.” For instance, John Deere took its first steps in telematics when it eliminated some of the paperwork necessary to get machines to do a certain type of pass in the field while harvesting. Now, a machine can execute its job in the field, and not only does the farmer know where the machine is at all times, but it also sends a work order back when the task is complete. Ultimately, this technology could be used in conjunction with AutoTrac and iTec Pro to elimi-
“Is there an opportunity for UAVs to play a role in simplifying this process, enabling new technologies to be applied to precision agriculture? I think there is,” he said. “The question is how does it evolve, and how is it applied?” Through machine optimization, logistics and precision support, John Deere is working to build a more autonomous agriculture industry, even if that means dragging end users along with the change, learning together as the technology evolves. “You don’t just start with technology,” Havens said. “It’s an integrated and complicated vision that has technology surrounding it.” “That’s what we have to make with autonomous systems: We have to make information that’s simple. That’s what our operators are expecting. That’s what they use personally when they use technology on a day-to-day basis.”
Mission Critical
•
Spring 2012
13
MARKETPLACE
Participate in AUVSI’s Member Marketplace, a NEW members-only network of free online classified advertisements featuring sections devoted to employment, research and for sale items.
Visit http://marketplace.auvsi.org today, or scan the barcode above to go to website.
MAXIMIZE YOUR VISIBILITY —
BECOME A CORPORATE MEMBER TODAY! · Discounts on Exhibits, Sponsorships, Advertising and AUVSI Products · Access to Members Only networking, education and Select VIP Events · Listing in Unmanned Systems Magazine and AUVSI’s Online Directory · Complimentary Job Listings on AUVSI’s Online Career Center · Research reports and knowledge sharing through AUVSI’s Online Community · Chapters and Regions around the Globe
Join today at www.auvsi.org/membership
Cropping up everywhere STATE OF THE ART
Leola, S.D.: The headquarter of Mack Robotics, South Dakota is home to the company’s Bin Bot invention. The robot can enter grain bins to empty out remaining grain without putting human life at risk. Read more on Bin Bot in Spotlight on Page 42.
Teddington, U.K.:
Cambridge, Mass.: In 2009, researchers from the Massachusetts Institute of Technology’s Offshore Aquaculture Engineering Center focused on building cages able to move under their own power that would aid in fish farming. Called Aquapods, the technology would enable fish farmers to rotate locations without towing cages behind boats.
Researchers at the National Physical Laboratory have developed an imaging technology that would intelligently harvest crops by looking beneath the leafy layers of that crop and predict precise size identification. The lab estimates that annual waste for crops can mean up to 100,000 pounds in lost revenue during an average year by poor harvest selection.
Ames, Iowa: Iowa-based inventor David Dorhout has created a hexapod robot with a face similar to WALL-E that could work in swarms to perform crop planting, tending and harvesting. To see a video of the massive robot, click here.
Pittsburgh, Pa.: The Carnegie Mellon Robotics Institute heads up the Comprehensive Automation for Specialty Crops (CASC) program, which involves industry and academic partners around the country developing and testing robotic technology for use on farms and orchards. Read more about CASC in Testing, Testing on Page 38.
San Diego, Calif.: San Diego company Vision Robotics Corp. develops robots for automating many chores, one of which is its Scout CAD, that uses six arms to scan trees for orange picking. Once the images are analyzed, the robot will generate at 3-D model, and then the arms can later go back and pick those fruit. The company also has robots planned for grape pruning and apple picking.
16
Mission Critical
•
Spring 2012
Gainesville, Fla.: Researchers from the University of Florida are testing using micro air vehicles to assess the health of citrus groves from disease.
F
rom coast to coast and heartland to heartland, agricultural robotic applications pop up all over the globe. While a lot of research has been done in the throughout the world, Europe is remarkable for its widespread adoption of robotic cow milking, while Japan’s myriad applications position the country as king of the robotic crop.
Maassluis, the Netherlands: The international company Lely makes the Astronaut A4, a machine that automates the process of milking cows; the cows can enter the walk-through robot on their own to be milked. The company, which introduced the first robotic milking system two decades ago, says 12,500 of its Astronaut systems have been installed worldwide.
Hamamatsu, Japan: Yamaha introduces the R-Max automated helicopter in 1990; by 2011 there were nearly 2,400 registered in the country for crop dusting, and they sprayed 30 percent of the country’s rice fields.
Sendai, Japan:
Beer-Sheva, Israel:
Japan’s Ministry of Agriculture plans on turning 600 acres of the area destroyed by the March 2011 earthquake and tsunami into a robotic farm, to advance the future of automation in agriculture.
Researchers at Ben-Gurion University of the Negev, working with European partners, are seeking to develop crop-picking robots able to identify, spray and pick a variety of crops, including grapes and apples that are sometimes hard to see among the surrounding foliage.
Tokyo: The Tokyo University of Agriculture and Technology has tested a “farming robotic suit” that would assist in the backbreaking labor of farm work simply by supplementing the user with an exoskeleton-style suit.
Perth, Australia: The University of Western Australia has had a robotic sheep shearer, called the Shear Magic Robot, since the 1990s, though the product has never been commercially available since it is not significantly faster or more efficient than human shearing.
Kensington, Australia: The University of New South Wales is developing a variety of automated farm equipment, including ground robots that can seed and weed crops.
Mission Critical
•
Spring 2012
17
Virginia Tech researchers seek out mysterious blankets of biomaterials By Danielle Lucey
A Rascal 110 UAV sits ready for a sampling mission at Virginia Tech’s Kentland Farm. All photos courtesy Schmale Laboratory at Virginia Tech.
S
omewhere, up in the air, live silent
passengers on a breeze. Kicked up
into the sky from as far away as other countries, these infinitesimally small microbes move in blanket-like waves. Not so silent is the havoc they can rain down on plant and animal life below, the results of which
Getting the project off the ground Ross has a background in dynamical systems and efficient space travel leveraged on gravity. By applying these concepts to the ways microbes move through the air, he can model and predict when these blankets
Farm, a campus-owned research farm about 10 miles west of Virginia Tech’s main Blacksburg, Va., campus. The facility has dedicated research areas and houses outreach and teaching space, where projects can bring in and educate stakeholders, members of the public and sometimes visit-
plague farmers, their crops, their animals
of microbes are on the move.
and even human health.
Schmale’s experience with unmanned aer-
functional and staffed in part through uni-
Called Lagrangian coherent structures by
ial vehicles puts the aircraft at the crux of
versity funding.
engineers, these dynamically distinct patches are typically talked about in terms of fluid
Schmale and Ross’ work, the tool that allows them to blend aerobiological monitor-
dynamics, relevant to ocean research. But
ing and plant pathology.
a project out of Virginia Tech hypothesizes
“We like to think about … microbial inva-
that similar structures occur in the air and waft between layers of the atmosphere. It’s these invisible blankets that brought Virginia Tech researchers David Schmale and Shane Ross together for their interdisciplinary research. Their work aims to determine when and where these waves of microbial
sions,” says Schmale, an associate professor in Virginia Tech’s Department of Plant Pathology, Physiology and Weed Science. He and Ross believe that pathogens move throughout the atmosphere and invade new territories. “We want to track them and find out where they’re coming from and where
invaders will happen and give farmers an
they’re going.”
early warning system to prepare for their
Virginia Tech is blessed with a large ag-
arrival.
18
ricultural research facility in its Kentland
Mission Critical
•
Spring 2012
ing school kids, says Schmale. The farm is
Most key for Schmale, however, is the facility’s 400-foot paved runway, perfect for the project’s fixed-wing unmanned systems. “Having the facility has been instrumental to our success,” says Schmale. “I would say in terms of challenges, getting that facility up and running, it didn’t happen overnight. We started all of our work basically on just a flat area of grass that the research farm gave us, and as we started to acquire resources, the administration started to respond with facilities.”
Going unmanned All this mysterious microbial movement falls under the umbrella of aerobiology, the study of the flow of life in the atmosphere, which is then broken down into three processes, says Schmale — release, horizontal transport and deposition. Fungi, for example, release their spores into the air, which are then swept up into turbulent currents. Then they’re transported over some distance and finally deposited onto another host crop. Release and deposition happen at the Earth’s surface, and there are many tools — like spore vacuums and Petri plates on stakes — to measure these two easily accessible processes. Horizontal transport, however, requires a way to assess the atmosphere and, as a result, is much harder to understand. There are a number of ways to do this, says Schmale, using full-size aircraft, balloons, even kites. “With full-scale aircraft you’ve got issues of at least risking one human life,” says Schmale, echoing that a lot of these missions fall into the dull, dirty and dangerous categories, “as AUVSI likes to reflect on quite a bit. So putting an unmanned system in the kicker is an important part of fulfilling research obligations and being able to do things fairly inexpensively and without the risk of human life. “It’s true, I mean a lot of the missions we do are very dull. I wouldn’t say they’re dangerous. They’re very dull and very dirty.” When they sample, the team goes out for a week and flies all day long. And since
A ground-based sampling device to collect Fusarium.
they’re using an autopilot, “the plane’s doing the flying; we’re just sitting back watching.” Sidestepping using a full-scale aircraft for this and using unmanned systems instead helps reduce pilot fatigue, says Schmale.
on Petri dishes containing a selective agar
Certificates of Authorization from the Fed-
medium.
eral Aviation Administration.
The team is using a SIG Rascal 110 for
The Penguin B comes with the largest price
their National Science Foundation-funded
For their research, Schmale, Ross and their
research, but they also fly a BlueBird Boo-
students use a series of four unmanned aer-
merang, a UAV Factory Penguin B and a
ial vehicles, all readily available as remote-
Senior Telemaster. All of the platforms are
controlled platforms, to gather pathogens
licensed to fly through their four separate
tag. It’s a fiberglass, “more professional, commercial body frame,” meanwhile the others are more hobby-based plywood platforms that aid in the project’s workload, says Schmale.
Mission Critical
•
Spring 2012
19
Airspace Invaders — continued from Page 19 The Rascal 110 is the project’s “tried and
activity still occurs above the altitude where
true platform,” says Schmale. It can fly for
the team is allowed to fly.
45 minutes with their current configuration, “and she carries the wing loading of our plates really, really well, and so we’ve used her exclusively for all our NSF work.” For
their
sensor-based
applications,
Schmale prefers the Penguin B, “because it’s a pusher, and of course we’re not disturbing airflow. We’re sort of in front of the plane, which allows us to channel things into the fuselage.” The team has published data on hundreds of their flights, though a very literal ceiling limits the research the project can do. While a lot of transport blankets occur in the operable airspace under Virginia Tech’s certificates of authorization, much of the
search on farms in the future. “The idea of a consultant that wants to set
“The pathogens are flying considerably
up a business that’s based in part on our
higher than the FAA will let us actually fly
technology would have to get a COA in
for our COAs,” says Schmale.
“They’re
order to do this, and I don’t know how they
moving through the planetary boundary
would do it,” he says. “You’d have to know
layer and beyond.”
in advance where you’d be flying, and
However, the higher a microbe goes, the more it will be exposed to the elements, says Schmale, like UV radiation, which is hard on things like spores and can kill them. The work at Kentland Farm is all restricted under the four COAs, which specify both altitude and operational restrictions.
you’d basically have to, more or less, say you’re going to fly anywhere in the contiguous U.S. Really, that’s the biggest restriction right now. There’s nothing more than that.” Growers, however, are amenable to the idea of UAVs buzzing over their farms, says Schmale. If a plane were able to go up as a wave of microbes were coming and detect
Having the Kentland Farm facility makes
where they were coming from, it could help
it easier for the FAA to clear them, says
a farmer know when to spray a fungicide.
Schmale. The FAA’s current policies, however, could limit implementation of this re-
CropCam in Canada has the farming mar-
The Rascal 110 is outfitted with sampling arms, which open and close during flight.
20
Mission Critical
•
Spring 2012
species,” he says. One of the most important mycotoxins in the United States, according to Schmale, is deoxynivalenol, also known by the much more descriptive name vomitoxin. “Symptoms that often occur from domestic animals consuming grain basically contaminated with this toxin will often exhibit symptoms of vomiting, even refuse to eat their feed, particularly in the case of swine, and death can ensue if levels are high enough,” warns Schmale. Another type of mycotoxin — known as zearalenone — plagues animals like pigs and mimics the hormone estrogen, resulting in reproductive problems for the animals, Preparing sampling devices for unmanned aerial vehicles to collect Fusarium in the lower atmosphere.
like swollen teats and ovaries. “It’s a really big deal,” says Schmale. Other kinds of Fusarium produce mycotoxins that lace common staple crops for humans, like wheat and corn. There are other species that cause skin diseases in humans and an eye disease called Fusari-
ket cornered in that country, says Schmale,
this group … causes a number of different,
because of its price and reliability. That
important diseases of relevance to plants,
model would also play well in the U.S., he
domestic animals and humans,” says
says.
Schmale.
“Those type of systems if you can get into
It can cause wilts, rots, cankers and blights
the few thousand dollar range, I think grow-
to many kinds of crops.
also creates pulmonary infections, with re-
“You’d be hard-pressed to find a crop that
zil, says Schmale.
ers would be really willing to entertain them, but then it’s the whole idea of actually being able to use it and make something of it in your business. And that’s where, if it’s not a hobby, it’s a business, and the FAA has to be involved.”
didn’t have some sort of Fusarium disease associated with it,” says Schmale. “It’s quite phenomenal.” There are about 80 biological species of the genus around the globe, and that iden-
Fungus fatale The main culprit in all this pathogen research is the extremely prolific fungus in the genus Fusarium, the sole microbe of interest in the NSF study. While not all species of Fusarium are dangerous — one is even approved for sale as food by the United Kingdom — many strains of the fungi cause widespread disease. “Fusarium is one of the most important genera of fungi on the planet Earth, and
tified number likely falls far short of the actual number of species, which Schmale estimates as perhaps twice that number or
um keratitis that particularly affects contact lens wearers that don’t frequently wash their lenses in solution, where the fungus implants onto the cornea outer layer of the eye and grows through its layers. Fusarium cent localized outbreaks occurring in Bra-
Gathering samples To actually collect these microbial samples, the UAVs are outfitted with a series of Petri dishes that open and close at set points in flight. “We have these sampling arms … that
more.
operated like a clam shell, that open and
Of the dangerous strains of Fusarium, some
Schmale.
produce dangerous toxins, called mycotoxins, commonly referred to as fungusproduced molds. These mycotoxins can
close remote control from the ground,” says
There are inner and outer sampling arms, and the configuration on the Rascal 110
contaminate food and feed, says Schmale.
can hold eight Petri plates. The plates con-
“We do a lot of analytical chemistry to
Fusarium.
detect these toxins in food and feed and ultimately attribute them to certain Fusarium
tain a specific medium that is selective for
“What’s neat about using these plates as a
Mission Critical
•
Spring 2012
21
Airspace Invaders — continued from Page 21 sampling device is that with this medium we
Once they land, they’ll detach the used
can basically select just for Fusarium and
plates, put on new ones and repeat the
kind of ignore all the other stuff we’re not
flights. All the sample analysis happens on
trying to track.”
the ground in their lab.
The arms are closed during takeoff and
Types of sampling surfaces in the plates in-
SPR, that allows users to detect an agent in
landing, so there is no exposure to the
clude things like grease, sticky tape and, in
real time, so the team could identify biologi-
plates until the arms are opened, prevent-
the last couple years, filter papers coated
cal agents while the UAV is flying.
ing any sample contamination.
with glycerin that allow them to collect a
The sampling pattern is all autonomous; meanwhile, the researchers open one set of arms for 10 minutes, close them and repeat the process with the second set of arms. “We have a sample that’s been collected over the period of 20 minutes, but it’s actually two independent samples, kind of like two separate flights, if you will,” says Schmale.
then inspected using DNA-based analyses back in the lab. The future of this research
Mission Critical
•
Spring 2012
They are also exploring a device from a Washington company that makes a technology called surface plasmon resonance, or
Once identified, the researchers can hypothesize where a sample was flowing from based on the time of year.
is likely this type of multimicrobial analysis,
“That’s a big component of what we do,”
says Schmale.
says Schmale. “Some of the species are
Other sensors on the aircraft include an ionic spore sensor, which takes air funneled through the nose of a pusher platform. The air is charged as it passes through this device, and the particles then are deposited
Members of the Schmale lab conduct an aerial sampling mission.
22
very varied sample of microbes, which are
on a surface of opposite charge.
exclusively localized to a certain region or certain crop type, so if you’re collecting a tropical species, for example, during the winter time here in Blacksburg, Va., then you might question, well, it’s likely coming from some warmer, tropical region.”
The team does back-trajectory analysis to
The students with no prior experience with
“What they might consider as a toy is a
test that hypothesis and looks for seasonal
unmanned systems tend to have a warm re-
tool for us,” says Schmale. “It sort of begs
trends in what they’re collecting. One of his
ception to the technology, he says.
the question of, what is the future of agri-
graduate students is focusing her research on these recurring trends from year to year.
“Everybody sort of gets excited … about things that fly. It’s just really cool,” says
Schmale says he’s very selective about
Schmale. “When you take something that
which grad students get to participate in his
can fly and then you turn it into a tool …
and Ross’ program, which blends students
that has some relevance to the system of
from many backgrounds and majors, some
agriculture, in general students get really
of which have never used unmanned sys-
excited.” Schmale says he’s impressed with
tems before.
how quickly young students take to the top-
“I have students who are trained solely through my department … and then we have students who also have a split role,
culture? And are these unmanned systems a part of it? Of course the answer coming from my side of the fence is yes.”
Danielle Lucey is managing editor of Mission Critical.
ic and the intelligent questions they have. Schmale also has an RC simulator where high schoolers can try flying for themselves.
who are basically co-advised, but their de-
“Of course they crash and burn pretty
gree will come from an engineering depart-
quickly after they try, but it’s all on a video
ment or from my department, for example,”
game, which is really nice,” he says. “They
says Schmale.
can appreciate how tough it really is.
Mission Critical
•
Spring 2012
23
Need to Learn More? No Time to Leave Your Desk?
Tune into AUVSI Webinars AUVSI webinars are a new and fastgrowing service that delivers expert speakers and hot topics in unmanned systems to your desktop. Topics over the coming year will include all facets of this dynamic industry.
Members receive a special discount on all webinar content. Listen to the webinars live, or download them to your computer later.
Check our website for the latest schedule: http://www.auvsi.org/publications/auvsiwebinarseries.
AUVSI’s quarterly publication that highlights special topics in the unmanned systems industry.
Each is an in-depth focus on one particular issue with information on the defense, civil and commercial applications of the technology as well as new developments and what the future may hold.
Upcoming Issues:
VOLUME 1
NO.2 • S
UMMER 20
11 • AU
VSI • 270
Robots aid Japa
0 South Qu
incy Street
, Suite 400
n
Commercial Robots: Summer Edition Security: Fall Edition Sensors: Winter Edition
V
OLUME
1 NO.1
• SPR
ING 20
11 •
I • 2 AUVS
700 S
incy S outh Qu
, Arlington
, VA 2 2 2 0 6
, USA
Unmanned syste ms fight fires
treet, S
uite 40
0, Arlin
g t o n , VA
22206
, USA
ected DOT’s connork w le vehic rtre Europe’s Sa s in a tr d a ro
Robots help p
olice
Inside this issu
e:
First responder
e
robots
Challeng nd Driver
Bli issue:
Inside this
tion
nsporta Smart Tra
MISSION
CRITICAL
•
Spring 201
1
MISSI ON CRITIC AL
I
If your company has technology in any of these arenas, then you can’t afford to miss this opportunity — book your advertising space today!
Contact Lisa Fick at fick@auvsi.org for more information.
•
Summer 2011
1
Dr. Daniel Schmoldt
Q&A
National Institute of Food and Agriculture Staff Director U.S. Department of Agriculture Dr. Daniel Schmoldt, National Institute of Food and Agriculture staff director at the U.S. Department of Agriculture, has helped get the agency involved in President Obama’s National Robotics Initiative. The USDA has teamed up with three other agencies — the National Science Foundation, NASA and the National Institutes of Health — to collect proposals, small and large, for new uses of robotics in the civil sector. While the farming industry has been slow to pick up robotics in the past, Schmoldt says there are now ample opportunities for unmanned systems to plow — or till, or harvest — their way into the industry. Dr. Daniel Schmoldt at AUVSI’s Unmanned Systems Program Review 2012. AUVSI photo.
Q:
How has the market for unmanned systems and robotics in agriculture evolved?
labor is probably the biggest cost they have.
There’s really only one significant funding
They see labor-saving technology as critical
authority for our agency where we have suf-
to their survival.
ficient discretion to use funds for something like [the Agriculture and Food Research Initia-
Because of the farm workers unions and things
tive]. It wasn’t increased this year, but at least
like that back in the ‘70s, there was a strong backlash against mechanization, because it was eliminating jobs for farm workers. A lot of the robotics that would have ensued and developed from that really didn’t have a chance, because the people that were doing the engineering and technology development were turned away from that. The funding wasn’t there; the incentives weren’t there. But in the last 10 years or so, there’s been a real strong push from the producers, from the stakeholders, because they have a lot of competition from international producers — for example apples from Chile and China, etc. — so they need to increase their production efforts, and
26
Mission Critical
•
Spring 2012
Q:
it wasn’t decreased, which is a good thing.
This is a presidential initiative, not a congres-
Q:
How is the USDA adjusting financially to fund the NRI?
sional one. There’s no new funding line for this, so every agency has to find some place in their discretionary expenditures and have
Do you think NRI funding will survive the current push for austerity on the federal level?
the appropriate legal authorities to support
In this administration, I think there’s an expec-
this program with their regular appropria-
tation, and maybe even a realization, that
tions. Unfortunately on our end … we have
robotics is one potential driver of economic
a whole bunch of very specific legal-funding
development going forward in the next 10
authorities, and those programs are supposed
to 20 years. It’s a burgeoning industry glob-
to only be working on certain things. It’s very
ally, and especially in Europe and Asia, and
prescriptive.
we’re going to see more and more economic development surrounding robotics. So if we
can advance that with this research and de-
quality tends to improve with newer technol-
jobs that are low skill, low wage, plus dan-
velopment program, the NRI, that certainly
ogy in automation.
gerous and ergonomically unfriendly.
seems to be a good way to support that.
Also there’s the opportunity to introduce sen-
It is already happening to some extent, be-
I think the interest is there and the rationale is
sors that can improve the practice; whether
cause there are new technologies coming in,
reasonably well placed for maintaining this
it’s harvesting or thinning, it can be improved
maybe not so much unmanned systems and
program. The exact levels it will be funded at
by the right sensors that humans don’t really
robotics, but other technologies that are help-
remains to be seen.
have or aren’t very good at. You’re going to
ing with this. It’s going to have to come with
see product quality improve.
the technology; it’s not going to be one or the other first. It’s going have to be introduced
Q:
How accessible are these unmanned systems and robotics to smaller momand-pop farms, as opposed to larger factory farms? What can be done to close this gap?
Obviously that’s very much a concern for us.
almost concurrently.
Q:
What does the testing process look like for emerging unmanned systems and robotic technology in agriculture? Are there any standards?
Q:
What kind of dream technology do you want to see down on the farm in terms of unmanned systems and robotics?
We like to have technology that’s scale neu-
There are some standards, for example
tral: It’s accessible from a cross-perspective
safety standards for agriculture equipment.
When I sort of go into a dream state about
for small- and medium-size producers as well
We have suggested in various meetings and
this, what would be nice to see is crop tend-
as large producers. There’s economy at scale
asked a couple of people for development
ing robots. That is whatever crop you’re talk-
associated with these things.
of standards for unmanned robotic systems in
ing about, whether it’s an orchard or tomato
agriculture. I can’t tell you in what stage that
field, that there would be sort of a swarm
is at right now.
of robots that could fairly continuously move
In some cases it’s not just economics; it’s not just cost. It’s other things relative to their operation — their age, who else they’re work-
If we’re able to help support the development,
ing with, their comfort level with technology
we’d like to see them eventually adopted. If
through the field and just monitor and perform opposite stimuli and coordinate among
“While [automation] eliminates more jobs in the short run, it creates more jobs in the long run. We want to see better jobs and get rid of these other jobs that are low skill, low wage, plus dangerous and ergonomically unfriendly.”
– Dr. Daniel Schmoldt
in general. That gets into the second piece of
there are safety roadblocks that sort of limit
accessibility: Do they have the time to use the
themselves along with the people managing
the adoption, we’ve encouraged the appro-
tech? There’s a learning curve, plus there’s a
the operation. Things get done, and they can
priate people — the people that set standards
time consideration.
look for pests, detect pests while they’re pull-
and deal with the legal team — to see these
ing weeds or cultivating weeds, or whatever
are dealt with in a timely fashion.
the operation might be. There’s just this fairly continuous revisiting of the crop production
Q:
How does more automation in agriculture affect the quality of the end product?
Robotics and the new automation can actually be friendlier, because instead of human
environment. The operator knows what’s go-
Q:
How has the automation of agriculture affected the job market for people working on farms? Are there any new emerging opportunities?
laborers, which get tired, they get fatigued,
While it eliminates more jobs in the short run,
they get distracted and all sorts of other
it creates more jobs in the long run. We want
things, machines do not. The end-product
to see better jobs and get rid of these other
ing and what’s being done and feels very comfortable. They know exactly what product quality they’re going to have and what yield they’re going to have at the end of the growing season.
Mission Critical
•
Spring 2012
27
John Deere
TIMELINE
1943
1837
28
1850
John Deere, a blacksmith and inventor, founds his namesake company. The first piece of farming technology he makes is a polished-steel plow that helps Midwest farmers plow through sticky prairie soil.
1800
The company’s history dates back to the early days of American westward expansion, giving it ample room to grow with, and lead, technology trends in farming.
Mission Critical
•
Spring 2012
1918
Motorized equipment on the farm becomes popularized, and the most popular piece of motorized equipment is the tractor. John Deere gets into the tractor business after buying out Waterloo Boy Tractors. The company sells Waterloo Boymodel tractors for five years before coming out with its first original Model “D” tractor.
1900
T
he first piece of unmanned equipment in the John Deere family was a horse.
The company makes the Hawkeye Riding Cultivator, the first piece of Deere equipment adapted for riding. At first the driver has to stand on the cultivator, but later models include seats.
1999
John Deere begins work on intelligent mobile equipment, including the use of satellites in farming and taking the first steps in automating agricultural processes.
1950
1863
John Deere makes military tractors, ammunition, aircraft parts and cargo, and mobile laundry units during World War II, in part because “limitation orders” restricted civilian production of farm equipment, repair parts and exports.
2011
2001
2002
John Deere works on the Autonomous Orchard Project with Carnegie Mellon University, Cornell University and the University of Florida.
2004 John Deere officially unveils the Gator vehicle platform to the U.S. military. Using some automated technologies adapted from agriculture products, warfighters can drive the vehicle or operate it as an unmanned ground vehicle. It’s most recent A3 model debuted in 2011.
2012
John Deere celebrates its 175th anniversary.
2000
AutoTrac becomes available for John Deere tractors. The technology fuses together satellite position data from the StarFire receiver with feeler data gathered from the row sensor.
John Deere debuts Machine Sync. New Machine Sync technology allows the combine operator to automatically control the location of the tractor and grain cart while simultaneously unloading during field operations.
Mission Critical
•
Spring 2012
29
Ripe for the picking MARKET REPORT
I
n robotic agriculture, just as with any
to Europe, with only 10 U.S. dairy farms
In a series of interviews conducted with
other industry, you reap what you sow.
owning robotics milkers as of 2009. In that
attendees at the 2011 World Ag Expo, a
year, there were more than 2,000 farms in
blog by The Robot Report, a news outlet
the world that employed the milkers. The
that tracks the state of business in robotics,
machines were first commercially available
reported that nearly 50 percent of robotic
to the United States in 2000, but were in ex-
tractors employ unmanned driving system
istence in places like the Netherlands start-
technology, however technologies like sen-
The world’s premiere conference in all
ing back in 1992. Purdue University dairy
sor and motion- or vision-based systems,
things farming is the World Ag Expo, held
specialist Mike Schutz, in an interview with
like berry harvesting robots, have until now
annually in Tulare, Calif. With more than
CBS News, attributes the slow adoption in
been slow to be adopted because of how
1,600 exhibitors from around the world,
the U.S. to the country’s cheap labor and
pricy the systems are.
the show picks 10 technologies every year
regulations that give farmers more incentive
that it deems the best to advance the field
to add cows than to make a smaller number
of agriculture.
of cows more efficient milkers.
In 2012, these Top 10 New Products in-
Another Top 10 New Products winner came
of seed, sprays and labor and the lower
cluded some with technologies from the
from specialty harvest company Oxbo Inter-
costs of these sensors. This environment
world of robotics, which could hold clues
national Corp. The Byron, N.Y., company’s
makes automated technology a prime in-
for the proverbial low-hanging fruit in the
9120 Raspberry Harvester features auto-
vestment for larger producers.
robo-ag world.
mated steering technology.
Swedish company DeLaval’s innovations in
“This machine could probably replace two-
company that is the lead material handling
dairy farming date back to 1894 when it
to three-hundred people out in the field, so
systems integrator of automation technolo-
held patents for early forms of milking ma-
when you compare that to what this is do-
gies, attended this year’s World Ag Expo
chines and the first centrifugal milk-cream
ing to your capital costs compared to the
and observed in his company blog that the
separator. This year, it was recognized for
labor cost you can pay it off in a year or
automation market is ripening.
the world’s first automated rotary parlor for
two,” said Scott Korthuis, a product man-
high-capacity milking, the DeLaval AMR.
ager with Oxbo, in another KVPR interview.
“Helping dairy farmers achieve a level of
The machine also employs Oxbo’s Orbitro-
dling suppliers just by the nature of what
sustainably that allows them to stay in busi-
tor, a selective picking system that seeks out
they do every day with their products, but
ness has been a focus and one of the ways
the best raspberries, in this case the IQF,
now, automation with material handling
that we have chosen to do that is with tech-
or individually quick frozen — the highest
is becoming more important as global de-
nology,” said Ben Kern, western regional
grade raspberry.
mand keeps rising and everyone wants to
Though farms aren’t currently over-
whelmed with robotic equipment, automating the process of farming is certainly a growing trend.
California’s KVPR Valley Public Radio.
of robotic milking technology, compared
SCAN IT
or
Robotic milkmaids all in a row: DeLaval’s Automatic Milking Rotary innovation. Photo courtesy DeLaval.
Click IT:
Click the photo or scan this barcode with your smartphone to see a video of the DeLaval Automatic Milking Rotary dariy management system
30
Mission Critical
•
Spring 2012
that the industry is slowly moving toward more automation because of the rising cost
Greg McGiffney of Bastian Solutions, a
“The agriculture and food handling businesses are big industries for material han-
be more efficient.”
director for DeLaval, in an interview with
The United States has been a slow adopter
The Robotics Report also says, however,
S trawberries ,
spraying and strength — Robotic farming in Japan By Liz Ruskin
The objects of their affection: These strawberries were for sale in a mid-range department store west of Tokyo for a little more than $1 per fruit. All photos by Liz Ruskin.
products, thanks in part to price supports and to policies that have discouraged cheaper imports. Japanese consumers tolerate the higher prices, because they tend to distrust the safety of food from overseas. That, plus sustained government support in a country known for its engineering prowess, seems to make up for the meager economies of scale. So, filled with a sense of national mission, Japanese engineers
S
have set about creating robots to do farm work. It’s a very Japatrawberries in Japan are prized jewels. They are sold, all
ruby red and perfectly aligned, in small trays in supermar-
kets and in the food halls of Tokyo department stores. Some, created for the high-end gift market, go for as much as $1 to $5 per
nese solution to uniquely Japanese problem, and it has put this country at the forefront of agribot research and development.
Strawberry-harvesting bot
berry. Bruising is not tolerated.
At an agricultural research and development campus north of To-
This level of perfection demands not just tender treatment, but a
harvesting robot that has been in the works since 2003. The ma-
lot of backbreaking labor — nearly 1,000 hours per quarter-acre greenhouse — just to pick, grade and pack. The trouble is, labor is in short supply in Japan. The farming population has shrunk by one-third over the past decade, the result of rural migration to the cities and Japan’s tight immigration policies. The average age of a farmer is now nearly 66 years old. One in four farm workers is
kyo, Dr. Shigehiko Hayashi recently demonstrated a strawberrychine was built to travel on rails through a greenhouse of elevated beds, looking for red objects with its three-camera-lens eyes. When its particular color conditions are met, it recognizes the object as a fruit and homes in. Then, using two side cameras arranged for stereoscopic vision, it determines the 3-D position of the fruit.
over 70.
Next it really shows its smarts, employing a “maturity assessment
The government, rallying the country to boost food self-sufficiency,
robot considers the hue saturation, the relative size of the red and
is investing in a range of automation projects, hoping robots will step up to help Japan increase its agricultural production. At first glance, Japanese agriculture doesn’t seem well suited to expensive machinery. Compared to the United States, farms here are tiny, just 4.5 acres on average, and often tucked into valleys or hemmed in by city sprawl. But these farms produce high-value
algorithm” to decide whether the fruit is ripe enough to pick. The non-red areas, and the margin between them. If it decides the fruit is ripe, its pincer-like hand reaches out for the stem, tilting its wrist to match the curve of the vine. It cuts and grasps in one motion. Holding the strawberry by a long stem, it then rotates to place the berry in a harvesting tray. It’s equipped with an optional suction device that can help hold the fruit for
Japan — continued from Page 31 the fruit the robot missed, but the robot could reduce their hours by more than half. Now on his fourth prototype, Hayashi says he’s getting close to commercial readiness. The cost per robot is now about $120,000. He figures he needs to bring it down to $70,000 or lower for it to be practical for a normal farm. He and his colleagues at IAM-BRAIN have also created a robot to that can grade and pack strawberries to the exacting standards of the Japanese market. While other machines rely on mechanical grippers, the IAM-BRAIN team found such machines couldn’t place the strawberries precisely next to each other on a tray, because the “fingers” tended to dislodge neighboring berries every time a new one was placed. Theirs, a two-armed contraption, uses suction alone to hold the berry, and it only A strawberry-harvesting robot created at the Japanese R&D institute known as IAM-BRAIN.
holds the fruit by its leafy green cap. To do that, it uses its vision system to determine exactly what angle to use to approach each berry. Its nuanced tilting gives the impres-
added security. Otherwise, the berry itself
Technology Research Advancement Institu-
goes untouched by man or machine.
tion, better known by its catchy acronym,
The robot repeats the process all the way down a row and then comes back up the
IAM-BRAIN. “It can only harvest 60 percent or 70 percent of the ripe fruit.”
sion that it is quizzically inspecting each target. The machine can sort by size and settle each in its proper tray. Dr. Satoshi Yamamoto, a colleague of Hayashi’s, says the current version of the packer would costs
same rails to pick berries from the other
The robot will often fail to detect a ripe ber-
about $86,000. He thinks it would be a
side of the aisle. Then it starts down the next
ry if an immature one is hanging in front of
practical choice for a strawberry farm if
aisle. When its tray is full, it brings it to a
it, he says, or if it is hidden behind leaves.
they can bring the price down to $60,000.
stacker, discharges it and takes an empty
Its success at judging ripeness depends on
one. It continues like that until it has worked
the cultivar. On types with a clear margin
the entire greenhouse.
between the ripe and unripe areas, its
“This machine is not perfect,” says Hayashi, of the Institute of Agricultural Machinery, Bio-oriented
judgment is about the same as a human’s, Hayashi says.
•
Spring 2012
future came early to some corners of the Japanese agricultural scene. Way back in 1983, an agricultural engineering profes-
nine seconds per berry, about the same as
sor from Kyoto University demonstrated
a person. The robot, however, could work
an intelligent robot for tomato harvesting.
well in conjunction with human harvesters,
Tokyo-based Iseki & Co. has had a robotic
he says. It was designed to work all night.
system for grafting seedlings on the market
The cooler air at night helps keep the fruit
since 1993.
illuminate its target.
Mission Critical
If agribots are the future of farming, the
It’s not particularly fast, either. It takes about
firm, and the robot has its own LED lights to
32
Robotic spraying
But the most prevalent unmanned systems at work on Japanese farms are in the air,
Workers, who typically harvest in the early
hovering over rice fields. Unmanned heli-
morning, would still come in early to pick
copters, on the market since 1990, are now
routinely used here for crop dusting, sav-
Only a few have been exported to the U.S.,
farmland on the northeast coast were inun-
ing Japan’s elderly farmers from countless
primarily for research purposes. They were
dated by saltwater and debris.
hours of manual spraying.
introduced to the South Korean market for
More than 2,300 are registered for use here, the vast majority versions of the Yamaha RMAX.
insecticide spraying in 2003, but fewer than 90 were registered by 2010. Korean farms are bigger, so they would benefit from a larger aircraft, says William Park,
The RMAX is roughly the size of a motor-
founder of Korea’s UAV Center and infor-
bike and weighs 128 pounds empty. It will
mation clearinghouse. He also says the un-
carry 66 pounds of pesticides or fertilizer,
manned helicopter had an abysmal safety
as liquid or granules. It can fly slow and
record in Korea, which he attributed to a
low, as little as three yards off the ground,
lack of training.
for an hour. Compared with manned helicopters, the
Unmanned tractors
RMAX is quieter and produces less chemi-
Among the best uses of robots is to send
cal drift. In 2003 unmanned helicopters for
them where human workers fear to tread,
the first time sprayed more acreage than
and by that yardstick the March 2011 tsu-
manned choppers. Now more than 30 per-
nami and nuclear disaster created ideal
cent of Japan’s fields are sprayed by these
proving grounds for agricultural robots.
UAVs.
About 10,900 hectares of rice fields in the
The RMAX is usually operated with a boxy controller, similar to those used for remote control toys, and is reportedly easy to fly. If the operator lets go of the control stick, the
Fukushima Prefecture will go unplanted this year, due to contamination from the nuclear meltdowns at the Fukushima Dai-ichi power plant. In addition, some 20,000 hectares of
The Japanese Ministry of Agriculture, Forestry and Fisheries announced earlier this year that it is funding a series of projects in the tsunami damage zone in which robots and unmanned tractors will be put to work rehabilitating fields in four communities in Miyagi, the prefecture that surrounds the city of Sendai and borders Fukushima. The government has committed to spending $52 million on the project over the next six years. Japan’s Institute of Agricultural Machinery has been busy preparing an unmanned tractor for use on the contaminated fields of Fukushima. The prototype can be operated from a control car a kilometer away. It has cameras on the front and side and navigates with the help of GPS technology. Its role, says the institute’s Keiji Hanawa, would be to crush the soil surface to prepare for the removal of the top two inches,
RMAX defaults to a dead-still hover. But the RMAX, which comes with or without GPS, is no toy. Prices for basic models approach six figures. Yamaha also makes an autonomous version, operated from a base station, that reportedly costs $270,000 or up to a $1 million depending on what equipment it comes with. Their potential for farmers goes beyond pesticide application. Scientists at IAM-BRAIN developed a “crop eye” for mounting under an RMAX helicopter. It is a growth-measuring device that is designed to fly over fields, taking pictures at red, near-infrared and sunlight wavelengths simultaneously. Image analysis software uses the pictures to determine the nitrogen content of the plants and thus gauge the growth index. The RMAX and its predecessors were designed in Japan, to meet Japan’s agricultural needs. Its popularity proves it’s well suited to Japan’s small farms. But how well would it work outside of its home country?
This strawberry-packing robot determines which way the berry is lying, then uses suction to pick it up by the green hull. The suctioning end effectuator is the blue object in the upper left.
Mission Critical
•
Spring 2012
33
where most of the nuclear contamination
But the GPS-based systems for auto-steering
an after-market product that farmers can
is concentrated. Radiation levels inside
on the market now are expensive and not
add to their existing tractors.
the sealed cabin of a control car would be
widely accepted by Japan’s farmers.
about half those in the field, he says.
Hanawa’s lower tech approach involves
The strength of youth
setting up a target lamp at the end of a row.
Some Japanese engineers are pursuing a
Cameras mounted on the tractor detect the
more direct approach for Japan’s aged
target, and the automatic steering control
farmers with devices that will help them
points the tractor toward it. As the tractor
regain the strength of youth. Inventors at
advances, the cameras also take a series
both the Institute of Agricultural Machinery
of pictures of small features on the ground.
and the Tokyo University of Agriculture and
The holy grail of agricultural robotics in
The system compares those points to correct
Technology have devised separate robotic
Japan would be a small, accurate autono-
the tractor’s lateral movement. With this sys-
suits to help farmers with the physical de-
mous tractor that’s commercially viable and
tem, a tractor can drive 100 yards without
mands of farming — carrying heavy ob-
simple to operate. Scores of companies
deviating so much as two inches, he says.
jects, crouching to weed or standing for
Is that safe enough? “We have to do this [remediation work],” says the institute’s spokeswoman, Momoko Fujii. “It may not be perfectly safe, but we have to do it.”
and research facilities here have tried to produce one, with mixed success.
One limitation: The maximum distance is 330 yards, which is just enough for most
Meanwhile, Hanawa is perfecting a self-
Japanese farms and probably laughable to
steering tractor that does not rely on ex-
an American farmer, he acknowledges.
pensive GPS systems. Driving a perfectly straight line — for seeding, ridge forming and creating field boundaries — is key to farming efficiency, especially on small plots.
But, he says, if you use two target lamps and move them alternately, you could maintain a straight line for as long as you like. He hopes to eventually turn this system into
hours with arms held high to prune, for example. The TUAT version, which Prof. Shigeki Toyama has been working on for 15 years, looks like something out of the movie “Iron Man.” It is a motorized exoskeleton that is strapped to the body. It employs eight electric motors to give more power to the user’s arm and leg muscles, and it has sensors to detect movement. It can be operated by voice command. One version, according to news agency Agence France-Presse, weighs 30 kilograms and is designed to help farmers pull radishes and other root vegetables. The lighter version weighs 23 kilograms and is intended for harvesting grapes. The inventors hope to bring the price down to about $6,000 once it’s in mass production. But will it appeal to farmers? That’s a potential barrier to deploying of any kind of agribot in Japan: The farmers they are trying to help are elderly, a population that has been slow to adopt new technologies. Few Japanese people over 65, a government survey shows, even use email. Liz Ruskin is a freelance journalist living in Tokyo. She has worked as a newspaper journalist for 14 years — as a Washington correspondent for McClatchy Newspapers and, before that, as a reporter for the Anchorage Daily News. She has also worked in radio.
34
Mission Critical
•
Spring 2012
How green is my valley? Hard to tell with flight restrictions UNCANNY VALLEY
P
lants may grow from the ground up, but the view of an orange grove from the top down also provides valuable
information, especially for precision agriculture.
farming,” says Ehsani.
Ehsani hopes to use aerial imaging to show
His project in particular is reviewing the health of Florida produce in the wake of a disease called citrus greening. The bacterial disease, transmitted from plant to plant
early onset of the disease. Though aerial imaging with manned aircraft has been around for decades, Ehsani sees unmanned systems fitting in as another method.
A farm concept that observes and responds
via insects called psyllids, can be difficult to
“If we can monitor the overall health of the
to variations in a field of crops, this agri-
detect, and trees can remain symptomless
tree, and that can be done with a multispec-
cultural management style is Reza Ehsani’s
for years before the infection becomes visu-
tral image, we can from the multispectral
specialty. An associate professor at the Cit-
ally evident. Once visible, it can kill a tree
aerial image identify which trees start to
rus Research and Education Center at the
very quickly.
decline for any reason,” he says. “And we
University of Florida, Ehsani set out to find an alternative, cheap method of assessing
“Greening is … a dangerous concern right
can go look at that particular tree.”
now in Florida, because it’s a disease that’s
The traditional method for identifying citrus
widespread in the state and there is no cure
greening is to have human operators walk
“People use satellite images and aerial im-
for it,” he says. “So to manage it, there are
or drive through each row in a grove and
ages to monitor crop health, crop disease,
two ways. One is identifying the infected
look for symptoms. When they see it, they
and also they can try to predict yield, so
trees and then removing them or try to
mark it and take a sample for later confir-
there’s a lot of different applications that
spray a lot of chemicals to kill the insect
mation. Once identified, then they’d go
aerial imaging can provide to farmers and
that transmits the bacteria.”
back and remove the tree.
crop health from the top down.
A quadrotor with a multispectral imaging sensor. Prof. Reza Ehsani from the University of Florida hopes to use this method to detect a citrus disease ravaging groves in Florida. Photos courtesy Reza Ehsani.
36
Mission Critical
•
Spring 2012
“This process is recommended if your infec-
with it, you need to get some authorization
you go through the whole process of get-
tion rate is less than 1 percent,” he says.
through the FAA, and that’s a very lengthy
ting certified you cannot really use it for re-
Using this human method, there is also a
process.”
search, so that’s kind of a disappointment,”
40 percent error rate, where citrus greening goes undetected. His idea is to use a quadrotor with a multispectral camera to seek out the greening and then send in people to monitor those trees. “It can do targeted scouting, so you
The recent ruling that the FAA must provide
he says.
airspace for small UAS by 2015 leaves
Another issue with nearly all farm equip-
Ehsani playing the waiting game.
ment is it has to be economical.
“We surely thought that we were exempt as
“The thing is that other people tried to use
a hobbyist, but it turns out that, really, if I
this type of low-cost platforms … but most of the previous attempts were
send the people to specific
not too successful, because
trees … and check if that
we know that to operate
tree has a sign of symp-
them you have to be a very
toms.”
good RC pilot. … And the
A friend turned Ehsani onto
thing is that if you make a
unmanned systems as a
small mistake you crash. And
hobby two years ago, and
if you crash you not only lose
it occurred to him the tech-
the airplane, but you lose a
nology would translate well
lot of expensive sensors that
to his research.
you have on it like a camera.”
“I looked at it and I thought, ‘Man, this is very easy to
The cost of losing an un-
operate.’ Before that I didn’t
manned asset, though, is
have any background in RC
A resolution comparison between a satellite image of a grove versus a quadcopter.
operating. I’d never even tried it; it sounded difficult.” Using
targeted
small compared to using manned aircraft. The cost of aerial imaging
scouting
with manned platforms re-
would likely improve the
quires a minimum amount of
productivity and likely the cost of hiring
go and do it as a hobby on a weekend it’s
area to cover. Ehsani estimates the current
scouts, which is about $100 per acre, says
okay, but if I try to use it really for research
minimum amount is around 10 acres and
Ehsani.
to collect data, I cannot. That’s why we are
the starting cost would be around $5,000
not flying right now.”
or more, depending on how close the grove
All of this is fine, in theory, but the project hit a major hurdle once Ehsani, who is
From the about five hours of flying he did
fairly new to using unmanned systems tech-
perform, Ehsani was pleased with the re-
nology, realized the Federal Aviation Ad-
sults.
ministration largely limits where and when unmanned systems can fly. “About two years ago we came across … these multi-rotor copters that are commercially available,” says Ehsani. “This is something that’s been used by hobbyists for a while. We said, well, if we could use this, that would be great.”
“Our initial data suggests that it’s very, very useful and it could be a very useful tool for growers to use,” says Ehsani.
is to an airport. These flights would have to be continuous, as often as a couple times a month. He doesn’t believe unmanned vehicles will replace manned vehicles in these large fields, but if you have a smaller grove, “then these systems could be very, very at-
Right now, commercial use is lacking, but
tractive tools, because now you can start
Ehsani is hoping to show this type of tech-
taking advantage of the benefits of aerial
nology isn’t just for applications that have
imaging at a very affordable rate.”
caused unmanned systems a PR problem, like use by paparazzi. People can benefit
After a handful of flights, Ehsani stopped
from the systems through research like his,
using the UAV because of this barrier.
he says.
“We collected some initial data, but then
“I guess everybody else is in our shoes in
we learned that, really, to use this as a hob-
terms of they will see a lot of potential for
by is okay, but if you try to do any research
this type of platform, but right now unless
Aerial imaging with manned vehicles has been in use since the 1970s, says Ehsani. “But if you go and talk to growers, you will see that they are not using this technology. Very, very few growers that I know are using these technologies,” says Ehsani.
CMU-led automation program puts robots in the field TESTING, TESTING
By Brett Davis
W
hen researchers at Carnegie Mellon
culture] extension people,” says Marcel
over and over again. There are some sta-
University wanted to evaluate the
Bergerman, the CASC project manager
tistics, I don’t know the exact numbers, but
usefulness of their ground robot for
and a systems scientist with Carnegie Mel-
about 30 percent of accidents in orchards
orchard growers, they did something radi-
lon’s Field Robotics Center. “Not to engi-
can be somehow related back to ladders,”
cal — they put it in the hands of actual or-
neers, we gave them to the extension peo-
says Bergerman.
chard growers.
ple, and they take these machines to actual
The team leaders of the Comprehensive Automation for Specialty Crops (CASC) program have developed a machine, based
orchards, commercial orchards with growers, and they go in and do the experiments themselves.”
To avoid that, CASC developed an automated version of Toro’s E-Workman platform, named the Autonomous Prime Mover. They delivered APM systems to agriculture
on a commercial Toro platform, that can
Previously, orchard workers would take a
extension programs and growers last year,
autonomously trundle down orchard rows,
ladder, climb it to perform a task, climb
keeping one in-house for system modifica-
allowing human workers atop it to prune,
back down and move the ladder.
tions and testing. The APM has a scissor
thin fruit, train and tie trees, or harvest.
“Now this is driving slowly down the row,
“For the first time in my career, we devel-
and instead of literally going up the ladder,
oped robots … and we gave them to Penn
down the ladder, and moving the ladder,
State and Washington State, to the [agri-
lift on top of it that can carry two workers, who can control the vehicle if they want but who no longer have to climb up and down ladders. The APM vehicle in “bin dog” mode. Photo courtesy CASC.
38
Mission Critical
•
Spring 2012
“It’s an interesting case. It’s autonomous
100%
but manned,” says Sanjiv Singh, the proj-
90%
CASC has developed two generations of the machines, which have grown a little bit simpler over time. The first version had an iPad interface, but “iPads aren’t meant for readability in sunshine” and they can heat up, even in the winter, Singh says. “We’ve gone back and done something a little bit lower tech but more sturdy, and it will work in any lighting condition,” he says. The interface now sports an E-ink panel, like a Kindle e-book reader, and has more discrete buttons, a slider bar for speed control and a foot pedal.
Increase in worker produc0vity
ect director and a research professor at the Robotics Institute.
Increased Efficiency with self-‐guided pla;orm compared to Ladder
80% 70% 58%
60% 50%
52%
50%
50%
46% 41%
41%
40%
34%
33% 30% 20%
20% 10% 0%
Tying 1-‐yr-‐old apple trees
Applying wire clamps, pinching 2-‐yr-‐old apple trees
Tying 3-‐yr-‐old apple trees
Tying, pinching, thinning 3-‐yr-‐old apple trees
Tying, pinching 3-‐yr-‐old apple trees
Thinning 5-‐yr-‐old peaches
Thinning 3-‐yr-‐old peaches
Applying wire clamps, pinching 2-‐yr-‐old apple trees
Hanging drier sheets for sDnk bug control
PalmeIe branch tying
The results of a timed trial between the APM and ladder-using workers.
“You put it at the beginning of the row, flip the switch, and … you press the foot pedal and the vehicle starts creeping in the row,” Singh says. “When it gets to the end of the row, it detects the open space with a laser and stops automatically. The workers atop the scissors lift use a joystick to drive it to the next row. … It’s an autonomous vehicle that does one thing only.” That one-trick pony has a good trick, however. CASC loaned the vehicles to the agriculture extension programs at the partner universities, which in turn lent them to orchards. In one case, the extension program also set up a John Henry-style competition
decisions. Mobility means a vehicle, such
them more productive, more efficient and
as the APM, that allows autonomous mow-
more safe.”
ing and spraying. Manipulation means higher level functions such as pruning, thinning and harvesting.
CASC is funded at $6 million by the U.S. Department of Agriculture Specialty Crop Research Initiative, established by the
“Let’s jump forward into the future — 10,
2008 farm bill, with matching funds from
20, 30 years. A robot will go out and do
industry and university partners. Those part-
the pruning and the thinning and the har-
ners, in addition to Penn State and Wash-
vesting for you,” Bergerman says. “That’s
ington State, include Oregon State, Purdue,
not our focus right now. The focus is how
the USDA Agricultural Research Service
do you augment human workers to make
Appalachian Fruit Research Station, Vision
between the APM and a man on a ladder. The video, which was shown at AUVSI’s Unmanned Systems Program Review 2012, showed that the automated platform is capable of making much better time.
CASC The overarching philosophy of the CASC program is to go “from information management to mobility to manipulation,” Bergerman says. Information is data about crop health and yield, using such things as soil and moisture sensors, automated calipers and counters, and fruit counting and sizing systems, to allow farmers to make better management
CASC leaders Sanjiv Singh and Marcel Bergerman at the Mid-Atlantic Fruit and Vegetable Convention. AUVSI photo.
Mission Critical
•
Spring 2012
39
Two workers thin green fruit at Allan Bros. Orchards in Prosser, Wash. In timed trials in Pennsylvania comparing workers on the vehicle with workers on ladders, CASC obtained up to a 58 percent efficiency increase. Photo courtesy CASC.
Robotics, Toro, DBR Conveyor Concepts,
discuss how farmers might use it and how
Spensa Technologies and Trimble.
which leads to irregularity at the ends of
they feel about it.
the rows.
Its initiatives include developing an auto-
In the first year, Singh says, the planned
mated bug trap that can trap and count
“Somebody asked me about that today
equipment tended to be expensive and the
specific types of insects, an automated tree
when I was giving my talk: ‘I don’t believe
reaction was “pretty muted.” Now farm-
caliper and counter, a harvest augmenta-
your machines are going to be able to deal
ers and growers “have started to get a
tion system, and a scout vehicle that as-
with the ends of rows.’ I said you don’t want
little more excited about it … because they
sesses crop load.
us to deal with that, because you wouldn’t
can actually see connection to their work,”
be able to justify the extra cost associated
Singh says.
with that. You’d be better off making your
The program has included the work of 69 students in its three-year life, including sum-
Price is a huge consideration in the agricul-
mer students and those who have worked
ture market, he says.
with it for its whole existence.
“If you suggest a $5,000 user interface for
“One of the reasons the program was cre-
a military system, it’s not a big deal. No
ated, besides creating new technology, is
one even asks about that. You couldn’t have
to educate a new generation of ag engi-
a $5,000 control interface for an ag robot;
neers,” says Bergerman. “That’s been miss-
it just would not compute,” he says.
ing in the U.S. for a long time now.”
That also means that the machines can’t
End users
be made as smart as the orchard grow-
Singh and Bergerman attend events such as
have said that the machines couldn’t work
the recent Mid-Atlantic Fruit and Vegetable Convention in Hershey, Pa., where they spoke with Mission Critical. They use such events to demonstrate their technology and
40
Mission Critical
•
Spring 2012
rows slightly shorter and giving the vehicle a little bit more room to turn around in the flat area.” In other words, “If you’re going to automate orchards, you’re going to have to build them to suit,” he says. “This is kind of the educational process. … We build these machines to spur people to ask questions.” Brett Davis is editor of Mission Critical.
ers might like. Singh says some growers for them because they’re not smart enough to negotiate the ends of tree rows, where
For More Information:
the ground may slope up or down. Grow-
http://www.cascrop.com
ers tend to use as much land as they can,
Future ag robots need better grips, better planning, lower price tags TECHNOLOGY GAP
F
or robotics and unmanned systems to
ments,” the report says. Members of the
make serious inroads into the agricul-
working groups that wrote the report
tural market, toughness, ease of use
“identified novel robotic hands, tactile
and affordability may be the keys. Systems to be used in the field in agriculture must be tough and able to endure rough weather, like their military counterparts, but
sensing and highly accurate, physically realistic simulators as important enablers for autonomous manipulation.” • Planning: Robust, dynamic 3-D path
the farmers who will ultimately use them
planning “remains an open problem.
generally don’t have the same deep pock-
An important aspect of this problem
ets as the Pentagon.
is the notion of a robot’s situational
“In the agriculture world, we have this other kind of constraint, because you have some of the characteristics that you have in the military world, but the tolerance for price is much lower,” says Sanjay Singh of Carnegie Mellon University, the project director
awareness, i.e., the robot’s ability to autonomously combine, interleave, and integrate the planning of actions with appropriate sensing and modeling of the environment.” • Sensing and perception: “Participants
of the multi-partner Comprehensive Auto-
believed that new sensing modalities as
mation for Specialty Crops program (for
well as more advanced, higher resolu-
more on that, see Page 38).
tion, lower cost versions of existing
An added complication is that most agricultural environments are relatively unstruc-
modalities would be areas of important progress,” the report says.
tured, not strictly regulated like a factory
• Mechanical hardware: Safety is key,
floor, and exist in an environment of inex-
and “participants therefore felt that
pensive labor.
inherently safer motors and mechanisms with increased strength-to-weight ratio
Roadmap needs In 2009, more than 140 robotics experts from industry, laboratories and academia teamed to produce a roadmap for the industry, entitled “A Roadmap for U.S Robotics — From Internet to Robotics.”
would represent an important enabling technology.” This area of research also
The Department of Agriculture has several requirements in mind for systems it may fund. It is seeking to back robotics research that will “enhance food production, processing and distribution,” and wants: • Automated systems for inspecting, sorting, processing and handling animal or plant products after they have been harvested or created • Systems for inspecting, sorting and handling plants and flowers in greenhouse or nurseries • Systems for vaccinating and deworming large numbers of live animals • Rapid-sensing systems for detecting defects, ripeness, physical damage, microbial contamination, size, shape and other attributes of plant and animal products • Systems for monitoring air and water quality.
Getting a grip Not all of this new technology needs to be very precise, however, especially for some agricultural use.
includes energy-efficient systems, as
In a paper published in 2006, researchers
robots will have to operate off-tether
from Turin University in Italy noted that agri-
for extended periods, and improved
cultural robots don’t need the precise move-
means of locomotion beyond wheels, as
ments required of factory robots and can
“outdoor environments oftentimes exhibit
They noted that in the field of service ro-
get by with movement errors in the range of
highly variable terrain properties” while
botics, which includes agriculture, improve-
a few millimeters. Such a level of precision
indoor ones may have stairs, ladders,
ments in several key areas would be re-
might lead to a really badly built automo-
ramps, escalators or elevators.
bile, but would be perfectly acceptable for
quired, including in the areas of:
picking fruits and berries and would allow
NRI
robots to be built for much less money.
manipulation systems function well
The National Robotics Initiative, announced
in carefully engineered and highly
last year, builds on the roadmap to acceler-
They also concluded that agricultural ro-
controlled environments, such as factory
ate the development and use of robotics in
floors and assembly cells, but cannot
the United States. One of the government
handle the environmental variability
agencies backing the program, which will
and uncertainty associated with open,
fund promising research, is the U.S. Depart-
dynamic and unstructured environ-
ment of Agriculture.
• Manipulation: “Currently, autonomous
bots might first find a home in greenhouses, which can be built to be much more regular than outdoor farmland. Greenhouse plants can be set up in an orderly way, and the greenhouses themselves could be built to allow robots to move up and down the rows.
Mission Critical
•
Spring 2012
41
Against the grain
In an industry sidelined by safety laws, could Bin Bot dig farmers out of a grain bin conundrum? By Danielle Lucey
F
arm work may sound like it’s only dull
In 2010, 51 workers were engulfed and 26
his own, and, like quicksand, the grain en-
and dirty. But just how dangerous it can
were killed working in grain bins, accord-
gulfs a person in seconds, or it can fall on
actually be inspired one company to try
ing to a study by Purdue University. This
top of the worker, like an avalanche.
and put a stop to the rising trend of deaths
number is the highest on record and nearly
associated with grain bin accidents.
twice the prior year’s accident number.
Jerome Mack, now president of Leola, S.D.-
Workers die one of a few ways. Many grain
occur, are often catastrophic. According to
based Mack Robotics, lost his best friend in
bins, similar to silos but much smaller, em-
the U.S. Occupational Safety and Health
one of these accidents in 1998. At the time,
ploy a sweeping arm to push along grain
Administration, there have been more than
Mack ran a company that used robotics to
that isn’t emptied from the bin by gravity. If
500 grain bin explosions in the last 35
enable easier swine reproduction through
the arm gets caught, a worker may enter
years, resulting in 677 injuries and 184
artificial insemination. But inspired to keep
the bin and try to push or kick it along. If
fatalities. Workers can also perish falling
the number of grain bin accidents down, he
his clothes get stuck on the arm, it can tear
from tall structures in a grain handling fa-
refocused and formed his own grain robot-
off limbs or rip apart his body.
cility.
Suffocation is the main cause of death
Mack Robotics’ goal is simple: to get the
ics company.
Due to the large amount of dust in a grain bin, explosions are possible and, if they
when a worker attempts to move grain on
The new, explosion-proof version of Bin Bot. All photos courtesy Mack Robotics.
The Bin Bot lowering the sweep after moving it to its desired position.
42
Mission Critical
•
Spring 2012
number of grain bin deaths down to zero. The company formed one year ago, and given a change in OSHA policy, its timing might prove impeccable.
The robot is 100 percent teleoperated by
demonstrations to allow farmers the chance
one person through wireless communica-
to see if Bin Bot will work at their location.
tions and cameras with lighting supplying a video feed. This is a decrease in the number of people it typically takes to work a
Recognizing the dangerous reality the
grain bin, which is between three and four.
United States’ approximately 13,000 grain
The robots currently cost between $15,000
elevator operators face, OSHA has a zero-
and $17,500 depending on attachment
entry policy with grain bins and heavily
options.
fines employers that allow humans to enter. The problem? Currently, human entry is the only way to ensure grain bins are thorough-
The company is also working on an updated version, one that’s explosion-proof, useful given the widespread grain bin explosion issue and that it’s an industry standard for much farm equipment. Though Outtrim says cannot speculate the
Attachment options include a scoop, which is useful for typical housekeeping and
cost savings involved using Bin Bot, that’s not Mack Robotics’ goal.
helps clean up the grain and push it out
“Our ultimate goal is to reduce the number
of the bin. The push attachment can move
of injuries and ultimately deaths that are oc-
In the market of robotic grain cleaners, Kris-
more product than the scoop. Finally there
curring. No matter how big or small [the
tin Outtrim, marketing manager for Mack
is a forklift, which is helpful when a grain
operation], it’s going to be an asset to that
Robotics, says she’s yet to see a similar
sweeper is acting up. It can apply pressure
company because of that effect.”
technology at the trade shows she attends.
to it, and if the grain avalanches, it can lift
ly cleaned and emptied.
So what are farmers to do? “That is a very, very good question, because that is what these elevator managers and farmers and all these people are
the sweep up. Though the company is still in a promotional phase with Bin Bot, it, in addition
Danielle Lucey is managing editor of Mission Critical.
to trade shows, has begun setting up live
asking. … What are we supposed to do?” says Outtrim. “For one thing that’s how they’ve always done it, and for another thing nobody’s offered a solution. A lot of the comments we get at the shows are ‘We need something like this because they’re telling us we can’t do this, but nobody is providing a solution.’” Bin Bot has been in a prototype phase since November 2011. In addition to the inspiration garnered from the loss of his friend, a farmer approached Mack interested in
The Bin Bot does some housekeeping inside a grain bin.
turning his hog solution into a grain bin solution. Though Mack couldn’t turn the one machine into a multifunctional system, it inspired him to create the brand new Bin Bot. Using his previous experience, Mack worked with a team of engineers to make the Bin Bot. One of the requirements was to make it powerful but small, able to fit through a 24-inch-diameter opening, the smallest bin door Mack has seen. The machine is battery operated, with a 12hour life. The machine is about 6-by-2 feet and weighs 800 pounds.
SCAN IT
or
Click IT:
Click the photo or scan this barcode with your smartphone to see Bin Bot work alongside a sweep
Mission Critical
•
Spring 2012
43
Flying where the green grass grows END USERS
K
eenan Amundsen is a University of
It is also superior to the current industry
Nebraska-Lincoln
profes-
practice of driving cameras over the turf by
sor of turfgrass genetics who is using
mounting them to golf carts and then piec-
assistant
some new technology to make his job more efficient.
ing together the resulting images.
mant longer than cool-season turf species. Pixobot’s Singh says the helicopter used in the initial tests employed a Canon SD750 point-and-shoot, but the spring iteration will
“With the helicopter, we can just hover
use a Sony Nex 5N, which features a larg-
As a turfgrass breeder, he studies the turf to
over the area. We get a live feed through
er sensor and better resolution.
predict areas prone to stress, which allows
the digital camera, so we can orient the
for site-specific management. He needs to
helicopter the way we want and take the
evaluate his green charges on a variety of
pictures,” Amundsen says. “I can sit at my
visual traits, including shoot density, color
desk and drink my coffee in the air condi-
and uniformity. With 40 acres of small
tioning while the computer is analyzing the
“You have to go through the plots sever-
grass plots to evaluate, it can be a time-
images, and I don’t have to get out there for
al times, and that just adds to the time it
consuming job.
hours at a time.”
takes,” Amundsen says. “With one picture,
“Walking through the plots takes several
The Pixobot will be used this spring to study
days. If we can speed up the process by
the timing of spring green-up of buffalo
taking an image and then analyzing the im-
grass, a warm-season turf, which stays dor-
The plots are also usually rated for one variable at a time, meaning multiple passes are required for a walking evaluation.
you can analyze that picture and pull out all that data from the one image. It greatly improves the efficiency of the whole process.”
age computationally, it dramatically accelerates the whole process,” he tells Mission Critical. He found a way to do just that at a chance meeting last fall with Vishal Singh, a multimedia design specialist for UNL’s edMedia, who also happens to own a company called Pixobot, which markets a small helicopter for aerial surveilling and other uses. “We got to talking about his interests and my interests, and it was just a natural fit,” Amundsen says. They did some testing with the vehicle last year, mounting an off-the-shelf digital camera to the helicopter to photograph the turf from the air. “The idea is to mount it with a digital camera so that we can hover over our turf areas, taking several static images. Then we bring the pictures back to the office and do the computational analysis to start rating the different plots,” Amundsen says. The Pixobot sped up the process dramatically, and is a better alternative to fixedwing aircraft, which must make multiple passes, and blimps, which are more prone to wind drift. 44
Mission Critical
•
Spring 2012
Vishal Singh flies the Pixobot. Photo courtesy Keenan Amundsen.
REACH THE
UNMANNED SYSTEMS AND
ROBOTICS COMMUNITIES Capture the attention of the most influential leaders on a daily, weekly and monthly basis. Advertise in Unmanned Systems magazine, unmanned systems:mission critical, Unmanned Systems eBrief or AUVSI.org.
For more information call +1 571 225 7779 or email marketing@auvsi.org