The Bleeding Edge: Bots and Rots

A new generation of farmers are reevaluating their industry’s relationship with nature and the value of ecological biodiversity. According to the Pesticide Action Network North America (PANNA), pesticide usage wipes out soil and plant biodiversity, which decreases crop resilience, decreases pesticide efficacy, and increases pest pressure, triggering even more pesticide applications. Pesticide use begets more pesticide use.

So why don’t humans just drink wines made from grapes that don’t spoil so easily? Unfortunately, these hybrids have been deemed as “less than” because they tend to be overwhelmingly aromatic, or “foxy”, rather than complex. This is where breeding comes into play as a solution, only now, it’s more advanced than monks pollinating crosses with paint brushes. Breeding for specific traits is not like genetic engineering where the genes of your choice are inserted into a host organism like a cassette tape. Breeding takes two to three decades of methodically crossing parent and progeny strains until the right characteristics are achieved. However, with some advanced genetic insights and rapid screening technology, this discovery time can be compressed. This is where project VitisGen comes in.

As stated on the VitisGen website, VitisGen projects, “are multi-disciplinary, collaborative projects focused on decreasing the time, effort and cost involved in developing the next generation of grapes. Incorporating cutting edge genomics technology and socioeconomic research into the traditional grape breeding process will speed up the ability to identify important genes related consumer-valued traits like disease resistance, low temperature tolerance and enhanced fruit quality. Identifying these genes will help grape breeding programs from around the world to more rapidly develop new grape varieties that will appeal to a wide range of consumers, while also addressing grower and producer needs.” VitisGen, a USDA program, “represents a new model of scientific collaboration. The integration of the needs of multiple interests, breeders, growers, fruit processors and consumers, into a single outcome will result in novel grape varieties that are beneficial to producers, processors, and consumers.”

This article discusses recent findings on how the third iteration of the VitisGen3 project leverages artificial intelligence and machine learning technology (AI) to speed up this process. The results in this paper are based on interviews from the scientists that published this recent work in the scientific journal, Frontiers in Plant Science titled, “Deep semantic segmentation for the quantification of grape foliar diseases in the vineyard”. Dr. Lance Cadle-Davidson is the USDA co-project director and Dr. Yu Jiang is an AI and machine learning specialist who provide commentary in the Q&A section below. According to Cadle-Davidson, “Our goal is to identify breeding lines that are more disease resistant and require less fungicide than our current varieties.”

Research Objective

This trial measures how an artificial intelligence (AI) system measures up to an expert human field scout at detecting and quantifying powdery or downy mildew risk in a vineyard. This information is used to select for disease resistant seeds to speed up the selective process in breeding programs. This technology may have broader commercial vineyard application in the future, but this is not currently the primary purpose for the technology.

infection of downy mildew with a side-by-side comparison of human and HMASS detection (interference). Yellow circles show the effects of poor lighting.

The AI system used a custom 20-megapixel stereo camera optic to take pictures and use them as a data source to pinpoint downy and powdery mildew on a grapevine canopy to direct breeding operations. Strobe lights were used to provide a consistent light source to minimize variability in factors such as shading, cloud cover or time of day. Images are processed using Hierarchical Multiscale Attention Semantic Segmentation (HMASS), in which algorithms

The

View daily updates on County-wide network maps help turn two-dimensional camera data into three dimensional results that could be correlated to the relevant disease, the magnitude of its presence and the GPS location of each frame (FIGURE 2)

In the field of deep learning, segmentation is a method that processes camera pictures in a way that groups pixels based on their similarity. Algorithms that use neural networks can be employed to split these picture groups into segments which correlate to usable information once they are “calibrated” using annotated images to train the network. Once the network is trained, they are validated to check that the performance is acceptable using a metric called Intersection over Union (mIoU).

This processed data is then used to compare AI results to that of a human collector and is represented in the color-filtered results shown in figure 3, below, which shows what the camera or human field scout sees (left), what the human field scout records as data (center) and what the HMASS model records (right).

Trial Description

Experimental vineyards were infected with downy and powdery mildew at an infection rate anywhere between 0 and 100 percent. This trial sought to improve the method of data collection using MHASS technology and to compare the difference between human and HMASS technology in determining the assessment mildew infection.

Conclusions

This trial demonstrated that the ground imaging system used for computer vision, coupled with HMASS processing technology—its “brain”—was a good tool to assess foliar disease in a high throughput manner. This will prove useful in fungicide trial evaluation, genetic mapping, and breeding programs. As reported by the journal’s authors, the results that support these findings are as follows:

• “Experimental results showed that the HMASS model achieved acceptable to good segmentation accuracy of downy mildew (mIoU > 0.84) and powdery mildew (mIoU > 0.74) infections in testing images, demonstrating the model capability for symptomatic disease segmentation.“ o mIoU is a metric that takes into account how AI models are performing versus a known standard. These algorithms review factors such as pixel accuracy, how overlapping leaves can be distinguished from each other and how the data is compared.

• “With the consistent image quality and multimodal metadata provided by the imaging system, the color filter and overlapping region removal could accurately and reliably segment grapevine canopies and identify repeatedly imaged regions between consecutive image frames, leading to critical information for infection severity calculation. “

• “Image-derived severity rates were highly correlated (r > 0.95) with human-assessed values and had comparable statistical power in differentiating fungicide treatment efficacy in both case studies.”

(FIGURE 4)

Based on the results above, the authors concluded that “the developed approach and pipeline can be used as an effective and efficient tool to quantify the severity of foliar disease infections, enabling objective, highthroughput disease evaluation for fungicide trial evaluation, genetic mapping, and breeding programs.”

Post-Mort Q&A with Lance

Cadle-Davidson and Yu Jiang

Are there any specific grape varieties that you are using for this experiment? If so which grapes?

Cadle-Davidson: The goal of this project is different than any standard vineyard project. We are looking at huge amounts of plant data from all types of wine grapes with collaborators from across the entire U.S. contributing data. While this helps us build a database of how powdery mildew manifests itself across many different environments across the country, this makes the challenge of finding the disease even greater. Our goal is to identify breeding lines that are more disease resistant and require less fungicide than our current varieties. This technology is being used in grape breeding programs across the U.S. which focus on the varieties relevant to their region. These are breeding programs supply new varieties for nurseries which, in turn, will propagate, distribute and sell them to growers.

What is the VitisGen project and how is it connected to robots?

Cadle-Davidson: VitisGen is a 15-year, $20 million project to build powdery mildew resistance across the country. Every public grape breeder at a federal institution is part of the project. We have a strong stakeholder in the National Grape Research Alliance.

For the last five years we have been imaging powdery mildew at a microscopic level. We have recently gained a 60-fold increase in the research that we have been able to accomplish. It’s really revolutionized our science by increasing the rate of screening new seedlings. We want to deliver a disease-resistant varieties. Now we are taking that robotic technology to the vineyard.

Who are all the groups working on this and what is the USDAARS’s role?

Cadle-Davidson: There are 25 researchers on the VitisGen project with about 30 collaborators in total, as well as an organization called Breeding Insight, a program that specializes in high tech breeding services which the USDA-ARS