Page 1

3D plant analysis over time: understanding plant architecture and growth (and function) using imaging technologies Dr Xavier Sirault Research Team Leader – Imaging and image analysis Engineer Scientist in Plant Phenomics High Resolution Plant Phenomics Centre, CSIRO Plant Industry, Canberra, ACT, Australia


Research context International C4 rice consortium International Rice Research Institute Bill and Melinda Gates foundation funding

Wheat Yield Consortium International wheat and maize improvement Centre (CIMMYT), Mexico

Global Rice Science Partnership (GRiSP) International Rice Research Institute Consultative Group on International Agricultural Research funding (PhD student Ms Katherine Meacham)


nt rs

lers

Photosynthesis and food production (1/2) To date crops have been adapted to - allocate the maximum amount of biomass to grain (wheat and rice) - fully exploit the interception of sunlight (to a certain extent)

Current Yield =cul HI tivars Biomass *

Current culti vars + tin Biomass = I Abs  RUE ** Light-use efficiency  Light interception

Current culti vars + tin

Rate of canopy closure, architecture and canopy size • Leaf area development rate (leaf size, rate of leaf emergence) • Canopy structure (leaf angle, leaf branching, curvature, azimuthal and leaf area distribution) • Leaf Area Duration Fewer wasteful tillers Larger ears *Richards (2000) J. Exp. Bot.: 51: 447–458 grains ** Hay and Walker (1989)Larger An introduction to the physiology of crop yield

Combined photosynthetic rate of leaves in the canopy corrected for respiratory losses • ФPSII, ETR, qP, NPQ,…

Fewer wasteful t illers Larger ears Larger grains

Wastef ul ti llers


Photosynthesis and food production (2/2) What is the maximum efficiency that photosynthesis can convert solar energy into biomass?

highest ξmax achieved in non-limiting conditions 2.4 % (C3) & 3.7% (C4) Insufficient/ineffective capacity to utilise all radiation incident on a leaf Opportunities Canopy architecture and/or photosynthetic capacity per unit leaf area + Photosynthetic efficiency Source: Zhu et al. 2008 Current Opinion in Biotechnology 19:153–159 Long et al 2006, PCE 29:315-330 Zhu et al 2007 Plant Phys 145:513-526


Breeding for RUE = understanding its plasticity and its genetics Assaying accurately each genotype (from a large population of genetic variants) for a large number of traits simultaneously:

‘Phenotype‘ hyper-surface ‘Phenotype‘ surface response response

RUE -

Canopy architecture (3D reconstruction and metrics) Rate of canopy establishment and LAI development (organ tracking in time) N and pigment distribution (hyperspectral imaging) Max assimilation rate, ETR (fluorescence imaging) Leaf senescence (fluorescence and/or visible imaging) Conductance patchiness (IR imaging) …

Adapted from A Gallais “Theorie de la selection en amelioration des plantes” ed Masson


n-dimensional imaging platform for crop plant An integrated multi-sensing tools to quantify the biological processes involved in the development of plants

Dataset for each plant is 3-dimensional and multi-spectral (~50s per plant for 1,000,000 vertices) - one image every 3deg “Process control� – environment inside imaging station is monitored continuously


3D imaging technologies for crop plant Methods: • Stereo-techniques: silhouette and texture and Point cloud (LiDAR) • Distortion correction and metric calibration • Spectral distortion correction          

Rice (Oryza sativa) Cotton (Gossypium species) Corn (Zea mays) Canola (Brassica rapa) Tobacco (Nicotiana tabacum) Millet (Setaria species) Eucalyptus species Wheat (Triticum aestivum) Brachypodium distachyon Tomato (Solanum lycopersicum)


An automated software solution to analyse structure and function in 3D

Segmentation of organs using a range of computer vision techniques

(Under Provisional Patent)

Paproki et al. (2012) BMC Plant Biology 12:63


Automated features extraction and quantification (1/2)

Metric measurements: Height, Internode length, Peduncle angles, Surface orientation in space, Stem diameters, Branching structure, Flowering structure‌ (Under Provisional Patent)

Paproki et al. (2012) BMC Plant Biology 12:63


Automated features extraction and quantification (2/2)

Plant specific metrics • Total leaf area, • Leaf area density • Plant height • 3D-shape descriptors (e.g. Symmetry, Convex hull)

Organ specific metrics • Leaf area • Leaf dimension • Symmetry • Shape factors • M-Rep and B-Rep Paproki et al. (2012) BMC Plant Biology


Automated features extraction and quantification: validation squared Pearson correlation R2w=0.957, R2l=0.948 , R2s=0.887 Intra-class correlation (consistency amongst observers) ICCw≃0.974, ICCl≃0.967, ICCs≃0.941,

Paproki et al. (2012) BMC Plant Biology 12:63


Dynamic data: automatic tracking of organs for longitudinal analysis

(Under Provisional Patent)

Paproki et al. (2012) BMC Plant Biology 12:63


An automated software solution to analyse structure and function in 3D Data workflow and 3D model analysis pipeline

Data acquisition

3D reconstruction

Automated segmentation

Data extraction

Paproki et al. (2012) BMC Plant Biology 12:63


Generalisation of the pipeline to cereals (1/2)

Spline fitting / margin extraction

Corn segmentation (Zea mays)

Leaf width distribution Sirault et al. unpublished


Generalisation of the pipeline to cereals (1/2) Azucena

Volume (mm3) Oryza sativa IR64 Azucena Moroberrekan

Nippon Bare

Nippon Bare

July 20th

August 10th

(Meacham et al. PhD work, unpublished)


3D data fusion of functional information

FIR to RGB mapping

240 FIR images from 2 angles (calibration 10mK - Measurement Standards Laboratory NZ)

Automated Far-infrared overlay on 3D structure (Zea mays) Guo and Sirault unpublished


Testing and integration of new imaging technologies Laser induced fluorescent transient (LIFT) (calculate suite of photosynthetic characteristics)

Jimenez, Furbank and Sirault (unpublished)

Kolber, Osmond, Sirault and Furbank

Hyperspectral imaging (350nm to 1100nm): • optical leaf properties ( e.g. transmittance, reflectance,) • biochemical properties (i.e. [N] and [pigment] & distribution) (e.g. PROSPECT modelling)


Measuring photosynthetic parameters using LIFT 360 degree scan of Fv/Fm, PQ pool and Pmax acquired at 9 elevation levels in a wheat plant within PlantScan acquisition system

systematic patterns in vertical distribution of the fluorescence characteristics Kolber, Osmond, Sirault and Furbank (unpublished)


Going further: understanding leaf morphology using computer vision Application of atlasing to Gossipium hirsutum species: Okra-leafed Siokra x normal-leafed MCU-5 F6 RILs population

Sirault et al. (unpublished)


Understanding leaf morphology using computer vision Registration

• • •

Rigid transform alignment (Centroid matching, Principal axis alignment, …) Non Rigid registration for correspondence registration (EM-ICP…) Smoothing

Generated statistical shape model or leaf ATLAS

Use in “Process control” for 3D reconstruction Paproki et al. (submitted)


Understanding leaf morphology using computer vision

Let mother nature tell the story! Paproki et al. (submitted)


Relevance of these traits in the field

Up to 1 million 10m2 wheat plots are planted each year in Australia for evaluation of genotypic performance in target environments THIS IS HIGH THROUGHPUT IN REAL CONDITIONS.!

Mostly, only final yield is recorded by breeders (Photo credit: Michelle Watt)


High Resolution field phenomics platforms

Accurate phenotyping (High spatial resolution)

Increasing throughput

Understanding the production environment (High temporal resolution)


Field-based phenomics: software pipeline development (1/3) (Jimenez et al)

(Deery et al)


Field-based phenomics: software pipeline development (2/3) Measuring biomass in high throughput in the field Brassica napus

Volume (disparity map)

1000 plots between 10-00 and 14-00h

Panicum miliaceum

Biomass (g/0.5m2) Multiple modalities

Tan et al. - unpublished


Field-based phenomics: software pipeline development (3/3) Counting ears

Jimenez et al. - unpublished


Concluding remarks • These tools and their associated analysis pipelines are starting to provide data for developing: 1. 3D dynamic, phenotypic and biochemical models of plant growth 2. 3D dynamic model of the response of genotypes to variable environments • Associated with mathematical/mechanistic modelling (radiative & photosynthetic) and virtual laboratory environments, it provides a way forward to up-scaling single plant response to canopy level using simulation 3D architectural model coupled with a radiative model to quantify the contribution of specific responses of plant architecture in terms of light intercepted by the crop

Source: OpenAlea

• Canopy level phenotyping occurs in parallel to validate these mathematical models


Thank you CREDITS Analysis pipeline – “Automated Phenomics Team” Jurgen Fripp, Ron Li, Anthony Paproki, CSIRO ICT Changming Sun, David Lovell, CMIS and TCP TB Chuong Nguyen ,(CMIS OCE post doc) Jianming Guo, CSIRO PI Xiao Tan (PhD student) HRPPC Team Robert Furbank, Jose Berni-Jimenez, David Deery, Helen Daily, (Scott Berry), Katherine Meacham, CSIRO PI Mechanical engineering Nick Binos, Peter Kuffner, Chris Swain Automation engineering Peter Mortimore, Ralph Dick, NVSI Electrical engineering: Paul Macarounas

3D analysis over time: understanding plant architecture and grwoth using image technologies  

PDF of the talk at the PhenoDays 2012 in Wageningen, NL