Pre-XVIII CAVyT - Ravi Kiran

Decision-Support Technologies and their Impact on Road Asset Management Systems Ravi Kiran Co-Founder & CEO Lonrix Ltd, New Zealand

1

Contents 1. Some Impactful Technologies in the Last Decade

2. GIS & GPS Advances 3. Videos & Virtual Field Inspections 4. Machine Learning & Deep Learning Technologies (AI) 5. Lessons we Learned – RAMS Implementation

2

Some Impactful Technologies in the Last Decade 1. Internet of Things (IOT)

2. Improvements in Laser technology – Lidars and Data Capturing Tech 3. Digital Twins (VR & AR)

4. Expansion of Cloud Computing 5. Artificial Intelligence Models 6. Video Streaming Technology 7. Precision & Portability in GPS 8. Spatial Info on Demand – (Frequently updated Satellite Imagery) 9. Smartphones 3

Contents (Recap) 1. Some Impactful Technologies in the Last Decade

2. GIS & GPS Advances 3. Videos & Virtual Field Inspections 4. Machine Learning & Deep Learning Technologies (AI) 5. Lessons we Learned – RAMS Implementation

4

GIS & GPS Advances: 1. GPS/GNSS Sensors are part of a. Smartphones b. Sports Cameras c. Survey equipment (improved) 2. Improved Base Maps (Google, Esri, Open Street, etc..)

3. Availability of satellite imagery

5

GIS & GPS Advances (Use Cases): 1. GPS sensors in Smartphone to Geo-Tag information 2. GIS method to calculate area

6

GIS & GPS Advances (Smartphone App)

Replace paper based /XL forms with Smartphone apps.

& Geo-Tagging of Inspection Data

7

GIS & GPS Advances (Smartphone App)

Replace paper based /XL forms with Smartphone apps.

& Geo-Tagging of Inspection Data

8

GIS & GPS Advances (Smartphone App)

Replace paper based /XL forms with Smartphone apps.

& Geo-Tagging of Inspection Data

9

GIS & GPS Advances (Maps) Calculate Areas on a Map:

1 0

Contents (Recap) 1. Some Impactful Technologies in the Last Decade

2. GIS and it’s Advancements 3. Videos & Virtual Field Inspections 4. Machine Learning & Deep Learning Technologies (AI) 5. Lessons we Learned – RAMS Implementation

1 1

Manual Field Inspections

2000km Flies twice a month Joe: Asset Manager Network/ Contract

1. Drive the Network with the local team 2. Help them with the planning of a forward works program 3. Ensures that the asset is performing as per the contract 4. Make use of an asset management system to analyse the data and manage works programs

A Typical Scenario 12

Videos & Virtual Field Inspections Schedules a Virtual Drive over with the Local Team

Joe: Asset Manager

Records a video with GoPro

John: Local Team Member

Specialised Software – Connects the video to LRMS and uploads video to allow streaming from the cloud

Joe: Asset Manager

Cloud-Based Asset Management Software 13

Videos & Virtual Field Inspections 1. Video Source a. Survey Vehicle b. GPS Sensor enables Sports/Smartphone/360 Degree Camera

2. Cloud Based Asset Management System (Inventory & Condition Data) 3. Video Streaming Servers 4. Cloud-Based Software to Link all these Blocks

14

Videos & Virtual Field Inspections (In Action) Step 1: Process Recorded Video & Auto-Cloud Upload

15

Videos & Virtual Field Inspections (Example) Step 2: Virtual Field Inspection

16

Videos & Virtual Field Inspections (Example) Step 2: Virtual Field Inspection

17

Videos & Virtual Field Inspections (Example) Step 2: Virtual Field Inspection

18

Videos & Virtual Field Inspections (Example) Step 2: Virtual Field Inspection

19

Videos & Virtual Field Inspections (Example) Step 2: Virtual Field Inspection

20

Videos & Virtual Field Inspections (Use Cases): 1. Validate works programs 2. Record field notes 3. Visual inspections including pavement rating (subjective) 4. Add or find missing off-pavement assets and road furniture 5. Inspect off-pavement assets and road furniture (if video is clear)

6. Validate data accuracies combined with engineering judgement

21

Contents (Recap) 1. Some of Impactful Technologies in the Last Decade

2. GIS and it’s Advancements 3. Videos & Virtual Field Inspections 4. Machine Learning & Deep Learning Technologies (AI) 5. Lessons we Learned – RAMS Implementation

22

Machine Learning: Predicting Probable Treatment Type 1. Four years of condition data 2. Maintenance & surface type information 3. General Information – Traffic, Road Classification, etc..

4. Data was processed using JunoViewer to build a Fingerprint of each segment 5. About 13 models were explored 6. Three models were finalised for the final tests

23

Machine Learning: Predicting Probable Treatment Type Model Performance Results Predicted

Observed

Average of 3 Data Samples

Predicted Totals

None

Chipseal

Asphalt

Rehab

Obs. Totals

None

261

46

10

0.0

317

Chipseal

54

179

4

3

240

Asphalt

6

1

47

0

54

Rehab

4

9

5

5

23

325.7

235.0

65.3

8.0

24

Deep Learning (AI) – Distress Identification The Famous Cat vs Dog – Deep Learning Program - 2012

25

Deep Learning (AI)– Distress Identification Video Source

Videos Processed with a Specialized Software

Upload processed videos to streaming cloud & Deep Learning Models

Asset Management System

26

Deep Learning (AI)– Distress Identification (In Action)

27

Deep Learning (AI)– Distress Identification (In Action)

28

Deep Learning (AI)– Distress Identification (In Action)

29

Deep Learning (AI)– Information (Use Cases) 1. Distress information send to an asset management platforms 2. Rating applied 3. Further used in deterioration models 4. Can help in building a program and assess budgets

30

Contents (Recap) 1. Some of Impactful Technologies in the Last Decade

2. GIS & GPS Advances 3. Videos & Virtual Field Inspections 4. Machine Learning & Deep Learning Technologies (AI) 5. Lessons we Learned – RAMS Implementation

31

Lessons We Learned - RAMS 1. Lane based centreline GPS is the Backbone for any RAMS

2. Tendency is to underestimate the accuracies of GPS from a Smartphone and Action cameras 3. Finding a right solution to make sense of the available Data 4. Interconnection of Technologies

32

Interconnected - Technologies Mapped Network Condition & Video Data

Road Asset Management Systems

Evidence Based Informed Decisions Data Captured with Smartphone

Distress info & Det Modelling Outputs

GIS Data Representation

33

Thank You ravi@lonrix.com

34

Credits: 1. JunoViewer Asset Management System 2. Dr Fritz Jooste – Director, Lonrix Ltd 3. Sean Rainsford – Fulton Hogan, New Zealand 4. Philip van der Wel – Lonrix Ltd 5. Manuel Augero – Lonrix Ltd

35

References: JunoViewer Asset Management Platform https://www.bankmycell.com/blog/how-many-phones-are-in-the-world https://www.maxar.com/products/spatial-on-demand - On Demand Datasets

https://www.reuters.com/article/us-smartphone-decade-in-review-idUSKBN1YR1SC https://towardsdatascience.com/the-decade-of-artificial-intelligence-6fcaf2fae473 https://www.reportlinker.com/p05379577/Global-Intelligent-Transportation-Systems-ITS-Industry.html?utm_source=GNW

36

Questions ravi@lonrix.com

37