Realtime merging problems

Page 1

VACANCY INTERNSHIP PROJECT We are looking for students interested in carrying out their master thesis project at DAT.Mobility

Real time data source merging Introduction/context At DAT.Mobility we’re responsible for processing large amount of (near)real time traffic data. In order to use multiple sources (GSM data, floating car data, loop detector data, traffic signal data, parking data) we’ll have to merge these flows into consistent estimates of traffic parameters (e.g. flows and speeds) for a network. And that’s when problems arise: the sources have different latencies, report over different time periods and sometime have inconsistent meanings. Another problem is the location of the measurements (points, lines). Here is your challenge! The challenge In your project you’ll have to create a reusable method how to cope with new sources. How to integrate a new data flow and to validate it. And finally we’ll have a taste of the pudding! In practice, you’ll have to search for possible solutions on the internet and use your common sense to create this method. You’ll also have to think of a method how you can validate the integrated data, based on the incoming data.

Latency Almost every source has its own latency. Some will deliver data in milliseconds after registration, some will count for a minute and deliver after that. Others will only deliver historical data and all variations between.

Different time periods As said before, the sources are all different. Also on the reporting periods. One delivers data for each registration (VRI), some report over a minute and others report using running averages. Can we all use them? And if yes, how?

Location All the measurements are of different locations. The floating car data reports about trajectories, the loop detectors about a single point and the VRI data is even reporting on point and direction! How can the data be merged? That’s on you to find out!

Validate You’ll have to create a method to validate the integration of data. Is the integrated data reliable? The incoming data should be used to validate.

Proof of concept We have a running environment where you can implement and test your method.


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.