affairs > tag this
BIG DREAMS, BIG DATA HAMAD BIN KHALIFA UNIVERSITY’S QATAR COMPUTING RESEARCH INSTITUTE (QCRI) IS HELPING BRING THE BIG DATA DREAM A LITTLE CLOSER. THROUGH THE HOME-GROWN RHEEM SYSTEM, DR JORGE-ARNULFO QUIANÉ-RUIZ AND THE DATA ANALYTICS TEAM AT QCRI HOPE TO INSPIRE MORE LOCAL BUSINESS TO TAKE A LEAP TOWARDS THIS EXCITING FUTURE. BY AYSWARYA MURTHY 48 > QATAR TODAY > OCTOBER 2016
O
ver here at Qatar Today we have been almost fanatically reporting and writing about Big Data and the Internet of Things in the past few years. It was impossible not to, considering how technocrats seem giddy with excitement preaching the incredible potential of this technology. In Doha, which hosts some top -level conferences and exhibitions across sectors, it’s been increasingly difficult to drop in at Meeting Room A or Hall 342 and not hear about this impending tectonic shift in the way we live. We are waiting at the edge of a technological leap, apparently. But once you step out of the pristine, airconditioned seminar rooms in Education
City and into the heat, dust and reality, this vision of the future is less clear. Qatari businesses still seem to be waiting on the sidelines. It calls for a significantly qualitative (or “disruptive”, as the kids are saying these days) shift in business that might be scary for some. Or maybe businesses here feel that the time isn’t right yet (though certainly no one would admit to this thought). Either way, Qatari corporations have been largely silent about the data they collect and how they use them. We should know – we asked many of them. It was therefore enthralling to hear tangible examples of Qatar Inc.’s tentative foray into Big Data applications during our chat at Qatar Computing Research Institute (QCRI), a research institute of Hamad bin Khalifa University, with Dr JorgeArnulfo Quiané-Ruiz, who leads the team that developed RHEEM. This application reportedly solves a substantial problem that businesses encounter when they try to dabble in Big Data, viz. processing data from different platforms where they are mined, categorised and stored differently. Companies can add as many new features as they want, on the fly, as well as replace or supplement existing platforms quickly. In other words, businesses can now stop worrying about HOW to query all these terabytes of data but instead start thinking about WHAT to query. For example, a telecom provider might have several platforms to analyse different sets of data – one gathered from social media that tracks their service levels and satisfaction. Another set of data of existing customers can be used to analyse preferences and promote offers. Another is used to improve infrastructure and services and a fourth may be used to attract potential customers. An executive shouldn’t have to worry about how many platforms they
A telecom provider might have several platforms to analyse different sets of data – one gathered from social media that tracks their service levels and satisfaction, another to analyse preferences of existing customers and promote offers. Another is used to improve infrastructure and services and a fourth may be used to attract potential customers. 49 > QATAR TODAY > OCTOBER 2016
affairs > tag this does or they are still in an incubation phase” says Dr Quiane-Ruiz. “The application has been well-received at top conferences on databases where we have presented research papers and a system demonstration” he says. This is a good sign for the project.
RHEEM can be used as the infrastructure to easily deploy algorithms for analysing the data on the spot with a human expert only required to verify the results. DR QUIANE-RUIZ Team Lead, QCRI
50 > QATAR TODAY > OCTOBER 2016
have, but be able to query all data and receive accurate and useful information. This is where RHEEM can be utilised. Dr Quiané-Ruiz explains: “The idea for the application was conceptualised a couple of years ago when we found that almost every time we wanted to implement a new application, we had to learn a new platform. There are many platforms out there to deal with data, and it’s usual to choose and learn one of them to get good results. But a single platform would not be adequate for different use cases as we make platforms specific to certain needs, e.g., whether you are dealing with temporal data, volume of data, etc. Across all these platforms, one paradigm didn’t, and couldn’t, fit all. We spent a lot of time learning these platforms and realised that enterprises were also suffering because of this. Any big company would have different applications and hire experts for the different platforms, which proved to be tedious and very expensive. We wanted to do it better.” RHEEM has been under development for a year. QCRI currently hosts this open source project and it is expected that many other organisations and developers will join in the efforts to help it evolve further. “Currently there is no real equivalent of RHEEM in the market. While there are similar approaches, we can say QCRI is a leader in this domain. Several institutions like MIT and Cambridge University, and companies, like Intel, IBM and Facebook, are all working on this. But either they do not provide all features RHEEM
Big Data in Qatar RHEEM was developed to target a broad spectrum of industries and applications and QCRI is working with several local organisations to stress-test the app and come up with a proof of concept. For example, the team is working with an airline to help them optimise certain operations. “To take one case, this company is interested in streamlining the cargo booking process. This is usually done by a group of specialists who manually analyse each booking and decide whether to accept or not based on several factors like contents, volume, flight capacity, etc. With rapid growth, the operation has grown many-fold and needs to be managed systematically. RHEEM can help by automatically analysing huge amounts of booking records (which run into millions) and learn from it to approve/decline bookings, with the goal that each airplane is always full of cargo but not overloaded so as to optimise fuel costs. They have been achieving good results” he says. In order to help companies understand how RHEEM can help their businesses, irrespective of size, QCRI is also working with local community website, Qatar Living. Every day, the growing number of residents in Qatar post hundreds of questions on the forum: Where does one get the best shawarma? What are the rents going to be like next year? What is the process to renew one car registration? Today community members reply to questions based on their experience and expertise. With the help of RHEEM, these questions can be answered automatically after analysing the website data and also information available on the web and this can be done in the order of a few seconds. In another exciting and tangible case study, QCRI is collaborating with a local research institute that is a world leader in connected vehicle technology. For Qatar Mobility Innovations Center (QMIC), RHEEM could be a perfect fit, according to Dr Quiane-Ruiz. With a goal of improving traffic and road safety, QMIC has been collecting and analysing data through Bluetooth sensors placed on the roads. This data is used to track vehicles, analyse
apps
To process and analyse Big Data one needs at least two fundamental layers – a data processing platform and the application implemented on top of it. RHEEM separates these two layers and puts itself in the middle. It’s a translator between the applications and the platform. Rather than dealing with many platforms, you now have a single system that matches all your needs.
ML 4 ALL
NADEEF
CROSS-PLATFORM OPTIMIZER
more
RHEEM OPERTORS
core
Possibilities and challenges Today, all sectors have realized the potential and benefits of Big Data technology, says Dr Quiané-Ruiz. “Equally. From sports to MNCs, everyone is trying to incorporate more analytics into their business models. They know they can do it, and they should do it but most don’t yet know how,” he says, underscoring the growing pains this technology is experiencing across the world, not just in Qatar. For example, many companies continue to collect and store data, anticipating that they can get insights from it. But often the data is just sitting there. “That’s the big problem we have today and research companies around the world are involved in trying to make sense of this data and figure out how to analyse it fast,” he says. He doesn’t deny that some sectors need more education on how to treat Big Data to our benefit. We are yet to fully realise some of the immediate applications of Big Data – from improving driver behaviour to customised marketing. “When we think of educating road users, we immediately picture ad campaigns but not data leveraging. There are insurance companies in the US that use applications to track each driver and give them hints and try to improve their driving. With improved skills, the premium will go down. This results in fewer accidents, which not only helps the insurance companies make more money but it also benefits the public,” he says while illustrating a simple case study. Big Data’s preventive rather than reactive nature will eventually find crucial application in the medical sector – both in laboratories (in an example pertinent to Qatar, in analysing genome data which are typically huge) and hospitals. For all this to fall into place, you need tools (like the ones QCRI is developing) but you also need experts who can prepare, understand,
THE STRUCTURE OF RHEEM
ENGINE
platforms
their path and predict traffic. Even with small databases, this is a laborious task and creating appropriate algorithms can take several months, he says. Now with the connected vehicles project ramping up and heading closer to implementation, QMIC has to deal with more sensors and the huge amount of data resulting from them. “These volumes are beyond the scope of human analysts. But RHEEM can be used as the infrastructure to easily deploy algorithms for analysing the data on the spot with a human expert only required to verify the results,” he says.
EXECUTION OPERTATORS
more
and analyse the data. RHEEM only solves one part of the equation by simplifying the tools and making them more efficient, but the Data Analytics Centre at QCRI is also working on the other part in the context of other in-house research projects. To a large extent, this is not the case of academia working to meet the needs of businesses, but businesses and society trying to catch up to the dreams of science. Dr Quiané-Ruiz explains that with their research agenda complete, the team will start educating enterprises on how exactly the system can impact bottom lines. This will complete RHEEM’s own capabilities in this direction, which is also being continuously developed. “We are currently in the process of stabilising the system. We hope to launch Phase 2 in the summer of 2017 where we will start approaching companies directly and show them what Big Data can do for them. The interest from the business community is present; our presentations in conferences and media outreach have been well received, with organisations approaching us with their problems, asking how RHEEM can help. It’s just a question of time,” he says 51 > QATAR TODAY > OCTOBER 2016