PUTTING DATA AT THE HEART OF ENERGY TRADING
Sponsored by
INTRODUCTION
In a digital world, data is King, and while energy and commodities have been going through a period of digitalizing for the last several years, there can be no doubt that coming up with data management strategies has also played an important, if not critical, role in those strategies as well. Indeed, this was strongly emphasized in Commodity Technology Advisory’s disruptive technologies research1 over the last few years, where more than half of all of those questioned in the industry were planning on further investment in data management and data mining. This was also where other new technologies like AI, for example, were thought likely to be applied for data mining purposes along with the use of enhanced visualization techniques.
Since that report was issued, much has already shifted due to industry events and drivers. Increased price volatility, particularly in energy, because of the Russia-Ukraine war and the current stage in the ongoing energy transition, along with digitalization and an increase in speed and data volumes around automated trading, has only re-affirmed many of its findings. Supreme among those would be the need for reliable and up to the minute data to support all trading and associated activities. Despite that, many in the industry still struggle with data – either in terms of being able to extract and utilize it from systems like ETRM or CTRM or, being able to access it in the right time frame and at the right quality.
Modern technologies have also meant that more data is available and collectible than ever before. All forms of data, both structured and unstructured, can be usefully used in trading and risk management if it is of sufficiently good quality and provided in a timely manner. A lot of effort is spent identifying, collecting,
and processing all types of data from a variety of sources by most trading or commodity-related firms. While some data like weather and pricing, for example, can be provided by data aggregating services, other data sources are web content often obtained via screen scraping, documents, images and so on, that need proprietary methods to obtain, cleanse, and utilize them. This is why there has been, and continues to be, significant investment in data and data management, whether that is in the form of data lakes or sophisticated data management applications.
Given the increasingly real-time nature of trading in many commodities and instruments, and given the speed of change in markets, access to high quality and timely data can make the difference between profit and loss or, good and poor decisions. This paper looks at those data management needs within trading and risk in energy and commodities, and it specifically examines FIS’s offerings. 1
CTRM – WHERE GOOD DATA GOES TO DIE?
CTRM software or its energy commodity version – ETRM, is often likened to a hole in which good data is placed but cannot be used outside of the pre-canned reports provided by the C/ ETRM. The CTRM can become a black hole insofar as where data is concerned despite the use of a variety of tools like BI used to try to access and utilize that data. In addition, in the last few years, the internet and regulations to promote openness and access to data, has meant that there is more data than ever that is useful to traders, risk managers and others within the energy or commodities enterprise. As stated above, that data now comes in all formats, structured and unstructured but of course, a lot of it is in the form of time series data as well. This means that data management has become a key aspect of business strategies and of digitalization efforts in the industry.
Providing users with the ability to identify data sources and then rapidly access them is now a key activity within most energy and commodity firms. This isn’t limited to price and weather data but can include a multitude of different types and formats of data from proprietary asset availability information on a website to satellite
imagery that allows estimates of crop yield, inventory, and so on. Of course, much of this data doesn’t sit in the C/ETRM at all, yet it is needed as back up or, input into decisions and algorithms that are related to processes recorded within the E/CTRM.
THE BASIC NEEDS OF A DATA MANAGEMENT TOOL
So, what is it that is needed to manage and access data in the manner that many seek in energy and commodities?
It is the ability to integrate with other systems – like E/CTRM – as well as almost any form of data source that has been identified. That data must then be made available in a timescale that keeps it useful in a fast-
moving business environment while also allowing users to cleanse, validate, and enrich the data automatically via easy to set up rules and processes. It must also provide the data in a format that is useful and allow
users to modify it via algorithms, calculations, rules and so on while ensuring an audit trail of transformations. Workflow is another important aspect of the solution –not just in terms of setting up the acquisition, cleansing and transformation processes but also to alert users to its availability as much data is time sensitive. It also needs to be able to handle huge amounts of data in a timely manner without performance issues and to allows visualization of the data in ways that aid users in decision making.
Use of an effective and comprehensive data
management application not only helps eliminate a lot of manual work with data including working with spreadsheets, but it helps to eliminate additional errors and costs. By being able to automate the acquisition, cleansing and transformation of many sources of data and make that data available to other users and applications across the enterprise in a timely manner, it reduces costs and risks and increases effectiveness. The data is then consumed and utilized in areas like risk, invoicing, settlement, position management and hedging, contract, and counterparty management and much more.
FIS OFFERS A COMPREHENSIVE SOLUTION
FIS has successfully integrated and deployed a combination of its data management and aggregation application – Market Data Analyzer- Energy Edition FIS Market Data Analyzer- Energy Edition formerly known as MarketMap Energy and FAME, with its E/CTRM solution - FIS® Energy Trading, Risk and Logistics Platform across several energy and commodity firms.
FIS Market Data Analyzer- Energy EditionMarket Data Analyzer- Energy Edition is its scalable big data management solution that can be hosted in the cloud offering users comprehensive and quality checked market data coverage along with the ability to rapidly add public or internal data sources and feeds. It affords users the confidence that their market and fundamental data is provided in a timely and quality fashion including complete support for time series and derived curves. It can easily be integrated with E/CTRM and other solutions including FIS’s own E/CTRM.
FIS Market Data Analyzer- Energy Edition includes the provision of over 400 aggregated data feeds (including over 250 exchanges) as well as extensive calculation and visualization tools. FIS also manages many of the data provider contracts so that users do not have to worry about multiple contacts and invoices with/from data providers. In addition, it is capable of web scraping and on-demand data acquisition to provide a holistic data solution. It is a modular solution that comes armed with all the tools, workflow and alerting that users need. Users have access to streaming, real-time and delayed
data, and the ability to rapidly integrate to consuming systems like E/CTRM and, when combined with the FIS® Energy Trading, Risk and Logistics Platform, it
affords users a rich and deeply functional platform that being from one vendor, offers support advantages and include 24 by 7, follow the Sun support.
FIS MARKET DATA ANALYZER - ENERGY EDITION PLATFORM
Several FIS clients have deployed this solution in areas like oil and gas, electric power and financial commodities trading to affect. In one instance, a major energy and commodities company that trades millions of barrels of crude oil and products every day required reliable access to hundreds of public multilingual websites for production, statistical, and logistical information to reduce operational costs. Despite investing in a data lake, they still could not solve the problem of acquiring and maintaining the data from across 1,000+ public websites and the Multi-lingual nature of the data sources was a further complexity. By adopting FIS Market Data
Analyzer- Energy Edition Market Data Analyzer- Energy Edition from FIS, the company was able to transform energy, commodity, statistical and economic websites, documents, and emails into high-quality data available in near-real time and at end of day. They accrued some serious benefits from this that included things like,
• Better and faster curve management for traders,
• Reduced operational risk,
• Reduced complexity and chance for errors,
• Reduced costs via a hosting arrangement,
• Ability to add new data feeds in the format that they wanted.
Two other clients manage power grids. One of these deployed the FIS solution to help manage meter data that was coming in in huge quantities and needed to be converted in to 5-minute increment time series. The other struggled with a vast array of spreadsheets holding data from multiple disparate sources. It used the FIS solution to capture and store this data, manipulate it to produce complex derived curves at different granularities and make all that data available to a wide range of analytical tools and applications across the enterprise including with the FIS Energy Trading, Risk
and Logistics Platform.
A large integrated financial services group has also deployed the solution successfully to transform market data and trades coming in from variety of exchanges and market data providers across a variety of commodities. It also organized its market data and curve analytics into a centralized server and the distributed it to its Commodity Trading system and Risk Systems. It also used data visualization and other analytical tools to work with the data.
THE BURDEN OF FUNDAMENTAL RESEARCH
While trade data like prices, volumes, and so on, is relatively easily obtained in energy and commodities via consistent feeds from any number of data aggregators, fundamental data can be much trickier to handle. This sort of data involves items such as demand, supply, production, consumption, seasonality, weather, shipping costs, storage capacity, traffic patterns and regional trends, to name a few.
Most usually, these data are freely available on public platforms and websites but, because of this, they are neither stable, nor guaranteed. It often takes teams of internal researchers and data scientists, scraping websites, translating languages, reformatting inputs and cleaning the data, to even begin to make it useful.
Then, no sooner do you have a good set of inputs established, than one of the main feeds goes down, or a new, better one comes up, and you have to start all over again. This endless cycle typifies fundamental research and it can consume countless hours.
DATA AS A SERVICE: A BETTER BUSINESS MODEL?
Energy and commodity companies looking for faster, more accurate insights across businesses and functions, require access to high-volume, diverse sets of data, so that they can make informed and timely decisions. It’s a cost of doing business that many firms simply absorb, yet it doesn’t have to be that way.
Machine learning is a powerful way of extracting knowledge from massive amounts of data and while a great deal of machine learning research has been focused on improving the accuracy and efficiency of training and inference algorithms, there was less attention paid to the important problem of monitoring the quality of data fed. Errors in the raw data can nullify any speed and accuracy benefits for training and assessment. This argument points to the need for a data-centric approach to machine learning that treats training and serving data as an important production asset, on par with the algorithm and infrastructure used for learning. FIS Market Data Analyzer – Energy Edition can address these challenges for you and lets you focus on the machine learning and data science. Thanks to data as a service (DaaS), the labor and cost involved in transforming fundamental research into useful insights can now be whittled down to a minimum
while also making the ongoing information stream a whole lot more reliable.
Major energy and commodities companies trade millions of barrels of crude oil or other commodities, such has natural gas, power, agricultural products, or metals and minerals, every day. They require access to hundreds of public multilingual websites for production, statistical and logistical information to reduce their operational cost. As a cost of doing business, they may choose to invest in a massive data lake, along with a sizeable team of data scientists, to monitor more than 1,000 public websites that change, add and delete in number every business day. The process can be extremely complex and unreliable. The burden it places on human resources and operating budgets can become intolerable.
FIS’ Market Data Analyzer DAAS SOLUTION
FIS works with major companies to shift their research operations to DaaS, offered by FIS Market Data Analyzer- Energy Edition – and invariably, dramatic changes occur in very short order.
Market Data Analyzer- Energy Edition establishes an open pipeline to a complete fundamental research data factory, supporting decision-makers with:
• Instantaneous access to 1,000+ feeds in preferred formats
• Tens of new feeds added every month
• Better and faster curve management for traders
• Reduced complexity and cost by hosting the data on FIS servers
• Reduced operational risk
• High-quality data supported by FIS’ energy and commodity data experts.
With Market Data Analyzer- Energy Edition, traders can begin to predict markets with greater accuracy. Their time to market can be reduced, as they begin receiving quality research in timely fashion and eliminate the extra work of maintaining individual custom curves for every trader. By hosting their data with FIS, they can add scale to their operations, and reduce OpEx relative to infrastructure and infrastructure maintenance. And, all the while, FIS domain experts ensure that their quality of data remains high and on point, in line with their stated business objectives.
SUMMARY
These real-world examples demonstrate how a proper data management strategy can unlock the potential of data from within and outside of the enterprise affording competitive advantage, reducing the potential for error and poor decisions, and reducing costs. This is why data management is a key and critical aspect of a digitalization strategy within energy and commodity firms.
FIS’s data FIS Market Data Analyzer- Energy Edition
Market Data Analyzer- Energy Edition allows users to properly manage data from many different sources quickly and easily with over 250 energy data feeds and historical data provided out of the box. Available
hosted and in the cloud, and by combining it with the FIS C/ETRM platform, the user benefits from working with one experienced supplier that can provide superior 24 by 7 support. It can also be easily and readily integrated with other E/CTRM solutions and a host of other commonly used applications in use in energy and commodity firms.
In a time of rapid change, multiple challenges and high volatilities – not just in commodity prices but in FX and interest rates as well, deploying state of the art data management can make a world of difference in dealing with the risks, opportunities and challenges in energy and commodities.
SPONSORED BY FIS
FIS is a leading provider of technology solutions for financial institutions and businesses of all sizes and across any industry globally. We enable the movement of commerce by unlocking the financial technology that powers the world’s economy. Our employees are dedicated to advancing the way the world pays, banks and invests through our trusted innovation, system performance and flexible architecture. We help our clients use technology in innovative ways to solve business-critical challenges and deliver superior experiences for their customers. Headquartered in Jacksonville, Florida, FIS is a member of the Fortune 500® and the Standard & Poor’s 500® Index.
getinfo@fisglobal.com
https://view.ceros.com/fis-global/market-dataanalyzer-energy-edition/p/1
To learn more, visit www.fisglobal.com. Follow FIS on Facebook, LinkedIn and Twitter (@FISGlobal).
ABOUT
Commodity Technology Advisory LLC
Commodity Technology Advisory is the leading analyst organization covering the ETRM and CTRM markets. We provide the invaluable insights into the issues and trends affecting the users and providers of the technologies that are crucial for success in the constantly evolving global commodities markets.
Patrick Reames and Gary Vasey head our team, whose combined 60-plus years in the energy and commodities markets, provides depth of understanding of the market and its issues that is unmatched and unrivaled by any analyst group.
For more information, please visit: www.comtechadvisory.com
ComTech Advisory also hosts the CTRMCenter, your online portal with news and views about commodity markets and technology as well as a comprehensive online directory of software and services providers. Please visit the CTRMCenter at: www.ctrmcenter.com
PO Box 1269, New Waverly, TX 77358
+1 832 687 4736
Prague, Czech Republic
+420 775 718 112
ComTechAdvisory.com
Email: info@comtechadvisory.com