Global Press eBook 2010
CONTENTS Wall Street Journal
Wall Street Journal
Investors.com 7 The Atlantic 8-11 Reuters Insider 12 Business Line 13 Sify Finance
Forbes 15-16 Forbes 17 Financial Times 18-19 The Atlantic 20-21 USA Today 22-24 The Atlantic 25 Xconomy 26-28 Boston Business Journal
CRN 31 DNA 32 Financial Times 33-34 BBC 35-36 The Boston Globe
Boston Business Journal
CNBC 40 Publi News 41-42
Wall Street Journal
TRADERS STEP UP TESTING AFTER ‘FLASH CRASH’ BY KRISTINA PETERSON DATE: JUNE 22, 2010 NEW YORK—Program traders looking to reduce mistakes that can cost thousands of dollars in the blink of an eye increasingly are testing algorithms using past market data to simulate trading. The range, complexity and size of the process, known as backtesting, has become more popular since the May 6 “flash crash,” when the stock market dropped nearly 1,000 points in a matter of minutes before rebounding. “It’s no longer an afterthought,” said Richard Tibbetts, chief technology officer at trading-technology firm StreamBase Systems. “If you’re going to be committing tens of millions of dollars to a trade that’s quantitatively driven, you want to be very confident in your model.” Most companies test their trading strategies by running them through simulations created by vast databases of past market data. But programs have to factor in the effect of the algorithm itself. “The mere presence of your trading activity would change the trading activity of other firms—that’s what makes backtesting quite chal- lenging,” Mr. Tibbetts said. Just testing reams of past data isn’t always enough to see what algorithms can handle. StreamBase engineers adapted a Nintendo Wii controller to help generate random numbers fed into a backtesting simulation used at a trade show, though the company tends to use less-playful number-generation tools in normal circumstances. Other companies are roping in higher-powered computer chips to speed up the barrage of data launched at the trading formulas. Firms are increasing the number of simulations and the level of detail created for the tests, Mr. Tibbetts said. New regulations such as circuit breakers are creating different trading parameters that have to be accurately reflected in the simulated environment for tests to predict an algorithm’s performance accurately. Since the market’s precipitous plunge May 6, trading firms have been hustling to make sure their systems can deal with the rush of market messages that accompanied the huge spike in trading activity during the flash crash.
Wall Street Journal
“Data rates peaked at extremely high levels during that period,” said Howard Pein, chief executive of trading-software company CodeStreet. On May 6, the number of messages sent across all live data feeds reached a high of more than 2.8 million messages a second at 2:43 p.m. EDT, according to the website MarketDataPeaks.com, which tracks messaging. That’s sharp increase from the not-distant past, when market data feeds used to peak at tens of thousands of updates a second. Algorithms and market data platforms were “stressed in ways they hadn’t been stressed before,” said Mr. Pein. Now firms are backtesting systems to make sure they can handle 500,000 updates a second and more, Pein said. “These are projects that are specifically commis- sioned to deal with the consequences of May 6.” Shock over the velocity of the flash crash has also had ripple effects. Traditional backtesting used to be largely confined to preparing algorithms before they actually entered the market. Companies were reluctant to run testing programs alongside their algorithms once they went live, as they often add small delays, or latency, to execution times. But after the sharp drops in stocks during the financial crisis and, more recently, the May 6 plummet, traders are willing to run programs that monitor and tweak algorithms in real time, said John Bates, chief technology officer of Progress Software Corp. and founder of the Apama data-processing platform. “In the traditional trading world, people used to turn off their risk management because it slowed things down and the potential upside was great, but the world has changed,” Mr. Bates said. “Fear of regulations and the fact that we’ve seen you can lose large amounts of money in a short period of time has really turned that around.”
RABOBANK GLITCH EYED IN STERLING BLIP BY KATIE MARTIN DATE: JULY 21, 2010
Wall Street Journal
Updated: A computer glitch at Dutch bank Rabobank caused a sudden and fleeting slump in sterling, Wednesday traders at other banks said, marking the second such currency-market blip for the week. The pound tanked by around 1% against the dollar at the outset of European trading hours, causing confusion for a moment before the currency quickly sprang back. Rabobank declined to comment. The mishap was the mirror image of a shocking spike in the Swedish krona just 24 hours earlier. That came from an as-yet- unknown source, but traders aren’t linking it to Rabobank. Neither rumble caused any lasting disruption in the markets, though the culprits are likely left with red faces and modest losses.
asset classes such as equities of late.
The events offer a reminder that the usually super-efficient foreign-exchange markets are vulnerable to the same kind of sharp moves that have affected other
“These things happen in equities, commodities and elsewhere, and foreign exchange is no different,” said Giles Nelson, the London-based deputy chief technology officer at Progress Software Corporation, which produces market-monitoring systems. “The consensus in the mar- ket is that this was a computer-based trading error, but ultimately there would have been a human involved somewhere,” he said. The episodes show that some organizations lack robust computerized checking mechanisms to vet trades before they hit the markets, Nelson added. The talk on London’s trading floors is that Rabobank’s electronic trading systems tried to dump up to £3 billion of sterling into the market in one hit, rather than splitting up the trade to minimize market impact. It is unclear whether the computer itself failed, or whether it was programmed incorrectly, but the bank stopped the trade within a few minutes, traders elsewhere said. At around the same time Tuesday, a similar event caused what one trader described as “fun and games” in the Swedish krona, as that cur- rency rocketed against the euro, again without any solid fundamental trigger. The euro sank by an eye-watering 14% against the krona for a very short period of time on at least one trading system, hitting a low of 8.16 krona. Again, the euro immediately sprang back.
One trader said that the order was placed on the Swedish krona accidentally and the Norwegian krone was the intended target. That was “a very painful mistake” he said. The moves in both the krona and the pound were likely to have been exaggerated by generally quiet summertime trading conditions. Dur- ing a busier period, they may have been less noticeable or met with stronger flows in the other direction, limiting their impact. Neither of these events had any significant effect on currencies more broadly, and neither was on the same scale as the so-called May 7 “flash crash” in U.S. equity markets, which sent ripples throughout the financial markets. Indeed, the drop in the pound didn’t breach the previous day’s low. Nonetheless, the fear among market insiders is that these events could easily have been more serious, distorting prices more generally, and they can affect any dealing firm in any market. Last summer, for example, a trader at London brokerage PVM Oil Futures Ltd. ramped up the price of a key oil futures contracts by trading while drunk. His actions attracted the attention of regulators such as the U.K.’s Financial Services Authority, which later fined him.
Wall Street Journal
“These things happen regularly, and we only hear about some of them,” said Nelson. “The danger is, as we saw from the case of PVM, they can cause greater regulatory scrutiny of these markets.”T
he generally, and they can affect any dealing firm in any market.
Last summer, for example, a trader at London brokerage PVM Oil Futures Ltd. ramped up the price of a key oil futures contracts by
trading while drunk. His actions attracted the attention of regulators such as the U.K.’s Financial Services Authority, which later
“These things happen regularly, and we only hear about some of them,” said Nelson. “The danger is, as we saw from the case of
PVM, they can cause greater regulatory scrutiny of these markets.”
PROGRESS EXEC TACKLES FAST, COMPLEX TRADES BY: J. BONASIA DATE: AUGUST 2, 2010 LOCATION: HTTP://WWW.INVESTORS.COM/NEWSANDANALYSIS/ARTICLE/542316/201008021816/ PROGRESS-EXEC- TACKLES-FAST-COMPLEX-TRADES.ASPX
Regulators still can’t explain why the Dow Jones industrial average plunged nearly 1,000 points in the span of minutes in last May’s Flash Crash. But John Bates aims to help find out — and reduce the chances of such freak occurrences happening in the future. This month Bates, chief technology officer of Progress Software (PRGS), joined the Technology Advisory Committee for the U.S. Commodity ...
MARKET DATA FIRM SPOTS THE TRACKS OF BIZARRE ROBOT TRADERS BY ALEXIS MADRIGAL DATE: AUGUST 4, 2010
Mysterious and possibly nefarious trading algorithms are operating every minute of every day in the nation’s stock exchanges. What they do doesn’t show up in Google Finance, let alone in the pages of the Wall Street Journal. No one really knows how they oper- ate or why. But over the past few weeks, Nanex, a data services firm has dragged some of the odder algorithm specimens into the light. The trading bots visualized in the stock charts in this story aren’t doing anything that could be construed to help the market. Unknown entities for unknown reasons are sending thousands of orders a second through the electronic stock exchanges with no intent to actually trade. Often, the buy or sell prices that they are offering are so far from the market price that there’s no way they’d ever be part of a trade. The bots sketch out odd patterns with their orders, leaving patterns in the data that are largely invisible to market participants. In fact, it’s hard to figure out exactly what they’re up to or gauge their impact. Are they doing something illicit? If so, what? Or do the patterns emerge spontaneously, a kind of mechanical accident? If so, why? No matter what the answers to these questions turn out to be, we’re witnessing a market phenomenon that is not easily explained. And it’s really bizarre. It’s thanks to Nanex, the data services firm, that we know what their handiwork looks like at all. In the aftermath of the May 6 “flash crash,” which saw the Dow plunge nearly 1,000 points in just a few minutes, the company spent weeks digging into their market record- ings, replaying the day’s trades and trying to understand what happened. Most stock charts show, at best, detail down to the one-minute scale, but Nanex’s data shows much finer slices of time. The company’s software engineer Jeffrey Donovan stared and stared at the data. He began to think that he could see odd patterns emerge from the numbers. He had a hunch that if he plotted the action around a stock sequentially at the millisecond range, he’d find something. When he tried it, he was blown away by the pattern. He called it “The Knife.”
This is what he saw:
“When I pulled up that first chart, we saw ‘the knife,’ we said, that’s certainly algorithmic and that is weird. We continued to refine our software, honing the algorithms we use to find this stuff,” Donovan told me. Now that he knows where and how to look, he could spend all day for weeks just picking out these patterns in the market data. The examples that he posts online are just the ones that look the most interesting, but at any given moment, some kind of bot is making moves like this in the stock exchange. “We probably get 10 stocks in any 10 minutes where we see something like this,” Donovan said. “It’s happening all the time.” These odd bots don’t really make sense within the normal parameters of the high-frequency trading business. High-frequency traders do employ algorithms to look for patterns in the market and exploit them, but their goal is making winning trades, not simply sending quotes into the financial ether. Here’s the way a stock trade is supposed to work: a buyer says they’ll pay some amount for 100 shares of a company, a seller makes an ask for slightly more money, and the two of them usually meet in the middle. Perhaps a middle man (no joke intended) helps match buyer and seller and takes a cut. That’s the role that a lot of high-frequency traders play: they help make markets work. Regulatory changes over the past several years have extended their usefulness and provided a nice business model for those that can move quickly to provide options for buyers and sellers. “Under the maker-taker model, market participants that offer to provide, or make, liquidity by posting an order to buy or sell a certain number of shares at a particular price receive a rebate,” explained Michael Peltz in a June feature for Institutional Investor. “Those that execute against that order -- that is, take the liquidity -- have to pay a fee. Exchanges earn the difference between the rebate they pay and the fee they charge. The SEC limits taker fees to 0.30 cents a share; rebates tend to be lower for economic reasons, but for high frequency firms trading millions of shares a day, they can make for a pretty good living.” In a sense, they take nickel-and-diming down an order of magnitude or two. The advantage is that their trades are low-risk: they rarely hold positions for very long and any individual stock, future, or currency can’t really sink the boat. High-frequency traders have become a target for all kinds of people, but most of them appear to make their money being a little faster and little smarter than their competitors. And if they are playing by the rules, they improve the quality of markets by minuscule amounts trade after trade after trade. But the algorithms we see at work here are different. They don’t serve any function in the market. University of Pennsylvania finance pro- fessor, Michael Kearns, a specialist in algorithmic trading, called the patterns “curious,” and noted that it wasn’t immediately apparent what such order placement strategies might do. Donovan thinks that the odd algorithms are just a way of introducing noise into the works. Other firms have to deal
with that noise, but the originating entity can easily filter it out because they know what they did. Perhaps that gives them an advantage of some millisec- onds. In the highly competitive and fast HFT world, where even one’s physical proximity to a stock exchange matters, market players could be looking for any advantage. “They are moving the high-frequency services as close to the exchanges as possible because even the speed of light matters,” in such a competitive market, said Stanford finance professor Peter Hansen. Given Nanex’s data, let’s say that these algorithms are being run each and every day, just about every minute. Are they really a big deal? Donovan said that quote stuffing or market spoofing played a role in the Flash Crash, but that event appears to have had so many causes and failures that it’s nearly impossible to apportion blame. (It is worth noting that European markets are largely protected from a similar event by volatility interruption auctions.) But already since the May event, Nanex’s monitoring turned up another potentially disastrous situation. On July 16 in a quiet hour before the market opened, suddenly they saw a huge spike in bandwidth. When they looked at the data, they found that 84,000 quotes for each of 300 stocks had been made in under 20 seconds. “This all happened pre-market when volume is low, but if this kind of burst had come in at a time when we were getting hit hardest, I guarantee it would have caused delays in the [central quotation system],” Donovan said. That, in turn, could have become one of those dominoes that always seem to present themselves whenever there is a catastrophic failure of a complex system. There are ways to prevent quote stuffing, of course, and at least one of the members of the Commodity Futures Trading Commission’s Technology Advisory Committee thinks it should be outlawed. “Algorithms that might be spoofing the market are something that should be made illegal,” said John Bates, a former Cambridge profes- sor and the CTO of Progress Software. But he didn’t want this presumably negative practice to color the more mundane competitive practices of high-frequency traders. “There is algorithmic terrorism and then there is reverse engineering, which is probably just part of good business practice,” Bates said. For now, Donovan plans to keep putting out the charts, which he calls “crop circles,” of the odd trading algorithms at work. That’s an apt name for the visualizations we see of this alien world of bot trading. And it certainly gets at a central mystery surrounding them: if trading firms aren’t sending out these orders, how are they getting into the market? On the quantitative trading forum, Nuclear Phynance, the consensus on the patterns seemed to be that they simply just emerged. They were the result of “a dynamical system that can enter oscillatory/unstable modes of behaviour,” as one member put it. If so, what you see here really is just the afterscent of robot traders gliding through the green-on-black darkness of the financial system on their way from one real trade to another. No matter why the bots end up executing these behaviors, the Nanex charts offer a window onto a kind of market behavior that’s fasci- nating and oddly beautiful. And we may never have seen them, if not for the mildly obsessive behavior of one dedicated nerd. “Who looks at millisecond charts?” Donovan said. “You’d never see those patterns in any other fashion. The SEC and CFTC certainly weren’t.” Here are a few more bots at work with explanations of what’s going on.
Here we see a “flag repeater” being executed on the BATS Exchange, the third-largest equity market after the NYSE and NASDAQ. 15,000 quote requests were made in 11 seconds in a repeating pattern. Each iteration upped the quote a penny until $9.36, and then the algorithm went down the same way, a penny at a time.
This is an extreme closeup of just one second of trading of the stock SHG, the Shinhan Financial Group. This is 760 quotes from a total of 10,000 made in 12 seconds.
This chart shows a different kind of strategy. It represents 56,000 quotes in one second all at the same price (the top chart) but with the size of the order increasing by one (i.e. 100 shares) all the way up to 40,000.
Finally, we see what Donovan calls the â€œstubby trianglesâ€? chart. It shows high quotes being made and then immediately followed by a stub order of $0.01 (basically canceled in most contexts). The quote is then remade at a lower price and followed with another stub quote. This cycle happened at the rate of 380 quotes a second. [This last description was clarified thanks to the kindness of author Joe Flood.]
TECH EXPERT: STOP “FLASH CRASHES” BY BOOSTING SURVILLANCE DATE: AUGUST 30, 2010 LOCATION: http://insider.thomsonreuters.com/link.html?ctype=groupchannel&chid=3&cid=138679& start=0&end=314&shareToken=MzplMWVmZTNkMi0xMThkLTRhNWItOTc3ZS1lZjMzYmNmYjNiNmI %3D
Progress Software’s John Bates appears on Reuters TV to discuss how surveillance is key to preventing another flash crash.
PROGRESS SOFTWARE TO EXPAND INDIA OPERATIONS DATE: SEPTEMBER 4, 2010 Progress Software India (PSI) is set to expand its scale of operations in India by adding to staff strength and increasing investments over the next one year, according to Mr Rob Levy, Executive Vice-President and Chief Product Officer. The staff expansion will mainly be at the R & D Centre at Hyderabad. At present, it employs around 260 and plans to add about 20 per cent more. Interestingly, 30 per cent of the total product development resource of the US-based company is in India and there are definite plans to push it to 50 per cent.
Given the talent pool available and strategy of the company to involve India in product development, which will be as much as 50 per cent, PSI wants to make India one of its hubs for R&D. “We look forward to this expansion as an important investment in India,” he told Business Line. Post the acquisition of Savvion in January (for $49 million), PSI has become an independent unit in the Asia-Pacific Japan region for the Massachusetts, US-headquartered Progress Software, which has 1,800 employees worldwide. The company is betting big on its Progress Responsive Process Management (RPM) suite, a next generation business solution, which en- ables enterprises to increase business performance. The market opportunity is in the range of $10 billion and Progress Software is the only company with the ability to deliver, he said. With domain expertise in several areas of business, the company is talking to system integrators to implement the RPM. Areas such as financials sector, airlines, healthcare in US, which needs electronic records, insurance and banking have potential for the suite. “We are also working on the next version of the RPM, which should be ready in another year,” Mr Levy said.
PROGRESS SOFTWARE TO EXPAND R&D CENTRE AT HYDERABAD BY K. RAJANI KANTH DATE: SEPTEMBER 24, 2010 Progress Software Corporation, a Nasdaq-listed provider of infrastructure software products, is in the process of expanding its India re- search and development (R&D) centre at Hyderabad to accelerate development of newer versions of its new Responsive Process Manage- ment (RPM) suite. The company has a global R&D strength of 700, of which 180 professional are based out of its 40,000-sft Hyderabad centre. It is now acquiring more space and hopes to go in for recruitment to fill this up by December. Once through, the Hyderabad centre will house 60 per cent of the company’s global R&D staff, Rob Levy, executive vice-president and chief product officer of Progress Software, told Business Standard. “RPM for us is a big opportunity and the market for this type of solution, according to IDC Research, is expected to be greater than $10 bil- lion. Even capturing a part of it will be good. In order to do this, we need to grow,” he added. Progress RPM suite is a human-centric platform providing business users with real-time visibility into business processes and systems, in- cluding legacy processes both inside and outside the business. The RPM suite is the first product developed by the company while the three existing products as Apama, Actionall and Savvion (with development centres in Mumbai and Coimbatore) had been acquired. Progress had released Version-I of the RPM suite at the end of April 2010, and is currently looking at expanding its partner eco-system for marketing the same. “Right now, dialogues are on with all the top-4 IT companies in the country for RPM. What we expect from them is to bring to the table their expertise and reach as collaborative go-to-market partners,” Levy said. Each of these IT companies need to establish centres of excellence (CoEs), get to know the product and find projects to work on, he said, while refusing to draw any time line for the deals to crystallise. Stating that the over $500-million company was growing at a double digit rate on RPM since this April, Levy said they were looking for extension of the RPM suite next year, either through acquisitions or building new versions in-house. “As the chief product officer, I always look for acquisitions. We will start talking about RPMâ␣TMs Version-II beginning next year when we will think about what technology we need to acquire,” Levy said, adding India was a market with a lot of technologies and many opportuni- ties for acquisitions.
THE HIGH-FREQUENCY TRADING CHALLENGE: EXPLAIN IT BY EMILY LAMBERT SEPTEMBER 13, 2010 High-frequency trading is presenting all sorts of challenges these days, what with quote stuffing and other news. But at least one challenge it presents could be tackled pretty easily – defining “highfrequency trading.”
Last week Hirander Misra, chief of the tech firm Algo Technologies, said at a derivatives conference in Switzerland that the industry needs to define the phrase for the regulators, press, and public. It’s not just day traders gleefully rubbing their hands together in front of super-fast computers, apparently, as they lob quote bombs at the New York Stock Exchange, CME Group and other trading venues. To his point, if people want to write rules for this, they need a definition. So I asked some industry folks for short and sweet descriptions of high- frequency trading, in Twitter-style 140 characters or less. (The character limit was a suggestion, not a rule.) How did they do? 1. James Overdahl, spokesman for Principal Traders Group (the ones formerly known as high-frequency traders) “The use of information technology to quickly receive and analyze market data in order to identify and act upon profitable trading op- portunities. Trading strategies include 1) market making; and 2) arbitrage across related products.” Result: Fail! 93 characters over. 2. Bill McNeill, director of trading at HTG Capital Partners “All-electronic submissions and cancels of orders in milliseconds or microseconds.” Result: Clear as day. A+ 3. John Bates, chief technology officer of Progress Software “Automated analysis of real-time market data to detect patterns that indicate a trading opportunity & instantly place orders in the mar- ket.” Result: Makes sense on a second read. B+.
4. Robert Newhouse, founder and former chief executive of Ballista Securities “Any trading activity that leverages high-throughput, low-latency computer technologies in order to interact with the marketplace for short-duration transaction times.” Result: Nice try. Over by 26 characters. 5. Gus Sauter, chief investment officer of Vanguard Group “High frequency trading is not a strategy. It is merely trading very quickly– typically microseconds. There are a number of strategies that utilize high frequency trading. It’s difficult to identify how much of high frequency trading is accounted for by a given strategy. However, we believe the lion’s share is attributable to nimble traders who make a small arbitrage profit by scouring the markets for price discrep- ancies and trading to eliminate them. Such trading activity has led to lower transaction costs and greater liquidity, benefitting all market participants.” Result: C. Good summary, but why not just stop at “microseconds?” Over by 434 characters.
6. Hirander Misra, chief executive of Algo Technologies “HFT is defined as being highly automated black box proprietary trading, based on mathematical computer driven algorithms, usually re- sulting in large volumes of order entry, revision and cancellation in very short time horizons. Such trading strategies can vary and can be liquidity provision strategies with a large percentage of passive orders on the book, predictive trading based on predefined time intervals and arbitrage trading exploiting price discrepancies in the same instrument listed across multiple venues or across multiple correlated asset classes. Some strategies also employ liquidity detection techniques to seek out larger institutional orders sitting in exchange order books to further inform their trading decisions. HFT strategies can be deployed in stand alone proprietary trading firms, hedge funds and investment banks and as such the typical definition of the types of firms that conduct HFT is becoming less well defined.” Result: Good lord, Misra. Is anyone still reading? Over by 807 characters
FORBES CIO VIDEO NETWORK: CONSIDER CULTURE AT THE CORE BY DAVE BENSON DATE: SEPTEMBER 22, 2010 HTTP://VIDEO.FORBES.COM/FVN/CIO/PROGRESS-CIO-DAVE-BENSON-ON-CORPORATE-CULTURE
INNOVATION DRIVES TRADING SURGE BY JENNIFER HUGHES DATE: SEPTEMBER 27 , 2010 Among the 225,000-plus iPhone applications in the Apple store sits one called Merlin, an app named after Credit Suisse’s online trading platform. Clients armed with a password can see, and deal, the bank’s live options prices from their iPhone. “Its simple to use,” says Martin Wiedmann, head of global foreign exchange sales and distribution at Credit Suisse. Foreign exchange, or FX, came out of the credit crisis well, but the point to take from Credit Suisse’s iPhone app and rival bank offerings is that the market is not resting on its laurels and instead is developing new technology and ways of trading. The market’s momentum was also captured in a report from the Bank for International Settlements this month. The triennial market survey is the industry benchmark and showed average daily trading volumes in FX have reached $4,000bn a day – a new record and up 20 per cent on the last survey in 2007. Market insiders say the BIS numbers probably do not reflect the actual market peak. The BIS collects data in April and volumes spiked higher in early May as the eurozone sovereign debt crisis reached its height. With short-term interest rates around the world held at extremely low levels, many investors have turned to FX. “It used to be that FX was relatively stable and the volatility took place in [interest] rates. Now with quantitative easing, rates are basi- cally zero, and when you take out that volatility, it goes elsewhere – and one of those places is FX,” says Alan Bozian, chief executive of CLS Bank, the FX settlement system. Other themes were evident in the BIS survey. In spite of the borderless trading that characterises the FX world, London, the traditional centre of the market, actually increased its dominance, taking 37 per cent and seeing off threats from New York (17.9 per cent) and a resurgent Tokyo, which pipped Switzerland to third place with 6.2 per cent. A lot of the rise in trading came in the spot market, where a near 50 per cent rise took volumes to $1,500bn a day. While some of the big- gest emerging-market economies raised market share, the bulk of the rise came in the so-called majors – trading in dollar, euro and yen and to some degree, sterling and the Swiss franc. This probably means the rise is in no small part the result of algorithmic, or “algo,” trading – the use of fast-moving computer models. These hit the headlines in the equity markets following their suspected involvement in the May 6 “flash crash” in the US markets.
Could a similar event disrupt FX? That could be more serious because of the market’s role as part of the global payments system, giving it a day-to-day systemic importance no other asset class has. Market observers think a “flash crash” unlikely. They point out that FX trading is concentrated in far fewer currency pairs than there are stocks in any major market and, as a result, each pair is watched by more people than virtually any single stock.“FX is emerging as an asset class where people can and do speculate, but mostly people are still trading because they need to buy or sell that particular currency,” says Giles Nelson, deputy chief technology officer of Progress Software, which develops algo-related products. “The FX marketplace still has banks at its centre. I don’t believe there is the same potential for high-fre- quency traders to influence prices as there might be elsewhere.” The vast sums traded in FX have indeed given the banks a central role they long ago gave up in some other asset classes. Currently it is one they welcome as FX has become increasingly popular with senior bank executives keen to develop revenues that do not rely on increasingly expensive bank capital. This does not, however, preclude technological investment. The top FX banks have warned their would-be rivals to be ready for a long, expensive, IT-driven slog if they want to compete. Kevin Rodgers, global head of FX derivatives at Deutsche Bank, the biggest FX bank by trading volumes, says: “The question is, do they have the strategic patience to grind out what they need to build in the face of the incumbents who are continuing to forge ahead?” For Mr Rodgers, this is work on developing electronic options trading. “The options market is light years behind spot in terms of e-trading since anything to do with derivatives is several orders of magnitude more complicated,” he says. “But that’s where the market is going and as more and more volume gets transacted then that prompts banks, and the algo shops, to work on the challenges.” Mr Rodgers reckons about 30 per cent of the bank’s FX derivatives volumes comes from the bank’s Autobahn platform – and that has tripled as a percentage in the past three years. “More and more of our volume is electronic,” he adds. “Where I’m spending resources and IT is on our e-commerce platform.” That goes for all the big banks.“Star traders are great but they aren’t scaleable,” says Frederic Boillereau, global head of FX and metals at HSBC. “FX today is mainly about e-distribution, e-risk, and managing the flows. We’re always thinking about how to do that business better.” Just down the road from HSBC in Canary Wharf sits Barclays Capital, which has invested over a number of years to build itself into a top- three force. “What I work on and worry about is making sure that I’m getting the electronic platform married up with highquality content and other value-adds,” says Ivan Ritossa, who oversees FX and e-commerce at the bank. “Now the challenge is we have to think a little like a content provider online – it’s not just about trading but what else you can offer, too.” The process has been helped by the increased importance of FX to bank profits and also by other, longer-term trends, such as the increasing correlations between asset classes. Jeff Feig, global head of G10 FX at Citigroup, says that has meant growing links between the asset classes within the bank. “When we first merged with Travellers [in 1998], the equity traders and the fixed-income guys had no interest in sitting down and talking to us. Today there’s interest from every market and trading desk. The correlations and interactions have increased massively,” he says. Combine greater interest from other asset classes with the development of new technology, such as Credit Suisse’s iPhone app, and you have a market very confident about its future.
REGULATORS FINGER DUMB ALGORITHM IN ‘FLASH CRASH’ BY ALEXIS MADRIGAL DATE: OCTOBER 1, 2010
The country’s financial regulators have delivered their final report on the mysterious May 6 “Flash Crash,” in which the Dow plunged 10 percent in just minutes -- and it turns out that a dumb algorithm is partly to blame. A single large sell order executed by a rather crude software program sent the already-stressed market into a downward spiral. It was obvious that computerized trading systems clearly played some important role in the Flash Crash, but the Securities and Exchange Commission and Commodity Futures Trading The new report details precisely what happened. A large firm, reportedly Waddell & Reed, sold more than $4 billion of S&P500 futures known as E-Minis. Their sell algorithm only took into account market volume when making trades, not price or time. The algo could only “know” what it was told to know, and because of the unusual market conditions, it sold all of the contracts in 20 minutes. This large fundamental trader chose to execute this sell program via an automated execution algorithm (“Sell Algorithm”) that was pro- grammed to feed orders into the June 2010 E-Mini market to target an execution rate set to 9% of the trading volume calculated over the previous minute, but without regard to price or time. As the Sell Algorithm went to work, traders bought up the contracts normally, but other algorithms started to pile on, increasing the volume of trades. Because the Sell Algorithm’s sell rate was pegged to the volume of the market, it started going faster and faster. The Sell Algorithm used by the large trader responded to the increased volume by increasing the rate at which it was feeding the orders into the market, even though orders that it already sent to the market were arguably not yet fully absorbed by fundamental buyers or cross-market arbitrageurs. In fact, especially in times of significant volatility, high trading volume is not necessarily a reliable indicator of market liquidity. The net effect was that the entire order was complete within 20 minutes, and that sent the markets into a panic. “The interaction between automated execution programs and algorithmic trading strategies can quickly erode liquidity and result in disorderly markets,” the regulators wrote. “As the events of May 6 demonstrate, especially in times of significant volatility, high trading volume is not necessarily a reliable indicator of market liquidity.”
As John Bates, a member of the technical advisory committee for the CFTC and the CTO of Progress Software, explained, “Liquidity was choked off as traders tried to make sense of the situation, which was nearly impossible.” The Flash Crash drew attention to the speed and automation of the markets. The speed of the decline (and the subsequent) recovery shocked just about everyone but the high-frequency specialists.Many wondered whether the crash resulted from bad actors, prompting calls for increased surveillance of high-frequency traders. That’s still a good idea, but in this case, the problem looks more like simple incompetence, which is a harder thing to regulate against. One key strategy could be implementing pre-trade risk monitoring of algorithmic strategies, Bates said. “The report underscores a powerful need for better pre-trade back-testing of a strategy to analyze potential impacts. Testing under realis- tic negative conditions might have stopped the firm from using the algorithmic approach used. It also highlights the need for better real- time monitoring of the trading process, using technology that is available but underused for pre-trade risk monitoring, market monitoring and market surveillance.” MIT’s Andrew Lo told us this summer that we need to “resynchronize the regulatory infrastructure with the technology of our time.” Because the traders aren’t waiting to implement new technologies, and their speed really matters.
“We’re seeing innovations that dramatically increase the speed and throughput of the market, and that works great until it doesn’t,” Lo concluded.
MASSIVE COMPUTER-DRIVEN SELL ORDER TRIGGERED MAY 6 PLUNGE BY ADAM SHELL DATE: OCTOBER 2, 2010 NEW YORK — A massive computerdriven sell program initiated by a mutual fund company was the trigger for the May 6 stock market “flash crash” that sent the Dow tumbling almost 600 points in a matter of minutes and has left a lasting scar on Main Street inves- tors, federal regulators say. A report issued Friday by the Securities and Exchange Commission (SEC) and the Commod- ity Futures Trading Commission (CFTC) sheds some new light on what caused the market mayhem that day, a trading session that started out turbulent as fears surrounding the European debt crisis roiled global markets.
The report, however, only attempts to explain what happened that day. It does not address specific policy responses or new measures that would preclude a similar meltdown hap- pening in the future. A separate committee of market regulators will review Friday’s report and make recommendations at a later date, the SEC said. “This report identifies what happened and reaffirms the importance of a number of the actions we have taken since that day,” SEC chairman Mary Schapiro and CFTC chairman Gary Gensler said in a statement. “We now must consider what other investor-focused measures are needed to ensure that our markets are fair, efficient and resilient, now and for years to come.” One remedy already put in place has been circuit breakers, which force markets to pause, or stop trading, when price movements exhibit abnormal activity. If an individual stock suffers a drop of 10% or more over a five-minute period, trading in that stock is now halted for five minutes. The report’s findings Here’s a quick snapshot of what the SEC and CFTC in their report say happened on May 6: 1. Massive sell program is activated. At 2:32 p.m. ET, at a time when the market was already volatile, an unidentified mutual fund com- pany put in a computer sell program of 75,000 E-Mini contracts — these are futures contracts tied to the Standard & Poor’s
500 index that allow traders to bet on the direction of the index. The trade was valued at $4.1 billion. Only two trades of equal or bigger size have occurred in the past 12 months, and both were executed by the same unidentified fund company. 2. Trade execution is totally automated. Normally, trades of this large size are executed by a combination of manual orders, sales of big blocks of shares by a trading intermediary, such as a broker, as well as automated computer trades that are put in motion by a computer algorithms. Typically, when computer algorithms are used, these trades target volume, price and time variables. But this trade was 100% automated, using computer algorithms. It also was designed to feed orders based on trading volume, but not price or time. 3. Trade duration is compressed in short time span. In the two earlier trades of this size entered by this same mutual fund company, the trade was executed using a combination of manual trades placed over the course of the day and several computer-driven trades, all of which took into account volume, time and price. The earlier trades also took more than five hours to complete. But on May 6, the algorithm-only trade took only volume into account, which resulted in the trade being executed in a super-fast 20 minutes. 4. Size and speed of trade results in too few buyers. By flooding the market with such a big order in such a short period of time, there were not enough buyers to snap up all the inventory of stock futures up for sale. High-frequency traders were the likely buyers of the initial batch of orders. But between 2:41 p.m. and 2:44 p.m. ET they sold about two-thirds of the 3,300 E-Mini contracts they had bought in an effort to reduce risk. At the same time, there was not enough buying by large institutional investors, referred to as “fundamental buyers’ in the report, to fill all the sell orders. 5. Selling knocks prices down and causes liquidity to dry up. Selling due to the big algorithm trade by the mutual fund company, coupled with resulting selling by high-frequency and other traders, drove down the price of the E-Mini futures by roughly 3% in a four-minute span ending at 2:44 p.m. At the same time, other traders were driving down the price of an exchange traded fund that tracks the Standard & Poor’s 500 index, by 3% as well. With market volatility skyrocketing, the high-frequency traders began buying and selling E-Mini contracts between each other to limit their risk and exposure in a falling market. The SEC report refers to this trading as “hot potato”-type volume. This quick backand-forth trading volume, the SEC and CFTC say, only gives the illusion of adequate liquidity in the market, when the reality is that there are few buyers out there willing to put money to work. 6. Buyers of E-Mini futures dry up. With no buyers out there, the value of the E-Mini futures fell more than 5% between 2:41 and 2:45 pm. Sell contracts swamped buy contracts. The level of “net sellers” was 15 times larger than during a similar time period in the previous three days.
7. Five second pause brings back buyers. At around 2:45 p.m., a five-second pause in E-Mini trades was triggered, during which time sell pressured dropped and buying interest picked up. When trading resumed, prices stabilized and shortly thereafter began to recover. 8. Stock buyers disappear too. A brief pause in trading on many of Wall Street’s automated trading systems around 2:45 p.m. also gave individual stock traders a chance to figure out what was going on. The report said before trading resumed, traders had to determine if the sharp drop in the market was due to an erroneous trade or some cataclysmic event they were not aware of, and how big an impact the plunge had on their risk levels. Based on their assessment of heightened risk, many players, including high-frequency traders, withdrew from the market completely. The result: a sharp, sudden drop in prices of individual stocks. Between 2:40 p.m. and 3 p.m., 98% of all trades were done at prices with- in 10% of their 2:40 value. But with liquidity “completely evaporated” in a number of stocks, some trades only found buyers at “irrational prices” as low as one penny a share. 9. Buyers, armed with better information, resume buying. The crazy drop in stocks, some to as low as a penny a share, was fleeting. “As market participants had time to react and verify the integrity of their data and systems, buying and selling interest returned and an orderly price discovery process began to function,” the report said. 10. Prices return to pre-flash-crash levels. Around 3 p.m., most stocks had reverted back to trading at prices reflecting true consensus values,” the report said.
Weaknesses acknowledged But retail investors have not returned to the stock market in an aggressive way. They have yanked money out of stock funds ever since. Stock funds have suffered outflows every month since the flash crash occurred. While the report issued no recommendations on making sure this type of chaos doesn’t disrupt markets again, it did address some key lessons learned. It acknowledged that under stressed market conditions, the automated execution of a large sell order can trigger extreme price movements. Noting the “interconnectedness” of markets, the SEC and CFTC said they are working to come up with consis- tent circuit breakers that apply the same to all the different markets and trading venues. The SEC and CFTC also stressed in the report that they are trying to identify any “unintentional or potentially abusive or manipulative conduct” that may cause system delays and make it hard for investor to determine the price of securities in a fair and orderly way. “The report underscores a powerful need for better pre-trade back-testing of a strategy to analyze potential impacts,” says John Bates, chief technology officer at Progress Software. “Testing under realistic negative conditions might have stopped the firm from using the algorithmic approach used. It also highlights the need for better real-time monitoring of the trading process, using technology that is available but underused for pre-trade risk monitoring, market monitoring and market surveillance.” That a crash of this magnitude could happen so fast, with so few controls in place, shows just how important it is to “light up” (or make more transparent) the trading process, he adds. “Until this happens, I’m afraid, flash crashes remain inevitable,” says Bates.
USA Today 24
LESSONS FROM THE ALGORITHM-FUELED MAY 6 FLASH CRASH BY ALEX EICHLER DATE: OCTOBER 4, 2010 On May 6, the Dow plunged 1,000 points in less than half an hour. By the closing bell, the market had recovered to close about 3 percent down on the day, but investors, naturally, wanted to know why the “flash crash” had happened. Now, a new report from the Securities and Exchange Commission and Commodity Futures Trading Commission offers an answer: an automated trading algorithm sold off its company’s contracts too quickly, leading to a snowball effect that launched a panic. * How It Happened According to the SEC-CFTC report, the algorithm in question was designed to sell based only on market volume, not based on price or time. As a result, when the program took effect on May 6 “against a backdrop of negative market sentiment and thinning liquidity,” it “executed the sell program extremely rapidly in just 20 minutes,” touching off a full-scale market disruption. The report doesn’t name the company whose algorithm set off the flash crash, but it’s believed to be Waddell & Reed, a financial planning company based in Kansas. * Hedging Strategies Aren’t Reliable James Hamilton at Econbrowser points out that the sudden plunge exposes market vulnerabilities we usually ignore. “This episode serves as another reminder, as if we did not have enough already from the financial fireworks in the fall of 2008, that the faith many market participants have in their hedging strategies is misplaced,” he writes. “Aggregate risks can- not be hedged by the aggregate market ... It also should remind us that it can be a really stupid plan to sell regardless of the price. And it is still a really stupid plan even if it comes out of what somebody told you is supposed to be a very smart computer algorithm.” * Preventing Future Crashes The Atlantic’s Alexis Madrigal notes that some market implosions are caused by “bad actors”, but “simple incompetence... is a harder thing to regulate against.” Still, there are options: Madrigal quotes the CFTC’s John Bates on the need for “better pre-trade back-testing” of strategies like the runaway algorithm, and “better real-time monitoring of the trading pro- cess, using technology that is available but underused.” * The Fallout: Less Trust Than Ever At MarketWatch, Nick Godt writes that the market has become a riskier place in recent years, and “big periods of volatility, as took place in May... have been increasingly common and are likely to continue.” He concludes that “the funny feeling surrounding computer trading and its ensuing controversy, perhaps, is the uselessness of the system at the moment. It’s the big guys playing against each other and tripping over their highly efficient computerized trading programs — and they’re the ones getting hurt.” * A Failure to Communicate David Warsh, of the blog Economic Principals, sees the May 6 crash as indicative of a wider erosion of “civic liquidity” -- the simple give-and-take among reasonable people that keeps political discourse civil and Internet discussions flame-free. “The fragmented world of financial markets may be the lesser danger,” he warns. “Intransigence, the breakdown of political back-and-forth, probably is the greater threat.” * Blame the Terminator! The Big Picture’s Barry Ritholtz looks at contemporary markets and sees “a system without humans charged with maintaining orderly markets, with software bots swapping shares with other silicon-based life forms.” It puts him in mind of Skynet, the self-aware computer network from James Cameron’s Terminator franchise. “Instead of putting SKYnet in charge of national defense, we have put it in charge of our markets and economy,” Ritholtz writes. “The end result — minus the spectacular special effects — seems to have been the same.”
REINVENTING PROGRESS SOFTWARE—BOSTON’S NEXT BILLION-DOLLAR COMPANY? BY WADE ROUSH DATE: MARCH 24, 2010
Most businesspeople around Boston can give you a thumbnail description of anchor companies like EMC (storage devices and information management) or Nuance (speech recognition) or Boston Scientific (medical devices). But what does Progress Software do, exactly? Considering that it’s the largest softwareonly company headquartered in Massachusetts— with $500 million in annual revenues, 1,800 employees around the world, and 70 of the Fortune 100 among its customer base—the Bedford, MA, business has a remarkably indistinct profile. But Richard Reidy, who joined Progress (NASDAQ: PRGS) shortly after its founding in 1981 and became CEO in 2009, is out to change that. Under his leadership, the company’s disparate operating units and products—the legacy of 14 acquisitions and at least two big restructurings since the company went public in 1991—are being brought together under the slogan “One Progress.” That means not just eliminating redundant staff (the company laid off 250 workers, more than 12 percent of its workforce, back in December), but making it much more obvious to customers how the company’s products and services, which are designed to help companies build and run business applications, fit together. Reidy says he wants to double the newly consolidated organization’s revenues to $1 billion per year, and make it the go-to software provider for organizations seeking to improve their “operational responsiveness”—a bit of management lingo that boils down to knowing more, sooner, about changes in the conditions affecting your business and having the means to act on them. In a way, increasing this responsiveness is the whole reason companies buy enterprise software. But in practice, things don’t always mesh. You might think of Progress as the BASF of the enterprise software market. The German chemical manufacturer’s tagline is, “We don’t make a lot of the products you buy; we make a lot of the product you buy better.” Well, Progress doesn’t make a lot of the software that big corporations use, but it makes that software work better.
Its main specialty is “business process management” software, which ties together other systems such as financial, human resources, and supply chain management programs and makes it easier for managers to routinize repeated tasks and spot problems. Closely related to that is the company’s “complex event processing” software, which can spot patterns in business data that aren’t detectable by humans, allowing organizations in industries such as financial services, transportation and logistics, and telecommunications to respond faster to changing conditions. Finally, Progress is still strong in its original business, selling tools that help software developers build and debug business applications of all stripes. On March 15, Progress released a new product called the “Progress Responsive Management (RPM) Suite” that combines the company’s event processing and process management tools into a single platform, managed through a so-called “control tower” that presents business leaders with real-time alerts, performance indicators, and interactive tools. Reidy said in a statement that the suite, which represents a reshuffling and re-assembly of software components that came into Progress through the January acquisition of Savvion, among other deals, provides managers with “total control over their business.” Recently, Reidy and Progress Software’s chief technology officer, John Bates, visited Xconomy for an extended joint interview. The unification campaign provided the jumping-off point, but we also talked about the company’s history, its strategy for acquiring and integrating new companies, and the way its software can help customers in industries like finance and transportation. Along the way, I also got to quiz Reidy and Bates about the company’s role in the Boston high-tech ecosystem, and their take on trends like cloud computing and the mobile revolution.
Below is Part 1 of my interview with Reidy and Bates; we’ll publish Part 2 tomorrow. Xconomy: Progress Software has been around for a long time, but it isn’t an easy company to describe in a sentence or two. How do you sum up the company? Richard Reidy: If there’s a unifying theme through our history, it’s always been providing whatever a developer needs to build, deploy, manage, and integrate a complete, soup-to-nuts business application. Now, what a “business application” is has changed quite a bit from 25 years ago. As a result, Progress has evolved over 25 to 30 years to take advantage of the new types of business applications and environments. We started off in a dentist’s office—and I was there. The world of application development was very simple back then. You had a screen—and in those days it wasn’t even a graphical screen, it was a green IBM screen—and you’d put stuff in, you’d store it on disk, and you’d pull it out to process it or print an invoice. It was a closed environment. We started the company around building development tools that you could use to build an entire application. This was before Java, but it had the promise of Java: write once and deploy everywhere. You could be a three-person software company, hang out your shingle, and write software that would run on 50 Unix or PC environments. Since then, obviously there are new interfaces; client-server, the Internet, cloud computing, and Web-based software; and a whole bunch of different environments and requirements. Applications have matured, and the integration requirements have become much more complex. We plug all the various gaps, so that whatever you need to build, deploy, manage, and merge in business applications, you can get it from Progress. X: Describe how the company got to this stage, with so many separate parts that need unifying. RR: About 10 years ago we embarked on a diversification strategy, which culminated in us starting companies on our own but also acquiring other companies—14 of them so far. For example, we started Sonic Software, which pioneered the whole category of enterprise service clouds, which is now the standard way people interact with clouds. People think of it as Sonic, but we started it. Over the course of seven or eight years, we built up a collection of products and divisions and companies with different brands that were utterly unassociated with Progress. You had to carry business cards that were from Sonic or Data Direct. If you wanted to buy a Sonic or a Data Direct product you had to go to two different offices and call two different numbers for tech support. And we had doubled the size of the company to half a billion dollars a year, but now we want to get to a billion. To do that, we have changed our strategy. More and more, we are going to integrate and go to market as Progress Software, with a more complete set of products that can be bundled and sold together. We’re pretty much 80 percent of the way there now, and we are now beginning to proselytize and promote Progress Software as a brand.
X: Why the change in philosophies from diversification to unification? Was the old philosophy mistaken in some way? RR: No, it was for a very good reason, and I used to be the biggest separatist in the company. I ran one of those divisions, and I wanted nothing to do with Boston [i.e., headquarters]. For Progress, it was the right strategy at the time, based on the fact that we were trying to become best-of-breed in each segment. The market at the time was accepting of companies that did it this way, versus pulling everything together. I thought we did a good job, and one of the reasons we were successful is that we were able to keep a lot of these teams together. There’s nothing wrong with that approach, but it just isn’t the right thing for the next transformation we have to go through. Our size and scale is different, so there is a lot more to the synergies than we might have had 10 years ago. And the market has changed. People are not looking for best-of-breed, they are looking for solutions. So, I make no apologies. If there is anything we did wrong, it may have been waiting for too long. Maybe we should have started a year or two sooner. X: What do you want the Progress Software brand to represent now? RR: Going forward, okay, we’ve got all this stuff, now how do you simplify? We have a product that allows customers to get visibility into their infrastructure. We have other products that allow them to sense and respond to events as they occur. Then there is implementing the business processes themselves. If you have visibility, and you can sense and respond to what’s going on, and based on that you can dynamically change your business process, we call this overall benefit “operational responsiveness.” See, think, and act.
So those are three three categories of products: business transaction management, complex event processing, and business process management. We think that a combined category will emerge at some point in the future that includes all of these—visualizing, analyzing, and acting. The best name we have for it right now is “business event and process management.” I should be clear—you don’t have to buy all these products together. You can buy them separately and combine them. Progress Software’s strategy going forward is to be the company that brings operational responsiveness to business. X: How about some examples of how these capabilities would help specific industries? RR: You can think of a number of verticals—financial services, transportation, and logistics. One very hot trend in financial services right now is that everybody up to President Obama is talking about concerns about high-frequency algorithmic trading. The concern is that this puts a lot of institutions at risk. So one of the solutions that Progress has been working on with customers in the banking community is real-time market surveillance. This means actually detecting patterns that indicate insider trading and market abuse and being able to stop that before it affects the market. Traditionally, you might find out that something nefarious had gone on months afterward, by looking through a database with analytics software. But if this even moved the market by even one percentage point, think of all the money that has changed hands that you can’t unwind. So we’ve been working with regulators and banks to gain visibility on that data. First we have to see all the exchanges and the data flowing into them, to be able to sense and respond to patterns that indicate insider trading and market abuse. For example, was there anybody trading at greater than 300 percent of their usual volume in a public instrument within 5 seconds before a news article came out with some information? When you get that kind of data, you need to invoke business processes to automatically cross-reference it with past cases. Has this happened before? Is there some pattern to indicate this might not be just a one-off? You can create business processes to raise alerts and manage them and discover trends and patterns. I can also give you an example from the transport and logistics industry. We have been working with customers from all over the world. One is Royal Dirkzwager, a Dutch logistics company that manages the container ports in Rotterdam. There’s a lot of complex thinking going on there about the availability of berths and docks in the ports. But what has happened for the last 100 years is that a ship might radio ahead and say “Okay, I’m going to arrive at 2:00 p.m.,” and some crusty old geezer on the dockside with binoculars would look for the ship. There is tremendous waste for the shipping lines; let’s say you steam at full speed to make it to the port but then there isn’t a berth for you. You burned all that fuel unnecessarily. But you can gain visibility on events in ports, and on the allocation of ships. You can potentially track every ship in the world and relate it to the availability of ports and docks and manpower and resources, then you can send real-time course and speed changes to ships to get them to the right port at the right time and optimize the use of those resources. That’s saving millions of dollars every year for Royal Dirkzwager.
PROGRESSING IN REAL-TIME BY KYLE ALSPACH OCTOBER 29, 2010 In a decade, John Bates went from being a computer science professor in Britain to a C-level executive at a half-billion-dollar U.S. company. Now Bates is a key figure leading the company, Bedford- based Progress Software Corp., in what executives believe is a groundbreaking new approach to real-time management of businesses.
Boston Business Journal
Bates, 40, is chief technology officer and senior vice president for corporate development at Progress. His path to the position was unusual, but Progress CEO Rick Reidy said it’s no surprise Bates has been able to accomplish all he’s done. “John has the very rare ability to combine technology knowledge and vision with real-life cases,” Reidy said. “You can find people who are very technical and might have product vision, and you can find people who understand business and industry. But it’s a rare person that can bridge the two.” In 1996, Bates secured a tenured position teaching computer science at Cambridge University. But four years later he would leave, inspired to start a business based on research he’d done while studying and teaching there. The research looked at an area now known as complex event processing, which seeks to find ways to take real-time action in response to data from ongoing events. Bates’ company, Apama, found early success in the capital markets industry using the technology to detect pat- terns in trading. “As a small team, circa 25 people, we were winning accounts like JP Morgan, Deutsche Bank, those kinds of organizations,” Bates said. “They saw that we had something unique, and that we were meeting a burning business problem.” Progress Software (Nasdaq: PRGS) eventually saw the same thing. After a yearlong courtship that began in early 2004, Progress acquired Apama for $30 million in April 2005. And Progress brought on Bates as vice president of products for the division that absorbed Apama — the real-time division, which would later be renamed the Apama division. Bates would ultimately become general manager of the Apama division, until his promotion to CTO and corporate development head last December.
“This has been a great fit for me. I mean, I love running a business, but I also like doing the technology vision, evangelizing, exploration of new opportunities,” he said. Bates is now helping to pilot the next major phase for Progress in making businesses more responsive to real-time events. It’s called responsive process management, and it seeks to help businesses to dynamically adjust to events by detecting opportunities and threats as they happen. Progress likes to refer to it as a “business navigation system,” helping to guide a business the way a GPS can direct a driver. “Lots of firms do business intelligence — they record stuff that’s happened, and they analyze it offline for planning,” Bates said. “The key word there is offline. This is about online ... (Businesses) need to see things that are happening and respond to them immediately.” Key industries Progress is targeting with the technology include financial services, telecommunications and transportation, such as the airline industry. Airlines, for instance, might use responsive process management to ensure that airlines avoid having passengers sitting on a runway in the event of something unexpected, Bates said. In the financial world, the technology is being used to detect market abuse and insider trading as it’s happening, he said. Giles Nelson, who co-founded Apama with Bates and is now deputy CTO at Progress, said Bates has a long record of “visionary” ideas. But just as importantly, Bates is courageous when it has come to implementing the ideas, Nelson said. “He’s often done things which have really pushed him in directions in which he was naturally uncomfortable,” Nelson said. “But he kept going and was tenacious. As part of his DNA, he wants to succeed in whatever he’s doing.” Bates has had to acquire business and financial savvy over time, however. His family has no history in business — “they still to this day don’t know what I do,” Bates said — and he has no MBA. “Although that would’ve been extremely helpful,” he said with a laugh. Reidy said you’d never know that Bates was self-taught when it comes to the financial world. Reidy recalled a dinner he attended with Bates and two financial industry experts, in which it was clear that Bates was as “deeply immersed in their industry as they are.”
Boston Business Journal
Bates also possesses a broad knowledge of world affairs, politics and many other subjects, Reidy said. “Perhaps his British accent just makes him sound more learned than most, but it works,” Reidy said. Despite living away from his home country, Bates said he’s relished life in New England. A favorite activity for himself, his wife, Jill, and 3-year-old daughter, Eleanor, is visiting a home Bates bought on Kezar Lake in Maine
PROGRESS SOFTWARE’S STRATEGIC SHIFT ROB LEVY, EXECUTIVE VP AND CHIEF PRODUCT OFFICER FOR PROGRESS SOFTWARE TALKS ABOUT HIS COMPANY’S MOVE AWAY FROM VERTICAL PRODUCTS IN FAVOR OF INTEGRATED SUITES. DATE:OCTOBER 29, 2010
DATE: OCTOBER 30, 2010
RESPONSES TO THE HIGH-SPEED CHALLENGE DATE: NOVEMBER 16, 2010 A few weeks ago Jeremy Grant, editor of FT Trading Room, offered readers a high-speed challenge: <http://www.ft.com/cms/s/0/ a98d6fd2-e01a-11df-9482-00144feabdc0.html#axzz155tcqDqH> The arms race in trading speed is now absurd. Of course, we know that it all has to do with high-frequency trading strategies. But to the average punter it beggars belief that trading at these speeds a) matters and b) is good for capital formation generally. I have yet to hear a decent explanation from an industry practitioner for this A number of industry experts responded – and thanks to everyone for taking the time to send them in. After an exhaustive, latency-sensitive analysis of the responses, the best reaction received was from Tim Edwards, Remco Lenterman and Robin van Boxsel at IMC Financial Markets in the Netherlands. Here’s what they had to say: “Essentially, a market maker provides liquidity, which we believe is positive within financial markets, if not a fundamental condition of them. Speed is an essential tool for market makers to manage risk by controlling the amount of time that their quotes are placed on an exchange. This is commonly referred to as exposure time. For every quote in the market that a market maker provides, they are exposed to that quote for the time it takes for a cancellation to be processed, or the time it takes to remove the exposure following a market move or a move in a related instrument. Basically the higher the speed, the lower the time between when information is received and the time when such information is incorpo- rated into prices. For any given order, the value of this fraction of a second exposure is very low. However, across an entire market venue, this adds up to very large numbers.
In the cases where exchange speeds are high, it enables market makers to manage their risk better and therefore they are willing to quote narrower spreads and for bigger size. The relationship between speed, spread and liquidity is evident on many exchanges and clearly adds value to all participants. It is clear that in the past 10 years, major markets have become substantially more liquid with narrower spreads and lower transactions costs. Speed has played an essential role in this development. A case in point is that markets that continue to have slow systems and/or low bandwidth have very little displayed liquidity (Hong Kong, Australia, Osaka Stock Exchange) and that in order to facilitate increased liquidity, they need to change these systems, which is happening. The role of an exchange is to supply a venue for buyers and sellers to access securities. This is the core of capital formation, spreading risk from those who cannot bear it to those who seek it. Market makers even out the distribution of risk through portfolio management, and give liquidity where there is not a natural counterparty. Exchanges are providing exactly the service we expect of them by increasing speeds and lowering order acknowledgement times, aiding market participants to provide the liquidity we expect to be available in a market centre. We hope we have been able to shed some light on this subject.” Tim Edwards, Remco Lenterman and Robin van Boxsel at IMC Financial Markets. We also have a couple of runners-up, from Hirander Misra, chief executive of Algo Technologies and Giles Nelson, deputy chief technology officer at Progress Software.
Here are some edited excerpts: “Fast trading technology is being used to level the playing field by providing transparency and efficiency to a public market. A faster exchange-matching engine acts as an enabler and assuming the business model stands up, liquidity providers will be more attracted to the fastest platform as it reacts quicker to price movements when they update their quotes. As a result, they also have less chance of being picked off by arbitrageurs than those on a slower platform when the same instrument is traded across platforms. This is one of the primary reasons why Chi-X Europe gained market share from the LSE and fared better than Turquoise and now with the LSE Group upgrading their systems the scenario could play out in reverse.” Hirander Misra, chief executive of Algo Technologies “I would suggest that you think of high frequency trading as being just the latest stage in the evolution of electronic trading. And this, as you know, has evolved very rapidly over the past decade because of cheaper and faster computers and networks. It’s led to many innovations and benefits: electronic crossing networks, algorithmic trading, online retail trading, smaller order sizes, the overall increase in trading volume, more price transparency, greater trader productivity, more accessible liquidity, spreads between buy and sell prices tightening, broker commissions reducing, competition between exchanges and so smaller exchange fees – none of these things would have happened without electronic trading. Mifid couldn’t have happened; it simply wouldn’t have been financially viable for the many alternative European equity-trading venues to launch without cheap access to networks and computers. Without these we would still have greedy, monopolistic exchanges with high transaction prices.” Giles Nelson, chief technology officer, Progress Software Click here to read Giles Nelson’s full reply <http://apama.typepad.com/my_weblog/2010/11/a-postcard-to-jeremy-grant.html>
Financial Times 34
WHY IT NEEDS TO BE REAL-TIME (AND MUST NOT BE SECRETIVE) DATE: DECEMBER 15, 2010 Each week we are asking chief technology officers and other high-profile tech decision-makers three questions. Answering today is Dr John Bates, chief technology officer of Progress Software, a company that specialises in real-time computing and provides business software infrastructure to more than twothirds of the Fortune 100 companies. Progress Software is a US firm based in Bedford, Massachusetts, and employs 1,800 people. It’s annual results are imminent, but during its most recent financial year the company made a $46.3m profit on a turnover of $515.5m (£330.1m). What’s your biggest technology problem right now? The biggest challenge for technology firms is to make themselves responsive in real-time to the business opportuni- ties at stake and the threats that could turn into disaster. For example social networks, this could turn into a real IT must provide visibility to humans: For example, how effective are business promotions? If you talk to a customer, where are the opportu- nities for up-sell? And when you have visibility, how do you respond? It could be an automated response, for example algorithmic trading or automated supply chain logistics, or information that guides call centre operatives. Automated real-time responsiveness requires a real-time safety net, a real-time firewall. This year we’ve had dozens of [banking industry] algorithms going wrong, causing multimillion dollar losses around the world. If you do high-frequency trading you need high-frequency risk management - constantly monitor your exposure, prices, volume spikes. You need to be able to give feedback to risk and compliance officers. For many companies - banking, energy, telecoms, logistics and others - responding to real-time information is now business critical, but many companies are still figuring out what to do. [Real-time systems] won’t be able to boil the ocean and discover everything, but systems are able to learn. This is not going to be the all-thinking solution, but if you go from no visibility to some visibility, and see the bottlenecks and problems, that’s worth something.
What’s the next big tech thing in your industry? Location - making better use of location-based services. Telecoms firms will be able to sell messages to consumers based not just on loca- tion but the context of the customer - but that has to be on an opt-in basis [where customers agree to get such messages].
For example, we are working on a service for a major entertainment group, for its amusement parks, that will give visitors a customised experience. The system knows where you are, which rides you want to visit, it monitors the length of the queues at the rides and then will make recommendations where you may want to go next to personalise and optimise the experience. Lufthansa has a location-aware social networking system, you can tell the system what your interests are, for example an investor who wants to meet entrepreneurs, and the system will bring them together. The other thing are social networks. They will become business tools, and could even spell the end of email. If you treat your organisation like a social network, not like an org chart, you may find that the most important person in your firm is Fred the engineer who sits towards the bottom of the org chart. What’s the biggest technology mistake you ever made - either at work or in your own life? I’ve made lots of mistakes, but the biggest mistake was when I was working on my first company. We went into stealth mode and didn’t talk to anybody about our project. Instead we should have shouted it from the roof tops and got the customers on board as soon as pos- sible and get their case studies into the development. Now, when we develop something new like our responsive management system, we get customers on board early. So I hope I’ve learned my lessons from the mistake of being a secretive squirrel. below £5 million annually from the coming year.
THAT’S PROGRESS: FIRM RAISES CURTAIN ON ACT III BY D.C. DENISON DATE: DECEMBER 27, 2010
The Boston Globe
When Richard Reidy, chief executive of Progress Software Corp., addressed 25 newly hired sales people earlier this month, he called it a company in transformation. Reidy described Progress as standing at a crossroads. Over the past 30 years, Progress has grown into one of the largest technology companies in the state by selling middleware, the crucial software that ties together the different tech- nology products corporate customers use. Progress’s airline customers, for example, use its software to connect reservation systems with baggagehandling operations and frequentflyer programs. But Progress is in a business that is dominat- ed by what Reidy calls the big three — IBM, Microsoft, and Oracle — and that is changing fast. And the company will have to change with it.
line of products before the legacy sales fall too far.
Progress makes a lot of its money from “legacy’’ products — updates and packages for longtime customers. But that revenue stream is declining, and the company will have to replace it with a faster-growing
“If Progress can get the lines to cross, the strategy will be a success,’’ said John R. Rymer, an analyst at Forrester Research Inc. in Cam- bridge. “It’s not an exaggeration to say that they are relaunching the company.’’ So Reidy is rebranding the company, combining most of its products and brands into one “suite,’’ and creating a more coherent marketing and organizational plan. “We’ve been very successful selling itty-bitty components, but now we need to combine all those products into one brand,’’ Reidy said.
In sales materials, Progress no longer even uses the term middleware. Instead, the sales team is selling its software products under a uni- fied concept called Responsive Process Management (RPM), which the company describes, broadly, as the ability to help customers see what’s going on in their business processes at any given moment and act quickly to fix problems, control processes, sell more, or cut costs. A consumer goods company, for example, could use RPM software to monitor competitors’ prices and promotions in real time and react quickly against competitive threats. A shipping company could use it to advise its vessels on the most efficient course to take, based on weather and tide conditions. The new brand was officially unveiled at a meeting with industry analysts last spring. Reaction appeared positive; the company’s stock price rose steadily during the last three months of 2010, from around $30 a share at the end of the summer to more than $40 in December. If Reidy’s transformation succeeds, it will be the company’s third act. Founded in 1981 to sell a single software product, Progress started expanding significantly in the 1990s, acquiring smaller companies and adding products, while making inroads into industries like banking, insurance, and travel.
The Boston Globe
Today, Progress sells as many as 15 software products that are used by nearly 140,000 organizations in more than 180 countries, including 70 percent of the Fortune 100. The company has more than 1,600 employees worldwide, including 508 in Massachusetts. Last week, Progress said year-end revenues were up 7 percent over the previous year, to $529.1 million. One challenge is that Progress’s products are better known under their own names than the company’s. At the sales meeting, Reidy said that to grow annual revenue to $1 billion from $500 million, the company has to go from having the best technology in a number of dispa- rate products to “a more valuable combination’’ under one brand. The change at Progress “is much bigger than just a rebranding,’’ said Maureen Fleming, an analyst at the Framingham technology firm IDC. “A lot of enterprises buy software that runs your business as usual. Progress is trying to create a new level of responsiveness, where companies can identify the first moment of opportunity, the first moment where you can make a difference.’’ “Up till now, we’ve been the best in a wide variety of areas, but our products were all sold as independent brands,’’ Reidy said. “From now on, we want to succeed as Progress Software.’’
PROGRESS SOFTWARE IN HIRING MODE BY KYLE ALSPACH DATE: DECEMBER 28, 2010 Progress Software Corp. (Nasdaq: PRGS) expects to hire an estimated 50 new employees for its Massachusetts operation in 2011, as the company expands deeper into the market for real-time business management software, CEO Richard Reidy said. “As a company overall we’re definitely in hiring mode,” Reidy said in an interview.
Boston Business Journal
The Bedford, Mass.-based company employs 508 in the state and 1,600 worldwide, and expects to add a total of about 200 more worldwide employ- ees next year, he said. Hiring in Massachusetts will be across the board, from R&D to corporate office staff, Reidy said. The company’s Responsive Process Manage- ment (RPM) suite is a major force behind the expansion, he said. RPM aims to help businesses to dynamically adjust to events by detecting op- portunities and threats in real time.
markets, telecommunications and transportation, such as the airline industry.
Key industries Progress is targeting with the technology include capital
Progress unveiled RPM last March, so 2011 will be the company’s first full year focusing on the product, Reidy said. “We’re betting the farm on it,” he said. RPM should be the primary driver of the 20 percent revenue growth that the company’s enterprise business solutions portfolio expects to see next year, Reidy said. Overall, Progress is forecasting revenue of between $560 million and $570 million in 2011, representing 7-8 percent growth over 2010. The company is aiming to become a $1 billion company within about five years, Reidy said. Shares in Progress rose last week after the company announced a three-for-two stock split, along with a 28-percent boost to its profit on quarterly revenues of $145.2 million.
REGULATORS CAN’T DEFEND MARKETS DATE: DECEMBER 8, 2010 LOCATION: HTTP://WWW.CNBC.COM/ID/15840232/?VIDEO=1688540443&PLAY=1 The regulators don’t have the tools to tackle problems such as insider trading and flash crashes, Dr John Bates, CTO of Progress Software, told CNBC Wednesday.
ECONOMIE, FINANCES, BANQUES, ASSURANCES JUNE 14, 2010
Comment démasquer un trader malveillant ?: Pourquoi des systèmes de surveillance peu performants laissent le champ libre à des traders malhonnêtes, par Dr. Giles Nelson de Progress Software “ Il y a quelques semaines une nouvelle malveillance défrayait la chronique, provoquant à la Bourse de New York un krach boursier vertigineux pendant plusieurs minutes. En effet un trader travaillant dans une salle de marché d’une des plus grandes banques, veut passer un ordre de quelques millions de dollars mais il se trompe et tape “milliards” au lieu de “millions”, et clique sur la valeur d’un géant des produits de consommation aux Etats-Unis. En quelques nano-secondes, les actions du groupe chutent de 37%. Cet exemple n’est qu’un cas parmi d’autres. Voici quelques cas de malveillance : le frontrunning, où le courtier place ses propres ordres avant l’ordre d’un client. Il peut ainsi générer un profit sans risque. Le trading pré-organisé, où deux parties s’accordent sur une heure pour acheter et vendre un produit, jouant ainsi artificiellement sur la demande et les prix. De nombreuses opérations sont considérées à tort comme du “trading corrompu”. Il est donc important d’identifier les pratiques qui n’en sont pas. Le “trading haute fréquence” ou HFT consistant à utiliser la technologie pour surveiller et soumettre des ordres, est décrié et est parfois perçu comme une pratique abusive. Mais pas plus abusive, que deux traders pré organisant une transaction téléphonique. Oui, même un simple combiné peut être l’outil d’un “trade corrompu”. De même, le trading algorithmique est perçu à tord comme une malédic- tion. Le Crédit Suisse a été récemment sanctionné pour avoir perdu le contrôle de son système, et diffusé plusieurs centaines de milliers d’ordres erronés. Pourtant, il ne s’agissait pas d’une tentative de manipulation mais d’une simple erreur humaine. Aucun contrôle n’existait pour protéger le marché, les algorithmes n’avaient pas été assez testés. Trading algorithmique et haute fréquence présentent d’énormes avantages : performance accrue des traders, meilleure liquidité du marché, spreads plus serrés et meilleurs rendements. Perdre cela pour une simple question de perception s’avèrerait dangereux. Les technologies ont considérablement accéléré et complexifié les marchés. Tous les acteurs doivent pourtant s’adapter pour que l’ensemble fonctionne.
Trop peu, trop tard Le fonctionnement et l’activité des marchés ont évolué. Le volume total des ordres a significativement augmenté, tandis que la taille des ordres individuels diminuait de moitié ces cinq dernières années. Dès lors, les places boursières et organismes de régulation sont tenus de surveiller en temps réel les marchés afin de détecter tout comportement suspect. Des technologies combinées de Business Intelligence et de traitement des événements sont en cours de déploiement, afin d’offrir des fonctionnalités de recherche et d’analyse plus performantes et ainsi empêcher tout comportement malveillant. Une complexité accrue La prolifération des plates-formes de trading complexifie le travail des organismes boursiers. Prenons le cas des titres européens “post-MiFiD”, qui ont permis la création de plates-formes de liquidité alternatives. Auparavant, les liquidités étaient concentrées sur les places boursières nationales. Désormais, les courtiers peuvent acheminer un ordre d’un client vers plusieurs plates-formes. Une place boursière n’a donc plus une image globale de la transaction. Les banques doivent assumer davantage de responsabilités con- cernant la surveillance des flux circulant sur le marché via leur intermédiaire, ou générés par leurs propres activités de trading. Elles subissent des pressions importantes des organismes de régulation, afin d’avoir une vue continue et en temps réel de leur business.
Ramener la confiance sur les marchés Comment redorer le blason des institutions financières et de leurs traders ? Deux options : la première, les gouvernements et organ- ismes de régulation introduisent un grand nombre de règles afin d’essayer de réduire les “comportements malhonnêtes”. Cela se traduira inévitablement par la prise de mesures autoritaires contre le HFT et le trading algorithmique, entraînant une perte de produc- tivité. Les traders malveillants seront également en meilleure position pour utiliser ces règles à leur profit. La deuxième option, la meil- leure selon moi, consiste à ce que les organismes de régulation améliorent encore la transparence et l’ouverture des marchés grâce à une technologie adaptée. Des logiciels de surveillance en temps réel sont en cours de déploiement dans les salles des marchés, les places boursières, ainsi que par les organismes de régulation. Ces outils surveillent l’activité de trading en temps réel, et détectent immédiatement les abus “.
PROGRESS SOFTWARE CORPORATION 14 OAK PARK DRIVE BEDFORD, MA 01730, USA TEL: 1-781-2800-4000 TOLL FREE: 1-800-477-6473