Hacker News new | past | comments | ask | show | jobs | submit login
Oil crash busted broker’s computers and inflicted big losses (bloomberg.com)
204 points by wclax04 on May 8, 2020 | hide | past | favorite | 176 comments



“Five days, including the weekend, with the coronavirus going on and a complex system where we have to make many changes, was not a sufficient amount of time,” he said. “The idea we could have bugs is not, in my mind, a surprise.” He also acknowledged the error in the margin model Interactive Brokers used that day.....We have called the CFTC and complained bitterly,” Peterffy said. “It appears the exchanges are going scot-free.”

Thomas Peterffy must think we are idiots. Anyone who trades commodity contracts for any period of time knows that the real cost of the contract is the actual cost of the commodity - storage costs. When storage costs spike and the actually commodity spot costs go down, the future will become negative!

One way to get a handle on storage costs is think of them being inversely proportional to the value density. The higher the value density, e.g. gold the less the storage costs. Oil is not so dense so storage costs matter. Financial instruments like the Treasury Bonds and the S&P futures contract have zero storage costs. Storage cost is of-course different than carry cost (the cost of funding your long position).

On another aside, I have known folks who have worked at IB in the past, and their systems absolutely suck dead goats. Huge masses of legacy C++ code with poor testing. Most of these brokerage firms have legacy code base from the 90s that is poorly understood. They also have nonexistent organizational quotient around code validation, correctness and testing their risk models. A futures margin model is not something one can whip up over a weekend but a good CS undergraduate can program one over a couple months.

Sorry for the IB customers but I have zero sympathy for IB or should I say negative ;)


Peterffy is talking out of his ass, they had more than 5 days. CME's current specifications for order entry/market data has been in place for years now. Whoever was writing their software on their backend failed to read the documentation.

https://www.cmegroup.com/confluence/display/EPICSANDBOX/Posi...


While I don't doubt that what you say is true, systems which, by spec, should be able to handle a given situation, that then years after they were written finally are called upon to handle that situation, fail more often than not.

The CEO also said they would pay $100million to clean up, which is not chump change, even for them I bet.

I'm not saying they were blameless, but I think most systems which have never had a given variable go negative, that years after they were written finally have that variable go negative, will have problems.


Nat gas prices went negative pretty famously (in the biz) around 2010 or so. I give everyone a pass based on what you said until then. After that no free passes. IB screwed up.


They are not the first people on the planet to encounter edge cases. You either test thoroguhly enough for them or you don't.


CME requires certification before you can deploy a connector to them.

If they really cared or if this was as obvious as everyone here is claiming it is, negative price support would be part of their certification suite. It is not.


> On another aside, I have known folks who have worked at IB in the past, and their systems absolutely suck dead goats. Huge masses of legacy C++ code with poor testing.

As a customer I’m not surprised at all. Their software is terrible, especially the simplified web application and their mobile app. Terrible. But their fees are incredibly low. You can convert a million dollar to other popular currencies for less than 20$ at interbank rates. Just nuts.


At interbank rates? Ok, that is _pretty_cool_. I will check that out, thanks!


Thanks for the information. I have been using IB and their API and had always wondered about the quality of their code base. I assumed there was a lot of legacy with their systems - I just had little idea of how well they were managing things internally.

For my limited requirements, Interactive Brokers does seem to meet needs, but do you know of a platform that is highly regarded that is accessible for clients outside of the States?

Interactive Brokers seemed to tick a lot of boxes at the time when I chose them.


The outside the US part, I think, will greatly narrow your universe of choices. Have you looked into Lime Brokerage? They are now owned by Wedbush but they used to have top tier technology.


No, it is the first time I have heard of them. I will check them out.

Cheers


Thanks for pointing it out, I looked at Lime, but the pricing is prohibitive for the amount of shares I would be trading. From what I can tell you would be looking at about $2,000 or more in fees per month.


This is not really an issue of whether prices can go negative though. It'ss whether IB supported negative prices, and whilst it's true the drop happened quickly, this was a known likelihood for a few weeks. The fact that IB kept letting people go long at positive prices, knowing they couldn't execute stop loss trades at negative prices is frankly terrible.


You couldn’t sound more clueless here. A rare screwup by IB for sure but the only reason you’re hearing about it is because the CEO is a standup guy. (Very sharp one too.) Other brokers would no doubt sweep this under the rug and/or pursue their customers for the losses.


You sound like an employee ;)

On a serious matter, doesn't Peterffy owns a huge chunk of IB and a brokerage is as riskless of a transaction business as you can get to in the financial industry as possible.

IB's frontends used to lead the market and their pricing is decent too. Who else lost $100 million because they did not program futures and span margin properly?


Not an employee but a customer for about 16 years. Not sure their front end ever led the market and they’re far from flawless. But they’re one of a very small number of ethical outfits in a very scummy industry.


Same here, customer for 18yrs. In 2002, $1 trades with honest exchange routing was unheard of.


Spreads have negative prices often enough


Agreed. IB's data is also not so accurate much of the time; the way it reports volume in particular is misleading. I've seen traders abandon IB after big losses because their strategies required actual live volume data that didn't suck, and nearly any other broker provides this.


What are some recommended alternative platforms that provide api access?


I know a few who moved to tradestation. But there are others as well.


> Peterffy said there’s a problem with how exchanges design their contracts because the trading dries up as they near expiration. The May oil futures contract -- the one that went negative -- expired the day after the historic plunge, so most of the market had moved to trading the June contract, which expires May 19 and currently trades around $24 a barrel.

> “That’s how it’s possible for these contracts to go absolutely crazy and close at a price that has no economic justification,” Peterffy said. “The issue is whose responsibility is this?”

It’s pretty well known that commodity futures contracts are a game of hot potato for most investors as the expiration date approaches. But the Interactive Brokers CEO doesn’t offer an alternative solution. How would the contracts be structured instead that would avoid this? I don’t see how it would be possible.


The other issue here is that there IS an economic justification: everybody bid up the price of storage. If it costs me $60 to store a barrel I can sell for $10 later, then I'll pay someone $40 to take it off my hands now. An asset became a liability, hardly a rare concept to anyone who has owned a car they had to pay someone to tow away... CEO is just passing blame.


Interesting point. Have we commoditized and securitized the storage of commodities yet? Can someone purchase oil storage futures contracts to mitigate this risk?


Go betweens exist and on Planet Money one was interviewed: https://www.npr.org/2020/04/22/842095406/episode-993-negativ...

Their website: https://www.thetanktiger.com

> The objective of the Tank Tiger is to serve as a clearinghouse and a single point of contact for parties who desire terminal tank services or utilization of midstream assets and to bring them together with the companies who own these assets and seek further utilization. It also provides for tenants who are leasing storage to find subleasing opportunities, when it makes sense for them to do so. Our expertise in this field provides a fast and efficient connection so that the uncertainty of storage availability is eliminated from the supply and trading equation. The Tank Tiger can facilitate and reduce market inefficiencies and illiquidity by providing this service.


Yep, but it's worth pointing out that this firm just connects you up with people and you have to deal with individual storage owners to buy something. Very high friction to firing up your brokerage app and going long /CL.


I think there might be a market for that but probably not as formal as the trading of oil futures themselves. Basically a more private transaction on a marketplace that isn't as visible or as liquid as the futures.


OTC-traded contracts are just as formal. I’d say that these markets are generally more formal in practice due to the lack of retail actors who don’t understand the microstructure, sources of risk, or how to price their assets individually and across their book.


You can buy FFAs


I've only recently started learning about how trading works in depth so I'm probably way off, but isn't he just avoiding the obvious answer? The broker and the clearing house. I thought that's a large part of why we have them? It just so happens that counterparty risk includes handling of massive amounts of physical goods so they'll have to charge larger commissions to cover the additional risk on the contracts. It would probably reduce the number of speculators at the same time, reducing the risk of this happening to begin with.


When the price mayhem was happening there was a discussion here on HN, and someone claimed that brokers would either never let retail investors handle contracts with physical delivers, or forcibly close the positions several days before the deadline.

Looks like Interactive Brokers fucked up in more than one way here.


Don't know why you're being downvoted. Physical delivery IS a third rail for retail investors and brokers should absolutely have a field in their settings page that authorizes them to close the position x days or hours before maturity.


There are ETFs that track oil futures (basically like a stock, but backed by oil instead of a company). It's been a while since I've looked at any of this, but I think USO is still the most prominent.

There are plenty of things to watch out for with these ETFs. You pay ongoing expense fees. And ETFs, especially those that aren't just holding containers for assets, can have subtleties in their prospectuses that cause their value to fluctuate in counterintuitive ways. There's still a lot to be cautious about.

However, compared to the actual futures, they're more suitable for casual investors, for reasons such as what we see here. They can't go below 0 and don't necessarily involve margin. And the ETF will typically deal with things like rolling the futures position ahead of expiry.


Sorry but this is terrible advice. If trading Oil Futures is akin to playing Russian Roulette then trading Oil ETFs is akin to juggling live hand grenades. One will go off as soon as you stop!

Most commodity and leveraged ETFs are designed to benefit just one party - the designer of the ETF. There are plenty of articles on USO and its travails.


I'm not recommending USO or oil itself as an investment, and I agree that you will be paying money to the ETF manager if you get involved in it.

But I don't agree that USO is _more_ dangerous for a casual trader to trade than oil futures, for the reasons I mentioned.

Removing the overall fluctuations of the oil market, the relative problem with ETFs is that they bleed away value over time. That's a real issue, but I wouldn't compare that to juggling a live hand grenade.

Edit: you did not say it was more dangerous, my mistake. I do think that ETFs are less dangerous, for the reasons I mentioned.


I owned USO and sold it on 4/21. I lost money of course. It seems USO could never go negative according to what you said and I could only go to 0 but I learned my lesson to stay out of futures for good


If I just wanted a ticker symbol for the price of oil to put on a 'market health' dashboard, with no intention of actually investing in the ETF at all, would USO suffice?


No. USO is not symptomatic of the health of the Oil market [1]. USO is a bizarre beast and any analogy fails to explain it. The only thing indicative of the health of the Oil market is spot and futures prices. [2]

This, unfortunately (or for some, fortunately) requires understanding the structure of the Oil market and how Oil is bought, sold, delivered and transported and yes stored! Spot prices, futures, oil grades, hubs, contango, etc.

[1] https://www.bloomberg.com/news/articles/2020-04-30/investors... [2] https://oilprice.com/oil-price-charts


You probably want the front month WTI and possibly Brent futures price.

USO has unpredictable drift due to contract rolling and often trades at a significant premium/discount to NAV.


> And the ETF will typically deal with things like rolling the futures position ahead of expiry.

This listed as making commodity ETFs more suitable for "casual investors" is the exact reason why they always lose money on ETFs. Retail investors for the most part do not understand contango or backwardation and do not understand (even though it's listed at the beginning of every prospectus) that these are not buy and hold instruments. In fact, I'd argue that it's easier to understand roll costs by actually having to roll futures contracts yourself (which is not difficult at all) vs having it obfuscated away in an ETF.


Definitely no. ETFs merely hide the fact that the underlying are futures contracts. ETFs make these futures contracts seem like stocks so any Tom Dick or Harry thinks they understand it and buys them. It's a very leaky abstraction.

> They can't go below 0

One month ago people know that futures can't go below 0. What guarantees ETFs will never go below zero? It's economically absurd to think an ETF holding negatively priced assets will still have positive value.

I would argue that if an investor isn't knowledgeable enough to trade an underlying, the investor shouldn't trade an ETF of these.


The asset value underlying an ETF can be negative, but then the owner of the ETF will just have a worthless piece of paper. The ETF managers will have a problem on their hands regarding the negative amount, though.

In contrast, a futures contract is an agreement to make a future trade, so it can keep going against you past 0. If you are able to take physical delivery, your worst-case scenario is that you pay the money you said you would, and you get your oil. But if you are a casual day-trader type, you probably don't have the ability to take physical, so you may be in trouble (over a barrel, literally).

I agree with you about the dangers of ETFs and about knowledgeability. I didn't mean to advocate for ETFs on an absolute basis, just to make a relative statement about them vs. futures.


And as you'd expect ETF managers "fixed" this by shuffling things so that the "same" ETF is now backed by a different mix of futures they're less worried about.

My opinion is that ETFs are legalised bucket shops and ought to go away. The Oil Futures market represents an actual need, Alice wants to sell her oil knowing what she'll get for it when it's delivered in a week's time, Bob wants to be able to lock in today's prices for next week's oil. There is clearly a deal to be made there and if people who don't actually need oil want to tie themselves up in it and maybe improve liquidity I guess I won't stop them. But Oil ETFs are just a way to gamble on the value of the Oil Futures, and that's why we forbade bucket shops. Negative future prices show that the mechanisms which are supposed to make ETFs safer than bucket shops are flawed, and IMO too flawed to continue to accept them as a legitimate business.


> It’s pretty well known that commodity futures contracts are a game of hot potato

Very much so. I wrote software for financial traders in the 1990s, and I heard tell of a couple of clerks (in this context, sort of "trader intern") who thought they were smart enough to do a little commodity metal trading on the side. However, they didn't quite understand the details of contract expiration, and so supposedly they ended up with 25,000 pounds of copper delivered to somebody's parents house. Oops!


There are contracts that specify physical delivery, but they also specify the warehouse or storage facility the commodity will be delivered to. The buyer can either collect the commodity or pay the storage facility to hold it for them. It's not likely it would be delivered to someone's house unexpectedly, because someone would need to pay for the additional cost of transportation.


Yeah, the way the story was told the thought they could save some money and the hassle of making a deal with a warehouse. Why pay for something they're never going to use? So they agreed to pay for transportation in the event of delivery, thinking they'd never get charged for it. But I could well believe this was a trader urban legend.


A source to back you up, this is precisely how it works. The CME link goes into more detail on how physical delivery works for base metals.

Other physically settled commodity contract have different delivery locations, /CL (WTI crude) delivery takes place at a terminal and storage facility in Cushing, OK.

https://www.cmegroup.com/education/courses/introduction-to-b...


Oil and gas also have the benefit of published prices for most pipeline transport, although you still need a buyer/terminal that's physically connected to the network to take delivery.


That's fake. Delivery doesn't work that way. The contract will specify one or a few acceptable delivery locations, always some industrial shipping depot or something.


It certainly could be apocryphal, but ....

I thought the commodity was coal. And the issue was that the trader was at a fancy new office park with a river view that was an old dock specifically for coal.


There is this old story in the same vein (and of course also [citation required]): https://thedailywtf.com/articles/special-delivery


Sorry, but this story is utter bullshit. Source is unreliable and the only link in the story is to the movie "Trading Places" and the maybe the "WTF Stock Exchange" might be a clue.


Sounds apocryphal.


Could be! It was definitely a friend-of-a-friend territory. But often legends of failure are used to warn against real dangers. Little Red Riding Hood is surely fictional, but wolves in the woods definitely weren't.

And at least in Chicago in the 1990s, traders and clerks could definitely be wild. During slow periods on the CME floor somebody would get a transparent trash bag, declare it a $20 bag, and then walk around with it. People would write their names on $20s and throw them in. Once they'd made the rounds, they'd shake the bag and have somebody draw from it. I myself saw thousands of dollars change hands like this. And clerks would regularly bet one another about jumping into the Chicago river from Upper Wacker, which is a fair drop.

Or there was one incident that was witnessed by a couple of our traders, as it happened in their pit at the CBOT. One afternoon among all the colored coats, they see somebody in a polar bear costume walking around the trading floor between the pits. One trader turns to another and says, "I'll pay you $100 if you punch the polar bear." The trader thinks about it, takes the $100, goes over, lays out the polar bear, and then in the chaos goes back to his pit. This turned out to be a bad trade, as the polar bear was there as a fundraiser for the Lincoln Park Zoo, and one of the people on the zoo's board was also on the exchange's board. Oops!


Also told on Page 34 of Ron Insana's Traders Tales: https://books.google.com/books?id=NyeVNsWvJHAC&pg=PA32&lpg=P...


Holy moly! Yes, that's exactly when I was working there. Glad to know there's some confirmation for that one!


Everybody has heard of this happening to their friend's cousin's ex's colleague. In practice you can't trade futures without a broker, and your broker won't let you take futures to delivery unless you convince them you have the facilities to deliver or take delivery. And if you do take delivery of copper futures on the main exchanges (CME or LME), you get "warehouse receipts" which entitle you to turn up at some industrial estate and collect your copper, and which can be resold.


Financial settlement against a spot market in a large port.


WTI (West Texas Intermediate, the one that went negative) is already settled at a specific place: Cushing, Oklahoma. Unlike with Brent Crude you can't just drive a tanker up to Cushing Oklahoma. So if you buy a WTI contract you are promising to take delivery of 42,000 gallons of oil there...it's the nature of the contract that there's no way around this.


ICE offers an equivalent cash settled contract, that takes it's value from CME's closing prices.

https://www.theice.com/products/213/WTI-Crude-Futures


Right, but the reason it went negative is that there was no place to store the oil (demand for storage went through the roof because demand for actual oil plummeted). So that if you held one of these contracts, you had to pay someone to store your oil (directly or indirectly, if doing a cash-settled contract). If the people in the article did cash-settled contracts they would have been in just as much trouble.


Curious -- did they settle at a price where holders had to pay up on the below zero spot?


Cash settled contracts exist. These people just chose not to buy them. The question is how to structure physically settled contracts. After all, oil needs to get delivered to someone at some point.


If you get into the market for physically settled contracts with no intention of taking delivery, then you're almost certainly a speculator. I'm not sure that it's the market's job to make that safer for you.

I am not justifying inaccurate pricing. Burning speculators is fine, but give everyone accurate information.


> If you get into the market for physically settled contracts with no intention of taking delivery, then you're almost certainly a speculator.

Hard disagree. There are lots of reasons people with legitimate hedging concerns who don't intend to take physical delivery prefer the physical contract to a CSC (if it's even available).


Would you list some of those reasons, instead of just making the bare assertion?


* Higher liquidity, lower slippage

* Different regulatory capital requirements

* Some commodities don't have cash settlement

* Sometimes a CSC doesn't count (as much) from a regulatory/insurance perspective if you're hedging a position in a commodities basket or whatever

I'm sure there are more - I don't work in commodities.


These futures contracts are for delivery at a specific location. What if you just dont want a million barrels 2000 miles away? Cash settled seems a lot more flexible for lots of industrial use.


Or possibly someone who needs large amounts of oil and want to hedge against fluctuations?

Freight industry, airliner, etc.


If you need large amounts of oil, then either you plan on taking physical delivery, so you can just let the contract expire, or you don't plan on taking delivery of that oil, in which case you want cash settlement contracts.


Good lord, if you actually need the oil, then take delivery!

The issue here is total speculators using PHYSICALLY settled contracts to speculate who don't want delivery.

The number of people who need WTI are pretty few - refiners and some small others. Seriously - with WTI you still need to refine it - airlines CANNOT just load WTI into their tanks.

These sob stories from folks who supposedly were bidding to take delivery of physical oil (unrefined) from a condo somewhere are ridiculous.

To trade futures you need to certify you understand them. I'd love to see the form this guy filled out listing what was likely a fair bit of bogus experience.

I'm fine with all these idiots getting burned.


Airlines buy futures in Jet A or Jet A-1. They have no interest in crude oil because they aren't refiners and they have nothing to do with it.

The freight industry will similarly buy futures in bunker fuel, diesel, or whatever exactly they use to fuel their vehicles. Again, they're not refiners and have no use for raw crude.

It's the oil refiners that buy futures in crude. Well, them and speculators.


> Airlines buy futures in Jet A or Jet A-1. They have no interest in crude oil

It doesn't matter. Airlines absolutely buy crude futures to hedge against changes in fuel cost. It might be impossible to buy futures in the exact good you need, or it might be too expensive due to illiquidity and slippage.

It's like how beer manufacturers buy aluminum futures even though they almost never take delivery on the futures. They're just going to buy the processed aluminum from their regular processed aluminum supplier, but they can still hedge some of the price changes with the easily available physical aluminum futures.


There are jet fuel futures. If they speculate on crude and then jet fuel itself goes up because e.g. a big jet fuel refinery blew up, they're screwed, because the crude future didn't actually hedge against the specific problem they're facing, and now they have to pay a lot more for jet fuel. Or it could be any other problem that specifically affects jet fuel but not crude in general.

Anyway, as usual, Wikipedia has a relevant article, and it seems that airlines do both: https://en.m.wikipedia.org/wiki/Fuel_hedging


That would seem counter to, "with no intention of taking delivery," from your parent's comment.


The contract is for a particular kind of crude delivered at a specific location. Most hedgers would still be better off closing the futures contract and taking the delivery they actually want instead.


It's far from a solved problem. Similar issues, albeit not quite as volatile exist for TBA mortgage bonds--except it's not usually problem of storage and more a problem of delivery.


Incorrectly assuming values can never be negative is an all-too-common occurrence in trading and financial software. In 2012 Swedish stock futures trading was suspended for a time because their matching engine used an unsigned type for order quantities and someone submitted an order with a negative value which wrapped around to 4 billion: https://www.reuters.com/article/markets-sweden-bug/swedish-s...

Interactive Brokers' software is usually very solid. I'm surprised they weren't ready to handle this, the possibility of oil going negative had been discussed for some time before it happened.


I particularly dislike the attitude of the CEO which shows he either doesn't know or pretends that he doesn't know how such contracts work. Please read the contract specs and educate yourself a little, Mr. Peterffy, they're public and free.

> Peterffy said there’s a problem with how exchanges design their contracts because the trading dries up as they near expiration. The May oil futures contract -- the one that went negative -- expired the day after the historic plunge, so most of the market had moved to trading the June contract, which expires May 19 and currently trades around $24 a barrel.

> “That’s how it’s possible for these contracts to go absolutely crazy and close at a price that has no economic justification,” Peterffy said. “The issue is whose responsibility is this?”

Nobody ever promised neither liquidity nor positivity of prices, it is the fault of the brokerage, plain and simple. Thankfully $100M is something that IBKR can take on their books (they have $3B of cash according to the latest filling).


Yes this is a very CYA statement. IB has been in this business for a long time, Peterffy knows very well that expiring futures contracts can experience all kinds of liquidity problems and large swings in price.


But if it’s never happened before, and you’re some random programmer making this stuff, I’m not surprised there are all kinds of assumptions being made when choosing data types and validating inputs...


"No economic justification"? That has to be faux-naïf. Yes, I was shocked at first to hear of prices turning negative. For about thirty seconds. Once I learned that the oil had to be delivered somewhere without enough storage, and under conditions of extreme low demand, it was perfectly obvious that you'd end up paying someone else to take the oil off your hands. Either Peterffy is too stupid to understand mass-media reports on his own field of business – not likely – or he's disingenuous, which says something about how he sees his clients.


Peterffy is a long time industry vet. He brought a lot of tech to trading in the 80's. He knows very well how things work.


>Interactive Brokers' software is usually very solid.

That is certainly not my experience. Endless bugs in TWS over the years, and the support people are unbelievably rude. Every time I try to report a bug they start out by blaming me, it usually takes 2-3 back and forth rounds until they admit it's actually broken and tell me they'll forward the issue to the tech people. After that it's radio silence and you never know if they'll actually fix it or not.

And let's not get into the disaster that is their API...


Completely agree. Their mobile app, TWS, API, and websites are all a nightmare. I left them as soon as I stopped trading on margin (the only really compelling reason to use them for a retail trader/investor).


I'll admit I haven't worked with their API or done anything too fancy with it. My experience is all relatively infrequent click trading, chart generation, and such.


> Interactive Brokers' software is usually very solid.

Except their web portal. 40-50% of the time it is unable to load my portfolio data (even without the current market value, just the number of stocks and cash balances). At the same time I log into the mobile app, it forces a logout on the web app (why?) and it is able to load the balances and portfolio. No explanation. Same network, no adblocker or other browser plugin.


The single session thing is annoying. I'm guessing it is related to their various data provider regulations. The large brokers can get around this because, well, you are not really seeing pool depth and the like.


I'm going to guess there's two teams involved on the IB side of things. There's financial engine team which processes and handles trades. And there is the UI team that displays the data and allows people to create trades. I bet the engine handled these fine and issue was that the UI team had data validation checks to prevent negative values.


Nope. It was not possible to trade at negative.


If you look at the CME website using the wayback machine, you can clearly see a hi and low limit for the price. There is no high limit, but the low limit is set at "0.01". [1] This is kind of weird, because the snapshot is pulling current market prices, and still showing an incorrect hi/lo limit. It looks like they have since removed the hi/lo display: [2]. Someone should take a screenshot of this.

[1] https://web.archive.org/web/20200117115242/https://www.cmegr...

[2] https://www.cmegroup.com/trading/energy/crude-oil/light-swee...


> Its software couldn’t cope with that pesky minus sign, even though it was always technically possible -- though this was an outlandish idea before the pandemic -- for the crude market to go upside down.

Wow, just wow. They are handling millions (billions?) of dollars every day and couldn't find the time to test that they can just DISPLAY a minus sign. That's insane.

And it's not even that outlandish. People were saying it could go into the negative weeks before it happened. This just seems like pure laziness. Just pretend everything is business as usual.


This is an industry that couldn't anticipate the year 2000 coming to happen.


Makes me wonder about their testing - any decent tester is going to throw a negative number in as a matter of course.

I have a gut feeling that this bug was flagged and closed as WONTDO because "that'll never happen"


Just as likely it was a feature. If you really think "that'll never happen", then the right thing to do is explicitly ignore negative prices or refuse to send orders at negative prices, and that's exactly the kind of failsafe I'd want to have in my trading software, since you can get spurious prices for all kinds of reasons.

It turned out the assumption was wrong and yeah, you should remove the logic handling that once it's evident futures may go negative, and you should have a process capable of making that change with only a day or two notice. But being robust the rest of the time at the expense of mishandling a once-in-fifty-years event is not in itself a bug.

If you disagree, should a trading system also allow negative prices for precious metals futures? Stocks? Currencies? Options? Bonds? Futures on stocks or bonds? I can contemplate all of those trading negative in extraordinary, contrived scenarios but I would design systems today not to trade them at negative prices.


But the fact that they didn't display negative prices to customers indicates that they were explicitly ignoring them in a very very bad way. I'm suspecting an abs() function was involved.


Wasn't the original purpose of futures to let farmers and others lock in prices early so they can mitigate risk? Speculation on futures seems dumb if you have no intention of taking delivery.


Taking physical delivery and hedging are not one and the same. It’s entirely possible to use a cash settled future to hedge against market movements; the farmer sells at a steep loss, but their cash settled wheat futures offset a large percentage of the loss on a cash basis.

The distinction you’re looking for here is those who are speculating on market prices, vs. those who are hedging against market prices. If you use or sell oil in large amounts, it makes sense to use futures to stabilize your downside risk, even if those futures are cash settled.

That being said, I think that cash settled futures make purely speculative trading much easier, so you’d have a good point if that’s what you were heading towards.


That is the point I was trying to make. Basically if you're in the business of producing or buying and selling the commodity the futures are for you. If you're just speculating how does that help anybody? I guess you could make the argument that having more eyes on the market means there is more information so the price is a better reflection of the true value.


It's a good question. Most markets have (or at least used to; I've been out of this a long time) a special class of speculator called a market-maker; they get discounted fees in exchange for providing liquidity. That way a seller doesn't have to wait around for a buyer.

Speculators in theory also provide liquidity, and in theory also contribute to keeping the prices "correct" (meaning at levels that reflect what's known). But it's not like any given speculator personally cares about that; they're just looking to gamble and win. Last I heard there was reasonable evidence that more gets spent on speculation than is delivered in benefits to the economy. So you suspicion is not unwarranted.

But there's another class of people who neither create nor consume the product, but have some financial interest in something related. E.g., suppose you sell farm equipment. You know that if wheat farmers have a bad year, you'll have a bad year, because they will put off buying your new tractors. To even things out, you can use wheat derivatives to essentially buy insurance on wheat prices. If wheat prices are normal, you lose a little money. But if they fall through the floor, you make money, hopefully counterbalancing the income from lost sales.

When this activity is significant enough, it can lead to the creation of synthetic commodities. E.g., if you are in the snowplowing business, maybe you want to insure against winters being abnormally snowy. You could maybe do something with a fuel oil future. But that's kinda tenuous. Instead now you can just trade weather futures: https://www.cmegroup.com/trading/weather/


Minor nit: market makers typically aren’t there to speculate, they’re there to provide liquidity and make small profits per trade doing so. Most market makers try their best to trade down to no position overnight, since they don’t want to be long or short in anything.


Thanks! Maybe the market maker I worked for was atypical. We definitely had opinions on where the market was going and made good money from that, especially in times of high volatility.


It's hard to make money by pure market-making, so it's common for market-makers to also do some amount of speculation.

After 2008, banks were banned from speculating (ish) [1], but were allowed to do market-making. It's common for market-making desks to do speculative trades under cover of market-making activity. It's hard to conclusively prove that any given trade is speculation rather than market-making (which involves hedging), so they generally get away with it.

[1] https://en.wikipedia.org/wiki/Volcker_Rule


Ah, yes. We were purely a proprietary trading firm. As my boss explained it, the traders who started the company brought a pile of money with them, and it was our job to make it a bigger pile of money. It was in some ways very pure; for my first year there we didn't even have the company name on the door. We already knew where the place was, and money spent on frivolities just meant less money to trade with.


If you only had farmers and individuals purchasing their goods allowed to trade futures, there is direct incentive for either party to manipulate the physical market through their actions, in ways that they only could, that wouldn't necessarily make any economic sense and could have spillover effects into the real economy. By having a more open market for futures, you theoretically have a more efficient market for all participants which you mention in your last sentence.


The standard answer is that speculators provide liquidity.

People that want to buy or sell a commodity typically only want to do a certain number of transactions. What happens if they can’t find someone to take the other side of a trade? It would be annoying to have to wait a week until someone else is interested. Moreover, they may want a different quantity than you do, or they may want a different delivery date.

There is also price discovery as you allude to. A farmer selling pork doesn’t necessarily want to have to track the details of the Chinese economy and the weather patterns in Europe just to get a fair price for their goods. Speculators help with that.


You may hedge your risk of prices of a different but related product.

E.g. a bread manufacturer wants stability on their cost of goods, so they may buy wheat futures even though they don't ever want to receive direct shipment of wheat, they're dealing with specific regional flour suppliers but the wheat futures at a major location are a good proxy for that price.


Liquidity is the standard answer.


You might want to invest in oil futures without taking delivery because you the thing you want to buy in volume doesn't have a contract, but is highly correlated. Airlines and jet fuel for example, though I think that has a market now.


More people than farmers (or manufacturers, or whatever picturesque production business) buy futures without speculative intent. Most futures trades are for (indirect) hedging purposes. E.g. if I made burger buns I would buy and sell wheat futures even if I never actually took delivery of wheat futures, because I can hedge part of my exposure.

Modulo the fact that almost no one a naive observer might accuse of "speculating" is actually speculating, I do agree that speculating on commodities futures is dumb, because taking a directional bet on anything you don't have inside information into is dumb.


>Speculation on futures seems dumb if you have no intention of taking delivery

What? How is it any more dumb than speculating and buying a stock hoping it will go up?

Speculation in futures is what gives those farmers liquidity in the markets.

Futures is a zero-sum game.. so in my world futures make way more sense to trade than everything throwing money into stocks to magically make money out of thin air until they don't.


A lot of businesses are impacted by the price of oil even if they don’t directly take delivery. It serves as a really valuable hedge for airlines for example, where it’s a key cost driver, even if it has to go through a refinery first.


You don't need to trade a physically settled contract for that though! Just hedge on a cash basis or hedge directly on the item you do need delivery on.

Seriously, do people enter into contracts like this in other parts of their life?


Cash settled contracts add a layer of indirection: we have to agree on a method to determine the price of physical oil on our preferred date. There is a popular cash-settled future for WTI crude. It trades on ICE and uses the settlement price of the CME future as its reference price, so exactly the same issue arises there.

As for trading exactly the item you will need delivery on, that may be hard to find. Standardising on a contract that's "close enough" allows crude oil producers to trade with airlines, bus companies to trade with refineries, etc, even if they all care about different products.


To clarify, it’s the farmer or resource producer shorting or selling the future to protect against the price falling (where the short will make money) while the buyer of a future (processing industries, farmers needing feed, etc.) does so to protect against high prices (where the price goes up unexpectedly). You can also do this with options on futures for presumably more leverage.

That said, it varies by regional availability. For instance, Canada has fewer options: https://www.producer.com/2017/11/hedging-with-u-s-futures-an... To properly hedge a Canadian producer using a US future you’d need to also hedge against the Canadian dollar, presumably. And hope that the weather and such is similar enough.


Capital One was short oil futures because they had loaned money to energy companies and wanted to hedge risk. They accidentally ended up with a big enough position that they almost had to be classified as a market maker.


Who would they lock their prices in with if not for speculators?


I'm surprised IB let speculators trade in a contract going to delivery. I worked as a risk manager in a commodity trading firm and only hedgers qualified to take delivery were permitted to hold contracts going to delivery.


IB automatically closes out your position before delivery can happen if you don't close it out.. so not sure why you are surprised IB lets them trade?

>I worked as a risk manager in a commodity trading firm and only hedgers qualified to take delivery were permitted to hold contracts going to delivery.

Ehh.. what? I've been part of a CTA for many many years, we have trading programs.. our clients trade in our programs.. the entire industry never takes delivery yet trades all these contracts which have delivery (metals, ags, energies). Very confused what you are talking about.. everyone rolls out of these before first notice dates.. brokers are on your back a week before the FND are coming up.


Wasn't this a cash settled contract?


No.


It's not a cash settled contract.

"Delivery shall be made free-on-board ("F.O.B.") at any pipeline or storage facility in Cushing, Oklahoma with pipeline access to Enterprise, Cushing storage or Enbridge, Cushing storage."

Source: https://www.cmegroup.com/trading/energy/crude-oil/light-swee...


It may have been ICE's crude contract, which is cash settled: https://www.theice.com/products/213/WTI-Crude-Futures

"The West Texas Intermediate Light Sweet Crude Oil futures contract is cash settled against the prevailing market price for US light sweet crude."


Sure, but a majority of trading was done on CME futures. In either case, you still ran into the issue of negative prices. USO only recently started explicitly stating that they would also potentially invest in ICE futures alongside CME futures.


Yes


They are cash-settled.



That's no the contract in which IBKR has lost money.

These are: QM NYMEX, and WTI ICE. Both cash-settled.


CL is not cash settled, it is physical delivery. It is traded on NYMEX.


IBKR doesn't allow to trade CL in the day of settlement, only cash-settled contracts:

QM - https://www.cmegroup.com/trading/energy/crude-oil/emini-crud...

WTI - https://www.theice.com/products/213/WTI-Crude-Futures


If you don't know what is going on, then why on earth would you risk so much money?

IB fucked up, no doubt, but these idiots are trading shit they know nothing about.

Don't trade on margin.


Well, yeah, it's reasonable to expect investors to appropriately researching something before buying, and be cognizant of the risk that it could crash.

It's not reasonable for them to expect to deal with a platform that misrepresents the state of the market and executes trades at a non-market price, as was happening here. (It was telling them the oil futures still had a positive price when it was negative, and making them pay on that basis.)


> Well, yeah, it's reasonable to expect investors to appropriately researching something before buying, and be cognizant of the risk that it could crash.

Is it? Isn't this the entire reason risk management departments exist?


I don't think they knew in this case they were potentially trading on margin. The broker didn't know either which is crazy.


If anyone is trading futures and did not know they were trading on margin or trading highly levered instruments then they either are trading on a platform that has zero compliance or they misrepresented themselves as an investor.


The broker calculated the margin requirements based on the assumption that the price could not go negative. So the investors thought they were risking $30 per contract when it was actually a couple orders of magnitude larger than that.

Effectively the broker lied to them about how leveraged they were.


Anyone who thought they were risking a max of $30/contract has no absolutely no business trading futures. Most of them are 1000bbl contracts and the disclosure docs that you read before being granted trading enablement are crystal clear. The margin required to be posted is not the limit of what you can lose. (If it was, margin calls would be much less of a thing.)


The margin is a measure of how much you might reasonably expect to lose. IB was clearly miscalculating it.


IB asked for very low margin, $30 per contrat for long positions, normal is around $8000. Margin for futures is based on risk. They could perfectly thought that minimum price was 0.01, and that was what IB was telling to them.


It's really odd that this bug occured, as IB has no issue pricing credit spreads with negative values. Must be an issue specific to commodities futures contracts. I wonder what data types they were using.


I’ve written code (a year or so ago) against the IB API and it seemed clear to me that the API was a thinish skin over multiple backend systems. The feed you get for products from different markets (even different futures markets) was different - the population of fields in price and trade feed was wildly inconsistent. I’m guessing each market is accessed in by a different IB system. Negative prices are a feature of some massively traded futures - interest rate futures for example - but can effectively never occur for index futures or the like.


From what I've seen, almost all financial companies developed a bunch of systems for different security types (which all have different rules and edge cases) independently, and only tried to tie them together as time went on. It's a recipe for a lot of confusion and inconsistency.

There are tons of opportunities in finance to make short-sighted proclamations like "the number of futures in this kind of contract is always 100" or "this type of security can't go negative", and have it be true at the time, but false 5 years later when they add a new type of contract.


Kind of orthogonal, but I’ve often faced pushback from Product Managers when trying to future proof a feature that I’m going to ship. The pushback is usually “oh, there’s no way our customers will ever need that!”. Only until many years later will it blow up in your face, with costly consequences.


I have had similar experiences. Even if the future-proofing is very inexpensive, they'd rather waste more work un-doing the future-proofing than the total amount of work that went into it in the first place, because it's "unnecessary".


For futures on physical deliverable objects (well, I guess most futures are such, but anyway) -- would volatility and speculation be dampened/improved if the clearinghouse forced everyone (or the seller) participating in a trade to certify that they had rights to the specific thing being traded? Could actually produce the contract -- like the oil producer is certified to have <xyz> barrels allowed to be sold?

My notion is that if much of the trading (and it can be shown by futures volumes) cannot possibly be on actual physically deliverable quantities, then most must be "speculation" by people who cannot actually produce the asset. Would this be a help to stabilize the market?

I know it all has to get settled in the end by the expiration date, but just an idea.


It likely will settle the markets by driving spreads through the roof and costing buyers/sellers of the physical items.

The underlying theme here is that speculators are not providing utility to the market, which is wrong. They are bridging time and risk.


Some companies can use this as a proxy to protect themself against variation of a product they need. For example an airline can use this to protect itself against price variation of kerosene once they have sold a ticket. As kerosene is not directly available on commodity market they use this future because their price are strongly correlated to kerosene price.


> then most must be "speculation" by people who cannot actually produce the asset.

Let's just call it what it is: gambling.


Fuck IB and this “trader”, idiots should lose their shirts, that’s one of the intended outcomes of an efficient market.

That said, there is an issue here with futures contracts: you can get very very large leverage when the price is near zero. This is the real issue with instruments that can negative price and just like their are “circuit breakers” in markets for big price swings, there should be breakers for entering the “near zero” range.


An efficient market that don't let traders operate? Did you even read the article?

Futures contracts that CAN BECOME NEGATIVE don't let large leverage when price is near zero, that's NOT TRUE. Future contracts margin is calculated with SPAN, and if it's done correctly, it considers the scenarios where price can go below 0.


>Futures contracts that CAN BECOME NEGATIVE don't let large leverage when price is near zero, that's NOT TRUE

It clearly is. If I can buy a contract for 1c, I can get 100,000 contracts for 1000usd. Then if the price rises of falls by 1usd, I'm up/down 100,000 dollars. Can you think of any retail product with that sort of leverage?

That's the danger of putting zero in a denominator.


You're ignoring the margin requirements.

Assuming your broker is competent (which the broker in this story is not), they won't let you buy 100,000 contracts for 1000 USD unless you can cover the potential downside, which would be of millions (possibly tens of millions) of dollars. So you can't actually leverage $1000 into going up for down $100,000; you would be tying up $X,000,000 to go up for down $100,000.

Unfortunately IB was not competent and they did not calculate the margin requirements correctly, which allowed their customers to leverage themselves improperly.


Sounds like IB hired the Robin Hood team.


You can't buy a contract for 1c. You need to pay the margin, and the margin for futures is not calculated based on current price, it's calculated with SPAN, that considers different scenarios in which you can lose money.

For example, for this contract Bloomber says IB asked for $30 margin. But the margin is usually $7000 for this contract, that it was IB should have requested as collateral at least for each contract. in a day with that volatility should be much higher in IB, as they take that also in consideration, probably around $20000 per contract.

The problem was that IB didn't consider scenarios in which the price can go below 0. The software was designed in that way. But it shouldn't.


Hopefully, one of the things that we price in going forward is the volatility in the price of oil vs alternatives, especially those that can be produced domestically in a way decoupled from international events. That includes natural gas and especially renewables. Remember that just a 6 years ago, oil was north of $100/barrel [1], and recently it's close to zero. Wind on the other hand has had a steadily reducing LCOE[2][3]. Solar's LCOE is also steadily reducing [3].

Volatility has a cost. With oil, it's one that the US and other countries hae historically tried to dampen with various industrial and political methods (the national strategic oil reserve, military/political "influence" on foreign oil producers, subsidies for domestic production), but seems like the current situation is beyond those methods' ability to control.

1. https://www.macrotrends.net/1369/crude-oil-price-history-cha...

2. https://www.energy.gov/sites/prod/files/2015/08/f25/LCOE.pdf

3.https://en.wikipedia.org/wiki/Cost_of_electricity_by_source#...

4. https://en.wikipedia.org/wiki/Cost_of_electricity_by_source#...


I've written real-time trading systems trading on IB. Their client library is a real pile of shit, so it's not surprising to hear that their internal systems are too. Reassuring to hear that they're going to eat the US$100M loss themselves instead of letting their customers have it. (Not that they were going to get US$9M out of this Shah guy anyway.) They never did us any wrong when we were their customers.

Does this pose a risk of IB going insolvent? If they do, is there a risk of their customers being just another creditor of a bankrupt corporation, with respect to the stocks and futures that IB holds on behalf of those customers? Or are those instruments held in bailment, or actually by some other company, rather than as IB assets?


Would be nice to have more details on exactly what happened with the trades. Did the trades clear? Did IB liquidate the contracts before expiration? Was someone on the hook for taking physical delivery?


I feel like any sane clearing house should’ve rejected these trades.


> Crude was actually around negative $3.70 a barrel when Shah’s screen had it at 1 cent

A bit of a design flaw, I wonder did they spend as much money on the software as Equifax did?

if (-3.70 != 1) { send an alert; }


Would any of the traders mentioned that owe all that money to Interactive Broker's have to pay the total amount? Or is that part of Interactive Brokers' loss claim.


From the story:

> Customers will be made whole, Peterffy said. “We will rebate from our own funds to our customers who were locked in with a long position during the time the price was negative any losses they suffered below zero.”

I'd imagine this is so they don't get sued and have some shoddy code pop up during trial.


or they're raising capital and the end justifies the means

same result of avoiding litigation, but even fewer consequences


Comments about storage costs are nice theory but don’t tell me those storage costs went up 50 dollars a barrel to cause CLK0 to go from +10 to -40 within something like 30 minutes. It was all forced liquidation by brokers like IB and people absolutely bamboozled by negative prices puking their positions. Nothing to do with costs for actual storage.



More to the point is that a bunch of dumbasses were speculating on oil futures without understanding the nature of the market, and no ability to take delivery on the oil in any event. Nobody should be holding those when they're so close to expiry unless they know what they're doing.


About two months ago I delivered to a private client a (monthly-basis) prediction model for ethanol prices that included Brent -- in logarithm -- as a predictor.

I know the monthly spot average of that statistic (fetched from FRED, the Fed of St. Louis system) won't turn negative, but still...


Another entry should be added to the “things programmers believe about commodity prices.”



I don’t really understand why the traders would end up owing money. If they thought they were paying 0,01 $/bbl but we’re actually “paying” -37 $/bbl wouldn’t IB owe them?


@tester89 > I don’t really understand why the traders would end up owing money. If they thought they were paying 0,01 $/bbl but we’re actually “paying” -37 $/bbl wouldn’t IB owe them?

My thoughts precicely, is there anyone on here that could explain the intricacies of these kind of trades?


He sold it (or IB did so on his behalf at end of day) at -$37.


i understood that he bought at -3.7, and then it dropped further to -37.


I would recommend http://www.saxobank.com , it's a kick ass broker.


I am playing my second tiniest violin for them right now.


Oh a guy with $77,000 in his day trading account didn't know futures can go negative. I'm shocked.

Ironically this is the same guy who will sell you a gym contract without a cancellation option and blame you for not doing your research when you owe him $1100.


As usual, this nonsense headline is clickbait for the truth - some traders couldn't trade when oil went below zero.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: