"Veteran traders would usually wait in anticipation for the weekly report of gas-inventory figures by the U.S. Energy Information Administration released on Thursday at 10.30 AM and then dive into the busiest trading window of the week. This is no longer true as most traders are now staying out of the market due to the HFTs new strategy - sending floods of orders in an effort to trigger huge price swings just before the data gets released, also known as “banging the beehive”."
edit: Fast algo puts orders out on multiple equity exchanges and then hedges itself with a slow algo in the futures market.
At what point will HFT drive out the proper functioning of a Market?
Have there been any studies on this? If human traders mostly reacted to "real" news (the Orange juice crop is bad this year) then human trading was mostly linked to actual changes that affect the price mechanism
But if large volumes of trades are speculative, or even worse, are directed at affecting the behaviour of other large Market players, is there not a point where the selling of actual OJ is just noise in the Market?
One can anticipate HFT bots acting like the strange Amazon pricing of obscure books in the millions of dollars, bots competing against bots to acquire more of the rent
redistribution. If we can imagine that we can imagine a failed crop with prices being driven down in a frenzy.
Surely some research exists on this? Where is the price signal tipping point?
The HFT boys are creating vast amounts of liquidity and squeezing the spread down to unprecedented lows. The volatility created by their trading is essentially invisible to a retail investor who observes the markets day-to-day rather than minute-to-minute.
It seems to me that people are fixated on volume, which is essentially an irrelevant figure - holding a big position for a couple of milliseconds has no meaningful impact on anyone but other HFTs. If we smooth our data to a resolution of minutes rather than milliseconds, market behaviour looks no different today than a decade ago. Even a supposed disaster like the "flash crash" of 2010 corrected itself within five minutes.
I'd be curious to see if the same debates were happening when computerised trading was first introduced. The magnitude of change was far greater, but there's broad consensus that computerisation led to fairer and more efficient markets. I just don't understand how faster trades can be bad in and of themselves. If your objection is to speculation in principle, then by all means argue for a Tobin tax; I just don't see why it matters very much whether that speculation occurs over the course of days, hours, minutes or milliseconds.
Stop-market orders, which turn into a market order when the stop is breached, are obsolete. Worst case one should set a stop-limit order, which "becomes a limit order that will be executed at a specified price (or better)" when the stop is triggered. "The benefit of a stop-limit order is that the investor can control the price at which the order can be executed" [1].
Your asset manager should know this and broker advise you of it. Then again, I don't think retail traders should have such unrestricted, unsupervised access to the exchanges.
It just means that your stop-loss order is working on unsmoothed millisecond-resolution data when it should be working on smoothed minute-resolution data.
When your stop-loss order executes, it's not going to execute at a smoothed minute-resolution price. It's going to execute at a price that's currently in the book. That means that it's not going to stop your losses if you trigger it with smoothed minute-resolution data. Twenty years ago, it would have. But today it won't, because anybody who took your smoothed minute-resolution price would be giving you an exploitable arbitrage opportunity.
I'm not claiming HFT causes securities market price crashes; maybe it does or maybe it doesn't. I'm claiming that, if a security is crashing, HFT gives it the chance to crash in milliseconds or less, rather than minutes.
Just so I'm not misunderstanding you....so you're saying that flash crashes would not and have not executed stop loss orders? (And also wouldn't trigger far out of the money purchase orders?)
That seems contrary to my understanding of how the market works. If a stock is trading at x and I have a stop loss in at (x-10) and the stock trades down to (x-20), even for a few moments, I would assume my order would execute (this is assuming FIFO execution of orders, and also assuming sufficient volume at that price to exhaust all the standing orders once we are down in that region).
The great-grandparent comment said that the market (specifically stop-loss orders) ought to work the way you thought the grandparent comment said it did work. The grandparent comment, to which you replied, actually described what would happen if the market worked the way it had been proposed to work. You are correct about how the market actually does work, and my comment explains why it works that way.
Holy crap now I'm even more confused. I just want to know if stop loss orders will or will not execute during a flash crash, when the crash price trades below the stop loss price and their is sufficient volume to eat through all the standing orders.
Yes they will trade.
No they will not trade (if they will not, then I'd be curious to know why, which perhaps has been explained above.)
EDIT: Rereading your comment above again, you seem to be of the belief that it will execute - which was the entire point of my argument. People talk as if HFT has no downsides, this to me seems like a clear downside. And saying "well just don't use stop loss orders, or use stop limit orders" isn't a valid counterpoint. A stop limit can still execute, the problem is when the stock dips down for a two minutes and then returns to where it was, but now you no longer hold the stock.
Sorry for the confusion. Absent experience, I believe that stop-loss orders will execute during flash crashes, because you don't know if a crash is a flash until it's over, at which point it's too late to stop your loss. Is that clearer?
It won't. High frequency trading slices the gap between offer and purchase into an enormous number of tiny slices, and HFT companies compete to see who can collect the most number of slices. But that does not affect the underlying factors that lead to most offers and purchases of stock.
For example you could offer OJ futures at improperly high prices a billion times per second, but that does not mean anyone will buy them. I'm not aware of any evidence that Amazon's weird million-dollar books have affected the price of popular new books.
In addition, I wonder by what criteria one would evaluate how "properly" the markets are functioning. Who decides what is proper? For example, based on the business fundamentals it seems ludicrous to me that Apple would have a lower P/E ratio than GM--but it currently does.
>For example, based on the business fundamentals it seems ludicrous to me that Apple would have a lower P/E ratio than GM--but it currently does.
Their P/E ratios are very similar. Why does that surprise you? Apple is a very mature company. They're currently making huge earnings (which naturally lowers the P/E ratio if those earnings are not expected to continue at that level), and they're a big long-term risk because nobody knows exactly what's going to happen with Android vs. iOS. It's extremely plausible that margins in the smart phone and tablet markets will take a dive in the medium term as a result of vigorous competition, which is where Apple derives the bulk of their profits. Expecting Apple to be doing five years from now as well as they've done for the past five years is to expect them to come out with something new which is as revolutionary as the original iPhone. Maybe they will, but the market obviously isn't betting on that happening.
On the other hand, GM is not doing great earnings wise, but there is no obvious reason to expect that their existing customers are going to evaporate, or that their margins are going to change significantly from what they already are.
My point was that the typical criteria for evaluating a market are the prices that it produces--but reasonable people can disagree about what the prices "should" be. (As evidenced by this sub-thread.)
> In addition, I wonder by what criteria one would evaluate how "properly" the markets are functioning. Who decides what is proper? For example, based on the business fundamentals it seems ludicrous to me that Apple would have a lower P/E ratio than GM--but it currently does.
Couple of things:
1. Reference to Apple PE ratio is likely a market driven phenomenon that has occurred before to the other technology company that grew very large very fast (MSFT) ie. sometime around 2000, most mutual/institutional investment funds literally owned more of MSFT (and now they likely do of AAPL) than they were legally allowed to own. At this cap, given that these buyers are the largest "long term" drivers of a stock's directionality, the stock must change direction. The second part of this effect is that now a bunch of them are underwater, and the psychology of holding a bad trade will affect whether they decide to book the loss (likely they wont for a while). TLDR: Apple is simply too large, relative to the tech sector, for its stock price growth to match its business fundamentals.
2. Re: markets functioning properly - we should remember that the "markets" are literally a construct. For all the logical arguments made about how HFT reduce the bid/offer spreads, I'm philosophically opposed to them. Mark Cuban has articulated why better than I can: http://blogmaverick.com/2010/05/09/what-business-is-wall-str...
In a nutshell, if we constructed the markets fundamentally to make it easier for businesses in the real economy to raise and price capital, and HFT starts to account for a multiple of that, then the purpose of the "market" construct has been hijacked.
This leads to all sorts of gnarly questions about how to decide what the right volume is etc, so I recognize its'a thorny area - just pointing out that HFT is not so benign, and oftentime comes with consequences that far outweigh the benefit, and happen too often to ignore.
Playing devil's advocate here: if you believe the purpose of the market is to help long-term investors buy and sell at rational/efficient prices (in other words, price discovery), then why is relative volumes of HFT vs. long-term investment relavent.
I'm not saying it's the best metric, or even a good metric, but there's quantitative evidence that when new news comes out and prices move rapidly, HFT reduces the time it takes for markets to settle down after big price moves. By many measures, this makes the markets more rough, since prices move more abruptly. On the other hand, with more rapid price discovery, fewer long-term investors trade at non-consensus prices during the transition period.
The ones that say Apple has the vision and skill to enter and exploit many more massive untapped markets as we've seen them do repeatedly compared to GM's fairly static markets?
To oversimplify, there are two kinds of trading strategy: value-based and momentum-based.
Value investors judge investments by the expected revenue from it if they hold on to it for long. HFT does not bother them, except in so far as it increases the amount of uncertainty that you can pick up bargains when you see them due to uncertainty.
Momentum investors, or speculators, judge investments based on whether they think they can soon sell higher than they buy. HFT is supposed to be bad for them, since very high volatility makes the kind of judgements they go in for harder. Usually, HFT is itself a kind of momentum investing, albeit of a strange sort.
If HFT reduces the returns from momentum investing while leaving value investment strategies largely unharmed, it might correct bad incentives in finance and so be a very good thing.
I think that HFT is responsible for over 70% of the volume on the major exchanges these days. Very little of what happens on the exchanges anymore is directly attributable to long positions.
I'm not sure how current this is, but the average time a stock is held is roughly 20 seconds [1] and that's definitely not long-term value investing.
I think we're already a long ways away from Kansas Dorothy, and I don't think we'll be going back any time soon.
This is only looking at the volume on public exchanges. There are a lot of regulations around public exchanges which prevent them from operating efficiently. For instance, unless the stock has a very small price, you cannot offer sub-penny prices on this exchange.
Most retail trades actually never see the exchange, they are sold in bulk by brokers to places like Knight or Getco who internalize the order flow. They cross customer orders and take on some orders. Since this does not happen on a public exchange, they can give sub-penny price improvements.
The same happen with dark pools where many hedge funds will send their trade to obtain better executions.
When the market makers who handle these order flow start carrying to much risk on their book, or if they don't want to take the opposite side of your trade, they send it to the stock exchange.
This means that the stock exchange is mostly a place where high-frequency traders meet to offset their exposures to one another. In this respect, it is not surprising, nor problematic that 70% of the volume come from HFT.
That all makes sense, but I would argue that we're still losing something very important.
There is a natural tension between intermediaries and suppliers in highly transparent and competitive markets. The threat of disintermediation minimizes rent seeking behaviors. Over the last few decades that threat seems very weak in the financial markets, which I believe will have negative consequences for our economy and society.
Yeah. I don't think it's surprising either, in fact when I was first writing the post I assumed HFT volume was closer to 95% of trade volume but I couldn't find any public numbers above 70%.
I think that dark pools are where the next crash will come from, but I'm by no means a financial wizard. The way I perceive it, dark pools and other nearly-invisible investment exchanges are scary in that the ramifications of dark pool trading can spill over into the light world (as it were) with dire consequences.
We'll see, I'm also thinking that the next crash could just as easily be caused by a rogue algorithm as a rogue trader.
Dark pools sound more ominous than they often really are. Most of the time, it's just trades that are done directly between two instituions (often via a broker-dealer like BGC or ICAP) rather than via an exchange.
Maybe you actually know a lot about dark pools, but by expressing fear without expressing knowledge, you give the impression that you're mostly afraid because the name "dark pool" sounds like some kind of unregulated secret exchange run out of a meat locker by the Russian mafia.
I think Kid Dynamite (a retired trader, non-HFT) has some pretty reasonable articles about dark pools:
Disclaimer: my employer helps mutual funds, pension funds, universities, etc. place large orders on the market, using a variety of techniques (including dark pools) to try and smooth out the market impact of these big trades and minimize the the amount clients lose to market participants who are trying to anticipate their short-term trading behavior.
Yes, I am aware of what dark pools are, and I'm not trying to imply fear. It's irrational to fear things you can't change, and it's unreasonable for me to deplore someone else allocating their money in an interesting fashion. I don't mean Dark pools are inherently bad because they're dark. These Dark pools expose society to unmanaged risk because of the implicit guarantees of support from the general population.
It's not so much evil as it is secret, and secrets with public shares are interesting secrets indeed.
As I said, I don't live in fear of Dark pools or any financial instrument, but it's important to understand that there is a significant difference between the normal sale and purchase of securities and the activity which happens in Dark pools. Otherwise there would be no need to draw a distinction.
> over 70% of the volume on the major exchanges these days. Very little of what happens on the exchanges anymore is directly attributable to long positions.
this isn't the same as being 70% of price movement. HFT is comprised mostly of market makers, who have books that, over the course of the day, are close to net zero. they do a lot of buys, but they also do a lot of sells.
most price moves over the course of a day are actually driven by people who take positional views, and buy or sell large positions. so actually, we're not really that far from kansas
That's not the whole story. If 70% of the volume is HFT, it means you necessarily have a lot of HFT trading against HFT, which means a lot of HFT is going to be liquidity taking.
Fortunately, that's only on public exchanges. Most of the liquidity provision happens before trade even hit those exchanges.
I read this morning in Nate Silver's new book (recommended by Fred Wilson and published September 2012) "The Signal and the Noise" that the average time a stock was held was 6 years a few decades ago, recently the average is closer to 6 months.
For the most part HFT just decreases or eliminates arbitrage opportunities, or price discrepancies between what a piece of information says the price will be and what the price currently is.
Arbitrage has always been a 'feature' of the market and although there are increased barriers to entry for arbitrage, economic theory says that by eliminating arbitrage opportunities through trading the difference away the market is being made more efficient
I've been wondering if an exchange that prevented HFT would prosper in the current climate. I'm sure that plenty of companies aren't a fan of their market cap being at the whim of an algorithm and the large number of swings it would undergo.
Wouldn't they prefer an exchange that offered liquidity in minutes or even hours, opposed to fractions of a second?
They'll still be traded over the counter elsewhere at high frequency, so I think the end result would be widening spreads on the new exchange to cover the variation over a time tick (whatever length that is), and no one (even long term investors) would actually use the new exchange.
What is your definition of H though? If someone decides to offer a large block of shares for sale, and they break it into 100 lots, are they a high-frequency trader under this definition? What if it's 10,000 lots? 10,000,000?
This is a data structure problem at its core - the fact that someone entered one billion BUY orders at $15.51 should not prevent me from seeing that there's an outstanding order at $15.52. The stock market employs the queue only for legacy reasons.
"This is a data structure problem at its core - the fact that someone entered one billion BUY orders at $15.51 should not prevent me from seeing that there's an outstanding order at $15.52. The stock market employs the queue only for legacy reasons."
(1) I'm not sure I understand what you're saying about price information. You seem to be saying that level 2 quotes don't exist. I assure you that for the major exchanges, level 2 quotes do exist, but they're more expensive than what most discout brokerages/Yahoo/Google give you.
(2) I'm not sure what you mean about queuing at price levels being a legacy artifact. There needs to be some objective rule for deciding who trades with whom when there are multiple participants at a given price level. In most markets, it's first-come-first-serve (a queue). I read that MS POOL was going to try prioritizing based on size, but I haven't heard anything since. So, if it's not first-come-first-serve, and there are two offers at $5.43 and a trader comes in and lifts one of the offers (not enough size to lift both), which of the two offers should get lifted?
Betfair offers exchange sports betting. There, you always have 5 seconds to cancel an order even after it's match, and they suspend trading near a major event, like a goal in a football match. This seems to work quite well.
So cancellations made immediately before e.g. a goal are rolled back? Otherwise that sounds exploitable, as the probability of a goal increases dramatically before the goal takes place.
You could make the (weak) argument that HFT has created market inefficiencies due to excess liquidity and furthering information asymmetry. Most exchanges (NYSE, NASDAQ, BATS, etc) have "circuit breakers" to prevent a massive sell-off. As far as "noise in the market", you could make the argument that speculative trading is simply that: noise. In this case it is just an algobot doing the noise trading, rather than some idiot that thinks he's found the next best way to "beat the market".
OP is referring to a case of a buggy algorithm in a bot which was setting a ridiculous price on a used textbook on Amazon: http://www.michaeleisen.org/blog/?p=358
Information can cause one of three price reactions: up, down, or flat. Prices stay the same if the market accurately anticipated the information. Chances are, though, that the information will change prices. One thus knows that when an EIA report comes out, ceteris paribus, prices are more likely to spike or dive than stand still.
"Veteran" traders set up limit orders so that if prices rose they'd buy and if prices fell they'd sell. It's a momentum trade. Unfortunately, they were sloppy, submitting orders before the data came out. Arbitrageurs realised that these lazy limit orders could themselves be cannibalised by nudging the price around to see if it triggers any hidden orders prematurely. The order's premature execution would then create a tiny, temporary momentum effect that the arbitrageur could ride. This is called banging the beehive.
The trader pushed out was setting up information-less standing orders before the report released. The trader adding information to the market, e.g. through unique analysis, is not concerned by pre-release volatility.
It is possible, here, that a massive lazy limit was prematurely triggered and subsequently mis-interpreted. It's tough to say. With limited information I'd caution against Nanex's assumption of the low prior probability insider information hypothesis.
Basically the company was about to release the findings from a key clinical trial. The stock price would either soar or it would flop.
Tuesday 12pm * stock price is $25
Tuesday 12pm - 12:27pm * massive wave of selling starts the price of the stock dropping
Tuesday 12:27pm - exchange halts trading at $11.81
Mind you, at the exact same time as this was happening, the company was releasing very positive clinical trial data.
So what happened?
The guess is that some large trader did some investigating and found out that a lot of people had stop-loss orders on the stock. The trader slowly accumulated a large position, then rapidly sold off shares right before the announcement.
This caused the stock price to drop, which initiated a number of stop loss orders, which further eroded the stock price. Since it's a small company with a pretty small float, it doesn't take much volume to really swing the price.
The trader then bought back a number of shares (at a very reduced price).
"Information can cause one of three price reactions: up, down, or flat."
Then there is no cause. Prices change somewhat randomly whether there is some information or not. The only correct predictive model is something like " there is 80pc chance that the price will be between -10 and 30 of its current value". Anything else is reading in a crystal ball.
I didn't invent any of these, read Thinking Fast and Slow if you have any doubts.
By price reaction I meant deviation from the path without the information. I was qualitatively representing the ensemble, i.e. parametric space, of possible price movements on (-∞, ∞) by mapping them to up <- (-∞,0), down <- (0,∞), and flat <- 0.
Let Px(t) be the price at time t. Px(t+δ) given Px(t), i.e. P[Px(t+δ)|Px(t)] is D = Gaussian(Px[t], vol) in an efficient market. We know relevant information is going to be added to the market at t+δ, but we do not precisely know its content until t+δ. The market can, however, make an educated guess. This information is a kernel with a central tendency* and an entropy, i.e. second moment, H. The price at t+δ is now D' = Gaussian(Px[t], vol + Hω), where ω is the weight of the information. Since H and ω are both positive we can see that D' is platykurtic with regards to D. D'-D, the impact of knowing that salient information will hit the market at a certain time, looks like a short butterfly P&L.
Thanks for the book recommendation; I enjoy Kahneman's work.
*If ω were very low or H very high, e.g. a unitary distributed kernel would have infinite Shannon's entropy, you are correct in assuming that it would have no impact on D', i.e. D'-D would be zero.
Commodity futures is not my specialty but I looked into the lagging quotes from leveraged natural gas ETFs and from natural gas producers.
It's well known that tracking index for gold (GLD) correlates to the tracking index of gold producers (GDX) and they diverge and converge. People would do pair-trading on GLD-GDX pairs to bet on convergence.
Given that there's a binary catalyst event for natural gas, I wonder if there's a viable strategy to identify the thinly traded natural gas producers and exploit the stale quotes by market-makers (if the laggards are not responding as actively as the more liquid traded natural gas companies like CHK).
On the other side, I wonder if in a highly volatile event where a lot of punters would bid up leveraged natural gas ETF (GASL) above its intrinsic value, and you can take the other side of that trade and hedge with more liquidly traded UNG or FCX options or futures market. Just food for thought.
EDIT: Also I wonder if there's a spread between IV in UNG options and in the futures market to do a calendar or ratio spread.
This is such utter bullshit. Markets are being programmatically gamed, and seriously bright young minds are still flocking to the Vampire Squid's embrace, hoping for a chance to help them do that in exchange for a slice of that phat, fraudulent loot.
The ad industry's efforts have led to online privacy being all but non-existent, but Wall Street has turned our economies into shit and saddled the world with massive amounts of debt that will never be repaid.
But you can consider the ad industry far more "innocent" than Wall Street. The latter has known exactly what it's doing and causing all along.
The most likely explanation: no one got anything early.
Venue timestamps can often disagree by a significant amount. It is very likely that SIAC (distributors of CQS and CTS) simply are not well synced to the reference clocked used to distribute the report.
Nanex spends a lot of time doing analysis based on precision timing without providing any sort of error analysis as to how well timestamps produced from different references synchronize. It would lend credibility to their hypotheses if they either provided their own reference or provided some measure of margin of error to the timestamps they so heavily rely upon.
For example, if Nanex had stated that they measured the time of release and the time of trades themselves as they appeared in CQS as observed from their machines then they could make a much more concrete statement as to whether: (a) the report was "early" or (b) the trades were "early" relative to one another. Most likely they would observe that the report appeared "early" as measured to their system clock (or possibly some other reference), but the trades did not appear before the report. Of course, Nanex would be smart enough to account for transit latency and other what-not.
The data are printed with the CME's and NYSE's official timestamps. It would not be unprecedented for the CME's timestamps to diverge from, say, the NYSE's. That would present itself as a consistent temporal dislocation between the CME and NYSE, but would be invisible within each data-set (sort of like your calendar putting itself into the wrong time-zone where - the times are off, but they're consistently off, i.e. spacing between events is preserved).
Instead, we see a signal within each dataset and symmetry between them. It could still be a data aberration, but that would implicate the CME and NYSE of having irregular clocks, a far more serious problem than some twat front-running a government report.
Most likely is that this is a real trading signal, but not one based on insider information.
Relative timestamps between venues don't matter. More importantly, using timestamps without a common reference point makes it impossible to assert what happened first, because the absolute value of the timestamp does not matter. What matters is what "really" happened first. Of course, everyone's view of "first" can differ, but that's a different issue.
Whether there is signal here or not wasn't my point. There may well could be. My point was simply that the analysis provided by Nanex is spurious without a definition of the timestamps, where they came from, and their relationship to known reference points. Taking three events with a timestamp from each of: CQS, CME, and an arbitrary point in time such as 10:30:00 and then asserting that events observed within a CQS or CME event stream happened "before" the arbitrary point in time, 10:30:00, is impossible.
Unless either (a) one observes CQS, CME, and the event scheduled for 10:30:00 from a known reference and accounts for all variables (differing transit latencies, etc) or (b) each of the generating events are known synchronized and the current state of that synchronization is observable (CQS lags CME by 475us, government event source leads CME by 1.2ms, etc). Clearing (b) is not happening today. (a) can be accomplished by Nanex, if they so choose.
I've seen exchanges screw up clock time more often than I care to remember, often for the stupidest reason. I once had an exchange claim it was "due to a roofing contractor accidentally interfering with the satellite used to get a GPS time signal".
Network syncing time is hard, my team (at a bank) built our own infrastructure to monitor time sync errors on our own internal servers, we synced externally with two independent clocks. An atomic clock in London and NIST in the US. But we also modelled the exchange's time as well.
We had six independent connections to the exchange (hitting different servers) and we'd reverse engineered the exchanges internal infrastructure so based upon the timing of their messages, the tcp packet headers and their own timestamps we knew exactly what clocks their internal servers were running on (more valuably in our case is it also gave us their internal latency figures). Half the time we knew when their clocks were drifting before they did and could tell them which internal server had the wrong time.
This stuff is hard, you need to know a lot about network latency and time to make sure you're doing it right (have a look at the NTP spec for starters), there's probably only a handful of people in the world who are capable of nailing it and probably most of them work in the military or atomic physics research labs.
> This stuff is hard, you need to know a lot about network latency and time to make sure you're doing it right (have a look at the NTP spec for starters), there's probably only a handful of people in the world who are capable of nailing it and probably most of them work in the military or atomic physics research labs.
Since Hacker News uses a dynamic sorting algorithm for comments, "below" can quickly become "above". It's best to just say something with a link, like, "See my post elsewhere in this thread (https://news.ycombinator.com/item?id=5147300)."
This is the first time I heard about that meme, which means your comment contained the most usable information I gained across all comments in this thread.
Then you will probably find knowyourmeme.com to be educational and edifying ;). HN, however, is not usually where I want to find out about these things, especially when they're so wholly tangential.
Or someone got the report a few hours early, didn't want to be seen to jump the gun (insider trading, and all that) and kicked off a new trading strategy 400ms earlier than they intended...
Which begs the question of whether it was accidentally early.
It may well be that the 'early' trader knew they weren't the only one with early access to the data and intentionally set their trades to be a tiny bit early, to increase their chances of getting out ahead of the people who schedule their trades for just-after the scheduled announcement.
Which may well indicate a larger problem than one guy with a tip.
Absolutely. I'm just saying there isn't much justification to assume it was a mere mistake.
Particularly given the trades that show up just after the report was released, but before a person could conceivably process them and initiate the relevant strategy.
That alone suggests gamesmanship, even if the "early" trader went earlier than he meant to.
> Particularly given the trades that show up just after the report was released, but before a person could conceivably process them and initiate the relevant strategy.
Couldn't they be trading based on some natural language processing algorithm?
and of course that also is only possible, when they know the exact time of when the news would go out. So that, they can time their activity just milliseconds before that, using tech.
For a lot of these economic numbers and earnings report, they are provided to news organizations prior to the announcement with instructions that the information cannot be released prior to a certain time.
So, for this example, that time was 10:30AM. The moment it hits that time, the reports get pushed out into the newswires and reported on-air.
This allows the reporters to digest the numbers and figure out how to report it.
Yes, but they have strong motivation not to abuse it. I've seen one case where an news org broke an embargo due to fat fingering a release, that news org lost early access privileges for a year as did every sister company that belonged to the same news group.
If you specialize in financial news losing numbers is a huge deal, you'll lose thousands of customers over it. And when those customers are often $1000/month subscribers it can easily mean a financial loss of tens of millions of dollars.
Within news organizations the information is typically restricted to 1-2 named individuals who have restrictions on trading. If they leak the information they can generally be criminally prosecuted.
I'm talking about government figure releases (employment, inflation, etc and the like).
In that case it was probably just a contractual issue between Google and their printer, to the wider market it would have been the same as if Google had accidentally released their numbers early.
(Actually even in the governments case it's still a contractual issue, it's just that governments have huge power by withdrawing early access rights)
If RR traded on those numbers they would have violated criminal insider trading law.
Outsider here, but I can't think of a good reason to give this info to news people before traders, and as the parent said, there are obvious bad reasons.
Much of the news ia a facade. Reporters often get things like economic numbers and speeches prior to the event.
Things like the State of the Union Address and candidate speeches are usually provided to the press beforehand so that they can package together a story. Next time you watch a major political speech, listen to the pundits prior to the speech. They will talk about all the things that the candidates are about to say.
With regards to economic numbers, there's very good reasons for them to do it this way (pre-released to media and embargoed). If it is not pre-released, reporters will need to take time to report the numbers which gives certain people an advantage.
For example, if you're monitoring Bloomberg for the CPI and the reporter is a slow typer, a person monitoring Reuters would have an advantage over you. Secondly, this forces a situation where reporters are in a hurry to get the numbers out which could potentially lead to errors. With these types of numbers, an error could have impact of billions of dollars in trades.
Well it seems to be known that the report is due for 10.30am, so it is possible. Just an additional possibility, though nanex are far more experienced in explaining odd activity in the markets than I am :)
yes, I saw that, but think it in this way, that its a very unreasonable way, to give the time in advance. Its better for investers to have some luck with this, if the time is not given in advance, just the day roughly. The news will drop at any time during the 12 hours before or after the given day.
This would cut the advantage those hardwired trading machines have, that do tons of trades in few seconds or even milliseconds.
This should be one of the reforms against such things, in the interest of common folk investors.
If your idea of investing involves reacting to news and trading stocks/bonds/commodities/derivatives on internationally automated markets then just donate all your money to a good cause because at least then you'll get a tax deduction out of it.
Or they had an algorithm that was cocked and ready for the data release, engineered for maximum speed at the cost of fail-safes, which had its moment of premature excitement.
Some back of the envelope calculation for UNG (ETF in the top Nanex graph) profits for the early mover.
In the second before the announcement UNG was trading at $18.72 High and $18.55 Low. According to some minutely data I saw 400,000 shares where traded between 10:29:00 and 10:30:00; I believe this ties out with the first Nanex chart where there are 490,000 shares traded, with majority of it coming in 400 ms before the 10:30:00 mark. 4 Seconds after the 10:30:00 report release the price had stabilized at around 18.51. I will consider this $18.51 the fair price with all resonably fast algos having made their post release moves.
Let us assume that there was a single trader/algo who got the report early and executed all of the 400,000 share sells 400ms before 10:30:00 and all other market participants only bought. Additionally assume that the average fill for these sell trades was the average of the High and the Low at (18.72 + 18.55)/2= $18.635. I believe this fair because looking at the first Nanex graph the early trades are somewhat uniformly distributed between the high and the low. In a simple arb on UNG, where the trader went short 400ms before the the announcement and closed the position at the fair price a few seconds after the announcement he stands to make a profit of (18.635 - 18.51) * 400,000= $50,000.
For a trade that lasts 5 seconds, making $50,000 is nothing to sneeze at, it is not that much in grand scheme of things. Additionally other ETFS and futures were impacted and could have made more or less money.
TL;DR: If one guy captured all the profit from the early UNG trade, the max he made was roughly $50,000.
If the total value of any trades that might have been made in the claimed 400ms window was only $50k, it would suggest to me that there was nothing untoward going on, and that this story is most likely nothing more than a clock error. For any of the HF firms, prop houses and Hedge Funds that I've run into (which is not a short list, but my no means exhaustive, given that I work in finance) $50,000 is not interesting action - especially since transaction fees will eat away a reasonable chunk of that.
It is a clock issue, as observed by others in this thread.
I would point to http://news.ycombinator.com/item?id=5146571 because its clear from nanex's response that he hasn't fully thought through the clocking issues with using CQS data without observing it directly himself.
After seeing some of their posts earlier and comparing it to live data I record at the colocations, I've concluded that they have clock issues which makes these types of anomalies appear frequently. Or they have a bad data vendor.
Interestingly enough, even the regulators don't have good (only millisecond-resolution) trade data.
CQS has had issues in the past. You should know this.
More importantly, why wouldn't you invest in colocations and collect the data yourself using direct feeds (with GPS clock synchronization etc to validate the data)?
We do. You are grossly misinformed. The charts correctly show the sequence and times of this event. I'm not going to engage this discussion further, though pmail is fine.
As a bystander who does not have the background necessary to evaluate either of your claims at face value, I would really like to hear your explanation for why you disagree with his explanation.
A simple analogy: you are an advertiser using Facebook. You tell them you want to get 1000 impressions, and facebook gives you a report at the end of the day saying that they gave the number of impressions you paid for.
So you look at yesterday's view count and today's view count and notice that somehow the view count only increased by 900. Something is afoot!
Nanex's argument is tantamount to saying "that means we must have lost 100 organic views today"
My argument is tantamount to saying "I actually bothered to look at our access logs (which we record on our servers) and only saw 900 that we could definitively attribute to real Facebook users. Is it possible that the report is incorrect or falsified?"
Back to the current situation. There are many sources of market data. Each individual exchange generates its own feed, and with the major exchanges (NYSE, NASDAQ, BATS, DirectEDGE, ...) you can colocate in the exchange data centers (NYSE and ARCA are in Mahwah NJ, NASDAQ is in Carteret NJ, and various other exchanges are located in New Jersey and Chicago) and record the data yourself. There is a unified tape (CQS/CTS) which combines and disseminates a combined record (across all exchanges). This is used to determine the "national best bid/offer" -- the prices people are willing to buy/sell at.
The process of CQS generation is fraught with problems, but lots of older traders and academic types use CQS data because its much cheaper to get that data than to get data from individual exchanges directly. However, you are subject to the quirks of the combination process, including subtleties regarding timestamping data (since this data includes trades and quotes from Chicago and from New Jersey, the sequence of events may appear different if you record from chicago or new york or philadelphia or some other place; if you ask the exchanges to timestamp directly, you have to worry about clock delay and skew between the exchanges' servers).
nanex is saying that it is acceptable to depend on that data and any anomalies must have occurred outside of the recording process. I am saying that the recording process can create the types of anomalies that nanex is showing, and that the only way to be sure is to record the data directly and carefully synchronize your recording machines. AND when you do that you see that there really is no anomaly.
Just to emphasize how sloppy the exchanges are with regards to timing: on the BATS exchange they use multiple servers to run trades and generate quotes, and every once in a while you see messages appear to be out of time order because the individual machines weren't properly synchronized (although, if you filter for a single ticker, messages are always in chronological order)
There's quote feed solutions that do co-location at the exchange servers and deliver the quotes to you in individual channels (ARCA/BATS/EDGX-A etc) and also SIAC feeds.
That looks like a hardware product. You still need to purchase market data access to the relevant exchanges to get that data, and those are expensive (NASDAQ costs, for example, run upwards of 20K/mo to get the lowest-latency data)
They should just spring for the data before making accusations -- the problem is that when you cry wolf all the time no one will take them seriously when a real case comes around.
Honest question: How are timestamps different from any other user-supplied data? Is the timeline of events ever recorded using an unsynchronized clock for relative comparison?
hmm, its not only the activity just before the official announcement, but also, just after it. There is no way possible that people can just make make a trade 'after' hearing a report 100-200 milliseconds after the report.
The trading before the official announcement is a concrete proof, if this is official, and bug/error free. But then again, this is not my forte.
To give an example of how this could happen (not saying this is what happened, but I've heard this happened before):
Suppose you left ntpd running and automatically adjusting the clock every hour.
If your clock is running faster than pool.ntp.org, and you are synchronizing to it, you may end up adjusting in the middle of an event. Because your clock is running fast, you would jump back in time, breaking the sequence of time (this is somewhat equivalent to what you see during daylight savings time if you aren't intelligent in the way you handle the backwards hour shift)
In this case, if the adjustment was forward in time, there would be a gap.
I'm sure there are lots of buggy NTP implementations out there that "adjust the clock every hour", but the way it's supposed to work is by continuously varying the speed of the clock (for example, using adjtime()) to correct any discrepancies. At no point should the clock jump backwards or forwards, or even have milliseconds that are more than X percent longer or shorter than usual.
"is plotted with official exchange timestamps" suggests that they are doing it wrong. You can't compare apples to oranges without knowing how the exchanges are timed.
For those who do latency tests, this is a very important point: you should always be on the lookout for what clock is recording the 'start' and the 'stop' and to be sure to consider clock skew.
To get a sense for how far timestamps can diverge, OATS -- the reports that are sent to the Financial Industry Regulatory Authority -- require that machines be synced to within 3 seconds of NIST (which is nearly 7.5x longer than the 400ms quoted).
When you measure latency, you always have to be careful about clock issues at the point where you measure the start and the point where you measure the end. This is old-hat for sysadmins and others who deal with these types of issues. This is why round-trip latency numbers are easier to work with: both the start and the end times are taken on the same clock.
It's clear, given nanex's responses, that they depended on someone else to give the timestamps. Before concluding that someone had inside information ahead of time, they should check their processes. It's like someone claiming they built a perpetual energy machine because they confused power with energy (i want to say it was paul newman but the name escapes me -- this actually happened)
Could be completely explained by someone having the right relay for this data in the right place at the right time (which would have been the result of a considerable outlay of resources and compensation to establish, thus boosting economic output long ahead of any potential killing(tm) made here).
And I can't see any particularly efficient way to level the playing field with a government that barely understands facebook and fair use. Just try to explain microsecond latency to them, go ahead, really, should be even more fun than a series of tubes(tm).
Not disagreeing with you, but wanted to mention that I recently learned that quartz clocks typically drift half a second PER DAY, which was shocking to me. This seems to imply that computers are subject to the same drift unless they are syncing via NTP many times a day.
(I don't know a whole lot about the intricacies of trading) Is there a minimum standard "resolution" for trades, or is everything sort of a best effort based on the speed of the computing hardware involved?
This has nothing to do with the main subject matter of the article but those graphs are almost unintelligible. I didn't even try to decode what they were trying to communicate.
The graphs are brilliantly clear to me, but I have a background in electronic derivatives trading and so am used to staring at wiggling psychedelic surfaces for hours on end. Nanex's audience is algorithmic securities traders. Imagine trying to explain a circuit diagramme to a room of uninitiated economists ("is there a reason the wire goes all squiggly right there?").
A little help on how to read the 2nd graph or maybe some source material to teach me. I'm not exactly sure how to even begun to research what kind of map it is, it looks like the matrix had an unfortunate mishap.
What you are looking at is a graph of the liquidity in the order book over time. A security does not have 1 price it has 2, the bid and the ask. The difference is called the spread. The bottom is showing the amount of liquidity at each price level. I don't know of a good source on how to read these charts since most people build their own viz tools.
Awesome, this explaination and electronic derivatives trading lead me on the right path of reading candlestick graphs. I discovered stockcharts.com from doing some basic research and now have a pretty good understanding of them. I won't be trading any time soon but I can at least read it to some extent. Thanks a bunch. Sorry if I got a bit off topic, but I hate not knowing stuff.
I don't think that increased trading activity supports a conclusion that the report leaked.
People may have anticipated increased trading after the rapport release and may have prepared algorithms to try to gain during this event. The algorithms may have started working before the release.
I'm not saying this was actually the case, but my theory is as well supported as the claim in the post.
I even hazard that whether the report "leaked" or not is irrelevant. The really interesting thing here is that, in a world of algorithmic trading, being 400ms late to a party can cost millions of dollars.
No, I'm not, but based on the quality of this post alone, I don't value them much. A respectful source would present a compelling argument for such a claim, whereas this post looks like a marketing trick to attract attention.
You'd only stick algos to automatically sell at a given time if you were mad (as you'd act on a move, not a time) - and the down and half-up tick indicate straddles set around the position.
This may have been a single rogue performing an initial sell, causing a drop which caused all the other pre-set strategies held by folks with very close to the exchange links to also sell, and then subsequently buy back to the midpoint.
It is worth pointing out that the EIA Natural Gas Report comes out weekly (every Thursday at 10:30) and the market reacts within a few milliseconds.
Since it's not the report producer's job for investors' computers to rapidly parse it, they should have some fun in phrasing and presenting the information in different ways each time. If nothing else, it could lead to an explosion in NLP and content parsing technology ;-)
Since it takes a while to digest the report after having seen it, chances are that they were in possession of the report far earlier than T-400ms but waited until they were in a time window where they knew the regulators would not come after them.
This is how fortunes are made. By taking advantage of loopholes in the regulatory mechanism.
I don't know the way these reports are structured, but is it regular enough where there's even a possibility that a bot could digest, analyze, and act on the information there in near real time?
How can you read this chart, and infer from it that the activity is due to having already read the report?
Surely everyone knew that the report would be released at 10:30, and they had strategies (or hedged positions) that were not as likely to be influenced by the contents of the report as by the market forces surrounding their orders?
> It is worth pointing out that the EIA Natural Gas Report comes out weekly (every Thursday at 10:30) and the market reacts within a few milliseconds. This is because the report centers on one number which makes it easy for machines to process and take action.
True but I also assume a large part of algo trading is deducing up/down signals from new information and acting quickly. I've heard the bigger reason high-freq algo trading is used is to mask larger moves in position in noisy trade bundles, avoiding price shock.
Do they hammer the webserver for this? Is the JSON pushed somehwere? I'm in Europe, so numbers are probably too high, but httping results for the file suggest this is not how it's done:
% httping http://ir.eia.gov/ngs/wngsr.json
PING ir.eia.gov:80 (http://ir.eia.gov/ngs/wngsr.json):
connected to 205.254.135.25:80 (286 bytes), seq=0 time=459.92 ms
connected to 205.254.135.25:80 (286 bytes), seq=1 time=230.15 ms
connected to 205.254.135.25:80 (286 bytes), seq=2 time=232.81 ms
connected to 205.254.135.25:80 (286 bytes), seq=3 time=235.23 ms
I would love to have some background information about how this trading works. Are there any technical details available how this works?
HFT traders are actually physically positioning their systems to be as unhindered by physics as possible. They likely have servers in DC just to check ir.eia.gov, and definitely have servers at (or as near as possible) to the exchange on direct lines with trading systems.
I'd be shocked if anybody big was doing that. There are news agencies (Reuters, Bloomberg, etc.) that get the reports early and sell exchange-quality news feeds. People subscribe to those news feeds, and at the appointed time, messages go out to the subscribers.
This shows that not all investors acted on the information at once. But doesn't show that the report got out early.
To the point of "private investors", if you are expecting to be able to see a news release the second it drops, spend 10 seconds reading it, make quick predictions on stock movements, and then buy/sell equities based on that... computers have had you beat for a long time on that game. Find another or automate that.
"There is the old story about the market craze in sardine trading when the sardines disappeared from their traditional waters in Monterey, California. The commodity traders bid them up and the price of a can of sardines soared. One day a buyer decided to treat himself to an expensive meal and actually opened a can and started eating. He immediately became ill and told the seller the sardines were no good. The seller said, "You don't understand. These are not eating sardines, they are trading sardines."
If we can't solve this problem, with the rise of machine driven microtrading, is there really any reason to place any faith in the stock market as a private investor?
You have to realize the distinction between trading and investing. Investing is putting money into something in the belief that over the long term, it will likely return enough money to be worth the opportunity cost and the risks involved. Trading is seeing a price difference between here and there (in 4 dimensional coordinates) and making money by moving the goods.
The stock market was created for investors, who by investing grow the economy as a whole, but it has been taken over by traders, who are helpful for greasing the wheels of the exchange but ultimately not necessary in anything like today's numbers. It would seem by competition for a commodity service they should start killing each other off but so far it seems there are still enough amateurs in the game to keep them well-fed.
So in short, make sure what you are doing is investing and not trading. Investing still pays off. Trading is like jumping into a tank full of robotic sharks.
I mean, consider: a report got released 400ms early. So the price of the futures fell 400ms early, and some trading firm made money off of that fall (instead of some other trading firm making said money.)
What's the impact on you of the price changing 400ms early? Even if you held that asset, were you really planning your trades to the second, let alone the millisecond, to begin with? No? Then it doesn't matter.
Buy-and-hold broad index funds for decades on end, then you need only fear macroeconomics and political risk.
This was futures, not stocks, but I see where you're going.
Market makers aren't competing with buy-and-hold investors. Price fluctuations like this don't affect people that are investing in the fundamental performance of the underlying company. The price may swing by a few percent randomly in either direction, but in the long run, a growing company will have a growing stock price.
Equities make up much of my portfolio and I haven't been disappointed. As cynical as everyone is about how "big companies suck", they have historically done a pretty good job creating value. And I think that will continue for some time.
>As cynical as everyone is about how "big companies suck", they have historically done a pretty good job creating value. And I think that will continue for some time. //
It's not like resources can run out or anything. You just dig up more.
Financial efficiency and value often appear to be at odds. Amazon certainly appear to spend less resources in the delivery of goods. Low cost of acquisition isn't necessarily correlated with greater value in terms of human fulfilment.
I've often wondered why we don't have [more/widespread] community kitchens, less work needed for food production, reduced waste and transportation costs, etc.. Why can't I go somewhere and get a quality, healthy meal that is cheaper than what I can make at home.
The short answer is "be the change you want to see in the world"
The long answer is a consumer's collective. If you are in the US a good example are the credit unions, where fees (of all kinds) largely don't exist. In the US, Consumer's Collectives legally have a democratic corporate structure. Private corporations legally are more like a dictatorship. Public corporations have a board of directors and are legally more like an oligarchy (unless one single person constitutes most of the board).
Assuming you're an engineer, it is probably not cheaper for you to cook at home if you factor in the cost of your own time. As soon as you involve a bunch of people that have no connection to each other and ask them to start doing work, they start wanting money. Hence, no community kitchens of software engineers: they can make a lot more money programming than by cooking you your food.
Trading stock is, and pretty much always has been, gambling. So your strategy should be the same as blackjack: Know the rules and know your limits. For the market, you should also be in for the long term, and always use limit orders.
Limit orders make sure that your buy or sell order are done exactly at what you want. Legally, the trade cannot execute unless it's at or below your limit if you're buying, or at or above if you're selling. That, more than anything else, protects you from small time fluctuations caused by HFT.
Going for the long term is really where the focus should be, though. HFT algos tend to fight cents or fractions of a cent(the event referenced here caused a 2% drop in the futures price, which came out to around 6 cents per : http://quotes.ino.com/charting/index.html?s=NYMEX_NG.H13.E&#... ). As a private investor, your focus shouldn't be on getting rich in a week. It should be making sure that the pile of money you have now gets bigger every year. Your limits shouldn't be the HFT-like fractions of a cent or tiny percentages, they should be in the 5-10% range for a return.
Also, since you're not going to win fighting HFT, don't bother. What they do shouldn't effect your overall strategy, because you're not in that space.
We could significantly alleviate the problem by limiting by law the trading frequency. Traders ultimately depend on the law to recognize the validity of their transactions. There is no value to society in high frequency trading. Mandating a full second in a market that operated quite well when slow-reacting humans conducted all the transactions should be more than sufficient.
Will laws work? I imagine the big investment banks will just set up dark pools in countries with favorable laws and just trade there instead. Added benefit: no more taxes!
The solution is to realize that high-frequency traders are playing a different game than you, even though they're on the same playing field. They do weird things but it's probably not hurting your returns. (It wasn't HFT that imploded the big banks, Enron, and Worldcom, right?)
They do weird things but it's probably not hurting your returns.
I'm not so sure. The financial system is a nonlinear dynamical system. The hallmark of such systems is that small local perturbations can lead to very large changes in system-wide state. High-frequency trading vastly increases the number of small perturbations, and while most remain local, there is a finite probability that some will percolate upward in scale. So micro-scale trading may increase our exposure to catastrophe.
Does money just grow on trees in the magic stock market? The amount of sustained non-bubble growth the stock market can generate is limited, not unbounded. It follows that if the HFT bots steals a slice of it, then the slice the regular gamblers get is smaller than it would otherwise have been.
They're already getting a slice. Look at any stock quote and notice that asks and bids are different amounts. That's where they get their money from: transactions, not growth. (Look up the expression "delta neutral".)
Is there a magic money tree? That involves a lot more economics classes than I took, so I won't even attempt to answer.
trading fast isn't the problem. it's quotes being made with no intent to ever execute them. like someone at an auction making a bid and then saying "just kidding"
Some have suggested to charge a percentage of a cent on all these requests rather than banning or limiting the rate of transactions. Seems reasonable to me.
Small investors can only buy on bad news and long where applicable (i.e good company). Bad news and poor technical performance even if for a period is guaranteed almost to be driven down by HFT. And sometimes HFT over does it (maybe on purpose to let the suckers flood back in). I.e. Apple going to 430, Netflix to mid 50s after similar shorts/run ups and battle of machines.
We all will be using HFT soon via proxy or already are. I would say this, possibly in the long run it is better as there will always be HFT algorithms that buy on the dip and take into account technical input as well as human input. Humans are irrational but machines only take a part of that into the algorithm, relying heavily on technicals, futures and news in addition.
But there will also be massive shorting efforts as this used to be a tenured skill, but now a machine can match anyone on shorting. Flash crashes are now always possible but also recover quickly. The only way to change this is to charge more for trades and throttles, then there will always be inequality there as well. This also sort of lessens run away bull runs as well as the machines will always pull back first or buy first if past thresholds are met and futures line up.
> Small investors can only buy on bad news and long where applicable (i.e good company). Bad news and poor technical performance even if for a period is guaranteed almost to be driven down by HFT. And sometimes HFT over does it (maybe on purpose to let the suckers flood back in). I.e. Apple going to 430, Netflix to mid 50s after similar shorts/run ups and battle of machines.
I think Apple's a weird case. They had a "down" quarter, which caused the tech analysts to go all crazy and claim that Apple's a sell. So that caused people to sell, which drove down the price, and dropping below 500 probably triggered a lot of people's stop loss strategies, which drove down the price even more...
I agree that corruption is the larger and possibly more pressing issue, but that doesn't seem to be what happened here. I could be wrong of course, but this seems like a technical flaw and not intentional.
Every sell is someone else's buy, and every buy is someone else's trade.
If you saw a bunch of activity happening milliseconds before it should, why would you be the other party to someone you suspect is committing fraud? You will be the primary victim of the fraud.
If you saw a bunch of activity happening milliseconds before it should, why would you be the other party to someone you suspect is committing fraud?
If no such report had come out milliseconds later, the activity wouldn't be suspect. How could a person (or in this case, given the timeframe, an algorithm) possibly distinguish this spike from a 'legitimate' spike? It doesn't make sense to blame the victim of a fraud when the victim has no way of knowing at the time that they are being defrauded.
To the millisecond? I suppose it's possible. Even so, rumors drive spikes all the time. Someone could put out a false report, committing fraud in the reverse direction. I still doubt such a fraud could be detected at the time in any remotely reliable way.
The report is a government report, in machine-readable format, that is taken from a government server. If you wanted to put out a false report, you would have to hack the server or the connection to the server.
Are you saying that, after reading the natural gas report and having hours to digest it, you could make a better decision of where to hedge your position / whether to place buy/sell orders, than by simply knowing when the report was to come out, and anticipating a storm of activity immediately preceeding it?
If the time that the report was due out was not known in advance, I could see this being suspicious. But just finding a flurry of activity at a time when you judge that it might have been prudent to wait and read the news first, that is not any evidence of illegal activity or fraud.
It is evidence that these day traders are getting more confident in the ability of their robots. That is true.
Rule # Evleventy Billion Six Hundred Thirty Eight:
Look around the room and try to spot the sucker. If you can't find him...
True... it's not like there are a lot of rubes out there with orders 400ms before data is scheduled to be released, who aren't still going to be there 400ms later.
The question is who gets to do the fleecing of the rubes.
So if someone did figure out a way to play the system and get it a little early that's interesting... if someone on the inside is playing fast and loose to make sure the right people get first crack at any small orders that happen to have been left in the system, that's a problem.
But, yeah, it shouldn't be super big bucks at stake...beyond some small initial volume, anyone you're going to trade with is going to know what's up.
Remind me of the film Trading Places and the Orange Juice crop reports.
A pico or nano second is in this digital age is an advantage - it is the fairness of that advantage be it routing or some more underlying more sinister aspect that is the real issue here.
Maybe that news was published 400ms or even a little bit earlier. Such little difference human beings cannot notice. Those who trade early were those who utilized A.I. systems to watch and analyze the report and trade accordingly, 400ms is long enough for big expensive machines to accomplish that task. To add even more conspiracy into the story, you can imagine someone was paid to publish the report just about 400ms earlier than supposed to by those who use computers to trade, and not very many people will notice, or just like post said feds don't even think this matters.
There is a lot of philosophizing over whether HFT harms or helps the market, etc. Much of the pro camp centers around liquidity, but as someone else mentioned, much of that liquidity is absorbed by offsetting HFT.
Rather than get lost in all of the gnarly details, however, I think it is easier to simply look at the purpose of the market and ask whether HFT serves or harms that purpose. IMO, it is pretty clear that it represents a hijacking of the market's true purpose and functioning in the service of that purpose.
For example, is it helpful in setting a price which reflects true supply and demand that we have algorithms designed specifically to manipulate the pricing mechanism by creating artificial supply and demand. These algorithms place phony orders, never intended for execution, but instead merely to trigger a move from the other side. How can that possibly be helpful to such a fundemental market mechanism as pricing?
HFT uses the market for an entirely different purpose. Anyone who defends it must acknowledge this point and argue that the purpose is good if they wish to defend HFT honestly. Otherwise, to couch pro HFT arguments in terms of it being supportive of the market's true purpose and functioning is to mislead.
I assume that such a simple question is intended to allow its author to make a point. So, perhaps we can skip the dance and get right to what your point is?
Else, if you really are unaware of the market's purpose, perhaps you can spend a little time on Google. There is much available information online that you might find helpful in answering your question.
What perplexes me here is the enthusiastic response received for the development and deployment of bitcoin mining hardware (for which the motive is giving a hardware edge to people in order to make lots and lots of money) versus the condemnation of HFT (for which the motive is giving a hardware edge to people in order to make lots and lots of money). Both cases require a significant outlay of resources and talent and I have respect for the work that went into each.
And this gets at a bunch of people, including yourself, claiming that HFT somehow defeats the "purpose of the market(tm)." But if I look this up on Wikipedia, I get the notion that it exists to help corporations raise money and secondarily to provide an indicator of the general economic mood. I fail to see how HFT goes against this. That said, I'd support everything Mark Cuban suggested short of the trading tax (because I believe (without proof) that it would have unintended negative consequences far beyond HFT).
Mark Cuban aside, I mostly see various people who, to quote an Amazon review of "Broken Markets: How High Frequency Trading and Predatory Practices on Wall Street are Destroying Investor Confidence and Your Portfolio", sound like the reviewer's grandparents explaining why they don't use computers. No hard proof - just lots and lots of fear-mongering.
So much so that I'm starting to believe that the underlying issue here is resentment and jealousy that they didn't pull these hacks off themselves and walk away with the big bucks.
Finally, how is HFT worse than the "good old days(tm)" where major brokerages all had seats on trading room floors from which they could manipulate the bid/ask spread with impunity? I'd counter that it's actually better now that anyone capable of getting the cash and talent upfront can get into the same game that's been played since the very beginning, except that the pace is continually accelerating and those left out in the cold are succumbing to future shock. Buffett knows how they'll deal with the events preceding an actual Singularity(c) if one occurs.
But that's just my opinion, feel free to disagree.
Ah, now there's an honest response. Thanks for the thoughtful reply.
I'm not sure why the comparison between bitcoin and HFT, however. My admittedly limited knowledge of the former informs me that the purpose of using compute intensive operations to mine bitcoins is to prevent arbitrary creation by limiting the number and means of creation, thereby attributing value to bitcoins.
I will allow that my understanding could be off by some degree but, in any case, there is no comparison. There is nothing in the design, purpose, or function of the market that is served by accruing advantage to whomever simply has the best hardware. Hence, I am really missing the leap that "well, if it works for bitcoin it should work for anything". The market is designed for economic purposes such as raising capital and allowing broader participation in the economic output of society via investment opportunities. To see how counterproductive HFT is to these purposes, one need only take it to the extreme. That is, what if all transactions were performed by HFT algos? Would the market continue to serve its purpose and function the same? Or would it simply become some strange self-serving system that benefits an entirely different group with entirely different agendas and objectives? How would an IPO even work in this scenario?
Now, going back to the point in my original post, one can argue that the new system is better or fairer, or whatever. But one cannot honestly argue that it is the same or, worse, better serves its current purpose due to HFT.
Your argument that HFT is no worse than the good old days is a straw man, so there's not much I need to say there. I will add though that whether spreads, prices, etc are manipulated by major brokerages or "anyone capable of getting the cash and talent up front", it does subvert confidence in the market and its core mechanisms.
1. I don't understand bitcoin mining but that's no obstacle to rendering a negative judgment on a comparison between building bitcoin mining farms and building an HFT firm.
2. The market is designed for raising capital and HFT is deleterious to that purpose.
So, looking at the market today, GOOG is 772, AMZN is 264, FB is 29, ZNGA is 3, and LNKD is 126. Sounds like they're raising capital to me. What am I missing?
3. I'll just dismiss your comparison of HFT profiting from bid/ask spreads to the previous roll of pit bosses doing the same thing as a straw man. And then I'll close with an unsubstantiated claim that HFT has reduced confidence in the market because it's obvious(tm).
Now I'm assuming that because Mark Cuban has wisely decided against investing in markets he doesn't understand that you're reasoning that no one understands the market?
Which to me is as much poppycock as the belief that the market is %100 efficient 100% of the time (for if so, there would be no housing bubbles, no dotcom booms, and more recently, no fiscal cliff chaos and witness http://en.wikipedia.org/wiki/Renaissance_Technologies which whose performance would be effectively impossible if so).
Might I propose an alternate path from futilely fleeing the event horizon of this mini technological singularity? Instead of saying "I don't understand HFT so it's bad" why not use the same physics currently in use to understand the ensemble behavior of 10^23++ molecules to derive higher-level trading strategies? It's worked in the past (http://en.wikipedia.org/wiki/Didier_Sornette) and I have no reason to believe it cannot be made to work again (which means to me that smart people are already working on it or have already figured it out).
" So if I can make a dollar or more per share with a .16 risk, why do I care about HFT's? They might even help me if they decide to move the price up a few pennys when Im close to my profit target and they may hurt me if I was a few pennys away from the stop.(all my stops are in my head, never manually till close to the target or stop. And even then I may sell at market if last 2 moves were upticks. Ithink HFTS hurt the other greedy scalpers and market makers and specialists and whatever goniff's are swimming around looking for a quick edge. But how would they hurt the buy and hold value investor who lets say buys AAPL at 100 and knows its going way higher and doesnt sell till its at 500. Who got hurt. Maybe a penny in slippage somewhere. Irrelevent to the long term player and even to me, the shorter term player. Its the scalpers that get hurt and they supposedly are providing liquidity anyway so their job is done. How do I get affected?"
It's hard to get to the meat of your arguments because they are riddled with logical fallacies. You seem to be trying to wedge as many in as possible.
Regarding your itemized "summary":
1. You seem to forget that YOU are the one asserting the validity of YOUR comparison between HFT and bitcoins. So, the onus is on YOU to prove it. So far, you have failed. Beyond that, I simply stated that I am no bitcoin expert. But, I believe my summary of bitcoins was sufficient and relevant to draw the distinction. If it wasn't, then you should have explained how I missed the mark and why that miss is relevant to your argument.
2. Are you kidding? So the fact that there are people who have cancer but not yet died from it proves that cancer is good for you? I mean your argument here is literally: HFT exists; there are stocks that are doing well; therefore HFT cannot be harmful. Wow.
3. You introduced the straw man. Don't blame me for noticing. As for much of the rest of the stuff you wrote, I never said it.
Ditto the Mark Cuban stuff. Not sure why you invoke him or the question of understanding the market for that matter. Likewise with the 100% market efficiency comment. I never referred to any of that. How many straw men are we up to now?
You do a really good job of arguing with yourself, but you fail to address my original point. Your talk about bid/ask spreads does at least address market mechanisms, but even if we were to accept all of the great wonders that HFT does for spreads at face value, that is a far cry from rebutting my OP, as spreads are but a small part of the picture. In fact, it is so insignificant to this discussion, let's call that one a red herring.
I also noticed that you left my example in the extreme completely untouched. Wise decision.
In any event, you joke about my not offering evidence, but the irony is that much of this actually is self-evident. For instance, who do you think is sitting on the other side of the trade when HFTs profit? I won't ask you to buy my trademarked brand of common sense this time though: http://www.nytimes.com/2012/12/04/business/high-speed-trades...
That's an actual study, not an Amazon review from some random "day trader". BTW, it is slightly hilarious that you referred to that review as "evidence".
A fun thought experiment. Suppose someone invents a time machine that gives the correct price of all securities at all future points in time.
1) Would it be against current rules to trade on this information?
2) If someone did use this machine surreptitiously to their own gain, how quickly will they approach owning 100% of everything?
3) If the entire data set of future prices were made publicly available, what would happen to markets? I mean, exactly, what would happen to stock prices?
2) Depends what they're trading, and how much capital they have to start with. If our inventor can only buy and sell the S&P 500 they can't make a whole lot - it generally climbs at a pretty steady, known rate; when it moves dramatically it's usually a fall rather than a rise, so you need to be able to short to take advantage of those moments. If the inventor's buying and selling junk bonds, or advanced derivatives contracts, they can make money pretty quickly for as long as they can find people willing to trade with them.
Also the total value available to be made in HFT is pretty low (on the order of tens of billions of dollars/year). To make serious money you need enough capital to be able to make somewhat longer-term investments.
3) Prices for mature, stable, dividend-paying companies would increase a little bit so that their P/E matched that of Treasury bonds. Futures contracts would probably stick around (although they're now essentially just another form of loan) but options would disappear, because you'd have to be stupid to buy/sell them.
The effect on the industry is that there's a lot less inefficiency available to exploit, so much less money to be made. The industry contracts; market-making becomes a boring way to earn small amounts of money, like car insurance. The smart people go elsewhere.
For riskier investments and especially young companies we'd see more dramatic shifts; a share in Facebook is now worth however much a share in Facebook is worth as a mature company (discounted by the risk-free interest rate). So a few would shoot up to 40x their current value, while many would drop low enough to be delisted.
In the slightly longer term an IPO becomes some kind of weird singularity - everyone knows which companies are going to be huge, as soon as they go public. The SEC would hopefully relax the rules (because their reporting rules are now basically obsolete) so we'd see companies being listed much sooner, or even funded based on whether or not they show up in the future data. Suddenly, every startup is a megahit, and there are many more of them, because it's now a safe way to make enormous amounts of money, so everyone wants to found one (or at least, everyone who has it in them to found one that works). The market abhors safe ways to make lots of money, so all the investment money available pours into this, and we hit the singularity in fairly short order.
They wouldn't even need to trade securities. Just search the future data for some successful tech IPOs and then invest in those companies early and heavily. Returns would be fantastic, and the risk of feedback from your future-influenced actions would be much lower.
Chances are, the existence of a perfect trader would cause enormous shifts in how a market handles trading. This trader would, by default, be barred from trading, because her influence on the market would be staggering to the effect that it would influence the market itself. Even Warren Buffet has to temper things he says, so as not to accidentally push the market around.
But, assuming the FTC just decided to take a nap on the whole thing, I imagine those shifts in trading would so quickly dilute value that you'd end up with a system very similar to one we see now: machines doing the majority of the trading, but rather than on algorithms, more on observing Perfect Trader X.
If I were the trader with perfect information, I'd program my trading algorithms to lose (or at least not be guarenteed to win) 40% of the time. This would make you look merely like a incredibly fortunate trader, and not a perfect one. Just make sure that your algorithm guarantees an X% return per year.
On 3, I think it depends on your time-travel rules. Because the first thing that would happen is that people would flock to retool to produce the higher value commodities (if corn price > wheat price, people would stop making wheat and start making corn), causing a supply glut and price collapse.
I think based on "invent a time machine" the implication is that Perfect Trader X (I imagine him dressed like Racer X) would make a trade, then gather future information based on his latest trade, then make the next trade.
Is there any way to subscribe to these Nanex posts? Couldn't find an RSS read but I'd like to slowly dip into HFT, and they like a good source of trading news.
>Earthquakes rarely coordinate their timing with market hours, to name one example.
When speaking of trading and earthquakes, it's hard not to mention Nick Leeson.
The beginning of the end occurred on 16 January 1995, when Leeson placed a short straddle in the Singapore and Tokyo stock exchanges, essentially betting that the Japanese stock market would not move significantly overnight. However, the Kobe earthquake hit early in the morning on 17 January, sending Asian markets, and Leeson's trading positions, into a tailspin. Leeson attempted to recoup his losses by making a series of increasingly risky new trades (using a Long-Long Future Arbitrage), this time betting that the Nikkei Stock Average would make a rapid recovery. However, the recovery failed to materialize.
Leeson left a note reading "I'm Sorry" and fled Singapore on 23 February. Losses eventually reached £827 million (US$1.4 billion), twice the bank's available trading capital. After a failed bailout attempt, Barings was declared insolvent on 26 February.
Delay every trade by 10 seconds. Not only does it prevent spikes like this, but it will ensure trades done with thousands of transactions per second by software are brought down as to not cause stupid crashes : http://en.wikipedia.org/wiki/2010_Flash_Crash
To make shenangins more obvious, what if 1 minute were the maximum resolution that any trade could happen? Say, every order gets a random number of seconds between 0 and 60 added to it before it is executed. Or even longer. What would happen if everyone gets 10 minutes to digest any news?
This sounds like the type of regulation that people outside of an industry put on the industry with good intentions but really no idea what the consequences would be.
Consider a company who holds a press conference announcing something huge (either positive or negative). Anyone wanting to buy or sell in this tiny window pretty much gets shafted by such a system.
I think that's exactly the point. So the whole world gets to trade after they've digested the news, not the guy with the fastest computer or the shortest wire to the exchange.
And why exactly shouldn't the guy who's invested in the best hardware have an edge? Should perhaps we also mandate that all software engineers use exactly the same 2004-era Acer Pentium 4 laptop so that any difference in productivity is strictly due to programming skills?
This. The entire reason for having this market is to reward making better decisions about allocating society's resources, not the same decisions imperceptibly faster. The market's clearing system is flawed in ways that reward huge misinvestments in solving the wrong problem.
Didn't see anything about rewarding better decisions about allocating society's resources. I did see a lot about raising capital for corporations and for providing an indicator of the general mood of the economy though.
The stock market is effectively a giant casino for pros. If you're not one, you're just plain dumb if you try to play their game by their rules.
And of course, with the sort of thinking you're putting forth, we should obviously OUTLAW that 66 GH/s bitcoin hardware. It's so unfair to everyone else who can only afford a cluster of 8 AMD 7970s, no?
And the remedy for that is for the demographic who believes in "hot streaks" in casinos and professional athletics, who thinks the lottery is a good investment of their cash, and fashions themselves as homebrew "masters of the universe" to listen to Warren Buffet and invest in index funds rather than trying to outcook Bobby Flay in his own kitchen.
"And why exactly shouldn't the guy who's invested in the best hardware have an edge? Should perhaps we also mandate that all software engineers use exactly the same 2004-era Acer Pentium 4 laptop so that any difference in productivity is strictly due to programming skills?"
Apples and oranges. Should we also mandate that doctors only use scalpels?
Any analogy quickly becomes ridiculous as soon as you compare HFT to anything of actual use in this society.
You mean the people trying to shaft the other people would cancel each other out and the ACTUAL larger market trends would work themselves out over the long run?
Yes, I am admittedly absolutely naive to the inner workings of the market.
So what would happen in the press conference case? You would make your move based on the news knowing that sometime in the next 10 minutes it would be executed. No one else has any advantage so they can't necessarily get it done quicker. The value of short term moves is reduced, so everyone plays with a longer view.
There is one huge problem with this, which is that under no circumstances can you reward people for placing larger orders than they can actually execute. If you do, you encourage people to make large orders in the hope that only part of them will execute, which can have catastrophic consequences if for some reason the whole trade executes in some unexpected case.
Any form of lottery is vulnerable to this. If you clamp all the timestamps to ten minute intervals, then you need some arbitrary rule to decide which of the valid trades in some time window to execute. If you randomly choose a few, you encourage people to put in more orders than they can execute. If you satisfy them partially, you encourage people to put in larger orders than they can execute.
As a result, the only way to do this that encourages proper behavior is to have some arbitrary competitive rule in place. The easiest and most obvious such arbitrary rule is the fastest order wins. This might seem anti-competitive at first glance, because it keeps out the tiny investors who can't get enough capital to compete in this area. But really, it's not that bad because it only takes a few million dollars if that to get top-notch hardware in a Manhattan or Chicago datacenter. In addition, market-making is by-and-large a commodity service: the entire HFT community can only make money when the market is inefficient, and market-maker competition makes the market more efficient. The HFT world makes money off of market inefficiencies, which are capped by the size of the market and the amount of outside investment. It's a hyper-competitive world of arbitrary rules, in place to encourage a more efficient market for investors. So you don't want to do it yourself, and you don't want to regulate them because you want HFT to be a insane low-margin nightmare for the benefit of everyone else.
Interesting. Thanks for the detailed response. I guess I can always naively assert that traders would figure out how to manage that bit of risk (that more of their trades would execute than they really wanted) but your explanation makes sense. Fastest order wins and, yes, that is arbitrary. Imperfect, but it keeps the market relatively efficient.
I know very little about finance, so treat this as a genuine question: what would be wrong with preventing people from buying or selling in such a tiny window? What would be the downside?
Or perhaps to encourage more long-term thinking there is a rate limit to the number of trades you can execute. We could start slowly: 1 trade per second, perhaps.
If the report is accessed via HTTP, I wouldn't be surprised if the clocks on the government server are off by a few ms, so all the HFTs that are pounding the URL are trying to get access to the data. The first one who got it made their trades before everyone else.
What is the benefit of trading over ms resolutions. What problem is it solving?
Wouldn't trade be more efficient if it were lock stepped - say one trade per hour (per day?): you agree your trade and the exchange processes it on the hour.
What would be lost that benefits the pseudo-capitalism of these systems by having such a regime. How would this negatively impact production.
This part is actually a good thing. That said, I remain gobstopped that HFT is such a whipping boy in a world where no one responsible for the 2008 crash has been prosecuted or jailed.
The incident here seems to either be a clocking artifact or an excellent exploit of a momentary inefficiency in the market - just the sort of thing traders spend their careers on hunting down and exploiting - so why... so... serious???
There are in fact places where the exchange processes trades once per hour. They are called "Call markets". Their disadvantage is trades only take place once per hour.
>What is the benefit of trading over ms resolutions. What problem is it solving?
The problem it's solving is that people want to do it. Exchanges on which they can't are outcompeted by exchanges on which they can.
>Wouldn't trade be more efficient if it were lock stepped - say one trade per hour (per day?):
No.
>you agree your trade and the exchange processes it on the hour.
You "agree" your trade outside the exchange? What do you think an exchange is?
>What would be lost that benefits the pseudo-capitalism of these systems by having such a regime. How would this negatively impact production.
Spreads would get wider, i.e. more investor cash would get creamed off by middlemen, leaving less for the productive companies it was actually invested in.
>The problem it's solving is that people want to do it. //
As a generality I'd say that's false. Why buy in to a system that makes a tiny proportion of the populous vastly wealthy only because those people already are wealthy.
>You "agree" your trade outside the exchange? //
No, hence the conjunction. Perhaps "you issue a bid or offer" would have been better?
>Spreads would get wider //
OK, can you give a reason why that happens and why it leaves more with middlemen. At the moment a change in price so transient as to pass in milliseconds gets exploited to extract value from the system. In essence surely a greater spread means more to lose from selling quickly, that would appear to tend to stabilise.
Presumably the middle-men here are primarily market-makers.
To this layman it appears that traders currently extract value by reacting fastest to the rapid variations. That the value extracted far exceeds the original notion of guaranteeing trades in order to provide liquidity. Slow the variations and there are less opportunities to extract capital. Again this seems like it would tend to money being invested for longer term growth.
>As a generality I'd say that's false. Why buy in to a system
Because it gives you a better price. There are two sides to any trade; people go to the HFTs because their prices are better than anyone else's. If they weren't providing value, no-one else would trade with them.
> that makes a tiny proportion of the populous vastly wealthy only because those people already are wealthy.
HFT has greatly democratized market-making; in the old days stockbroking was an old-boy's network, virtually impossible for new participants to enter. Nowadays, three or four blokes with computers and one investor can start a new trading firm, and many of the biggest HFT players started that way.
>OK, can you give a reason why that happens and why it leaves more with middlemen
Because there's a higher risk. If I offer to buy microsoft for $50 but that offer has to stay out there for an hour, and news of a lawsuit comes out 20 minutes later, I'm going to lose lots of money. So the middlemen need to be able to bear that risk, they need much bigger capital reserves, and they can't afford to offer the penny spreads we see nowadays (because they need to make a greater profit to offer the same return on their bigger capital reserve).
> At the moment a change in price so transient as to pass in milliseconds gets exploited to extract value from the system.
Where's the value being "extracted" from? Certainly not from a fundamentals trader, who's getting the best possible price with the smallest possible spread (sadly, regulations require shares to be sold in increments of $0.01 and no smaller, so there's always $0.005/share to be made on every trade, which by modern standards is absolutely huge - and is why the HFT guys are willing to spend so much on low latency to maximize their chances of getting that $0.005/share. But the fundamentals trader always pays exactly $0.005/share; the HFTs are just fighting among themselves for who gets it).
>there's always $0.005/share to be made on every trade //
I [clearly] don't know enough to know if you're exactly right, but lower the latency and increase the trades and there you have it. Aren't bids and offers listed in pips (like $1.4032).
As I see it production, processing, administration, etc. are the only value inputs. When a 400ms glitch extracts something of the order 1e5 USD the value that money represents comes from those inputs. Yes liquidity is an administrative input but the way the system is set up trades appear to extract far greater amounts of money than their value to society; of course that money comes from other investors, but money is not value, the value the money ineffectively represents is brought to the system by those said inputs.
The problem appears to be that those in a position to rectify the aberration are too busy getting rich off it to care.
Thanks for your input(!) and education.
As an aside, do you [or anyone] know of something along the lines of a (FOSS) toy stock market program, something that allows one to model a market futz with parameters on trades (like changing minimum stock increments, or fixing time periods) and see the effects graphically. Like an ecological modelling program I suppose.
>lower the latency and increase the trades and there you have it
It is true that lowering the latency and more to the point narrowing the spreads increases trade volume, which I kind of glossed over, but again if you lower your margins and sell more of your product you're not extracting value but creating it. Fundamentals traders are (hopefully) buying and selling shares for good reason; helping them do it quicker and more cheaply is a good thing.
>Aren't bids and offers listed in pips (like $1.4032)
AIUI there's an exception for low-value shares, but most are required to be sold in increments of $0.01. (Of course that only applies to stocks traded on public exchanges, which is by no means all or even most HFT activity)
>As I see it production, processing, administration, etc. are the only value inputs. When a 400ms glitch extracts something of the order 1e5 USD the value that money represents comes from those inputs. Yes liquidity is an administrative input but the way the system is set up trades appear to extract far greater amounts of money than their value to society; of course that money comes from other investors, but money is not value, the value the money ineffectively represents is brought to the system by those said inputs.
You're right that there's a kind of "tragedy of the commons" going on; because there's that massive $0.005/share to be made and free competition on latency to be the company to get it, the competing companies naturally push harder and harder until they're all spending $0.004999/share on FPGA programming and the smartest employees they can find to get that $0.005. But it is at least kind of circumscribed; it's that fixed (ish) quantity of money getting wasted, nothing more.
>The problem appears to be that those in a position to rectify the aberration are too busy getting rich off it to care.
Maybe. I've seen elsewhere in these comments that large institutions are now trading directly with HFT players like GETCO and Knight, because they can offer better prices (narrower spreads - less than $0.01) there than they can publicly. These guys are now doing their own trade crossing, effectively acting as a private exchange - and competition between these private exchanges will make the spreads narrower still, and reduce the rents the market makers get. Of course, there are all the downsides of a private exchange - without a public order book it's a shark pool in the same way as the bond market.
For those who are skeptical of Nanex, or want some background on some of the problems with HFT, here is a Dec 2012 analysis by Credit Suisse. HFT Measurement Detection and Response
I really hate that I can't find articles when I need to. However, I have read that investment companies build datacenters as close to internet hubs as possible, to take advantage of the extra milliseconds gained.
If I recall correctly, this is notable in Manhattan.
I always get a kick out of reading these NANEX reports. Reading a new report usually means I spend 30 or 40 minutes filling in the gaps (giant) in my knowledge. Are there any other organizations that put out similar quality reports?
Would putting a minimum time barrier on HFT systems help? Basically say, you can't make any trades faster than 100ms. You still get the benefit of fast HFT systems, but not the insane microsecond stuff going on now.
Those are some heavy graphs, but I wonder what was the spread before the release of the news?
Traders buying early on the news might be dealing with a big spread as everybody is expecting the news to come out.
Is there some kind of wargame where we could all write little algorithms to trade/compete with one another for a couple minutes, and then check the winners/learn strategies?
Somebody has to provide evidence (and, you know, a perpetrator) to a prosecutor first. But there's no real "investigation team", save maybe FINRA.
If FINRA detects something they can refer an investigation to the S.E.C. or other law enforcement body. But they don't exactly have a stellar track record. And nobody even has to listen to them; they just gather evidence and make recommendations. It's very much a dog with no teeth.
The high-frequency traders would pay $1,000,000 /year salaries to high-speed riddle solvers who trained themselves on Jeopardy shows for high-speed buzzer pressing.
Is cost/return a factor in these things? I'd hope not. If expense stops prosecution, the offender can learn where the cut off lies, and just offend below the benchmark.
Somewhat like the way I drive (speed camera goes at 10kmh above limit, I drive at just under this).
A rough approximation off the graph: volume before the announcement ~190,000. The announcement pushed the price down by ~$0.30/share, so by trading before the announcement, they avoided losses of ~$57,000.
So, that initial downwards spike at -400ms that immediately kicks back up to the halfway point?
Those can only possibly be pre-programmed strategies. From the big boys. 'Cos you can't stack 'em exchange side last I checked, if they're price dependent. Which means they have a gloriously low-latency-close-to-the-exchange-link.
Which means this news was leaked well before the event.
Sure you can "stack" them. If you place a sell order larger than any of the buy orders on the exchange, then what you will see is a "stack" of executed orders beginning with the highest-priced buy order and working down to cheaper ones.
In any case, the bounce back looks to happen over ~100 ms or so, which is plenty of time for anyone's strategy to catch up. I don't see how you can conclude there is some big player who knew "well before the event."
What if someone got the natural gas report was earlier than 400 ms but used a computer whose clock set 400ms too early?
It then would be the regular "insider" knowledge trying to pretend business is as usual. But he got noticed because instead of starting abusing the system at 10:30:000 he unknowingly started abusing it at 10:29:400.
Petty amounts of money anyway and nobody is forcing to gamble in the stock market ; )
"Veteran traders would usually wait in anticipation for the weekly report of gas-inventory figures by the U.S. Energy Information Administration released on Thursday at 10.30 AM and then dive into the busiest trading window of the week. This is no longer true as most traders are now staying out of the market due to the HFTs new strategy - sending floods of orders in an effort to trigger huge price swings just before the data gets released, also known as “banging the beehive”."
edit: Fast algo puts orders out on multiple equity exchanges and then hedges itself with a slow algo in the futures market.