Happy to answer any questions about this. It's been a side project that turned into a full blown obsession. There is nothing too secret about the system since it's more about having a solid platform that you can plug your strategies into. I'd probably even open source it but I'd have to clean up all my hacks :)
"Tech Trader is a fully autonomous trading system live with no human intervention or updates, now for over 10 years. It is unique from conventional algorithmic systems, not only because it actually is fully automated, but because it takes a "human" approach to markets. It is not quant. It is not stat-arb. It is not high frequency. It is a program that looks at stocks the same way a person does but with the cold discipline and infinite attention span of a machine. It is analogous to having a thousand independent traders each focusing on a single stock, as opposed to a single quant manager trying to make sense of a thousand datapoints. A person doesn't think through stats, correlations, or complex math models when trading, and neither does Tech Trader. Tech Trader leverages technology to do what human traders do at scale rather than approach markets from the point of view of an academic, mathematician, or scientist.
Since its launch in Dec 2012, Tech Trader has been trading live capital completely on its own, fully automated in the truest sense with no human input, no tweaking, no updates. It is, for all intents and purposes, an autonomous hedge fund, one of the first to truly trade unsupervised for years on end. Whereas many "automated" or "AI" funds may have a hundred scientists providing the actual intelligence behind the curtain, the creator of Tech Trader consists of just one person - a self-taught individual going by the gaming moniker pftq, who created the system at age 21 and has long moved on to other interests. "
Awesome write up. I have a similar project in Go myself, although I just use minute bar data instead of realtime ticks.
Can you share your approach for plugging in various strategies? I quickly learned that having a pluggable strategy system is tricky as it could span across multiple layers of the system.
Also, with backtesting, are you storing and replaying all the quote/tick data? or just using the historical aggregates?
I wish this was better, but honestly, I'm just hard coding them right into the BUY loop. So, I need to restart the app anytime I want to change something. That's why I build this logic to dump and reload the state into a gob file (go memory dump essentially). Ideally, you'd have some type of format to write out your algorithm, and then have a way of hot loading it or something. But, I don't change these enough to really do that yet.
For back testing, I download all raw trades and quotes, and put them into 1 file sorted by time (1 file per day). Then, I compress them using lz4. This allows me to sort of replay the entire market and build up all my intraday backtesting from the source. This took me a long time to figure out and build but has been so worth it. So, I have an off-line script that basically, loops through these files, and replays the market, and makes simulated trades, and then spits out what would have happened. There is a GUI for that too so you can go in an explore the trades and see what triggered the buy and sell. I have seen nothing that goes backtesting for intraday like this.
This is super inefficient but I'm just building the aggregates on the fly. I could probably cache them somewhere but it takes maybe 4-5 minutes to replay a days worth of trades/quotes and build all this so I haven't bothered yet.
Count me in the group of solo algo trading developers (Go and Python and Tradingview Pinescript and other sruff). Planning to send you an email but maybe we can form a group/discord or something online.
I've been algo trading as a side project for a few years now, but all through TD Ameritrade. I get all my quotes and whatnot with them, and it's free with my accounts. I also get other data elsewhere, which is free as well.
I was curious in getting a websocket setup with Polygon after seeing your post, but I noticed it's $29 for first package with the websocket feature. Then I noticed that the package with real-time data is the Advanced package at $200 a month. I'll admit, real-time with unlimited API calls sounds pretty sweet, as my strats rely on to the second data. However, I don't know if I can justify $200 as my algos run no more than 78% win rates with a normal bankroll funding it all. I'm looking to save wherever I can while I build these things and get a passive income stream rolling in.
I would love to know which package you were using, as I didn't see it in the time I quickly read your post? Also, any pro/cons to that specific package? Any and all other details are welcome, and if you would rather respond to my email, it's in my profile. Thanks!
You connect to IB's TWS API... If you execute your trade there only, wouldn't it be an option to fetch the price in real-time from the IB API and not having to use polygon.io at all?
I have no idea: does IB send the price feed in real time? (they certainly send the data in real-time to TWS as it constantly updates right? But is the order book available through their API?)
Basically and even though I know this was published on polygon.io's blog, would that work by only using IB / TWS's API?
Yeah, I initially tried that actually. They have very low resolution data and do not provide raw trades/quotes (something like a message every 250ms), you cannot watch the entire market so you're stuck with like 100 tickers [1] vs 5500 tickers, and fetching historical data is very cumbersome. You can basically pull down pre-aggregated candlestick data and I wanted to base things off the raw trades/quotes. I actually probably spend a month or two trying to make this work. So, that's why I'm using polygon.io in that you can stream trades/quotes for the entire stock market in real-time, without any caps, which is amazing!
I've been going through this journey myself. I started learning on Tradingview, then bought Build Alpha to discover how to test strategies. I chose Portfolio 123 for my automated factor trading but had been working towards creating a program/basket trading system like yours that can act on intraday data.
I moved to long-term investment until I could build a simulator capable of verifying the correctness of my investment strategies using fuzzy testing ideas stolen from TiggerBettle.
I have almost two years of polygon quotes and trades for the whole market captured with a monotonic timestamp to be able to replay the data –and test the handling of polygon socket glitches.
I'm focusing initially on capturing the data in a way that allows fast replay and aggregation, similar to what Kafka can do with topics but in-process using zig and custom memory-mapped data structures. My idea is to be able to generate signals like VIX (once I add options data), ETFs, and indexes and hopefully be faster at doing so than others :), please HN folks, call me out here if I'm being too naive.
This has been a three-year learning process for me. I have been a retail investor for +10 years, but over the last three years, I've gone deep into learning algo-trading, drank del Prado Kool-aid, and read numerous trading and investment books.
I'm now focusing on my technical chops to build the engine to build order books for individual stocks, baskets, and indexes with realistic market prices. I aim to develop a system that can get as close to the market price in the next dollar bar as possible.
This has been a very lonely journey, and after reading the responses to this post, I'd love to connect with others on a similar path. Sending you an email!
I've been really wanting to use Go, but as you say, much of the community is Python due to the data analysis strengths. To the detriment of the other things Python does do poorly.
Can you give some thoughts with your experimentation on the following from a Go perspective.
1. Supported TA libraries in Go. I'm familiar with TAlib (python), bloom, etc.
- certain forks tailored to real time rather than historical (eg: no re-compute on ticks)
2. Data storage (article mentioned you're all in memory). I've been using S3 & ArticDB
3. If your in-data memory is treating you well for multiple TA calculations (example: in Python, you can compute & save pickled dataframes - and re-read those over longer time periods)
> Supported TA libraries in Go. I'm familiar with TAlib (python), bloom, etc. - certain forks tailored to real time rather than historical (eg: no re-compute on ticks)
I've been basically, just manually coding the algorithms from python into Go. ChatGPT is amazing at this. I really only just about 4 so it was a one time thing.
> Data storage (article mentioned you're all in memory). I've been using S3 & ArticDB
Yeah, I ran into issues and then was like what would be the fastest, then just went in-memory. I download all raw trades/quotes each night and store then into gob+lz4 compressed files. Then for backtesting and stuff I can load these in and build the aggrogate bars on the fly.
> If your in-data memory is treating you well for multiple TA calculations (example: in Python, you can compute & save pickled dataframes - and re-read those over longer time periods)
Yeah, I have a historical lookup table that I build nightly too. This gives me a reference point when I'm doing % change calculations and stuff. I should probably have mentioned that.
I'd be careful using ChatGPT for this. I tried the same recently to bootstrap spot yields from par, which is a fairly common code problem with hundreds of examples online. ChatGPT couldn't do it. It produced code that looked right, but would fail my tests abysmally. I ended up writing it by hand. I hope you are validating their code with known data.
Yeah, I'm heavily testing everything it writes. So, I'm very sure it's correct or get it close enough and then code it. I've seen ChatGPT add imports, functions, etc that don't even exist in Go. What I've found so useful though, is not even coding, but asking it how to solve problems, and then having it code things up. With Google, you need to pretty much already know how to solve the problem then go looking for answers. With ChatGPT you can ask it how it would solve this problem. For algorithms and stuff it have been unreal. Thanks for the warning though. You're totally right.
I had not looked at Mojo... thanks for the pointer.
In the past, I've had issues with library compatibility with compiled derivative languages of interpreted ones (eg: Crystal of Ruby, etc, etc).
Know if Mojo directly uses existing Python ecosystem?
I'm excited for Mojo, but it'll be years before it's ready for prime time as a general purpose language. It's not even available to download yet AFAIK...
I've used ATR bots for years, and would love to hear your thoughts on how what you're building delineates itself as a distinct product strategy beyond just the programming language GO.
I've used wonderbit, zigz, and some of the others.
No way. Not even close. That would be the dream though. Taxes are insane in Canada. Like 50% insane. Housing and kids just eat money. Maybe if I was renting and didn't have a family but that's not happening.
