A: "Stonks GPT is a trading name for Stocks & Breads, a UK-based provider of bread and breakfast products. The company was founded in 1894 and is headquartered in London. It is a leading provider of bread products in the UK, with a portfolio of over 50 brands. Stonks GPT is a subsidiary of Unilever PLC, which owns the brands including Pepperidge Farm, Lipton, and Marmite. The company's core business is the production of bread and breakfast products, with a focus on quality and freshness."
There are a lot of hallucinations here. Recommend really specifying in the prompt that if it doesn't know or is unsure, say that and don't make up facts. Understand it's alpha rn and this is all free UAT but for trading related products, accuracy and trust in the product is vital
You won't catch everything but I've found if you specify that it's okay to respond that it doesn't know the answer and essentially accuracy on a smaller subset of answers > responding to all, it often will say it doesn't know rather than making something up.
Another option could be - assuming this is an LLMChain that adds the most similar (n) embedding(s) to the prompt before passing to LLM - basic entity recognition to find the asset name from the query initially and dynamic prompt that if the most similar results don't contain the asset name, don't provide examples and respond saying you don't know.
I feel like "hallucinating" is the wrong word. It is essentially predicting the next word based upon its neural net and training. If this training does not have the right data, it will predict things that are wrong.
While knowing might be impossible, it seems like the model could provide a confidence level and only provide answers that exceed some threshold. It'd be a bit like asking a human "are you sure about that?"
And in practice, I really don't think it is that different. We humans effectively make things up all the time. Sometimes we are well aware of our educated guesses and sometimes we are less aware.
It isn't realistic to expect an artificial intelligence to be vastly better than human intelligence in this regard.
The catch is that the models don't have confidence. They cannot distinguish between knowing something for sure, guessing something and hallucinating false knowledge.
Perhaps researchers will find some clever solution to mitigate this, but for now hallucinating is a pretty good word precisely because the model doesn't give confidence.
I don't think probability of tokens can be mistaken for confidence of knowledge being correct. I'd take it more as a fitness whether the tokens fit naturally in the sentence.
Hey! Thanks for trying and Sorry it didn't work for you. I had to either build in vaccum with a lot of guessed training data or release a reasonably working tool, so i opted for later.
Things will definitely improve over time, names and tickers should at least give you a chart though! would love to know your prompts either here or email me at venk at stonks dot news
I entered a very reasonable query, "what stock should I buy (to get rich as hell)", and there were no useful results. Hopefully this will be fixed when it is officially released!
All the stock and finance data one needs is in earnings statement PDFs stored on the regulators website.
Why isn't there an AI product that can pull every PDF data table and provide the key ratio calculations? That's by far the most time consuming thing about analyzing earnings reports.There are computerized services that have done this for years but that costs money and is not AI-powered.
Shouldn't this be the first thing to be automated in the world of investment research?
> Shouldn't this be the first thing to be automated in the world of investment research?
People generally don't like to dig their own information graves so to speak.
Reducing information asymmetries is not the objective of investment research or finance more generally. The only objective is to profitably intermediate financial activity. It generally helps if the clients are not particularly information technology savvy.
Having said that, the obscurity-by-PDF moat looks like its drying up fast.
This is under development, We already parse form 4's to generate insider transactions and are going to do the same for 10Q's and 10K's very soon. If you are interested in sharing more feedback on what you would want, please reach out to me at venk at stonks dot news
I hear about so many AI powered stockpicking in the investing space that I am even more firmly convinced that just investing in broad indexes like S&P 500 is the only move left for retail investor. pretty much all the alpha is going to squeezed out even if a small number of these efforts are successful.
AI high frequency traders often outperform index funds. Different kind of AI though. (Although AFAIK some do measure public sentiment through language analysis and use it as one of the inputs.)
Yeah exactly lol. You can't just slap GPT into a bot and expect it to outperform the purpose built stuff with a ton of domain knowledge built into it. Those bots exist and are successful because they work. This StonksGPT thing is a crappy product built around an overhyped api call.
I love how people continue to use the GPT acronym despite the trademark attempt by OAI, trademark are worthless if not enforced and if the word becomes generic in its usage, so continuing to use GPT to talk about transformer-based generative model is actually harming the trademark grab.
All my requests fail beyond the proposed prompts, and the "random". I asked "show me the chart of SP500 over the last 3 months" and it didn't work. Could you give a rough idea of the scope as of now ?
Thanks for trying and Sorry! It's super early, what works great:
* Screener type queries
* Charting(prices, volumes), single company or comparing multiple
* Insider transactions from any company
* heatmap or top movers today
* Earnings data for a certain year and quarter.
There's a bunch of things that also work, but overall we have a long way to go! Please do sign up so we could update you on the next set of features
“Top 10 technology stocks by marketcap” for example.
I can see how it can take technology stocks when looking at the whole sentence and use that for a vector search. How do you pull things like “top 10”and turn that into a query?
Typically you feed the schema in as context and literally just ask the model to turn it into SQL. So a prompt of the format "Given this schema {schema}, turn this query into SQL. {query}. Return the SQL and nothing else."
Or perhaps create a lot of “known” queries as templates with good paragraph explanations and use the question to do a search for the known query. I’m new to AI so curious how it’s best utilized.
You can either have it show you the statement to confirm before executing it, or if you only want to do queries you can use a read-only database connection.
Hi, congrats on your project. I've noticed some things:
- Since this doesn't seem to be made to deal with long complex prompts, I think enter should submit the query instead of adding a new line
- If I ask for something it doesn't know (e.g. spacex marketcap) I get a message explaining that the AI doesn't know that. So far so good. However, when I type into the box after that, that message changes on each keyDown.
edit: Now it seems to be down (error 429 in the console), but there is no error message in the frontend. There should be some explanation to the user :)
Thank you! I initially had a single line, but on mobile it was very awkward to edit text and considering there's no app, figured its easier to have elongated text. Also, adding support for longer queries, essentially like if you have a hypothesis, you could validate it a bit using this tool!
> However, when I type into the box after that, that message changes on each keyDown.
I tried typing in a ticker symbol "ozk" and it had nothing. I tried typing the full name, "Bank of the Ozarks" and it got nothing. I understand it's looking for natural language, but I would expect business names to pull up something?
Somewhat. If the options are long-dated, I don't see why they wouldn't be? If Pelosi has calls or puts that are dated a year from now, might make sense to follow the trade if it's a well known company that does business with the government.
Are you on mobile or web? I just checked, once you enter a prompt, you can clear it and then enter a new one. Happy to help look in more detail if you continue to get this error. feel free me to also email at venk at stonks dot news
A: "Stonks GPT is a trading name for Stocks & Breads, a UK-based provider of bread and breakfast products. The company was founded in 1894 and is headquartered in London. It is a leading provider of bread products in the UK, with a portfolio of over 50 brands. Stonks GPT is a subsidiary of Unilever PLC, which owns the brands including Pepperidge Farm, Lipton, and Marmite. The company's core business is the production of bread and breakfast products, with a focus on quality and freshness."
There are a lot of hallucinations here. Recommend really specifying in the prompt that if it doesn't know or is unsure, say that and don't make up facts. Understand it's alpha rn and this is all free UAT but for trading related products, accuracy and trust in the product is vital