I wish there was a light version of gpt3 with maybe like 100 api query limit per day so just amateurs like me got to play with this API too on weekends.
I have no agenda of changing the world and I have nothing substantial to fill in the gpt3 invite form but sometimes the best ideas come from tinkering and I just wish they gave some limited access to everyone.
You can run that and deploy your own GPT-2 API locally, or in the cloud. I'd also recommend checking out Max Woolf's gpt-2-simple, which is a library that makes it really easy to fine tune GPT-2 with your own text (if you've played the free version of AI Dungeon, this is how they trained their model):
out of the box probably not. gpt-3 is pretrained on a much larger corpus and is a much higher capacity model, which is why its been performing very well in zero-shot learning examples like the one above. That being said you should try it.
Note the huggingface library is a python library, so you'd technically be writing some code, though the library API is super user friendly and the community is very helpful.
"Recommendation system is so successful in many products and services we interact everyday" ->
"Recommendation system[S] [ARE] so successful in many products and services we interact [WITH] every[ ]day"
I'm sorry, but four errors within the first sentence is just not good enough - did the author even read this themselves just once before hitting publish? Show just a little bit more respect toward the people who give their attention to what you've written.
Thanks for pointing out the errors. I got them fixed in the blog.
This is also a good use case of GPT3, checking grammar and spelling. I might build an application for this.
Thanks for fixing them - and sorry if I was overly harsh. Sharing your writing with the world is nerve-wracking enough already without having the spelling and grammar shot down on HN. Truth be told, my first thought was "maybe this is some example of text created by GPT-3?" But I think you're right that it would actually serve well as a combined spelling and error checker to catch things like that - I hope you do build an application for it, that would be neat.
Sure, neither is mine. But if you don't have the skills to proofread what you've written yourself, then get someone to help that has. I don't expect to be able to write something in Chinese deserving of a wide audience all on my own either.
A lot of people write content like this for nothing or next to nothing. If the bar to publish a blog post is "needs an editor" then there's a lot of content that simply won't exist any more, because people don't want to pay money to read blogs (at least I know I won't).
That is quite probably untrue, and I'd wager many best selling and critically acclaimed books contained a number of 'howlers' on the publication of their first editions.
For anyone else curious, the pricing according to gwern [1]:
Explore: Free tier: 100K [BPE] tokens, Or, 3-month trial, Whichever comes first
Create: $100/mo, 2M tokens/mo, 8 cents per additional 1k tokens
Build: $400/mo, 10M tokens/mo, 6 cents per additional, 1k tokens
Scale: Contact Us
Expensive but (unfortunately) pretty standard market price for cloud NLP stuff. The price is about $40 per million tokens.
For comparison, cloud translation costs per million characters range from $6 (Azure bulk pricing) to $20 (Google/AWS without bulk discounts). If we assume the average BPE token length is 5 characters, GPT-3 will cost about $8 per million characters.
In one of the Reddit comments about a service (PhilosopherAI.com) using the API outlining the possible total costs:
> That makes for 400 million tokens in 2 or 3 weeks, which puts me at like $4000/mo minimum
Oh dear, that means...
> Scale: Contact Us
So while they are using the API for free for now, by these metrics, they are least going to be soon having a running cost of at least $4K+/mo. That is an expensive toy. The same goes for AI Dungeon.
> App Store earnings will be used to cover OpenAI costs & develop new features. Thank you for your support.
Well I will be seeing a tsunami of complaints from OpenAI projects that they can't pay up for their "game changers" since they will be priced out quickly. OpenAI Wins, Everyone else loses.
If this is for simple projects, then I wouldn't dare build an entire startup based on someone else's API. OpenAI will win by default here.
I was thinking about Ai dungeon the other day. I can't imagine it really is sustainable with those 10$ subscriptions covering the free play folks too. What happens whenever they get featured on some bigger website?
An important note, as others have said, is that not every player gets the same model. Free players, as far as I'm aware, only get GPT-2 ("only" is a strong word, as up until GPT-3, GPT-2 was the state of the art in text generation and is still extremely impressive in its own right).
Not exactly, GPT-3 uses a variant of BPE [1], so one token can correspond to a character, an entire word or more, or anything in between. The paper [2] says a token corresponds to 0.7 words on average.
If you told me in 2015 that I can give just a few examples of movie recommendations, on a model trained for general text, and get perfectly coherent recommendations, I wouldn’t have believed you.
Indeed. But now OpenAI has become more like another DeepMind, but starting to rent out their APIs to its users, indicating that they probably won't ever release models for their research.
I don't fully disagree, but I also think there is a layer of complication here that goes beyond OpenAI's hopes of monetization (which are clearly a factor).
I maintain an open source ML deployment platform, and I've interacted with a bunch of teams that have used it to deploy GPT-2. It was actually the platform AI Dungeon built their app on. GPT-2 is a beast to deploy—it's huge (almost 6 GB fully trained), requires GPUs, and scales fairly poorly. You need to autoscale GPU instances aggressively to handle any kind of real time inference situation with it, and even with spot instances, that gets expensive quick.
GPT-2 is 1.5 billion parameters, and at the time, was scandalously large. GPT-3 is 175 billion. For a model that large, there's real questions around whether it's even feasible for the average team to use it if it is not hosted somewhere else as a third party API.
From that perspective, I think the value OpenAI captures with the API is less about the exclusivity of the model itself, but the exclusivity of their infrastructure. Because of that, I wouldn't be surprised to see them open source the model for research.
However, I 100% agree that the fact that the model still isn't open is concerning, and it casts some doubts on whether or not it will ultimately happen in the future.
I guess it depends how you think about it. From computational perspective, it's heavyweight. On the other hand, it really optimizes for minimal human effort, on several levels:
* GPT-3 is conceptually simple: the effort to invent it and engineer it amortizes well.
* Once the model is trained, it can be reused to very different tasks.
One thing that's cool about GPT-3 is how versatile it can be, so I'm exploring a natural language search interface, which is something that traditional recommendation algorithms can't do.
Are you planning to incorporate GPT-3 into your startup? I'd love to hear about what use cases you envision it solving. If you're interested in connecting, my email is in my profile and my website.
Very creative use of the technology. I wonder when GPT3 will evolve to handle more languages, and what if any steps they take to reduce cultural bias? If I was in Japan these recommendations would certainly be less useful. :)
Given this blog post I have no idea if it even works well for English. Sure, the demo video contains sensible recommendations but what gives? Most of those I've seen seem to be included in the training data shown in the image below. If you have the term "AI" in your blog's blurb readers might expect the author to actually perform an evaluation of stuff they write ads about.
Presumably when a new movie (not included in GPT3 training data) comes out the recommendations are not going to be too good, unless someone spends a lot of money retraining the underlying model?
This will be the case for everything built on GPT-3, but you've got to imagine it's possible for them to keep refreshing it, eventually training or fine tuning it on a daily basis as their web crawl updates.
"Supporters of the Strong AI Hypothesis insisted that consciousness was a property of certain algorithms – a result of information being processed in certain ways, regardless of what machine, or organ, was used to perform the task. A computer model which manipulated data about itself and its ‘surroundings’ in essentially the same way as an organic brain would have to possess essentially the same mental states. ‘Simulated consciousness’ was as oxymoronic as ‘simulated addition’.
Opponents replied that when you modelled a hurricane, nobody got wet. When you modelled a fusion power plant, no energy was produced. When you modelled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modelled the human brain, why should you expect real thought to occur?"
Could you please give an explanation of panpsychism applied to software that make sense? Spoiler: you cannot and it's totally refuted empirically.
If I code a hello world, what kind of cognitive abilities does this have?
BTW I guess panpsychism researchers should start talking to rocks maybe one day they'll find one that is an AGI! /s
The position is that information not physical matter is ontologically primary and that substrate independent integrated information processing is identical to consciousness.
A hello world program and a rock are not integrated information processing systems. The former is an automata which can be contained in an integrated information processing systems and the latter is a more arbitrary construct which does very little information processing at all (much less integrated).
Suppose there’s a highly-detailed molecular dynamics simulation of every atom in your brain: every ion channel in every neuron, any neurotransmitters wafting about, and whatever other mechanisms might be relevant (ephaptic coupling?) for good measure.
This would be frightfully expensive to build and run (BlueBrain blew billions on a very stripped down version)l Nevertheless, it’s not obvious to me that this MD model wouldn’t “think” in the same way you do.
The simulation you describe, if precise enough is isomorphic to a real brain so obviously it's tautologic that it would think the same way as the real brain do (qualias aside).
But panpsychism apply to anything, e.g a rock.
And your statement does not apply to a standard neural network.
Still seems like an interesting question if you replace “think” with “inferred”, “learnt” (in the statistical sense), or any other synonym that’s in opposition to “memorized and parroted”.
Intersteller (with -er, indeed)? How did it pick that one up as an alternative for Inception? There is something with that title, but no information in IMDb.
Anyway, for the rest it looks similar to Netflix' recommendation: similarity in actors, directors, keywords. A toy.
I have no agenda of changing the world and I have nothing substantial to fill in the gpt3 invite form but sometimes the best ideas come from tinkering and I just wish they gave some limited access to everyone.