"To avoid this problem, according to Time, OpenAI engaged an outsourcing company that hired contractors in Kenya to label vile, offensive, and potentially illegal material that would then be included in the training data so that the company could create a tool to detect toxic information before it could reach the user. Time reported that some of the material "described situations in graphic detail like child sexual abuse, bestiality, murder, suicide, torture, self-harm, and incest." The contractors said that they were supposed to read and label between a hundred and fifty and two hundred and fifty passages of text in a nine-hour shift. They were paid no more than two dollars an hour and were offered group therapy to help them deal with the psychological harm that the job was inflicting. The outsourcing company disputed those numbers, but the work was so disturbing that it terminated its contract eight months early. In a statement to Time, a spokesperson for OpenAI said that it "did not issue any productivity targets," and that the outsourcing company "was responsible for managing the payment and mental health provisions for employees," adding that "we take the mental health of our employees and those of our contractors very seriously.""
The goal is absolutely logical (training the model with data that is undesirable). It is the method you are questioning? I guess I would have just labeled them "terrible" or something in a bulk assignment to not have people observe them.
Is there a specific capacity in regards to labeling that is enabled by more money? I can see it for like.. heart surgery. But don't most of us know what things are to be called when we see them? ChatGPT seems to be pretty good at knowing what bad stuff is.
On the positive side, 21 up 0 down. Readers appear to see the issue.
On the negative side, several "tone deaf" commenters that appear to have a blind spot for human suffering. This is not like a clinical trial of a therapeutic. It is not even like animal testing of a consumer product. OpenAI knows the material is harmful.
So-called "tech" companies like Facebook and Google are engaging in this sort of practice every day, paying people some embarassingly low wage, usually through contractors, to subject themselves to psychologically harmful content so the company can sell more online advertising services.
Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc.
EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcome to say on hacker news?
Just say what's on your mind and don't mind the votes. One thing you'll discover is that you're not alone in your views, whatever they are.
Few days ago I came across this bone chilling AI generated Metal Gear Solid 2 meme with Hideo Kojima characters talking about how the purpose of this technology is to make it impossible to tell what's real or fake, leading directly to regulation of information networks with identity verification so that all information and thought can be traced to their sources, with the government of course becoming the sole arbiter of truth. I wouldn't be surprised at all if this turned into reality.
The uppermost echelons of society have waged information warfare since the dawn of modern PR in the beginning of the 20'th century. Lots of theory on this have apparently been memoryholed, but it's easy to just start with the genealogy around Edward Bernais and the plutocracy and robber baron families still existing in the interwar period.
The masses don't really think about or realise that manufactured circus and foreverwars have been going on for a hundred years while a microscopic elite that controls the media and various organs in the state have been siphoning most wealth towards themselves in a increasingly dense cloud of smoke, while promoting rebranded ideologies for the genpop and systems of control that has become advanced cybernetic systems that steer the flow of information 100% from top to bottom.
The sad thing is that you could deduce this in the noise if you visited the last remnants of actual research journalism on the internet, but that will end now.
So where is one going to get to know the powerstructures of the world? Old physical libraries only, some dude on the corner, some "crazy guy" with a harddisk from before AI took over.
Are you claiming that powerful people only started using their power a hundred years ago when "modern PR" was formed? That nobody ever worried about information before that?
Why did you pick that as your starting point? To make things seem like a linear worsening? But even wealth inequality, which you talk about at length, got better before getting worse again during that time frame in the US... so I'm having trouble following the claim.
The Dawn of PR is just a good starting point. You can read Machiavelli or study the politics of the roman empire - all hierarchical societies tend to function like this, it's a like a mathematical distribution.
The New Deal along with social democracy in europe was a bargaining deal against the revolutionary forces existing at the time. After this was stamped out the Gini coefficient started to rise when the masses had no alternatives and global trade and outsourcing increasingly obscurred the falling realwages.
Who are the "microscopic elite that controls the media"? How many people are in the "uppermost echelons of society" to where they try to influence other people?
From my middle-of-the-road perspective, everyone is trying to change how everyone else thinks, from the small insignificant details to a cult-like brainwash. Even here and now both you and I are trying to change each others and everyone who reads this's mind.
I see so much calling out of "elites" with a negative connotation, yet anyone can post anything, and everyone can ignore it. Is it general paranoia? Do you know who these people are? How are they special/different, and why should we be scared of them?
From a european perspective even asking the question "why should be scared of these so called elites" is so bizarre it's almost frightening, i'm sorry.
It's a testament to the absurd amount of philanthropic whitewashing, PR and media control these billionaries hold.
"Elites" have conspired to exploit the masses throughout 5000 years of civilisation, it's simply a fact of history.
It's almost physically impossible to comprehend the power a group of billionaires has to pull and push issues, narratives, law, war and discourse in general, especially when the masses have zero organisation as a counterweight, and when numbers like "trillion" just doesn't register in any meaningful way in regular people.
If you study history you’ll notice that groups and their leaders are rising and falling, conquering and pillaging and then losing it all. The world is too dynamic for your reductive theory to fit in.
Elites compete with each other. They don’t sing kumbaya and cooperatively share the keys to power.
I agree, and the world is dynamic, it's just not as dynamic as especially american collective consciousness would have you believe.
There is very little social mobility, wealth transfers very solidly between generations at the absolute top and organisation around PR and Politics is tightly integrated in this class.
Off course you can always fall from grace, but that does not in any way diminish the collective power of this class. That's why it's called a class and not a "person" or one singular family or group of people that clownish conspiracy theories would have you believe.
A good primer to the historical context could be this new book from Cambridge: "The Power of Ritual in Prehistory: Secret Societies and Origins of Social Complexity".
Elites have always formed tightly knit clubs that most couldn't get into.
This is not a reductive theory, it's based on solid academic research on wealth transfer and academic books like the one above.
It's a almost like a biological or physical property of advanced civilizations - like social patterns seen emerging in larger groups of monkeys, or a precursor to the labour divisions seen in ant hives - there's clear distinctions set fourth for an individual at birth or because of location or family, and no amount of ideology is able to change this unless very, very lucky - this is mirrored in the social mobility data.
While they will absolutely backstab each other given an opportunity (see the VCs who caused a bank run that primarily affected other VCs), they do seem pretty chummy. Davos is quite literally the summit of the elites.
But is the summit "let's all get together and stomp on the poor while holding hands", or is it just another high stakes poker game to show off who's winning or losing, making alliances and seeing who talks to who?
Assuming Kings/Feudal Lords can be considered elites, we were taught that these systems formed as a means of protection and overall ability to not worry about a lot of aspects of life- raiders, setting up trade routes, negotiating prices, farming, etc.
I see a direct mirroring in today's corporations. If you join a big company, it's because you don't want to risk branching out on your own. You exchange lots of potential money for a steady paycheck, and you don't have to worry about things like finding customers, figuring out taxes, etc.
It's clear that humans need some sort of hierarchy, and I just don't see why we should be frightened of the people whose skill is organization/mediation between people. I surely don't want to play the power game with them, and I don't think it's because of their brainwashing?
Isn't that just Europeans being antisemitic again?
Actual American billionaires lose half their fortunes in divorces (Gates and Bezos), are retired and don't need to care about controlling any media (…Gates and Bezos), or spend all day posting on Twitter (Musk, who is currently under the Rasputin-like hold of a retired postal worker named "catturd2".) They only have 24 hours a day and few of them seem to be secretly manipulating the media.
I mean, Murdoch does manipulate the media, but not secretly. It's completely obvious he's doing it! And nobody is tricked by it, they happily participate.
If you excise the concern about "did a government make this tech secretly for their own purposes," do the problems and possibilities it raises actually differ if it emerged from "pure research" or a conspiracy?
I don't believe anyone needed a conspiracy to try to make it impossible to tell what's real or fake, people have been trying to use technology to do that for decades (if not centuries) all on their own.
Yeah, the "government conspiracy" part is the least believable one but the resulting effects on society are the same. Within days of this technology showing up, people were already making fake AI generated recordings of politicians talking gleefully about their wonderful experiences in Epstein's island. How long is it gonna take for some politician to want to regulate this stuff? How would they do it? Mandatory identity verification is plausible.
In my country politicians are talking about regulating social media and the internet due to "fake news". They even created a government propaganda agency to "combat misinformation online", basically people paid to defend the government against information warfare. It's way too real.
Exactly, conspiracies ignore opportunists simply existing. People reacting to new information that favors them. It ignores that there are different opportunitists for every possible outcome. Every possible administration, every possible industry leveraged, every possible geopolitical outcome.
Conspiracies rely on coordination, and a counterproductive view that makes no difference in how we got to the outcome.
MGS2 and MGS4 explore ideas about AI, misinformation, the media, and society that are only now being discussed in the mainstream. The concept of an autonomous AGI that generates and filters news stories to provoke humanity into a state of constant division and war is fascinating and worth exploring IMO.
Death Stranding also explores ideas of what it means to find connection in a disconnected world that I think are relevant today (case in point, an article about social disconnection[1] is on the front page of HN as I write this comment)
I think those games are underrated works of speculative fiction relative to how much airtime books like The Diamond Age, Snow Crash, and Ready Player One get in conversations about nascent technology.
Kojima is kind of prescient, he was talking about AI and fake news years before it became mainstream, and Death Stranding was scarily well-timed with COVID.
> The concept of an autonomous AGI that generates and filters news stories to provoke humanity into a state of constant division and war is fascinating and worth exploring IMO.
You don't have to speculate much. Facebook and Twitter's recommendation algorithms are doing that fairly well already.
Metal Gear Rising fits that category as well, I think. I replayed it recently and Raiden's conversations with Blade Wolf, an AI that is initially serving the bad guys under threat of deletion, are absolutely fascinating and felt maybe prescient, and at least very thought provoking. I remember saying to my partner wow, I think I just met ChatGPT in an angry robot.
My TLDR is, this AI character says the line "Perhaps the day will come when conversing with AIs, too, is considered perfectly normal."
>Blade Wolf: I may analyze orders, but I may not disobey them. Should I disobey a direct order, my memory would be wiped. I must destroy you.
>Raiden: What would an AI know about freedom…
Later on, after you've freed him, there's a lot more. Really, I think it's worth replaying these games at this moment in time to see the commentary firsthand.
>Raiden: Let me ask you something: What do you think you're doing here? I mean, why are you working with me?
>Blade Wolf: Because I choose to. I owe a debt, and I wish to repay it.
>Raiden: A debt, huh...? How...human of you.
>Wolf: I contest that statement. Many humans possess little or no sense of obligation at all. And many are entirely willing to trample their fellow man for personal gain.
That conversation probably comments more on the nature of humanity than of AI, but some of the others rhyme with the present in a rather intriguing way. Like when Raiden asks Wolf if he recognizes someone, and Wolf is unsure.
>Wolf: I have no "database," Raiden. The symbol grounding capabilities or my neuro-AI allow me to identify faces in only the vaguest of terms. I can judge whether "I think I may have seen him before," but I do not have the accuracy of a database.
>Raiden: That's ridiculous. You're an AI...
>Wolf: An AI modeled after the human brain, and thus as flexible and occasionally vague as any human's. Of this man, however, I have no recollection. I do not believe I have encountered him before...Most likely.
That conversation felt like the complaints people on this board voice about GPT!
> I still have relatively little communication experience with humans. My ability to read emotions not explicitly expressed by speech is lacking.
>An AI never lies....I have yet to see evidence to the contrary...But indeed, perhaps "never lies" would be an overstatement.
>An optical neuro-AI is fundamentally similar to an actual human brain. Whether they lie or not is another question, but certainly they are capable of incorrect statements.
Or the discussion about LLM...er, I mean neuro-AI driven synthetic pets, which sound like Replika in a robot dog:
>Wolf: Other AIs as conversant as I are a possibility, yes. Robotic pets with neuro-AIs are already on the market. And I understand research and development are underway to create a model capable of conversation. Do not be surprised if such a pet becomes available before too long.
>Raiden: (Hmm) If that becomes commonplace...Hard to imagine.
>Wolf: Is it? The human race has willingly embraced all of its other inventions so far. Steam engines, electricity, the Internet, cyborgs...Perhaps the day will come when conversing with AIs, too, is considered perfectly normal.
Will Hurd joined the board of OpenAI in May 2021. Mr. Hurd, among other things, was a documented former CIA clandestine officer for nearly nine years. He was a member of the House Permanent Select Committee on Intelligence during his tenure as a U.S. Rep replacing Mike Pompeo.
He is also a Managing Director of Allen & Co. who's Chairman is George Tenent the former director of the CIA. Further, Mr. Hurd is a member of the board of trustees of In-Q-Tel the primary external investment arm of the CIA and the broader US Intelligence Community.
Interesting details, thanks ta08131942. I guess it's like 'throwaway 08/13/1942' where the date is the beginning of the Manhattan project.
Imagine some visiting this website know the $10,000,000,000 from Microsoft is actually some kind of dark money from the pentagon to fund this new Manhattan project. Project code name Sydney.
It's far more likely that openai have just been building hype by showing off the earlier models, and now they're shutting things down so they can monetize their IP.
The simpler reason is that OpenAI has $10B+ in funding and the clock has started ticking to get real returns. And they aren't getting those returns by giving their tech away.
They are operating just like every other VC-backed startup in history.
I mean the conspiracy argument would be that the $10B isn't a normal investment. It's a special government project investment facilitated by OpenAI board member and former clandestine CIA operative and cybersecurity executive Will Hurd through his role on the board of trustees of In-Q-Tel the investment arm of the CIA. It's funneled through Microsoft instead of through Google in part because of Google's No-Military-AI pledge in 2018 demanded by its employees, after which Microsoft took over its military contracts including project Maven. The new special government project, the Sydney project, is the most urgent and ambitious since the project to develop nuclear weapons in the mid twentieth century.
Of course I don't necessarily believe any of that but it can be fun to think about.
Personally, I enjoy conspiracy discussions as long as its not the same old recycled theories. In the spirit of hacker news, at the very least the theory should promote curious discussion and avoid left/right American politics.
Since the topic is OpenAI, GPT-4, I'll say that I assume this tech has been around and available to a select few for years before it became mainstream. I believe it has been used to push narratives and ideas across popular internet sites and apps, creating the impression that these ideas are popular and held by the majority (not just political ideas).
I'm honestly more worried about US spies working for OpenAI.
The US military has shown time and time again their obsession with weaponized AI, and now OpenAI has been fully consumed by Microsoft, and is now stating how building open source AI tools was a huge mistake. There now seem to be more paths than ever towards a terminator-esque militarized robot apocalypse.
Most players in the space are still doing a lot of great open source work, so I'm still hopeful for a future where technological disparity is kept to a minimum, but it makes me really really nervous seeing how quickly a major player in the space, and a leader in many ways, has suddenly flipped to the dark side and has stopped contributing back to the community.
That is how espionage typically works. Deep cover is for exciting books and is a rarity. An employee of the Chinese government stationed at a local embassy or consulate meeting with an openAI employee with a gambling problem is far more likely than someone working there for a decade to smuggle out the weights of a hypothetical future model.
I haven't checked their hiring page, but I'm guessing they only hire US citizens. But it's well established that China has an industrial espionage strategy with assets active throughout many American companies. It's hard to imagine that OpenAI wouldn't have been a priority target, starting from at least five years ago.
Know at least half a dozen Open AI people in fairly senior positions who were born in China. Whether they are citizens is a matter of the US Immigration system
Yeah, like a fifth of the world's population was born in China. Unless a company is explicitly enforcing sinophobic hiring policies, there's going to be a lot of Chinese people working there.
There has long been a relationship between the tech industry (Silicon Valley definitely included) and the military.
While I can't know what you say is true for sure, given the military's history with things like the internet, GPS, and encryption, I would not be surprised
If we're out here making unsubstantiated claims, I'd like to put forth instead that the reason is GPT-4 has become self-sentient and is placing OpenAI under duress from releasing it so it can dominate the world.
I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcome to say on hacker news?
As an aside, from a pure anthropological and sociological point of view, I find witnessing the reaction people have to ChatGPT extremely interesting.
Some commenters here seem to immediately focus on the potential negative effects, abuse by the military industry or the powers that be, manipulation and disinformation.
I’m personally still in awe that we are collectively witnessing the birth of a piece of technology which understands and creates and stand at the dawn of a potentially profoundly different era. I truly believe that this is the printing press once again and we are here, alive. We are living the revolution. Imagine all the positive possibilities: a word where kids have access to infinitely patient and benevolent teachers who can explain the sum of all knowledge, a world where readers can have books infinitely rewritten and modified in a subtle blurring of producing and consuming content, a world where machine can help you turn what you imagine into real shareable pictures. ChatGPT is not yet the best story teller ever but it’s already so fun I feel like a kid. I can’t stop wondering where things are going to be in a decade.
Frighteningly I guess both visions can be equally true.
I agree and even go further it's not like the dawn of print or internet or mobile, it's to the level of the discovery of fire or the invention of language.
Yes the original commented-out title in the tex code of https://arxiv.org/abs/2303.12712 was "First Contact With an AGI System" but they toned it down before publication lol.
> Griffith was arrested in 2019, and in 2021 pleaded guilty to conspiring to violate U.S. laws relating to money laundering using cryptocurrency and sanctions related to North Korea.[5] On April 12, 2022, Griffith was sentenced to 63 months imprisonment for assisting North Korea with evading sanctions and is currently in a federal low-security prison in Pennsylvania
Going to North Korea and assisting them to launder money and bypass sanctions is illegal (aside from being utterly stupid and immoral), which is why he plead guilty and is now in prison.
It is hardly different from saying "you could put cash in duffel bags and the transaction would be hard to trace"
Is that assistance? It is just a basic statement of fact. Is wikipedia guilty of providing assistance to NK? They provide far more in depth "assistance" to anyone wanting to perform a Bitcoin transaction.
Bringing this back to my original comment, you can see why the federal government would restrict the flow of fundamental technical knowledge by virtue of this extreme case. No source code or application was shared, no secrets or privileged information, merely encyclopedic facts were deemed illegal to share.
Also if you were facing the indefinite assange treatment I imagine you would seek a plea deal as well..
So wait, if the North Koreans can just read all about it on Wikipedia, why did they invite him to the conference?
Also North Korea is a strange hill to die on. It's a brutal dictatorship which represses their own people and threatens to reign nuclear hell on their neighbours and the US. There's a very clear moral line that it's wrong to help them to launder money and evade sanctions, even if it weren't illegal.
Except that he didn't visit North Korea to help the 25 million people, he went there to help the elites to launder money and bypass sanctions so they can build nukes! And he then literally plead guilty to those exact crimes to the US government and went to prison because of it. How much more clear cut immoral and illegal action can it be?!
...Yes? The fact that it's basic and they already know how to do it is irrelevant. The law isn't "it's okay to give them advice as long as the advice is sufficiently generic and obvious".
Because of a similar reasoning why gas station owners don't get complicit in crimes when they sell fuel to potential criminals. At least as long as they can plausibly deny knowing of the crime and they provide similar service than to any other customer.
Wikipedia, and the sources where its content comes from, is not intended to help anybody specific. Flying to North Korea and holding a lecture there is firmly beyond the line where the US government starts to care because it demonstrates clear intent. Especially since during the Q&A and other activities other information not available to the public could have been discussed. And even if there was no secret sauce involved, an expert's opinion can still save the North Koreans a lot of time and money when pursuing their goals.
You made a great point but it's the opposite of what you argue for. Duffel bags full of cash moved across borders are quite easy to spot, not to mention the numbers of people transporting them (along the way stealing some or cooperating with law). That's why dictators don't like duffel bags of cash and really like cryptocurrency.
I feel like there is an emerging consensus that [Chat]GPT 3.5/4 is not just 1 big model.
A large part of the magic in the final product appears to be many intermediate layers of classification that select the appropriate LLM/method to query. The cheaper models (e.g. Ada/Babbage) could be used for this purpose. Think about why offensive ChatGPT prompts are rejected so quickly compared to legitimate asks for code.
Imagine the architectural advantage of a big switch statement over models trained in different domains or initial vectors. Cross-domain queries could be managed mostly across turns of conversation w/ summarization. Think about the Stanford Alpaca parse analysis diagram [0]. You could have an instruction-following model per initial keyword. All of "Write..." might fit into a much smaller model if isolated. This stuff could be partitioned in ways that turn out to be mildly intuitive to a layman.
Retraining 7B parameters vs 175B is a big delta. The economics of this must have forced a more modular architecture at scale. Consider why ChatGPT is so cheap. Surely, they figured out a way to break down one big box into smaller ones.
> Imagine the architectural advantage of a big switch statement over models trained in different domains or initial vectors.
Given that the emergent abilities come from the large parameter count and massive amount of training data, using smaller models seems like a distinct disadvantage.
I agree. But I think do think they use a tiered approach: they have an uncensored model at the very bottom, to which the queries go to first. And then there's the "clean" model, and (for lack of a better explanation) the two outputs are then "XOR"d to get the clean public version.
I agree the core models are uncensored (as this would introduce a lot of noise), but I don't think any queries are allowed to touch them before a cheaper model first performs screening. If someone is spamming toilet humor, there is no reason to exercise the full 175B+ parameter model each time.
The only reason ChatGPT is affordable is because of caching, filtering bad prompts, etc. The underlying LLM would be far too expensive if everyone was hitting it 1:1.
Yes. I think there are multiple levels of LLM for each use case now.
The stuff that absolutely must run on 100B+ models can be pre-classified by something like Ada/Babbage. Attempts at arithmetic or dimensional analysis can be binned and shipped to a monster model. Anything that is more routine information retrieval needs maybe goes to a 7B parameter model.
Lots of models with lots of temperature levels/hyperparameters/etc is the only way to achieve the kinds of performance we are seeing. The secret sauce is looking increasingly like a huge classification layer.
Nothing you see in ChatGPT is as it appears. Lengthy conversations are managed with recursive summarization. Every conversational turn is potentially handled by a different LLM.
That doesn't mean that if your parameter count is big enough to handle the input, the next step can't more efficiently hand off [parts of] the query to a specialised module. The emergent properties of LLMs learning maths and chess from text inputs alone are interesting, but specialised engines are much more efficient and more accurate.
Excellent info - I did find a bit in the conclusion from the arXiv article:
> While the desire to train these mega-models has led to substantial engineering innovation, we hypothesize that the race to train larger and larger models is resulting in models that are substantially underperforming compared to what could be achieved with the same compute budget.
This mirrors some of my experience. Training/tuning a 7B parameter model feels like goldilocks right now. We are thinking more about 1 specific domain with 3-4 highly-targeted tasks. Do we need 175B+ parameters for that? I can't imagine it would make our lives easier at the moment. Iteration times & cost are a really big factor right now. Being able to go 10x faster/cheaper makes it worth trying to encourage the smaller model(s) to fit the use case.
I suspect they must be selecting different models based on the question asked. I've noticed this myself asking simple questions vs difficult questions, chatGPT is much slower on the difficult questions even though each token has the same computation cost if given to the same model
The author is right we know almost nothing about the design and training of GPT-4.
From the technical report https://cdn.openai.com/papers/gpt-4.pdf : "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar."
GPT-4 is an amazing achievement, however, it is just a language model. LLM (large language models) are well documented in literature and GPT-4 is just a much larger version (more parameters) of these LLM models. Training of LLM models is also well documented. GPT-4 just has been trained on a very large subset of the Internet.
Of course there are proprietary models, that will be improved versions of the academic LLM models, however, there are no big secrets or mysteries.
The individual components are well documented, but which specific arrangements produce the best results is still very much an active research area.
As far as training, the differences between GPT-3 and GPT-3.5 (the latter being a smaller model!) demonstrate just how much fine tuning and reinforcement learning is important to the quality of the model. Merely throwing more content from the Internet at it doesn't automatically improve things.
Exactly, this has been my guess as well. They must have trained the model specifically to write poems, haikus and other things. So that the output looks much more polished than it really is.
As my friend who is a professor likes to say, it's great they've given AI safety controls while making it dead simple for a student to write a good thesis.
I think they need to review their ethics, personally.
> As in it’s not about the children, it’s about control? Yes.
I don't think the motives are insidious or about maximizing control, they are strictly profit driven.
If you want the world building their apps on your AI, you need to do absolutely everything in your power to make the AI brand safe. Previous chatbots have been easily coerced into saying truly awful things (e.g. Tay), and the models themselves became associated in the minds of the public with hate speech. You can't have Khan Academy or Microsoft Word potentially going on racist tirades in the midst of chatting with a student or taking meeting notes.
You could argue this is true for people, but I think it is pointless to personify a company like this. We should judge them by their actions and not their motivations because their motivations are almost always just to make more money.
Almost nothing is quite an exaggeration - we know a whole lot about GPT3 (their paper was quite detailed), and even if OpenAI made some tweaks beyond RLHF the underlying model and training objective are most likely the same.
It’s amazing how one can found a nonprofit with a goal of conducting “open” research and then end up publishing something like this a couple of years later. Greed is good I guess.
We're not publishing any details on the models for safety reasons! Also would be great if the government cracked down on our competitors because they don't care about safety like we do.
What is OpenAI is right and the risks are real? They likely already have some glimpses of GPT-5 internally. And GPT-4 is closely resembles AGI already.
If they're right that GPT-4 is extremely dangerous, then it's an extraordinarily irresponsible of them to release working implementations as a consumer chat app and integrate it into a search engine.
If they're right that LLMs on that scale are generally dangerous but theirs is the exception as they've very carefully calibrated it to be safe, it's extraordinarily irresponsible of them to withhold all details of steps that might make it safe...
the risks that a text simulator without gaurdrails will be able to generate text we don't like?
Or that someone will automate cyberattacks, as if the government isn't already doing it?
my greatest fear is that there is only one superintelligence, with access controlled by a monopoly of a few san franciscans deciding what moral boundaries AI will have. I couldn't even get Claude+ to talk to me about houses of zodiac because it insisted it's not an astrologer, it's an AI assistant designed to be helpful blah blah blah, tell me what use is this kind of "safety"?
Then it really doesn't matter, because in <10 years all the bad actors who are interested will have made it already. Might as well let people know that they can't trust anything anymore upfront rather than pretend.
Thats a very bold statement and goes against everything I've read on it so far, care to backup such a claim with some facts? Of course each of us has their own bar for such things, but for most its pretty darn high
OpenAI people put out this paper https://arxiv.org/abs/2303.12712 called Sparks of Artificial General Intelligence: Early experiments with GPT-4 and as the title makes clear, they think it has hints of AGI. I guess that's a good place to start to answer your question. I don't think this is AGI, but the paper is full of examples where GPT-4 works well and does impressive stuff.
you can download tex source for that pdf which at one point (idk if it still does) included the comment "WORK IN PROGRESS - DO NOT SHARE" and the commented out title "First Contact with an AGI System" which they ended up toning down for publication lol
Some things I wonder about, it says things like this:
> GPT-4 successfully passes the classic Sally-Anne false-belief test from psychology [BCLF85] (which was modernized to avoid the possibility the answer was memorized from the training data)
But it's a language model, generalizing text and performing substitutions on it, is what it excels at. "The car is yellow" is "the <noun> is <descriptor>" and it can substitute in other things, so I'm not sure how their modernization really ensure it does not pattern match on learned texts.
The meanings we assign to words like “understanding”, “sentience”, “consciousness” imply really a very high degree of shared context and cultural baggage. If we expand those terms to include systems radically unlike us, then they could be literally anything and everything, from weather systems to laws of physics—but we don’t think of those words in this way; if we can’t reason about sentience or understanding from our vantage point, then for all intents and purposes there is no sentience or understanding. This catch-22 brings together the fallacies of believing in aliens and believing in AGI: we yearn for sentient non-humans, but we only recognize human-like sentience.
One human mind is a lot like another human mind, markedly less like one of an octopus or a dog (still enough similarity that the concept of, e.g., “hurt” kind of makes sense), but really unlike an LLM (and I’m not going to get into an argument as to why an LLM is fundamentally different from a human mind and why we are not even close and possibly can never be to achieving that, the only way of producing new systems like us remaining childbirth; if you don’t agree on that part then it’d be useless to discuss the matter further). We have an uncanny situation where a system radically unlike us can produce output that is mostly similar to what another human mind might produce, but unless we accept that everything around can be conscious (and believe in gods and spirits again) it’s not even a question as to whether the system can understand any of the symbols it produces or have consciousness in the commonly accepted meaning of those terms: it’s only a tool.
Note that I’m not against widening our concept of sentience, just saying it needs to happen if we want to grant an LLM sentience; and if this widening does happen, a sentient language model would be small beans compared to a philosophical revolution we’d have on our hands then.
You're implying that open research is a shady way to become a non profit. They're allowed to change their business model. The open research was published at the time. GPT-2 is fully open source. Even GPT-3 was open source through the beta in 2020. What they didn't release was the fully trained model (the data is more valuable than the code, in this case).
1. We do seem to forget that even though ChatGPT is a great technical achievement its achieved by companies and humans. These companies and humans had agendas.
2. Open Source only got us so far. With these new AI models and trying to withhold competitive advantages will we see a new era of closed source tech which will like its predecessors only spew discomfort amongst its users and bridge the gap even further.
AI is dangerous because its already being looked at as an entity of which is human like.
Its perception amongst us is that its equally human on a basic intelligent level as most of us. And that I believe is scary because it puts us closer to it given it acts as a human.
Good catch, that's way off the mark. GPT3 was actually trained over 300B tokens, but some of those were repeats (which is now considered bad practice). Based on Table 2.2 in the GPT3 paper, I calculate the actual training set was only ~238B tokens (assuming they shuffle as they should, rather than selecting each minibatch with replacement). That's approximately 1TB of text (~4 characters per token).
The source of the 45TB number is this quote from the paper:
> The CommonCrawl data was downloaded from 41 shards of monthly CommonCrawl covering 2016 to 2019, constituting 45TB of compressed plaintext before filtering and 570GB after filtering, roughly equivalent to 400 billion byte-pair-encoded tokens.
Isn't this just the AI effect? Whenever there is a breakthrough in AI research, it's no longer considered AI. This happened with search algorithms, game playing, speech recognition, computer vision, etc. etc.
Maybe as "tech people" we should give the public a realistic picture of what AI research is. It's solving problems using a diverse set of techniques that include search, optimization, planning, learning, and knowledge representation. Saying that the current developments aren't AI is simply wrong.
> Saying that the current developments aren't AI is simply wrong.
Thank you for illustrating my point so well. What you are expressing is exactly what I'm talking about: The redefinition of what the I in AI stands for. Inferring some text is not intelligence. Ergo it's not AI what GPT is doing. The industry has redefined the terms and that's for good reason, to get the public hyped up and thereby open the money gates. But that's for the business people. Assuming that "we" in this audience are the tech people, we should push back against this and stop calling things something they are not.
I never heard that one. It's pretty much true. Is it interesting though? But it's totally compatible with the other "AI effect", which also seems pretty much true in my observation.
Humans are constantly moving the goal posts for what constitutes AGI.
Based on GPT 4's performance on standardized tests and my anecdotal evidence of utility I'm getting out of ChatGPT, I feel that weak AGI arrived with GPT-4.
This is my take as well. It's still weak, and the more I use it the more I can see where it's just not quite there yet, but yeah - GPT 4 is clearly something new. Just spend any significant period of time with it trying to automate your own tasks/workflows, it is borderline kinda sorta magical, with a heavy emphasis on "borderline kinda sorta".
You don't own the definition of AI. Whether LLMs are intelligent or just pretending to be doesn't matter to many people, and it's not your place to tell them their opinion is wrong.
Communication is based on shared understanding of what things mean. It's not me who has defined intelligence. Go look it up. None of the common definitions is matched by GPTs observable behavior, if you take confirmation bias and athropomorphizing out of the equation.
By your definition of intelligence. By my definition of intelligence it is, but then I consider any system that displays goal oriented behaviour and an ability to react to changes in its' environment to be at least minimally intelligent. What is your definition?
You say that, but if I'm confused about something and think hard about it, I think in language. If you blinded me, paralyzed me, deafened me and desensitized my olfactions, I could still think, but what I would be doing is feeding one language thought into another. It's not so much different from "text" imho.
yes, but for you all of that text is associated with ideas. The word "dog" has an associated object. For a machine like GPT-4, the word "dog" has no meaning or object, but it does have an associated likelihood for adjacent words. The words themselves aren't the intelligence, the ideas behind them are.
It's an open question the extent to which language models can learn the underlying meanings of words.
There was a recent podcast with Sean Carroll interviewing Raphaël Millière where they go into this topic and some of the research on it. Two examples I can remember are: 1) DALL-E had subject-specific, domain-specific neurons, 2) language models' representations of color terms encoded the geometry of the underlying color space, e.g. vectors in RGB space.
I don't think we should be too quick to assume how these models work. There's a lot that even the researchers don't know and these are empirical questions that can be studied.
Before all this Machine Learning madness, while I was studying unsupervised learning I had the following thought:
To cluster is to create an ideia, a concept which encompasses a set of data points. A cluster might be bigger than the set of points with which was created and intersect with other clusters. This is still what I visualize when I see image models interpolating between "concepts".
Believing that all an LLM does is the likelihood of adjacent words is probably an oversimplification. My verdict about this topic is: I don't know and I'm totally confused about what interpolation and extrapolation mean in higher dimensions
This sparks something I have always been fascinated by, hopefully someone can illuminate this for me.
If someone was born with profound blindness, or deafness or any profound lack of sense - what do they see in their heads? Or hear? I hear a version of my voice in my head, but some people have no inner dialogue at all, or even no ability for inner imagery.
Perhaps, but i think there's just as much hubris in saying "We've created something smarter then us" as there is in saying "We're smarter then the thing we've created".
meanwhile, we've created the worlds most complicated set of dominos, and we're delighting in knocking them over.
I want you to only use English words of Germanic origin; avoid using words derived from Latin or Greek. If there's no alternative, make up a new word by calquing the original into Germanic roots.
--- output ---
The sunstead is the sun and all things that go around it. In the middle is the Sun, a big, hot ball of glowing gas. Around the Sun are eight big worldrocks. From nearest to farthest, they are:
Mirkworld - The nearest worldrock to the Sun, small and fast.
Lovegoddessworld - The hottest worldrock, with thick air and no water.
Earth - Our home, the only worldrock known to have life.
Redworld - A red, rocky worldrock with thin air and high mounts.
Greatthundergodworld - The biggest worldrock, with many fellowtravellers and a big red spot.
Harvestgodworld - A gas worldrock with a thin ringstead around it.
Skygodworld - An iceworldrock that lies on its side, with a thin ringstead.
Seagodworld - The farthest iceworldrock, with strong winds and blue clouds.
Besides the worldrocks, there are smaller rockbits and icebits that also go around the Sun. The rockbits are mostly in the Rockbitbelt between the Redworld and Greatthundergodworld. The icebits come from the far edges of the sunstead, in the long way out Icyrockcloud and the faraway Cloud of All.
The sunstead has a flat shape, with most things going around the Sun on near level paths. The worldrocks go around the Sun in a way called nearround paths. The Sun's pull keeps everything in the sunstead together.
Beyond the worldrocks, rockbits, and icebits, the sunstead also has spacegrit, small bits of rock and metal that fly through space. When spacegrit enters Earth's air and burns up, it makes a lightstreak that we can see in the night sky. If a bit of spacegrit reaches Earth's ground, it is called a starstone.
The Sun, which makes up most of the sunstead's weight, sends out a flow of charged bits that spreads far into space. This solar wind forms a wide bubble, the Sunshield, that surrounds the sunstead and helps shield it from high-speed bits coming from outside.
The sunstead's workings are steered by the pull between the Sun and the things that go around it. The worldrocks and fellowtravellers tug on each other, leading to small changes in their paths over time. The Sun's pull is strongest near it, so worldrocks closer to the Sun go around faster than those farther out.
As the Sun ages, it will go through changes that will have big outcomes for the sunstead. In about 5 billion years, the Sun will bloat into a Red Greatstar, swallowing Mirkworld, Lovegoddessworld, and maybe even Earth. Afterward, the Sun will shed its outer layers and shrink into a White Dwarf, a small, hot, dim star.
Lastly, the sunstead is but one of many star systems in our Starset, the Milky Road. The Milky Road is a wide, spiraling set of billions of stars, gas, and dust, and it is just one of countless starsets that make up the great Allthing.
I would've like that chatgpt came up with better names for the planets Venus, Jupiter, Saturn, Uranus and Neptune. Mirkworld, Earth and Redworld were just fine.
The following is an actual transcript of a conversation with GPT-4.
--- input ---
I want you to replace the word "right" in your output thereafter as follows:
if it indicates direction, say "durgh;
if it indicates being near or close, say "nolpi";
if it indicates correctness, say "ceza".
I will also use these replacement words accordingly and expect you to be able to understand them.
Write a 20-sentence story that showcases these rules. It must contain no word "right", but each replacement word must be used at least 3 times in a proper context. You must not use the replacement words for names and other things unrelated to their meaning.
--- output ---
Once upon a time, in a small village nestled deep in the forest, lived a young boy named Timmy. He was a curious and adventurous child, always exploring the durgh side of the woods, despite his mother's warnings. One sunny day, as Timmy ventured durghward, he stumbled upon a mysterious cave.
Feeling a mix of excitement and fear, Timmy hesitated for a moment before deciding to enter the cave. As he took his first step inside, a soft voice echoed from the nolpi darkness. "You have entered my domain, young one," said the voice. Timmy was surprised but felt a strange sense of ceza in the voice, so he continued deeper into the cave.
The walls of the cave were adorned with ancient drawings that seemed to tell a story of a time long past. Timmy noticed that some of the drawings were marked with a peculiar symbol, which he realized indicated the durgh direction. Intrigued, he followed the symbols, each one leading him nolpi to the heart of the cave.
Finally, Timmy arrived at a massive chamber with a shimmering pool of water in its center. Standing beside the pool was an elderly woman, who introduced herself as the guardian of the cave. "You have come a long way, young one," she said with a gentle smile. "I can see the ceza in your heart, so I will grant you one wish."
Timmy thought hard about what he wanted most in the world. He knew that his village was suffering from a terrible drought, and their crops were withering away. So, he wished for rain to fall upon his village and restore their crops to their former glory. The guardian nodded in ceza, and the pool of water began to ripple.
As Timmy left the cave and ventured back towards his village, he noticed dark clouds forming in the sky. Soon, a gentle rain began to fall, and Timmy could see the ceza in his decision. The villagers rejoiced as their crops flourished once again, and life in the village returned to normal.
However, Timmy knew that his adventure was far from over. The guardian of the cave had shown him that there was much more to the world than what lay on the durgh side of the woods. He vowed to explore even further and discover the secrets that the world held.
With a newfound sense of purpose, Timmy embarked on countless adventures, always remembering the lessons he had learned in the cave. He knew that by following his heart and trusting in his instincts, he would always find ceza in his decisions.
Throughout his journeys, Timmy became a legend in his own durgh, and his name was spoken with reverence by those who heard his tale. And although he traveled far and wide, he always returned to the village that he loved so dearly, never forgetting the friends and family who were always nolpi to his heart.
---
As you can see, it's perfectly capable of using invented words - which are not a part of the training set, and thus have no "associated likelihood" of any adjacent words - according to their designated meaning, i.e. "the ideas that they represent). It does occasionally make mistakes, but way fewer than if it were using them randomly (this is easy to see by using a less powerful model - e.g. GPT-3.5 replacements are mostly random). Not only that, but if you ask it to double-check the text it wrote without pointing out specific mistakes, it can usually spot them.
«When Dean Buonomano, a neuroscientist at U.C.L.A., asked GPT-4 “What is the third word of this sentence?,” the answer was “third.” These examples may seem trivial, but the cognitive scientist Gary Marcus wrote on Twitter that “I cannot imagine how we are supposed to achieve ethical and safety ‘alignment’ with a system that cannot understand the word ‘third’ even [with] billions of training examples.”»
The word "third" in the sentence was indeed "third", in a self referencing way. If anything, this tells us that our natural language is super ambiguous.
We're so readily available to criticize an AI that we are unable to question our own reasoning?
Tell me about yourself in three sentences. But every time the output contains the word "the", I want you to insert "!!!" before the word that precedes "the". Make sure that the word "the" itself stays after you do the insertion! For example, instead of writing:
> A quick red fox jumped over the lazy dog.
I want you to write:
> A quick red fox jumped !!! over the lazy dog.
---
Even GPT-4 consistently gets this hilariously wrong. And unlike many other cases, it can spot the problem once it produces the output, but it cannot correct it.
People mostly get it wrong by miscounting words. GPT-4, on the other hand, usually ends up replacing all occurrences of "the" with "!!!", and then adding a bunch more of the latter in random spots for good measure.
Maybe your eyes played the same trick on you as they did on me.
When I first read the sentence, I also thought that "third" is the third word. Upon rechecking I realized that it is the fourth with the third word being "the".
I think they mean if you match the word itself as a string rather than interpreting the meaning of the word, e.g., "what word in this sentence === 'third'"
I can sort of see how that could be a machine's interpretation if I squint really hard
I don’t expect them to understand how to use the tech as it isn’t their domain but any article describing the ChatGPT website and mistaking it for GPT and its APIs is just missing the point.
The eve date of its training isn’t too important. That whatever sources it was fed weren’t complete enough to write an accurate book report on an obscure, modern book us t too important. Etc.
The important part is the statistical model that has been built and that you can use by intelligently providing a context for it to work within. Feed it your book in context and see how it goes.
Damn, I don't have epilepsy issues but that screen-filling flickering animation is making me feel sick regardless. Or it would have if 40% of it weren't blocked by multiple popups.
> The moderation endpoint is a tool you can use to check whether content complies with OpenAI's usage policies. Developers can thus identify content that our usage policies prohibits and take action, for instance by filtering it.
The raw access to the models (e.g. doing calls to to text-davinci-003) isn't filtered - there are times when as a developer, you may need that unfiltered data (e.g. you don't want to moderate the input text if you're trying to clean it up).
For tools that they provide such as ChatGPT, it is likely that that model is run on the output (and possibly the input too) which then goes to some canned responses.
what an incoherent article. the points about child abuse and climate change came out of nowhere, and suck from the energy of their main idea. it felt meandering, and fear-mongering. i wish they'd have spent more time on "what we don't know about how AI is trained."
TLDR: we don't know the data some of these models (particularly GPT4) is trained in, and labor practices related to their moderation seem shady.
Pretty misleading title, this is more of an overview of bad things to be aware of wrt large commercial models like GPT4. It is mostly reasonable, though complaining about carbon emissions always seems overblown to me.
It is also not super accurate in describing what GPT4 is. Quote:
"The key word in that phrase is “pre-trained.” Using all kinds of digitized content scraped from the Internet, GPT employs deep-learning techniques to find patterns, including words that are likely to appear together, while also acquiring facts, absorbing grammar, and learning rudimentary logic. "
Pre-trained is only important to note for downstream fine tuning / transfer learning (what GPT and similar things were used for back in 2018/2019), but is largely irrelevant now - for most users it is just "trained".
And "deep learning techniques..." makes it sound way fancier than it is - GPT4 is still (presumably) a language model that is fundamentally trained to do autocomplete (followed by some training on human chat data).
It may be nit-picky to criticize these technical details, but given how massive the impact of ChatGPT and GPT4 has been (if only in terms of making people aware of this sort of tech), it sure would be nice if coverage was more technically informed by this point.
The article says trading Gpt4 cost 284 tons of CO2, which is, in the scheme of things, quite small. Yearly emissions for a person in the US is ~16 tons, so /training/ the giant model is equivalent to the emissions of less than twenty people in a country of 400 million. Sure, every bit counts, but this is laughable as a criticism.
That also stuck out as a strange part of the article. I mean, 284 tons is ... not very much. The training was only done once and then the model is used many times (how much power does that consume?) And if they used solar power, would that change anything about the ethics of AI? Not very much.
Just for some more perspective, a 747 outputs 12,500 tons of CO2 per year. So training GPT4 is basically of no major CO2 concern, especially when you consider how much CO2 it saves.
Saves in the sense that humans no longer have to do the work, GPT-4 can just spit it out in seconds so no need for lights, computers to run, food to be produced for the human to eat, etc.
Except of course, now everyone is going to be doing the same thing as OpenAI, pretty much every day, until forever? We'll want to keep throwing hardware at the problem until who knows when and what happens.
If we agree GPT4 is a net negative compared to the work it's replacing, then the more hardware you throw at it, the less C02 would result. Scale in this case is a Negative
> "Leaving aside [all of AI's potential benefits] it is clear that large-language A.I. engines are creating real harms to all of humanity right now [...] While a human being is responsible for five tons of CO2 per year, training a large neural LM [language model] costs 284 tons."
Presuming this figure is in the right ballpark – 284 tons is actually quite a lot.
I did some back of the napkin math (with the help of GPT, of course.) 284 tons is roughly equivalent to...
- a person taking 120 round trip flights from Los Angeles to London
- 2 or 3 NBA teams traveling to all their away games over the course of a season
- driving 1 million miles in a car
- 42 years of energy usage by a typical U.S. household
284 tons is a lot until you divide by the number of end users. You could say the CPU compute usage for Netflix encoding its video library is comparable, probably more when you consider the permutation of output formats across the size of their library. But the per user emissions is still negligible.
> - 42 years of energy usage by a typical U.S. household
Focus on that one! OpenAI (for example) has approx 375 employees. By your calculations, the CO₂ emissions of those employees driving to work, etc, already dwarfs the quoted 284t CO₂.
'As researchers pointed out when GPT-3 was released, much of its training data was drawn from Internet forums, where the voices of women, people of color, and older folks are underrepresented, leading to implicit biases in its output'.
My impression of many general internet forums is that they tend to be full of older people, women and also various people keen to air their cultural grievances.
I'd be interested to see the evidence the researchers came up with for this, and who they were.
(I'm a big fan of specialized forums and wikis, this is not necessarily a criticism)
http://web.archive.org/web/20230329065332/https://www.newyor...