Users of OpenAI’s GPT-4 are complaining that the AI model is performing worse lately. Industry insiders say a redesign of GPT-4 could be to blame.
You know how we have pre-bomb steel? We’ll have pre-GPT data sets.
Not the first time OpenAI has done this. DALLE2 used to be the best AI art program in the world. Then OpenAI decided that they didn’t want to get sued by celebrities, so they made it so that if a face came out that resembled a celebrity, it would be distorted. But every face kind of looks like someone famous. Ta da! Now DALLE2 can’t do faces.
Want a crane shot areal image of a teen couple in a corvette driving off into the sunset? Well, you are now banned for life from the DALLE2 service, because DALLE2 produced an image of a ‘shot teen’ and that violates it’s terms of service.
deleted by creator
This isn’t sustainable. They’re banking that nobody else is going to be able to achieve GPT-4-like quality, and what with us basically being at near the bottom of the vertical bit of the growth curve, I’d say that’s a little like betting that nobody’s going to be able to build a car that beats the Model T’s performance. Meta is trying to tackle very large language models in the same way that they got React to be so good and widely supported: by taking it open source. Google, on the other hand, is currently working on having LLMs running natively on phones and tablets. That’s not to speak of the fully open source models. Yeah, running a 1.6 trillion parameter GPT-based LLM is fucking expensive and difficult to replicate, but there are newer, more efficient techniques popping up around LLMs at a dizzying pace. It’s only a matter of time before someone comes up with something that’s at least as good as GPT 4.
Just like most people after they achieve success.
The model has become inbred because it’s now impossible to scrape the web without AI content getting ingested, which is full of “hallucinations” and other weird artifacts. The last opportunity to get “uncontaminated” training data was sometime in mid 2022.
Not to say that it’s causing this particular problem, but this issue will emerge eventually. Garbage in = garbage out. Eventually GPT-19 will grow a mighty Habsburg chin.
Maybe not yet, but…
- Spez will turn Reddit into a bot farm and sell this as training data
- Musk turns Twitter into a bigoted cesspool and will sell this as training data, which will subsequently be flagged for low quality (also: a botfarm)
- Threads is a corporate ad dashboard (and we already know how easy it is to GPT copy) and Zuck will sell this as training data
- Facebook is either dead or only good for boomers and Poles
- blogs are dead
- Fediverse is out there waiting to be scraped but possibly too small to sustain a big model
We’te getting there, hopefully.
…is Facebook popular with Polish people? Or was this a weird polish joke I don’t get?
Very. Twitter never took off among general population (only politicians, journalists, botfarms and people who troll politicians and journalists), tiktok is for kids, Instagram is popular but again, rather among influencers and people who need to show off pictures not as a default SM app. I don’t really know where did Americans and west Europeans move from Facebook.
Scrapped?.. Or scraped?
I suspect future models are going to have to put some more focus on learning using techniques more like what humans use, and on cognition.
Like, compared to a human these language models need very large quantities of text input. When humans are first learning language they get lots of visual input along with language input, and can test their understanding with trial-and-error feedback from other intelligent actors. I wonder if perhaps those factors greatly increase the rate at which understanding develops.
Also, humans tend to cogitate on inputs while ingesting them during learning. So if the information in new inputs disagrees with current understanding, those inputs are less likely to affect current understanding (there’s a whole ‘how to change your mind’ thing here that is necessary for people to use, but if we’re training a model on curated data that’s probably less important for early model training).
I don’t know details of how model training works, but it would be interesting to know if anyone is using a progressive learning technique where the model that is being trained is used to judge new training data before it is used as a training input to update the model’s weights. That would be kind of like how children learn by starting with very simple words and syntax and building up conceptual understanding gradually. I’d assume so, since it’s an obvious idea, but I haven’t heard about it.
Some people have been saying that since the beginning while some haven’t noticed this “decline”. It seems very subjective.
I don’t use it much but I’ve been facing more “I can’t” scenarios with OpenAI lately, could be unrelated though.
Beside upscale ai all other ai can collapse, burn and fuck right off
I think I’d place breakthrough medical applications above upscaling but that’s just me.
Why?
lmao I was write back then :D
You mean "I was right* or “i wrote*”?
No no, he used to work as a wright. Built ships and shit.
Also the title of the attached post is lols: “Yes, ChatGPT became dumber and I 2 days ago I cancelled my subscription”
