I don’t believe that. At least from an implementation perspective we’re extremely early on, and I don’t see why the tech itself can’t be improved either.
Maybe it’s current iteration has hit a wall, but I don’t think anyone can really say what the future holds for it.
LLMs have been around since roughly 2016 2017 (comment below corrected me that Attention paper was 2017). While scaling the up has improved their performance/capabilities, there are fundamental limitations on the actual approach. Behind the scenes, LLMs (even multimodal ones like gpt4) are trying to predict what is most expected, while that can be powerful it means they can never innovate or be truth systems.
For years we used things like tf-idf to vectorize words, then embeddings, now transformers (supped up embeddings). Each approach has it limits, LLMs are no different. The results we see now are surprisingly good, but don’t overcome the baseline limitations in the underlying model.
The “Attention Is All You Need” paper that birthed modern AI came out in 2017. Before Transformers, “LLMs” were pretty much just Markov chains and statistical language models.
Transformers, the foundation of modern “AI”, was proposed in 2017. Whatever we called “AI” and “Machine Learning” before that was mostly convolutional networks inspired by the 80’s “Neocognitron”, which is nowhere near as impressive.
The most advanced thing a Convolutional network ever accomplished was DeepDream, and visual Generative AI has skyrocketed in the 10 years since then. Anyone looking at this situation who believes that we have hit bedrock is delusional.
From DeepDream to Midjourney in 10 years is incredible. The next 10 years are going to be very weird.
It’s this. When boards and non-tech savvy managers start making decisions based on a slick slide deck and a few visuals, enough will bite that people will be laid off. It’s already happening.
There may be a reckoning after, but wall street likes it when you cut too deep and then bounce back to the “right” (lower) headcount. Even if you’ve broken the company and they just don’t see the glide path.
It’s gonna happen. I hope it’s rare. I’d argue it’s already happening, but I doubt enough people see it underpinning recent lay offs (yet).
I can see the statement in the same way word processing displaced secretaries.
There used to be two tiers in business. Those who wrote ideas/solutions and those who typed out those ideas into documents to be photocopied and faxed. Now the people who work on problems type their own words and email/slack/teams the information.
In the same way there are programmers who design and solve the problems, and then the coders who take those outlines and make it actually compile.
LLM will disrupt the programmers leaving the problem solvers.
There are still secretaries today. But there aren’t vast secretary pools in every business like 50 years ago.
The only people who would say this are people that don’t know programming.
LLMs are not going to replace software devs.
AI as a general concept probably will at some point. But LLMs have all but reached the end of the line and they’re not nearly smart enough.
LLMs have already reached the end of the line 🤔
I don’t believe that. At least from an implementation perspective we’re extremely early on, and I don’t see why the tech itself can’t be improved either.
Maybe it’s current iteration has hit a wall, but I don’t think anyone can really say what the future holds for it.
LLMs have been around since roughly
20162017 (comment below corrected me that Attention paper was 2017). While scaling the up has improved their performance/capabilities, there are fundamental limitations on the actual approach. Behind the scenes, LLMs (even multimodal ones like gpt4) are trying to predict what is most expected, while that can be powerful it means they can never innovate or be truth systems.For years we used things like tf-idf to vectorize words, then embeddings, now transformers (supped up embeddings). Each approach has it limits, LLMs are no different. The results we see now are surprisingly good, but don’t overcome the baseline limitations in the underlying model.
The “Attention Is All You Need” paper that birthed modern AI came out in 2017. Before Transformers, “LLMs” were pretty much just Markov chains and statistical language models.
Oh really! The analysis has been established since the 80’s. Its so far from early on that statement is comical
Transformers, the foundation of modern “AI”, was proposed in 2017. Whatever we called “AI” and “Machine Learning” before that was mostly convolutional networks inspired by the 80’s “Neocognitron”, which is nowhere near as impressive.
The most advanced thing a Convolutional network ever accomplished was DeepDream, and visual Generative AI has skyrocketed in the 10 years since then. Anyone looking at this situation who believes that we have hit bedrock is delusional.
From DeepDream to Midjourney in 10 years is incredible. The next 10 years are going to be very weird.
The next 10 years
Wrong, this is also exactly what people selling LLMs to people who can’t code would say.
It’s this. When boards and non-tech savvy managers start making decisions based on a slick slide deck and a few visuals, enough will bite that people will be laid off. It’s already happening.
There may be a reckoning after, but wall street likes it when you cut too deep and then bounce back to the “right” (lower) headcount. Even if you’ve broken the company and they just don’t see the glide path.
It’s gonna happen. I hope it’s rare. I’d argue it’s already happening, but I doubt enough people see it underpinning recent lay offs (yet).
I don’t know if you noticed but most of the people making decisions in the industry aren’t programmers, they’re MBAs.
I can see the statement in the same way word processing displaced secretaries.
There used to be two tiers in business. Those who wrote ideas/solutions and those who typed out those ideas into documents to be photocopied and faxed. Now the people who work on problems type their own words and email/slack/teams the information.
In the same way there are programmers who design and solve the problems, and then the coders who take those outlines and make it actually compile.
LLM will disrupt the programmers leaving the problem solvers.
There are still secretaries today. But there aren’t vast secretary pools in every business like 50 years ago.
It’ll have to improve a magnitude for that effect. Right now it’s basically an improved stack overflow.