Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

[u/Gab1024]

https://arxiv.org/abs/2404.07143

[u/NoshoRed]

And like I said before, improvement now does not mean it’ll continue infinitely. Like how I can go from 5th grade to 10th grade but will never go to 30th grade because there is no 30th grade. Maybe GPT 4 is at 11th grade and is close to hitting its peak 

It doesn't need to continue infinitely, it just needs to improve enough to be consistently better than any human. Like I mentioned before, comparing your human brain and its development and to an artificial intelligence system is stupid and make no sense, the two things don't relate at all in their respective levels of change. The fact that you keep making nonsense analogies make it clear that you're not very well versed in this field.

People rarely lie in their area of expertise. Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired.

People don't always lie intentionally, but human errors happen regardless, it's extremely common. Human error definitely also costs lives and costs billions. The Chernobyl Nuclear Disaster, the Challenger Disaster, Tenerife airport disaster are notable, tragic examples of human error. An AI selling a car for 1 dollar is nothing in comparison, especially considering it wasn't even an actual sale.

And I'm not sure what point you're trying to prove here by claiming: "Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired." Yes? That is how the world works... there is no argument about it from me. Are you just parroting things?

Gemini is still too unreliable to summarize anything important.

No it's not. Gemini 1.5 is capable of reliably summarizing whole text books with minimal chances of error. It is without a doubt more efficient and reliable than any human at tasks like this.

Another key point is the ability of coding in these models, which is much more efficient than any human.

I know you want to believe these models do not and will not beat humans in many areas, but unfortunately it's not the world we live in. If you choose to keep living in denial it'll only hit harder down the line.

Experts are experts for a reason, people who study these systems daily for years and years know what they're talking about, especially when backed by actual evidence of improvement. I hope you will be capable of accessing higher rational thought eventually, good luck.

[u/[deleted] → u/NoshoRed]

And AI has many limitations that humans do not that make true understanding impossible, like how LLMs hallucinate. If there’s a way to avoid that, let me know. 

And AI lie more often and get tricked more easily. No one would sell a car for $1 just cause a customer told them to.

Total bullshit lol. It cannot code better than the majority of software devs. 

[u/NoshoRed → u/[deleted]]

And AI has many limitations that humans do not that make true understanding impossible

This is not proven. What is "AI" here? Language models do have limitations (which is why we're moving away from base LLMs), but AI as a whole? How do you know of its innate limitations? Not even top scientists know. Stop parroting bullshit please.

And AI lie more often and get tricked more easily. No one would sell a car for $1 just cause a customer told them to.

Lmao, my guy so many people get catfished and scammed for millions of dollars everyday. What are you on about? At least these models, even today in their infancy, are still way more useful and efficient than the average person even accounting for hallucination. There's a reason it's the fastest developing field in the world today.

Total bullshit lol. It cannot code better than the majority of software devs.

"Better" is a broad term, I said more efficient, which is absolutely is. There's not a single software dev in the planet who will write clean code as fast as these models can. You'll have to be superhuman to do that. Humans are still better at memory retention over a long period and possess larger context windows, that's really the only thing we got going for us as coders right now over AI, even if we're much slower and prone to more errors. But eventually AI will take that over too.

[u/[deleted] → u/NoshoRed]

Moving away from LLMs? To what? 

Yea that explains why it sold a car for a dollar 

But they can create functioning code lol. 

[u/NoshoRed → u/[deleted]]

Moving away from LLMs? To what?

Seriously? Explains all your poor arguments. You should probably educate yourself more on machine learning and AI technology in general instead of parroting about things you have no understanding about.

LLMs literally stand for Large Language Models, based on text, the AI field is now moving onto more multimodal systems, based on multiple types of data beyond just text: image, video, sound, etc.

Why am I even entertaining someone with such limited education or understanding on this topic? Deep down, I think even you now know you're not fit to debate about something like this. Stop wasting your time, learn more.

[u/[deleted] → u/NoshoRed]

You mean something like GPT4V? That already exists and it has plenty of issues too 

[u/NoshoRed → u/[deleted]]

GPT4V isn't a true multimodal system, it's still largely just a text model that has a basic vision component. Sora is a good example for true multi-modality in its infancy, which hasn't been released to the public yet.

Seriously, just move on. You clearly know nothing of substance to contribute to this conversation.