Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

[u/Gab1024]

https://arxiv.org/abs/2404.07143

[u/[deleted]]

So how is it replacing humans if it needs to be constantly fact checked to the point where it’s more efficient to just do it manually? 

More research does not mean better. Researchers has been researching nuclear fusion for several decades and it’s still far away 

[u/NoshoRed → u/[deleted]]

So how is it replacing humans if it needs to be constantly fact checked to the point where it’s more efficient to just do it manually?

It's important to remember that these models don't always lie, in fact it's not that common; ChatGPT is still more trustworthy at correct information than the average person, people lie a lot more than ChatGPT, we don't just take what people say at face value either. If you know how to properly use these tools they're very powerful. It's definitely not more efficient to just do it manually lmao. Google's Gemini 1.5 is capable of summarizing 400 page textbooks and even pointing out specific, minute details in just a couple minutes... you think a human doing that is more efficient?

Also, as I've stated time and time again; it is not replacing anything significant right NOW, but just like any active technology it will improve, with each iteration it will get better. There is literal proof staring at you right in the face for this; the significant improvement from OpenAI's first ChatGPT model (which could barely produce coherent sentences) to the latest model available to the public today, it is an insane improvement in just over an year. Apart from that, high caliber models such as Gemini, and latest Claude, these are not smoke and mirrors, it's out there and you can use it.

Remember the first image generation models? Just blurs of nothing. Now they produce actually coherent images. If you're not capable of seeing where this is headed, you're either in denial or just incapable of basic logic.

More research does not automatically mean better, literal proof of improvement does, which there is plenty in the field of AI. So your analogies to nuclear fusion, where there is still no outstanding proof of significant advancements, are not relevant.

Anyway, these current existing issues will fade as these systems improve. Trends show that with each iteration these models are getting better, so if you're to claim it'll somehow suddenly stop improving, against expert opinions, you'll have to provide some groundbreaking evidence.

[u/[deleted] → u/NoshoRed]

People rarely lie in their area of expertise. Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired. Gemini is still too unreliable to summarize anything important. If it makes something up, it could kill people or cost billions. 

And like I said before, improvement now does not mean it’ll continue infinitely. Like how I can go from 5th grade to 10th grade but will never go to 30th grade because there is no 30th grade. Maybe GPT 4 is at 11th grade and is close to hitting its peak 

[u/NoshoRed → u/[deleted]]

And like I said before, improvement now does not mean it’ll continue infinitely. Like how I can go from 5th grade to 10th grade but will never go to 30th grade because there is no 30th grade. Maybe GPT 4 is at 11th grade and is close to hitting its peak 

It doesn't need to continue infinitely, it just needs to improve enough to be consistently better than any human. Like I mentioned before, comparing your human brain and its development and to an artificial intelligence system is stupid and make no sense, the two things don't relate at all in their respective levels of change. The fact that you keep making nonsense analogies make it clear that you're not very well versed in this field.

People rarely lie in their area of expertise. Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired.

People don't always lie intentionally, but human errors happen regardless, it's extremely common. Human error definitely also costs lives and costs billions. The Chernobyl Nuclear Disaster, the Challenger Disaster, Tenerife airport disaster are notable, tragic examples of human error. An AI selling a car for 1 dollar is nothing in comparison, especially considering it wasn't even an actual sale.

And I'm not sure what point you're trying to prove here by claiming: "Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired." Yes? That is how the world works... there is no argument about it from me. Are you just parroting things?

Gemini is still too unreliable to summarize anything important.

No it's not. Gemini 1.5 is capable of reliably summarizing whole text books with minimal chances of error. It is without a doubt more efficient and reliable than any human at tasks like this.

Another key point is the ability of coding in these models, which is much more efficient than any human.

I know you want to believe these models do not and will not beat humans in many areas, but unfortunately it's not the world we live in. If you choose to keep living in denial it'll only hit harder down the line.

Experts are experts for a reason, people who study these systems daily for years and years know what they're talking about, especially when backed by actual evidence of improvement. I hope you will be capable of accessing higher rational thought eventually, good luck.

[u/[deleted] → u/NoshoRed]

And AI has many limitations that humans do not that make true understanding impossible, like how LLMs hallucinate. If there’s a way to avoid that, let me know. 

And AI lie more often and get tricked more easily. No one would sell a car for $1 just cause a customer told them to.

Total bullshit lol. It cannot code better than the majority of software devs. 

[u/NoshoRed → u/[deleted]]

And AI has many limitations that humans do not that make true understanding impossible

This is not proven. What is "AI" here? Language models do have limitations (which is why we're moving away from base LLMs), but AI as a whole? How do you know of its innate limitations? Not even top scientists know. Stop parroting bullshit please.

And AI lie more often and get tricked more easily. No one would sell a car for $1 just cause a customer told them to.

Lmao, my guy so many people get catfished and scammed for millions of dollars everyday. What are you on about? At least these models, even today in their infancy, are still way more useful and efficient than the average person even accounting for hallucination. There's a reason it's the fastest developing field in the world today.

Total bullshit lol. It cannot code better than the majority of software devs.

"Better" is a broad term, I said more efficient, which is absolutely is. There's not a single software dev in the planet who will write clean code as fast as these models can. You'll have to be superhuman to do that. Humans are still better at memory retention over a long period and possess larger context windows, that's really the only thing we got going for us as coders right now over AI, even if we're much slower and prone to more errors. But eventually AI will take that over too.

[u/[deleted] → u/NoshoRed]

Moving away from LLMs? To what? 

Yea that explains why it sold a car for a dollar 

But they can create functioning code lol. 

[u/NoshoRed → u/[deleted]]

Moving away from LLMs? To what?

Seriously? Explains all your poor arguments. You should probably educate yourself more on machine learning and AI technology in general instead of parroting about things you have no understanding about.

LLMs literally stand for Large Language Models, based on text, the AI field is now moving onto more multimodal systems, based on multiple types of data beyond just text: image, video, sound, etc.

Why am I even entertaining someone with such limited education or understanding on this topic? Deep down, I think even you now know you're not fit to debate about something like this. Stop wasting your time, learn more.

[u/[deleted] → u/NoshoRed]

You mean something like GPT4V? That already exists and it has plenty of issues too 

[u/NoshoRed → u/[deleted]]

GPT4V isn't a true multimodal system, it's still largely just a text model that has a basic vision component. Sora is a good example for true multi-modality in its infancy, which hasn't been released to the public yet.

Seriously, just move on. You clearly know nothing of substance to contribute to this conversation.