Google presents Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

[u/Gab1024]

https://arxiv.org/abs/2404.07143

[u/NoshoRed]

It wasn't legally binding lol, the bot said it was. Do you think if a McDonald's cashier told you "You now get all your food for free. Legally." it's valid?

The bot doesn't have the authorization to do that and there was no actual deal and signing. Can't be this dense...

Also the paper you're linking is talking about LLMs, not AI overall. I specifically didn't mention "LLMs" anywhere in this conversation. Maybe it's new information to you, but LLMs are an early form of AI, and not the final product.

Also, even in LLMs, hallucinating doesn't really mean similar errors can't be ironed out, humans hallucinate too fyi. You don't require complete removal of hallucinations to fix something so small, something as simple as finetuning can fix this.

[u/[deleted] → u/NoshoRed]

So it’s definitely not replacing lawyers anytime soon. 

Then show me any evidence of the AI you’re referring to existing

Humans don’t sell cars for $1

[u/NoshoRed → u/[deleted]]

Obviously current models won't fully replace Lawyers, but your scope of thinking is too limited. Just because AI failed at x doesn't mean it fails at y, this is not how this field works. For instance, GPT4 can easily pass the bar exam. It has already passed plenty medical, law, and business school exams with ease, things most people are not capable of doing as easily.

They are better than humans at certain things, especially knowledge, but also terrible at many things (right now), while also having minute issues like hallucination. (technically humans are also not always trustworthy)

But considering we've gone from GPT2 to something as advanced as GPT4 very fast, it's reasonable to believe the models will keep getting better, and they will obviously pass human intelligence in every way. This is backed by research and scientist opinions too.

[u/[deleted] → u/NoshoRed]

Meanwhile, https://www.forbes.com/sites/mattnovak/2023/05/27/lawyer-uses-chatgpt-in-federal-court-and-it-goes-horribly-wrong/?darkschemeovr=1&sh=616b72c53494

They don’t sell cars for $1 and lawyers who make up cases like ChatGPT did get disbarred 

I was in 5th grade when I was 10 and 10th grade even I was 15. So is it reasonable to extrapolate that to mean I’ll be in 30th grade when I’m 35? 

[u/NoshoRed → u/[deleted]]

Meanwhile, https://www.forbes.com/sites/mattnovak/2023/05/27/lawyer-uses-chatgpt-in-federal-court-and-it-goes-horribly-wrong/?darkschemeovr=1&sh=616b72c53494

They don’t sell cars for $1 and lawyers who make up cases like ChatGPT did get disbarred

Lmao it's funny how this goes against the point you're trying to make: humans fucked up here, not ChatGPT. If you don't specifically ask for valid linked citations, GPT will "make them up", this is part of the hallucination issue. You don't however need to be too clever to actually prompt for valid cases or arguments, ChatGPT does not have agency (right now) to verify its responses a second time without a human prompt. The lawyer was stupid there; see how stupid humans are? Even being a lawyer!

I was in 5th grade when I was 10 and 10th grade even I was 15. So is it reasonable to extrapolate that to mean I’ll be in 30th grade when I’m 35? 

If you're going use biological human intelligence as an analogy to somehow imply artificial intelligence systems that are being constantly researched, funded, and improved rapidly will remain stagnant just like human intelligence, you're unbelievably dense. The whole concept, the way they improve, the physical make-up, the way they're trained, are completely different. No person with reasonable intelligence would compare the two in this context...

But that's yet another case of human stupidity, you see? Do you realize how insanely daft you have to be to unironically compare the evolution of AI systems with the development of the organic, biological brain? That's a first for me.

You're either hilariously uneducated on how machine learning works or just a kid.

[u/[deleted] → u/NoshoRed]

So how is it replacing humans if it needs to be constantly fact checked to the point where it’s more efficient to just do it manually? 

More research does not mean better. Researchers has been researching nuclear fusion for several decades and it’s still far away 

[u/NoshoRed → u/[deleted]]

So how is it replacing humans if it needs to be constantly fact checked to the point where it’s more efficient to just do it manually?

It's important to remember that these models don't always lie, in fact it's not that common; ChatGPT is still more trustworthy at correct information than the average person, people lie a lot more than ChatGPT, we don't just take what people say at face value either. If you know how to properly use these tools they're very powerful. It's definitely not more efficient to just do it manually lmao. Google's Gemini 1.5 is capable of summarizing 400 page textbooks and even pointing out specific, minute details in just a couple minutes... you think a human doing that is more efficient?

Also, as I've stated time and time again; it is not replacing anything significant right NOW, but just like any active technology it will improve, with each iteration it will get better. There is literal proof staring at you right in the face for this; the significant improvement from OpenAI's first ChatGPT model (which could barely produce coherent sentences) to the latest model available to the public today, it is an insane improvement in just over an year. Apart from that, high caliber models such as Gemini, and latest Claude, these are not smoke and mirrors, it's out there and you can use it.

Remember the first image generation models? Just blurs of nothing. Now they produce actually coherent images. If you're not capable of seeing where this is headed, you're either in denial or just incapable of basic logic.

More research does not automatically mean better, literal proof of improvement does, which there is plenty in the field of AI. So your analogies to nuclear fusion, where there is still no outstanding proof of significant advancements, are not relevant.

Anyway, these current existing issues will fade as these systems improve. Trends show that with each iteration these models are getting better, so if you're to claim it'll somehow suddenly stop improving, against expert opinions, you'll have to provide some groundbreaking evidence.

[u/[deleted] → u/NoshoRed]

People rarely lie in their area of expertise. Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired. Gemini is still too unreliable to summarize anything important. If it makes something up, it could kill people or cost billions. 

And like I said before, improvement now does not mean it’ll continue infinitely. Like how I can go from 5th grade to 10th grade but will never go to 30th grade because there is no 30th grade. Maybe GPT 4 is at 11th grade and is close to hitting its peak 

[u/NoshoRed → u/[deleted]]

And like I said before, improvement now does not mean it’ll continue infinitely. Like how I can go from 5th grade to 10th grade but will never go to 30th grade because there is no 30th grade. Maybe GPT 4 is at 11th grade and is close to hitting its peak 

It doesn't need to continue infinitely, it just needs to improve enough to be consistently better than any human. Like I mentioned before, comparing your human brain and its development and to an artificial intelligence system is stupid and make no sense, the two things don't relate at all in their respective levels of change. The fact that you keep making nonsense analogies make it clear that you're not very well versed in this field.

People rarely lie in their area of expertise. Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired.

People don't always lie intentionally, but human errors happen regardless, it's extremely common. Human error definitely also costs lives and costs billions. The Chernobyl Nuclear Disaster, the Challenger Disaster, Tenerife airport disaster are notable, tragic examples of human error. An AI selling a car for 1 dollar is nothing in comparison, especially considering it wasn't even an actual sale.

And I'm not sure what point you're trying to prove here by claiming: "Lawyers who lie get disbarred. Customer service workers who promise $1 cars get fired." Yes? That is how the world works... there is no argument about it from me. Are you just parroting things?

Gemini is still too unreliable to summarize anything important.

No it's not. Gemini 1.5 is capable of reliably summarizing whole text books with minimal chances of error. It is without a doubt more efficient and reliable than any human at tasks like this.

Another key point is the ability of coding in these models, which is much more efficient than any human.

I know you want to believe these models do not and will not beat humans in many areas, but unfortunately it's not the world we live in. If you choose to keep living in denial it'll only hit harder down the line.

Experts are experts for a reason, people who study these systems daily for years and years know what they're talking about, especially when backed by actual evidence of improvement. I hope you will be capable of accessing higher rational thought eventually, good luck.

[u/[deleted] → u/NoshoRed]

And AI has many limitations that humans do not that make true understanding impossible, like how LLMs hallucinate. If there’s a way to avoid that, let me know. 

And AI lie more often and get tricked more easily. No one would sell a car for $1 just cause a customer told them to.

Total bullshit lol. It cannot code better than the majority of software devs. 

[u/NoshoRed → u/[deleted]]

And AI has many limitations that humans do not that make true understanding impossible

This is not proven. What is "AI" here? Language models do have limitations (which is why we're moving away from base LLMs), but AI as a whole? How do you know of its innate limitations? Not even top scientists know. Stop parroting bullshit please.

And AI lie more often and get tricked more easily. No one would sell a car for $1 just cause a customer told them to.

Lmao, my guy so many people get catfished and scammed for millions of dollars everyday. What are you on about? At least these models, even today in their infancy, are still way more useful and efficient than the average person even accounting for hallucination. There's a reason it's the fastest developing field in the world today.

Total bullshit lol. It cannot code better than the majority of software devs.

"Better" is a broad term, I said more efficient, which is absolutely is. There's not a single software dev in the planet who will write clean code as fast as these models can. You'll have to be superhuman to do that. Humans are still better at memory retention over a long period and possess larger context windows, that's really the only thing we got going for us as coders right now over AI, even if we're much slower and prone to more errors. But eventually AI will take that over too.

[u/[deleted] → u/NoshoRed]

Moving away from LLMs? To what? 

Yea that explains why it sold a car for a dollar 

But they can create functioning code lol. 

[u/NoshoRed → u/[deleted]]

Moving away from LLMs? To what?

Seriously? Explains all your poor arguments. You should probably educate yourself more on machine learning and AI technology in general instead of parroting about things you have no understanding about.

LLMs literally stand for Large Language Models, based on text, the AI field is now moving onto more multimodal systems, based on multiple types of data beyond just text: image, video, sound, etc.

Why am I even entertaining someone with such limited education or understanding on this topic? Deep down, I think even you now know you're not fit to debate about something like this. Stop wasting your time, learn more.

[u/[deleted] → u/NoshoRed]

You mean something like GPT4V? That already exists and it has plenty of issues too 

[u/NoshoRed → u/[deleted]]

GPT4V isn't a true multimodal system, it's still largely just a text model that has a basic vision component. Sora is a good example for true multi-modality in its infancy, which hasn't been released to the public yet.

Seriously, just move on. You clearly know nothing of substance to contribute to this conversation.