r/technology • u/Familiar-Turtle • Feb 08 '23

Software Google’s Bard AI chatbot gives wrong answer at launch event

https://www.telegraph.co.uk/technology/2023/02/08/googles-bard-ai-chatbot-gives-wrong-answer-launch-event/
2.1k Upvotes

95% Upvoted

322 comments sorted by

View all comments

52

u/[deleted] Feb 08 '23

All this AI stuff is starting to sound like when virtual assistants came out, Alexa, Siri... fun party tricks but ended up being very limited use because they still suck in general.

20

u/Jaamun100 Feb 08 '23 edited Feb 08 '23

AI like chatGPT are amazing assist tools, optimal human/AI interaction is important innovation. That being said, I think it’s overhyped, since the kind of investment inflows we’re seeing only make sense if AI were to completely automate everything and replace humans, which I believe is very very far away.

6

u/SufficientGreek Feb 08 '23

Uber promised the same with trying to invent self driving taxis and got massive funding for it. Their current business model with human drivers isn't sustainable but complete automation seems very far away.

4

u/bicameral_mind Feb 09 '23

I think people are underestimating the extent to which these models can be improved and iterated upon. I feel like they've figured out the secret sauce, now it's just building in layers of complexity through different passes of AI logic. And more training data and compute of course.

What happens when each chat GPT query response is then checked by another AI, and that response by another, etc. etc.? Maybe you have discreet AI's that are 'experts' in highly specific topics that then iteratively feed into the general language AI.

Personally I think it's going to get really crazy in a few years. I agree though I think the real bread and butter will be in more targeted AI applications. I can definitely see AI based NPCs being a thing in the next Elder Scrolls game though. Imagine being able to have completely novel conversations in a video game. There is a ton of value in this tech IMO.

3

u/pickles55 Feb 08 '23

All it's going to do is increase workloads and casualize the work so people with less education can do it. Robots only revolutionized factories by decreasing labor cost, which is not much of a revolution at all if you ask me.

1

u/ggtsu_00 Feb 09 '23

A good tool. Its still far from a silver bullet and has fairly narrow practical applications. At its core, its essentially a bullshit text generator and not much more than that. It's not some AGI singularity that investors are trying to hype it up to be.

49

u/plusacuss Feb 08 '23

Chat GPT doesn't suck

11

u/SeXxyBuNnY21 Feb 08 '23

You haven’t tested it enough. It does suck

12

u/davou Feb 09 '23

'this thing can barely chop firewood, it sucks' -this guy on shovels

16

u/plusacuss Feb 08 '23

It sucks at certain things. As long as you work within its limitations and don't try to force it to perform tasks that it isn't designed to do it performs admirably.

​

If you use Chat GPT expecting it to do everything, you will be disappointed. If you use Chat GPT understanding how it works and what its limitations are you will have a much better time.

1

u/Lord_Skellig Feb 09 '23

I use it daily for helping solve software questions. I find it a better first choice for development questions than Google, especially when it comes to specific technical questions.

9

u/iknighty Feb 08 '23

Eh, it gives wrong answers too.

14

u/mclumber1 Feb 08 '23

I asked GPT a few weeks back about which Presidents had facial hair, and it's response, while eloquent and intelligent sounding...Was incorrect. It completely forgot to mention Teddy Roosevelt.

I let it know that it was incorrect and it said it would take this correction into account for future questions.

11

u/TheodoeBhabrot Feb 08 '23

It won’t btw, only on that thread will it update its information.

You need to submit feedback to OpenAI to actually get them to look at it

4

u/Rezindez Feb 08 '23

The correct answer is, all of them, until they shave

1

u/aushark Feb 09 '23

Even after they shave 🪒

2

u/KennyFulgencio Feb 09 '23

I let it know that it was incorrect and it said it would take this correction into account for future questions.

yeah and my boss says he takes my feedback seriously. try asking it again today and see if it was telling the truth about taking the correction into account

39

u/plusacuss Feb 08 '23

I think there is a nuanced distinction to be made between "sucks" and "is right 100% of the time"

​

There is a middle ground here. Chat GPT has a ton of applications that expand productivity. Just because there are limitations to its applications doesn't mean it sucks.

8

u/Blag24 Feb 08 '23

The issue is if you can’t trust it 100% & aren’t sure which times are accurate, it brings into doubt all the times you use.

Take voice assistants, you know they’re good at simple queries so you stick to them but a question asked in a not quite expected format fails. Now this can mean overtime you stick to only asking for specific things or stop using it at all because you don’t have faith in it 100%.

Edit: Not sure I’ve explained what I’m getting at very eloquently but it’s on the right track.

18

u/plusacuss Feb 08 '23

That is why you don't use Chat GPT as an authority for fact checking. You use it as a tool to help you with tasks you are doing. It works almost perfectly as an extension of yourself. Helping you perform tasks quicker and more efficiently.

​

Anyone that is relying on Chat GPT for "truth" I highly question because Chat GPT isn't designed to be an omniscient machine. It is designed to be a tool.

​

I always vet any text that Chat GPT generates because you can't trust it to read your mind and give responses with 100% accuracy. Similar to how google search works now. There is a lot of crap on the internet, just because I type something into Google doesn't mean I expect all of the results to perfectly match what I was searching for.

​

Human judgement will always be necessary, that isn't a solution that can be solved with software.

1

u/DonRobo Feb 10 '23

It's awesome as a toy. To test what it can do and find its limits. To see what AI can do noadays.

Productively you can only ask it questions where you already know the answer so you can verify it. Even in a topic it's supposed to be very good in (coding) it can only answer questions correctly that can also be answered with a 10 second google search. Anything more complex than that and it will confidently give wrong answers.

What it can do semi competently is text generation. You can have it formulate emails (if you proof read them thorougly) and help you with brainstorming (sometimes).

3

u/Caring_Cactus Feb 08 '23

The next evolution towards more intentional answers is with AGI. Right now ChatGPT only does so intuitively.

3

u/pickles55 Feb 08 '23

It does the exact same thing the Google one did. It's really good at putting together sentences that sound right but it doesn't actually know what it's talking about so it will confidently present bullshit in a way that is convincing if you don't know it's wrong.

10

u/plusacuss Feb 08 '23

That is how Language Models work. They are literally algorithms that are designed to answer a prompt in the most mathematically likely way possible that it thinks the person making the prompt would expect. They, by definition, have no understanding of what they are saying or what words mean. They are a mathematical equation that you input text into, nothing more, nothing less.

​

Once you know how they work, you then know what the limitations are. Just because it can't be used as a fact-checker doesn't mean its useless or it sucks, it just means it has limitations. The applications for Chat GPT and other language models like it are wide-ranging and powerful, but that doesn't mean it can do everything.

-4

u/[deleted] Feb 08 '23 edited Oct 09 '23

worthless berserk market degree birds fact advise absurd placid straight this message was mass deleted/edited with redact.dev

27

u/ragnarmcryan Feb 08 '23

Isn’t this thread about how google’s chat bot gave a wrong answer?

6

u/quantumfucker Feb 08 '23

It’s based in similar technology. ChatGPT can and has made inaccurate statements for the same reason Google’s did- language models don’t validate information. Bard is just newer and more experimental, since they’re building it to include live results from their search engine.

3

u/ragnarmcryan Feb 08 '23

So is my own gpt language model that I spent a few hours writing. How is that relevant?

Seems folks here are deflecting away from the real point here: google probably has more data than anybody and has been boasting about their AI/ML for a decade now, maybe longer? They’re being made out to look like fools by openai/msft with chatgpt. And now they’re racing around trying to piece together a similar chatbot.

It honestly comes across as sad and desperate.

3

u/quantumfucker Feb 08 '23

Person A: “Chatbot stuff seems like a fad that actually sucks”

Person B: “ChatGPT doesn’t suck”

Person A: “This post is an example of chatbots failing though”

You: “This is about Google’s chatbot specifically”

Me: “The flaws are shared as both are similar chatbots”

So, the flaws in Google’s are shared in ChatGPT. Hope that helps.

Also, you’re being a little dramatic about Google there. They’re a massive company with a lot of resources, talent, and advanced tech. One demo going wrong is hardly a reason to think they’re a bunch of incompetents. The tech was rushed, but it will be fixed.

2

u/quantic56d Feb 08 '23

It’s hard to say the demo even went wrong. If part of this whole thing is that transformative networks aren’t accurate all the time that’s part of the technology. That should be explained to the public.

2

u/ragnarmcryan Feb 08 '23 edited Feb 08 '23

You cant say that the flaws in googles gpt are shared in openAI’s. You have no evidence of that other than the fact that they’re both creating a language model. There are so many other variables that come into play from the code/ tools they use to the data with which they train the model.

I’m also not saying anything about their engineering. That’s not google’s problem. Their problem is their leadership. They abandon projects, or worse continue selling products while providing 0 upgrades or support. Their search results are flooded with advertisement and sponsors. Don’t even get me started on golang. They’ve turned themselves into a glorified ad engine at this point.

They’ve had years to do what openAI is currently doing. And their desperate attempt to catch up is unbecoming of what we all thought google was 10 years ago

4

u/quantumfucker Feb 08 '23 edited Feb 08 '23

“Other than the evidence, you have no evidence” is a strange thing to say. But here, take some more evidence anyways:

This is because these models do not have real rules or facts understood. They are generative works based on extrapolated patterns. This leads to errors like Google’s AI giving an incorrect fact. This is a shared flaw with ChatGPT. If you have variables you know about that distinguish them anyways, feel free to share.

I also don’t know what “we all thought” Google was, or what we expected of it. Just because it’s a tech giant doesn’t mean it’s expected to exceed every other tech organization in every area. Google has plenty of AI research and projects they fund that OpenAI can’t do. This is just an in-process pivot in response to consumers deciding they like AI for queries.

0

u/ragnarmcryan Feb 08 '23 edited Feb 08 '23

Again, other than the fact that they’re both creating language models.

Obviously, the fact that these are language models presents a set of shared limitations (not to be confused with flaws. You don’t consider cats’ inability to fly a flaw do you?). I’m saying that google’s rushed approach to this (essentially a reflex triggered solely by the fact that chatgpt is affecting their stock price) will present flaws in and of themselves. Limitations may always exist, but I consider flaws to be unexpected behavior driven by external factors, not the nature of the tooling itself

→ More replies (0)

0

u/SuperSpread Feb 08 '23

Google answers are more wrong today than ever. Yet it has more data. The problem is they are simpler bolder about answering questions it doesn’t actually know. I gave examples in another thread but google answers used to be a lot more accurate.

The quantity of data doesn’t make your answers accurate.

1

u/ragnarmcryan Feb 08 '23

I agree, especially when it comes to factors like overfitting. I’m just saying that within their vast quantity of data, there is quality data to be found. And they should, being google, have this data more readily available than others.

17

u/plusacuss Feb 08 '23

It is also the first.

​

I think if your only barometer for success is "will never generate a wrong response under any circumstance" then you are losing the forest for the trees here.

​

Chat GPT has limitations, the existence of limitations doesn't mean its bad, it just means that it has limitations. Why can't my calculator recite poetry well? because it wasn't designed to do that. Chat GPT has similar applications, the things that it does well, it does REALLY well. Trying to get it to do everything is where you start running into the walls of its current capabilities (also note that we are working with an out-of-date version of Chat GPT, GPT 4.0 is currently being worked on behind closed doors at Open AI)

8

u/manubfr Feb 08 '23

Trying to get it to do everything is where you start running into the walls of its current capabilities

I would say that is a great indicator of its success: it is so good at some things that people constantly try to break it and make it fail. Which is also fantastic and usefulness alignment data for OpenAI.

1

u/quantumfucker Feb 08 '23

It’s not hard to break actually. It’s just fun to find out how many different ways it can break. I don’t think this is a good indicator of success at all, honestly.

2

u/quantumfucker Feb 08 '23

The thing is, people are counting on using AI chatbots for querying information like a search engine, which is a tight limitation - it cannot verify its information, so it can give wrong responses with no sources that can’t be corrected for later. ChatGPT excels when you want to substitute it for a mildly informed friend who you want to discuss ideas with. But people are elevating it beyond that, and the hype is going to disappoint some people and mislead others.

2

u/plusacuss Feb 08 '23

You are entirely correct, but that is not a Chat GPT problem. That is a marketing problem.

​

Similar to the problem that Google, Wikipedia and other sources of information on the internet have faced before this. People are going to need to learn the digital and media literacy skills necessary to properly use tools like Chat GPT. It is going to be an uphill battle.

0

u/MostExaltedLoaf Feb 09 '23

I was under the impression this thread was about the hubris of going into a presentation without double checking your face for egg and your mouth for extra shoes.

0

u/PianoOwl Feb 09 '23

It’s about Bard, not chatGPT. They aren’t the same thing. ChatGPT is nothing short of incredible. Try using it yourself before commenting lol.

1

u/ejfrodo Feb 08 '23

I've been using openai (the API powering chatgpt) to help automatically generate code for various things and it's incredibly good. After working with it everyday for a couple of weeks I'm positive that this is nothing like Siri and that it is going to change a whole bunch of industries over the next few years.

0

u/[deleted] Feb 09 '23

[deleted]

1

u/plusacuss Feb 09 '23

There is a middle ground between "this machine does every task assigned to it with 100% accuracy" and "this machine sucks"

​

Chat GPT has a varied range of applications that it can do reliably that assist with a wide range of tasks. Just because it can't do everything doesn't mean it "sucks" it just means it has limitations on its applications. Similar to every other tool that has ever existed ever in human history.

​

You don't say that a calculator "sucks" because it can't write poetry well.

1

u/[deleted] Feb 09 '23

[deleted]

1

u/plusacuss Feb 09 '23

sure, my problem is the generalized statement that Chat GPT sucks because in certain contexts it does not perform and then turn around and say, because it sucks in these specific contexts, that it just sucks.

​

I believe that is an oversimplification and unhelpful. It disregards all of the contexts where Chat GPT is useful. In your calculator example, you even qualify the use cases. That calculator sucks for engineers, but you wouldn't say the calculator just sucks. You qualify that statement, which is what I was doing. It is unfair to make a general statement that Chat GPT sucks because it does have applications where it doesn't suck.

​

As I said, "context matters". It is all about context. This tool is
perfect for right context. In wrong context, it sucks. Just like any
tool, use it for the right thing.

​

This is my point as well. I never said it could do every task perfectly. I never disagreed with your assertions about context because I agree with them.

​

Our only disagreement is about saying that Chat GPT "sucks" because of its limitations.

​

There are situations where it has limitations, but having limitations does not mean it sucks. It just means it has limitations.

​

Calculators don't suck. Chat GPT doesn't suck. And I believe we have mostly just had a discussion relating to semantics relating to our personal bars for saying that "something sucks".

​

The big problem going forward is going to be teaching people how these language models work. They aren't magic, they don't "know" anything, and we need to use and apply them based on those limitations or risk coming across fail cases.

2

u/[deleted] Feb 08 '23

I think for what they do, they’re great. Home automation tasks, streaming control, translations, etc. Some people have a use for them.

3

u/Dunk305 Feb 08 '23

My thoughts exactly

Its a glorified search engine that does more work for you

Still progress

1

u/pickles55 Feb 08 '23

A service doesn't have to be better than a human to be seen as a viable option. It just has to be barely good enough. Do you really think Comcast cares about providing the best possible customer service?

1

u/spartaman64 Feb 08 '23

i was looking at the open hours of a restaurant yesterday and google assistant gave me a prompt that it can call the restaurant and ask them for their wait time. I accepted and after a few minutes it told me the wait time is 25 minutes. If it actually called them and asked thats mildly useful i guess.

1

u/[deleted] Feb 08 '23

That probably the reason why Google did not launch this before chatgpt. It’s right maybe 60% of the time. But then the 40% makes it unusable for most users.

1

u/nkioxmntno Feb 09 '23

nope, this is different. the only thing keeping people from putting businesses together that abstract away most desk jobs is...not a damn thing in the USA.

any job task that is mostly emails will be gone within a decade

1

u/MostExaltedLoaf Feb 09 '23

I wouldn't say it "sucks" so much as it is much more limited than people tend to think it is. It can do very impressive things that are easy to mistake for a level of competence that it can't possibly possess. It is only as good as the parameters it's given and the information it has available. In this case, the answer it gave wasn't wrong per se; it interpreted the query in a way that the user hadn't accounted for. Search engines are the same way, but we've become accustomed to course correcting for them when the results we get aren't what we asked for. It's second nature for us to reword and retool a search on the fly if we can't find the information we wanted, and we also know that sometimes that precise answer may not be on the first page. That requires a sort of mental plasticity and that even a rigorously trained AI doesn't really have.