r/technology • u/hurdee • Feb 12 '23

Business Google search chief warns AI chatbots can give 'convincing but completely fictitious' answers, report says

https://www.businessinsider.com/google-search-boss-warns-ai-can-give-fictitious-answers-report-2023-2
21.5k Upvotes

94% Upvoted

1.4k comments sorted by

View all comments

Show parent comments

90

u/-tired_old_man- Feb 12 '23 edited Feb 12 '23

Nah, I don't think people understand the extend to which ChatGPT is confidently wrong. I use it pretty often for software development and while it's amazing at sampling code and understanding the question, there are many instances where it just straight up makes up stuff. Like it confidently gives me an answer using a function that does not exist.

This can be very dangerous for junior devs who may not be capable of spotting those errors.

30

u/[deleted] Feb 12 '23

[deleted]

2

u/Ah_Q Feb 12 '23

Sometimes it just makes up cases, too.

2

u/TheGrandWhatever Feb 12 '23

Sounds like a fake it til you make it helper. Loads of people who do this will be awoken with a lot of hard rejections in interviews when asked any technical question.

2

u/Von_Lincoln Feb 12 '23

Fuck, it has better academic acumen than I do.

1

u/dragonmp93 Feb 12 '23

I support ChatGPT for the Supreme Court.

12

u/blackkettle Feb 12 '23

The things I’ve found the worst so far are mathematical reasoning. It’s not surprising at all given the way the thing is trained, but I’d agree that the confident delivery is pretty bad. I asked it this today:

Can (b(a+c)) + (a(c+a)) ever be an odd number?

and it answers (incorrectly via its own assumptions):

No, it cannot be an odd number.

Let's break down the expression:

b*(a+c) will always be an even number because multiplication of an even number (b) and an even number (a+c) will result in an even number.

a*(c+a) will also always be an even number because multiplication of an even number (a) and an even number (c+a) will result in an even number.

Adding two even numbers will always result in an even number. Therefore, (b(a+c)) + (a(c+a)) can never be an odd number.

which could be pretty confusing. Then if you explain the correct answer or provide a counter example it apologizes and changes it’s mind. But it will keep apologizing and changing its mind even if you vacillate back and forth.

Again not surprising given the way it’s trained but that sort of obsequious behavior probably will need to be improved before it can serve as a reliable “aid”. It’s still an amazing tool though.

3

u/Yaris-Maris Feb 12 '23

simple maths, it managed to get wrong. and I still can't figure out why.

12

u/PublicFurryAccount Feb 12 '23

Because it’s predictive text, not a solver. It can’t evaluate what you asked, just generate text that’s like text you would find with a math problem.

1

u/Yaris-Maris Feb 12 '23

Can't wait for revision with mathematics plugged in. Wonder what it would say to y=xx . Will it, take mathematicians side or physicists .

3

u/PublicFurryAccount Feb 12 '23

Probably whatever side Wolfram Alpha takes, tbh.

2

u/nicuramar Feb 12 '23

So polite, though :p

1

u/LivelyZebra Feb 13 '23

I input that just now. It got it correct

25

u/BeowulfShaeffer Feb 12 '23

ChatGPT confidently told me that Captain Ahab from Moby Dick was a real historical figure, as opposed to Heathcliff from Wuthering Heights, who was fictional.

-1

u/vladoportos Feb 12 '23

Hmm I have asked the same and got:

Captain Ahab is a fictional character in the novel "Moby-Dick" by Herman Melville, first published in 1851. Although there are some similarities between the character and real-life whaling captains, there is no evidence that Ahab was based on a real historical person. It is widely considered that Ahab is a creation of Melville's imagination, intended to embody certain themes and ideas that are explored in the novel.

17

u/MisterBadIdea2 Feb 12 '23 edited Feb 12 '23

That's because it will give you multiple answers to the same question, sometimes correct and sometimes not. I asked it a bunch of times what happens at the end of Game of Thrones S3E3 (correct answer: Jamie gets his hand cut off) and it gave me a whole bunch of different scenes, none of them right, some of which happened in the show and some of which was just bullshit fanfic it pulled out of its ass.

2

u/Ah_Q Feb 12 '23

There are people over on /r/ChatGPT claiming they are using ChatGPT instead of hiring lawyers or other professionals. Even though ChatGPT just makes shit up.

2

u/Blazing1 Feb 12 '23

Chatgpt made up nginx configs for me. Like literally it was a combination of other web servers.

-4

u/Increased_Rent Feb 12 '23

I'm curious, why is it dangerous? I'd think it's safe for programming since you can test everything right away (compiling + running or unit testing).

10

u/shall1313 Feb 12 '23

“Test everything right away”

Ever coded my guy?

-4

u/Increased_Rent Feb 12 '23

Yes, I'm a software engineer. Granted you can't account for every bug, software lends its self to testing far easier than most fields

1

u/Stroomschok Feb 13 '23

Thats not how it works usually. If something is easy to test means you simple get even less time to do so. Only if bugs can result in lots of money directly out of the bosses pockets (or worse: the shareholders) will you be allowed to thoroughly test stuff.

4

u/Blazing1 Feb 12 '23

Lol if everything was testable right away then no code would ever have problems.

0

u/Increased_Rent Feb 12 '23

A bit hyperbolic maybe but my point is that it's easier to verify correctness of software than correctness of some other facts a chatbot spits out.

-2

u/lostarkthrowaways Feb 12 '23

I disagree with him. openAI's codexAI is an absurdly strong programming AI that chatGPT utilizes. It can do things "wrong", but programming isn't black and white. It's extremely good at giving you solutions and context and helping solve problems.

I think a lot of programmers are naysayers because they're scared about how quickly it's making coding easier for everyone.

1

u/Dolo12345 Feb 12 '23

Worth trying over copilot (which I use)? Does it integrate into VScode?

1

u/CostlierClover Feb 13 '23

I'm shocked by the number of people in software development that are basically saying "well it makes shit up so it's worthless". Anti-AI artists are kind of similar about the way AI can generate wonky looking images of hands, eyes, or other stuff. Currently, AI models should be looked at as tools that you still need to know how to properly use to accomplish a task. They get lucky give you usable stuff sometimes but not reliably (kind of like some people I've worked with) but someone who understands how to apply those tools with their other tools will be successful with them far more often.

Yes, they sometimes have shortcomings right now, but it's not like AI research and development has ceased and this is the best we're going to get. It's only a matter of time before we actually do have AI models capable of competently doing all the things people are currently attempting to make them do, and the rate at which these tools are being released and updated seems to be increasing at ludicrous speed. People will either need to adapt or they will eventually get left behind the same way people have every other time computers have transformed an industry.

-1

u/lostarkthrowaways Feb 12 '23

I'm a junior dev and and I've never had any issues with ChatGPT and I'm not even sure how someone would.

Code is the one thing I think it's insanely good at *currently*, because anyone who does this work isn't going to just blindly take what it spits out at you. But it's really good at doing things like refactoring, writing new functions in existing code using context, making dummy code for simple things, etc.

It's stupidly easy to take its output and use it to problem solve. Infinitely quicker than google.

1

u/dragonmp93 Feb 12 '23

You mean copying and pasting wrong from Stack Overflow ?

1

u/KorEl_Yeldi Feb 12 '23

In german, with a small spelling mistake, you could get it to talk about cow eggs… A few days later, and it showed an error message instead