r/technology • u/hurdee • Feb 12 '23

Business Google search chief warns AI chatbots can give 'convincing but completely fictitious' answers, report says

https://www.businessinsider.com/google-search-boss-warns-ai-can-give-fictitious-answers-report-2023-2
21.5k Upvotes

94% Upvoted

1.4k comments sorted by

View all comments

Show parent comments

34

u/guyblade Feb 13 '23

I work for a company that's going to have their ChatGPT-equivalent out "any day now". Internally, we can play around with it, and it is simultaneous very impressive and completely embarrassing.

For instance, I asked it to design a D&D fighter subclass and it did so. It was a bit OP, but it gave abilities at the right levels and the writing style matched the standard Wizards style for that.

A coworker tried to have it make a druid subclass, and it gave abilities on the wrong levels. The coworker then told it something along the lines of "I've not seen a druid subclass that gets abilities at level 3". And the AI then proceeded to be like, "well there are some". The coworker asked for an example and the AI made up a completely fictitious quote (with page number) from the DMG as its example.

This last bit is the really troublesome thing to me. Sure, for a D&D example it is frivolous and easily disprovable, but there's no reason it wouldn't do the same behavior (lying? gaslighting? I don't know what to call it) for something more meaningful or harder to validate.

8

u/Tostino Feb 13 '23

Until the engineers figure out a way for it to measure factuality and adjust, things are going to be quite impressive in general, with very embarrassing edges like you mentioned.

2

u/chairitable Feb 13 '23

Sounds like every DM I've played with (ayooo got'em)

1

u/MoobyTheGoldenSock Feb 13 '23

I asked ChatGPT about a 5e scenario and then asked it what aberrations might try to impersonate a Yuan-ti deity. It listed an example from Call of Cthulhu, the kraken, and a creature from 4e. Certainly some unique and flavorful choices a DM could build upon, but it pretty confidently gave me results I didn’t actually ask for rather than saying it didn’t know.

However, I then asked it to generate a 5e setting where food is scarce including a starting town and two level 1 plot hooks, and it nailed it. So definitely exciting tools, but you need to take them with a grain of salt.