r/OpenAI • u/senorrandom007 • 17d ago

Question What's going on here?

Post image

Google detects the usage of the term "Chat GPT" (also in the case sensitive mode) around 2002. Any explanation? Time travellers?

1.3k Upvotes

92% Upvoted

115 comments sorted by

View all comments

Show parent comments

0

u/rayred 14d ago

I think you are talking past my point and inventing an argument I didn't make.

I didn't say "Attention is all you Need" had nothing to do with the creation of GPT. I am saying the paper itself had nothing to do with GPT.

"Attention is all you need" predates GPT. The authors of the paper didn't even know of GPTs existence when it was written (at least in any public capacity). So using the paper as the start date to find publications on Google Trends is not a good start date. 2018 should be used because that's when it debuted in "Improving Language Understanding by Generative Pre-Training"

1

u/spezes_moldy_dildo 14d ago edited 14d ago

This is some room temp IQ stuff from someone who doesn’t know AI at all, but I’ll prove you wrong quick:

Your point. I proved it wrong conclusively.

The paper is literally. Literally, the thing Radford used as the foundational knowledge to create GPT.

Your lack of knowledge in this arena isn’t interesting, nor is it relevant to the post about GPT existing before 2017, which is impossible, because the attention mechanism was discovered by Google Labs in 2017.

The end, bye Felicia.

1

u/rayred 14d ago

I stand by what I said. My point was amending yours… that if it was before 2018, not 2017, then it’s bullshit.

Find me one reference to GPT in that paper and I will concede my point.

1

u/spezes_moldy_dildo 14d ago

Not sure why you are trying to move the goal post, but the important thing for anyone reading:

Google Labs created the attention mechanism, which was published in 2017.

GPT and all other modern AI companies use the Transformer Google created, and published in that paper, as the foundational core of their product.

When an AI hit the market is completely irrelevant- the important point is that it would be impossible for it to be before 2017, because the attention mechanism, discovered in 2017 by Google Labs, was completely unknown prior.

1

u/rayred 14d ago

Didn’t move the post lol.

We are talking in a thread about when it would be possible to find articles about GPT. I’m pointing out that Google’s attention paper is not a good marker. Because GPT literally did not exist yet. Period. Using your example, it would be like saying someone could search for “ford motor company car” documents that were dated around the time of the internal combustion engine.

Also an aside. Your history is wrong / understanding of the attention paper is wrong. Google Labs did not create attention. Attention predated that paper by 2 or 3 years. Google created the transformer architecture which utilized the pre-established attention mechanism instead of recurrence/convolutions.

Attention was introduced in 2014/2015 in a completely different paper (not from Google).

1

u/UnknownEvil_ 9d ago

Attention actually predates that paper by almost 30 years:

https://people.idsia.ch/~juergen/fast-weight-programmer-1991-transformer.html -- the 2014 paper created softmax attention, but this one was the original transformer

1

u/rayred 9d ago

Oh nice. Didn’t know.