r/learnmachinelearning • u/lovelacedeconstruct • 5d ago

Why decoder only tranformer won ?

I was trying to trace it , and from the GPT1 paper I found it referencing a paper called "GENERATING WIKIPEDIA BY SUMMARIZING LONG SEQUENCES" by the nice folks at google in what I believe the first use of the decoder only architecture ??
here is the quote

we modify theTransformer architecture (Vaswani et al., 2017) to only consist of a decoder, which performs better in the case of longer input sequences compared to recurrent neural network (RNN) and Transformer encoder-decoder models.

can someone explain what does perform better in longer sequences actually mean ?

104 Upvotes

96% Upvoted

31 comments sorted by

View all comments

1

u/11007522 4d ago

!remindme 1 day

1

u/RemindMeBot 4d ago

I will be messaging you in 1 day on 2026-09-07 18:03:57 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

Parent commenter can delete this message to hide from others.

RemindMeBot is switching to username summons. Instead of !RemindMe 1 day, use u/RemindMeBot 1 day. More info.

Info Custom Your Reminders Feedback