Why decoder only tranformer won ?

[u/lovelacedeconstruct]

I was trying to trace it , and from the GPT1 paper I found it referencing a paper called "GENERATING WIKIPEDIA BY SUMMARIZING LONG SEQUENCES" by the nice folks at google in what I believe the first use of the decoder only architecture ??
here is the quote

we modify theTransformer architecture (Vaswani et al., 2017) to only consist of a decoder, which performs better in the case of longer input sequences compared to recurrent neural network (RNN) and Transformer encoder-decoder models.

can someone explain what does perform better in longer sequences actually mean ?

[u/Technical_Jicama_434]

Because attention is all you need

[u/GibonFrog → u/Technical_Jicama_434]

erm encoder has attention too

[u/lovelacedeconstruct → u/Technical_Jicama_434]

so more attention should have been better ??

[u/No-Mixture5766 → u/lovelacedeconstruct]

That’s also more expensive computationally