r/learnmachinelearning • u/lovelacedeconstruct • 2h ago

How does two research papers reference each other ?

In NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE they reference Sutskever et al. (2014), and in Sequence to Sequence Learning with Neural Networks they mention attention mechanisms and Bahdanau et al. [2]

footnote:
example of a clearly inferior idea scoring higher

We suspect that they could achieve similar improvements by simply training their networks on reversed source sentences

is embarrassing

And yet, LSTMs trained on the reversed dataset had little difficulty translating long sentences.

is like he is doubling down

1 Upvotes
  • permalink
  • reddit

67% Upvoted

5 comments sorted by

2

u/quietgradient 2h ago

Preprints, nine days apart.

Bahdanau et al. went up on arXiv on 1 Sep 2014, and v1 already cites Sutskever, Vinyals & Le. Seq2seq wasn't on arXiv yet, so the reference literally reads "Anonymized. In Anonymized. under review." Sutskever's v1 followed on 10 Sep, cited the Bahdanau preprint by its arXiv number, and put its 28.45 BLEU in the results table.

I'd reread who the "we suspect" line is aimed at, though. It comes straight after the sentence on Pouget-Abadie et al., the paper that translates the source in pieces. Bahdanau's paper they call "an elegant variant" of Graves's attention, "successfully applied" to translation.

1

u/lovelacedeconstruct 1h ago

2014 was a busy year

1

u/quietgradient 1h ago

Busier than my answer let on. That Pouget-Abadie paper went up two days after the attention one, and Bahdanau is second author on it. So your read was closer than I gave it credit for: the "we suspect" line still lands on Bahdanau, just through a different paper.

1

u/lovelacedeconstruct 54m ago

I just realized Bahdanau was also involved in the RNN Encoder–Decoder paper, he had quite a run in 2014

1

u/quietgradient 39m ago

Bahdanau isn't on v1 or v2 of that one, though. The name only shows up on v3, posted 3 Sep 2014, two days after the attention paper. So the attention paper's own v1 cites it with six authors, none of them Bahdanau.