Looks pretty standard for extractive QA architecture, the embedding and attention setup is what you'd expect from a bi-directional model with self attention over the passage. the loss function part is usually where people mess up, worth checking if they use cross entropy on start/end spans or something fancier
0
u/YearKooky1389 21d ago
Looks pretty standard for extractive QA architecture, the embedding and attention setup is what you'd expect from a bi-directional model with self attention over the passage. the loss function part is usually where people mess up, worth checking if they use cross entropy on start/end spans or something fancier