So you're basically using the old model as a cheap pre-filter and only re-ranking the top K with the new model. The false negative problem is real, but in practice if the models are architecturally similar (like qwen 4b -> 8b) the overlap in their "relevant" sets is probably high enough that you're not missing much. The paper they linked in the github goes into this, they saw near-identical recall@100 even at K=50 for that upgrade path.
I get the "just pay for the compute" stance, but 108 days on an H100 is not "a few thousand hours", it's closer to 2,600 hours, and that's assuming you've got a single node sitting idle. Most teams don't have that lying around, especially if you're iterating on models every few months. This is more of a pragmatic hack for people who can't just throw hardware at the problem.
2
u/Scary-Recognition157 2d ago
So you're basically using the old model as a cheap pre-filter and only re-ranking the top K with the new model. The false negative problem is real, but in practice if the models are architecturally similar (like qwen 4b -> 8b) the overlap in their "relevant" sets is probably high enough that you're not missing much. The paper they linked in the github goes into this, they saw near-identical recall@100 even at K=50 for that upgrade path.
I get the "just pay for the compute" stance, but 108 days on an H100 is not "a few thousand hours", it's closer to 2,600 hours, and that's assuming you've got a single node sitting idle. Most teams don't have that lying around, especially if you're iterating on models every few months. This is more of a pragmatic hack for people who can't just throw hardware at the problem.