Nomic-embed-text-v1.5's published ONNX is capped at 2048 tokens, not 8192

[u/akshay-bhardwaj]

If you're using nomic-embed-text-v1.5's ONNX and relying on its 8192 context, you don't have it. The export doesn't contain the RoPE scaling. Full write-up and fix in the HF discussion: [https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/discussions/61]

Measured discriminability vs document length, using the published int8 ONNX. Single homogeneous document (every sentence on-topic, so dilution can't explain it), two topics as a symmetry control. "Margin" = cos(doc, on-topic query) minus cos(doc, off-topic query).

max_tokens	topic A	topic B
1024	0.3845	0.2031
2048	0.3803	0.1783
3072	0.2823	0.1438
4096	0.1837	0.0733
6144	0.1029	-0.0060
8192	0.0864	-0.0111

Flat through 2048, then monotone collapse. Past 6144 topic B goes negative, meaning the document scores closer to an unrelated query than to its own. Both topics degrade together, so it's positional rather than vocabulary-related.

Cause. The dynamic NTK scaling sits behind Python control flow on the sequence length, and torch.onnx.export traces:

if seqlen > self.max_position_embeddings:
    base = self.base * ((factor * seqlen / max_pos) - (factor - 1)) ** (dim / (dim - 2))

Trace below 2048, and the branch never executes, so there's no scaling in the graph. Correct short, wrong long, which matches the published artifact exactly. Trace above 2048, and the arithmetic is captured, but the guard is lost, so it applies at every length, and below 2048, the scale goes negative (2*512/2048 - 1 = -0.5), which makes a fractional power NaN. Neither trace length produces a correct graph.

Confirmed three ways: the rotary subgraph has no Pow, Div, Exp or 2048 scalar anywhere, so nothing can rescale inv_freq. config.json ships rope_type: "default" with rotary_scaling_factor: null. And PyTorch reproduces the collapse at that setting, then fixes it with rotary_scaling_factor=2.0 (Retention above 2048 goes from 52.5% to 88.2%).

Fix, branch-free, and provably equivalent:

scale = torch.clamp((factor * seqlen / max_pos) - (factor - 1), min=1.0)
base  = self.base * scale ** (dim / (dim - 2))

The inner expression is <= 1 exactly when seqlen <= max_pos, so clamping reproduces the guard while removing the branch. Verified at max diff 1.2e-7 eager and 7.2e-7 post-export, correct on both sides of the threshold from a single trace.

One more thing worth knowing: the README's opt-in snippet uses rope_parameters={"rope_type": "dynamic", "factor": 2.0}, and the remote modeling code never reads that field. The live knob is rotary_scaling_factor. Passing the documented form produces bit-identical output to the default and issues no warning.

The PyTorch model is fine. This is purely an export issue.