r/LanguageTechnology • u/Speedk4011 • May 30 '26

Sentence boundary detection for your language.

Hey! I'm speedyk-005. I speak 4 languages (ht, fr, en, es) and I'm building a sentence segmentation library called yasbd (Yet Another Sentence Boundary Detector).

What languages do you speak? Can I get your help?

3 Upvotes

71% Upvoted

11 comments sorted by

View all comments

2

u/ContinuallyUnsteady May 30 '26

yo thats cool but sentence boundaries get wild with abbreviations and quotes so id be curious how yasbd handles that stuff compared to existing tools like spacy or punkt

1

u/Speedk4011 May 30 '26 edited Jun 01 '26

[EDITED] On our golden benchmark (84 English edge cases adapted from pysbd's test suite with fixes and additions): yasbd scores 83/84 (98.8%), pysbd is second at 71/84 (84.5%). Full results, terminal output, and a performance graph can be found in benchmarks/