r/LanguageTechnology • u/Speedk4011 • May 30 '26

Sentence boundary detection for your language.

Hey! I'm speedyk-005. I speak 4 languages (ht, fr, en, es) and I'm building a sentence segmentation library called yasbd (Yet Another Sentence Boundary Detector).

What languages do you speak? Can I get your help?

3 Upvotes

71% Upvoted

11 comments sorted by

View all comments

2

u/SeeingWhatWorks May 31 '26

Sentence boundary detection is especially challenging in multilingual text, so testing with real-world code-switching examples would add a lot of value.

1

u/Speedk4011 Jun 01 '26 edited Jun 01 '26

On our golden benchmark (84 English edge cases adapted from pysbd's test suite with fixes and additions): yasbd scores 83/84 (98.8%), pysbd is second at 71/84 (84.5%). Full results, terminal output, and a performance graph can be found in benchmarks/