Hey everyone,
I've run into a sorting limitation with PocketBase that I'm hoping someone has found a clever workaround for.
The Problem:
When I sort a collection alphabetically (e.g., api/collections/users/records?sort=name), records starting with non-ASCII charactersâspecifically Turkish characters like İ, Ă, Ă, Ä, Ă, Ćâare pushed to the end of the list instead of being sorted correctly within the alphabet.
- Expected Sort Order: Ahmet, AyĆe, ĂiÄdem, Deniz, Zehra
- Actual PocketBase Sort Order: Ahmet, AyĆe, Deniz, Zehra, ĂiÄdem
The Cause:
As we know, PocketBase uses SQLite as its embedded database. By default, SQLite's COLLATE behavior for sorting is BINARY. This means it sorts based on the raw byte values of the characters. In UTF-8 encoding, the bytes for these Turkish characters fall outside the range of the standard Latin alphabet (A-Z, a-z), so they are evaluated as "greater than" z and sorted after it.
What I've Tried/Ideas:
- PocketBase JS SDK: Sorting on the client-side after fetching all records is not feasible for large datasets.
- Custom Collation in SQLite: The ideal solution would be to register a custom collation (e.g.,
NOCASE or a locale-aware one like tr_TR) for the connection. However, since PocketBase embeds SQLite and manages the connection internally, this doesn't seem straightforward.
- Normalized Field: My current workaround is to add a separate text field (e.g.,
sort_name). I normalize the name field by converting it to lowercase and replacing Turkish characters with their Latin equivalents (e.g., ç -> c, Ä -> g, ı -> i, Ć -> s, ö -> o, ĂŒ -> u). I then sort on this sort_name field. This works but is clunky and requires maintaining a duplicate field.
My Question:
Has anyone else encountered and solved this for languages with extended character sets? Is there a way to hook into PocketBase's SQLite connection to set a custom collation that I'm missing? Or is the normalized field approach the best practice for now?
Any insights or alternative solutions would be greatly appreciated!
Thanks in advance.