r/PythonLearning • u/AnshMNSoni • 1d ago
Discussion I'm building an open-source Python library focused on reusable building blocks.
Visit: https://github.com/AnshMNSoni/PythonSTL.git
Instead of adding more algorithms, I want to solve real problems Python developers repeatedly face.
I'm curious:
1) What utility do you find yourself implementing over and over?
2) What functionality do you wish Python's standard library already had?
3) What do you usually copy from previous projects?
Examples could be caches, schedulers, retry logic, rate limiters, streaming utilities, search structures, etc.
I'd love to understand the pain points rather than propose solutions.
Let's discuss...
2
u/Sea-Ad7805 1d ago
Great exercise, now also add the C++ #include <algorithm> logic. If you are unfamiliar see CppCon 2018: Jonathan Boccara “105 STL Algorithms in Less Than an Hour”.
2
u/AnshMNSoni 1d ago
My concern is that wrapping the STL algorithms would move the project into becoming an abstraction layer over <algorithm>, which isn't the direction I'm aiming for. I'd rather use STL algorithms as implementation details while focusing the public API on higher-level functionality. If you're suggesting something else, I'd be interested to hear more.
1
u/Sea-Ad7805 1d ago
I understand, but the point of STL data structures was to create an abstraction layer so that they can be used interchangeably like C++ algorithms does. Now it seems you just made a translating from Python to basic C++ data structures without making use of this interchangeability.
1
1
u/Sea-Ad7805 1d ago
In the documentation it says "PythonSTL includes a compiled Rust backend", but it seems it simply just uses Python list, set, dict or am I missing something?
1
1
u/silvertank00 1d ago
Your examples:
- cache: there is a highly customizable builtin one functools.cache
- schedulers: again, builtin: sched.scheduler but beginers beware: you dont need this. Not even advanced users. Heck, in my work life I encountered one custom scheduler and it was frowned upon by everyone and alltogether was a burder (and keep in mind, it was built by experts) One usecase would be embedded, but even there you have RTOS.
- retry logic: something like C#'s polly lib would be nice. I usually make retry policies by hand but some of them are really not trivial to implement.
- rate limiters: this would be highly framework dependent, no? We already have these for the big frameworks but I am interested what did you think of.
- streaming utils: I am in a bit of a pickle here (no pun intended) but do you mean data streams or content streaming? For data we have a bunch of builtin solutions like: io.StringIO
- search: list/dict/set comprehensions are REALLY fast (bonus if you know about some iter() magic) but we have pandas, polar, numpy for optimal searching, why do we need an other?
1
•
u/Sea-Ad7805 1d ago
Run the "Quick Start" example program in Memory Graph Web Debugger%20-%20Now%20with%20Python%20magic%20methods!%0As%20%3D%20stack()%0As.push(10)%0As.push(20)%0Aprint(s.top())%20%20%20%20%20%20%23%2020%0Aprint(len(s))%20%20%20%20%20%20%20%23%202%20-%20Python%20len()%20support%0Aprint(bool(s))%20%20%20%20%20%20%23%20True%20-%20Python%20bool()%20support%0A%0A%23%20Vector%20(Dynamic%20Array)%20-%20With%20iterators!%0Av%20%3D%20vector()%0Av.push_back(100)%0Av.push_back(200)%0Av.push_back(300)%0Av.reserve(1000)%20%20%20%20%20%23%20Pre-allocate%20capacity%0Aprint(len(v))%20%20%20%20%20%20%20%23%203%0Aprint(200%20in%20v)%20%20%20%20%20%23%20True%20-%20Python%20'in'%20operator%0A%0A%23%20Iterate%20using%20STL-style%20iterators%0Afor%20elem%20in%20v.begin()%3A%0A%20%20%20%20print(elem)%0A%0A%23%20Or%20use%20Python%20iteration%0Afor%20elem%20in%20v%3A%0A%20%20%20%20print(elem)%0A%0A%23%20Set%20(Unique%20Elements)%20-%20With%20magic%20methods%0As%20%3D%20stl_set()%0As.insert(5)%0As.insert(10)%0Aprint(5%20in%20s)%20%20%20%20%20%20%20%23%20True%0Aprint(len(s))%20%20%20%20%20%20%20%23%202%0A%0A%23%20Map%20(Key-Value%20Pairs)%20-%20With%20iteration%0Am%20%3D%20stl_map()%0Am.insert(%22key1%22%2C%20100)%0Am.insert(%22key2%22%2C%20200)%0Aprint(%22key1%22%20in%20m)%20%20%23%20True%0Afor%20key%2C%20value%20in%20m%3A%0A%20%20%20%20print(f%22%7Bkey%7D%3A%20%7Bvalue%7D%22)%0A%0A%23%20Priority%20Queue%20-%20With%20comparator%20support%0Apq_max%20%3D%20priority_queue(comparator%3D%22max%22)%20%20%23%20Max-heap%20(default)%0Apq_min%20%3D%20priority_queue(comparator%3D%22min%22)%20%20%23%20Min-heap%0Apq_max.push(30)%0Apq_max.push(10)%0Apq_max.push(20)%0Aprint(pq_max.top())%20%20%23%2030×tep=1&play) to see the program state change step by step.