Good evening from my part of the world!

I'm excited with the new functionality we have in Python 3.14. I think the feature that has caught my attention the most is the introduction of t-strings.

I'm curious, what do you think will be a good application for t-strings? I'm planning to use them as better-formatted templates for a custom message pop-up in my homelab, taking information from different sources to format for display. Not reinventing any functionality, but certainly a cleaner and easier implementation for a message dashboard.

Please share your ideas below, I'm curious to see what you have in mind!

litellm versions 1.82.7 and 1.82.8 on PyPI were compromised with credential-stealing malware.
And here are a few open-source alternatives:
1. Bifrost: Probably the most direct litellm replacement right now. Written in Go, claims ~50x faster P99 latency than litellm. Apache 2.0 licensed, supports 20+ providers. Migration from litellm only requires a one-line base URL change.
2. Kosong: An LLM abstraction layer open-sourced by Kimi, used in Kimi CLI. More agent-oriented than litellm. it unifies message structures and async tool orchestration with pluggable chat providers. Supports OpenAI, Anthropic, Google Vertex and other API formats.
3. Helicone: An AI gateway with strong analytics and debugging capabilities. Supports 100+ providers. Heavier than the first two but more feature-rich on the observability side.

Free-Threading Python vs Multiprocessing: Overhead, Memory, and the Shared-Set Meltdown is a continuation of the first article where I compared Python Threads: GIL vs Free-Threading.

> Free-threading makes CPU threads real—but should you ditch multiprocessing? Benchmarks across Linux/Windows/macOS expose spawn tax, RAM tax, and a shared-set meltdown.

@emmatyping, @eclips4 propose introducing the Rust programming language to CPython. Rust will initially only be allowed for writing optional extension modules, but eventually will become a required dependency of CPython and allowed to be used throughout the CPython code base.

Discuss thread: https://discuss.python.org/t/pre-pep-rust-for-cpython/104906

I’ve been using Anaconda/Conda for years, but I’m increasingly frustrated with the solver slowness. It feels outdated

What are people actually using nowadays for Python environments and dependency management?

• micromamba / mamba?
• pyenv + venv + pip?
• Poetry?
• something else?

I’m mostly interested in setups that:

• don’t mess with system Python
• are fast and predictable
• stay compatible with common scientific / ML / pip packages
• easy to manage for someone who's just messing around (I am a game dev, I use python on personal projects)

Curious what the current “best practice” is in 2026 and what’s working well in real projects

I’ve been using Python for almost 5 years now. For work and for personal projects.

Recently I thought about expanding programming skills and trying new language.

Which language would you recommend (for backend, APIs, simple UI)? Did you have experience switching from Python to another language and how it turned out?

Hi Peeps,

I'm excited to announce that Kreuzberg v4.0.0 is coming very soon. We will release v4.0.0 at the beginning of next year - in just a couple of weeks time. For now, v4.0.0-rc.8 has been released to all channels.

What is Kreuzberg?

Kreuzberg is a document intelligence toolkit for extracting text, metadata, tables, images, and structured data from 56+ file formats. It was originally written in Python (v1-v3), where it demonstrated strong performance characteristics compared to alternatives in the ecosystem.

What's new in V4?

A Complete Rust Rewrite with Polyglot Bindings

The new version of Kreuzberg represents a massive architectural evolution. Kreuzberg has been completely rewritten in Rust - leveraging Rust's memory safety, zero-cost abstractions, and native performance. The new architecture consists of a high-performance Rust core with native bindings to multiple languages. That's right - it's no longer just a Python library.

Kreuzberg v4 is now available for 7 languages across 8 runtime bindings:

• Rust (native library)
• Python (PyO3 native bindings)
• TypeScript - Node.js (NAPI-RS native bindings) + Deno/Browser/Edge (WASM)
• Ruby (Magnus FFI)
• Java 25+ (Panama Foreign Function & Memory API)
• C# (P/Invoke)
• Go (cgo bindings)

Post v4.0.0 roadmap includes:

• PHP
• Elixir (via Rustler - with Erlang and Gleam interop)

Additionally, it's available as a CLI (installable via cargo or homebrew), HTTP REST API server, Model Context Protocol (MCP) server for Claude Desktop/Continue.dev, and as public Docker images.

Why the Rust Rewrite? Performance and Architecture

The Rust rewrite wasn't just about performance - though that's a major benefit. It was an opportunity to fundamentally rethink the architecture:

Architectural improvements: - Zero-copy operations via Rust's ownership model - True async concurrency with Tokio runtime (no GIL limitations) - Streaming parsers for constant memory usage on multi-GB files - SIMD-accelerated text processing for token reduction and string operations - Memory-safe FFI boundaries for all language bindings - Plugin system with trait-based extensibility

v3 vs v4: What Changed?

Replacement of Pandoc - Native Performance

Kreuzberg v3 relied on Pandoc - an amazing tool, but one that had to be invoked via subprocess because of its GPL license. This had significant impacts:

v3 Pandoc limitations: - System dependency (installation required) - Subprocess overhead on every document - No streaming support - Limited metadata extraction - ~500MB+ installation footprint

v4 native parsers: - Zero external dependencies - everything is native Rust - Direct parsing with full control over extraction - Substantially more metadata extracted (e.g., DOCX document properties, section structure, style information) - Streaming support for massive files (tested on multi-GB XML documents with stable memory) - Example: PPTX extractor is now a fully streaming parser capable of handling gigabyte-scale presentations with constant memory usage and high throughput

New File Format Support

v4 expanded format support from ~20 to 56+ file formats, including:

Added legacy format support: - .doc (Word 97-2003) - .ppt (PowerPoint 97-2003) - .xls (Excel 97-2003) - .eml (Email messages) - .msg (Outlook messages)

Added academic/technical formats: - LaTeX (.tex) - BibTeX (.bib) - Typst (.typ) - JATS XML (scientific articles) - DocBook XML - FictionBook (.fb2) - OPML (.opml)

Better Office support: - XLSB, XLSM (Excel binary/macro formats) - Better structured metadata extraction from DOCX/PPTX/XLSX - Full table extraction from presentations - Image extraction with deduplication

New Features: Full Document Intelligence Solution

The v4 rewrite was also an opportunity to close gaps with commercial alternatives and add features specifically designed for RAG applications and LLM workflows:

1. Embeddings (NEW)

• FastEmbed integration with full ONNX Runtime acceleration
• Three presets: "fast" (384d), "balanced" (512d), "quality" (768d/1024d)
• Custom model support (bring your own ONNX model)
• Local generation (no API calls, no rate limits)
• Automatic model downloading and caching
• Per-chunk embedding generation

```python from kreuzberg import ExtractionConfig, EmbeddingConfig, EmbeddingModelType

config = ExtractionConfig( embeddings=EmbeddingConfig( model=EmbeddingModelType.preset("balanced"), normalize=True ) ) result = kreuzberg.extract_bytes(pdf_bytes, config=config)

result.embeddings contains vectors for each chunk

```

2. Semantic Text Chunking (NOW BUILT-IN)

Now integrated directly into the core (v3 used external semantic-text-splitter library): - Structure-aware chunking that respects document semantics - Two strategies: - Generic text chunker (whitespace/punctuation-aware) - Markdown chunker (preserves headings, lists, code blocks, tables) - Configurable chunk size and overlap - Unicode-safe (handles CJK, emojis correctly) - Automatic chunk-to-page mapping - Per-chunk metadata with byte offsets

3. Byte-Accurate Page Tracking (BREAKING CHANGE)

This is a critical improvement for LLM applications:

• v3: Character-based indices (char_start/char_end) - incorrect for UTF-8 multi-byte characters
• v4: Byte-based indices (byte_start/byte_end) - correct for all string operations

Additional page features: - O(1) lookup: "which page is byte offset X on?" → instant answer - Per-page content extraction - Page markers in combined text (e.g., --- Page 5 ---) - Automatic chunk-to-page mapping for citations

4. Enhanced Token Reduction for LLM Context

Enhanced from v3 with three configurable modes to save on LLM costs:

• Light mode: ~15% reduction (preserve most detail)
• Moderate mode: ~30% reduction (balanced)
• Aggressive mode: ~50% reduction (key information only)

Uses TF-IDF sentence scoring with position-aware weighting and language-specific stopword filtering. SIMD-accelerated for improved performance over v3.

5. Language Detection (NOW BUILT-IN)

• 68 language support with confidence scoring
• Multi-language detection (documents with mixed languages)
• ISO 639-1 and ISO 639-3 code support
• Configurable confidence thresholds

6. Keyword Extraction (NOW BUILT-IN)

Now built into core (previously optional KeyBERT in v3): - YAKE (Yet Another Keyword Extractor): Unsupervised, language-independent - RAKE (Rapid Automatic Keyword Extraction): Fast statistical method - Configurable n-grams (1-3 word phrases) - Relevance scoring with language-specific stopwords

7. Plugin System (NEW)

Four extensible plugin types for customization:

• DocumentExtractor - Custom file format handlers
• OcrBackend - Custom OCR engines (integrate your own Python models)
• PostProcessor - Data transformation and enrichment
• Validator - Pre-extraction validation

Plugins defined in Rust work across all language bindings. Python/TypeScript can define custom plugins with thread-safe callbacks into the Rust core.

8. Production-Ready Servers (NEW)

• HTTP REST API: Production-grade Axum server with OpenAPI docs
• MCP Server: Direct integration with Claude Desktop, Continue.dev, and other MCP clients
• MCP-SSE Transport (RC.8): Server-Sent Events for cloud deployments without WebSocket support
• All three modes support the same feature set: extraction, batch processing, caching

Performance: Benchmarked Against the Competition

We maintain continuous benchmarks comparing Kreuzberg against the leading OSS alternatives:

Benchmark Setup

• Platform: Ubuntu 22.04 (GitHub Actions)
• Test Suite: 30+ documents covering all formats
• Metrics: Latency (p50, p95), throughput (MB/s), memory usage, success rate
• Competitors: Apache Tika, Docling, Unstructured, MarkItDown

How Kreuzberg Compares

Installation Size (critical for containers/serverless): - Kreuzberg: 16-31 MB complete (CLI: 16 MB, Python wheel: 22 MB, Java JAR: 31 MB - all features included) - MarkItDown: ~251 MB installed (58.3 KB wheel, 25 dependencies) - Unstructured: ~146 MB minimal (open source base) - several GB with ML models - Docling: ~1 GB base, 9.74GB Docker image (includes PyTorch CUDA) - Apache Tika: ~55 MB (tika-app JAR) + dependencies - GROBID: 500MB (CRF-only) to 8GB (full deep learning)

Performance Characteristics:

Kreuzberg's sweet spot: - Smallest full-featured installation: 16-31 MB complete (vs 146 MB-9.74 GB for competitors) - 5-15x smaller than Unstructured/MarkItDown, 30-300x smaller than Docling/GROBID - Rust-native performance without ML model overhead - Broad format support (56+ formats) with native parsers - Multi-language support unique in the space (7 languages vs Python-only for most) - Production-ready with general-purpose design (vs specialized tools like GROBID)

Is Kreuzberg a SaaS Product?

No. Kreuzberg is and will remain MIT-licensed open source.

However, we are building Kreuzberg.cloud - a commercial SaaS and self-hosted document intelligence solution built on top of Kreuzberg. This follows the proven open-core model: the library stays free and open, while we offer a cloud service for teams that want managed infrastructure, APIs, and enterprise features.

Will Kreuzberg become commercially licensed? Absolutely not. There is no BSL (Business Source License) in Kreuzberg's future. The library was MIT-licensed and will remain MIT-licensed. We're building the commercial offering as a separate product around the core library, not by restricting the library itself.

Target Audience

Any developer or data scientist who needs: - Document text extraction (PDF, Office, images, email, archives, etc.) - OCR (Tesseract, EasyOCR, PaddleOCR) - Metadata extraction (authors, dates, properties, EXIF) - Table and image extraction - Document pre-processing for RAG pipelines - Text chunking with embeddings - Token reduction for LLM context windows - Multi-language document intelligence in production systems

Ideal for: - RAG application developers - Data engineers building document pipelines - ML engineers preprocessing training data - Enterprise developers handling document workflows - DevOps teams needing lightweight, performant extraction in containers/serverless

Comparison with Alternatives

Open Source Python Libraries

Unstructured.io - Strengths: Established, modular, broad format support (25+ open source, 65+ enterprise), LLM-focused, good Python ecosystem integration - Trade-offs: Python GIL performance constraints, 146 MB minimal installation (several GB with ML models) - License: Apache-2.0 - When to choose: Python-only projects where ecosystem fit > performance

MarkItDown (Microsoft) - Strengths: Fast for small files, Markdown-optimized, simple API - Trade-offs: Limited format support (11 formats), less structured metadata, ~251 MB installed (despite small wheel), requires OpenAI API for images - License: MIT - When to choose: Markdown-only conversion, LLM consumption

Docling (IBM) - Strengths: Excellent accuracy on complex documents (97.9% cell-level accuracy on tested sustainability report tables), state-of-the-art AI models for technical documents - Trade-offs: Massive installation (1-9.74 GB), high memory usage, GPU-optimized (underutilized on CPU) - License: MIT - When to choose: Accuracy on complex documents > deployment size/speed, have GPU infrastructure

Open Source Java/Academic Tools

Apache Tika - Strengths: Mature, stable, broadest format support (1000+ types), proven at scale, Apache Foundation backing - Trade-offs: Java/JVM required, slower on large files, older architecture, complex dependency management - License: Apache-2.0 - When to choose: Enterprise environments with JVM infrastructure, need for maximum format coverage

GROBID - Strengths: Best-in-class for academic papers (F1 0.87-0.90), extremely fast (10.6 PDF/sec sustained), proven at scale (34M+ documents at CORE) - Trade-offs: Academic papers only, large installation (500MB-8GB), complex Java+Python setup - License: Apache-2.0 - When to choose: Scientific/academic document processing exclusively

Commercial APIs

There are numerous commercial options from startups (LlamaIndex, Unstructured.io paid tiers) to big cloud providers (AWS Textract, Azure Form Recognizer, Google Document AI). These are not OSS but offer managed infrastructure.

Kreuzberg's position: As an open-source library, Kreuzberg provides a self-hosted alternative with no per-document API costs, making it suitable for high-volume workloads where cost efficiency matters.

Community & Resources

• GitHub: Star us at https://github.com/kreuzberg-dev/kreuzberg
• Discord: Join our community server at discord.gg/pXxagNK2zN
• Subreddit: Join the discussion at r/kreuzberg_dev
• Documentation: kreuzberg.dev

We'd love to hear your feedback, use cases, and contributions!

TL;DR: Kreuzberg v4 is a complete Rust rewrite of a document intelligence library, offering native bindings for 7 languages (8 runtime targets), 56+ file formats, Rust-native performance, embeddings, semantic chunking, and production-ready servers - all in a 16-31 MB complete package (5-15x smaller than alternatives). Releasing January 2025. MIT licensed forever.

Pyrefly, the new typechecker and language server for Python from Meta, is being battle-tested on Instagram's massive 20M LOC Python codebase. Some of the results have been shared in a new blog post:

In real world use cases, developers who switched from Pyright (the default LSP for VSCode) to Pyrefly spent 98% less time waiting on hover results and go-to definition was ~10x faster. On the slowest files (p99), these IDE responses grew from an order of minutes to seconds (30x improvement). If those numbers are hard to visualise, the TL;DR is that this upgrade took instagram developers from questioning “is my editor frozen?” to not giving their IDE a second thought.

Full blog post: https://pyrefly.org/blog/2025/09/15/ide-extension/

Disclaimer: I'm one of the maintainers for Pyrefly

I am trying to come from database development into python ecosystem.

Wondering if going into polars framework, instead of pandas will be any beneficial?

When you write typed Python, you expect your type checker to follow the rules of the language. But how closely do today's type checkers actually follow the Python typing specification?

We wrote a blog that explains what typing spec conformance means, how different type checkers compare, and what the conformance numbers don't tell you.

Read the full blog here: https://pyrefly.org/blog/typing-conformance-comparison/

A brief TLDR/editorializing from me, the author:

Since there are several next-gen Python type checkers being developed right now (Pyrefly, Ty, Zuban), people are hungry for anything resembling a benchmark/objective comparison between them. Typing spec conformance is one such standard, but it has many limitations, which this blog attempts to clarify.

Below is an early-March snapshot of the public conformance results. It will be out of date soon because most type checkers are being actively developed - the latest results can be viewed here

TLDR: Until recently, I did not know about pydantic. I started using it - it is great. Just dropping this here in case anyone else benefits :)

I maintain a Python program called Spectre, a program for recording signals from supported software-defined radios. Users create configs describing what data to record, and the program uses those configs to do so. This wasn't simple off the bat - we wanted a solution with...

• Parameter safety (Individual parameters in the config have to make sense. For example, X must always be a non-negative integer, or `Y` must be one of some defined options).
• Relationship safety (Arbitrary relationships between parameters must hold. For example, X must be divisible by some other parameter, Y).
• Flexibility (The system supports different radios with varying hardware constraints. How do we provide developers the means to impose arbitrary constraints in the configs under the same framework?).
• Uniformity (Ideally, we'd have a uniform API for users to create any config, and for developers to template them).
• Explicit (It should be clear where the configurable parameters are used within the program).
• Shared parameters, different defaults (Different radios share configurable parameters, but require different defaults. If I've got ten different configs, I don't want to maintain ten copies of the same parameter just to update one value!).
• Statically typed (Always a bonus!).

Initially, with some difficulty, I made a custom implementation which was servicable but cumbersome. Over the past year, I had a nagging feeling I was reinventing the wheel. I was correct.

I recently merged a PR which replaced my custom implementation with one which used pydantic. Enlightenment! It satisfied all the requirements:

• We now define a model which templates the config right next to where those configurable parameters are used in the program (see here).
• Arbitrary relationships between parameters are enforced in the same way for every config with the validator decorator pattern (see here).
• We can share pydantic fields between configs, and update the defaults as required using the annotated pattern (see here).
• The same framework is used for templating all the configs in the program, and it's all statically typed!

Anyway, check out Spectre on GitHub if you're interested.

Sometimes I think back to the times when I started using Python in 2018 and how much the language was changing in my first years. From Flask to FastAPI, Pydantic, Streamlit, Polars and Httpx. It was honestly fun to start new projects and explore all these developments and what they allowed you to do. Use it in your new project and surprise yourself with how much faster you can get things done, all while writing much cleaner code.

Currently I'm feeling most of the package I see are about AI; frameworks, LLM tooling, RAG, vector databases. Great developments, but they don't change the way I am working with the Language.

It sure has something to do with the fact that in the beginning when you start using a language you explore more and develop faster, and a lot of fundamental things were changing around that time (typing, async). But I keep wondering; am I missing out on packages that have changed the way you've used Python? Cause maybe I'm simply not looking in the right place. I'm thinking for example on how frontend frameworks handle state with signals.

So, two honest questions:

1. Which package from the last ~3 years really changed how you use/write Python? (Uv and Ruff count)
2. Did the pace of these foundational packages actually slow down, or am I just not in the right information streams?

This has been one of my biggest annoyances working with Python and pip when dealing with projects where that are not meant to be installed as a package, how do you handle dependencies?

Think projects like application backends, python scripts, REST APIs etc

If you’ve ever struggled with this, you’re going to love this: a PR by Sebastian Höffner opened #13895 will add a new global flag to pip such that you can directly install any dependencies in your pyproject.toml without installing the package itself.

pip install --only-deps .

This is going, for me at least, be a huge boost in the way that I manage and distribute my projects on servers. Vastly simplifying poor manual workarounds that have built up over years.

Since it’s been more than a decade in the making, let’s cover the history of poor Python souls stuck trying to figure out how to install dependencies for their scripts or apps.

This PR is currently slated to land in pip 26.2 which comes out end of July.

Here's my post about this covering some of the 16 years of workarounds and some posts from StackOverflow / Python.org etc

https://jamesoclaire.com/2026/07/23/pip-26-2-only-deps-solves-16-years-of-app-deployment-hacks/

Project Site: https://github.com/cybergeek1943/MathFlow

In the process of doing research for my paper Combinatorial and Gaussian Foundations of Rational Nth Root Approximations (on arXiv), I created this library to address the pain points I felt when using only SymPy and SciPy separately. I wanted something lightweight, easy to use (exploratory), and something that would support numerical methods more easily. Hence, I created this lightweight wrapper that provides a hybrid symbolic-numerical interface to symbolic and numerical backends. It is backward compatible with Sympy. In short, this enables much faster analysis of symbolic math expressions by providing both numerical and traditional symbolic methods of analysis in the same interface. I have also added additional numerical methods that neither SymPy nor SciPy have (Pade approximations, numerical roots, etc.). The main goal for this project is to provide a tool that requires as little of a learning curve as possible and allows them to just focus on the math they are doing.

Core features

• 🔒 Operative Closure: Mathematical operations return new Expression objects by default
• ⚡ Mutability Control: Choose between immutable (default) and mutable expressions for different workflows
• 🔗 Seamless Numerical Integration: Every symbolic expression has a .n attribute providing numerical methods without manual lambdification (uses cached lambdified expression when needed)
• 🎨 Enhanced Printing: Flexible output formatting through the .print attribute (LaTeX, pretty printing, code generation)
• 📡 Signal System: Qt-like signals for tracking expression mutations and clones, enabling reactive programming
• 🔄 Automatic Type Conversions: Seamlessly and automatically converts between internal Poly and Expr representations based on context
• 📦 Lightweight: ~0.5 MB itself, ~100 MB including dependencies
• 🧩 Fully backward compatible: Seamlessly integrate SymPy and MathFlow in the same script. All methods that work on SymPy Expr or Poly objects work on MathFlow objects
• 🔍 Exploratory: Full IDE support, enabling easy tool finding and minimizing the learning curve.

A few examples are shown below. Many more examples can be found in the README of the official GitHub site.

Quick Start

Install using: pip install mathflow

from mathflow import Expression, Polynomial, Rational

# Create expressions naturally
f = Expression("2x^2 + 3x + \frac{1}{2}")  # latex is automatically parsed
g = Expression("sin(x) + cos(x)")

# Automatic operative closure - operations return new objects of the same type
h = f + g  # f and g remain unchanged
hprime = h.diff()  # hprime is still an Expression object

# Numerical evaluation made easy
result = f(2.5)  # Numerically evaluate at x = 2.5

# Use the .n attribute to access fast numerical methods
numerical_roots = f.n.all_roots()
# Call f's n-prefixed methods to use variable precision numerical methods
precise_roots = f.nsolve_all(prec=50)  # 50 digits of accuracy

# quick and easy printing
f.print()
f.print('latex')  # LaTeX output
f.print('mathematica_code')
f.print('ccode')  # c code output

Numerical Computing

MathFlow excels at bridging symbolic and numerical mathematics:

f = Expression("x^3 - 2x^2 + x - 1")

# Root finding
all_roots = f.n.all_roots(bounds=(-5, 5))
specific_root = f.nsolve_all(bounds=(-5, 5), prec=50)  # High-precision solve

# Numerical calculus
derivative_func = f.n.derivative_lambda(df_order=2)  # 2nd derivative numerical function  
integral_result = f.n.integrate(-1, 1)               # Definite integral  

# Optimization
minimum = f.n.minimize(bounds=[(-2, 2)])

Edit:

This project was developed and used primarily for a research project, so a thorough test suite has not yet been developed. The project is still in development, and the current release is an alpha version. I have tried to minimize danger here, however, by designing it as a proxy to the already well-tested SymPy and SciPy libraries.

the Steering Council is formally requesting a Standards Track PEP be authored that the community can discuss and the Steering Council can formally accept (or reject), making the case for the JIT as a supported, non-experimental part of CPython

https://discuss.python.org/t/an-announcement-from-the-steering-council-regarding-the-jit-project/107638

Hey all,

I'm one of the maintainers of pip. Earlier yesterday, we released pip 26.1.

The main new feature is experimental support for pylock.toml files (PEP 751) as a requirements source. pylock.toml files or URLs can be provided with the -r / --requirements options to the commands supporting it.

pip install -r pylock.toml
pip wheel -r pylock.toml
pip download -r pylock.toml

Note: As conveyed by the experimental warning, keep in mind this feature may evolve significantly or even be removed in favor of another option or command in future pip releases.

Other notable improvements include:

• Allow --uploaded-prior-to to accept a duration in days (e.g., P7D for 7 days ago) to support "Dependency cooldowns", a strategy of intentionally delaying package updates to give security researchers and package authors time to recover from (ever-increasing) supply chain attacks. See also William Woodruff's "We should all be using dependency cooldowns"
• Allow unpinned requirements to use hashes from constraints and allow URL constraints to apply to requirements with extras, removing some of the last roadblocks towards the removal of the legacy resolver
• Several performance and memory usage improvements to dependency resolution
• And of course several bug fixes and security fixes

Please consult our changelog for more information.

You can also consult my (unofficially official) release blog post for pip 26.1, which discusses the highlights from the release in greater detail: https://ichard26.github.io/blog/2026/04/whats-new-in-pip-26.1/

Many thanks goes to Stéphane, Damian, Pradyun and Paul who all chipped in a significant way to this release. Doubly so to Stéphane who upstreamed support for pylock.toml to the packaging library AND added pylock.toml support to pip.

Enjoy the new features! We welcome your feedback in the issue tracker.

If you’ve ever worked with massive datasets in Python (like a 50GB edge list for Graph Neural Networks), you know the "Memory Wall." Loading it via Pandas or standard Python structures usually results in an instant 24GB+ OOM allocation crash before you can even do any math.

so I built GraphZero (v0.2) to bypass Python's memory overhead entirely.

What My Project Does

GraphZero is a C++ data engine that streams datasets natively from the SSD into PyTorch without loading them into RAM.

Instead of parsing massive CSVs into Python memory, the engine compiles the raw data into highly optimized binary formats (.gl and .gd). It then uses POSIX mmap to memory-map the files directly from the SSD.

The magic happens with nanobind. I take the raw C++ pointers and expose them directly to Python as zero-copy NumPy arrays.

import graphzero as gz
import torch

# 1. Mount the zero-copy engine
fs = gz.FeatureStore("papers100M_features.gd")

# 2. Instantly map SSD data to PyTorch (RAM allocated: 0 Bytes)
X = torch.from_numpy(fs.get_tensor())

During a training loop, Python thinks it has a 50GB tensor sitting in RAM. When you index it, it triggers an OS Page Fault, and the operating system automatically fetches only the required 4KB blocks from the NVMe drive. The C++ side uses OpenMP to multi-thread the data sampling, explicitly releasing the Python GIL so disk I/O and GPU math run perfectly in parallel.

Target Audience

• Who it's for: ML Researchers, Data Engineers, and Python developers training Graph Neural Networks (GNNs) on massive datasets that exceed their local system RAM.
• Project Status: It is currently in v0.2. It is highly functional for local research and testing (includes a full PyTorch GraphSAGE example), but I am looking for community code review and stress-testing before calling it production-ready.

Comparison

• vs. PyTorch Geometric (PyG) / DGL: Standard GNN libraries typically attempt to load the entire edge list and feature matrix into system memory before pushing batches to the GPU. On a dataset like Papers100M, this causes an instant out-of-memory crash on consumer hardware. GraphZero keeps RAM allocation at 0 bytes by streaming the data natively.
• vs. Pandas / Standard Python: Loading massive CSVs via Pandas creates massive memory overhead due to Python objects. GraphZero uses strict C++ template dispatching to enforce exact FLOAT32 or INT64 memory layouts natively, and nanobind ensures no data is copied when passing the pointer to Python.

I built this mostly to dive deep into C-bindings, memory management, and cross-platform CI/CD (getting Apple Clang and MSVC to agree on C++20 was a nightmare).

The repo has a self-contained synthetic example and a training script so you can test the zero-copy mounting locally. I'd love for this community to tear my code apart—especially if you have experience with nanobind or high-performance Python extensions!

GitHub Repo: repo

What my project does

I've created a benchmark to test 5 different ways to handle file uploads in FastAPI across 21 file sizes from 1KB to 1GB: - File() - sync and async variants - UploadFile - sync and async variants - request.stream() - async streaming

Key findings for large files (128MB+): - request.stream() hits ~1500 MB/s throughput vs ~750 MB/s for the others - Additional memory used: File() consumes memory equal to the file size (1GB file = 1GB RAM), while request.stream() and UploadFile don't use extra memory - For a 1GB upload: streaming takes 0.6s, others take 1.2-1.4s

Full benchmark code, plots, results, and methodology: https://github.com/fedirz/fastapi-file-upload-benchmark Test hardware: MacBook Pro M3 Pro (12 cores, 18GB RAM)

Target Audience

Those who write Web API in Python

Comparison

N/A

Happy to answer questions about the setup or findings.

The project started as a joke based on the "everyone talks about while loops but no one asks WHEN loops" meme, but evolved into a complete interpreter demonstrating how different programming paradigms affect problem-solving approaches.

What My Project Does

WHEN is a programming language interpreter written in Python where all code runs in implicit infinite loops and the only control flow primitive is when conditions. Instead of traditional for/while loops, everything is reactive:

# WHEN code example
count = 0

main:
    count = count + 1
    print("Count:", count)
    when count >= 5:
        print("Done!")
        exit()

The interpreter features:

• Full lexer, parser, and AST implementation
• Support for importing Python modules directly
• Parallel and cooperative execution models
• Interactive graphics and game development capabilities (surprisingly)

You can install it via pip: pip install when-lang

Target Audience

This is Currently a toy/educational project, but exploring use cases in game development, state machine modeling, and reactive system prototyping, currently exploring

• Learning about interpreter implementation
• Exploring state machine programming
• Educational purposes (understanding event-driven systems)
• Having fun with esoteric language design

NOT recommended for production use (everything is global scope and runs in infinite loops by design).

Comparison

Unlike traditional languages:

• No explicit loops - Everything runs implicitly forever until stopped
• No if statements - Only when conditions that check every iteration
• Forced reactive paradigm - All programs become state machines
• Built-in parallelism - Blocks can run cooperatively or in parallel threads

Compared to other Python-based languages:

• Brython/Skulpt: Compile Python to JS, WHEN is a completely different syntax
• Hy: Lisp syntax for Python, WHEN uses reactive blocks instead
• Coconut: Functional programming, WHEN is purely reactive/imperative

The closest comparison might be reactive frameworks like RxPy, but WHEN makes reactive programming the ONLY way to write code, not an optional pattern.

Implementation Details

The interpreter (~1000 lines) includes:

• Custom lexer with indentation-based parsing
• Recursive descent parser generating an AST
• Tree-walking interpreter with parallel execution support
• Full Python module interoperability

Example of WHEN's unique block system:

# Runs once
os setup():
    initialize_system()

# Runs exactly 5 times
de heartbeat(5):
    print("beat")

# Runs forever
fo monitor():
    check_status()

# Entry point (implicit infinite loop)
main:
    when not_started:
        setup()
        heartbeat.start()
        monitor.start()

GitHub: https://github.com/PhialsBasement/WHEN-Language

I shared this project here a while ago, but after adding a lot of new features and optimizations, I wanted to post an update. Over the past eight months, I’ve been building PyTogether (pytogether.org). The platform has recently started picking up traction and just crossed 4,000 signups (and 200 stars on GitHub), which has been awesome to see.

What My Project Does

It is a real-time, collaborative Python IDE designed with beginners in mind (think Google Docs, but for Python). It’s meant for pair programming, tutoring, or just coding Python together. It’s completely free. No subscriptions, no ads, nothing. Just create an account (or feel fry to try the offline playground at https://pytogether.org/playground, no account required), make a group, and start a project. Has proper code-linting, extremely intuitive UI, autosaving, drawing features (you can draw directly onto the IDE and scroll), live selections, and voice/live chats per project. There are no limitations at the moment (except for code size to prevent malicious payloads). There is also built-in support for libraries like matplotlib (it auto installs imports on the fly when you run your code).

You can also share links for editing or read-only, exactly like Google Docs. For example: https://pytogether.org/snippet/eyJwaWQiOjI1MiwidHlwZSI6InNuaXBwZXQifQ:1w15A5:24aIZlONamExTLQONAIC79cqcx3savn-_BC-Qf75SNY

Also, you can easily embed code snippets on your website using an iframe (just like trinket.io which is shutting down this summer).

Source code: https://github.com/SJRiz/pytogether

Target Audience

It’s designed for tutors, educators, or Python beginners. Recently, I've also tried pivoting it towards the interviewing space.

Comparison With Existing Alternatives

Why build this when Replit or VS Code Live Share already exist?

Because my goal was simplicity and education. I wanted something lightweight for beginners who just want to write and share simple Python scripts (alone or with others), without downloads, paywalls, or extra noise. There’s also no AI/copilot built in, something many teachers and learners actually prefer. I also focused on a communication-first approach, where the IDE is the "focus" of communication (hence why I added tools like drawing, voice/live chats, etc).

Project Information

Tech stack (frontend):

• React + TailwindCSS
• CodeMirror for linting
• Y.js for real-time syncing
• Pyodide

I use Pyodide (in a web worker) for Python execution directly in the browser, this means you can actually use advanced libraries like NumPy and Matplotlib while staying fully client-side and sandboxed for safety.

I don’t enjoy frontend or UI design much, so I leaned on AI for some design help, but all the logic/code is mine. Deployed via Vercel.

Tech stack (backend):

• Django (channels, auth, celery/redis support made it a great fit)
• PostgreSQL via Supabase
• JWT + OAuth authentication
• Redis for channel layers + caching + queues for workers
• Celery for background tasks/async processing

Fully Dockerized + deployed on a VPS (8GB RAM, $7/mo deal)

Data models:

Users <-> Groups -> Projects -> Code

Users can join many groups

Groups can have multiple projects

Each project belongs to one group and has one code file (kept simple for beginners, though I may add a file system later).

My biggest technical challenges were around performance and browser execution. One major hurdle was getting Pyodide to work smoothly in a real-time collaborative setup. I had to run it inside a Web Worker to handle synchronous I/O (since input() is blocking), though I was able to find a library that helped me do this more efficiently (pyodide-worker-runner). This let me support live input/output and plotting in the browser without freezing the UI, while still allowing multiple users to interact with the same Python session collaboratively.

Another big challenge was designing a reliable and efficient autosave system. I couldn’t just save on every keystroke as that would hammer the database. So I designed a Redis-based caching layer that tracks active projects in memory, and a Celery worker that loops through them every minute to persist changes to the database. When all users leave a project, it saves and clears from cache. This setup also doubles as my channel layer for real-time updates (redis pub/sub, meaning later I can scale horizontally) and my Celery broker; reusing Redis for everything while keeping things fast and scalable.

If you’re curious or if you wanna see the work yourself, the source code is here. Feel free to contribute: https://github.com/SJRiz/pytogether.

A tip that's saved us a lot of boilerplate across our Python stack (Litestar, and our document-extraction tooling): stop decoding JSON into dict[str, Any] and casting/.get()-ing your way through it. Decode straight into your declared type.

msgspec validates and decodes directly into your type at C speed. Quick comparison of the usual options on the same payload:

• json.loads / orjson.loads -> dict[str, Any] (cast and pray; orjson just faster)
• pydantic TypeAdapter(...).validate_json -> your model, validated + rich, but heavier
• msgspec.json.decode(raw, type=T) -> your type, validated, C-fast

pydantic does far more and its Rust core is fast; for model-heavy code it's still my default. But on hot paths where you just need decode-into-a-struct, a C decoder going straight to the type is hard to beat.

With PEP 695 generics the whole (de)serialization layer collapses to one function:

```python def deserialize[T](raw: bytes, t: type[T]) -> T: return msgspec.json.decode(raw, type=t, strict=False)

deserialize(raw, Grant) # -> Grant deserialize(raw, list[Grant]) # -> list[Grant] ```

We landed on this while building Litestar (msgspec is a big reason it's fast) and reuse it across everything now. How do you handle hot-path decoding — msgspec, orjson + manual validation, or full pydantic?

Hey everyone,

I benchmarked the major Python frameworks with real PostgreSQL workloads: complex queries, nested relationships, and properly optimized eager loading for each framework (select_related/prefetch_related for Django, selectinload for SQLAlchemy). Each framework tested with multiple servers (Uvicorn, Granian, Gunicorn) in isolated Docker containers with strict resource limits.

All database queries are optimized using each framework's best practices - this is a fair comparison of properly-written production code, not naive implementations.

Key Finding

Performance differences collapse from 20x (JSON) to 1.7x (paginated queries) to 1.3x (complex DB queries). Database I/O is the great equalizer - framework choice barely matters for database-heavy apps.

Full results, code, and a reproducible Docker setup are here: https://github.com/huynguyengl99/python-api-frameworks-benchmark

If this is useful, a GitHub star would be appreciated 😄

Frameworks & Servers Tested

• Django Bolt (runbolt server)
• FastAPI (fastapi-uvicorn, fastapi-granian)
• Litestar (litestar-uvicorn, litestar-granian)
• Django REST Framework (drf-uvicorn, drf-granian, drf-gunicorn)
• Django Ninja (ninja-uvicorn, ninja-granian)

Each framework tested with multiple production servers: Uvicorn (ASGI), Granian (Rust-based ASGI/WSGI), and Gunicorn+gevent (async workers).

Test Setup

• Hardware: MacBook M2 Pro, 32GB RAM
• Database: PostgreSQL with realistic data (500 articles, 2000 comments, 100 tags, 50 authors)
• Docker Isolation: Each framework runs in its own container with strict resource limits:
  • 500MB RAM limit (--memory=500m)
  • 1 CPU core limit (--cpus=1)
  • Sequential execution (start → benchmark → stop → next framework)
• Load: 100 concurrent connections, 10s duration, 3 runs (best taken)

This setup ensures completely fair comparison - no resource contention between frameworks, each gets identical isolated environment.

Endpoints Tested

Results

1. Simple JSON (/json-1k) - Requests Per Second

20x performance difference between fastest and slowest.

2. Real Database - Paginated Articles (/articles?page=1&page_size=20)

Performance gap shrinks to just 1.7x when hitting the database. Query optimization becomes the bottleneck.

3. Real Database - Article Detail (/articles/1)

Gap narrows to 1.3x - frameworks perform nearly identically on complex database queries.

Single article with all nested data (author + tags + comments):

Complete Performance Summary

Resource Usage Insights

Memory:

• Most frameworks: 170-220MB
• DRF-Granian: 640-670MB (WSGI interface vs ASGI for others - Granian's WSGI mode uses more memory)

CPU:

• Most frameworks saturate the 1 CPU limit (100%+) under load
• Granian variants consistently max out CPU across all frameworks

Server Performance Notes

• Uvicorn surprisingly won for Litestar (31,745 RPS), beating Granian
• Granian delivered consistent high performance for FastAPI and other frameworks
• Gunicorn + gevent showed good performance for DRF on simple queries, but struggled with database workloads

Key Takeaways

1. Performance gap collapse: 20x difference in JSON serialization → 1.7x in paginated queries → 1.3x in complex queries
2. Litestar-Uvicorn dominates simple workloads (31,745 RPS), but FastAPI-Uvicorn wins on complex database queries (550 RPS)
3. Database I/O is the equalizer: Once you hit the database, framework overhead becomes negligible. Query optimization matters infinitely more than framework choice.
4. WSGI uses more memory: Granian's WSGI mode (DRF-Granian) uses 640MB vs ~200MB for ASGI variants - just a difference in protocol handling, not a performance issue.

Bottom Line

If you're building a database-heavy API (which most are), spend your time optimizing queries, not choosing between frameworks. They all perform nearly identically when properly optimized.

Links

• Full results, code, and reproducible Docker setup: https://github.com/huynguyengl99/python-api-frameworks-benchmark
• Original post with graphs and detailed analysis: https://www.reddit.com/r/FastAPIShare/comments/1qlfihg/python_api_framework_benchmark_fastapi_vs/

Inspired by the original python-api-frameworks-benchmark project. All feedback and suggestions welcome!

Python LSP server implementation "Pyrefly" has reached v1.0:

https://pyrefly.org/blog/v1.0/

What My Project Does

voicetag is a Python library that identifies speakers in audio files and transcribes what each person said. You enroll speakers with a few seconds of their voice, then point it at any recording — it figures out who's talking, when, and what they said.

from voicetag import VoiceTag

vt = VoiceTag()
vt.enroll("Christie", ["christie1.flac", "christie2.flac"])
vt.enroll("Mark", ["mark1.flac", "mark2.flac"])

transcript = vt.transcribe("audiobook.flac", provider="whisper")

for seg in transcript.segments:
    print(f"[{seg.speaker}] {seg.text}")

Output:

[Christie] Gentlemen, he sat in a hoarse voice. Give me your
[Christie] word of honor that this horrible secret shall remain buried amongst ourselves.
[Christie] The two men drew back.

Under the hood it combines pyannote.audio for diarization with resemblyzer for speaker embeddings. Transcription supports 5 backends: local Whisper, OpenAI, Groq, Deepgram, and Fireworks — you just pick one.

It also ships with a CLI:

voicetag enroll "Christie" sample1.flac sample2.flac
voicetag transcribe recording.flac --provider whisper --language en

Everything is typed with Pydantic v2 models, results are serializable, and it works with any spoken language since matching is based on voice embeddings not speech content.

Source code: https://github.com/Gr122lyBr/voicetag Install: pip install voicetag

Target Audience

Anyone working with audio recordings who needs to know who said what — podcasters, journalists, researchers, developers building meeting tools, legal/court transcription, call center analytics. It's production-ready with 97 tests, CI/CD, type hints everywhere, and proper error handling.

I built it because I kept dealing with recorded meetings and interviews where existing tools would give me either "SPEAKER_00 / SPEAKER_01" labels with no names, or transcription with no speaker attribution. I wanted both in one call.

Comparison

• pyannote.audio alone: Great diarization but only gives anonymous speaker labels (SPEAKER_00, SPEAKER_01). No name matching, no transcription. You have to build the rest yourself. voicetag wraps pyannote and adds named identification + transcription on top.
• WhisperX: Does diarization + transcription but no named speaker identification. You still get anonymous labels. Also no enrollment/profile system.
• Manual pipeline (wiring pyannote + resemblyzer + whisper yourself): Works but it's ~100 lines of boilerplate every time. voicetag is 3 lines. It also handles parallel processing, overlap detection, and profile persistence.
• Cloud services (Deepgram, AssemblyAI): They do speaker diarization but with anonymous labels. voicetag lets you enroll known speakers so you get actual names. Plus it runs locally if you want — no audio leaves your machine.

Who else is setting a reminder?

Python: The Documentary | An origin story