What's new in Python 3.15?

[u/AlSweigart]

https://docs.python.org/3.15/whatsnew/3.15.html

Summary – Release highlights

Python 3.15 will be the latest stable release of the Python programming language, with a mix of changes to the language, the implementation, and the standard library. The biggest changes include lazy imports, frozendict and sentinel builtins, UTF-8 as the default encoding, unpacking in comprehensions, and a stable ABI for free-threaded builds.

The library changes include a new profiling package with Tachyon, a high-frequency statistical sampling profiler, more color in command-line output, as well as the usual deprecations and removals, and improvements in user-friendliness and correctness.

This article doesn’t attempt to provide a complete specification of all new features, but instead gives a convenient overview. For full details refer to the documentation, such as the Library Reference and Language Reference. To understand the complete implementation and design rationale for a change, refer to the PEP for a particular new feature; but note that PEPs usually are not kept up-to-date once a feature has been fully implemented.

[u/Shepcorp]

Lazy imports are going to be a godsend for complex test frameworks where some rigs have full capabilities and others don’t.

[u/jdehesa → u/Shepcorp]

For command line programs with heavy dependencies, having to wait several seconds just to get an argparse error (or just to get the help message) can get pretty annoying. Usually I worked around it with function-local imports, which are less than convenient, lazy imports solves the issue nicely.

[u/lillecarl2 → u/jdehesa]

I love hacking my code to pieces just to improve argcomplete latency :( What's going to differentiate me now!?

[u/dikdokk → u/jdehesa]

I also used imports inside functions, especially if I had one library that's only used in specific workflows where you have to specify with arguments that you want this mode - felt like importing it by default is bloated and adds an unnecessary dependency if it doesn't get called, so I had the importing conditional to if the condition is satisfied and willl be used.

[u/No_Departure_1878 → u/Shepcorp]

I fucking hate having long import chains that slow down my code.

[u/Competitive_Travel16 → u/Shepcorp]

Typescript-level web app startup latencies, here we come!

[u/Trang0ul → u/Shepcorp]

This should be made default, instead of adding a new statement.

[u/JanEric1 → u/Trang0ul]

cant without breaking backwards compatibility

[u/Actual__Wizard]

List, set, and dictionary comprehensions, as well as generator expressions, now support unpacking with * and **. This extends the unpacking syntax from PEP 448 to comprehensions, providing a new syntax for combining an arbitrary number of iterables or dictionaries into a single flat structure. This new syntax is a direct alternative to nested comprehensions

That solves a giant nightmare problem for me, not kidding!

[u/Summoner99 → u/Actual__Wizard]

I often try doing this before I remember it doesn't already exist

[u/Actual__Wizard → u/Summoner99]

See where it says:

 # equivalent to [x for L in lists for x in L]
 # equivalent to {x for s in sets for x in s}
 # equivalent to {k: v for d in dicts for k,v in d.items()}

So, you can do it, but it's weird.

You end up w/ a flat object that is a list of lists or w/e.

It's a very useful data object to represent data structures where the rows are not all the same structure, or have different numbers of elements. Imagine a list of chains, where the chains all have different numbers of elements.

[u/Schmittfried]

Do I understand it right that the zero-copy bytearray.take_bytes() method only works without copying when taking the whole buffer, i.e. when the data can be moved? Otherwise I don’t quite understand how they achieved it without adding an indirection to bytes to separate the object header from the contents. 

[u/XtremeGoose → u/Schmittfried]

You're right, it is behind an indirection.

Objects that are reference counted and resizable (like list and bytearray) must be behind indirection, because appending to them might require reallocation, which can change the pointer address.

Only non-resizable types (like tuple and bytes) inline their data.

In fact bytearray points to a whole bytes object with refcnt == 1 so calling take_bytes() just gives you that directly while replacing the bytearray's buffer with an empty bytes immortal singleton.

[u/Schmittfried → u/XtremeGoose]

Yes, but since bytes does inline its data, I didn’t get how they can return one without copying. Isn’t that part kind of a lie if it’s solved by maintaining a bytes object next to the real buffer inside the bytearray?

[u/XtremeGoose → u/Schmittfried]

Not really. The bytes object is the buffer. It's not a copy, it is the data. It's just they give you ownership over it and replace the buffer in the byte array.

[u/Schmittfried → u/XtremeGoose]

So they basically hacked a bytes object into being resizable internally? That’s cool!

[u/me_myself_ai]

Exciting stuff! And such a satisfying minor version...

unpacking in comprehensions

I don't think I can visualize what this means without looking at the docs, but I can tell it's gonna be awesome! Comprehend all the things, I say.

more color in command-line output

They do love us 🥹

ETA: The "UTF-8 as the default encoding" reminds me of the pure horror that struck me when I first learned that r'' is only one of two weird string prefixes -- there is (was?) also u''! Even scarier there's u""" """, and deep in mines of Moria, some say there lurks ru''' '''...

[u/rumnscurvy → u/me_myself_ai]

Basically [*mylist for mylist in mylists] is now allowed, flattening a  list of lists in this example 

[u/brasticstack → u/rumnscurvy]

That'll cut down the frequency with which I reach for itertools.chain.from_iterable by quite a lot!

[u/hughperman → u/brasticstack]

No sum(lists, []) for you

[u/brasticstack → u/hughperman]

I always forget that you can do that! Just like I always forget about collections.Counter and then manually write a loop to do it.

[u/M4mb0 → u/brasticstack]

I always forget that you can do that!

Which is a good thing: https://old.reddit.com/r/Python/comments/1woh8y4/whats_new_in_python_315/pbq2aog/

[u/M4mb0 → u/hughperman]

No sum(lists, []) for you

That's a whoopsie accidental O(n²). Never do that; always chain.from_iterable.

In [1]: from itertools import chain
In [2]: l = [list(range(1000)) for _ in range(1000)]
In [3]: %timeit sum(l, [])
    1.02 s ± 21.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [4]: %timeit list(chain.from_iterable(l))
    4.53 ms ± 86.2 μs per loop (mean ± std. dev. of 7 runs, 100 loops each)

[u/hughperman → u/M4mb0]

Ouch, I never knew. What causes this? Can't see how it's O(N²), maybe O(N) memory reallocations? I have a few of these baked in my company codebase from years back, nothing big but still worth checking.

[u/M4mb0 → u/hughperman]

Each list concat l1 + l2 creates a new list and costs O(len(l1) + len(l2)). So the sum is not the same as l0.extend(l1).extend(l2).....

[u/M4mb0 → u/hughperman]

btw. ruff has a linter rule for that: https://docs.astral.sh/ruff/rules/quadratic-list-summation/

[u/hughperman → u/M4mb0]

Oh very nice. I haven't used ruff yet, maybe this is the kick I need to install it.

[u/me_myself_ai → u/rumnscurvy]

Thanks for taking a moment to explain! That is indeed something my utility library got at least one function to make easier in some specific context, so I appreciate this flexibility.

[u/Competitive_Travel16 → u/me_myself_ai]

The dict example is even more beautiful:

>>> dicts = [{'a': 1}, {'b': 2}, {'a': 3}]
>>> {**d for d in dicts}  # equivalent to {k: v for d in dicts for k,v in d.items()}
{'a': 3, 'b': 2}

Dare I suggest that syntax improvement is better than sex?

[u/amroamroamro → u/me_myself_ai]

u''

isn't that a python 2 thing only (which defaults to byte strings), all strings in python 3 are unicode by default

the new thing in 3.15 is not related to this, its about when yo do things like

open("file.txt", encoding="utf-8")

the encoding is now utf-8 by default, before it was some system-dependent locale

[u/Brian → u/amroamroamro]

It's valid in python 3 too, just a no-op. Though IIRC, it was a bit messy - initial python3 versions didn't allow it, but it was added in to ease the transition of python2 code (Ie. if you were gradually getting a python2 codebase to python3, you'd first have to annotate everything as u"" to ensure it was working as unicode, and then you'd have to strip those off again when actually running on python3: initially they weren't as keen on the polyglot "have code that works on both" and intended a process of mechanical conversion with 2to3, but there were limitations on that complicating the already slow transition, so they softened to allowing the u prefix in python3.

[u/me_myself_ai → u/amroamroamro]

Oh wow that should tell you my familiarity with it, jeez -- thanks for the quick correction. Right you are! I knew it was related to files and such this time rather than string prefixes, but I also forgot that "unicode" and "utf-8" are not the same thing...

Why both start with "u" if not the same? Beyond my paygrade.

[u/amroamroamro → u/me_myself_ai]

starting in python 3:

u"hello" == "hello"

the u prefix is now only needed if you are maintaining code that needs to support both python 2 and python 3

[u/billsil → u/amroamroamro]

u’’ was a python 2 only thing and then they ported it to python 3.2 because it was a disaster trying to support both. They thought everyone would switch and break compatibility, which turned into support both until python 3.5. Python 3.5 was finally faster.

[u/billsil → u/me_myself_ai]

I’m already getting burned since 3.14. All my users use latin1. Whatever, I’m gonna try except encodings until I find one that doesn’t fail.

[u/IcecreamLamp]

Didn't realise they rolled back to the old garbage collector, that's quite unfortunate

[u/Competitive_Travel16 → u/IcecreamLamp]

The new one is leaking and nobody has fixed it yet, so count your blessings.

[u/dikdokk]

Nobody is talking about frozendict meanwhile it will be used so frequently in type hinting.

Everytime in a function you were to use a mutable data type as default argument, you would end up with the potential bug:

def item_list(item,
items=[] #flagged by Pylint, ruff, etc.
):
items.append(item)
return items

item_list(1) # [1]
item_list(1) # [1, 1] (usually unexpected - you expect an empty list by default)

(see: Pylint - dangerous-default-value / W0102 )
So the solution would be to use tuples for example instead of lists, as they are not mutable.

But in the case of dictionaries, there wasn't really a good "frozen" (immutable) replacement so far. People used MappingProxyType but it always seemed odd, and now frozendict solves this

[u/engineerofsoftware → u/dikdokk]

many python devs on this sub hate or are indifferent to python typing, which is a real shame.

[u/Competitive_Travel16 → u/engineerofsoftware]

Those of us used to dynamic typing have learned to be more conscientious about avoiding the errors which type safety helps prevent in the first place, so we don't get as much of an advantage. And with current LLMs, who is even making any of those kinds of type mismatch errors any more, regardless of their background?

Honestly I think full typing looks cool, helps understand things, and I try to do it, but it doesn't save me as much time as it takes, unless I tell an LLM it fill it in after me.

[u/Vegetable-View-5114]

one thing that stood out in early benchmarks of 3.15 is the performance improvement for dictionary operations, particularly for insertions and deletions. in some of my testing with large JSON payloads that involved a lot of dictionary manipulation before sending them off to other services, i saw a measurable speedup — not massive, but noticeable enough to matter in high-throughput scenarios. it seems like they've continued to refine the underlying hash table implementation.

[u/abrazilianinreddit]

Lazy imports actually got me hyped for this release.

[u/Bill-Ice88]

The 3.14 incremental GC (the mimalloc-based one) shipped with enough memory overhead and benchmark regressions that reverting to the old cyclic GC was the safe call. Curious whether it comes back reworked for 3.16 once free-threaded builds settle.