Ideally we'll get most of this done for 3.12. FWIW, there isn't enough time if it's just me, so consider this a call out to whoever can help. 😄 Feel free to adjust the TODO list and (hidden) "analysis" table above. Also feel free to reach out to any others that might be interested in pitching in here. Thanks!!!

@erlend-aasland, @corona10, @kumaraditya303, etc.

@terryjreedy (for _tkinter)

I've got WIP branches for datetime, io, pickle, collections, and ssl. The three former are up as draft PRs.

(FYI, I have quite a bit of CPython time scheduled for next week.)

@markshannon

Regarding the freelists in _asyncio, Kumar and Guido had some thoughts over at #91375 (comment).

Elementtree was isolated/ported in gh-92123.

_lsprof is also done, AFAICS. ... just noticed rotatingtree.c; sorry 'bout the noise.

Thanks for the updates!

@serhiy-storchaka for _tkinter. I have nothing to do with it.

AFAICS, array should be fine; array_reconstructor is in the module state already.

I will try to find which modules that I can support for this issue :)

I will try to find which modules that I can support for this issue :)

Feel free to pick up any of my draft PRs! :)

I tried submitting some PRs(Port _datatime module, Convert _pickle module to use heap types) for multi-phase initialization last year (delayed due to PEP 687 not being accepted at that time).
Maybe now that I can start something new! Can I try to work for _decimal module, if no one has claimed it yet.

I tried submitting some PRs(Port _datatime module, Convert _pickle module to use heap types) for multi-phase initialization last year (delayed due to PEP 687 not being accepted at that time).

Looks like you can reopen your PRs :)

Maybe now that I can start something new! Can I try to work for _decimal module, if no one has claimed it yet.

Sure you can! I started doing some work on _decimal some weeks ago, but it is very far from complete; feel free to start with that branch :) https://github.com/erlend-aasland/cpython/tree/isolate-decimal

I volunteer to review the PRs as I have been doing, and leave _asyncio for me to handle.

@erlend-aasland I'd like _datetime static types to be heap types to help #113601. Would you mind if I open an issue and take the type conversion part from your POC (without putting them into some states)?

@erlend-aasland I'd like _datetime static types to be heap types to help #113601. Would you mind if I open an issue and take the type conversion part from your POC (without putting them into some states)?

Converting static types to heap types is not the blocker issue for the datetime. The problem of datetime is its C API.

It reminds me the problem of the AST C API. We tried different things but we had multiple crashes. At the end, I moved AST types to the PyInterpreterState, they are initialized at the first access. This way, from any code path, you always meet the same types. Before, depending on how types were created, you might meet different types and so PyAST_Check() returned false which was very surprising.

I can help you if you open an issue. But please, first, move definition of these types to PyInterpreterState (put PyTypeObject* pointers there), in a new "datetime" state even if it's shared library.

Maybe already known, but reloading any module (e.g.-R) seems to be possibly unsafe:

import sys
from _decimal import Decimal as D1
del sys.modules['_decimal']
from _decimal import Decimal as D2

print(a := D1(1), b := D1(1), a == b, a is b)  # load
print(c := D2(1), d := D2(1), c == d, c is d)  # reload
print(a == c)

• 3.12: Static Decimal type, single-phase-init

1 1 True False
1 1 True False
True

• 3.13: Heap Decimal type, multi-phase init

1 1 True False
1 1 True False
False   # a and c have different module states

Indeed, PyInterpreterState should be effective for _datetime to be used in main and isolated sub-interpreters.

What do you mean by unsafe? If something's unsafe at the C level (e.g. undefined behaviour, crashes), that's a bug.

This behaviour is known; see https://docs.python.org/3/howto/isolating-extensions.html#surprising-edge-cases and the warning in https://docs.python.org/3/library/sys.html#sys.modules

What do you mean by unsafe?

Thanks. I suspect _datetime 's crashes with multi-phase init come from incorrect state accesses from binary (or ternaly) functions. I also think the init check in test_capi should be reconsidered. However, I agree to use PyInterpreterState if we need to check multiple module states. I have not fully investigated, though.

EDIT: The _datetime crash can be seen in the refleak test.

I've opened a _datetime issue: #117398

@Yhg1s, could we get an exception for 3.13 like we got for 3.12, regarding porting stdlib extensions to multi-phase init? You let us land those changes up to beta 2.

This time around there are only two modules left to be done, and the PRs for both are close enough that I expect they could be wrapped up before beta 2 (end of May).

@vstinner

From #118357 (comment):

Which crash? This PR is related to issue #117398 which doesn't mention any crash.

I said at #103092 (comment) in this thread:

I'd like _datetime static types to be heap types to help #113601.

You replied:

I can help you if you open an issue.

Then, I opened the issue #117398. I think we are aware of #113601, whose original issue was raised a year ago and you said there (#100911 (comment)):

It will work as expected once more extensions will be converted to the multi-phase initialization API.

We know well that the porting resolves existing different issues derived from static types and single-phase init. My volunteer is following the generic approach that seems to be supported by active core devs, especially by you. Also, the recent crash case you reported at #116467 on _ctypes (actually ctypes) was actually fixed by multi-phase init.

@kumaraditya303 has not been in good health, right? And @erlend-aasland appears to be not working individually on this project, and to be exhausted: #110415 (comment).

As to _datetime C-API, there seems to be no technical reason to block the isolation. Even if it blocks, I do not understand why there was no attempt to unblock nor to investigate further.

Such change is too late for beta1. It's better to wait for the beginning of the 3.14 dev cycle.

How do you think of the situation?

In my experience, such change is likely to cause regression and need more time to be stabilized.

Again, how do you think of the delay? Maybe isolated _datetime could be stabilized even at this point if we tried to work without bias nor reluctance? Why we cannot stabilize _datetime in the regular way with many eye-balls on 3.13?

Please open a dedicated issue for the datetime crash.

@Yhg1s, could we get an exception for 3.13 like we got for 3.12, regarding porting stdlib extensions to multi-phase init? You let us land those changes up to beta 2.

I don't think that it's a good idea. Regressions are complex to debug/fix, and it's better to continue the work in Python 3.14 and learn how to work around Python 3.13 limited isolation.