Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistency in handling None handlers in the Python and C implementations of pickle #93627

Open
serhiy-storchaka opened this issue Jun 9, 2022 · 1 comment
Labels
3.10 only security fixes 3.11 only security fixes 3.12 bugs and security fixes type-bug An unexpected behavior, bug, or error

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Jun 9, 2022

In the Python implementation of pickle setting some handlers to None causes a fallback to other methods.

  • Setting Pickler.reducer_override to None has the same effect as not setting this attribute -- falling back to other methods.
  • Setting the dispatch_table to None leads to falling back to __reduce_ex__.
  • Setting the __reduce_ex__ attribute to None leads to falling back to __reduce__.
  • Setting the __setstate__ attribute to None leads to falling back to the default implementation of __setstate__ (setting __dict__ and slots).

In the C implementation all this leads to TypeError: 'NoneType' object is not callable.

The copy module follows the Python implementation of pickle, except that it fails for any false value of __reduce__, not just None.

In most other code setting a dunder method to None leads to a TypeError (either raise explicitly or just generic "'NoneType' object is not callable").

We should decide what behavior should be in the pickle and copy module. On one hand, the Python implementation was primary. On other hand, it was inconsistent between the pickle and copy modules and was already changed in the past, the C implementation of pickle is now used by most users, and it is more consistent with other code.

See also:
https://mail.python.org/archives/list/[email protected]/thread/YGAK34DRWJFSIV2VZ4NC2J24XO37GCMM/
https://docs.python.org/3.10/reference/datamodel.html#id2
#70146
#55781
#60755

Linked PRs

@serhiy-storchaka serhiy-storchaka added type-bug An unexpected behavior, bug, or error 3.11 only security fixes 3.10 only security fixes 3.12 bugs and security fixes labels Jun 9, 2022
@eendebakpt
Copy link
Contributor

@serhiy-storchaka How can we move forward with this issue? Adapting the different components is something I can help out with, but I do not have sufficient experience to make an informed decision on which behavior is desired and whether we can make backwards incompatible changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.10 only security fixes 3.11 only security fixes 3.12 bugs and security fixes type-bug An unexpected behavior, bug, or error
Projects
Status: No status
Development

No branches or pull requests

2 participants