Revert: PyJType for java.util.Map now extends collections.abc.Mapping #503
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The changes in #469 are incompatible with the cpython changes in python/cpython#28748. Testing the current dev_4.2 branch on python 3.12 results in
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases
from test_maps and test_builtins.The problem is that in python
java.util.Map
is extendingcollections.abc.Mapping
. The metaclass forcollections.abc.Mapping
isabc.ABCMeta
. In Python 3.12java.util.Map
inherits theabc.ABCMeta
metaclass. When jep tries to define a type forjava.util.AbstractMap
, or any other mapping implementation, it extends fromjava.lang.Object
with a metaclass ofPyJType
but it also extends from thejava.util.Map
with a metaclass ofabc.ABCMeta
and as the error mentions you cannot mix these two metaclasses.In earlier versions of Python the metaclass was not inherited by
java.util.Map
so the metaclass was simplytype
, which is compatible withPyJType
so jep works. The type forjava.util.Map
is defined using PySpec_FromType and the documentation for this function helpfully mentions the new incompatibility "Changed in version 3.12: The function now finds and uses a metaclass corresponding to the base classes provided in Py_tp_base[s] slots. Previously, only type instances were returned."I believe the new behavior of Python is correct, so Jep must be updated to work with newer Python. Similar problems have always existed when using metaclasses in pure python and I have been experimenting with the solutions mentioned on this stackoverflow post which is similar to advice I have found elsewhere. I tried to make a new Metaclass for
java.util.Map
which combines bothPyJType
andabc.ABCMeta
. However sincejava.util.Map
is defined with a PyType_Spec it is necessary to use the new PyType_FromMetaclass function and my attempts to use that function with a combine metaclass result inTypeError: Metaclasses with custom tp_new are not supported.
I think we may be able to get around that error by defining a custom class to act as the parent class tojava.util.Map
which uses the combined metaclass and is defined by callingtype()
, without a spec, however even if that compiles and runs the same problem theoretically exists, thetp_new
forabc.ABCMeta
would not run for thejava.util.Map
type. I have become convinced that we should not introduce abc types into the type hierarchy for Java types, the mixing of metaclasses is just too complicated.I think the only reasonable solution is to revert #469. We have not released this feature yet so there is no compatibility issues if we remove it. I'd be happy to try other solutions if anyone else has ideas. I really like the idea of using the
collections.abc
base classes to increase compatibility between Java and Python with minimal code but I don't see away to make the Metaclasses align. In the future I think we can achieve better interoperability betweenjava.util.Map
anddict
, by defining the necessary functions(items()
,keys()
, andvalues()
) in c but I am not confident I will have time to implement that before the release of 3.12 later this year(Contributions are always welcome if someone else is interested in working on this) so for now I am just proposing a reversion. Unfortunately the new PyBuiltins interface introduces indev_4.2
was relying on this interoperability to create dicts so that functionality has also been removed.This change reverts #469 and the new code in PyBuiltins that depends on it. With this change the unit tests will pass on Python 3.12