I intend to send a PR later today.

However, I am not sure if I can revert the removal + deprecate after the feature freeze, or if I must only revert the removal in 3.11 and deprecate it in 3.12+. Will start with a PR to main anyway.

EDIT: I won't send a PR before there is a consensus.

Removal discussed at #91367

The discussion started in this larger issue about re "reorganization": #91308

I don't understand the purpose of the re.template() function and the re.TEMPLATE flag. It doesn't seem to disable backtracking:

>>> re.compile(r"((a)b\2)", re.IGNORECASE).match("aba").groups()
('aba', 'a')
>>> re.template(r"((a)b\2)", re.IGNORECASE).match("aba").groups()
('aba', 'a')

The only thing which seems to be changed is that repetition is not supported by TEMPLATE:

>>> re.template(r"a+")
re.error: internal: unsupported template operator MAX_REPEAT
>>> re.template(r"a*")
re.error: internal: unsupported template operator MAX_REPEAT

Code search for "TEMPLATE" in re/sre source code files of Python 3.10:

Lib/re.py:    TEMPLATE = T = sre_compile.SRE_FLAG_TEMPLATE # disable backtracking
Lib/sre_compile.py:            if flags & SRE_FLAG_TEMPLATE:
Lib/sre_constants.py:SRE_FLAG_TEMPLATE = 1 # template mode (disable backtracking)
Lib/sre_constants.py:        f.write("#define SRE_FLAG_TEMPLATE %d\n" % SRE_FLAG_TEMPLATE)
Lib/sre_parse.py:    "t": SRE_FLAG_TEMPLATE,
Lib/sre_parse.py:GLOBAL_FLAGS = SRE_FLAG_DEBUG | SRE_FLAG_TEMPLATE
Modules/sre_constants.h:#define SRE_FLAG_TEMPLATE 1

cc @serhiy-storchaka

To be honest, I don't understand the purpose of the re.template() function either and I think that deprecating and removing it is a good thing.

If we decide to not revert the change, I suggest to document the removal in What's New in Python 3.11 and explain how to port impacted applications. For example, replace re.template() with re.compile().

If we decide to not revert the change, I suggest to document the removal in What's New in Python 3.11 and explain how to port impacted applications. For example, replace re.template() with re.compile().

I suppose we should do that even if we "just" deprecate it.

This was removed in b09184bf05 without a depreciation period.
Despite being undocumented, projects actually use this function, see e.g. rpm-software-management/dnf#1827

PEP 387 says: https://peps.python.org/pep-0387/#backwards-compatibility-rules

This policy applies to all public APIs. (...) Others are explicitly not part of the public API. They can change or be removed at any time in any way.

The rationale here is that since re.template() and re.TEMPLATE are not documented, they are not part of the public API and so are not covered by PEP 387 deprecation policy.

Yes, projects use and abuse private and undocumented functions, and yes, they are broken when these private/undocumented functions are changed or removed.

I suggest to document the removal, but don't revert it. Projects using it can replace re.template() with re.compile(). Or you can even add re.template = re.compile and re.TEMPLATE = 0 to not have to modify the code of affected applications.

@pablogsal: As the Python 3.11 release manager, do you have an opinion on this removal?

I cannot really relate to the "not documented == private" assumption, but I'll leave that decision to core contributors. However, if that is the consensus it would be great to have that written in the PEP.

As for this specific case, re.template was included in __all__ hence I'd argue it was explicitly part of the public API (unlike re.TEMPLATE).

I am surprised that this function was used anywhere. I searched on GitHub before removing it and did not found anything relevant on first 20 or like pages.

The original intention of this function is not clear, but it was never correctly implemented. Instead, it started raising an error for repetition operations since 29c4ba9. I suppose that it was not the original intention. The comment about disabling backtracking is not completely true, because an alternation and a conditional pattern involve backtracking too.

I am surprised that this function was used anywhere.

People will use almost anything that'll get their job done. This particular example is almost 14 years old and I guess nobody remembers why they used it. There might be more code like this outside of our search possibilities (e.g. proprietary code). That's why I think a deprecation period is in order :)

Undocumented functions do not require a deprecation period. A removal notice in whatsnew will suffice.

This is doubly true for function that is flimsy, never fully implemented, and was at least partially disabled.

Well, I removed it because I was going to add new features containing "template" in the name, and I thought that it can be confusing.

But if you think that it is worth, I am not opposing to reverting #32300 and adding a deprecation warning instead.

I don't find any "re\.template\( in the top PypI 5,000 packages.

GitHub's new search tool finds only 19 Python files matching /\bre\.template\(/, many of which are forks or duplicates.

1. https://github.com/elyscape/dnf/blob/5965627a81be59f6dbb9ff2362be1aba8f897237/dnf/cli/term.py#L284
2. https://github.com/venaxyt/mysterium/blob/8cac9b1bbdb882873a0e260bfb8bfd273bcfcbc3/modules/re.py#L99
3. https://github.com/containerbuildsystem/osbs-client/blob/f719759af18ef9f3bb45ee4411f80a9580723e31/make_mock_json.py#L142
4. https://github.com/th3r00t/pyShelf/blob/aef30f3dcab4014d16a0f65388e92b6310f673a4/src/backend/lib/config.py#L61
5. https://github.com/shoptime/trex/blob/75fea0e2f79a6be25917011456bbfef31aebdbaf/support/mongoengine.py#L20
6. https://github.com/brinkqiang/dnf/blob/6517f47c94bdf2da2b1ce11605babee4d8cb8756/dnf/cli/term.py#L291
7. https://github.com/tizenorg/platform.upstream.yum/blob/40014cdf8aae2f0a8c3eccd7f908e6ed1f5ddd65/output.py#L268
8. https://github.com/lovaya/Ryven/blob/a11e361528d982a9dd3c489dd536f8b05ffd56e1/Ryven/packages/auto_generated/re/nodes.py#L355
9. https://github.com/Distrotech/yum/blob/f4e54aeed297158c563828aa3ebb93d0c8ce7e38/output.py#L334
10. https://github.com/rpm-software-management/yum/blob/4ed25525ee4781907bd204018c27f44948ed83fe/output.py#L334
11. https://github.com/TheApacheCats/yum/blob/5c9c8db7b380457f71482a5ed28cd5fdb653a37b/output.py#L334
12. https://github.com/timgates42/KomodoEdit/blob/980b5b00afda0144dd204c19de175b1a32ff12a8/src/codeintel/test2/test_python.py#L2220
13. https://github.com/ryanuber/dnf/blob/fd8c108be591d52da6758214cb2b32a54d1afc31/dnf/cli/output.py#L334
14. https://github.com/itmightbemo/KomodoEdit/blob/7f125d5140994e213ee892afac0210d7fa293699/src/codeintel/test2/test_python.py#L2182
15. https://github.com/fatherlinux/containers-deep-dive/blob/76b3dbeb21c84d9536dc47660e958da520815b9e/containers201/rootfs/usr/share/yum-cli/output.py#L334
16. https://github.com/cgwalters/dnf/blob/b81feda59ab79bc8a1ade4a54ee4b35789f2b75b/dnf/cli/term.py#L276
17. https://github.com/SublimeCodeIntel/CodeIntel/blob/5a84f609c55972297119c2d7bc9d045222ee6d93/codeintel/test2/test_python.py#L2230
18. https://github.com/alamatic/komodoedit/blob/19b97c35c59f1d232fa51bb0c96cb4cf5cc86248/src/codeintel/test2/test_python.py#L2082
19. https://github.com/rmccue/codeintel/blob/5a6d47672c35d5fb1911312a8e3973284abd165d/test2/test_python.py#L2165

I have no opinion on this particular removal/deprecation, but just quickly want to point to PEP8 (https://peps.python.org/pep-0008/#public-and-internal-interfaces) and it's directive that:
To better support introspection, modules should explicitly declare the names in their public API using the __all__ attribute. Setting __all__ to an empty list indicates that the module has no public API.. Previously, template was in __all__ which is also pre-faced with a # public symbols comment.

Perhaps there are reasons why this does not hold true for CPython, but it would be good if that could be documented somewhere. As a package maintainer I have recently been struggling to correctly determine what is and isn't public in a codebase that does not currently document this. So guidelines such as the one in the PEP and general agreement on those guidelines would be helpful for me and the wider ecosystem I think.

But if you think that it is worth, I am not opposing to reverting #32300 and adding a deprecation warning instead.

Yes, I believe that would be in the spirit of the policy. I will try to draft a PR shortly and then I will leave it for the others to decide if it's worth it.

How long the deprecation period should be? Would the only 3.11 be enough?

The policy speaks of 2 releases.

I've opened #93161

@rhettinger:

Undocumented functions do not require a deprecation period.

I disagree.
Is this documented somewhere, or is it tribal knowledge (which should be documented, now that we have PEP-387 written)?

Thank you @hugovk for your investigation and PR.

Thank you @hroncok for the PR ;)