Python Initialization Configuration¶

Added in version 3.8.

Python can be initialized with Py_InitializeFromConfig() and the PyConfig structure. It can be preinitialized with Py_PreInitialize() and the PyPreConfig structure.

There are two kinds of configuration:

The Python Configuration can be used to build a customized Python which behaves as the regular Python. For example, environment variables and command line arguments are used to configure Python.
The Isolated Configuration can be used to embed Python into an application. It isolates Python from the system. For example, environment variables are ignored, the LC_CTYPE locale is left unchanged and no signal handler is registered.

The Py_RunMain() function can be used to write a customized Python program.

Example¶

Example of customized Python always running in isolated mode:

int main(int argc, char **argv)
{
    PyStatus status;

    PyConfig config;
    PyConfig_InitPythonConfig(&config);
    config.isolated = 1;

    /* Decode command line arguments.
       Implicitly preinitialize Python (in isolated mode). */
    status = PyConfig_SetBytesArgv(&config, argc, argv);
    if (PyStatus_Exception(status)) {
        goto exception;
    }

    status = Py_InitializeFromConfig(&config);
    if (PyStatus_Exception(status)) {
        goto exception;
    }
    PyConfig_Clear(&config);

    return Py_RunMain();

exception:
    PyConfig_Clear(&config);
    if (PyStatus_IsExit(status)) {
        return status.exitcode;
    }
    /* Display the error message and exit the process with
       non-zero exit code */
    Py_ExitStatusException(status);
}

PyWideStringList¶

type PyWideStringList¶

List of wchar_t* strings.

If length is non-zero, items must be non-NULL and all strings must be non-NULL.

Methods:

PyStatus PyWideStringList_Append(PyWideStringList *list, const wchar_t *item)¶

Append item to list.

Python must be preinitialized to call this function.

PyStatus PyWideStringList_Insert(PyWideStringList *list, Py_ssize_t index, const wchar_t *item)¶

Insert item into list at index.

If index is greater than or equal to list length, append item to list.

index must be greater than or equal to 0.

Python must be preinitialized to call this function.

Structure fields:

Py_ssize_t length¶: List length.

wchar_t **items¶: List items.

PyStatus¶

type PyStatus¶

Structure to store an initialization function status: success, error or exit.

For an error, it can store the C function name which created the error.

Structure fields:

int exitcode¶: Exit code. Argument passed to exit().

const char *err_msg¶: Error message.

const char *func¶: Name of the function which created an error, can be NULL.

Functions to create a status:

PyStatus PyStatus_Ok(void)¶: Success.

PyStatus PyStatus_Error(const char *err_msg)¶

Initialization error with a message.

err_msg must not be NULL.

PyStatus PyStatus_NoMemory(void)¶: Memory allocation failure (out of memory).

PyStatus PyStatus_Exit(int exitcode)¶: Exit Python with the specified exit code.

Functions to handle a status:

int PyStatus_Exception(PyStatus status)¶: Is the status an error or an exit? If true, the exception must be handled; by calling Py_ExitStatusException() for example.

int PyStatus_IsError(PyStatus status)¶: Is the result an error?

int PyStatus_IsExit(PyStatus status)¶: Is the result an exit?

void Py_ExitStatusException(PyStatus status)¶: Call exit(exitcode) if status is an exit. Print the error message and exit with a non-zero exit code if status is an error. Must only be called if PyStatus_Exception(status) is non-zero.

Note

Internally, Python uses macros which set PyStatus.func, whereas functions to create a status set func to NULL.

Example:

PyStatus alloc(void **ptr, size_t size)
{
    *ptr = PyMem_RawMalloc(size);
    if (*ptr == NULL) {
        return PyStatus_NoMemory();
    }
    return PyStatus_Ok();
}

int main(int argc, char **argv)
{
    void *ptr;
    PyStatus status = alloc(&ptr, 16);
    if (PyStatus_Exception(status)) {
        Py_ExitStatusException(status);
    }
    PyMem_Free(ptr);
    return 0;
}

PyPreConfig¶

type PyPreConfig¶

Structure used to preinitialize Python.

Function to initialize a preconfiguration:

void PyPreConfig_InitPythonConfig(PyPreConfig *preconfig)¶: Initialize the preconfiguration with Python Configuration.

void PyPreConfig_InitIsolatedConfig(PyPreConfig *preconfig)¶: Initialize the preconfiguration with Isolated Configuration.

Structure fields:

int allocator¶

Name of the Python memory allocators:

PYMEM_ALLOCATOR_NOT_SET (0): don’t change memory allocators (use defaults).
PYMEM_ALLOCATOR_DEFAULT (1): default memory allocators.
PYMEM_ALLOCATOR_DEBUG (2): default memory allocators with debug hooks.
PYMEM_ALLOCATOR_MALLOC (3): use malloc() of the C library.
PYMEM_ALLOCATOR_MALLOC_DEBUG (4): force usage of malloc() with debug hooks.
PYMEM_ALLOCATOR_PYMALLOC (5): Python pymalloc memory allocator.
PYMEM_ALLOCATOR_PYMALLOC_DEBUG (6): Python pymalloc memory allocator with debug hooks.
PYMEM_ALLOCATOR_MIMALLOC (6): use mimalloc, a fast malloc replacement.
PYMEM_ALLOCATOR_MIMALLOC_DEBUG (7): use mimalloc, a fast malloc replacement with debug hooks.

PYMEM_ALLOCATOR_PYMALLOC and PYMEM_ALLOCATOR_PYMALLOC_DEBUG are not supported if Python is configured using --without-pymalloc.

PYMEM_ALLOCATOR_MIMALLOC and PYMEM_ALLOCATOR_MIMALLOC_DEBUG are not supported if Python is configured using --without-mimalloc or if the underlying atomic support isn’t available.

See Memory Management.

Default: PYMEM_ALLOCATOR_NOT_SET.

int configure_locale¶

Set the LC_CTYPE locale to the user preferred locale.

If equals to 0, set coerce_c_locale and coerce_c_locale_warn members to 0.

See the locale encoding.

Default: 1 in Python config, 0 in isolated config.

int coerce_c_locale¶

If equals to 2, coerce the C locale.

If equals to 1, read the LC_CTYPE locale to decide if it should be coerced.

See the locale encoding.

Default: -1 in Python config, 0 in isolated config.

int coerce_c_locale_warn¶

If non-zero, emit a warning if the C locale is coerced.

Default: -1 in Python config, 0 in isolated config.

int dev_mode¶

Python Development Mode: see PyConfig.dev_mode.

Default: -1 in Python mode, 0 in isolated mode.

int isolated¶

Isolated mode: see PyConfig.isolated.

Default: 0 in Python mode, 1 in isolated mode.

int legacy_windows_fs_encoding¶

If non-zero:

Set PyPreConfig.utf8_mode to 0,
Set PyConfig.filesystem_encoding to "mbcs",
Set PyConfig.filesystem_errors to "replace".

Initialized from the PYTHONLEGACYWINDOWSFSENCODING environment variable value.

Only available on Windows. #ifdef MS_WINDOWS macro can be used for Windows specific code.

Default: 0.

int parse_argv¶

If non-zero, Py_PreInitializeFromArgs() and Py_PreInitializeFromBytesArgs() parse their argv argument the same way the regular Python parses command line arguments: see Command Line Arguments.

Default: 1 in Python config, 0 in isolated config.

int use_environment¶

Use environment variables? See PyConfig.use_environment.

Default: 1 in Python config and 0 in isolated config.

int utf8_mode¶

If non-zero, enable the Python UTF-8 Mode.

Set to 0 or 1 by the -X utf8 command line option and the PYTHONUTF8 environment variable.

Also set to 1 if the LC_CTYPE locale is C or POSIX.

Default: -1 in Python config and 0 in isolated config.

Preinitialize Python with PyPreConfig¶

The preinitialization of Python:

Set the Python memory allocators (PyPreConfig.allocator)
Configure the LC_CTYPE locale (locale encoding)
Set the Python UTF-8 Mode (PyPreConfig.utf8_mode)

The current preconfiguration (PyPreConfig type) is stored in _PyRuntime.preconfig.

Functions to preinitialize Python:

PyStatus Py_PreInitialize(const PyPreConfig *preconfig)¶

Preinitialize Python from preconfig preconfiguration.

preconfig must not be NULL.

PyStatus Py_PreInitializeFromBytesArgs(const PyPreConfig *preconfig, int argc, char *const *argv)¶

Preinitialize Python from preconfig preconfiguration.

Parse argv command line arguments (bytes strings) if parse_argv of preconfig is non-zero.

preconfig must not be NULL.

PyStatus Py_PreInitializeFromArgs(const PyPreConfig *preconfig, int argc, wchar_t *const *argv)¶

Preinitialize Python from preconfig preconfiguration.

Parse argv command line arguments (wide strings) if parse_argv of preconfig is non-zero.

preconfig must not be NULL.

The caller is responsible to handle exceptions (error or exit) using PyStatus_Exception() and Py_ExitStatusException().

For Python Configuration (PyPreConfig_InitPythonConfig()), if Python is initialized with command line arguments, the command line arguments must also be passed to preinitialize Python, since they have an effect on the pre-configuration like encodings. For example, the -X utf8 command line option enables the Python UTF-8 Mode.

PyMem_SetAllocator() can be called after Py_PreInitialize() and before Py_InitializeFromConfig() to install a custom memory allocator. It can be called before Py_PreInitialize() if PyPreConfig.allocator is set to PYMEM_ALLOCATOR_NOT_SET.

Python memory allocation functions like PyMem_RawMalloc() must not be used before the Python preinitialization, whereas calling directly malloc() and free() is always safe. Py_DecodeLocale() must not be called before the Python preinitialization.

Example using the preinitialization to enable the Python UTF-8 Mode:

PyStatus status;
PyPreConfig preconfig;
PyPreConfig_InitPythonConfig(&preconfig);

preconfig.utf8_mode = 1;

status = Py_PreInitialize(&preconfig);
if (PyStatus_Exception(status)) {
    Py_ExitStatusException(status);
}

/* at this point, Python speaks UTF-8 */

Py_Initialize();
/* ... use Python API here ... */
Py_Finalize();

PyConfig¶

type PyConfig¶

Structure containing most parameters to configure Python.

When done, the PyConfig_Clear() function must be used to release the configuration memory.

Structure methods:

void PyConfig_InitPythonConfig(PyConfig *config)¶: Initialize configuration with the Python Configuration.

void PyConfig_InitIsolatedConfig(PyConfig *config)¶: Initialize configuration with the Isolated Configuration.

PyStatus PyConfig_SetString(PyConfig *config, wchar_t *const *config_str, const wchar_t *str)¶

Copy the wide character string str into *config_str.

Preinitialize Python if needed.

PyStatus PyConfig_SetBytesString(PyConfig *config, wchar_t *const *config_str, const char *str)¶

Decode str using Py_DecodeLocale() and set the result into *config_str.

Preinitialize Python if needed.

PyStatus PyConfig_SetArgv(PyConfig *config, int argc, wchar_t *const *argv)¶

Set command line arguments (argv member of config) from the argv list of wide character strings.

Preinitialize Python if needed.

PyStatus PyConfig_SetBytesArgv(PyConfig *config, int argc, char *const *argv)¶

Set command line arguments (argv member of config) from the argv list of bytes strings. Decode bytes using Py_DecodeLocale().

Preinitialize Python if needed.

PyStatus PyConfig_SetWideStringList(PyConfig *config, PyWideStringList *list, Py_ssize_t length, wchar_t **items)¶

Set the list of wide strings list to length and items.

Preinitialize Python if needed.

PyStatus PyConfig_Read(PyConfig *config)¶

Read all Python configuration.

Fields which are already initialized are left unchanged.

Fields for path configuration are no longer calculated or modified when calling this function, as of Python 3.11.

The PyConfig_Read() function only parses PyConfig.argv arguments once: PyConfig.parse_argv is set to 2 after arguments are parsed. Since Python arguments are stripped from PyConfig.argv, parsing arguments twice would parse the application options as Python options.

Preinitialize Python if needed.

Changed in version 3.10: The PyConfig.argv arguments are now only parsed once, PyConfig.parse_argv is set to 2 after arguments are parsed, and arguments are only parsed if PyConfig.parse_argv equals 1.

Changed in version 3.11: PyConfig_Read() no longer calculates all paths, and so fields listed under Python Path Configuration may no longer be updated until Py_InitializeFromConfig() is called.

void PyConfig_Clear(PyConfig *config)¶: Release configuration memory.

Most PyConfig methods preinitialize Python if needed. In that case, the Python preinitialization configuration (PyPreConfig) in based on the PyConfig. If configuration fields which are in common with PyPreConfig are tuned, they must be set before calling a PyConfig method:

PyConfig.dev_mode
PyConfig.isolated
PyConfig.parse_argv
PyConfig.use_environment

Moreover, if PyConfig_SetArgv() or PyConfig_SetBytesArgv() is used, this method must be called before other methods, since the preinitialization configuration depends on command line arguments (if parse_argv is non-zero).

The caller of these methods is responsible to handle exceptions (error or exit) using PyStatus_Exception() and Py_ExitStatusException().

Structure fields:

PyWideStringList argv¶

Set sys.argv command line arguments based on argv. These parameters are similar to those passed to the program’s main() function with the difference that the first entry should refer to the script file to be executed rather than the executable hosting the Python interpreter. If there isn’t a script that will be run, the first entry in argv can be an empty string.

Set parse_argv to 1 to parse argv the same way the regular Python parses Python command line arguments and then to strip Python arguments from argv.

If argv is empty, an empty string is added to ensure that sys.argv always exists and is never empty.

Default: NULL.

Initialization with PyConfig¶

Initializing the interpreter from a populated configuration struct is handled by calling Py_InitializeFromConfig().

The caller is responsible to handle exceptions (error or exit) using PyStatus_Exception() and Py_ExitStatusException().

If PyImport_FrozenModules(), PyImport_AppendInittab() or PyImport_ExtendInittab() are used, they must be set or called after Python preinitialization and before the Python initialization. If Python is initialized multiple times, PyImport_AppendInittab() or PyImport_ExtendInittab() must be called before each Python initialization.

The current configuration (PyConfig type) is stored in PyInterpreterState.config.

Example setting the program name:

void init_python(void)
{
    PyStatus status;

    PyConfig config;
    PyConfig_InitPythonConfig(&config);

    /* Set the program name. Implicitly preinitialize Python. */
    status = PyConfig_SetString(&config, &config.program_name,
                                L"/path/to/my_program");
    if (PyStatus_Exception(status)) {
        goto exception;
    }

    status = Py_InitializeFromConfig(&config);
    if (PyStatus_Exception(status)) {
        goto exception;
    }
    PyConfig_Clear(&config);
    return;

exception:
    PyConfig_Clear(&config);
    Py_ExitStatusException(status);
}

More complete example modifying the default configuration, read the configuration, and then override some parameters. Note that since 3.11, many parameters are not calculated until initialization, and so values cannot be read from the configuration structure. Any values set before initialize is called will be left unchanged by initialization:

PyStatus init_python(const char *program_name)
{
    PyStatus status;

    PyConfig config;
    PyConfig_InitPythonConfig(&config);

    /* Set the program name before reading the configuration
       (decode byte string from the locale encoding).

       Implicitly preinitialize Python. */
    status = PyConfig_SetBytesString(&config, &config.program_name,
                                     program_name);
    if (PyStatus_Exception(status)) {
        goto done;
    }

    /* Read all configuration at once */
    status = PyConfig_Read(&config);
    if (PyStatus_Exception(status)) {
        goto done;
    }

    /* Specify sys.path explicitly */
    /* If you want to modify the default set of paths, finish
       initialization first and then use PySys_GetObject("path") */
    config.module_search_paths_set = 1;
    status = PyWideStringList_Append(&config.module_search_paths,
                                     L"/path/to/stdlib");
    if (PyStatus_Exception(status)) {
        goto done;
    }
    status = PyWideStringList_Append(&config.module_search_paths,
                                     L"/path/to/more/modules");
    if (PyStatus_Exception(status)) {
        goto done;
    }

    /* Override executable computed by PyConfig_Read() */
    status = PyConfig_SetString(&config, &config.executable,
                                L"/path/to/my_executable");
    if (PyStatus_Exception(status)) {
        goto done;
    }

    status = Py_InitializeFromConfig(&config);

done:
    PyConfig_Clear(&config);
    return status;
}

Isolated Configuration¶

PyPreConfig_InitIsolatedConfig() and PyConfig_InitIsolatedConfig() functions create a configuration to isolate Python from the system. For example, to embed Python into an application.

This configuration ignores global configuration variables, environment variables, command line arguments (PyConfig.argv is not parsed) and user site directory. The C standard streams (ex: stdout) and the LC_CTYPE locale are left unchanged. Signal handlers are not installed.

Configuration files are still used with this configuration to determine paths that are unspecified. Ensure PyConfig.home is specified to avoid computing the default path configuration.

Python Configuration¶

PyPreConfig_InitPythonConfig() and PyConfig_InitPythonConfig() functions create a configuration to build a customized Python which behaves as the regular Python.

Environments variables and command line arguments are used to configure Python, whereas global configuration variables are ignored.

This function enables C locale coercion (PEP 538) and Python UTF-8 Mode (PEP 540) depending on the LC_CTYPE locale, PYTHONUTF8 and PYTHONCOERCECLOCALE environment variables.

Python Path Configuration¶

PyConfig contains multiple fields for the path configuration:

Path configuration inputs:
- PyConfig.home
- PyConfig.platlibdir
- PyConfig.pathconfig_warnings
- PyConfig.program_name
- PyConfig.pythonpath_env
- current working directory: to get absolute paths
- PATH environment variable to get the program full path (from PyConfig.program_name)
- __PYVENV_LAUNCHER__ environment variable
- (Windows only) Application paths in the registry under “SoftwarePythonPythonCoreX.YPythonPath” of HKEY_CURRENT_USER and HKEY_LOCAL_MACHINE (where X.Y is the Python version).
Path configuration output fields:

If at least one “output field” is not set, Python calculates the path configuration to fill unset fields. If module_search_paths_set is equal to 0, module_search_paths is overridden and module_search_paths_set is set to 1.

It is possible to completely ignore the function calculating the default path configuration by setting explicitly all path configuration output fields listed above. A string is considered as set even if it is non-empty. module_search_paths is considered as set if module_search_paths_set is set to 1. In this case, module_search_paths will be used without modification.

Set pathconfig_warnings to 0 to suppress warnings when calculating the path configuration (Unix only, Windows does not log any warning).

If base_prefix or base_exec_prefix fields are not set, they inherit their value from prefix and exec_prefix respectively.

Py_RunMain() and Py_Main() modify sys.path:

If run_filename is set and is a directory which contains a __main__.py script, prepend run_filename to sys.path.
If isolated is zero:
- If run_module is set, prepend the current directory to sys.path. Do nothing if the current directory cannot be read.
- If run_filename is set, prepend the directory of the filename to sys.path.
- Otherwise, prepend an empty string to sys.path.

If site_import is non-zero, sys.path can be modified by the site module. If user_site_directory is non-zero and the user’s site-package directory exists, the site module appends the user’s site-package directory to sys.path.

The following configuration files are used by the path configuration:

pyvenv.cfg
._pth file (ex: python._pth)
pybuilddir.txt (Unix only)

If a ._pth file is present:

Set isolated to 1.
Set use_environment to 0.
Set site_import to 0.
Set safe_path to 1.

The __PYVENV_LAUNCHER__ environment variable is used to set PyConfig.base_executable.

Py_GetArgcArgv()¶

void Py_GetArgcArgv(int *argc, wchar_t ***argv)¶

Get the original command line arguments, before Python modified them.

Multi-Phase Initialization Private Provisional API¶

This section is a private provisional API introducing multi-phase initialization, the core feature of PEP 432:

“Core” initialization phase, “bare minimum Python”:
- Builtin types;
- Builtin exceptions;
- Builtin and frozen modules;
- The sys module is only partially initialized (ex: sys.path doesn’t exist yet).
“Main” initialization phase, Python is fully initialized:
- Install and configure importlib;
- Apply the Path Configuration;
- Install signal handlers;
- Finish sys module initialization (ex: create sys.stdout and sys.path);
- Enable optional features like faulthandler and tracemalloc;
- Import the site module;
- etc.

Private provisional API:

PyConfig._init_main: if set to 0, Py_InitializeFromConfig() stops at the “Core” initialization phase.

PyStatus _Py_InitializeMain(void)¶: Move to the “Main” initialization phase, finish the Python initialization.

No module is imported during the “Core” phase and the importlib module is not configured: the Path Configuration is only applied during the “Main” phase. It may allow to customize Python in Python to override or tune the Path Configuration, maybe install a custom sys.meta_path importer or an import hook, etc.

It may become possible to calculate the Path Configuration in Python, after the Core phase and before the Main phase, which is one of the PEP 432 motivation.

The “Core” phase is not properly defined: what should be and what should not be available at this phase is not specified yet. The API is marked as private and provisional: the API can be modified or even be removed anytime until a proper public API is designed.

Example running Python code between “Core” and “Main” initialization phases:

void init_python(void)
{
    PyStatus status;

    PyConfig config;
    PyConfig_InitPythonConfig(&config);
    config._init_main = 0;

    /* ... customize 'config' configuration ... */

    status = Py_InitializeFromConfig(&config);
    PyConfig_Clear(&config);
    if (PyStatus_Exception(status)) {
        Py_ExitStatusException(status);
    }

    /* Use sys.stderr because sys.stdout is only created
       by _Py_InitializeMain() */
    int res = PyRun_SimpleString(
        "import sys; "
        "print('Run Python code before _Py_InitializeMain', "
               "file=sys.stderr)");
    if (res < 0) {
        exit(1);
    }

    /* ... put more configuration code here ... */

    status = _Py_InitializeMain();
    if (PyStatus_Exception(status)) {
        Py_ExitStatusException(status);
    }
}

Python Initialization Configuration¶

Example¶

PyWideStringList¶

PyStatus¶

PyPreConfig¶

Preinitialize Python with PyPreConfig¶

PyConfig¶

Initialization with PyConfig¶

Isolated Configuration¶

Python Configuration¶

Python Path Configuration¶

Py_GetArgcArgv()¶

Multi-Phase Initialization Private Provisional API¶

Table of Contents

Previous topic

Next topic

This Page