You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using PythonNet in a C# application to run multiple Python scripts sequentially, each contained in different modules. However, I am encountering inconsistent MemoryAccessViolation errors, which sometimes occur during program execution, at shutdown, or even seemingly at random times. These errors do not follow a specific pattern and are difficult to reproduce reliably. PythonEnv object only provides the path to the environment.
usingPython.Runtime;usingSystem.Diagnostics;namespaceDetune.PythonBridge{/// <summary>/// Manages the Python environment using pythonnet./// </summary>publicclassPythonNetEnv:IDisposable{/// <summary>/// Indicates if the environment is initialized./// </summary>publicboolIsInitialized{get;privateset;}privatereadonlyPythonEnv_env;privatePythonEngine?_engine;privatestring?_originalPath;privatestring?_originalPythonhome;privatestaticreadonlychar[]separator=[' '];privateint_currentWriter=-1;privateDateTimelastWriteTime=DateTime.MinValue;privateCancellationTokenSource_cts=new();/// <summary>/// Event triggered when a response from Pythonnet is received./// </summary>publiceventEventHandler<ConsoleResponse>?OnPythonnetResponse;/// <summary>/// Initializes a new instance of the <see cref="PythonNetEnv"/> class./// </summary>/// <param name="pythonEnv">The path to the Python environment directory.</param>publicPythonNetEnv(PythonEnvpythonEnv){_env=pythonEnv??thrownewArgumentNullException(nameof(pythonEnv));Initialize();}/// <summary>/// Initializes the Python environment./// </summary>privatevoidInitialize(){if(IsInitialized)return;// Construct the necessary pathsstringscriptsPath=Path.Combine(_env.EnvPath,"Scripts");stringlibraryPath=Path.Combine(_env.EnvPath,"Library");stringbinPath=Path.Combine(_env.EnvPath,"bin");stringexecutablePath=Path.Combine(_env.EnvPath,"Library","bin");stringmingwBinPath=Path.Combine(_env.EnvPath,"Library","mingw-w64","bin");// Get the current PATH environment variable_originalPath=Environment.GetEnvironmentVariable("PATH");_originalPythonhome=Environment.GetEnvironmentVariable("PYTHONHOME");// Set the new PATH environment variablestringnewPath=$"{_env.EnvPath};{scriptsPath};{libraryPath};{binPath};{executablePath};{mingwBinPath};{_originalPath}";Environment.SetEnvironmentVariable("PATH",newPath,EnvironmentVariableTarget.Process);// Set the PYTHONHOME environment variableEnvironment.SetEnvironmentVariable("PYTHONHOME",_env.EnvPath,EnvironmentVariableTarget.Process);// Extract the major and minor version numbers from the provided version stringstring[]versionParts=_env.Version.Split('.');if(versionParts.Length<2)thrownewArgumentException("Invalid Python version format. Expected format: main.current.path (e.g., 3.8.20)");stringmajorVersion=versionParts[0];stringminorVersion=versionParts[1];// Construct the Python runtime DLL path based on the versionstringpythonDllPath=Path.Combine(_env.EnvPath,$"python{majorVersion}{minorVersion}.dll");// Explicitly set the Python runtime DLL pathRuntime.PythonDLL=pythonDllPath;// Set PythonEngine.PythonHomePythonEngine.PythonHome=_env.EnvPath;if(!PythonEngine.IsInitialized){// Initialize the Python engine_engine=newPythonEngine();}elseDebug.WriteLine("Exception");// Allow Python threads to run independently of the main threadPythonEngine.BeginAllowThreads();IsInitialized=true;}/// <summary>/// Runs a Python script from a specific path with specified arguments./// </summary>/// <param name="scriptPath">The path to the Python script.</param>/// <param name="workingDirectory">The working directory for the script.</param>/// <param name="arguments">The arguments to pass to the script.</param>publicasyncTaskRunPythonScript(stringscriptPath,stringworkingDirectory,stringarguments){if(!IsInitialized)thrownewInvalidOperationException("Python environment is not initialized.");awaitTask.Run(()=>{stringcurrentDictionary=Environment.CurrentDirectory;using(Py.GIL()){try{Environment.CurrentDirectory=workingDirectory;dynamicsys=Py.Import("sys");dynamicio=Py.Import("io");ValidateScriptPath(scriptPath);ConfigurePythonArguments(sys,scriptPath,arguments);stringmoduleName=Path.GetFileNameWithoutExtension(scriptPath);Debug.WriteLine($"Importing module: {moduleName}");dynamicstdout_capture=io.StringIO();dynamicstderr_capture=io.StringIO();sys.stdout=stdout_capture;sys.stderr=stderr_capture;_cts=newCancellationTokenSource();_=Task.Run(()=>CaptureOutput(stdout_capture,false,0,_cts.Token));_=Task.Run(()=>CaptureOutput(stderr_capture,true,1,_cts.Token));try{ExecutePythonScript(scriptPath);}finally{_cts.Cancel();RestoreStandardOutputs(sys,stdout_capture,stderr_capture);}}catch(PythonExceptionex){Console.WriteLine($"Python error: {ex.Message}");Console.WriteLine($"Stack Trace: {ex.StackTrace}");}catch(Exceptionex){Console.WriteLine($"Error: {ex.Message}");Console.WriteLine($"Stack Trace: {ex.StackTrace}");}}Environment.CurrentDirectory=currentDictionary;});}privatestaticvoidValidateScriptPath(stringscriptPath){if(!File.Exists(scriptPath)){Console.WriteLine($"Script file not found: {scriptPath}");thrownewFileNotFoundException($"Script file not found: {scriptPath}");}}privatestaticvoidConfigurePythonArguments(dynamicsys,stringscriptPath,stringarguments){string[]argsArray=arguments.Split(separator,StringSplitOptions.RemoveEmptyEntries);usingvarpyList=newPyList([newPyString(scriptPath), .. argsArray.Select(arg =>newPyString(arg))]);sys.argv=pyList;sys.path.append(Environment.CurrentDirectory);}privateasyncTaskCaptureOutput(dynamiccaptureStream,boolerror,intinstanceId,CancellationTokentoken){intindex=0;stringcurrentOutput="";while(!token.IsCancellationRequested){try{using(Py.GIL()){varcurrentOutputValue=captureStream.getvalue();if(currentOutputValue!=null)currentOutput=currentOutputValue.ToString();// It happens sometimes here, while the code is running. Throws an error but the try-catch does not prevent exiting of the program.}}catch(System.AccessViolationExceptionex){Debug.WriteLine(ex);}if(index==currentOutput.Length){awaitTask.Delay(20,token);continue;}boolshouldWrite=_currentWriter==-1||_currentWriter==instanceId;boolsetWrite=false;if(!shouldWrite&&(DateTime.Now-lastWriteTime).TotalMilliseconds>=20)setWrite=true;if(shouldWrite||setWrite){_currentWriter=instanceId;OnPythonnetResponse?.Invoke(this,newConsoleResponse(error,(setWrite?Environment.NewLine:"")+currentOutput[index..]));index=currentOutput.Length;lastWriteTime=DateTime.Now;}}}privatestaticvoidExecutePythonScript(stringscriptPath){using(Py.GIL()){usingdynamicscope=Py.CreateScope();scope.Set("__name__","__main__");scope.Exec(File.ReadAllText(scriptPath));// It happens sometimes here, while the code is running. Throws an error but the try-catch does not prevent exiting of the program.}}privatestaticvoidRestoreStandardOutputs(dynamicsys,dynamicstdout_capture,dynamicstderr_capture){using(Py.GIL()){sys.stdout=sys.__stdout__;sys.stderr=sys.__stderr__;try{_=stdout_capture.getvalue().ToString();_=stderr_capture.getvalue().ToString();}finally{// Dispose of the captured streams if they implement IDisposableif(stdout_captureisIDisposablestdoutDisposable){stdoutDisposable.Dispose();}if(stderr_captureisIDisposablestderrDisposable){stderrDisposable.Dispose();}}}}/// <summary>/// Disposes of the Python environment, shutting down the Python engine./// </summary>publicvoidDispose(){if(!IsInitialized)return;try{_cts.Dispose();}catch(Exception){}try{Debug.WriteLine("Shutdown");PythonEngine.Shutdown();// Somewhere here does it happen at most. The catch does not work._engine?.Dispose();}catch(Exceptionex){Debug.WriteLine(ex);}finally{Environment.SetEnvironmentVariable("PATH",_originalPath,EnvironmentVariableTarget.Process);Environment.SetEnvironmentVariable("PYTHONHOME",_originalPythonhome,EnvironmentVariableTarget.Process);}}}}
If there was a crash, please include the traceback here:
There is no traceback the code is only exiting in Visual Studio
Additional Details:
The errors do not happen on the first or second call but usually occur on the third call.
I create a PythonNet object, let it run, dispose it, and start a new PythonNet object.
I have marked the lines in the code where the errors happen, but I don't understand why it works sometimes 4 times and sometimes only once. This is completely random.
Sometimes, the program ends with an access violation error code without throwing any errors.
The program runs to the end and then shuts down, where it most often exits with access violations.
Questions:
How can I properly reset or restart PythonNet between different Python script executions to avoid these memory access errors?
Is there a way to ensure that each Python script runs in a completely isolated environment within PythonNet?
What might be causing these random access violations, and how can I prevent them?
I am grateful for any suggestions or insights into resolving this issue.
This discussion was converted from issue #2516 on November 22, 2024 09:34.
Heading
Bold
Italic
Quote
Code
Link
Numbered list
Unordered list
Task list
Attach files
Mention
Reference
Menu
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Environment
Details
Describe what you were trying to get done:
I am using PythonNet in a C# application to run multiple Python scripts sequentially, each contained in different modules. However, I am encountering inconsistent
MemoryAccessViolation
errors, which sometimes occur during program execution, at shutdown, or even seemingly at random times. These errors do not follow a specific pattern and are difficult to reproduce reliably. PythonEnv object only provides the path to the environment.What commands did you run to trigger this issue? If you can provide a Minimal, Complete, and Verifiable example this will help us understand the issue:
There is no traceback the code is only exiting in Visual Studio
Additional Details:
Questions:
I am grateful for any suggestions or insights into resolving this issue.
Beta Was this translation helpful? Give feedback.
All reactions