Please note that this article is published incomplete, so complete details can be found here.
Check the relevant ticket here.
We’ve recently run an investigation made internally that involves the Process class and its UseShellExecute feature. This is an investigation about a pitfall when debugging the StartInfo property of a process that is to be executed from the Process class with the UseShellExecute feature enabled.
Before going into the details of this investigation, we’ll explain a bit about what is UseShellExecute. This property in the Process class describes whether the operating system shell (ShellExecute in Windows) is to be used when executing processes or the operating system should create a process directly from the executable file. This is available for all platforms, despite the usage of the “ShellExecute” name in the property.
The Investigation
We’re going straight to the investigation details and a case study about why debugging the StartInfo property in UseShellExecute-enabled process instances causes a confusing exception.
Symptoms
This investigation is a case study about the above subject. Suppose that you have this block of code that defines a process class that re-executes your .NET 6.0 application with the “runas” verb to execute it as an elevated user account.
var selfProcess = new Process
{
StartInfo = new(Path.ChangeExtension(Assembly.GetExecutingAssembly().Location, ".exe"))
{
UseShellExecute = true,
Verb = "runas",
Arguments = string.Join(" ", EnvironmentTools.arguments)
},
};
selfProcess.StartInfo = ProcessExecutor.StripEnvironmentVariables(selfProcess.StartInfo);
// Now, go ahead and start.
selfProcess.Start();
Normally, you’d check that all the parameters work as expected, such as taking a look at the StartInfo property value by putting it to a watch when debugging your program.
However, when you actually try to start the process after taking a look at that property and making sure that all the values are populated as expected, instead of your program executing the process, you’ll actually notice that your program throws an exception as seen in the below screenshot.
If you can’t see the screenshot, here’s a text version of the exception.
Investigation
Our verification using the example POC app, which its source code is short, verifies that this exception only gets thrown on Windows systems. Linux systems are unaffected. Here’s the source code:
Invocation of the Environment or the EnvironmentVariables property from the StartInfo property causes the above exception, and Google searches won’t help pinpoint the problem.
Further investigation of the .NET source code suggests that these properties are populated upon the first invocation of any of the two properties, as you can see here (taken from the .NET 8.0 source code):
The property getter doesn’t check the UseShellExecute property before populating the environment variable, which causes the environment variable lists to be populated, which is inappropriate for process instances with UseShellExecute enabled.
In our opinion, only checking for null in the Environment property getter is insufficient. Furthermore, when Start() is called on the process after this property is populated, the abstraction of the Process class, depending on your platform, is called to call the platform-specific methods of process starting function.
Investigating the Unix abstraction of the Process class, this piece of code snippet below only checks for redirection when ShellExecute is enabled.
However, when it comes to the Windows abstraction of the Process class, the StartCore checks to see if ShellExecute is used. In our case, StartWithShellExecuteEx() is called, and does the following checks:
- In line 32, if the username or the password is provided, the exception is thrown.
- In line 35, if any redirection is enabled, the exception is thrown.
- In line 38, line 41, and line 44, if you’re attempting to set the encoding to the process, the exception is thrown.
- In line 47, if the environment variable list is populated, intentionally (as in the POC) or not (as in StartInfo debugging), the exception is thrown.
Workaround
There is no way to nullify the internal environment variable list as pinpointed in line 47 except using the dirty private reflection hack. However, this hack lasts until you try to access the environment variables property again. This is the reflection hack used to nullify this field (taken from the source code of Nitrocid KS 0.1.0):
After nullifying this list, Start() works again. In our opinion, an exception should be thrown when UseShellExecute is enabled and an attempt to get environment variables is tried to reduce confusing errors.
Enjoy hacking!
Aptivi 
Thoughts?