{"id":12966,"date":"2023-09-27T06:58:49","date_gmt":"2023-09-27T06:58:49","guid":{"rendered":"https:\/\/www.digitaldesignjournal.com\/?p=12966"},"modified":"2023-09-27T17:35:35","modified_gmt":"2023-09-27T17:35:35","slug":"python-subprocess-tutorial","status":"publish","type":"post","link":"https:\/\/www.digitaldesignjournal.com\/python-subprocess-tutorial\/","title":{"rendered":"Python Subprocess Tutorial [Beginner To Advanced]"},"content":{"rendered":"\n<p>The Python <code>subprocess<\/code> module is a powerful tool that allows developers to interact with external processes and execute system commands from within their Python programs. <\/p>\n\n\n\n<p>It provides a way to create, manage, and communicate with subprocesses, making it an essential part of Python&#8217;s standard library for system administration, automation, and various other tasks.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Mastering Python Subprocess: A Comprehensive Tutorial<\/h2>\n\n\n\n<p>This comprehensive tutorial will walk you through the fundamentals of using the <code>subprocess<\/code> module in Python, starting with the basics and gradually diving into more advanced topics. <\/p>\n\n\n\n<p>Whether you need to run simple shell commands, manage complex pipelines of processes, or interact with long-running external programs, this tutorial will equip you with the knowledge and skills to do so effectively.<\/p>\n\n\n\n<p>Throughout this tutorial, you will learn how to:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Run simple commands and capture their output.<\/li>\n\n\n\n<li>Handle input and output streams for subprocesses.<\/li>\n\n\n\n<li>Manage errors and exceptions when working with external processes.<\/li>\n\n\n\n<li>Redirect input and output to and from files.<\/li>\n\n\n\n<li>Set environment variables for subprocesses.<\/li>\n\n\n\n<li>Handle timeouts and asynchronous subprocess execution.<\/li>\n\n\n\n<li>Create pipelines of processes for data processing.<\/li>\n\n\n\n<li>Send signals to running processes for control and termination.<\/li>\n\n\n\n<li>Deal with platform-specific considerations.<\/li>\n<\/ol>\n\n\n\n<p>By the end of this tutorial, you&#8217;ll have a solid understanding of how to harness the full potential of the <code>subprocess<\/code> module in Python, enabling you to automate tasks, integrate with external tools and services, and perform system-level operations with confidence.<\/p>\n\n\n\n<p>Let&#8217;s dive into the world of Python subprocesses and unlock a new realm of possibilities for your Python programming journey.<\/p>\n\n\n\n<p>Here&#8217;s a tutorial on how to use the <code>subprocess<\/code> module effectively:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">1. Import the <code>subprocess<\/code> module<\/h3>\n\n\n\n<p>First, import the <code>subprocess<\/code> module:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Learn More: <\/strong><a href=\"https:\/\/www.digitaldesignjournal.com\/what-is-subprocess-in-python\/\">What Is Subprocess In Python?&nbsp;<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Running a Simple Command<\/h3>\n\n\n\n<p>You can run a simple command by using the <code>subprocess.run()<\/code> function. For example, to run the <code>ls<\/code> command to list files in a directory:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\">result = subprocess.run(&#91;<span class=\"hljs-string\">\"ls\"<\/span>, <span class=\"hljs-string\">\"-l\"<\/span>])\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>This will execute the <code>ls -l<\/code> command and store the result in the <code>result<\/code> variable. You can access the return code, standard output, and standard error like this:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-3\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\">print(<span class=\"hljs-string\">\"Return Code:\"<\/span>, result.returncode)\nprint(<span class=\"hljs-string\">\"Standard Output:\"<\/span>, result.stdout)\nprint(<span class=\"hljs-string\">\"Standard Error:\"<\/span>, result.stderr)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-3\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Learn More: <\/strong><a href=\"https:\/\/www.digitaldesignjournal.com\/subprocess-run\/\">Learn Subprocess.run() in Python [Step-by-Step Examples]<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. Handling Input and Output<\/h3>\n\n\n\n<p>You can also pass input to the command and capture its output. For example, let&#8217;s run a simple Python script that reads input from standard input and prints it to standard output:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-4\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\">input_data = <span class=\"hljs-string\">\"Hello, subprocess!\"<\/span>\nresult = subprocess.run(&#91;<span class=\"hljs-string\">\"python\"<\/span>, <span class=\"hljs-string\">\"-c\"<\/span>, <span class=\"hljs-string\">\"print(input())\"<\/span>], input=input_data.encode(), text=<span class=\"hljs-literal\">True<\/span>, capture_output=<span class=\"hljs-literal\">True<\/span>)\n\nprint(<span class=\"hljs-string\">\"Return Code:\"<\/span>, result.returncode)\nprint(<span class=\"hljs-string\">\"Standard Output:\"<\/span>, result.stdout)\nprint(<span class=\"hljs-string\">\"Standard Error:\"<\/span>, result.stderr)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-4\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>In this example, we use the <code>input<\/code> parameter to provide input data as bytes, the <code>text<\/code> parameter to specify that the data is in text format, and the <code>capture_output<\/code> parameter to capture the standard output.<\/p>\n\n\n\n<p><strong>Learn More: <\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-output-to-variable\/\">Python Subprocess Output To Variable [Explained With Example]<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-output-to-file\/\">Python Subprocess Output To File [Explained]<\/a><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">4. Running Shell Commands<\/h3>\n\n\n\n<p>You can also run shell commands by setting the <code>shell<\/code> parameter to <code>True<\/code>. However, be cautious about shell injection vulnerabilities if you&#8217;re including user-provided input in the command.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-5\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\">command = <span class=\"hljs-string\">\"echo Hello, subprocess!\"<\/span>\nresult = subprocess.run(command, shell=<span class=\"hljs-literal\">True<\/span>, text=<span class=\"hljs-literal\">True<\/span>, capture_output=<span class=\"hljs-literal\">True<\/span>)\n\nprint(<span class=\"hljs-string\">\"Return Code:\"<\/span>, result.returncode)\nprint(<span class=\"hljs-string\">\"Standard Output:\"<\/span>, result.stdout)\nprint(<span class=\"hljs-string\">\"Standard Error:\"<\/span>, result.stderr)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-5\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Learn More: <\/strong><a href=\"https:\/\/www.digitaldesignjournal.com\/python-execute-shell-command-and-get-output\/\">Python Execute Shell Command And Get Output<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">5. Handling Errors<\/h3>\n\n\n\n<p>To check if the command executed successfully, you can examine the return code. Typically, a return code of 0 indicates success, while non-zero values indicate errors.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-6\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">if<\/span> result.returncode == <span class=\"hljs-number\">0<\/span>:\n    print(<span class=\"hljs-string\">\"Command executed successfully\"<\/span>)\n<span class=\"hljs-keyword\">else<\/span>:\n    print(<span class=\"hljs-string\">\"Command failed with return code:\"<\/span>, result.returncode)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-6\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">6. Advanced Features<\/h3>\n\n\n\n<p>The <code>subprocess<\/code> module offers many more features for more complex scenarios, such as redirecting input\/output, setting environment variables, and handling timeouts. <\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">6.1. Redirecting Input\/Output\/Error<\/h3>\n\n\n\n<p>You can control where the standard input, standard output, and standard error of a subprocess are directed. Use the <code>stdin<\/code>, <code>stdout<\/code>, and <code>stderr<\/code> parameters when creating a subprocess.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-7\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\n<span class=\"hljs-keyword\">with<\/span> open(<span class=\"hljs-string\">\"output.txt\"<\/span>, <span class=\"hljs-string\">\"w\"<\/span>) <span class=\"hljs-keyword\">as<\/span> outfile:\n    subprocess.run(&#91;<span class=\"hljs-string\">\"echo\"<\/span>, <span class=\"hljs-string\">\"Hello, subprocess!\"<\/span>], stdout=outfile)\n\n<span class=\"hljs-keyword\">with<\/span> open(<span class=\"hljs-string\">\"input.txt\"<\/span>, <span class=\"hljs-string\">\"r\"<\/span>) <span class=\"hljs-keyword\">as<\/span> infile:\n    subprocess.run(&#91;<span class=\"hljs-string\">\"cat\"<\/span>], stdin=infile)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-7\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>In this example, we redirect the standard output of the <code>echo<\/code> command to a file called &#8220;output.txt&#8221; and provide input to the <code>cat<\/code> command from &#8220;input.txt&#8221;.<\/p>\n\n\n\n<p><strong>Learn More: <\/strong><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-stdin\/\">Python Subprocess\u2019 Stdin [Full Guide With Examples]<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.2. Setting Environment Variables<\/h3>\n\n\n\n<p>You can set environment variables for the subprocess using the <code>env<\/code> parameter. This is useful when you need to customize the environment in which the subprocess runs.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-8\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\ncustom_env = {<span class=\"hljs-string\">\"CUSTOM_VARIABLE\"<\/span>: <span class=\"hljs-string\">\"custom_value\"<\/span>}\nresult = subprocess.run(&#91;<span class=\"hljs-string\">\"python\"<\/span>, <span class=\"hljs-string\">\"-c\"<\/span>, <span class=\"hljs-string\">\"import os; print(os.getenv('CUSTOM_VARIABLE'))\"<\/span>], env=custom_env, text=<span class=\"hljs-literal\">True<\/span>, capture_output=<span class=\"hljs-literal\">True<\/span>)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-8\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>Here, we set the <code>CUSTOM_VARIABLE<\/code> environment variable for the subprocess and use it within a Python script.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.3. Handling Timeouts<\/h3>\n\n\n\n<p>You can specify a timeout for the subprocess using the <code>timeout<\/code> parameter. If the subprocess doesn&#8217;t complete within the specified time, a <code>TimeoutExpired<\/code> exception is raised.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-9\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\n<span class=\"hljs-keyword\">try<\/span>:\n    result = subprocess.run(&#91;<span class=\"hljs-string\">\"sleep\"<\/span>, <span class=\"hljs-string\">\"5\"<\/span>], timeout=<span class=\"hljs-number\">3<\/span>, text=<span class=\"hljs-literal\">True<\/span>, capture_output=<span class=\"hljs-literal\">True<\/span>)\n<span class=\"hljs-keyword\">except<\/span> subprocess.TimeoutExpired:\n    print(<span class=\"hljs-string\">\"Command timed out\"<\/span>)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-9\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>In this example, we attempt to run the <code>sleep<\/code> command for 5 seconds but set a timeout of 3 seconds. The code will catch the <code>TimeoutExpired<\/code> exception when the command takes too long to complete.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.4. Piping Between Processes<\/h3>\n\n\n\n<p>You can create pipelines of processes by connecting them using pipes. For example, you can pipe the output of one command to another using the <code>stdout<\/code> of one subprocess as the <code>stdin<\/code> of another.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-10\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\np1 = subprocess.Popen(&#91;<span class=\"hljs-string\">\"ls\"<\/span>, <span class=\"hljs-string\">\"-l\"<\/span>], stdout=subprocess.PIPE)\np2 = subprocess.Popen(&#91;<span class=\"hljs-string\">\"grep\"<\/span>, <span class=\"hljs-string\">\".txt\"<\/span>], stdin=p1.stdout, stdout=subprocess.PIPE, text=<span class=\"hljs-literal\">True<\/span>)\n\noutput = p2.communicate()&#91;<span class=\"hljs-number\">0<\/span>]\nprint(output)\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-10\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>In this example, we list files in a directory with <code>ls -l<\/code> and then pipe the output to <code>grep<\/code> to filter for <code>.txt<\/code> files.<\/p>\n\n\n\n<p><strong>Learn More: <\/strong><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-pipe-with-example\/\">Python Subprocess Pipe With Example<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.5. Handling Exceptions<\/h3>\n\n\n\n<p>Make sure to handle exceptions that might occur during subprocess execution. Common exceptions include <code>subprocess.CalledProcessError<\/code>, <code>FileNotFoundError<\/code>, and <code>PermissionError<\/code>. Robust error handling ensures that your program doesn&#8217;t crash unexpectedly.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-11\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\n<span class=\"hljs-keyword\">try<\/span>:\n    result = subprocess.run(&#91;<span class=\"hljs-string\">\"non_existent_command\"<\/span>], check=<span class=\"hljs-literal\">True<\/span>, text=<span class=\"hljs-literal\">True<\/span>, capture_output=<span class=\"hljs-literal\">True<\/span>)\n<span class=\"hljs-keyword\">except<\/span> subprocess.CalledProcessError <span class=\"hljs-keyword\">as<\/span> e:\n    print(<span class=\"hljs-string\">\"Command failed with error:\"<\/span>, e)\n<span class=\"hljs-keyword\">except<\/span> FileNotFoundError <span class=\"hljs-keyword\">as<\/span> e:\n    print(<span class=\"hljs-string\">\"Command not found:\"<\/span>, e)\n<span class=\"hljs-keyword\">except<\/span> PermissionError <span class=\"hljs-keyword\">as<\/span> e:\n    print(<span class=\"hljs-string\">\"Permission denied:\"<\/span>, e)<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-11\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>These are some of the advanced features of the <code>subprocess<\/code> module. Depending on your specific use case, you may need to use different combinations of these features to achieve your desired functionality when interacting with external processes in Python.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.6. Running Background Processes<\/h3>\n\n\n\n<p>If you want to run a subprocess in the background and continue with your program without waiting for it to finish, you can use the <code>subprocess.Popen<\/code> class. This class allows you to start a process and continue with other tasks.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-12\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\n<span class=\"hljs-comment\"># Start a background process<\/span>\nbackground_process = subprocess.Popen(&#91;<span class=\"hljs-string\">\"python\"<\/span>, <span class=\"hljs-string\">\"my_script.py\"<\/span>])\n\n<span class=\"hljs-comment\"># Continue with other tasks<\/span>\n\n<span class=\"hljs-comment\"># Optionally, you can wait for the background process to finish later if needed<\/span>\nbackground_process.wait()\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-12\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Learn More: <\/strong><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-run-in-background-with-example\/\">Python Subprocess Run In Background [With Example]<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.7. Communicating with Long-Running Processes<\/h3>\n\n\n\n<p>For long-running processes where you need to interact with the subprocess during its execution, you can use <code>subprocess.Popen<\/code> in combination with <code>communicate()<\/code> to send input and receive output interactively.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-13\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\nprocess = subprocess.Popen(&#91;<span class=\"hljs-string\">\"python\"<\/span>, <span class=\"hljs-string\">\"-u\"<\/span>, <span class=\"hljs-string\">\"interactive_script.py\"<\/span>], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=<span class=\"hljs-literal\">True<\/span>)\n\n<span class=\"hljs-comment\"># Send input to the subprocess<\/span>\nprocess.stdin.write(<span class=\"hljs-string\">\"Hello\\n\"<\/span>)\nprocess.stdin.flush()\n\n<span class=\"hljs-comment\"># Read output from the subprocess<\/span>\noutput = process.stdout.readline()\nprint(output)\n\n<span class=\"hljs-comment\"># Optionally, wait for the subprocess to finish<\/span>\nprocess.wait()\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-13\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p><strong>Learn More:<\/strong> <a href=\"https:\/\/www.digitaldesignjournal.com\/subprocess-communicate\/\">How to use the subprocess Popen.communicate() method<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.8. Cross-Platform Considerations<\/h3>\n\n\n\n<p>Keep in mind that certain commands and behaviors may vary across different operating systems (Windows, Linux, macOS). Be aware of these platform-specific differences when using <code>subprocess<\/code>. You can use the <code>platform<\/code> module to detect the current operating system and adjust your code accordingly.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-14\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> platform\n\n<span class=\"hljs-keyword\">if<\/span> platform.system() == <span class=\"hljs-string\">\"Windows\"<\/span>:\n    <span class=\"hljs-comment\"># Windows-specific code<\/span>\n    <span class=\"hljs-keyword\">pass<\/span>\n<span class=\"hljs-keyword\">elif<\/span> platform.system() == <span class=\"hljs-string\">\"Linux\"<\/span>:\n    <span class=\"hljs-comment\"># Linux-specific code<\/span>\n    <span class=\"hljs-keyword\">pass<\/span>\n<span class=\"hljs-keyword\">elif<\/span> platform.system() == <span class=\"hljs-string\">\"Darwin\"<\/span>:\n    <span class=\"hljs-comment\"># macOS-specific code<\/span>\n    <span class=\"hljs-keyword\">pass<\/span>\n<span class=\"hljs-keyword\">else<\/span>:\n    <span class=\"hljs-comment\"># Code for other operating systems<\/span>\n    <span class=\"hljs-keyword\">pass<\/span>\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-14\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<h3 class=\"wp-block-heading\">6.9. Asynchronous Subprocesses<\/h3>\n\n\n\n<p>You can use Python&#8217;s <code>asyncio<\/code> library to run subprocesses asynchronously. This is particularly useful when you want to run multiple subprocesses concurrently and efficiently handle their outputs.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-15\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> asyncio\n<span class=\"hljs-keyword\">import<\/span> subprocess\n\n<span class=\"hljs-keyword\">async<\/span> <span class=\"hljs-function\"><span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title\">run_command<\/span><span class=\"hljs-params\">(command)<\/span>:<\/span>\n    process = <span class=\"hljs-keyword\">await<\/span> asyncio.create_subprocess_shell(command, stdout=asyncio.subprocess.PIPE, text=<span class=\"hljs-literal\">True<\/span>)\n    output, _ = <span class=\"hljs-keyword\">await<\/span> process.communicate()\n    <span class=\"hljs-keyword\">return<\/span> output\n\n<span class=\"hljs-keyword\">async<\/span> <span class=\"hljs-function\"><span class=\"hljs-keyword\">def<\/span> <span class=\"hljs-title\">main<\/span><span class=\"hljs-params\">()<\/span>:<\/span>\n    command1 = <span class=\"hljs-string\">\"ls -l\"<\/span>\n    command2 = <span class=\"hljs-string\">\"echo 'Hello, subprocess!'\"<\/span>\n\n    result1 = <span class=\"hljs-keyword\">await<\/span> run_command(command1)\n    result2 = <span class=\"hljs-keyword\">await<\/span> run_command(command2)\n\n    print(result1)\n    print(result2)\n\n<span class=\"hljs-keyword\">if<\/span> __name__ == <span class=\"hljs-string\">\"__main__\"<\/span>:\n    asyncio.run(main())\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-15\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>This example uses <code>asyncio<\/code> to run two subprocesses concurrently and asynchronously.<\/p>\n\n\n\n<p><strong>Learn More: <\/strong><a href=\"https:\/\/www.digitaldesignjournal.com\/python-asyncio-subprocess\/\">Python Asyncio Subprocess<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.10. Interacting with Streams<\/h3>\n\n\n\n<p>When dealing with processes that produce continuous streams of data, such as real-time logs or live data feeds, you can use the <code>stdout<\/code> and <code>stderr<\/code> streams to read data as it&#8217;s produced, rather than waiting for the process to complete.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-16\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\nprocess = subprocess.Popen(&#91;<span class=\"hljs-string\">\"tail\"<\/span>, <span class=\"hljs-string\">\"-f\"<\/span>, <span class=\"hljs-string\">\"my_log_file.log\"<\/span>], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=<span class=\"hljs-literal\">True<\/span>)\n\n<span class=\"hljs-keyword\">while<\/span> <span class=\"hljs-literal\">True<\/span>:\n    line = process.stdout.readline()\n    <span class=\"hljs-keyword\">if<\/span> <span class=\"hljs-keyword\">not<\/span> line:\n        <span class=\"hljs-keyword\">break<\/span>\n    print(<span class=\"hljs-string\">\"stdout:\"<\/span>, line.strip())\n\n    error_line = process.stderr.readline()\n    <span class=\"hljs-keyword\">if<\/span> <span class=\"hljs-keyword\">not<\/span> error_line:\n        <span class=\"hljs-keyword\">break<\/span>\n    print(<span class=\"hljs-string\">\"stderr:\"<\/span>, error_line.strip())\n\nprocess.wait()\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-16\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>This code continuously reads from both <code>stdout<\/code> and <code>stderr<\/code> streams of a subprocess.<\/p>\n\n\n\n<p><strong>Learn More:<\/strong> <a href=\"https:\/\/www.digitaldesignjournal.com\/subprocess-python-stdout\/\">Subprocess Python Stdout [In-Depth Tutorial]<\/a><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.11. Handling Signals<\/h3>\n\n\n\n<p>You can send signals to running processes using the <code>send_signal()<\/code> method of a <code>Popen<\/code> object. For example, to send the <code>SIGTERM<\/code> signal to gracefully terminate a process:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-17\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n<span class=\"hljs-keyword\">import<\/span> signal\n<span class=\"hljs-keyword\">import<\/span> time\n\nprocess = subprocess.Popen(&#91;<span class=\"hljs-string\">\"python\"<\/span>, <span class=\"hljs-string\">\"my_long_running_script.py\"<\/span>])\n\n<span class=\"hljs-comment\"># Allow some time for the process to run<\/span>\ntime.sleep(<span class=\"hljs-number\">5<\/span>)\n\n<span class=\"hljs-comment\"># Send a SIGTERM signal to gracefully terminate the process<\/span>\nprocess.send_signal(signal.SIGTERM)\n\n<span class=\"hljs-comment\"># Wait for the process to complete<\/span>\nprocess.wait()\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-17\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>This allows you to control and manage the lifecycle of subprocesses more precisely.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">6.12. Creating Custom Shells<\/h3>\n\n\n\n<p>You can create custom shells to execute multiple commands within the same shell environment. This is useful for complex sequences of commands.<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-18\" data-shcb-language-name=\"Python\" data-shcb-language-slug=\"python\"><span><code class=\"hljs language-python\"><span class=\"hljs-keyword\">import<\/span> subprocess\n\ncommands = &#91;\n    <span class=\"hljs-string\">\"echo 'Step 1'\"<\/span>,\n    <span class=\"hljs-string\">\"echo 'Step 2'\"<\/span>,\n    <span class=\"hljs-string\">\"echo 'Step 3'\"<\/span>,\n]\n\nshell_command = <span class=\"hljs-string\">\" &amp;&amp; \"<\/span>.join(commands)\n\nprocess = subprocess.Popen(shell_command, shell=<span class=\"hljs-literal\">True<\/span>, text=<span class=\"hljs-literal\">True<\/span>, stdout=subprocess.PIPE)\n\n<span class=\"hljs-keyword\">for<\/span> line <span class=\"hljs-keyword\">in<\/span> process.stdout:\n    print(line.strip())\n\nprocess.wait()\n<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-18\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">Python<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">python<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p>In this example, the <code>&amp;&amp;<\/code> operator is used to run multiple commands within the same shell session.<\/p>\n\n\n\n<p><strong>Learn More:<\/strong> <a href=\"https:\/\/www.digitaldesignjournal.com\/python-execute-shell-command-and-get-output\/\">Python Execute Shell Command And Get Output [In-Depth Guide]<\/a><\/p>\n\n\n\n<p>These advanced topics should give you more control and flexibility when working with the <code>subprocess<\/code> module in Python, especially in complex scenarios or when dealing with more challenging subprocess interactions.<\/p>\n\n\n\n<p>The official Python documentation for <code>subprocess<\/code> is a valuable resource for diving deeper into these topics: <a href=\"https:\/\/docs.python.org\/3\/library\/subprocess.html\" rel=\"nofollow noopener\" target=\"_blank\">https:\/\/docs.python.org\/3\/library\/subprocess.html<\/a><\/p>\n\n\n\n<p><strong>Read More;<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-detach-subprocess-and-exit-with-examples\/\">Python Detach Subprocess And Exit<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/subprocess-communicate\/\">How to use the subprocess Popen.communicate() method<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-output-to-file\/\">Python Subprocess Output To File<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-output-to-variable\/\">Python Subprocess Output To Variable<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/multiple-commands-with-ssh-using-python-subprocess\/\">Multiple Commands With SSH Using Python Subprocess<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-run-in-background-with-example\/\">Python Subprocess Run In Background<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-execute-shell-command-and-get-output\/\">Python Execute Shell Command And Get Output<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/face-recognition-based-attendance-system-using-python\/\">Face Recognition Based Attendance System Using Python<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-pipe-with-example\/\">Python Subprocess Pipe With Example<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-interactive\/\">Python Subprocess Interactive<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-awk-with-example\/\">Python Subprocess AWK<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.digitaldesignjournal.com\/what-is-subprocess-wait-with-example\/\">What is subprocess.wait() With Example<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>The Python subprocess module is a powerful tool that allows developers to interact with external processes and execute system commands &#8230; <a title=\"Python Subprocess Tutorial [Beginner To Advanced]\" class=\"read-more\" href=\"https:\/\/www.digitaldesignjournal.com\/python-subprocess-tutorial\/\" aria-label=\"More on Python Subprocess Tutorial [Beginner To Advanced]\">Read more<\/a><\/p>\n","protected":false},"author":14,"featured_media":12974,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[92],"tags":[154],"ppma_author":[155],"class_list":["post-12966","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-python","tag-subprocess-in-python","author-aniket-singh"],"authors":[{"term_id":155,"user_id":14,"is_guest":0,"slug":"aniket-singh","display_name":"Aniket Singh","avatar_url":{"url":"https:\/\/www.digitaldesignjournal.com\/wp-content\/uploads\/2023\/09\/Aniket_Singh.png","url2x":"https:\/\/www.digitaldesignjournal.com\/wp-content\/uploads\/2023\/09\/Aniket_Singh.png"},"0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/posts\/12966","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/comments?post=12966"}],"version-history":[{"count":16,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/posts\/12966\/revisions"}],"predecessor-version":[{"id":13013,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/posts\/12966\/revisions\/13013"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/media\/12974"}],"wp:attachment":[{"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/media?parent=12966"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/categories?post=12966"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/tags?post=12966"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.digitaldesignjournal.com\/wp-json\/wp\/v2\/ppma_author?post=12966"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}