{"id":2350,"date":"2021-04-13T14:17:30","date_gmt":"2021-04-13T12:17:30","guid":{"rendered":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/?p=2350"},"modified":"2021-06-10T15:07:32","modified_gmt":"2021-06-10T13:07:32","slug":"python-subprocess","status":"publish","type":"post","link":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/operating-system\/python-subprocess","title":{"rendered":"Python Subprocess: Run External Commands"},"content":{"rendered":"\n<p>Despite the many libraries on PyPI, sometimes you need to run an external command from your Python code. The built-in Python subprocess module makes this relatively easy. In this article, you&#8217;ll learn some basics about processes and sub-processes.<\/p>\n\n\n\n<p>We&#8217;ll use the Python subprocess module to safely execute external commands, capture the output, and optionally feed them with input from standard in. If you&#8217;re familiar with the theory on processes and sub-processes, you can safely skip the first section.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_17 counter-hierarchy counter-decimal ez-toc-transparent\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\">Table of contents<\/p>\n<span class=\"ez-toc-title-toggle\"><\/span><\/div>\n<nav><ul class=\"ez-toc-list ez-toc-list-level-1\"><ul class=\"ez-toc-list-level-2\"><li class=\"ez-toc-heading-level-2\"><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#Processes_and_sub-processes\" title=\"Processes and sub-processes\">Processes and sub-processes<\/a><\/li><li class=\"ez-toc-page-1 ez-toc-heading-level-2\"><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#Create_a_Python_subprocess_with_subprocessrun\" title=\"Create a Python subprocess with subprocess.run\">Create a Python subprocess with subprocess.run<\/a><\/li><li class=\"ez-toc-page-1 ez-toc-heading-level-2\"><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#Capture_output_of_a_Python_subprocess\" title=\"Capture output of a Python subprocess\">Capture output of a Python subprocess<\/a><\/li><li class=\"ez-toc-page-1 ez-toc-heading-level-2\"><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#Feeding_data_from_standard_input\" title=\"Feeding data from standard input\">Feeding data from standard input<\/a><\/li><li class=\"ez-toc-page-1 ez-toc-heading-level-2\"><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#Running_shell_commands\" title=\"Running shell commands\">Running shell commands<\/a><\/li><li class=\"ez-toc-page-1 ez-toc-heading-level-2\"><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#Caveats_to_look_out_for\" title=\"Caveats to look out for\">Caveats to look out for<\/a><\/li><\/ul><\/li><li class=\"ez-toc-page-1 ez-toc-heading-level-1\"><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#User_input_is_always_dangerous\" title=\"User input is always dangerous\">User input is always dangerous<\/a><ul class=\"ez-toc-list-level-2\"><li class=\"ez-toc-heading-level-2\"><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\/#Keep_learning\" title=\"Keep learning\">Keep learning<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h2 id=\"h-processes-and-sub-processes\"><span class=\"ez-toc-section\" id=\"Processes_and_sub-processes\"><\/span>Processes and sub-processes<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>A program that is executed on a computer is also called a process. But what is a process, exactly? Let&#8217;s define it more formally:<\/p>\n\n\n\n<dl class=\"wp-block-simple-definition-list-blocks-list\">\n<dt class=\"wp-block-simple-definition-list-blocks-term\"><strong>Process<\/strong><\/dt>\n\n\n\n<dd class=\"wp-block-simple-definition-list-blocks-details\">A process is the&nbsp;instance&nbsp;of a&nbsp;computer program&nbsp;that is being executed by one or more <a href=\"https:\/\/python.land\/python-concurrency\/python-threads\">threads<\/a>.<\/dd>\n<\/dl>\n\n\n\n<p>A process can have multiple <a href=\"https:\/\/python.land\/python-concurrency\/python-threads\">threads<\/a>, this is called multi-threading. In turn, a computer can run multiple processes at once. These processes can be different programs, but they can also be multiple instances of the same program. In our article on concurrency with Python, this is explained in great detail. The following images come from that article, too:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" width=\"514\" height=\"240\" src=\"https:\/\/python.land\/wp-content\/uploads\/2020\/12\/multithreading.png\" alt=\"\" class=\"wp-image-358\" srcset=\"https:\/\/python.land\/wp-content\/uploads\/2020\/12\/multithreading.png 514w, https:\/\/python.land\/wp-content\/uploads\/2020\/12\/multithreading-300x140.png 300w\" sizes=\"(max-width: 514px) 100vw, 514px\" \/><figcaption>A computer can run multiple processes at once, and a process can have multiple threads<\/figcaption><\/figure><\/div>\n\n\n\n<p>If you want to run an external command, it means you need to create a new process from your Python process. Such a process is often called a child process or a sub-process. Visually, this is what happens when one process spawns two sub-processes:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" width=\"389\" height=\"298\" src=\"https:\/\/python.land\/wp-content\/uploads\/2020\/12\/multiprocessing.png\" alt=\"\" class=\"wp-image-359\" srcset=\"https:\/\/python.land\/wp-content\/uploads\/2020\/12\/multiprocessing.png 389w, https:\/\/python.land\/wp-content\/uploads\/2020\/12\/multiprocessing-300x230.png 300w\" sizes=\"(max-width: 389px) 100vw, 389px\" \/><figcaption>A parent process spawning two sub-processes<\/figcaption><\/figure><\/div>\n\n\n\n<p>What happens internally (inside the OS kernel) is what&#8217;s called a fork. The process forks itself, meaning a new copy of the process is created and started. This can be useful if you want to parallelize your code and utilize multiple CPUs on your machine. That&#8217;s what we call <a href=\"https:\/\/python.land\/python-concurrency\/python-multiprocessing\">multiprocessing<\/a>.<\/p>\n\n\n\n<p>We can utilize this same technique to start another process, though. First, the process forks itself, creating a copy. That copy, in turn, replaces itself with another process: the process you were looking to execute.<\/p>\n\n\n\n<p>We can go the low-level way and do much of this ourselves using the Python subprocess module, but luckily, Python also offers a wrapper that will take care of all the nitty-gritty details, and do so safely too. Thanks to the wrapper, running an external command comes down to calling a function. This wrapper is the function <code>run()<\/code> from the <code>subprocess <\/code>library and that&#8217;s what we&#8217;ll use in this article.<\/p>\n\n\n\n<p>I thought it would be nice for you to know what&#8217;s going on internally, but if you feel confused, rest assured that you don&#8217;t need this knowledge to do what you want: running an external command with the Python subprocess module.<\/p>\n\n\n\n<h2><span class=\"ez-toc-section\" id=\"Create_a_Python_subprocess_with_subprocessrun\"><\/span>Create a Python subprocess with subprocess.run<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Enough with the theory, it&#8217;s time to get our hands dirty and write some code to execute external commands.<\/p>\n\n\n\n<p>First of all, you need to import the subprocess library. Since it is part of Python 3, you don&#8217;t need to install it separately. From this library, we&#8217;ll work with the run command. This command was added in Python 3.5. Make sure you have at least that Python version, but preferably you should be running the latest version. Check our detailed <a href=\"https:\/\/python.land\/installing-python\">Python installation instructions<\/a> if you need help with that. <\/p>\n\n\n\n<p>Let\u2019s start with a simple call to ls, to list the current directories and files:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">>>> import subprocess\n>>> subprocess.run(['ls', '-al'])\n\n(a list of your directories will be printed)<\/pre>\n\n\n\n<p>In fact, we can call Python, the binary, from our Python code. Let\u2019s request the version of the default python3 installation on our system next:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">>>> import subprocess\n>>> result = subprocess.run(['python3', '--version'])\nPython 3.8.5\n>>> result\nCompletedProcess(args=['python3', '--version'], returncode=0)<\/pre>\n\n\n\n<p>A line-by-line explanation:<\/p>\n\n\n\n<ol><li>We import the subprocess library<\/li><li>Run a subprocess, in this case the python3 binary, with one argument: <code>--version<\/code><\/li><li>Inspect the result <a href=\"https:\/\/python.land\/introduction-to-python\/variables\">variable<\/a>, which is of the type <code><a href=\"https:\/\/docs.python.org\/3\/library\/subprocess.html#subprocess.CompletedProcess\" target=\"_blank\" rel=\"noreferrer noopener\">CompletedProcess<\/a><\/code><\/li><\/ol>\n\n\n\n<p>The process returned code 0, meaning it executed successfully. Any other return code would mean there was some kind of error. It depends on the process you called what the different return code means. <\/p>\n\n\n\n<p>As you can see in the output, the Python binary printed its version number on standard out, which is usually your terminal. Your result may vary because your Python version will likely be different. Perhaps, you&#8217;ll even get an error looking like this: <code>FileNotFoundError: [Errno 2] No such file or directory: 'python3'<\/code>. In this case, make sure the Python binary is called python3 on your system too, and that it&#8217;s in the PATH.<\/p>\n\n\n\n<h2 id=\"h-capture-output-of-a-python-subprocess\"><span class=\"ez-toc-section\" id=\"Capture_output_of_a_Python_subprocess\"><\/span>Capture output of a Python subprocess<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>If you run an external command, you&#8217;ll likely want to capture the output of that command. We can achieve this with the capture_output=True option:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">>>> import subprocess\n>>> result = subprocess.run(['python3', '--version'], capture_output=True, encoding='UTF-8')\n>>> result\nCompletedProcess(args=['python3', '--version'], returncode=0, stdout='Python 3.8.5\\n', stderr='')<\/pre>\n\n\n\n<p>As you can see, Python didn&#8217;t print its version to our terminal this time. The subprocess.run command redirected the standard out and standard error streams so it could capture them and store the result for us. After inspecting the result <a href=\"https:\/\/python.land\/introduction-to-python\/variables\">variable<\/a>, we see that the Python version was captured from standard out. Since there were no errors, stderr is empty.<\/p>\n\n\n\n<p>I also added the option encoding=&#8217;UTF-8&#8242;. If you don&#8217;t, <code>subprocess.run<\/code> assumes the output is a stream of bytes because it doesn&#8217;t have this information. Try it, if you want. As a result, <code>stdout <\/code>and <code>stderr <\/code>will be byte arrays. Hence, if you know the output will be ASCII text or UTF-8 text, you&#8217;re better off specifying it so the run function encodes the captured output accordingly as well.<\/p>\n\n\n\n<p>Alternatively, you can also use the option text=True without specifying the encoding. Python will capture the output as text. I&#8217;d recommend specifying the encoding explicitly if you know it.<\/p>\n\n\n\n<h2><span class=\"ez-toc-section\" id=\"Feeding_data_from_standard_input\"><\/span>Feeding data from standard input<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>If the external command expects data on standard input, we can do so easily as well with the <code>input<\/code> option of Python&#8217;s <code>subprocess.run<\/code> function. Please note that I&#8217;m not going into streaming data here. We&#8217;ll build on the previous examples here:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">>>> import subprocess\n>>> code = \"\"\"\n... for i in range(1, 3):\n...   print(f\"Hello world {i}\")\n... \"\"\"\n\n>>> result = subprocess.run(['python3'], input=code, capture_output=True, encoding='UTF-8')\n>>> print(result.stdout)\n>>> print(result.stdout)\nHello world 1\nHello world 2<\/pre>\n\n\n\n<p>We just used Python to execute some Python code with the python3 binary. Completely useless, but (hopefully) very instructive!<\/p>\n\n\n\n<p>The code variable is a <a href=\"https:\/\/python.land\/introduction-to-python\/strings#Multiline_strings\">multi-line Python string<\/a> and we assign it as input to the <code>subprocess.run<\/code> command using the <code>input<\/code> option.<\/p>\n\n\n\n<h2><span class=\"ez-toc-section\" id=\"Running_shell_commands\"><\/span>Running shell commands<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>If you are looking to execute shell commands on Unix-like systems, by which I mean anything you would normally type into a Bash-like shell, you need to realize that these are often not external binaries that are executed. For example, expressions like&nbsp;<code>for<\/code>&nbsp;and&nbsp;<code>while<\/code>&nbsp;loops, or pipes and other operators, are interpreted by the shell itself.<\/p>\n\n\n\n<p>Python often has alternatives in the form of built-in libraries, which you should prefer. But if you need to execute a shell command, for whatever reason,&nbsp;<code>subprocess.run<\/code>&nbsp;will happily do so when you use the&nbsp;<code>shell=True<\/code>&nbsp;option. It allows you to enter commands just as if you were entering them in a Bash compatible shell:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">>>> import subprocess\n>>> result = subprocess.run(['ls -al | head -n 1'], shell=True)\ntotal 396\n>>> result\nCompletedProcess(args=['ls -al | head -n 1'], returncode=0)<\/pre>\n\n\n\n<p>But a warning is in place: using this method is prone to command injection attacks (see: <a href=\"#caveats\">caveats<\/a>).<\/p>\n\n\n\n<h2 id=\"caveats\"><span class=\"ez-toc-section\" id=\"Caveats_to_look_out_for\"><\/span>Caveats to look out for<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Running external commands is not without risks. Please read this section very carefully.<\/p>\n\n\n\n<h3>os.system vs subprocess.run<\/h3>\n\n\n\n<p>You might see code examples where <code>os.system()<\/code> is used to execute a command. The <code>subprocess<\/code> module is more powerful, though, and the official Python docs recommend using it over <code>os.system()<\/code>. Another issue with <code>os.system<\/code> is that it is more prone to command injection.<\/p>\n\n\n\n<h3>Command injection<\/h3>\n\n\n\n<p>A common attack, or exploit, is to inject extra commands to gain control over a computer system. For example, if you ask your user for input and use that input in a call to <code>os.system()<\/code> or a call to <code>subprocess.run(...., shell=True)<\/code>, you&#8217;re at risk of a command injection attack.<\/p>\n\n\n\n<p>To demonstrate, the following code allows us to run any shell command:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import subprocess\nthedir = input()\nresult = subprocess.run([f'ls -al {thedir}'], shell=True)\n<\/pre>\n\n\n\n<p>Because we directly used the user input, the user can run any command simply by appending it with a semicolon. E.g., the following input will list the \/ directory and echo a text. Try it for yourself:<\/p>\n\n\n\n<p><code>\/; echo \"command injection worked!\";<\/code><\/p>\n\n\n\n<p>The solution is <em>not <\/em>to try and clean the user input. You might be tempted to start looking for semicolons and rejecting the input of you find one. Don&#8217;t; hackers can think of at least 5 other ways to append a command in this situation. It&#8217;s an uphill battle.<\/p>\n\n\n\n<p>The better solution is to not use <code>shell=True<\/code>, and feed the command in a list as we did in the earlier examples. Input like this will fail in such cases because the subprocess module will make sure the input is an argument to the program you\u2019re executing, instead of a new command.<\/p>\n\n\n\n<p>With the same input, but with <code>shell=False<\/code>, you will get this:<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import subprocess\nthedir = input()\n>>> result = subprocess.run([f'ls -al {thedir}'], shell=False)\nTraceback (most recent call last):\n  File \"&lt;stdin>\", line 1, in &lt;module>\n  File \"\/usr\/lib\/python3.8\/subprocess.py\", line 489, in run\n    with Popen(*popenargs, **kwargs) as process:\n  File \"\/usr\/lib\/python3.8\/subprocess.py\", line 854, in __init__\n    self._execute_child(args, executable, preexec_fn, close_fds,\n  File \"\/usr\/lib\/python3.8\/subprocess.py\", line 1702, in _execute_child\n    raise child_exception_type(errno_num, err_msg, err_filename)\nFileNotFoundError: [Errno 2] No such file or directory: 'ls -al \/; echo \"command injection worked!\";'\n<\/pre>\n\n\n\n<p>The command is treated as an argument to&nbsp;<code>ls<\/code>, which in turn tells us that It can\u2019t find that file or directory.<\/p>\n\n\n\n<h1 id=\"058c\"><span class=\"ez-toc-section\" id=\"User_input_is_always_dangerous\"><\/span>User input is always dangerous<span class=\"ez-toc-section-end\"><\/span><\/h1>\n\n\n\n<p id=\"520f\">In fact, using user input is always dangerous, not just because of command injection. For example, suppose you allow a user to input a file name. After this, we read the file and show it to the user. Although this might seem harmless, a user could enter something like this: <code>..\/..\/..\/..\/configuration\/settings.yaml<\/code><\/p>\n\n\n\n<p id=\"076f\">Where&nbsp;<code>settings.yaml<\/code>&nbsp;might contain your database password\u2026 oops! You always need to properly sanitize and check user input. How to do that properly, is beyond the scope of this article though.<\/p>\n\n\n\n<h2 id=\"h-keep-learning\"><span class=\"ez-toc-section\" id=\"Keep_learning\"><\/span>Keep learning<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The following related resources will help you dive even deeper in this subject:<\/p>\n\n\n\n<ul><li>The <a href=\"https:\/\/docs.python.org\/3\/library\/subprocess.html\" target=\"_blank\" rel=\"noreferrer noopener\">official documentation<\/a> has all the details about the subprocess library<\/li><li>Our article on <a href=\"https:\/\/python.land\/python-concurrency\">Python concurrency<\/a> explains more about processes and threads<\/li><li>Our section on <a href=\"https:\/\/python.land\/the-unix-shell\">using the Unix shell<\/a> might come in handy<\/li><li>Learn some <a href=\"https:\/\/python.land\/the-unix-shell\/basic-unix-commands\">Basic Unix commands<\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Despite the many libraries on PyPI, sometimes you need to run an external command from your Python code. The built-in Python subprocess module makes this &#8230; <a title=\"Python Subprocess: Run External Commands\" class=\"read-more\" href=\"https:\/\/python.land\/operating-system\/python-subprocess\" aria-label=\"More on Python Subprocess: Run External Commands\">read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":2388,"parent":2359,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[30],"tags":[],"modified_by":"Erik van Baaren","menu_order":20,"_links":{"self":[{"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/posts\/2350"}],"collection":[{"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/comments?post=2350"}],"version-history":[{"count":16,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/posts\/2350\/revisions"}],"predecessor-version":[{"id":2499,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/posts\/2350\/revisions\/2499"}],"up":[{"embeddable":true,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/posts\/2359"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/media\/2388"}],"wp:attachment":[{"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/media?parent=2350"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/categories?post=2350"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/python.land\/wp-json\/wp\/v2\/tags?post=2350"}],"curies":[{"name":"wp","href":"https:\/\/web.archive.org\/web\/20211101101637\/https:\/\/api.w.org\/{rel}","templated":true}]}}