{"id":290,"date":"2024-01-27T14:09:03","date_gmt":"2024-01-27T14:09:03","guid":{"rendered":"https:\/\/learnpython.elegantwallp.com\/?p=290"},"modified":"2024-01-27T14:09:06","modified_gmt":"2024-01-27T14:09:06","slug":"python-threading","status":"publish","type":"post","link":"https:\/\/learnpython.elegantwallp.com\/2024\/01\/27\/python-threading\/","title":{"rendered":"Python Threading"},"content":{"rendered":"\n

Summary<\/strong>: in this tutorial, you\u2019ll learn how to use the Python threading module to develop multi-threaded applications.<\/p>\n\n\n\n

Single-threaded applications<\/h2>\n\n\n\n

Let\u2019s start with a simple program:<\/p>\n\n\n\n

from time import sleep, perf_counter def task(): print('Starting a task...') sleep(1) print('done') start_time = perf_counter() task() task() end_time = perf_counter() print(f'It took {end_time- start_time: 0.2f} second(s) to complete.') <\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
How it works.<\/p>\n\n\n\n
First, import the\u00a0sleep()<\/code>\u00a0and\u00a0perf_counter()<\/code>\u00a0functions from the\u00a0time<\/code>\u00a0module:<\/p>\n\n\n\n
from time import sleep, perf_counter<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
Second,\u00a0define a function\u00a0that takes one second to complete:<\/p>\n\n\n\n
def task(): print('Starting a task...') sleep(1) print('done')<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
Third, get the value of the performance counter by calling the\u00a0perf_counter()<\/code>\u00a0function:<\/p>\n\n\n\n
start_time = perf_counter() <\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
Fourth, call the\u00a0task()<\/code>\u00a0function twice:<\/p>\n\n\n\n
task() task()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
Fifth, get the value of the performance counter by calling the\u00a0perf_counter()<\/code>\u00a0function:<\/p>\n\n\n\n
end_time = perf_counter()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
Finally, output the time that takes to complete running the\u00a0task()<\/code>\u00a0function twice:<\/p>\n\n\n\n
print(f'It took {end_time- start_time: 0.2f} second(s) to complete.')<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
Here is the output:<\/p>\n\n\n\n
Starting a task... done Starting a task... done It took 2.00 second(s) to complete.<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
As you may expect, the program takes about two seconds to complete. If you call the task()<\/code> function 10 times, it would take about 10 seconds to complete.<\/p>\n\n\n\n
The following diagram illustrates how the program works:<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
First, the task()<\/code> function executes and sleeps for one second. Then it executes the second time and also sleeps for another second. Finally, the program completes.<\/p>\n\n\n\n
When the task()<\/code> function calls the sleep()<\/code> function, the CPU is idle. In other words, the CPU doesn\u2019t do anything, which is not efficient in terms of resource utilization.<\/p>\n\n\n\n
This program has one process with a single thread, which is called the main thread<\/strong>. Because the program has only one thread, it\u2019s called a single-threaded program.<\/p>\n\n\n\n
Using Python threading to develop a multi-threaded program example<\/h2>\n\n\n\n
To create a multi-threaded program, you need to use the Python threading<\/code> module.<\/p>\n\n\n\n
First, import the Thread<\/code> class from the threading<\/code> module:from threading import Thread<\/code>Code language: Python (python)<\/small><\/p>\n\n\n\n
Second, create a new thread by instantiating an instance of the\u00a0Thread<\/code>\u00a0class:<\/p>\n\n\n\n
new_thread = Thread(target=fn,args=args_tuple)<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
The Thread()<\/code> accepts many parameters. The main ones are:<\/p>\n\n\n\n
    \n
  • target<\/code>: specifies a function (fn<\/code>) to run in the new thread.<\/li>\n\n\n\n
  • args<\/code>: specifies the arguments of the function (fn<\/code>). The args<\/code> argument is a tuple.<\/li>\n<\/ul>\n\n\n\n
    Third, start the thread by calling the\u00a0start()<\/code>\u00a0method of the\u00a0Thread<\/code>\u00a0instance:<\/p>\n\n\n\n
    new_thread.start()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    If you want to wait for the thread to complete in the main thread, you can call the\u00a0join()<\/code>\u00a0method:<\/p>\n\n\n\n
    new_thread.join()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    By calling the join()<\/code> method, the main thread will wait for the child thread to complete before it is terminated.<\/p>\n\n\n\n
    The following program illustrates how to use the\u00a0threading<\/code>\u00a0module:<\/p>\n\n\n\n
    from time import sleep, perf_counter from threading import Thread def task(): print('Starting a task...') sleep(1) print('done') start_time = perf_counter() # create two new threads<\/em> t1 = Thread(target=task) t2 = Thread(target=task) # start the threads<\/em> t1.start() t2.start() # wait for the threads to complete<\/em> t1.join() t2.join() end_time = perf_counter() print(f'It took {end_time- start_time: 0.2f} second(s) to complete.')<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    How it works. (and we\u2019ll focus on the threading part only)<\/p>\n\n\n\n
    First, create two new threads:<\/p>\n\n\n\n
    t1 = Thread(target=task) t2 = Thread(target=task)<\/code><\/code><\/pre>\n\n\n\n
    Second, start both threads by calling the\u00a0start()<\/code>\u00a0method:<\/p>\n\n\n\n
    t1.start() t2.start()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    Third, wait for both threads to complete:<\/p>\n\n\n\n
    t1.join() t2.join()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    Finally, show the executing time:<\/p>\n\n\n\n
    print(f'It took {end_time- start_time: 0.2f} second(s) to complete.')<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    Output:<\/p>\n\n\n\n
    Starting a task... Starting a task... done done It took 1.00 second(s) to complete.<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    When the program executes, it\u2019ll have three threads: the main thread and two other child threads.<\/p>\n\n\n\n
    As shown clearly from the output, the program took one second instead of two to complete.<\/p>\n\n\n\n
    The following diagram shows how threads execute:<\/p>\n\n\n\n
    \"\"<\/figure>\n\n\n\n
    Passing arguments to threads<\/h2>\n\n\n\n
    The following program shows how to pass arguments to the function assigned to a thread:<\/p>\n\n\n\n
    from time import sleep, perf_counter from threading import Thread def task(id): print(f'Starting the task {id}...') sleep(1) print(f'The task {id} completed') start_time = perf_counter() # create and start 10 threads<\/em> threads = [] for n in range(1, 11): t = Thread(target=task, args=(n,)) threads.append(t) t.start() # wait for the threads to complete<\/em> for t in threads: t.join() end_time = perf_counter() print(f'It took {end_time- start_time: 0.2f} second(s) to complete.')<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    How it works.<\/p>\n\n\n\n
    First, define a\u00a0task()<\/code>\u00a0function that accepts an argument:<\/p>\n\n\n\n
    def task(id): print(f'Starting the task {id}...') sleep(1) print(f'The task {id} completed')<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    Second, create 10 new threads and pass an id to each. The\u00a0threads<\/code>\u00a0list is used to keep track of all newly created threads:<\/p>\n\n\n\n
    threads = [] for n in range(1, 11): t = Thread(target=task, args=(n,)) threads.append(t) t.start()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    Notice that if you call the join()<\/code> method inside the loop, the program will wait for the first thread to complete before starting the next one.<\/p>\n\n\n\n
    Third, wait for all threads to complete by calling the\u00a0join()<\/code>\u00a0method:<\/p>\n\n\n\n
    for t in threads: t.join()<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    The following shows the output of the program:<\/p>\n\n\n\n
    Starting the task 1... Starting the task 2... Starting the task 3... Starting the task 4... Starting the task 5... Starting the task 6... Starting the task 7... Starting the task 8... Starting the task 9... Starting the task 10... The task 10 completed The task 8 completed The task 1 completed The task 6 completed The task 7 completed The task 9 completed The task 3 completed The task 4 completed The task 2 completed The task 5 completed It took 1.02 second(s) to complete.<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
    It just took 1.05 seconds to complete.<\/p>\n\n\n\n
    Notice that the program doesn\u2019t execute the thread in the order from 1 to 10.<\/p>\n\n\n\n
    When to use Python threading<\/h2>\n\n\n\n
    As introduced in the\u00a0process and thread tutorial, there\u2019re two main kinds of tasks:<\/p>\n\n\n\n
      \n
    • I\/O-bound tasks \u2013 the time spent on I\/O is significantly more than the time spent on computation.<\/li>\n\n\n\n
    • CPU-bound tasks \u2013 the time spent on computation is significantly higher than the time waiting for I\/O.<\/li>\n<\/ul>\n\n\n\n
      Python threading is optimized for I\/O bound tasks. For example, requesting remote resources, connecting a database server, or reading and writing files.<\/p>\n\n\n\n
      A Practical Python threading example<\/h2>\n\n\n\n
      Suppose that you have a list of text files in a folder e.g., C:\/temp\/<\/code>. And you want to replace a text with a new one in all the files.<\/p>\n\n\n\n
      The following single-threaded program shows how to replace a substring with the new one in the text files:<\/p>\n\n\n\n
      from time import perf_counter def replace(filename, substr, new_substr): print(f'Processing the file {filename}') # get the contents of the file<\/em> with open(filename, 'r') as f: content = f.read() # replace the substr by new_substr<\/em> content = content.replace(substr, new_substr) # write data into the file<\/em> with open(filename, 'w') as f: f.write(content) def main(): filenames = [ 'c:\/temp\/test1.txt', 'c:\/temp\/test2.txt', 'c:\/temp\/test3.txt', 'c:\/temp\/test4.txt', 'c:\/temp\/test5.txt', 'c:\/temp\/test6.txt', 'c:\/temp\/test7.txt', 'c:\/temp\/test8.txt', 'c:\/temp\/test9.txt', 'c:\/temp\/test10.txt', ] for filename in filenames: replace(filename, 'ids', 'id') if __name__ == \"__main__\": start_time = perf_counter() main() end_time = perf_counter() print(f'It took {end_time- start_time :0.2f} second(s) to complete.') <\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
      Output:<\/p>\n\n\n\n
      It took 0.16 second(s) to complete.<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
      The following program has the same functionality. However, it uses multiple threads instead:<\/p>\n\n\n\n
      from threading import Thread from time import perf_counter def replace(filename, substr, new_substr): print(f'Processing the file {filename}') # get the contents of the file<\/em> with open(filename, 'r') as f: content = f.read() # replace the substr by new_substr<\/em> content = content.replace(substr, new_substr) # write data into the file<\/em> with open(filename, 'w') as f: f.write(content) def main(): filenames = [ 'c:\/temp\/test1.txt', 'c:\/temp\/test2.txt', 'c:\/temp\/test3.txt', 'c:\/temp\/test4.txt', 'c:\/temp\/test5.txt', 'c:\/temp\/test6.txt', 'c:\/temp\/test7.txt', 'c:\/temp\/test8.txt', 'c:\/temp\/test9.txt', 'c:\/temp\/test10.txt', ] # create threads<\/em> threads = [Thread(target=replace, args=(filename, 'id', 'ids')) for filename in filenames] # start the threads<\/em> for thread in threads: thread.start() # wait for the threads to complete<\/em> for thread in threads: thread.join() if __name__ == \"__main__\": start_time = perf_counter() main() end_time = perf_counter() print(f'It took {end_time- start_time :0.2f} second(s) to complete.') <\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
      Output:<\/p>\n\n\n\n
      Processing the file c:\/temp\/test1.txt Processing the file c:\/temp\/test2.txt Processing the file c:\/temp\/test3.txt Processing the file c:\/temp\/test4.txt Processing the file c:\/temp\/test5.txt Processing the file c:\/temp\/test6.txt Processing the file c:\/temp\/test7.txt Processing the file c:\/temp\/test8.txt Processing the file c:\/temp\/test9.txt Processing the file c:\/temp\/test10.txt It took 0.02 second(s) to complete.<\/code>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n
      As you can see clearly from the output, the multi-threaded program runs so much faster.<\/p>\n","protected":false},"excerpt":{"rendered":"
      Summary: in this tutorial, you\u2019ll learn how to use the Python threading module to develop multi-threaded applications. Single-threaded applications Let\u2019s start with a simple program: How it works. First, import the\u00a0sleep()\u00a0and\u00a0perf_counter()\u00a0functions from the\u00a0time\u00a0module: Second,\u00a0define a function\u00a0that takes one second to complete: Third, get the value of the performance counter by calling the\u00a0perf_counter()\u00a0function: Fourth, call the\u00a0task()\u00a0function […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[41],"tags":[],"class_list":["post-290","post","type-post","status-publish","format-standard","hentry","category-1-python-concurrency"],"_links":{"self":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts\/290","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/comments?post=290"}],"version-history":[{"count":1,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts\/290\/revisions"}],"predecessor-version":[{"id":291,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts\/290\/revisions\/291"}],"wp:attachment":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/media?parent=290"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/categories?post=290"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/tags?post=290"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}