{"id":299,"date":"2024-01-28T10:17:28","date_gmt":"2024-01-28T10:17:28","guid":{"rendered":"https:\/\/learnpython.elegantwallp.com\/?p=299"},"modified":"2024-01-28T10:17:29","modified_gmt":"2024-01-28T10:17:29","slug":"python-multithreading-example","status":"publish","type":"post","link":"https:\/\/learnpython.elegantwallp.com\/2024\/01\/28\/python-multithreading-example\/","title":{"rendered":"Python Multithreading Example"},"content":{"rendered":"\n<p><strong>Summary<\/strong>: in this tutorial, you\u2019ll learn how to use the Python threading module to develop a multithreaded program.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Extending the Thread class<\/h2>\n\n\n\n<p>We\u2019ll develop a multithreaded program that scraps the stock prices from the\u00a0Yahoo Finance website.<\/p>\n\n\n\n<p>To do that, we\u2019ll use two third-party packages:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>requests<\/code>&nbsp;\u2013 to get the contents of a webpage.<\/li>\n\n\n\n<li><code>lxml<\/code>&nbsp;\u2013 to select a specific element of an HTML document.<\/li>\n<\/ul>\n\n\n\n<p>First, install the\u00a0<code>requests<\/code>\u00a0and\u00a0<code>lxml<\/code>\u00a0modules using the\u00a0pip command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>pip install request lxml<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Next, define a new class called\u00a0<code>Stock<\/code>\u00a0that\u00a0inherits\u00a0from the\u00a0<code>Thread<\/code>\u00a0class of the\u00a0<code>threading<\/code>\u00a0module. We\u2019ll place the Stock class in\u00a0<code>stock.py<\/code>\u00a0module:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>import threading class Stock(threading.Thread): pass<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Then, implement the\u00a0<code><code>__init__()<\/code><\/code>\u00a0method that accepts a symbol and initializes the\u00a0<code>url<\/code>\u00a0instance variable based on the symbol:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>import threading import requests from lxml import html class Stock(threading.Thread): def __init__(self, symbol: str) -> None: super().__init__() self.symbol = symbol self.url = f'https:\/\/finance.yahoo.com\/quote\/{symbol}' self.price = None<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>For example, if you pass the symbol\u00a0<code>GOOG<\/code>\u00a0to the\u00a0<code>__init__()<\/code>\u00a0method, the URL will be:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>https:\/\/finance.yahoo.com\/quote\/GOOG<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>After that, override the\u00a0<code>run()<\/code>\u00a0method of the\u00a0<code>Thread<\/code>\u00a0class. The\u00a0<code>run()<\/code>\u00a0method gets the contents from the\u00a0<code>self.url<\/code>\u00a0and grabs the stock price:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>class Stock(threading.Thread): def __init__(self, symbol: str) -> None: super().__init__() self.symbol = symbol self.url = f'https:\/\/finance.yahoo.com\/quote\/{symbol}' self.price = None def run(self): headers = { \"User-Agent\": \"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/114.0.0.0 Safari\/537.36\" } response = requests.get(self.url, headers=headers) if response.status_code == 200: <em># parse the HTML<\/em> tree = html.fromstring(response.text) <em># get the price in text<\/em> price_text = tree.xpath( '\/\/*&#91;@id=\"quote-header-info\"]\/div&#91;3]\/div&#91;1]\/div&#91;1]\/fin-streamer&#91;1]\/text()') if price_text: try: self.price = float(price_text&#91;0].replace(',', '')) except ValueError: self.price = None def __str__(self): return f'{self.symbol}\\t{self.price}'<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>How it works.<\/p>\n\n\n\n<p>Make a request to the URL using the\u00a0<code>requests.get()<\/code>\u00a0method:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>headers = { \"User-Agent\": \"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/114.0.0.0 Safari\/537.36\" } response = requests.get(self.url, headers=headers)<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Notice that without valid headers, Yahoo will return 404 instead of 200.<\/p>\n\n\n\n<p>If the request is successful, the HTTP status code is 200. In this case, we get the HTML contents from the response and pass it to the\u00a0<code>fromstring()<\/code>\u00a0function of the\u00a0<code>html<\/code>\u00a0module from the\u00a0<code>lxml<\/code>\u00a0package:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>if response.status_code == 200: tree = html.fromstring(response.text)<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Every element on a webpage can be selected using something called XPath.<\/p>\n\n\n\n<p>To get the XPath of an element using Google Chrome, you inspect the page, right-click the element, select copy, and Copy XPath.<\/p>\n\n\n\n<p>The XPath of the stock price at the time of writing this tutorial is as follows:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>\/\/*&#91;@id=\"quote-header-info\"]\/div&#91;3]\/div&#91;1]\/div&#91;1]\/fin-streamer&#91;1]<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>To get the text of the element, you append the\u00a0<code>text()<\/code>\u00a0at the end of the XPath:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>\/\/*&#91;@id=\"quote-header-info\"]\/div&#91;3]\/div&#91;1]\/div&#91;1]\/fin-streamer&#91;1]\/text()<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Notice that if Yahoo changes the page structure, you need to change the XPath accordingly. Otherwise, the program won\u2019t work as expected:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>price_text = tree.xpath('\/\/*&#91;@id=\"quote-header-info\"]\/div&#91;3]\/div&#91;1]\/div&#91;1]\/fin-streamer&#91;1]\/text()')<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Once getting the price as text, we remove the comma and convert it to a number:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>if price_text: try: self.price = float(price_text&#91;0].replace(',', '')) except ValueError: self.price = None<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Finally, add the\u00a0<code><code>__str__()<\/code><\/code>\u00a0method that returns the string representation of the Stock object:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>import threading import requests from lxml import html class Stock(threading.Thread): def __init__(self, symbol: str) -> None: super().__init__() self.symbol = symbol self.url = f'https:\/\/finance.yahoo.com\/quote\/{symbol}' self.price = None def run(self): headers = { \"User-Agent\": \"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/114.0.0.0 Safari\/537.36\" } response = requests.get(self.url, headers=headers) if response.status_code == 200: <em># parse the HTML<\/em> tree = html.fromstring(response.text) <em># get the price in text<\/em> price_text = tree.xpath( '\/\/*&#91;@id=\"quote-header-info\"]\/div&#91;3]\/div&#91;1]\/div&#91;1]\/fin-streamer&#91;1]\/text()') if price_text: try: self.price = float(price_text&#91;0].replace(',', '')) except ValueError: self.price = None def __str__(self): return f'{self.symbol}\\t{self.price}' <\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Using the Stock class<\/h2>\n\n\n\n<p>The following\u00a0<code>main.py<\/code>\u00a0module uses the\u00a0<code>Stock<\/code>\u00a0class from the\u00a0<code>stock.py<\/code>\u00a0module:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>from stock import Stock symbols = &#91;'MSFT', 'GOOGL', 'AAPL', 'META'] threads = &#91;] for symbol in symbols: t = Stock(symbol) threads.append(t) t.start() for t in threads: t.join() print(t) <\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Output:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>MSFT 253.67 GOOGL 2280.41 AAPL 145.86 META 163.27<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>How it works.<\/p>\n\n\n\n<p>First, import the\u00a0<code>Stock<\/code>\u00a0class from the\u00a0<code>stock.py<\/code>\u00a0module:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>from stock import Stock<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Second, initialize a list of symbols:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>symbols = &#91;'MSFT', 'GOOGL', 'AAPL', 'META']<\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Third, create a thread for each symbol, start it, and append the thread to the threads list:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>threads = &#91;] for symbol in symbols: t = Stock(symbol) threads.append(t) t.start() <\/code><small>Code language: Python (python)<\/small><\/code><\/pre>\n\n\n\n<p>Finally, wait for all the threads in the threads list to complete and print out the stock price:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>for t in threads: t.join() print(t)<\/code><\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Summary: in this tutorial, you\u2019ll learn how to use the Python threading module to develop a multithreaded program. Extending the Thread class We\u2019ll develop a multithreaded program that scraps the stock prices from the\u00a0Yahoo Finance website. To do that, we\u2019ll use two third-party packages: First, install the\u00a0requests\u00a0and\u00a0lxml\u00a0modules using the\u00a0pip command: Next, define a new class [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[41],"tags":[],"class_list":["post-299","post","type-post","status-publish","format-standard","hentry","category-1-python-concurrency"],"_links":{"self":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts\/299","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/comments?post=299"}],"version-history":[{"count":1,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts\/299\/revisions"}],"predecessor-version":[{"id":300,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/posts\/299\/revisions\/300"}],"wp:attachment":[{"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/media?parent=299"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/categories?post=299"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/learnpython.elegantwallp.com\/wp-json\/wp\/v2\/tags?post=299"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}