{"id":5334,"date":"2020-03-04T20:07:07","date_gmt":"2020-03-04T19:07:07","guid":{"rendered":"https:\/\/pythonprogramming.altervista.org\/?p=5334"},"modified":"2020-03-05T09:18:26","modified_gmt":"2020-03-05T08:18:26","slug":"webscrapint-coronavirus-data","status":"publish","type":"post","link":"https:\/\/pythonprogramming.altervista.org\/webscrapint-coronavirus-data\/","title":{"rendered":"Webscraping coronavirus data:"},"content":{"rendered":"

I have this simple code to scrape<\/strong> the last data<\/strong> of cofirmed cases in Italy<\/strong> of coronavirus<\/strong> from a site with Python<\/strong>.<\/p>\n

I just did some basic webscraping, importing:<\/p>\n

import requests\r\nimport urllib.request\r\nimport time\r\nfrom bs4 import BeautifulSoup\r\n<\/pre>\n
Now that we have the tools, let’s use them<\/p>\n
url = \"https:\/\/lab24.ilsole24ore.com\/coronavirus\/\"\r\n\r\n# access the site\r\nresponse = requests.get(url)\r\nprint(response)<\/pre>\n
We made a request to the site of ilsole24ore. If we get this we get the response right:<\/p>\n
<Response [200]><\/pre>\n
We parse the html that we get as response<\/p>\n
soup = BeautifulSoup(response.text, \"html.parser\")<\/pre>\n
We get all the <h2> tag, because the data is into a h2 tag, like you can see in the code below.<\/p>\n
To get the window on the right that you see below, right click with the mouse and then choose inspect from the menu, then click on (1. click) the arrow on the top left, then click on the number of confirmed cases (click 2). You will se on the right the <h2> tag that contains the number.<\/p>\n
\"\"<\/a><\/p>\n
h2 = soup.findAll(\"h2\")<\/pre>\n
I can see that the first h2 is the one that I need; let’s see what we have<\/p>\n
print(h2)<\/pre>\n
We will see this<\/p>\n
[<h2 class=\"timer count-number\" data-speed=\"1000\" data-to=\"2706\" id=\"num_1\"><\/h2>, <h2 class=\"timer count-number\" data-speed=\"1000\" data-to=\"107\" id=\"num_2\"><\/h2>, <h2 class=\"timer count-number\" data-speed=\"1000\" data-to=\"276\" id=\"num_3\"><\/h2>, <h2 class=\"timer count-number\" data-speed=\"1000\" data-to=\"3089\" id=\"num_4\"><\/h2>, <h2 class=\"chartTitle\"><center>I numeri complessivi<\/center><\/h2>, <h2 class=\"chartTitle\">Il trend giorno per giorno<\/h2>, <h2 class=\"chartTitle\">L\u2019andamento nelle province con pi\u00f9 contagi<\/h2>, <h2 class=\"chartTitle\">L\u2019andamento delle 5 regioni con pi\u00f9 contagi<\/h2>, <h2 class=\"chartTitle\">I dati per provincia<\/h2>, <h2 class=\"chartTitle\">I nuovi tamponi giornalieri<\/h2>, <h2 class=\"chartTitle\">Ricoveri e terapie intensive<\/h2>, <h2 class=\"chartTitle\">Come crescono i ricoveri<\/h2>, <h2 class=\"chartTitle\">Il contagio nei Paesi europei<\/h2>, <h2 class=\"chartTitle\">I primi dieci Paesi al mondo per contagio<\/h2>]<\/pre>\n
as you can see, the first one it the one that we are looking for with the 2706 number.<\/p>\n
I am gonna transform the first of the h2 data (h2[0]) into a string and the print the sixth (5) element of the list obtained splitting by the apostrophes the string.<\/p>\n
h2 = str(h2[0])\r\nprint(\"Last data of confirmed case in Italy:\")\r\nprint(h2.split(\"\\\"\")[5])<\/pre>\n
Just to be clear if I split the h2.split(“\\””) I get this:<\/p>\n
['<h2 class=', 'timer count-number', ' data-speed=', '1000', ' data-to=', '2706', ' id=', 'num_1', '><\/h2>']<\/pre>\n
And the sixth element is “2706”, what I was looking for.<\/p>\n
The whole code<\/h2>\n
import requests\r\nimport urllib.request\r\nimport time\r\nfrom bs4 import BeautifulSoup\r\n\r\n# the site to scrape\r\n\r\nurl = \"https:\/\/lab24.ilsole24ore.com\/coronavirus\/\"\r\n\r\n# access the site\r\nresponse = requests.get(url)\r\n# now we parse the html with BS\r\nsoup = BeautifulSoup(response.text, \"html.parser\")\r\nh2 = soup.findAll(\"h2\")\r\nh2 = str(h2[0])\r\nprint(\"Last data of confirmed case in Italy:\")\r\nprint(h2.split(\"\\\"\")[5])<\/pre>\n
Let’s get the result with a shell here<\/h2>\n