Automating Tasks with Python
Python, with its simple syntax, versatility, and vast array of libraries, has emerged as the most popular choice for automating various mundane and repetitive tasks. Developers, data analysts, consultants and many other Python users can create scripts that interact with other software, services, and even simulate human interactions. Python’s extensive library ecosystem provides pre-built modules for various automation tasks, making it easier to automate processes in different domains.
Automation refers to the process of using technology, such as software or machines, to perform repetitive tasks or processes. It has become an essential aspect of our lives, allowing us to streamline processes, save time, and focus on more critical tasks.
The possibilities are endless when it comes to automating tasks with Python. Learn about it with these automation challenges and experiment with your own ideas.
Examples of tasks to be automated with Python:
1.Data extraction from webs
Web scraping involves pulling data from websites by writing scripts that scrape information from web pages. This can be very valuable for research, price monitoring, or aggregating information from multiple sources. For example, if you want to monitor hotel rooms ratings for a certain date and place, you can write a script to check prices until you find a good offer. Python offers libraries like Selenium and BeautifulSoup, which facilitate web scraping by interacting with web browsers and parsing HTML content.
Here’s an example of a script structure for web scraping using Selenium and BeautifulSoup:
import requests from bs4
import BeautifulSoup
# Make a GET request to the website
response = requests.get('https://www.example.com')
# Parse the HTML content using BeautifulSoup
soup = BeautifulSoup(response.content, 'html.parser')
# Extract specific information from the web page
title = soup.find('h1').text
paragraphs = soup.find_all('p')
# Print the extracted information
print('Title:', title)
print('Paragraphs:')
for paragraph in paragraphs:
print(paragraph.text)
In this example, we make a “GET” request to a website using the requests library, parse the HTML content using BeautifulSoup, and extract specific information like the title and paragraphs from the web page.
2. File Management Automation
Managing files on your computer can be a tedious task, especially when dealing with large quantities of them. Python’s OS module provides functionalities for automating file management tasks like copying, renaming and deleting files. For instance, you can create a small program to clean the downloads folder from time to time, or to move all image files (png or jpg) to a specific path on your laptop .
A sample of OS functions to sort files with Python:
import os
# Rename a file
old_name = 'old_file.txt'
new_name = 'new_file.txt'
os.rename(old_name, new_name)
# Delete a file
file_to_delete = 'path/to/file.txt'
os.remove(file_to_delete)
3. Automating Communications
Automating communications can be incredibly useful for sending reminders, notifications, or personalized messages. Libraries like “smtplib” and “PyWhatKit” allow to send emails and whatsapp messages.
With PyWhatKit you can open WhatsApp Web in your default browser and automatically send a message to a specified phone number.
Small business owners, like restaurants or hairdressers, can write scripts to text or email requests to confirm customer’s appointments and avoid last minute cancellations.
The code for automating emails would be something like:
import smtplib from email.mime.text
import MIMEText
def send_email(subject, message, sender, recipient):
msg = MIMEText(message)
msg['Subject'] = subject
msg['From'] = sender
msg['To'] = recipient
smtp_server = 'smtp.example.com'
smtp_port = 587
smtp_username = 'your_username'
smtp_password = 'your_password'
with smtplib.SMTP(smtp_server, smtp_port) as server:
server.starttls()
server.login(smtp_username, smtp_password)
server.send_message(msg)
# Send an email
subject = 'Reminder: Meeting Tomorrow'
message = 'Just a friendly reminder that we have a meeting scheduled for tomorrow.'
sender = 'sender@example.com'
recipient = 'recipient@example.com'
send_email(subject, message, sender, recipient)
We define the function “send_email()” that takes the subject, message, sender, and recipient as parameters and uses the smtplib library to send the pre-defined email.
4. Secure Password Generation
Generating secure passwords is crucial for protecting sensitive information. Nowadays is very common to update passwords regularly and always with many restrictions in order to be valid.
For example, we can define a function that takes the desired length and the required type of characters (upper case, numbers, etc). The function will generate a password by randomly selecting characters from the available character set defined in the string module.
5. Financial Reporting Automation
Automating reporting can be highly beneficial in banking, insurance, funds… enabling them to generate reports quickly and accurately. Excel spreadsheets are the most common tool in the Finance industry; therefore, a program using the Pandas library to clean, merge and manipulate several Excel files can be an excellent time- saving choice for automating data wrangling tasks without errors.
Here’s a sample of Python functions to read and export files in different formats:
import pandas as pd
# Read data from a CSV file and an Excel
data = pd.read_csv('data.csv')
data_excel = pd.read_excel('data2.xlsx')
# Remove duplicates and merge files
report = pd.merge(data,data_excel, on='id,how='left')
report = report.drop_dulicates()
# Save the report to a new Excel file
report.to_excel('report.xlsx', index=False)
6. Extracting Data from PDFs, CSVs, and Excels
Extracting data from various file formats like PDFs, CSVs, and Excels can be time-consuming. There are libraries like “tabula“, “csv”, and “openpyxl” that allow you to automate the extraction of data from these file formats. If you are an analyst that have to manually gather data from different sources on a regular basis, you can write a program to do the job for you automatically.
A Python script to pull data from a PDF and saving it to a CSV file using tabula would be as follows:
import tabula
# Read data from a PDF and extract tables
tables = tabula.read_pdf('data.pdf', pages='all')
# Convert each table to a DataFrame
dataframes = [table “” not found /]
# Save each DataFrame to a separate CSV file
for i, dataframe in enumerate(dataframes):
dataframe.to_csv(f'table_{i+1}.csv', index=False)
We use the “read_pdf()” function from the tabula library to read data from a PDF and extract tables. We then convert each table to a DataFrame and export each DataFrame to a separate CSV file using the “to_csv()” method.
7. Client services
Chatbots have become increasingly popular in various domains, including customer support and online client services. Libraries like “NLTK” and “ChatterBot” enable developers to build chatbots with natural language processing capabilities.
8. Navigating the Internet and Web Browsing Automation
Web browsing tasks like querying search engines to extract information from websites or perform actions on web applications can be automated thanks to Python libraries such as Selenium or BeautifulSoup. For example, if you need to check Google Trends for certain keywords, or export traffic figures from the Search Console, you can create a Python program to do it whenever you run it.
To automate web browsing using selenium:
from selenium import webdriver
# Create a new instance of the web driver
driver = webdriver.Chrome()
# Navigate to a website
driver.get('https://www.wikipedia.com')
# Perform actions on the web page # ...
# Close the web driver
driver.quit()
We import the webdriver class from the selenium library, create a new instance of the web driver (in this case, Chrome), navigate to a website using the “get()” method, perform actions on the web page, and finally close the web driver.
9. Conclusion
Automating boring and repetitive tasks with Python can significantly improve productivity and efficiency, save time and ensure a reliable outcome. From web scraping and file management to automating communications and generating secure passwords, Python has the tools to simplify these tasks. By harnessing the power of automation, you can focus on more critical aspects of your work and increase overall productivity.
