Real Python

Python's raise: Effectively Raising Exceptions in Your Code

2025-01-25T14:00:00+00:00

When you use the raise statement in Python to raise (or throw) an exception, you signal an error or an unusual condition in your program. With raise, you can trigger both built-in and custom exceptions. You can also include custom messages for more clarity or even re-raise exceptions to add context or handle further processing.

By the end of this tutorial, you’ll understand that:

Raising an exception in Python signals an error condition, halting normal program flow.
You use raise to initiate exceptions for error handling or to propagate existing exceptions.
You can raise custom exceptions by defining new exception classes derived from Exception.
The difference between raise and assert lies in their use. You use assert for debugging, while raise is used to signal runtime errors.
You can re-raise an exception by using a bare raise within an except block to preserve the original traceback.

Learning about the raise statement will allow you to handle errors and exceptional situations effectively in your code. By knowing how to use it, you’ll be able to develop more robust programs and improve the quality of your code.

To get the most out of this tutorial, you should understand the fundamentals of Python, including variables, data types, conditional statements, exception handling, and classes.

Free Bonus: Click here to download the sample code that you’ll use to gracefully raise and handle exceptions in your Python code.

Take the Quiz: Test your knowledge with our interactive “Python's raise: Effectively Raising Exceptions in Your Code” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Python's raise: Effectively Raising Exceptions in Your Code

In this quiz, you'll test your understanding of how to raise exceptions in Python using the raise statement. This knowledge will help you handle errors and exceptional situations in your code, leading to more robust programs and higher-quality code.

Handling Exceptional Situations in Python

Exceptions play a fundamental role in Python. They allow you to handle errors and exceptional situations in your code. But what is an exception? An exception represents an error or indicates that something is going wrong. Some programming languages, such as C, and Go, encourage you to return error codes, which you check. In contrast, Python encourages you to raise exceptions, which you handle.

Note: In Python, not all exceptions are errors. The built-in StopIteration exception is an excellent example of this. Python internally uses this exception to terminate the iteration over iterators. Python exceptions that represent errors have the Error suffix attached to their names.

Python also has a specific category of exceptions that represent warnings to the programmer. Warnings come in handy when you need to alert the user of some condition in a program. However, that condition may not warrant raising an exception and terminating the program. A common example of a warning is DeprecationWarning, which appears when you use deprecated features.

When a problem occurs in a program, Python automatically raises an exception. For example, watch what happens if you try to access a nonexistent index in a list object:

Python
      
        
      
    
>>> colors = [
...     "red",
...     "orange",
...     "yellow",
...     "green",
...     "blue",
...     "indigo",
...     "violet"
... ]

>>> colors[10]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range

In this example, your colors list doesn’t have a 10 index. Its indices go from 0 to 6, covering your seven colors. So, if you try to get index number 10, then you get an IndexError exception telling you that your target index is out of range.

Note: In the example above, Python raised the exception on its own, which it’ll only do with built-in exceptions. You, as a programmer, have the option to raise built-in or custom exceptions, as you’ll learn in the section Choosing the Exception to Raise: Built-in vs Custom.

Every raised exception has a traceback, also known as a stack trace, stack traceback, or backtrace, among other names. A traceback is a report containing the sequence of calls and operations that traces down to the current exception.

In Python, the traceback header is Traceback (most recent call last) in most situations. Then you’ll have the actual call stack and the exception name followed by its error message.

Note: Since the introduction of the new PEG parser in Python 3.9, there’s been an ongoing effort to make the error messages in tracebacks more helpful and specific. This effort continues to produce new results, and Python 3.12 and Python 3.13 has incorporated even better error messages.

Exceptions will cause your program to terminate unless you handle them using a try … except block:

Python
      
        
      
    
>>> try:
...     colors[10]
... except IndexError:
...     print("Your list doesn't have that index :-(")
...
Your list doesn't have that index :-(

The first step in handling an exception is to predict which exceptions can happen. If you don’t do that, then you can’t handle the exceptions, and your program will crash. In that situation, Python will print the exception traceback so that you can figure out how to fix the problem. Sometimes, you must let the program fail in order to discover the exceptions that it raises.

In the above example, you know beforehand that indexing a list with an index beyond its range will raise an IndexError exception. So, you’re ready to catch and handle that specific exception. The try block takes care of catching exceptions. The except clause specifies the exception that you’re predicting, and the except code block allows you to take action accordingly. The whole process is known as exception handling.

If your code raises an exception in a function but doesn’t handle it there, then the exception propagates to where you called function. If your code doesn’t handle it there either, then it continues propagating until it reaches the main program. If there’s no exception handler there, then the program halts with an exception traceback.

Exceptions are everywhere in Python. Virtually every module in the standard library uses them. Python will raise exceptions in many different circumstances. The Python documentation states that:

Exceptions are a means of breaking out of the normal flow of control of a code block in order to handle errors or other exceptional conditions. An exception is raised at the point where the error is detected; it may be handled by the surrounding code block or by any code block that directly or indirectly invoked the code block where the error occurred. (Source)

In short, Python automatically raises exceptions when an error occurs during a program’s execution. Python also allows you to raise exceptions on demand using the raise keyword. This keyword lets you handle your program’s errors in a more controlled manner.

Read the full article at https://realpython.com/python-raise-exception/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Executing Python Scripts With a Shebang

2025-01-25T14:00:00+00:00

In shell scripts, the shebang line (#!) specifies the path to the interpreter that should execute the file. You can place it at the top of your Python file to tell the shell how to run your script, allowing you to execute the script directly without typing python before the script name. The shebang is essential for Unix-like systems but ignored on Windows unless using specific compatibility layers.

By the end of this tutorial, you’ll understand that:

A shebang specifies the path to the Python interpreter in scripts, allowing direct execution.
You should include a shebang when a script needs direct execution, but not in import-only modules.
Best practices for shebangs include using /usr/bin/env for portability and ensuring the script is executable.
Shebangs have limitations, such as being ignored on Windows without compatibility layers like WSL.

To proceed, you should have basic familiarity with the command line and know how to run Python scripts from it. You can also download the supporting materials for this tutorial to follow along with the code examples:

Free Sample Code: Click here to download the free sample code that you’ll use to execute Python scripts with a shebang.

What’s a Shebang, and When Should You Use It?

In short, a shebang is a special kind of comment that you may include in your source code to tell the operating system’s shell where to find the interpreter for the rest of the file:

Python
      
#!/usr/bin/python3

print("Hello, World!")

If you’re using a shebang, it must appear on the first line in your script, and it has to start with a hash sign (#) followed by an exclamation mark (!), colloquially known as the bang, hence the name shebang. The choice of the hash sign to begin this special sequence of characters wasn’t accidental, as many scripting languages use it for inline comments.

You should make sure you don’t put any other comments before the shebang line if you want it to work correctly, or else it won’t be recognized! After the exclamation mark, specify an absolute path to the relevant code interpreter, such as Python. Providing a relative path will have no effect, unfortunately.

Note: The shebang is only recognized by shells, such as Z shell or Bash, running on Unix-like operating systems, including macOS and Linux distributions. It bears no particular meaning in the Windows terminal, which treats the shebang as an ordinary comment by ignoring it.

You can get the shebang to work on Windows by installing the Windows Subsystem for Linux (WSL) that comes with a Unix shell. Alternatively, Windows lets you make a global file association between a file extension like .py and a program, such as the Python interpreter, to achieve a similar effect.

It’s not uncommon to combine a shebang with the name-main idiom, which prevents the main block of code from running when someone imports the file from another module:

Python
      
#!/usr/bin/python3

if __name__ == "__main__":
    print("Hello, World!")

With this conditional statement, Python will call the print() function only when you run this module directly as a script—for example, by providing its path to the Python interpreter:

Shell
      
$ python3 /path/to/your/script.py
Hello, World!

As long as the script’s content starts with a correctly defined shebang line and your system user has permission to execute the corresponding file, you can omit the python3 command to run that script:

Shell

$ /path/to/your/script.py
Hello, World!

A shebang is only relevant to runnable scripts that you wish to execute without explicitly specifying the program to run them through. You wouldn’t typically put a shebang in a Python module that only contains function and class definitions meant for importing from other modules. Therefore, use the shebang when you don’t want to prefix the command that runs your Python script with python or python3.

Note: In the old days of Python, the shebang line would sometimes appear alongside another specially formatted comment described in PEP 263:

Python
      
    
#!/usr/bin/python3
# -*- coding: utf-8 -*-

if __name__ == "__main__":
    print("Grüß Gott")

The highlighted line used to be necessary to tell the interpreter which character encoding it should use to read your source code correctly, as Python defaulted to ASCII. However, this was only important when you directly embedded non-Latin characters, such as ü or ß, in your code.

This special comment is irrelevant today because modern Python versions use the universal UTF-8 encoding, which can handle such characters with ease. Nevertheless, it’s always preferable to replace tricky characters with their encoded representations using Unicode literals:

Python
      
>>> "Grüß Gott".encode("unicode_escape")
b'Gr\\xfc\\xdf Gott'

Your foreign colleagues who have different keyboard layouts will thank you for that!

Now that you have a high-level understanding of what a shebang is and when to use it, you’re ready to explore it in more detail. In the next section, you’ll take a closer look at how it works.

How Does a Shebang Work?

Normally, to run a program in the terminal, you must provide the full path to a particular binary executable or the name of a command present in one of the directories listed on the PATH environment variable. One or more command-line arguments may follow this path or command:

Shell
      
$ /usr/bin/python3 -c 'print("Hello, World!")'
Hello, World!

$ python3 -c 'print("Hello, World!")'
Hello, World!

Here, you run the Python interpreter in a non-interactive mode against a one-liner program passed through the -c option. In the first case, you provide an absolute path to python3, while in the second case, you rely on the fact that the parent folder, /usr/bin/, is included on the search path by default. Your shell can find the Python executable, even if you don’t provide the full path, by looking through the directories on the PATH variable.

Note: If multiple commands with the same name exist in more than one directory listed on the PATH variable, then your shell will execute the first it can find. As a result, the outcome of running a command without explicitly specifying the corresponding path may sometimes be surprising. It’ll depend on the order of directories in your PATH variable. However, this can be useful, as you’ll find out later.

Read the full article at https://realpython.com/python-shebang/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

How to Flush the Output of the Python Print Function

2025-01-25T14:00:00+00:00

Python’s flush parameter in the print() function allows you to control when to empty the output data buffer, ensuring your output appears immediately. This is useful when building visual progress indicators or when piping logs to another application in real-time.

By default, print() output is line-buffered in interactive environments and block-buffered otherwise. You can override this behavior by using the flush=True parameter to force the buffer to clear immediately.

By the end of this tutorial, you’ll understand that:

Flush in coding refers to emptying the data buffer to ensure immediate output.
flush=True in print() forces the buffer to clear immediately.
Clearing output involves managing buffer behavior, typically with newlines.
Turning off print buffering can be achieved using the -u option or PYTHONUNBUFFERED.

By repeatedly running a short code snippet that you change only slightly, you’ll see that if you run print() with its default arguments, then its execution is line-buffered in interactive mode, and block-buffered otherwise.

You’ll get a feel for what all of that means by exploring the code practically. But before you dive into changing output stream buffering in Python, it’s helpful to revisit how it happens by default, and understand why you might want to change it.

Free Sample Code: Click here to download the free sample code that you’ll use to dive deep into flushing the output of the Python print function.

Understand How Python Buffers Output

When you make a write call to a file-like object, Python buffers the call by default—and that’s a good idea! Disk write and read operations are slow in comparison to random-access memory (RAM) access. When your script makes fewer system calls for write operations by batching characters in a RAM data buffer and writing them all at once to disk with a single system call, then you can save a lot of time.

To put the use case for buffering into a real-world context, think of traffic lights as buffers for car traffic. If every car crossed an intersection immediately upon arrival, it would end in gridlock. That’s why the traffic lights buffer traffic from one direction while the other direction flushes.

Note: Data buffers are generally size-based, not time-based, which is where the traffic analogy breaks down. In the context of a data buffer, the traffic lights would switch if a certain number of cars were queued up and waiting.

However, there are situations when you don’t want to wait for a data buffer to fill up before it flushes. Imagine that there’s an ambulance that needs to get past the crossroads as quickly as possible. You don’t want it to wait at the traffic lights until there’s a certain number of cars queued up.

In your program, you usually want to flush the data buffer right away when you need real-time feedback on code that has executed. Here are a couple of use cases for immediate flushing:

Instant feedback: In an interactive environment, such as a Python REPL or a situation where your Python script writes to a terminal
File monitoring: In a situation where you’re writing to a file-like object, and the output of the write operation gets read by another program while your script is still executing—for example, when you’re monitoring a log file

In both cases, you need to read the generated output as soon as it generates, and not only when enough output has assembled to flush the data buffer.

There are many situations where buffering is helpful, and there are some situations where too much buffering can be a disadvantage. Therefore, there are different types of data buffering that you can implement where they fit best:

Unbuffered means that there’s no data buffer. Every byte creates a new system call and gets written independently.
Line-buffered means that there’s a data buffer that collects information in memory, and once it encounters a newline character (\n), the data buffer flushes and writes the whole line in one system call.
Fully-buffered (block-buffered) means that there’s a data buffer of a specific size, which collects all the information that you want to write. Once it’s full, it flushes and sends all its contents onward in a single system call.

Python uses block buffering as a default when writing to file-like objects. However, it executes line-buffered if you’re writing to an interactive environment.

To better understand what that means, write a Python script that simulates a countdown:

Pythoncountdown.py
      
    
from time import sleep

for second in range(3, 0, -1):
    print(second)
    sleep(1)
print("Go!")

By default, each number shows up right when print() is called in the script. But as you develop and tweak your countdown timer, you might run into a situation where all your output gets buffered. Buffering the whole countdown and printing it all at once when the script finishes would lead to a lot of confusion for the athletes waiting at the start line!

So how can you make sure that you won’t run into data buffering issues as you develop your Python script?

Add a Newline for Python to Flush Print Output

If you’re running a code snippet in a Python REPL or executing it as a script directly with your Python interpreter, then you won’t run into any issues with the script shown above.

In an interactive environment, the standard output stream is line-buffered. This is the output stream that print() writes to by default. You’re working with an interactive environment any time that your output will display in a terminal. In this case, the data buffer flushes automatically when it encounters a newline character ("\n"):

Read the full article at https://realpython.com/python-flush-print-output/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

How to Download Files From URLs With Python

2025-01-25T14:00:00+00:00

Python makes it straightforward to download files from a URL with its robust set of libraries. For quick tasks, you can use the built-in urllib module or the requests library to fetch and save files. When working with large files, streaming data in chunks can help save memory and improve performance.

You can also perform parallel file downloads using ThreadPoolExecutor for multithreading or the aiohttp library for asynchronous tasks. These approaches allow you to handle multiple downloads concurrently, significantly reducing the total download time if you’re handling many files.

By the end of this tutorial, you’ll understand that:

You can use Python to download files with libraries like urllib and requests.
To download a file using a URL in Python, you can use urlretrieve() or requests.get().
To extract data from a URL in Python, you use the response object from requests.
To download a CSV file from a URL in Python, you may need to specify the format in the URL or query parameters.

In this tutorial, you’ll be downloading a range of economic data from the World Bank Open Data platform. To get started on this example project, go ahead and grab the sample code below:

Free Bonus: Click here to download your sample code for downloading files from the Web with Python.

Facilitating File Downloads With Python

While it’s possible to download files from URLs using traditional command-line tools, Python provides several libraries that facilitate file retrieval. Using Python to download files offers several advantages.

One advantage is flexibility, as Python has a rich ecosystem of libraries, including ones that offer efficient ways to handle different file formats, protocols, and authentication methods. You can choose the most suitable Python tools to accomplish the task at hand and fulfill your specific requirements, whether you’re downloading from a plain-text CSV file or a complex binary file.

Another reason is portability. You may encounter situations where you’re working on cross-platform applications. In such cases, using Python is a good choice because it’s a cross-platform programming language. This means that Python code can run consistently across different operating systems, such as Windows, Linux, and macOS.

Using Python also offers the possibility of automating your processes, saving you time and effort. Some examples include automating retries if a download fails, retrieving and saving multiple files from URLs, and processing and storing your data in designated locations.

These are just a few reasons why downloading files using Python is better than using traditional command-line tools. Depending on your project requirements, you can choose the approach and library that best suits your needs. In this tutorial, you’ll learn approaches to some common scenarios requiring file retrievals.

Downloading a File From a URL in Python

In this section, you’ll learn the basics of downloading a ZIP file containing gross domestic product (GDP) data from the World Bank Open Data platform. You’ll use two common tools in Python, urllib and requests, to download GDP by country.

While the urllib package comes with Python in its standard library, it has some limitations. So, you’ll also learn to use a popular third-party library, requests, that offers more features for making HTTP requests. Later in the tutorial, you’ll see additional functionalities and use cases.

Using `urllib` From the Standard Library

Python ships with a package called urllib, which provides a convenient way to interact with web resources. It has a straightforward and user-friendly interface, making it suitable for quick prototyping and smaller projects. With urllib, you can perform different tasks dealing with network communication, such as parsing URLs, sending HTTP requests, downloading files, and handling errors related to network operations.

As a standard library package, urllib has no external dependencies and doesn’t require installing additional packages, making it a convenient choice. For the same reason, it’s readily accessible for development and deployment. It’s also cross-platform compatible, meaning you can write and run code seamlessly using the urllib package across different operating systems without additional dependencies or configuration.

The urllib package is also very versatile. It integrates well with other modules in the Python standard library, such as re for building and manipulating regular expressions, as well as json for working with JSON data. The latter is particularly handy when you need to consume JSON APIs.

In addition, you can extend the urllib package and use it with other third-party libraries, like requests, BeautifulSoup, and Scrapy. This offers the possibility for more advanced operations in web scraping and interacting with web APIs.

To download a file from a URL using the urllib package, you can call urlretrieve() from the urllib.request module. This function fetches a web resource from the specified URL and then saves the response to a local file. To start, import urlretrieve() from urlllib.request:

Python
      
>>> from urllib.request import urlretrieve

Next, define the URL that you want to retrieve data from. If you don’t specify a path to a local file where you want to save the data, then the function will create a temporary file for you. Since you know that you’ll be downloading a ZIP file from that URL, go ahead and provide an optional path to the target file:

Python
      
        
      
    
>>> url = (
...     "https://api.worldbank.org/v2/en/indicator/"
...     "NY.GDP.MKTP.CD?downloadformat=csv"
... )
>>> filename = "gdp_by_country.zip"

Because your URL is quite long, you rely on Python’s implicit concatenation by splitting the string literal over multiple lines inside parentheses. The Python interpreter will automatically join the separate strings on different lines into a single string. You also define the location where you wish to save the file. When you only provide a filename without a path, Python will save the resulting file in your current working directory.

Read the full article at https://realpython.com/python-download-file-from-url/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python and TOML: New Best Friends

2025-01-25T14:00:00+00:00

TOML stands for Tom’s Obvious Minimal Language. Its human-readable syntax makes TOML convenient to parse into data structures across various programming languages. In Python, you can use the built-in tomllib module to work with TOML files. TOML plays an essential role in the Python ecosystem. Many of your favorite tools rely on TOML for configuration, and you’ll use pyproject.toml when you build and distribute your own packages.

By the end of this tutorial, you’ll understand that:

TOML in Python refers to a minimal configuration file format that’s convenient to read and parse.
TOML files are primarily used for configuration, separating code from settings for flexibility.
pyproject.toml is crucial for package configuration and specifies the build system and dependencies.
Loading a TOML file in Python involves using tomli or tomllib to parse it into a dictionary.
tomli and tomllib differ mainly in origin, with tomllib being a standard library module in modern Python.

If you want to know more about why tomllib was added to Python, then have a look at the companion tutorial, Python 3.11 Preview: TOML and tomllib.

Free Download: Get a sample chapter from Python Tricks: The Book that shows you Python’s best practices with simple examples you can apply instantly to write more beautiful + Pythonic code.

Use TOML as a Configuration Format

TOML is short for Tom’s Obvious Minimal Language and is humbly named after its creator, Tom Preston-Werner. It was designed expressly to be a configuration file format that should be “easy to parse into data structures in a wide variety of languages” (Source).

In this section, you’ll start thinking about configuration files and look at what TOML brings to the table.

Configurations and Configuration Files

A configuration is an important part of almost any application or system. It’ll allow you to change settings or behavior without changing the source code. Sometimes you’ll use a configuration to specify information needed to connect to another service like a database or cloud storage. Other times you’ll use configuration settings to allow your users to customize their experience with your project.

Using a configuration file for your project is a good way to separate your code from its settings. It also encourages you to be conscious about which parts of your system are genuinely configurable, giving you a tool to name magic values in your source code. For now, consider this configuration file for a hypothetical tic-tac-toe game:

Config File
      
    
player_x_color = blue
player_o_color = green
board_size     = 3
server_url     = https://tictactoe.example.com/

You could potentially code this directly in your source code. However, by moving the settings into a separate file, you achieve a few things:

You give explicit names to values.
You provide these values more visibility.
You make it simpler to change the values.

Look more closely at your hypothetical configuration file. Those values are conceptually different. The colors are values that your framework probably supports changing. In other words, if you replaced blue with red, that would be honored without any special handling in your code. You could even consider if it’s worth exposing this configuration to your end users through your front end.

However, the board size may or may not be configurable. A tic-tac-toe game is played on a three-by-three grid. It’s not certain that your logic would still work for other board sizes. It may still make sense to keep the value in your configuration file, both to give a name to the value and to make it visible.

Finally, the project URL is usually essential when deploying your application. It’s not something that a typical user will change, but a power user may want to redeploy your game to a different server.

To be more explicit about these different use cases, you may want to add some organization to your configuration. One popular option is to separate your configuration into additional files, each dealing with a different concern. Another option is to group your configuration values somehow. For example, you can organize your hypothetical configuration file as follows:

Config File
      
    
[user]
player_x_color = blue
player_o_color = green

[constant]
board_size = 3

[server]
url = https://tictactoe.example.com

The organization of the file makes the role of each configuration item clearer. You can also add comments to the configuration file with instructions to anyone thinking about making changes to it.

Note: The actual format of your configuration file isn’t important for this discussion. The above principles hold independently of how you specify your configuration values. As it happens, the examples that you’ve seen so far can be parsed by Python’s ConfigParser class.

There are many ways for you to specify a configuration. Windows has traditionally used INI files, which resemble your configuration file from above. Unix systems have also relied on plain-text, human-readable configuration files, although the actual format varies between different services.

Over time, more and more applications have come to use well-defined formats like XML, JSON, or YAML for their configuration needs. These formats were designed as data interchange or serialization formats, usually meant for computer communication.

On the other hand, configuration files are often written or edited by humans. Many developers have gotten frustrated with JSON’s strict comma rules when updating their Visual Studio Code settings or with YAML’s nested indentations when setting up a cloud service. Despite their ubiquity, these file formats aren’t the easiest to write by hand.

TOML: Tom’s Obvious Minimal Language

Read the full article at https://realpython.com/python-toml/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #236: Simon Willison: Using LLMs for Python Development

2025-01-24T12:00:00+00:00

What are the current large language model (LLM) tools you can use to develop Python? What prompting techniques and strategies produce better results? This week on the show, we speak with Simon Willison about his LLM research and his exploration of writing Python code with these rapidly evolving tools.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

How to Deal With Missing Data in Polars

2025-01-22T14:00:00+00:00

Efficiently handling missing data in Polars is essential for keeping your datasets clean during analysis. Polars provides powerful tools to identify, replace, and remove null values, ensuring seamless data processing.

This tutorial covers practical techniques for managing missing data and highlights Polars’ capabilities to enhance your data analysis workflow. By following along, you’ll gain hands-on experience with these techniques and learn how to ensure your datasets are accurate and reliable.

By the end of this tutorial, you’ll understand that:

Polars allows you to handle missing data using LazyFrames and DataFrames.
You can check for null values in Polars using the .null_count() method.
NaN represents non-numeric values while null indicates missing data.
You can replace NaN in Polars by converting them to nulls and using .fill_null().
You can fix missing data by identifying, replacing, or removing null values.

Before you go any further, you’ll need some data. To begin with, you’ll use the tips.parquet file included in the downloadable materials that you can access by clicking the link below:

Get Your Code: Click here to download the free sample code that shows you how to deal with missing data in Polars.

The tips.parquet file is a doctored version of data publicly available from Kaggle. The dataset contains information about the tips collected at a fictitious restaurant over several days. Be sure to download it and place it in your project folder before getting started.

Note: The Parquet format is a format for storing large volumes of data. Disk size is minimized because of the compression algorithm it uses.

In addition, Parquet uses a columnar format and maintains metadata about each column’s content. This means columns can be searched very efficiently, often in parallel, without the need to search through the entire file.

The table below shows details about the columns in the tips.parquet file, along with their Polars data types. The text in parentheses beside each data type shows how these types are annotated in a DataFrame heading when Polars displays its results:

Column Name	Polars Data Type	Description
`record_id`	`Int64 (i64)`	Unique row identifier
`total`	`Float64 (f64)`	Bill total
`tip`	`Float64 (f64)`	Tip given
`gender`	`String (str)`	Diner’s gender
`smoker`	`Boolean (bool)`	Diner’s smoker status
`day`	`String (str)`	Day of meal
`time`	`String (str)`	Time of meal

As a starting point, you’ll investigate each of the columns in your data to find out whether or not they contain any null values. To use Polars, you first need to install the Polars library into your Python environment. To do this from a command prompt you use:

Windows PowerShell
      
PS> python -m pip install polars

Shell
      
$ python -m pip install polars

In a Jupyter Notebook, the command becomes:

Python
      
!python -m pip install polars

Either way, you can then begin to use the Polars library and all of its cool features. Here’s what the data looks like:

Python
      
        
      
    
>>> import polars as pl

>>> tips = pl.scan_parquet("tips.parquet")

>>> tips.collect()
shape: (180, 7)
┌───────────┬───────┬──────┬────────┬────────┬─────┬────────┐
│ record_id ┆ total ┆ tip  ┆ gender ┆ smoker ┆ day ┆ time   │
│ ---       ┆ ---   ┆ ---  ┆ ---    ┆ ---    ┆ --- ┆ ---    │
│ i64       ┆ f64   ┆ f64  ┆ str    ┆ bool   ┆ str ┆ str    │
╞═══════════╪═══════╪══════╪════════╪════════╪═════╪════════╡
│ 1         ┆ 28.97 ┆ 3.0  ┆ Male   ┆ true   ┆ Fri ┆ Dinner │
│ 2         ┆ 22.49 ┆ 3.5  ┆ Male   ┆ false  ┆ Fri ┆ Dinner │
│ 3         ┆ 5.75  ┆ 1.0  ┆ Female ┆ true   ┆ Fri ┆ null   │
│ 4         ┆ null  ┆ null ┆ Male   ┆ true   ┆ Fri ┆ Dinner │
│ 5         ┆ 22.75 ┆ 3.25 ┆ Female ┆ false  ┆ Fri ┆ Dinner │
│ …         ┆ …     ┆ …    ┆ …      ┆ …      ┆ …   ┆ …      │
│ 176       ┆ 40.55 ┆ 3.0  ┆ Male   ┆ true   ┆ Sun ┆ Dinner │
│ 177       ┆ 20.69 ┆ 5.0  ┆ Male   ┆ false  ┆ Sun ┆ Dinner │
│ 178       ┆ 20.9  ┆ 3.5  ┆ Female ┆ true   ┆ Sun ┆ Dinner │
│ 179       ┆ 30.46 ┆ 2.0  ┆ Male   ┆ true   ┆ Sun ┆ Dinner │
│ 180       ┆ 18.15 ┆ 3.5  ┆ Female ┆ true   ┆ Sun ┆ Dinner │
└───────────┴───────┴──────┴────────┴────────┴─────┴────────┘

First of all, you import the Polars library into your program. It’s considered good practice to import it using the alias pl. You then read the content of the tips.parquet file into Polars. To do this, you use the scan_parquet() function. This reads the file’s data into a Polars LazyFrame.

Unlike traditional DataFrames that store data, LazyFrames contain only a set of instructions—called a query plan—that defines how the data should be processed. To see the actual data, you still need to read it into a Polars DataFrame. This is called materializing the LazyFrame and is achieved using the .collect() method.

Before a LazyFrame materializes its data, its query plan is optimized. For example, Polars can choose to only read some data from the data source if those are enough to fulfill the query. Also, when you define a LazyFrame containing multiple instructions, there are no delays while you create it because you don’t need to wait for earlier data reads to complete before adding new instructions. This makes LazyFrames the preferred approach in Polars.

The result has a shape of 180 rows and 7 columns. This is shown in the output as the shape of the LazyFrame.

Next, you need to figure out if there’s any missing data you need to deal with:

Python
      
        
      
    
>>> (
...     tips
...     .null_count()
... ).collect()
shape: (1, 7)
┌───────────┬───────┬─────┬────────┬────────┬─────┬──────┐
│ record_id ┆ total ┆ tip ┆ gender ┆ smoker ┆ day ┆ time │
│ ---       ┆ ---   ┆ --- ┆ ---    ┆ ---    ┆ --- ┆ ---  │
│ u32       ┆ u32   ┆ u32 ┆ u32    ┆ u32    ┆ u32 ┆ u32  │
╞═══════════╪═══════╪═════╪════════╪════════╪═════╪══════╡
│ 0         ┆ 2     ┆ 4   ┆ 0      ┆ 0      ┆ 0   ┆ 2    │
└───────────┴───────┴─────┴────────┴────────┴─────┴──────┘

To check for the presence of nulls, you use .null_count() on your LazyFrame which adds in an instruction to find a count of the nulls in each column of your data. Normally, this would require a read of the entire file. However, because a Parquet file stores a count of nulls for each column in its metadata, obtaining the counts is instantaneous.

To actually trigger the data read, you again use the LazyFrame’s .collect() method. This will implement the optimized version of the plan contained within your LazyFrame to obtain the required data.

Read the full article at https://realpython.com/polars-missing-data/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Exploring Python's tuple Data Type With Examples

2025-01-21T14:00:00+00:00

In Python, a tuple is a built-in data type that allows you to create immutable sequences of values. The values or items in a tuple can be of any type. This makes tuples pretty useful in those situations where you need to store heterogeneous data, like that in a database record, for example.

Through this tutorial, you’ll dive deep into Python tuples and get a solid understanding of their key features and use cases. This knowledge will allow you to write more efficient and reliable code by taking advantage of tuples.

In this video course, you’ll learn how to:

Create tuples in Python
Access the items in an existing tuple
Unpack, return, copy, and concatenate tuples
Reverse, sort, and traverse existing tuples
Explore other features and common gotchas of tuples

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: How to Deal With Missing Data in Polars

2025-01-21T12:00:00+00:00

In this quiz, you’ll test your understanding of How to Deal With Missing Data in Polars.

By working through the questions, you’ll review your understanding of dealing with null values in Polars and also expand on what you learned in the tutorial.

You’ll need to do some research outside of the tutorial to answer all the questions. Embrace this challenge and let it take you on a learning journey.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Getters and Setters: Manage Attributes in Python

2025-01-20T14:00:00+00:00

Getter and setter methods allow you to access and mutate non-public attributes while maintaining encapsulation. In Python, you’ll typically expose attributes as part of your public API and use properties when you need attributes with functional behavior. This tutorial guides you through writing getter and setter methods, replacing them with properties, and exploring alternatives like descriptors for optimized attribute management.

By the end of this tutorial, you’ll understand that:

Getter and setter methods allow you to access and modify data attributes while maintaining encapsulation.
Python properties can replace getters and setters, providing a more Pythonic way to manage attributes with functional behavior.
You typically avoid using getters and setters in Python unless necessary, as properties and descriptors offer more flexible solutions.
Descriptors are an advanced Python feature that enable reusable attributes with attached behaviors across different classes.
In some scenarios, inheritance limitations may make traditional getters and setters preferable over properties.

Even though properties are the Pythonic way to go, they can have some practical drawbacks. Because of this, you’ll find some situations where getters and setters are preferable over properties. This tutorial will delve into the nuances of using different approaches, equipping you with the knowledge to make informed decisions about attribute management in your classes.

To get the most out of this tutorial, you should be familiar with Python object-oriented programming. It’ll also be a plus if you have basic knowledge of Python properties and descriptors.

Source Code: Click here to get the free source code that shows you how and when to use getters, setters, and properties in Python.

Take the Quiz: Test your knowledge with our interactive “Getters and Setters: Manage Attributes in Python” quiz. You’ll receive a score upon completion to help you track your learning progress:

Interactive Quiz

Getters and Setters: Manage Attributes in Python

In this quiz, you'll test your understanding of Python's getter and setter methods, as well as properties. You'll learn when to use these tools and how they can help maintain encapsulation in your classes.

Getting to Know Getter and Setter Methods

When you define a class in object-oriented programming (OOP), you’ll likely end up with some instance and class attributes. These attributes are just variables that you can access through the instance, the class, or both.

Attributes hold the internal state of objects. In many cases, you’ll need to access and mutate this state, which involves accessing and mutating the attributes. Typically, you’ll have at least two ways to access and mutate attributes. You can either:

Access and mutate the attribute directly
Use methods to access and mutate the attribute

If you expose the attributes of a class to your users, then those attributes automatically become part of the class’s public API. They’ll be public attributes, which means that your users will directly access and mutate the attributes in their code.

Having an attribute that’s part of a class’s API will become a problem if you need to change the internal implementation of the attribute itself. A clear example of this issue is when you want to turn a stored attribute into a computed one. A stored attribute will immediately respond to access and mutation operations by just retrieving and storing data, while a computed attribute will run computations before such operations.

The problem with regular attributes is that they can’t have an internal implementation because they’re just variables. So, changing an attribute’s internal implementation will require converting the attribute into a method, which will probably break your users’ code. Why? Because they’ll have to change attribute access and mutation operations into method calls throughout their codebase if they want the code to continue working.

To deal with this kind of issue, some programming languages, like Java and C++, require you to provide methods for manipulating the attributes of your classes. These methods are commonly known as getter and setter methods. You can also find them referred to as accessor and mutator methods.

What Are Getter and Setter Methods?

Getter and setter methods are quite popular in many object-oriented programming languages. So, it’s pretty likely that you’ve heard about them already. As a rough definition, you can say that getters and setters are:

Getter: A method that allows you to access an attribute in a given class
Setter: A method that allows you to set or mutate the value of an attribute in a class

In OOP, the getter and setter pattern suggests that public attributes should be used only when you’re sure that no one will ever need to attach behavior to them. If an attribute is likely to change its internal implementation, then you should use getter and setter methods.

Implementing the getter and setter pattern requires:

Making your attributes non-public
Writing getter and setter methods for each attribute

For example, say that you need to write a Label class with text and font attributes. If you were to use getter and setter methods to manage these attributes, then you’d write the class like in the following code:

Pythonlabel.py
      
    
class Label:
    def __init__(self, text, font):
        self._text = text
        self._font = font

    def get_text(self):
        return self._text

    def set_text(self, value):
        self._text = value

    def get_font(self):
        return self._font

    def set_font(self, value):
        self._font = value

In this example, the constructor of Label takes two arguments, text and font. These arguments are stored in the ._text and ._font non-public instance attributes, respectively.

Then you define getter and setter methods for both attributes. Typically, getter methods return the target attribute’s value, while setter methods take a new value and assign it to the underlying attribute.

Read the full article at https://realpython.com/python-getter-setter/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Constants: Improve Your Code's Maintainability

2025-01-19T14:00:00+00:00

In Python, constants are identifiers for values that don’t change during a program’s execution. Unlike some other languages, Python lacks built-in syntax for constants, treating them as variables that should remain unchanged. You define constants by following a naming convention: use all uppercase letters with underscores separating words. This signals to other developers that these variables should not be reassigned.

By the end of this tutorial, you’ll understand that:

Constants in Python are variables that should remain unchanged throughout execution.
Python lacks built-in syntax for constants, relying on conventions to signal immutability.
Defining a constant involves using uppercase letters with underscores for clarity.
Best practices include defining constants at the top of a file and using descriptive names.
Built-in constants like math.pi offer predefined, reliable values, unlike user-defined ones.

To learn the most from this tutorial, you’ll need basic knowledge of Python variables, functions, modules, packages, and namespaces. You’ll also need to know the basics of object-oriented programming in Python.

Sample Code: Click here to download sample code that shows you how to use constants in Python.

Understanding Constants and Variables

Variables and constants are two historical and fundamental concepts in computer programming. Most programming languages use these concepts to manipulate data and work in an effective and logical fashion.

Variables and constants will probably be present in each project, app, library, or other piece of code that you’ll ever write. The question is: what are variables and constants in practice?

What Variables Are

In math, a variable is defined as a symbol that refers to a value or quantity that can change over time. In programming, a variable is also a symbol or name typically associated with a memory address containing a value, object, or piece of data. Like in math, the content of a programming variable can change during the execution of the code that defines it.

Variables typically have a descriptive name that’s somehow associated with a target value or object. This target value can be of any data type. So, you can use variables to represent numbers, strings, sequences, custom objects, and more.

You can perform two main operations on a variable:

Access its value
Assign it a new value

In most programming languages, you can access the value associated with a variable by citing the variable’s name in your code. To assign a new value to a given variable, you’ll use an assignment statement, which often consists of the variable’s name, an assignment operator, and the desired value.

In practice, you’ll find many examples of magnitudes, data, and objects that you can represent as variables. A few examples include temperature, speed, time, and length. Other examples of data that you can treat as variables include the number of registered users in a web app, the number of active characters in a video game, and the number of miles covered by a runner.

What Constants Are

Math also has the concept of constants. The term refers to a value or quantity that never changes. In programming, constants refer to names associated with values that never change during a program’s execution.

Just like variables, programming constants consist of two things: a name and an associated value. The name will clearly describe what the constant is all about. The value is the concrete expression of the constant itself.

Like with variables, the value associated with a given constant can be of any of data type. So, you can define integer constants, floating-point constants, character constants, string constants, and more.

After you’ve defined a constant, it’ll only allow you to perform a single operation on it. You can only access the constant’s value but not change it over time. This is different from a variable, which allows you to access its value, but also reassign it.

You’ll use constants to represent values that won’t change. You’ll find lots of these values in your day-to-day programming. A few examples include the speed of light, the number of minutes in an hour, and the name of a project’s root folder.

Why Use Constants

In most programming languages, constants protect you from accidentally changing their values somewhere in the code when you’re coding at two in the morning, causing unexpected and hard-to-debug errors. Constants also help you make your code more readable and maintainable.

Some advantages of using constants instead of using their values directly in your code include:

Advantage	Description
Improved readability	A descriptive name representing a given value throughout a program is always more readable and explicit than the bare-bones value itself. For example, it’s easier to read and understand a constant named `MAX_SPEED` than the concrete speed value itself.
Clear communication of intent	Most people will assume that `3.14` may refer to the Pi constant. However, using the `Pi`, `pi`, or `PI` name will communicate your intent more clearly than using the value directly. This practice will allow other developers to understand your code quickly and accurately.
Better maintainability	Constants enable you to use the same name to identify the same value throughout your code. If you need to update the constant’s value, then you don’t have to change every instance of the value. You just have to change the value in a single place: the constant definition. This improves your code’s maintainability.
Lower risk of errors	A constant representing a given value throughout a program is less error-prone than several explicit instances of the value. Say that you use different precision levels for Pi depending on your target calculations. You’ve explicitly used the values with the required precision for every calculation. If you need to change the precision in a set of calculations, then replacing the values can be error-prone because you can end up changing the wrong values. It’s safer to create different constants for different precision levels and change the code in a single place.
Reduced debugging needs	Constants will remain unchanged during the program’s lifetime. Because they’ll always have the same value, they shouldn’t cause errors and bugs. This feature may not be necessary in small projects, but it may be crucial in large projects with multiple developers. Developers won’t have to invest time debugging the current value of any constant.
Thread-safe data storage	Constants can only be accessed, not written. This feature makes them thread-safe objects, which means that several threads can simultaneously use a constant without the risk of corrupting or losing the underlying data.

Read the full article at https://realpython.com/python-constants/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

pandas GroupBy: Your Guide to Grouping Data in Python

2025-01-19T14:00:00+00:00

The pandas .groupby() method allows you to efficiently analyze and transform datasets when working with data in Python. With df.groupby(), you can split a DataFrame into groups based on column values, apply functions to each group, and combine the results into a new DataFrame. This technique is essential for tasks like aggregation, filtering, and transformation on grouped data.

By the end of this tutorial, you’ll understand that:

Calling .groupby("column_name") splits a DataFrame into groups, applies a function to each group, and combines the results.
To group by multiple columns, you can pass a list of column names to .groupby().
Common aggregation methods in pandas include .sum(), .mean(), and .count().
You can use custom functions with pandas .groupby() to perform specific operations on groups.

This tutorial assumes that you have some experience with pandas itself, including how to read CSV files into memory as pandas objects with read_csv(). If you need a refresher, then check out Reading CSVs With pandas and pandas: How to Read and Write Files.

You can download the source code for all the examples in this tutorial by clicking on the link below:

Download Datasets: Click here to download the datasets that you’ll use to learn about pandas’ GroupBy in this tutorial.

Prerequisites

Before you proceed, make sure that you have the latest version of pandas available within a new virtual environment:

Windows PowerShell
      
PS> python -m venv venv
PS> venv\Scripts\activate
(venv) PS> python -m pip install pandas

Shell
      
$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ python -m pip install pandas

In this tutorial, you’ll focus on three datasets:

The U.S. Congress dataset contains public information on historical members of Congress and illustrates several fundamental capabilities of .groupby().
The air quality dataset contains periodic gas sensor readings. This will allow you to work with floats and time series data.
The news aggregator dataset holds metadata on several hundred thousand news articles. You’ll be working with strings and doing text munging with .groupby().

You can download the source code for all the examples in this tutorial by clicking on the link below:

Download Datasets: Click here to download the datasets that you’ll use to learn about pandas’ GroupBy in this tutorial.

Once you’ve downloaded the .zip file, unzip the file to a folder called groupby-data/ in your current directory. Before you read on, ensure that your directory tree looks like this:

./
│
└── groupby-data/
    │
    ├── legislators-historical.csv
    ├── airqual.csv
    └── news.csv

With pandas installed, your virtual environment activated, and the datasets downloaded, you’re ready to jump in!

Example 1: U.S. Congress Dataset

You’ll jump right into things by dissecting a dataset of historical members of Congress. You can read the CSV file into a pandas DataFrame with read_csv():

Pythonpandas_legislators.py
      
    
import pandas as pd

dtypes = {
    "first_name": "category",
    "gender": "category",
    "type": "category",
    "state": "category",
    "party": "category",
}
df = pd.read_csv(
    "groupby-data/legislators-historical.csv",
    dtype=dtypes,
    usecols=list(dtypes) + ["birthday", "last_name"],
    parse_dates=["birthday"]
)

The dataset contains members’ first and last names, birthday, gender, type ("rep" for House of Representatives or "sen" for Senate), U.S. state, and political party. You can use df.tail() to view the last few rows of the dataset:

Python
      
        
      
    
>>> from pandas_legislators import df
>>> df.tail()
      last_name first_name   birthday gender type state       party
11970   Garrett     Thomas 1972-03-27      M  rep    VA  Republican
11971    Handel      Karen 1962-04-18      F  rep    GA  Republican
11972     Jones     Brenda 1959-10-24      F  rep    MI    Democrat
11973    Marino        Tom 1952-08-15      M  rep    PA  Republican
11974     Jones     Walter 1943-02-10      M  rep    NC  Republican

The DataFrame uses categorical dtypes for space efficiency:

Python
      
        
      
    
>>> df.dtypes
last_name             object
first_name          category
birthday      datetime64[ns]
gender              category
type                category
state               category
party               category
dtype: object

You can see that most columns of the dataset have the type category, which reduces the memory load on your machine.

Read the full article at https://realpython.com/pandas-groupby/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Create and Modify PDF Files in Python

2025-01-19T14:00:00+00:00

Creating and modifying PDF files in Python is straightforward with libraries like pypdf and ReportLab. You can read, manipulate, and create PDF files using these tools. pypdf lets you extract text, split, merge, rotate, crop, encrypt, and decrypt PDFs. ReportLab enables you to create new PDFs from scratch, allowing customization with fonts and page sizes.

By the end of this tutorial, you’ll understand that:

You can read and modify existing PDF files using pypdf in Python.
You can create new PDF files from scratch with the ReportLab library.
Methods to encrypt and decrypt a PDF file with a password are available in pypdf.
Concatenating and merging multiple PDF files can be done using pypdf.
You can add custom fonts to a PDF using ReportLab.
Python can create interactive PDFs with forms using ReportLab.

To follow along with this tutorial, you should download and extract to your home folder the materials used in the examples. To do this, click the link below:

Download the sample materials: Click here to get the materials you’ll use to learn about creating and modifying PDF files in this tutorial.

Extracting Text From PDF Files With `pypdf`

In this section, you’ll learn how to read PDF files and extract their text using the pypdf library. Before you can do that, though, you need to install it with pip:

Shell
      
$ python -m pip install pypdf

With this command, you download and install the latest version of pypdf from the Python package index (PyPI). To verify the installation, go ahead and run the following command in your terminal:

Shell
      
        
      
    
$ python -m pip show pypdf
Name: pypdf
Version: 3.8.1
Summary: A pure-python PDF library capable of splitting,
 merging, cropping, and transforming PDF files
Home-page:
Author:
Author-email: Mathieu Fenniak <biziqe@mathieu.fenniak.net>
License:
Location: .../lib/python3.10/site-packages
Requires:
Required-by:

Pay particular attention to the version information. At the time of publication for this tutorial, the latest version of pypdf was 3.8.1. This library has gotten plenty of updates lately, and cool new features are added quite frequently. Most importantly, you’ll find many breaking changes in the library’s API if you compare it with its predecessor library PyPDF2.

Before diving into working with PDF files, you must know that this tutorial is adapted from the chapter “Creating and Modifying PDF Files” in Python Basics: A Practical Introduction to Python 3.

The book uses Python’s built-in IDLE editor to create and edit Python files and interact with the Python shell, so you’ll find occasional references to IDLE throughout this tutorial. However, you should have no problems running the example code from the editor and environment of your choice.

Reading PDF Files With `PdfReader`

To kick things off, you’ll open a PDF file and read some information about it. You’ll use the Pride_and_Prejudice.pdf file provided in the downloadable resources for this tutorial.

Open IDLE’s interactive window and import the PdfReader class from pypdf:

Python
      
>>> from pypdf import PdfReader

To create a new instance of the PdfReader class, you’ll need to provide the path to the PDF file that you want to open. You can do that using the pathlib module:

Python
      
        
      
    
>>> from pathlib import Path

>>> pdf_path = (
...     Path.home()
...     / "creating-and-modifying-pdfs"
...     / "practice_files"
...     / "Pride_and_Prejudice.pdf"
... )

The pdf_path variable now contains the path to a PDF version of Jane Austen’s Pride and Prejudice.

Note: You may need to change pdf_path so that it corresponds to the location of the creating-and-modifying-pdfs/ folder on your computer.

Now create the PdfReader instance by calling the class’s constructor with the path to your PDF file as an argument:

Python
      
>>> pdf_reader = PdfReader(pdf_path)

If you’ve been following along in Python Basics, then you’ll remember from Chapter 12, “File Input and Output,” that all open files should be closed before a program terminates. The PdfReader object does all of this for you, so you don’t need to worry about opening or closing the PDF file!

Now that you’ve created a PdfReader instance, you can use it to gather information about the PDF file. For example, to get the number of pages contained in the PDF file, you can use the built-in len() function like in the code below:

Read the full article at https://realpython.com/creating-modifying-pdf/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #235: Principles for Considering Your Python Tooling

2025-01-17T12:00:00+00:00

What are the principles you should consider when making decisions about which Python tools to use? What anti-patterns get in the way of making the right choices for your team? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Building Dictionary Comprehensions in Python

2025-01-14T14:00:00+00:00

Dictionary comprehensions are a concise and quick way to create, transform, and filter dictionaries in Python. They can significantly enhance your code’s conciseness and readability compared to using regular for loops to process your dictionaries.

Understanding dictionary comprehensions is crucial for you as a Python developer because they’re a Pythonic tool for dictionary manipulation and can be a valuable addition to your programming toolkit.

In this video course, you’ll learn how to:

Create dictionaries using dictionary comprehensions
Transform existing dictionaries with comprehensions
Filter key-value pairs from dictionaries using conditionals
Decide when to use dictionary comprehensions

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #234: Building New Structures for Learning Python

2025-01-10T12:00:00+00:00

What are the new ways we can teach and share our knowledge about Python? How can we improve the structure of our current offerings and build new educational resources for our audience of Python learners? This week on the show, Real Python core team members Stephen Gruppetta and Martin Breuss join us to discuss enhancements to the site and new ways to learn Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Ways to Start Interacting With Python

2025-01-07T14:00:00+00:00

There are multiple ways of interacting with Python, and each can be useful for different scenarios. You can quickly explore functionality in Python’s interactive mode using the built-in Read-Eval-Print Loop (REPL), or you can write larger applications to a script file using an editor or Integrated Development Environment (IDE).

In this video course, you’ll learn how to:

Use Python interactively by typing code directly into the interpreter
Execute code contained in a script file from the command line
Work within a Python Integrated Development Environment (IDE)
Assess additional options, such as Thonny and Python’s IDLE

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #233: PyCoder's Weekly 2024 Top Articles & Missing Gems

2025-01-03T12:00:00+00:00

PyCoder's Weekly included over 1,500 links to articles, blog posts, tutorials, and projects in 2024. Christopher Trudeau is back on the show this week to help wrap it all up by sharing some highlights and uncovering a few missing gems from the pile.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Building HTTP APIs With Django REST Framework

2024-12-31T14:00:00+00:00

REST is a loosely defined protocol for listing, creating, changing, and deleting data on your server over HTTP. The Django REST framework (DRF) is a toolkit built on top of the Django web framework that reduces the amount of code you need to write to create REST HTTP API interfaces.

In this course you’ll learn about:

The REST protocol
DRF Serializers and how to use them with Django objects
Using Django views and DRF ViewSet classes to create REST end-points
Multiple flavors of renderers and how to control their output
Specifying permissions and limiting who can see what data in your REST API

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #232: Exploring Modern Sentiment Analysis Approaches in Python

2024-12-20T12:00:00+00:00

What are the current approaches for analyzing emotions within a piece of text? Which tools and Python packages should you use for sentiment analysis? This week, Jodie Burchell, developer advocate for data science at JetBrains, returns to the show to discuss modern sentiment analysis in Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: How to Remove Items From Lists in Python

2024-12-18T12:00:00+00:00

In this quiz, you’ll test your understanding of How to Remove Items From Lists in Python.

By working through this quiz, you’ll revisit the different approaches to removing items from a list in Python, including .pop(), .remove(), the del statement, and more.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Programming Sockets in Python

2024-12-17T14:00:00+00:00

Sockets and the socket API are used to send messages across a network. They provide a form of inter-process communication (IPC). The network can be a logical, local network to the computer, or one that’s physically connected to an external network with its own connections to other networks. The obvious example is the Internet, which you connect to via your ISP.

In this video course, you’ll create:

A simple socket server and client
An improved version that handles multiple connections simultaneously
A server-client application that functions like a full-fledged socket application, complete with its own custom header and content

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Documenting Python Projects With Sphinx and Read the Docs

2024-12-10T14:00:00+00:00

Sphinx is a document generation tool that’s become the de facto standard for Python projects. It uses the reStructuredText (RST) markup language to define document structure and styling, and it can output in a wide variety of formats, including HTML, ePub, man pages, and much more. Sphinx is extendable and has plugins for incorporating pydoc comments from your code into your docs and for using MyST Markdown instead of RST.

Read the Docs is a free document hosting site where many Python projects host their documentation. It integrates with GitHub, GitLab, and Bitbucket to automatically pull new documentation sources from your repositories and build their Sphinx sources.

In this video course, you’ll learn how to:

Write your documentation with Sphinx
Structure and style your document with RST syntax
Incorporate your pydoc comments into your documentation
Host your documentation on Read the Docs

With these skills, you’ll be able to write clear, reliable documentation that’ll help your users get the most out of your project.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #231: Good Python Programming Practices When New to the Language

2024-12-06T12:00:00+00:00

What advice would you give to someone moving from another language to Python? What good programming practices are inherent to the language? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Handling or Preventing Errors in Python: LBYL vs EAFP

2024-12-03T14:00:00+00:00

Dealing with errors and exceptional situations is a common requirement in programming. You can either prevent errors before they happen or handle errors after they’ve happened. In general, you’ll have two coding styles matching these strategies: look before you leap (LBYL), and easier to ask forgiveness than permission (EAFP). In this video course, you’ll dive into the questions and considerations surrounding LBYL vs EAFP in Python.

By learning about Python’s LBYL and EAFP coding styles, you’ll be able to decide which strategy and coding style to use when you’re dealing with errors in your code.

In this video course, you’ll learn how to:

Use the LBYL and EAFP styles in your Python code
Understand the pros and cons of LBYL vs EAFP
Decide when to use either LBYL or EAFP

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #230: marimo: Reactive Notebooks and Deployable Web Apps in Python

2024-11-29T12:00:00+00:00

What are common issues with using notebooks for Python development? How do you know the current state, share reproducible results, or create interactive applications? This week on the show, we speak with Akshay Agrawal about the open-source reactive marimo notebook for Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Managing Dependencies With Python Poetry

2024-11-26T14:00:00+00:00

When your Python project relies on external packages, you need to make sure you’re using the right version of each package. After an update, a package might not work as it did before. A dependency manager like Python Poetry helps you specify, install, and resolve external packages in your projects. This way, you can be sure that you always work with the correct dependency version on every machine.

In this video course, you’ll learn how to:

Create a new project using Poetry
Add Poetry to an existing project
Configure your project through pyproject.toml
Pin your project’s dependency versions
Install dependencies from a poetry.lock file
Run basic Poetry commands using the Poetry CLI

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #229: The Joy of Tinkering & Python Free-Threading Performance

2024-11-22T12:00:00+00:00

What keeps your spark alive for developing software and learning Python? Do you like to try new frameworks, build toy projects, or collaborate with other developers? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: Expression vs Statement in Python: What's the Difference?

2024-11-21T12:00:00+00:00

In this quiz, you’ll test your understanding of Expression vs Statement in Python: What’s the Difference?

By working through this quiz, you’ll revisit the key differences between expressions and statements in Python, and how to use them effectively in your code.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: Interacting With Python

2024-11-21T12:00:00+00:00

In this quiz, you’ll test your understanding of the different ways you can interact with Python.

By working through this quiz, you’ll revisit key concepts related to Python interaction in interactive mode using the Read-Eval-Print Loop (REPL), through Python script files, and within Integrated Development Environments (IDEs) and code editors.

You’ll also test your knowledge of some other options that may be useful, such as Jupyter Notebooks.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: NumPy Practical Examples: Useful Techniques

2024-11-20T12:00:00+00:00

In this quiz, you’ll test your understanding of the techniques covered in the tutorial NumPy Practical Examples: Useful Techniques.

By working through the questions, you’ll review your understanding of NumPy arrays and also expand on what you learned in the tutorial.

You’ll need to do some research outside of the tutorial to answer all the questions. Embrace this challenge and let it take you on a learning journey.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Working With TOML and Python

2024-11-19T14:00:00+00:00

TOML—Tom’s Obvious Minimal Language—is a reasonably new configuration file format that the Python community has embraced over the last couple of years. TOML plays an essential part in the Python ecosystem. Many of your favorite tools rely on TOML for configuration, and you’ll use pyproject.toml when you build and distribute your own packages.

In this video course, you’ll learn more about TOML and how you can use it. In particular, you’ll:

Learn and understand the syntax of TOML
Use tomli and tomllib to parse TOML documents
Use tomli_w to write data structures as TOML
Use tomlkit when you need more control over your TOML files

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #228: Maintaining the Foundations of Python & Cautionary Tales

2024-11-15T12:00:00+00:00

How do you build a sustainable open-source project and community? What lessons can be learned from Python's history and the current mess that the WordPress community is going through? This week on the show, we speak with Paul Everitt from JetBrains about navigating open-source funding and the start of the Python Software Foundation.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: Namespaces and Scope in Python

2024-11-14T12:00:00+00:00

In this quiz, you’ll test your understanding of Python Namespaces and Scope.

You’ll revisit how Python organizes symbolic names and objects in namespaces, when Python creates a new namespace, how namespaces are implemented, and how variable scope determines symbolic name visibility.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: Python Dictionary Comprehensions: How and When to Use Them

2024-11-13T12:00:00+00:00

In this quiz, you’ll test your understanding of Python Dictionary Comprehensions: How and When to Use Them.

Understanding dictionary comprehensions is crucial for you as a Python developer because they provide a Pythonic tool for dictionary manipulation and can be a valuable addition to your programming toolkit.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: Basic Input and Output in Python

2024-11-13T12:00:00+00:00

In this quiz, you’ll test your understanding of how to use Python’s built-in functions input() and print() for basic input and output operations.

You’ll also revisit how to use readline to improve the user experience when collecting input, and how to format output using the sep and end keyword arguments of print().

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Formatting Floats Inside Python F-Strings

2024-11-12T14:00:00+00:00

You’ll often need to format and round a Python float to display the results of your calculations neatly within strings. In earlier versions of Python, this was a messy thing to do because you needed to round your numbers first and then use either string concatenation or the old string formatting technique to do this for you.

Since Python 3.6, the literal string interpolation, more commonly known as a formatted string literal or f-string, allows you to customize the content of your strings in a more readable way.

An f-string is a literal string prefixed with a lowercase or uppercase letter f and contains zero or more replacement fields enclosed within a pair of curly braces {...}. Each field contains an expression that produces a value. You can calculate the field’s content, but you can also use function calls or even variables.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #227: New PEPs: Template Strings & External Wheel Hosting

2024-11-08T12:00:00+00:00

Have you wanted the flexibility of f-strings but need safety checks in place? What if you could have deferred evaluation for logging or avoiding injection attacks? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: Variables in Python: Usage and Best Practices

2024-11-05T12:00:00+00:00

In this quiz, you’ll test your understanding of Variables in Python: Usage and Best Practices.

By working through this quiz, you’ll revisit how to create and assign values to variables, change a variable’s data type dynamically, use variables to create expressions, counters, accumulators, and Boolean flags, follow best practices for naming variables, and create, access, and use variables in their scopes.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Quiz: Using .repr() vs .str() in Python

2024-10-29T12:00:00+00:00

In this quiz, you’ll test your understanding of Python’s .__repr__() and .__str__() special methods. These methods allow you to control how a program displays an object, making your classes more readable and easier to debug and maintain.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Real Python

Python's raise: Effectively Raising Exceptions in Your Code

Handling Exceptional Situations in Python

Executing Python Scripts With a Shebang

What’s a Shebang, and When Should You Use It?

How Does a Shebang Work?

How to Flush the Output of the Python Print Function

Understand How Python Buffers Output

Add a Newline for Python to Flush Print Output

How to Download Files From URLs With Python

Facilitating File Downloads With Python

Downloading a File From a URL in Python

Using urllib From the Standard Library

Python and TOML: New Best Friends

Use TOML as a Configuration Format

Configurations and Configuration Files

TOML: Tom’s Obvious Minimal Language

The Real Python Podcast – Episode #236: Simon Willison: Using LLMs for Python Development

How to Deal With Missing Data in Polars

Exploring Python's tuple Data Type With Examples

Quiz: How to Deal With Missing Data in Polars

Getters and Setters: Manage Attributes in Python

Getting to Know Getter and Setter Methods

What Are Getter and Setter Methods?

Python Constants: Improve Your Code's Maintainability

Understanding Constants and Variables

What Variables Are

What Constants Are

Why Use Constants

pandas GroupBy: Your Guide to Grouping Data in Python

Prerequisites

Example 1: U.S. Congress Dataset

Create and Modify PDF Files in Python

Extracting Text From PDF Files With pypdf

Reading PDF Files With PdfReader

The Real Python Podcast – Episode #235: Principles for Considering Your Python Tooling

Building Dictionary Comprehensions in Python

The Real Python Podcast – Episode #234: Building New Structures for Learning Python

Ways to Start Interacting With Python

The Real Python Podcast – Episode #233: PyCoder's Weekly 2024 Top Articles & Missing Gems

Building HTTP APIs With Django REST Framework

The Real Python Podcast – Episode #232: Exploring Modern Sentiment Analysis Approaches in Python

Quiz: How to Remove Items From Lists in Python

Programming Sockets in Python

Documenting Python Projects With Sphinx and Read the Docs

The Real Python Podcast – Episode #231: Good Python Programming Practices When New to the Language

Handling or Preventing Errors in Python: LBYL vs EAFP

The Real Python Podcast – Episode #230: marimo: Reactive Notebooks and Deployable Web Apps in Python

Managing Dependencies With Python Poetry

The Real Python Podcast – Episode #229: The Joy of Tinkering & Python Free-Threading Performance

Quiz: Expression vs Statement in Python: What's the Difference?

Quiz: Interacting With Python

Quiz: NumPy Practical Examples: Useful Techniques

Working With TOML and Python

The Real Python Podcast – Episode #228: Maintaining the Foundations of Python & Cautionary Tales

Quiz: Namespaces and Scope in Python

Quiz: Python Dictionary Comprehensions: How and When to Use Them

Quiz: Basic Input and Output in Python

Formatting Floats Inside Python F-Strings

The Real Python Podcast – Episode #227: New PEPs: Template Strings & External Wheel Hosting

Quiz: Variables in Python: Usage and Best Practices

Quiz: Using .__repr__() vs .__str__() in Python

Using `urllib` From the Standard Library

Extracting Text From PDF Files With `pypdf`

Reading PDF Files With `PdfReader`

Quiz: Using .repr() vs .str() in Python