Real Python

Serialize Your Data With Python

2023-12-04T14:00:00+00:00

Whether you’re a data scientist crunching big data in a distributed cluster, a back-end engineer building scalable microservices, or a front-end developer consuming web APIs, you should understand data serialization. In this comprehensive guide, you’ll move beyond XML and JSON to explore several data formats that you can use to serialize data in Python. You’ll explore them based on their use cases, learning about their distinct categories.

By the end of this tutorial, you’ll have a deep understanding of the many data interchange formats available. You’ll master the ability to persist and transfer stateful objects, effectively making them immortal and transportable through time and space. Finally, you’ll learn to send executable code over the network, unlocking the potential of remote computation and distributed processing.

In this tutorial, you’ll learn how to:

Choose a suitable data serialization format
Take snapshots of stateful Python objects
Send executable code over the wire for distributed processing
Adopt popular data formats for HTTP message payloads
Serialize hierarchical, tabular, and other shapes of data
Employ schemas for validating and evolving the structure of data

To get the most out of this tutorial, you should have a good understanding of object-oriented programming principles, including classes and data classes, as well as type hinting in Python. Additionally, familiarity with the HTTP protocol and Python web frameworks would be a plus. This knowledge will make it easier for you to follow along with the tutorial.

You can download all the code samples accompanying this tutorial by clicking the link below:

Get Your Code: Click here to download the free sample code that shows you how to serialize your data with Python.

Feel free to skip ahead and focus on the part that interests you the most, or buckle up and get ready to catapult your data management skills to a whole new level!

Get an Overview of Data Serialization

Serialization, also known as marshaling, is the process of translating a piece of data into an interim representation that’s suitable for transmission through a network or persistent storage on a medium like an optical disk. Because the serialized form isn’t useful on its own, you’ll eventually want to restore the original data. The inverse operation, which can occur on a remote machine, is called deserialization or unmarshaling.

Note: Although the terms serialization and marshaling are often used interchangeably, they can have slightly different meanings for different people. In some circles, serialization is only concerned with the translation part, while marshaling is also about moving data from one place to another.

The precise meaning of each term depends on whom you ask. For example, Java programmers tend to use the word marshaling in the context of remote method invocation (RMI). In Python, marshaling refers almost exclusively to the format used for storing the compiled bytecode instructions.

Check out the comparison of serialization and marshaling on Wikipedia for more details.

The name serialization implies that your data, which may be structured as a dense graph of objects in the computer’s memory, becomes a linear sequence—or a series—of bytes. Such a linear representation is perfect to transmit or store. Raw bytes are universally understood by various programming languages, operating systems, and hardware architectures, making it possible to exchange data between otherwise incompatible systems.

When you visit an online store using your web browser, chances are it runs a piece of JavaScript code in the background to communicate with a back-end system. That back end might be implemented in Flask, Django, or FastAPI, which are Python web frameworks. Because JavaScript and Python are two different languages with distinct syntax and data types, they must share information using an interchange format that both sides can understand.

In other words, parties on opposite ends of a digital conversation may deserialize the same piece of information into wildly different internal representations due to their technical constraints and specifications. However, it would still be the same information from a semantic point of view.

Tools like Node.js make it possible to run JavaScript on the back end, including isomorphic JavaScript that can run on both the client and the server in an unmodified form. This eliminates language discrepancies altogether but doesn’t address more subtle nuances, such as big-endian vs little-endian differences in hardware.

Other than that, transporting data from one machine to another still requires converting it into a network-friendly format. Specifically, the format should allow the sender to partition and put the data into network packets, which the receiving machine can later correctly reassemble. Network protocols are fairly low-level, so they deal with streams of bytes rather than high-level data types.

Depending on your use case, you’ll want to pick a data serialization format that offers the best trade-off between its pros and cons. In the next section, you’ll learn about various categories of data formats used in serialization. If you already have prior knowledge about these formats and would like to explore their respective scenarios, then feel free to skip the basic introduction coming up next.

Compare Data Serialization Formats

There are many ways to classify data serialization formats. Some of these categories aren’t mutually exclusive, making certain formats fall under a few of them simultaneously. In this section, you’ll find an overview of the different categories, their trade-offs, and use cases, as well as examples of popular data serialization formats.

Later, you’ll get your hands on some practical applications of these data serialization formats under different programming scenarios. To follow along, download the sample code mentioned in the introduction and install the required dependencies from the included requirements.txt file into an active virtual environment by issuing the following command:

Shell
      
(venv) $ python -m pip install -r requirements.txt

This will install several third-party libraries, frameworks, and tools that will allow you to navigate through the remaining part of this tutorial smoothly.

Textual vs Binary

At the end of the day, all serialized data becomes a stream of bytes regardless of its original shape or form. But some byte values—or their specific arrangement—may correspond to Unicode code points with a meaningful and human-readable representation. Data serialization formats whose syntax consists purely of characters visible to the naked eye are called textual data formats, as opposed to binary data formats meant for machines to read.

The main benefit of a textual data format is that people like you can read serialized messages, make sense of them, and even edit them by hand when needed. In many cases, these data formats are self-explanatory, with descriptive element or attribute names. For example, take a look at this excerpt from the Real Python web feed with information about the latest tutorials and courses published:

Read the full article at https://realpython.com/python-serialize-data/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #182: Building a Python JSON Parser & Discussing Ideas for PEPs

2023-12-01T12:00:00+00:00

Have you thought of a way to improve the Python language? How do you share your idea with core developers and start a discussion in the Python community? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Single and Double Underscores in Python Names

2023-11-29T14:00:00+00:00

Python has a few important naming conventions that are based on using either a single or double underscore character (_). These conventions allow you to differentiate between public and non-public names in APIs, write safe classes for subclassing purposes, avoid name clashes, and more.

Following and respecting these conventions allows you to write code that looks Pythonic and consistent in the eyes of other Python developers. This skill is especially useful when you’re writing code that’s intended for other developers to work with.

In this tutorial, you’ll:

Learn about Python naming conventions that rely on using underscores (_)
Differentiate public and non-public names by using a single leading underscore
Use double leading underscores to leverage name mangling in Python classes
Explore other common uses of underscores in Python names

To get the most out of this tutorial, you should be familiar with Python variables, constants, functions, modules, and especially classes and object-oriented programming.

Get Your Code: Click here to download the free sample code that shows you how to use single and double underscores in Python names.

Public Interfaces and Naming Conventions in Python

As a Python programmer, you’ll frequently work with public interfaces, or application programming interfaces (API). An API is a type of programming interface that offers a service to other parts of a program or other programs.

For example, the Python standard library has many modules and packages that provide certain services. To use these modules and packages, you need to access their public components, such as classes, functions, variables, constants, and modules. All these objects are part of the module or package’s public interface. They’re available for you to use directly in your code.

However, many of these packages and modules define objects that aren’t intended for direct access. These objects are for the internal use of the specific package or module only. They’re not part of its public interface.

In the context of object-oriented programming, languages like C++ and Java have the notion of public and private methods and attributes. In these languages, you can use these types of class members as follows:

Public: You can use them in your own code or client code.
Private: You can use them only from inside the defining class and its subclasses.

These languages have specific keywords and syntax to define public and private members in their classes. Once you declare a member as private, you can’t use it outside the class because the language restricts access. So, private members aren’t part of the class’s public interface, and there’s no way to access them.

In contrast, Python doesn’t have the notion of public and private members. It has neither dedicated keywords nor syntax for defining them. So, you can always access the members of a Python class.

If Python doesn’t have a specific syntax to define when an object is part of a public interface, then how do you tell your users that they can or can’t use a given class, method, function, variable, constant, or even module in their code?

To approach this question, the Python community has well-established naming conventions. You should observe these naming conventions to explicitly indicate whether other developers should directly use your variables, constants, functions, methods, and modules in external code.

Note that the naming conventions don’t restrict access to objects. They’re just a warning to other developers using your code. Because of this, Python doesn’t use the terms public and private. Instead, it uses the terms public and non-public.

In Python, if a name starts with a letter in uppercase or lowercase, then you should consider that name public and, therefore, part of the code’s API. In contrast, if a name starts with an underscore character (_), then you should consider that name non-public, meaning it’s not a part of the code’s API.

The Python community uses the underscore character (_) as part of other naming conventions. Here’s a summary of what PEP 8 says about using this character in names:

Convention	Example	Meaning
Single leading underscore	`_variable`	Indicates that the name is meant for internal use only
Single trailing underscore	`class_`	Avoids naming conflicts with Python keywords and built-in names
Double leading underscore	`__attribute`	Triggers name mangling in the context of Python classes
Double leading and trailing underscore	`__name__`	Indicates special attributes and methods that Python provides
Single underscore	`_`	Indicates a temporary or throwaway variable

Note that only two of these conventions enforce specific Python behaviors. Using double leading underscores triggers name mangling in Python classes. You’ll learn more about this behavior in the section on name mangling.

Additionally, those names with double leading and trailing underscores that are listed in the Python data model trigger internal behaviors in specific contexts. You’ll also learn more about this topic in the section on dunder names in Python.

Note: Python gives special treatment to the single underscore (_), but only within match … case statements. You’ll learn more about this later.

In the following section, you’ll learn more about using a single leading underscore when naming your Python objects.

Single Leading Underscore in Python Names

Read the full article at https://realpython.com/python-double-underscore/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Advent of Code: Solving Puzzles With Python

2023-11-28T14:00:00+00:00

Advent of Code is an online Advent calendar where you’ll find new programming puzzles offered each day from December 1 to 25. While you can solve the puzzles at any time, the excitement when new puzzles unlock is really something special. You can participate in Advent of Code in any programming language—including Python!

With the help of this Code Conversation, you’ll be ready to start solving puzzles and earning your first gold stars.

In this video course, you’ll learn:

What an online Advent calendar is
How solving puzzles can advance your programming skills
How you can participate in Advent of Code

Advent of Code puzzles are designed to be approachable by anyone with an interest in problem-solving. You don’t need a heavy computer science background to participate. Instead, Advent of Code is a great arena for learning new skills and testing out new features of Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Python Rich Package: Unleash the Power of Console Text

2023-11-27T14:00:00+00:00

Python’s Rich package is a tool kit that helps you generate beautifully formatted and highlighted text in the console. More broadly, it allows you to build an attractive text-based user interface (TUI).

Why would you choose a TUI over a graphical user interface, or GUI? Sometimes a text display feels more appropriate. Why use a full-blown GUI for a simple application, when an elegant text interface will do? It can be refreshing to work with plain text. Text works in almost any hardware environment, even on an SSH terminal or a single-board computer display. And many applications don’t need the complexity of a full graphical windowing system.

In this tutorial, you’ll learn how Rich can help you:

Enhance the user interface of command-line tools
Improve the readability of console output
Create attractive dashboard displays for real-time tabular data
Generate well-formatted reports

Will McGugan, the author of Rich, has also developed the Textual package. Whereas Rich is a rich-text tool kit, Textual is a full application framework built on Rich. It provides application base classes, an event-driven architecture, and more.

There’s a lot you can do with Rich on its own, and its support for engaging, dynamic displays may well be sufficient for your app. By following this tutorial, you’ll experiment with many of the cool features of Rich, and you’ll finish up by using your skills to build a dynamically scrolling tabular display of crypto prices:

To fully understand Rich’s syntax for animations, you should have a good grasp of context managers. But if you’re a bit rusty, don’t worry! You’ll get a quick refresher in this tutorial.

Get Your Code: Click here to download free sample code that shows you how to use Rich for more beautiful Python code and apps.

Installing Rich

You can start using Rich very quickly. As always when starting a new project or investigation, it’s best to create a virtual environment first, to avoid polluting your system’s Python installation.

It’s quite possible to install Rich and use it with the built-in Python REPL, but for a better developer experience, you may want to include support for Jupyter notebooks. Here’s how you can install Rich so that it’ll work with either the REPL or Jupyter:

Windows PowerShell
      
PS> python -m venv venv
PS> venv\Scripts\activate
(venv) PS> python -m pip install rich[jupyter]

Shell
      
$ python -m venv venv
$ source venv/bin/activate
(venv) $ python -m pip install "rich[jupyter]"

Now that you’ve installed Rich in your new virtual environment, you can test it and also get a nice overview of its capabilities:

Shell
      
(venv) $ python -m rich

Running this command will make lots of magic happen. Your terminal will fill with color, and you’ll see several options for customizing your text-based user interface:

Apart from displaying colorful text in a variety of styles, this demo also illustrates a few more of Rich’s exciting features.

You can wrap and justify text. You can easily display any Unicode characters, as well as a wide choice of emojis. Rich renders Markdown and offers a syntax for creating elegantly formatted tables.

There’s much more that you can do with Rich, as you’ll discover throughout this tutorial. You can also try some of the command-line demos that Rich has thoughtfully provided for its subpackages, so you can get a flavor of what each one can do without writing any code.

Here are a few that you can try from your OS console. Execute them one by one to get a feel for the power of Rich:

Shell
      
(venv) $ python -m rich.table
(venv) $ python -m rich.progress
(venv) $ python -m rich.status

Most of these demos are very short, but if necessary, you can always interrupt them with Ctrl+C.

With the installation done, you’re ready to start exploring Rich.

Read the full article at https://realpython.com/python-rich-package/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Build Conway's Game of Life With Python

2023-11-22T14:00:00+00:00

Wouldn’t it be cool to build a Python game that only requires initial user input and then seems to take on a mind of its own, creating mesmerizing patterns along the way? You can do exactly that with Conway’s Game of Life, which is about the evolution of cells in a life grid.

Implementing the Game of Life algorithm is a good exercise with many interesting challenges that you’ll have to figure out. Specifically, you’ll need to build the life grid and find a way to apply the game’s rules to all the cells on the grid so that they evolve through several generations.

In this tutorial, you’ll:

Implement Conway’s Game of Life algorithm with Python
Build a curses view to display the Game of Life grid
Create an argparse command-line interface for the game
Set up the game app for installation and execution

To get the most out of this tutorial, you should know the basics of writing object-oriented code in Python, creating command-line interface (CLI) apps with argparse, and setting up a Python project.

You can download the complete source code and other resources for this project by clicking the link below:

Get Your Code: Click here to download the source code to build Conway’s Game of Life with Python.

Demo: Conway’s Game of Life With Python

In this tutorial, you’ll implement Conway’s Game of Life for your command line using Python. Once you’ve run all the steps to build the game, then you’ll end up with a fully working command-line app.

The demo shows how the app works and walks you through the evolution of multiple life patterns or seeds, which define the game’s initial state and work as starting points for evolving the cells in the life grid:

Throughout this tutorial, you’ll run through several challenges related to the game’s algorithm and also to writing and setting up a command-line application in Python. At the end, you’ll have a Game of Life app that will work like the demo above.

Project Overview

The Game of Life by the British mathematician John Horton Conway isn’t a game in the traditional sense. In technical terms, it’s a cellular automaton, but you can think of Game of Life as a simulation whose evolution depends on its initial state and doesn’t require further input from any players.

The game’s board is an infinite, two-dimensional grid of cells. Each cell can be in one of two possible states:

Alive
Dead

Each cell evolves to the next generation depending on the state of itself and its neighbor cells. Here’s a summary of the evolution rules:

Alive cells die if they have fewer than two (underpopulation) or more than three living neighbors (overpopulation).
Alive cells stay alive if they have two or three living neighbors.
Dead cells with exactly three living neighbors become alive (reproduction).

The game’s initial state is the seed, or initial life pattern. In this implementation, the life pattern will be a set of alive cells. The first generation results from applying the above rules to every cell in the seed. The second generation results from applying the rules to the first generation, and so on. So, each generation is a pure function of the preceding one.

The challenge in this project is to program the evolution algorithm in Python and then provide a command-line interface (CLI) to run the game with different life patterns.

Prerequisites

The project that you’ll build in this tutorial will require familiarity with general Python programming and especially with object-oriented programming. So, you should have basic knowledge of the following topics:

Working with conditional statements
Writing for loops and comprehensions
Working with Python strings, tuples, lists, and sets
Creating regular classes and data classes
Building command-line interfaces with argparse
Working with TOML files in Python

However, if you don’t have all this knowledge yet, then that’s okay! You might learn more by going ahead and giving the project a shot. You can always stop and review the resources linked here if you get stuck.

With this short overview of your Game of Life project and its prerequisites, you’re ready to start Pythoning. Have fun while coding!

Read the full article at https://realpython.com/conway-game-of-life-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Basics Exercises: Modules and Packages

2023-11-21T14:00:00+00:00

In Python Basics: Modules and Packages, you learned how to build an application by putting related code into separate files called modules. You also used the import statement to use modules in another file.

In this video course, you’ll practice:

Creating your own modules
Using modules in another file through the import statement
Organizing several modules into a package with __init__.py

Along the way, you’ll also get some insight into how to tackle coding challenges in general, which can be a great way to level up as a developer.

This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.

Note that you’ll be using IDLE to interact with Python throughout this course. If you’re just getting started, then you might want to check out Python Basics: Setting Up Python before diving into this course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

How to Render Markdown in a Django Application

2023-11-20T14:00:00+00:00

One of Django’s most popular features is the admin interface. For developers, it provides an out-of-the-box tool kit to work with your project’s data models. And it gives non-technical administrators an intuitive user interface (UI) for interacting with the database and creating website content. However, the Django admin doesn’t provide a way to format text-based content. Using Markdown with Django is a great solution.

Markdown is a plain text formatting language that can automatically convert to HTML for publishing on the Web. Due to its accessible syntax and widespread use in static site generators, productivity apps, and content management tools, it’s become a popular alternative to rich-text editors for both technical and non-technical content creators.

There are also several third-party What You See is What You Get (WYSIWYG) editors available for Django. These provide an interface for formatting text by using buttons on a menu bar while writing, similar to a word processor program. Other options include using a content management system such as Wagtail CMS, or working directly in HTML template files.

But when used in combination with the Django admin, Markdown provides a simpler alternative to a full-blown CMS and a friendlier user experience than working with HTML templates for both developers and content authors.

Many writers find it faster and more straightforward to work with Markdown than a WYSIWYG interface. Plus, there are additional benefits to separating the text and presentation layers of your web content, including increased security, less risk of syntax errors, and the ability to publish across multiple formats, such as email or PDF documents.

In this tutorial, you’ll learn how to render Markdown content with Django so that your website’s content creators can take advantage of the simplicity and convenience that have made Markdown such a popular choice for formatting text-based content. This tutorial assumes that you have some prior Django experience, so if you’re totally new to the framework, then you can learn to build your first Django app here.

Get Your Code: Click here to download the free sample code that shows you how to render Markdown in your Django apps.

How to Use Markdown for Content in a Django Application

In a Django application, incorporating Markdown can enhance the content management workflow for users who prefer to format their writing in plain text over a WYSIWYG editor. In this section of the tutorial, you’ll learn how to extend a Django application to allow the creation, management, and rendering of Markdown content.

You’ll walk through setting up a Django project and app, defining a model to store Markdown content, and creating Django templates to render that content as HTML. Additionally, you’ll explore using Python-Markdown to convert Markdown to HTML within your Django views.

By the end of this section, you’ll have a fully functional Markdown content management system within your Django application.

Create a Django App to Display Content

To render HTML content written as Markdown with Django, begin by creating a new Django project named dmd_project inside a Python virtual environment. You use dmd as a shorthand for Django Markdown:

Windows PowerShell
      
(venv) PS> mkdir django-markdown
(venv) PS> django-admin startproject dmd_project django-markdown

Shell
      
(venv) $ mkdir django-markdown/
(venv) $ django-admin startproject dmd_project django-markdown/

The only dependencies that you’ll need for this tutorial are Django and the Python-Markdown package, which provides an API for working with Markdown in Python. Install markdown into your virtual environment with pip:

Windows PowerShell
      
(venv) PS> python -m pip install markdown

Shell
      
(venv) $ python -m pip install markdown

Now create a new Django app with the manage.py startapp command:

Windows PowerShell
      
(venv) PS> cd django-markdown
(venv) PS> python manage.py startapp dmd_app

Shell
      
(venv) $ cd django-markdown/
(venv) $ python manage.py startapp dmd_app

You’ve used the startapp command to create the dmd_app/ folder, which contains the necessary files for composing that content through the Django admin interface, storing it in your project’s database, and displaying it to users as HTML. Add your new app to the INSTALLED_APPS setting in your project’s settings.py file:

Pythondmd_project/settings.py
      
    
INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "dmd_app",
]

To create a new database table for storing your project’s content, define a MarkdownContent model in the models.py file inside your dmd_app/ directory:

Pythondmd_app/models.py
      
    
from django.db import models

class MarkdownContent(models.Model):
    title = models.CharField(max_length=100)
    content = models.TextField()

    class Meta:
        verbose_name_plural = "Markdown content"

    def __str__(self):
        return self.title

Read the full article at https://realpython.com/django-markdown/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #181: Computational Thinking & Learning Python During an AI Revolution

2023-11-17T12:00:00+00:00

Has the current growth of artificial intelligence (AI) systems made you wonder what the future holds for Python developers? What are the hidden benefits of learning to program in Python and practicing computational thinking? This week on the show, we speak with author Lawrence Gray about his upcoming book "Mastering Python: A Problem Solving Approach."

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Embeddings and Vector Databases With ChromaDB

2023-11-15T14:00:00+00:00

The era of large language models (LLMs) is here, bringing with it rapidly evolving libraries like ChromaDB that help augment LLM applications. You’ve most likely heard of chatbots like OpenAI’s ChatGPT, and perhaps you’ve even experienced their remarkable ability to reason about natural language processing (NLP) problems.

Modern LLMs, while imperfect, can accurately solve a wide range of problems and provide correct answers to many questions. But, due to the limits of their training and the number of text tokens they can process, LLMs aren’t a silver bullet for all tasks.

You wouldn’t expect an LLM to provide relevant responses about topics that don’t appear in their training data. For example, if you asked ChatGPT to summarize information in confidential company documents, then you’d be out of luck. You could show some of these documents to ChatGPT, but there’s a limited number of documents that you can upload before you exceed ChatGPT’s maximum number of tokens. How would you select documents to show ChatGPT?

To address these shortcomings and scale your LLM applications, one great option is to use a vector database like ChromaDB. A vector database allows you to store encoded unstructured objects, like text, as lists of numbers that you can compare to one another. You can, for example, find a collection of documents relevant to a question that you want an LLM to answer.

In this tutorial, you’ll learn about:

Representing unstructured objects with vectors
Using word and text embeddings in Python
Harnessing the power of vector databases
Encoding and querying over documents with ChromaDB
Providing context to LLMs like ChatGPT with ChromaDB

After reading, you’ll have the foundational knowledge to use ChromaDB in your NLP or LLM applications. Before reading, you should be comfortable with the basics of Python and high school math.

Get Your Code: Click here to download free sample code that shows you how to use ChromaDB to add context to an LLM.

Represent Data as Vectors

Before diving into embeddings and vector databases, you should understand what vectors are and what they represent. Feel free to skip ahead to the next section if you’re already comfortable with vector concepts. If you’re not or if you could use a refresher, then keep reading!

Vector Basics

You can describe vectors with variable levels of complexity, but one great starting place is to think of a vector as an array of numbers. For example, you could represent vectors using NumPy arrays as follows:

Python
      
>>> import numpy as np

>>> vector1 = np.array([1, 0])
>>> vector2 = np.array([0, 1])
>>> vector1
array([1, 0])

>>> vector2
array([0, 1])

In this code block, you import numpy and create two arrays, vector1 and vector2, representing vectors. This is one of the most common and useful ways to work with vectors in Python, and NumPy offers a variety of functionality to manipulate vectors. There are also several other libraries that you can use to work with vector data, such as PyTorch, TensorFlow, JAX, and Polars. You’ll stick with NumPy for this overview.

You’ve created two NumPy arrays that represent vectors. Now what? It turns out you can do a lot of cool things with vectors, but before continuing on, you’ll need to understand some key definitions and properties:

Dimension: The dimension of a vector is the number of elements that it contains. In the example above, vector1 and vector2 are both two-dimensional since they each have two elements. You can only visualize vectors with three dimensions or less, but generally, vectors can have any number of dimensions. In fact, as you’ll see later, vectors that encode words and text tend to have hundreds or thousands of dimensions.
Magnitude: The magnitude of a vector is a non-negative number that represents the vector’s size or length. You can also refer to the magnitude of a vector as the norm, and you can denote it with ||v|| or |v|. There are many different definitions of magnitude or norm, but the most common is the Euclidean norm or 2-norm. You’ll learn how to compute this later.
Unit vector: A unit vector is a vector with a magnitude of one. In the example above, vector1 and vector2 are unit vectors.
Direction: The direction of a vector specifies the line along which the vector points. You can represent direction using angles, unit vectors, or coordinates in different coordinate systems.
Dot product (scalar product): The dot product of two vectors, u and v, is a number given by u ⋅ v = ||u|| ||v|| cos(θ), where θ is the angle between the two vectors. Another way to compute the dot product is to do an element-wise multiplication of u and v and sum the results. The dot product is one of the most important and widely used vector operations because it measures the similarity between two vectors. You’ll see more of this later on.
Orthogonal vectors: Vectors are orthogonal if their dot product is zero, meaning that they’re at a 90 degree angle to each other. You can think of orthogonal vectors as being completely unrelated to each other.
Dense vector: A vector is considered dense if most of its elements are non-zero. Later on, you’ll see that words and text are most usefully represented with dense vectors because each dimension encodes meaningful information.

While there are many more definitions and properties to learn, these six are most important for this tutorial. To solidify these ideas with code, check out the following block. Note that for the rest of this tutorial, you’ll use v1, v2, and v3 to name your vectors:

Python
      
        
      
    
>>> import numpy as np

>>> v1 = np.array([1, 0])
>>> v2 = np.array([0, 1])
>>> v3 = np.array([np.sqrt(2), np.sqrt(2)])

>>> # Dimension
>>> v1.shape
(2,)

>>> # Magnitude
>>> np.sqrt(np.sum(v1**2))
1.0
>>> np.linalg.norm(v1)
1.0

>>> np.linalg.norm(v3)
2.0

>>> # Dot product
>>> np.sum(v1 * v2)
0

>>> v1 @ v3
1.4142135623730951

You first import numpy and create the arrays v1, v2, and v3. Calling v1.shape shows you the dimension of v1. You then see two different ways to compute the magnitude of a NumPy array. The first, np.sqrt(np.sum(v1**2)), uses the Euclidean norm that you learned about above. The second computation uses np.linalg.norm(), a NumPy function that computes the Euclidean norm of an array by default but can also compute other matrix and vector norms.

Lastly, you see two ways to calculate the dot product between two vectors. Using np.sum(v1 * v2) first computes the element-wise multiplication between v1 and v2 in a vectorized fashion, and you sum the results to produce a single number. A better way to compute the dot product is to use the at-operator (@), as you see with v1 @ v3. This is because @ can perform both vector and matrix multiplications, and the syntax is cleaner.

While all of these vector definitions and properties may seem straightforward to compute, you might still be wondering what they actually mean and why they’re important to understand. One way to better understand vectors is to visualize them in two dimensions. In this context, you can represent vectors as arrows, like in the following plot:

The above plot shows the visual representation of the vectors v1, v2, and v3 that you worked with in the last example. The tail of each vector arrow always starts at the origin, and the tip is located at the coordinates specified by the vector. As an example, the tip of v1 lies at (1, 0), and the tip of v3 lies at roughly (1.414, 1.414). The length of each vector arrow corresponds to the magnitude that you calculated earlier.

From this visual, you can make the following key inferences:

v1 and v2 are unit vectors because their magnitude, given by the arrow length, is one. v3 isn’t a unit vector, and its magnitude is two, twice the size of v1 and v2.
v1 and v2 are orthogonal because their tails meet at a 90 degree angle. You see this visually but can also verify it computationally by computing the dot product between v1 and v2. By using the dot product definition, v1 ⋅ v2 = ||v1|| ||v2|| cos(θ), you can see that when θ = 90, cos(θ) = 0 and v1 ⋅ v2 = 0. Intuitively, you can think of v1 and v2 as being totally unrelated or having nothing to do with each other. This will become important later.
v3 makes a 45 degree angle with both v1 and v2. This means that v3 will have a non-zero dot product with v1 and v2. This also means that v3 is equally related to both v1 and v2. In general, the smaller the angle between two vectors, the more they point toward a common direction.

Read the full article at https://realpython.com/chromadb-vector-database/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Basics: Modules and Packages

2023-11-14T14:00:00+00:00

As you gain experience writing code, you’ll eventually work on projects that are so large that keeping all the code in a single file becomes cumbersome.

Instead of writing a single file, you can put related code into separate files called modules. You can put individual modules together like building blocks to create a larger application.

In this video course, you’ll learn how to:

Create your own modules
Use modules in another file through the import statement
Organize several modules into a package

This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.

Note that you’ll be using IDLE to interact with Python throughout this course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

JupyterLab for an Enhanced Notebook Experience

2023-11-13T14:00:00+00:00

Maybe you’ve already worked with Jupyter Notebooks from Project Jupyter to create documents containing runnable code. You can achieve even more with JupyterLab, a tool kit that you can use to document and share your research, teaching, and learning activities. It’s useful in a wide range of disciplines, from data analysis and data visualization to scientific study.

JupyterLab enhances your notebooks by providing a browser-based interface that allows you to use multiple notebooks together effectively. In addition, it offers you a comprehensive Markdown editor, file manager, file viewer, and an infrastructure that enables you to run code from a wide range of files.

In this tutorial, you’ll learn how to:

Share code between multiple Jupyter Notebooks
Debug a Jupyter Notebook
Create and manage Markdown files
Run embedded code from a range of different files
Manage and view different file types from a single interface
Access your operating system from within JupyterLab

Jupyter is a portmanteau word blended from the three programming languages Julia, Python, and R. Although you’ll focus on Python in this tutorial, you can use Jupyter with the other languages as well. Plus, this free application works on macOS, Linux, and Windows environments.

JupyterLab takes Jupyter Notebook usage to a different level, so you’ll get the most out of this tutorial if you’re already familiar with Jupyter Notebook.

Free Bonus: Click here to download notebooks and files that you can play with in JupyterLab.

Installing and Starting JupyterLab

The cleanest way of installing JupyterLab on a computer is to use a virtual environment. This will ensure that your JupyterLab work doesn’t interfere with any other Python projects or environments that you may already have. For this tutorial, you’ll create a new virtual environment named jl_venv. Select your operating system to get JupyterLab up and running:

If you haven’t already done so, download and install Python on your Windows computer. Then fire up a Windows PowerShell(Admin) or Terminal(Admin) prompt depending on the version of Windows that you’re using. Now type in the following commands:

Windows PowerShell
      
        
      
    
PS> mkdir jupyterlab_projects
PS> cd jupyterlab_projects
PS> python -m venv jl_venv
PS> jl_venv\Scripts\activate
(jl_venv) PS> python -m pip install jupyterlab

To keep things neat, you first create a new jupyterlab_projects folder for all of your JupyterLab work. Later you’ll create individual subfolders for each of your projects to keep everything organized.

Next, you create a virtual environment named jl_venv within this folder, which you then activate. If the activation is successful, then the virtual environment’s name will precede your Powershell prompt. If not, see the alert box below. Finally, you install JupyterLab into this virtual environment.

Note: When you run the activation script, you may receive an error stating that your system can’t run the script. Modern versions of Windows don’t allow you to run scripts downloaded from the Internet. This is a security feature.

To fix this, you need to type the command Set-ExecutionPolicy RemoteSigned, then answer Y to the question. Your computer will now run scripts that Microsoft has verified. Once you’ve done this, retry the jl_venv\Scripts\activate command, which should now work.

As you can see above, you have a PowerShell prompt that’s preceded by (jl_venv). This means anything you do from this point forward will stay in this environment and remain separate from any other Python work that you may have elsewhere.

Finally, for neatness, you’ll create a new folder named tutorial_project. This will serve as a working area for this tutorial. Then you start up JupyterLab from within it:

Windows PowerShell
      
(jl_venv) PS> mkdir tutorial_project
(jl_venv) PS> cd tutorial_project
(jl_venv) PS> jupyter lab

To install JupyterLab, fire up a terminal and run the following commands:

Shell
      
        
      
    
$ mkdir jupyterlab_projects
$ cd jupyterlab_projects
$ python3 -m venv jl_venv
$ source jl_venv/bin/activate
(jl_venv) $ python -m pip install jupyterlab

To keep things neat, you first create a new jupyterlab_projects directory for all of your JupyterLab work. Later you can create individual subdirectories for each of your projects to keep things organized.

Next, you create a virtual environment named jl_venv. If this command fails, see the alert box below. You then activate your virtual environment by running its activation script. Once the virtual environment is activated, its name precedes your terminal prompt. You install JupyterLab into this virtual environment.

Note: To create a virtual environment on Linux, you need to have the python3-venv package installed on your computer. If you don’t have this, then run the command sudo apt-get install python3-venv before running the python3 -m venv jl_venv command once more.

Your terminal prompt now shows that (jl_venv) is the active environment. This means anything you do from this point forward will happen in this environment and remain separate from any other Python work that you may have elsewhere.

Finally, for neatness, you’ll create a new directory named tutorial_project and then start up JupyterLab from within it:

Shell
      
(jl_venv) $ mkdir tutorial_project
(jl_venv) $ cd tutorial_project
(jl_venv) $ jupyter lab

To install JupyterLab, fire up a terminal and run the following commands:

Shell
      
        
      
    
$ mkdir jupyterlab_projects
$ cd jupyterlab_projects
$ python -m venv jl_venv
$ source jl_venv/bin/activate
(jl_venv) $ python -m pip install jupyterlab

To keep things neat, you first create a new jupyterlab_projects folder for all of your JupyterLab work. Later you can create individual subfolders for each of your projects to keep things organized.

Next, you create a virtual environment named jl_venv. You then activate your virtual environment by running its activation script. Once the virtual environment is activated, its name precedes your terminal prompt. You install JupyterLab into this virtual environment.

Finally, for neatness, you’ll create a new folder named tutorial_project and then start up JupyterLab from within it:

Shell
      
(jl_venv) $ mkdir tutorial_project
(jl_venv) $ cd tutorial_project
(jl_venv) $ jupyter lab

Of course, once you’ve finished this tutorial, you can delete tutorial_project and add in your own project-specific folders instead.

Note: If you wish, you could create a Samples subfolder within tutorial_project and save this tutorial’s downloadable files into it. These include completed versions of the notebooks that you’ll create later on, as well as some other files. This will also give you some files to play around with and will allow you to fully participate in the tutorial.

JupyterLab will start in your web browser, all ready for you to use. But before you dive in, you might want to know how to end your session:

To shut JupyterLab down, make sure everything is saved, and then use File → Shut Down to close the application before closing your browser. This will close everything down cleanly. Closing the browser alone doesn’t close the server, while crashing the server may cause data loss.
To restart, open either Powershell or your terminal, navigate to your jupyterlab_projects folder, then activate jl_venv. Finally, create or enter your specific project’s folder then start JupyterLab as before.
To deactivate your virtual environment, use the deactivate command. Your command prompt will return to normal.

Once you’ve installed and started JupyterLab, its server will start, along with a web browser connection to it. It may take a moment, but soon you’ll be looking at its main interface:

Because this is your first time running JupyterLab, the front screen shown above contains only a single Launcher window. This is where you can access everything else that’s on offer.

Note: Before you start using JupyterLab, you may like to change its appearance to make it easier for you to use. There are several options available to you:

You can hide or display various screen regions using View → Appearance. This is useful if you have a small monitor.
You can change the overall theme of the interface by opening Settings → Theme. Themes may help you see more clearly.
You can also increase and decrease various font sizes using the options under Settings and Settings → Theme. These may help clarify text.

You can even use your favorite coding font by accessing Settings → Settings Editor and then scrolling down the list of settings on the left until you reach Notebook. Once you’re there, fill out the font family and font size according to your preferences. Then close the Settings tab:

As you can see from the screenshot, you’ve updated the font within your notebooks. If you don’t like your adjustments, then click the big red Restore to Defaults button that appears at the top-right of the Settings screen, and no harm done.

In the upcoming sections, you’ll perform a range of tasks highlighting how JupyterLab’s tools enhance the capability of notebooks. You’ll also see some other interesting features as well.

Understanding JupyterLab Kernels

JupyterLab’s tools support you in your work. Although the tools are self-contained, by using some of them together, you get more out of them. This integration is probably JupyterLab’s most powerful feature.

A good starting point when learning JupyterLab is for you to know what its basic components are and how to make them work together. The diagram below shows an overview of these:

This diagram may look overwhelming at first because there are several parts. Don’t worry, you’ll soon see their relevance. The arrows show how various components interact. These interactions are one of the great benefits of JupyterLab. You’ll start with the central part of the application and the diagram: the kernel.

Read the full article at https://realpython.com/using-jupyterlab/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #180: Studying Python Software Architecture & Creating Lambda Expressions

2023-11-10T12:00:00+00:00

Have you moved through the fundamentals of Python, and are you now considering building a more extensive project or complete application? Where can you study the architecture of existing Python projects and learn best practices? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Document Your Python Code and Projects With ChatGPT

2023-11-08T14:00:00+00:00

Having good documentation is a critical feature of any successful Python project. In practice, writing documentation is hard and can take a lot of time and effort, so some developers don’t like to do it. Luckily, with large language models (LLMs) and tools like ChatGPT, you can quickly document your Python code and projects.

In Python, you can document your code using docstrings and then take advantage of them to enrich the project’s external documentation. ChatGPT can be of great help in writing both docstrings and external documentation.

In this tutorial, you’ll:

Build different ChatGPT prompts to generate Python docstrings
Use different styles while generating docstrings with ChatGPT
Add doctest tests and usage examples to Python docstrings
Create external documentation, such as README files and tutorials, with ChatGPT

To get the most out of this tutorial, you should have a ChatGPT account and know the basics of interacting with this tool using prompt engineering. You should also know the basics of how to document Python code.

Get Your Code: Click here to download free sample code for documenting Python code with ChatGPT.

Benefits of Using ChatGPT for Documenting Python Code

Having high-quality, up-to-date documentation is critical for any software project. Poor documentation can cause a project to fail or go unnoticed even if the codebase is well written and the project’s main idea is innovative and useful.

Writing good documentation takes considerable time and effort. That’s why using large language models (LLMs) like ChatGPT can be a viable alternative for providing your projects and code with proper documentation.

Note: Check out Episode 174: Considering ChatGPT’s Technical Review of a Programming Book of The Real Python Podcast for an interesting conversation about using ChatGPT to run a technical review on existing resources, such as books.

Some of the benefits of ChatGPT for documenting Python code include the following:

Increased productivity: It allows automation of tasks related to code documentation and its maintenance, which saves you considerable time and effort.
Improved quality: It helps ensure that your documentation is accurate, up-to-date, and comprehensive.
Enhanced user experience: It can produce engaging and user-friendly documentation, leading to a better user experience.
Reduced costs: It helps reduce the costs of creating and maintaining documentation.
Improved compliance: It can help ensure that the documentation complies with standards and regulations, making it more consistent and professional.

With ChatGPT, you can generate cool documentation for your Python code in almost no time. In the following sections, you’ll learn the basics of using ChatGPT as an assistant for creating coherent docstrings and user-friendly external documentation for your Python projects.

Effective ChatGPT Prompts for Writing Docstrings

The primary way to document Python code is through docstrings. In Python, a docstring is typically a triple-quoted string that occupies the first line of modules, functions, classes, and methods. This string has a special meaning for Python, which stores it in an attribute called .__doc__.

Many Python tools, including code editors and IDEs, take advantage of docstrings to provide real-time help when you’re writing your code. Docstrings are also part of Python’s built-in help system, which you can access with the help() function:

Python
      
        
      
    
>>> help(str)

Help on class str in module builtins:

class str(object)
 |  str(object='') -> str
 |  str(bytes_or_buffer[, encoding[, errors]]) -> str
 |
 |  Create a new string object from the given object. If encoding or
 |  errors is specified, then the object must expose a data buffer
 |  that will be decoded using the given encoding and error handler.
 |  Otherwise, returns the result of object.__str__() (if defined)
 |  or repr(object).
 |  encoding defaults to sys.getdefaultencoding().
 |  errors defaults to 'strict'.
 |
 |  Methods defined here:
 |
 |  __add__(self, value, /)
 |      Return self+value.
 |
 |  __contains__(self, key, /)
 |      Return key in self.
...

In this example, you call help() with the str class as an argument, and you get the class’s documentation page, which includes the class’s docstring:

Python
      
        
      
    
>>> print(str.__doc__)
str(object='') -> str
str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or
errors is specified, then the object must expose a data buffer
that will be decoded using the given encoding and error handler.
Otherwise, returns the result of object.__str__() (if defined)
or repr(object).
encoding defaults to sys.getdefaultencoding().
errors defaults to 'strict'.

In this case, you get the class’s docstring by accessing the .__doc__ attribute directly on the str class. As you can conclude, docstrings add a lot of value to your code. They’re the primary documentation that you and other Python developers will use to learn about any Python object.

You can also take advantage of your code’s docstrings when you’re building project documentation with a tool like Sphinx or MkDocs. These tools have plugins and features that allow you to extract the docstrings and make them part of your project’s external documentation, which can save you a lot of time.

Python has well-established conventions for writing good docstrings. Package, module, class, method, and function docstrings all have specific goals and should follow specific guidelines. You can find these guidelines and conventions in PEP 257.

Note: To dive deeper into documenting Python projects, check out Documenting Python Code: A Complete Guide.

Although PEP 257 provides a standard, you’ll actually find a healthy variety of docstring styles across the Python ecosystem. Here are a few common alternatives:

Read the full article at https://realpython.com/document-python-code-with-chatgpt/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Basics Exercises: Numbers and Math

2023-11-07T14:00:00+00:00

In Python Basics: Numbers and Math, you learned the mathematical skills that you’ll need as a Python programmer. Now, you’ll take those skills to the next level by actively practicing and applying what you’ve learned.

In this hands-on course, you’ll have the opportunity to reinforce your understanding of numbers and math in Python programming through a series of exercises and coding challenges. By actively engaging with the material, you’ll gain the confidence you need to apply your learning in real-world scenarios.

In this course, you’ll practice:

Creating integers and floating-point numbers
Using arithmetic expressions, math functions, and number methods
Formatting and displaying numbers in strings
Taking numbers as user input

In working through these exercises, you’ll deepen your knowledge and solidify your understanding of the Python language. You’ll not only gain confidence in your programming abilities but also enhance your problem-solving skills.

This course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out other Python Basics courses.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python News: What's New From October 2023

2023-11-06T14:00:00+00:00

October 2023 saw the release of the new Python 3.12. At the same time, focus has shifted to new developments in the language. The acceptance of the long-discussed PEP 703 means that developers can ramp up their work on a free-threading version of Python.

The steering council does an important job governing Python and its development. Nominations for the next steering council are now open. Currently, groups are being established that’ll support the steering council by specifically focusing on typing and documentation.

Dive in to learn more about the most important Python news from the last month.

Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.

Python 3.12 Released

The latest version of Python, Python 3.12, came out on October 2. You can read more about the release in last month’s newsletter.

If you haven’t tried Python 3.12 for yourself yet, then you should give it a quick spin! Especially if you’re working on a library or an application, it’s good to check that it works on the latest version of Python. You can install several versions of Python on your computer to try them side by side.

The biggest challenge with upgrading a project to a brand-new version of Python is that some of your dependencies may not be ready yet. In general, pure Python libraries should continue to work well, while you may experience issues with extension modules that need to be compiled especially for Python 3.12. However, most of the popular libraries are on the ball and are providing compatible wheels already.

Python Enjoys New Developments

Even though Python 3.12 has just been released, the core developers have been working on Python 3.13 for several months already. You can expect the next version of Python to be released in October 2024.

The first alpha version of Python 3.13 is now available. As it’s still early in development, you won’t find many new features yet.

Instead, the main differences between Python 3.12 and 3.13 so far are deprecations and removals of old functionality. In particular, the dead batteries identified in Python 3.11 have now been removed, and many private functions in Python’s C API have been removed.

There are several places where you can follow the discussions and work that goes into developing and maintaining Python, including GitHub, discussion forums, and PEP documents. Last month, core developers Pablo Galindo Salgado and Łukasz Langa unveiled a new platform for Python news: a podcast named core.py.

In the first episode, Pablo and Łukasz discuss some of the features that are in development for Python 3.13. These include an improved editing experience in the REPL and the Faster CPython project’s just-in-time (JIT) compiler prototype.

In the second episode, they cover PEP 703 and the road toward a version of Python without the GIL. The GIL—or global interpreter lock—is a mutex that ensures that only one thread accesses the Python interpreter at a time. A GIL has several advantages in single-threaded programs. However, it also makes parallel processing harder.

If you’re interested in everything that happens under the hood in Python, then you won’t find better guides than Pablo and Łukasz. They’ve both been instrumental in several of the recent new features of the language. Additionally, Pablo was the release manager for Python 3.10 and 3.11, while Łukasz had the same role for Python 3.8 and 3.9.

Steering Council Accepting Nominations

Read the full article at https://realpython.com/python-news-october-2023/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #179: Improving Your Git Developer Experience in Python

2023-11-03T12:00:00+00:00

Are you getting by with a few fundamental commands for Git when building your Python projects? Would you like to improve your version control techniques and become more efficient with the Git command line? This week on the show, Adam Johnson is back to talk about his new book, "Boost Your Git DX."

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Build a Hangman Game for the Command Line in Python

2023-11-01T14:00:00+00:00

Many people learn basic language and word skills by playing the game of hangman. First mentioned in print in the late nineteenth century and popularized in international media as the game show Wheel of Fortune, hangman appeals to many people. For beginning programmers who want a challenge or for more experienced coders looking for a bit of fun, writing hangman in Python is a rewarding endeavor.

Throughout this tutorial, you’ll build the hangman game in Python in a series of steps. The game will work as a command-line application. Throughout the process, you’ll also learn the basics of building computer games.

In this tutorial, you’ll:

Learn the common elements of a computer game
Track the state of a computer game
Get and validate the user’s input
Create a text-based user interface (TUI) for your game
Figure out when the game is over and who the winner is

To get the most from this tutorial, you should be comfortable working with Python sets and lists. You don’t need to have any previous knowledge about game writing.

All the code that you’ll write in this tutorial is available for download at the link below:

Get Your Game: Click here to download the free code that you’ll write to create a hangman game with Python.

Demo: Command-Line Hangman Game in Python

In this tutorial, you’ll write a hangman game for your command line using Python. Once you’ve run all the steps to build the game, then you’ll end up with a text-based user interface (TUI) that will work something like the following:

Throughout this tutorial, you’ll run through several challenges designed to help you solve specific game-related problems. At the end, you’ll have a hangman game that works like the demo above.

Project Overview

The first step to building a good computer game is to come up with a good design. So, the design for your hangman game should begin with a good description of the game itself. You also need to have a general understanding of the common elements that make up a computer game. These two points are crucial for you to come up with a good design.

Description of the Hangman Game

While many people know the game well, it’s helpful to have a formal description of the game. You can use this description to resolve programming issues later during development. As with many things in life, the exact description of a hangman game could vary from person to person. Here’s a possible description of the game:

Game setup: The game of hangman is for two or more players, comprising a selecting player and one or more guessing players.
Word selection: The selecting player selects a word that the guessing players will try to guess.
- The selected word is traditionally represented as a series of underscores for each letter in the word.
- The selecting player also draws a scaffold to hold the hangman illustration.
Guessing: The guessing players attempt to guess the word by selecting letters one at a time.
Feedback: The selecting player indicates whether each guessed letter appears in the word.
- If the letter appears, then the selecting player replaces each underscore with the letter as it appears in the word.
- If the letter doesn’t appear, then the selecting player writes the letter in a list of guessed letters. Then, they draw the next piece of the hanged man. To draw the hanged man, they begin with the head, then the torso, the arms, and the legs for a total of six parts.
Winning conditions: The selecting player wins if the hanged man drawing is complete after six incorrect guesses, in which case the game is over. The guessing players win if they guess the word.
- If the guess is right, the game is over, and the guessing players win.
- If the guess is wrong, the game continues.

A game in progress is shown below. In this game, the word to be guessed is hangman:

In this tutorial, to write the hangman game in Python, you’ll make a few additional design decisions:

The game will be between the computer and one human player.
The computer will act as the selecting player and will select the word to guess, process human input, and handle all output.
The human player is the guessing player, hereafter simply referred to as the player. When the player knows the word, they continue to guess correct letters until the word is complete.

With this basic understanding of the game and some design decisions for the computer version, you can begin creating the game. First, however, you need to know the common elements of computer games and understand how they interact to produce the desired result.

Common Elements of a Computer Game

Every game, whether it’s a computer game or not, has a set of elements that make it a game. Without these elements, there’s no game:

Initial setup: You get the game ready to play. This might mean setting the pieces on a chess board, dealing out cards, or rolling dice to see who goes first.
Gameplay: When people think of the game, this is what they normally think of. The flow of gameplay is controlled by a game loop, which keeps the game moving and doesn’t end until the game is over. The game loop ensures the following happens:
- The user input is gathered and processed.
- The game state is updated based on reactions to the user input. This may include checking for end-of-game conditions.
- Any required output changes are made to reflect the new game state.
- The game loop repeats.
Game ending: How the game ends depends on the game itself. Whether capturing the enemy king in checkmate, achieving a certain score in a card game, or having your piece cross the line in a board game, certain conditions determine the end of the game and the winner.

All games have a game loop. For example, in a chess game, you first set the pieces on the board and declare the current player. Then the game loop begins:

Read the full article at https://realpython.com/python-hangman/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Using the bpython Enhanced REPL

2023-10-31T14:00:00+00:00

The standard Python interpreter lets you run scripts from files or interactively execute code on the fly in a so-called read-evaluate-print loop (REPL). While this is a powerful tool for exploring the language and discovering its libraries through instant feedback on your code inputs, the default REPL shipped with Python has several limitations. Luckily, alternatives like bpython offer a much more programmer-friendly and convenient experience.

You can use bpython to experiment with your code or quickly test an idea without switching contexts between different programs, just like in an integrated development environment (IDE). In addition, bpython may be a valuable teaching tool in either a virtual or physical classroom.

In this course, you’ll learn how to:

Install and use bpython as your alternative Python REPL
Boost your productivity thanks to bpython’s unique features
Tweak bpython’s configuration and its color theme
Use common keyboard shortcuts to code more quickly
Contribute to bpython’s open-source project on GitHub

Before starting this course, make sure you’re already familiar with Python basics and know how to start the standard Python REPL in the command line. In addition, you should be able to install packages with pip, ideally into a virtual environment.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #178: Guiding Scientific Python Library Development

2023-10-27T12:00:00+00:00

How do you prepare a scientific Python project for sharing with others? Could you use some best practices and guidance for packaging, documentation, and testing? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Basics Exercises: Building Systems With Classes

2023-10-24T14:00:00+00:00

In Python Basics: Building Systems With Classes, you moved beyond the basics of object-oriented programming (OOP), and started to put those classes to work. In this exercise course you will employ these capabilities to build more complex systems and write readable, reusable code.

In this video course, you’ll practice the following:

Compose classes together to create layers of functionality
Inherit and override behavior from other classes to create variations
Creatively mix and match these approaches
Instantiate classes with attributes and methods
Override methods from a parent class

This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #177: Welcoming PyPI's Safety & Security Engineer Mike Fiedler

2023-10-20T12:00:00+00:00

You may remember a recent Python Package Index (PyPI) announcement about hiring a full-time security engineer. We've also mentioned several current security initiatives from PyPI. This week on the show, we talk with Mike Fiedler about accepting this new role and securing accounts on PyPI.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Basics Exercises: Object-Oriented Programming

2023-10-17T14:00:00+00:00

In Python Basics: Object-Oriented Programming, you learned how OOP, or object-oriented programming, is a method of structuring a program by bundling related properties and behaviors into individual objects.

In this video course, you’ll practice the following:

Create a class, which is like a blueprint for creating an object
Use classes to create new objects
Instantiate classes with attributes and methods
Use dunder methods like init() and str()

This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.

Note that you’ll be using IDLE to interact with Python throughout this course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #176: Building Python Best Practices and Fundamental Skills

2023-10-13T12:00:00+00:00

What fundamental developer skills are new Python users missing? What best practices might developers without a computer science background be lacking? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Using Python's min() and max()

2023-10-10T14:00:00+00:00

Python’s built-in min() and max() functions come in handy when you need to find the smallest and largest values in an iterable or in a series of regular arguments. Even though these might seem like fairly basic computations, they turn out to have many interesting use cases in real-world programing. You’ll try out some of those use cases here.

In this video course, you’ll learn how to:

Use Python’s min() and max() to find smallest and largest values in your data
Call min() and max() with a single iterable or with any number of regular arguments
Use min() and max() with strings and dictionaries
Tweak the behavior of min() and max() with the key and default arguments
Use comprehensions and generator expressions as arguments to min() and max()

Once you have this knowledge under your belt, then you’ll be prepared to write a bunch of practical examples that will showcase the usefulness of min() and max(). Finally, you’ll code your own versions of min() and max() in pure Python, which can help you understand how these functions work internally.

To get the most out of this video course, you should have some previous knowledge of Python programming, including topics like for loops, functions, list comprehensions, and generator expressions.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #175: Exploring the New Features of Python 3.12

2023-10-06T12:00:00+00:00

Python 3.12 is here! Our regular guests, Geir Arne Hjelle and Christopher Trudeau, return to discuss the new version. Geir Arne coordinated a series of preview articles with several members of the Real Python team this year, and his showcase tutorial, "Python 3.12: Cool New Features for You to Try," came out on October 2. Christopher's video course was posted the next day, covering the topics from the article with visual examples of Python 3.12 in action.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

What's New in Python 3.12

2023-10-03T14:00:00+00:00

It’s that time of year again, time for a new release of Python. Although Python 3.12 mostly focuses on internal improvements, there’s a little something for everyone. You can read up on all the new features by checking out the changelog.

In this video course, you’ll learn about the following changes:

Improvements to f-strings
More did-you-mean error messages
New features in the standard library
Additions to static typing
Changes to the CPython interpreter

In this video course, you’ll explore these changes and see how this new version of Python can work for you.

If you want to try any of the examples in this video course, then you’ll need to use Python 3.12. The Python 3 Installation & Setup Guide and How Can You Install a Pre-Release Version of Python? walk you through several options for adding a new version of Python to your system.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #174: Considering ChatGPT's Technical Review of a Programming Book

2023-09-29T12:00:00+00:00

What can you learn from feeding an entire book on Python programming into ChatGPT-4 and asking it to provide a technical review? What are the potential pitfalls of using an LLM as a learning tool? This week on the show, author Al Sweigart talks about his recent experiments using ChatGPT and Python.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Basics Exercises: Conditional Logic and Control Flow

2023-09-26T14:00:00+00:00

In Python Basics: Conditional Logic and Control Flow, you learned that much of the Python code you’ll write is unconditional. That is, the code doesn’t make any choices. Every line of code is executed in the order that it’s written or in the order that functions are called, with possible repetitions inside loops.

In this course, you’ll revisit how to use conditional logic to write programs that perform different actions based on different conditions. Paired with functions and loops, conditional logic allows you to write complex programs that can handle many different situations.

In this video course, you’ll use the following:

Boolean comparators: ==, !=, <, >, <=, >=
Logical operators: and, or, not
Conditional logic: if … elif … else
Exception handling: try … except
Loops: for, while
Control flow statements: break, continue

Ultimately, you’ll bring all of your knowledge together to build a text-based role-playing game. Along the way, you’ll also get some insight into how to tackle coding challenges in general, which can be a great way to level up as a developer.

This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.

Note that you’ll be using IDLE to interact with Python throughout this course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #173: Getting Involved in Open Source & Generating QR Codes With Python

2023-09-22T12:00:00+00:00

Have you thought about contributing to an open-source Python project? What are possible entry points for intermediate developers? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]