Real Python

How to Download Files From URLs With Python

2023-07-31T14:00:00+00:00

When it comes to file retrieval, Python offers a robust set of tools and packages that are useful in a variety of applications, from web scraping to automating scripts and analyzing retrieved data. Downloading files from a URL programmatically is a useful skill to learn for various programming and data projects and workflows.

In this tutorial, you’ll learn how to:

Download files from the Web using the standard library as well as third-party libraries in Python
Stream data to download large files in manageable chunks
Implement parallel downloads using a pool of threads
Perform asynchronous downloads to fetch multiple files in bulk

In this tutorial, you’ll be downloading a range of economic data from the World Bank Open Data platform. To get started on this example project, go ahead and grab the sample code below:

Free Bonus: Click here to download your sample code for downloading files from the Web with Python.

Facilitating File Downloads With Python

While it’s possible to download files from URLs using traditional command-line tools, Python provides several libraries that facilitate file retrieval. Using Python to download files offers several advantages.

One advantage is flexibility, as Python has a rich ecosystem of libraries, including ones that offer efficient ways to handle different file formats, protocols, and authentication methods. You can choose the most suitable Python tools to accomplish the task at hand and fulfill your specific requirements, whether you’re downloading from a plain-text CSV file or a complex binary file.

Another reason is portability. You may encounter situations where you’re working on cross-platform applications. In such cases, using Python is a good choice because it’s a cross-platform programming language. This means that Python code can run consistently across different operating systems, such as Windows, Linux, and macOS.

Using Python also offers the possibility of automating your processes, saving you time and effort. Some examples include automating retries if a download fails, retrieving and saving multiple files from URLs, and processing and storing your data in designated locations.

These are just a few reasons why downloading files using Python is better than using traditional command-line tools. Depending on your project requirements, you can choose the approach and library that best suits your needs. In this tutorial, you’ll learn approaches to some common scenarios requiring file retrievals.

Downloading a File From a URL in Python

In this section, you’ll learn the basics of downloading a ZIP file containing gross domestic product (GDP) data from the World Bank Open Data platform. You’ll use two common tools in Python, urllib and requests, to download GDP by country.

While the urllib package comes with Python in its standard library, it has some limitations. So, you’ll also learn to use a popular third-party library, requests, that offers more features for making HTTP requests. Later in the tutorial, you’ll see additional functionalities and use cases.

Using `urllib` From the Standard Library

Python ships with a package called urllib, which provides a convenient way to interact with web resources. It has a straightforward and user-friendly interface, making it suitable for quick prototyping and smaller projects. With urllib, you can perform different tasks dealing with network communication, such as parsing URLs, sending HTTP requests, downloading files, and handling errors related to network operations.

As a standard library package, urllib has no external dependencies and doesn’t require installing additional packages, making it a convenient choice. For the same reason, it’s readily accessible for development and deployment. It’s also cross-platform compatible, meaning you can write and run code seamlessly using the urllib package across different operating systems without additional dependencies or configuration.

The urllib package is also very versatile. It integrates well with other modules in the Python standard library, such as re for building and manipulating regular expressions, as well as json for working with JSON data. The latter is particularly handy when you need to consume JSON APIs.

In addition, you can extend the urllib package and use it with other third-party libraries, like requests, BeautifulSoup, and Scrapy. This offers the possibility for more advanced operations in web scraping and interacting with web APIs.

To download a file from a URL using the urllib package, you can call urlretrieve() from the urllib.request module. This function fetches a web resource from the specified URL and then saves the response to a local file. To start, import urlretrieve() from urlllib.request:

>>>

>>> from urllib.request import urlretrieve

Next, define the URL that you want to retrieve data from. If you don’t specify a path to a local file where you want to save the data, then the function will create a temporary file for you. Since you know that you’ll be downloading a ZIP file from that URL, go ahead and provide an optional path to the target file:

>>>

>>> url = (
...     "https://api.worldbank.org/v2/en/indicator/"
...     "NY.GDP.MKTP.CD?downloadformat=csv"
... )
>>> filename = "gdp_by_country.zip"

Because your URL is quite long, you rely on Python’s implicit concatenation by splitting the string literal over multiple lines inside parentheses. The Python interpreter will automatically join the separate strings on different lines into a single string. You also define the location where you wish to save the file. When you only provide a filename without a path, Python will save the resulting file in your current working directory.

Then, you can download and save the file by calling urlretrieve() and passing in the URL and optionally your filename:

Read the full article at https://realpython.com/python-download-file-from-url/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #166: Differentiating the Versions of Python & Unlocking IPython's Magic

2023-07-28T12:00:00+00:00

What are all the different versions of Python? You may have heard of Cython, Brython, PyPy, or others and wondered where they fit into the Python landscape. This week on the show, Christopher Trudeau is here, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python 3.12 Preview: More Intuitive and Consistent F-Strings

2023-07-26T14:00:00+00:00

Every new Python version brings many changes, updates, fixes, and new features. Python 3.12 will be the next minor version and is now in the beta phase. In this version, the core team has been working intensely to formalize and improve the syntax and behavior of one of the most popular Python features: f-strings.

F-strings, short for formatted strings, are string literals prefixed by either a lowercase or uppercase letter F. These strings provide a concise and clean syntax that allows the interpolation of variables and expressions.

In this tutorial, you’ll learn about:

Limitations of f-strings in Python versions before 3.12
Advantages of formalizing the f-string syntax
New capabilities and features of f-strings in Python 3.12
Formalization as the key to better error messages for f-strings

To get the most out of this tutorial, you should be familiar with the string data type, as well as with f-strings and string interpolation.

You’ll find many other new features, improvements, and optimizations in Python 3.12. The most relevant ones include the following:

Ever better error messages
Support for the Linux perf profiler
Support for subinterpreters
Improved static typing features

Go ahead and check out what’s new in the changelog for more details on these and other features.

Free Bonus: Click here to download your sample code for a sneak peek at Python 3.12, coming in October 2023.

F-Strings Had Some Limitations Before Python 3.12

You can use Python’s f-strings for string formatting and interpolation. An f-string is a string literal prefixed with the letter F, either in uppercase or lowercase. This kind of literal lets you interpolate variables and expressions, which Python evaluates to produce the final string.

F-strings have gained a lot of popularity in the Python community since their introduction in Python 3.6. People have embraced them with enthusiasm, turning them into a standard in modern Python programming. The reasons? They provide a concise and readable syntax that allows you to format strings and interpolate variables and expressions without needing the .format() method or the old-style string formatting operator (%).

However, to introduce f-strings, the CPython core development team had to decide how to implement them, especially how to parse them. As a result, f-strings came with their own parsing code. In other words, CPython has a dedicated parser for f-strings. Because of this, the f-string grammar isn’t part of the official Python grammar.

From the core developers’ point of view, this implementation decision implies considerable maintenance costs because they have to manually maintain a separate parser. On the other hand, not being part of the official grammar means that other Python implementations, such as PyPy, can’t know if they’ve implemented f-strings correctly.

However, the most important burden is on the user’s side. From the user’s perspective, the current f-string implementation imposes some limitations:

Reusing quotes or string delimiters isn’t possible.
Embedding backslashes isn’t possible, which means you can’t use escape characters.
Adding inline comments is forbidden.
Nesting of f-strings is limited to the available quoting variations in Python.

PEP 536 lists these limitations. However, exploring them with a few small examples will help you understand how they can affect your use of f-strings in your Python code.

Note: The examples that you’ll see in this section use Python 3.11. If you have a lower Python version installed, then you may get different output.

First, say that you need to interpolate a dictionary key in an f-string. If you try the following code, then you’ll get an error:

>>>

>>> employee = {
...     "name": "John Doe",
...     "age": 35,
...     "job": "Python Developer",
... }

>>> f"Employee: {employee["name"]}"
  File "<stdin>", line 1
    f"Employee: {employee["name"]}"
                           ^^^^
SyntaxError: f-string: unmatched '['

In this example, you try to interpolate the employee name in your f-strings. However, you get an error because the double quotes around the "name" key break the string literal. To work around this, you need to use a different type of quotation mark to delimit the key:

>>>

>>> f"Employee: {employee['name']}"
'Employee: John Doe'

Now you use double quotes for the f-string and single quotes for the dictionary key. Your code works now, but having to switch quotes can get annoying at times.

The second limitation of f-strings is that you can’t use backslash characters in embedded expressions. Consider the following example, where you try to concatenate strings using the newline (\n) escape sequence:

>>>

>>> words = ["Hello", "World!", "I", "am", "a", "Pythonista!"]

>>> f"{'\n'.join(words)}"
  File "<stdin>", line 1
    f"{'\n'.join(words)}"
                         ^
SyntaxError: f-string expression part cannot include a backslash

Read the full article at https://realpython.com/python312-f-strings/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Socket Programming in Python Part 1: Handling Connections

2023-07-25T14:00:00+00:00

Sockets and the socket API are used to send messages across a network. They provide a form of inter-process communication (IPC). The network can be a logical, local network to the computer, or one that’s physically connected to an external network, with its own connections to other networks. The obvious example is the Internet, which you connect to with your ISP.

In this two-part series, you’ll create:

A simple socket server and client
An improved version that handles multiple connections simultaneously
A server-client application that functions like a full-fledged socket application, complete with its own custom header and content

By the end of part one, you’ll understand how to use the main functions and methods in Python’s socket module to write your own client-server applications, including ones with multiple connections. In part two, you’ll dive into building a custom class and handling errors.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python & APIs: A Winning Combo for Reading Public Data

2023-07-24T14:00:00+00:00

Knowing how to consume an API is one of those magical skills that, once mastered, will crack open a whole new world of possibilities, and consuming APIs using Python is a great way to learn such a skill.

A lot of apps and systems that you use on a daily basis are connected to an API. From very simple and mundane tasks, like checking the weather in the morning, to more addictive and time-consuming actions, such as scrolling through your Instagram, TikTok, or Twitter feed, APIs play a central role.

In this tutorial, you’ll learn:

What an API is
How you can consume APIs with your Python code
What the most important API-related concepts are
How Python can help you read data that’s available through public APIs

By the end of this tutorial, you’ll be able to use Python to consume most of the APIs that you come across. If you’re a developer, then knowing how to consume APIs with Python will make you much more proficient, especially when it comes to integrating your work with third-party applications.

Note: In this tutorial, you’ll focus on how to consume APIs using Python, not how to build them. For information on building an API with Python, check out Python REST APIs With Flask, Connexion, and SQLAlchemy.

You can download the source code for the examples that you’ll see in this tutorial by clicking the link below:

Get the Source Code: Click here to get the source code you’ll use to learn about consuming APIs with Python in this tutorial.

Getting to Know APIs

API stands for application programming interface. In essence, an API acts as a communication layer, or interface, that allows different systems to talk to each other without having to understand exactly what the others do.

APIs can come in many forms or shapes. They can be operating system APIs, used for actions like turning on your camera and audio when joining a Zoom call. Or they can be web APIs, used for web-focused actions, such as liking images on your Instagram or fetching the latest tweets.

No matter the type, all APIs function mostly the same way. You usually make a request for information or data, and the API returns a response with what you requested. For example, every time you open Twitter or scroll down your Instagram feed, you’re basically making a request to the API behind that app and getting a response in return. This is also known as calling an API.

In this tutorial, you’ll focus more on the high-level APIs that communicate across networks, also called web APIs.

SOAP vs REST vs GraphQL

Even though some of the examples above are geared toward newer platforms or apps, web APIs have been around for quite a long time. In the late 1990s and early 2000s, two different design models became the norm in exposing data publicly:

SOAP (Simple Object Access Protocol) is typically associated with the enterprise world, has a stricter contract-based usage, and is mostly designed around actions.
REST (Representational State Transfer) is typically used for public APIs and is ideal for fetching data from the Web. It’s much lighter and closer to the HTTP specification than SOAP.

Nowadays, there’s a new kid in town: GraphQL. Created by Facebook, GraphQL is a very flexible query language for APIs, where the clients decide exactly what they want to fetch from the server instead of letting the server decide what to send.

If you want to learn more about the differences between these three design models, then here are a few good resources:

Even though GraphQL is on the rise and is being adopted by bigger and bigger companies, including GitHub and Shopify, the truth is that the majority of public APIs are still REST APIs. Therefore, for the purpose of this tutorial, you’ll learn only about REST APIs and how to consume them using Python.

APIs and `requests`: A Match Made in Heaven

When consuming APIs with Python, there’s only one library you need: requests. With it, you should be able to do most, if not all, of the actions required to consume any public API.

You can install requests by running the following command in your console:

$ python -m pip install requests

The examples in this tutorial have been tested with Python 3.11.2 and requests 2.31.0. However, any supported version of Python and requests should yield similar results.

Read the full article at https://realpython.com/python-api/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #165: Leveraging the Features of Your Database With Postgres and Python

2023-07-21T12:00:00+00:00

Are you getting the most out of your Postgres database? What features could you leverage to improve your Python project? This week on the show, Craig Kerstiens from Crunchy Data is here to discuss getting the most out of Postgres.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python's list Data Type: A Deep Dive With Examples

2023-07-19T14:00:00+00:00

The list class is a fundamental built-in data type in Python. It has an impressive and useful set of features, allowing you to efficiently organize and manipulate heterogeneous data. Knowing how to use lists is a must-have skill for you as a Python developer. Lists have many use cases, so you’ll frequently reach for them in real-world coding.

By working through this tutorial, you’ll dive deep into lists and get a solid understanding of their key features. This knowledge will allow you to write more effective code by taking advantage of lists.

In this tutorial, you’ll learn how to:

Create new lists in Python
Access the items in an existing list
Copy, update, grow, shrink, and concatenate existing lists
Sort, reverse, and traverse existing lists
Use other features of Python lists

In addition, you’ll code some examples that showcase common use cases of lists in Python. They’ll help you understand how to better use lists in your code.

To get the most out of this tutorial, you should have a good understanding of core Python concepts, including variables, functions, and for loops. You’ll also benefit from familiarity with other built-in data types, such as strings, tuples, dictionaries, and sets.

Free Bonus: Click here to download the sample code that will make you an expert on Python’s list data type.

Getting Started With Python’s `list` Data Type

Python’s list is a flexible, versatile, powerful, and popular built-in data type. It allows you to create variable-length and mutable sequences of objects. In a list, you can store objects of any type. You can also mix objects of different types within the same list, although list elements often share the same type.

Note: Throughout this tutorial, you’ll use the terms items, elements, and values interchangeably to refer to the objects stored in a list.

Some of the more relevant characteristics of list objects include being:

Ordered: They contain elements or items that are sequentially arranged according to their specific insertion order.
Zero-based: They allow you to access their elements by indices that start from zero.
Mutable: They support in-place mutations or changes to their contained elements.
Heterogeneous: They can store objects of different types.
Growable and dynamic: They can grow or shrink dynamically, which means that they support the addition, insertion, and removal of elements.
Nestable: They can contain other lists, so you can have lists of lists.
Iterable: They support iteration, so you can traverse them using a loop or comprehension while you perform operations on each of their elements.
Sliceable: They support slicing operations, meaning that you can extract a series of elements from them.
Combinable: They support concatenation operations, so you can combine two or more lists using the concatenation operators.
Copyable: They allow you to make copies of their content using various techniques.

Lists are sequences of objects. They’re commonly called containers or collections because a single list can contain or collect an arbitrary number of other objects.

Note: In Python, lists support a rich set of operations that are common to all sequence types, including tuples, strings, and ranges. These operations are known as common sequence operations. Throughout this tutorial, you’ll learn about several operations that fall into this category.

In Python, lists are ordered, which means that they keep their elements in the order of insertion:

>>>

>>> colors = [
...     "red",
...     "orange",
...     "yellow",
...     "green",
...     "blue",
...     "indigo",
...     "violet"
... ]

>>> colors
['red', 'orange', 'yellow', 'green', 'blue', 'indigo', 'violet']

The items in this list are strings representing colors. If you access the list object, then you’ll see that the colors keep the same order in which you inserted them into the list. This order remains unchanged during the list’s lifetime unless you perform some mutations on it.

You can access an individual object in a list by its position or index in the sequence. Indices start from zero:

>>>

>>> colors[0]
'red'
>>> colors[1]
'orange'
>>> colors[2]
'yellow'
>>> colors[3]
'green'

Positions are numbered from zero to the length of the list minus one. The element at index 0 is the first element in the list, the element at index 1 is the second, and so on.

Lists can contain objects of different types. That’s why lists are heterogeneous collections:

[42, "apple", True, {"name": "John Doe"}, (1, 2, 3), [3.14, 2.78]]

This list contains objects of different data types, including an integer number, string, Boolean value, dictionary, tuple, and another list. Even though this feature of lists may seem cool, in practice you’ll find that lists typically store homogeneous data.

Note: One of the most relevant characteristics of lists is that they’re mutable data types. This feature deeply impacts their behavior and use cases. For example, mutability implies that lists aren’t hashable, so you can’t use them as dictionary keys. You’ll learn a lot about how mutability affects lists throughout this tutorial. So, keep reading!

Okay! That’s enough for a first glance at Python lists. In the rest of this tutorial, you’ll dive deeper into all the above characteristics of lists and more. Are you ready? To kick things off, you’ll start by learning the different ways to create lists.

Read the full article at https://realpython.com/python-list/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Creating Web Maps From Your Data With Python Folium

2023-07-18T14:00:00+00:00

If you’re working with geospatial data in Python, then you might want to quickly visualize that data on a map. Python’s Folium library gives you access to the mapping strengths of the Leaflet JavaScript library through a Python API. It allows you to create interactive geographic visualizations that you can share as a website.

You’ll build a web map that displays the ecological footprint per capita of many countries and is based on a similar map on Wikipedia. Along the way, you’ll learn the basics of using Folium for data visualization.

In this video course, you’ll:

Create an interactive map using Folium and save it as an HTML file
Choose from different web map tiles
Anchor your map to a specific geolocation
Bind data to a GeoJSON layer to create a choropleth map
Style the choropleth map

You’ll use Folium inside of a Jupyter notebook, so the Folium library will render your maps directly in the Jupyter notebook. This gives you a good opportunity to visually explore a geographical dataset or include a map in your data science report.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python News: What's New From June 2023

2023-07-17T14:00:00+00:00

June 2023 welcomed a lot of new people and positions to the Python Software Foundation (PSF). Elections wrapped up for directors, and a new security developer in residence got to work. The organization is still looking to expand, though, and it announced a new deputy CPython developer in residence position that could go to a less seasoned programmer. Over at Pydantic, version 2 brings improved performance, while the sun went down on Python 3.7.

It’s summer, so get ready for some hot Python news!

Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.

PSF Elects New Directors

The Python Software Foundation (PSF) is the nonprofit organization behind Python. If you’re wondering what exactly the PSF does, the answer is a bunch! Here’s a bird’s-eye view of their role in the language and community:

The mission of the Python Software Foundation is to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers. (Source)

Elections to fill four seats on the PSF board of directors occurred during the second half of June. A term for a seat on the PSF board of directors lasts three years, and directors are eligible for reelection. To vote in this election, you had to be a registered PSF member with voting rights.

This year, there were a lot of nominees. In this crowded race, four people received the most votes and were elected to fill the seats:

If you’re curious about what it means to serve on the PSF board and what a director’s duties are, then you can watch a video in which three board members describe Life as a Python Software Foundation Director.

Congratulations to the new and returning board directors, and big thanks to all the other nominees for their continued engagement in the Python community. Community service is what keeps Python thriving.

Read the full article at https://realpython.com/python-news-june-2023/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #164: Constructing Python Library APIs & Tackling Jinja Templating

2023-07-14T12:00:00+00:00

What principles should you consider when designing a Python library? How do you construct a library API that's understandable and easy to use? This week on the show, Christopher Trudeau is here, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Unlock IPython's Magical Toolbox for Your Coding Journey

2023-07-12T14:00:00+00:00

When you’re executing Python code, the standard Python shell that comes with your Python installation is a natural tool to use. However, as you progress in your coding journey, you might find yourself seeking more powerful functionalities than the standard REPL offers. Luckily, IPython offers an enhanced version of interactive Python that can supercharge your capabilities.

Using the IPython shell is a fast way of learning Python and executing code without the need for a full-fledged integrated development environment (IDE), such as PyCharm. IPython adds great features to the Python experience with its magic commands, which the standard Python shell lacks. These can help you complete tasks more quickly.

In this tutorial, you’ll:

Learn the major aspects and functionality of the IPython shell
Write and execute code in IPython
Integrate IPython in your Python scripts
Use IPython’s magic commands in your coding sessions
Learn how to save your sessions persistently in a Python file

To get started with IPython, you don’t need to be far along in your Python journey. In fact, IPython is an excellent learning tool because it offers an intuitive interface. Are you ready to get started?

Free Bonus: Click here to download code resources and a cheat sheet so you can work magic with IPython.

Install and Initiate IPython on Your Machine

It only takes a few steps to install and initiate IPython. First, you’ll need to install Python on your system. You also might want to create and activate a virtual environment instead of installing IPython into your global Python.

Note: IPython comes installed with the Anaconda package. If you’ve installed the full Anaconda package, then you already have IPython.

To install IPython, use pip on the command line:

(venv) C:\Users\User>python -m pip install ipython

Just like that, you should see a message that the installation was successful. That means the package has installed all the dependencies, and you should be able to begin using the interactive shell. If you encounter any issues, please consult the documentation.

After the installation process is finished, you can initiate the interactive shell by executing the command ipython on the command line:

C:\Users\User>ipython
Python 3.11.4 (main, Jun 27 2023, 11:06:35) [Clang 14.0.0 (clang-1400.0.29.202)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.14.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]:

You’ll see the above message at the start of the session, confirming the successful installation of IPython on your machine. The shell initiates the first prompt, with In [1] indicating that it’s ready to accept input.

To exit the interactive shell, you can use the exit command, and you’ll switch from IPython to the command line. Now that you have IPython up and running, you’re ready to explore how it can transform your Python workflow. That’s what you’ll discover next.

Improve Your Interactive Python Workflow With IPython

The Python standard shell is a fantastic and versatile tool. However, IPython takes things up a notch by incorporating powerful features that can greatly enhance your coding productivity. While you can explore all of its strengths in the documentation, here are a few noteworthy distinctions that add up to a more user-friendly interface:

Numbered prompts can give you a stronger understanding of the input code and output results.
IPython has so many magic commands that the standard Python shell lacks, and they make working in the shell efficient.
You can access your code history within the shell.
It makes object introspection and code documentation accessible.

These advantages can make a huge difference in your life as a programmer. You’ll get to know these productivity boosters in the coming sections.

IPython is also available through Jupyter Notebooks. This browser-based interactive interface offers similar features, such as magic commands and numbered input and output. Its decoupled two-process model separates the front-end and back-end components of IPython into distinct processes. This allows the IPython kernel to function independently and handle code execution, namespace management, and communication between the shell and kernel.

Note: IPython isn’t the only interactive shell that you can use. In the Python-verse, options abound, and you can choose your favorite shell out of bpython, ptpython, IDLE, and others.

The standard Python shell allows you to interact with Python on the command line. It permits you to enter statements or expressions one at a time. Then, it executes them and displays the corresponding output:

In the above code examples, you’re able to define two variables, first_name and last_name, and then concatenate the two strings. You can create mini-programs in the Python shell by creating other objects, such as functions and class objects.

Read the full article at https://realpython.com/ipython-interactive-python-shell/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Mazes in Python Part 2: Storing and Solving

2023-07-11T14:00:00+00:00

If you’re up for a little challenge and would like to take your programming skills to the next level, then you’ve come to the right place! In this hands-on video course, you’ll practice object-oriented programming, among several other good practices, while building a cool maze solver project in Python.

This is the second part in a two-part series. Throughout the series, you’re going step by step through the guided process of building a complete and working project. This includes reading a maze from a binary file, visualizing it using scalable vector graphics (SVG), and finding the shortest path from the entrance to the exit.

In this video course, you’ll learn how to:

Define a specialized binary file format to store the maze on disk
Transform the maze into a traversable weighted graph
Use a graph search algorithm in the NetworkX library to find the solution.

To get the most out of this video course, make sure you’ve completed Part 1: Building and Visualizing.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Profiling in Python: How to Find Performance Bottlenecks

2023-07-10T14:00:00+00:00

Do you want to optimize the performance of your Python program to make it run faster or consume less memory? Before diving into any performance tuning, you should strongly consider using a technique called software profiling. It may help you answer whether optimizing the code is necessary and, if so, which parts of the code you should focus on.

Sometimes, the return on investment in performance optimizations just isn’t worth the effort. If you only run your code once or twice, or if it takes longer to improve the code than execute it, then what’s the point?

When it comes to improving the quality of your code, you’ll probably optimize for performance as a final step, if you do it at all. Often, your code will become speedier and more memory efficient thanks to other changes that you make. When in doubt, go through this short checklist to figure out whether to work on performance:

Testing: Have you tested your code to prove that it works as expected and without errors?
Refactoring: Does your code need some cleanup to become more maintainable and Pythonic?
Profiling: Have you identified the most inefficient parts of your code?

Only when all the above items check out should you consider optimizing for performance. It’s usually more important that your code runs correctly according to the business requirements and that other team members can understand it than that it’s the most efficient solution.

The actual time-saver might be elsewhere. For example, having the ability to quickly extend your code with new features before your competitors will make a real impact. That’s especially true when the performance bottleneck lies not in the underlying code’s execution time but in network communication. Making Python run faster won’t win you anything in that case, but it’ll likely increase the code’s complexity.

Finally, your code will often become faster as a result of fixing the bugs and refactoring. One of the creators of Erlang once said:

Make it work, then make it beautiful, then if you really, really have to, make it fast. 90 percent of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful! (Source)

— Joe Armstrong

As a rule of thumb, anytime you’re considering optimization, you should profile your code first to identify which bottlenecks to address. Otherwise, you may find yourself chasing the wrong rabbit. Because of the Pareto principle or the 80/20 rule, which applies to a surprisingly wide range of areas in life, optimizing just 20 percent of your code will often yield 80 percent of the benefits!

But without having factual data from a profiler tool, you won’t know for sure which parts of the code are worth improving. It’s too easy to make false assumptions.

So, what’s software profiling, and how do you profile programs written in Python?

Free Bonus: Click here download your sample code for profiling your Python program to find performance bottlenecks.

How to Find Performance Bottlenecks in Your Python Code Through Profiling

Software profiling is the process of collecting and analyzing various metrics of a running program to identify performance bottlenecks known as hot spots. These hot spots can happen due to a number of reasons, including excessive memory use, inefficient CPU utilization, or a suboptimal data layout, which will result in frequent cache misses that increase latency.

Note: A performance profiler is a valuable tool for identifying hot spots in existing code, but it won’t tell you how to write efficient code from the start.

It’s often the choice of the underlying algorithm or data structure that can make the biggest difference. Even when you throw the most advanced hardware available on the market at some computational problem, an algorithm with a poor time or space complexity may never finish in a reasonable time.

When profiling, it’s important that you perform dynamic analysis by executing your code and collecting real-world data rather than relying on static code review. Because dynamic analysis often entails running a slow piece of software over and over again, you should start by feeding small amounts of input data to your algorithm if possible. This will limit the amount of time that you spend waiting for results on each iteration.

Once you have your code running, you can use one of the many Python profilers available. There are many kinds of profilers out there, which can make your head spin. Ultimately, you should know how to pick the right tool for the job. Over the next few sections, you’ll get a quick tour of the most popular Python profiling tools and concepts:

Timers like the time and timeit standard library modules, or the codetiming third-party package
Deterministic profilers like profile, cProfile, and line_profiler
Statistical profilers like Pyinstrument and the Linux perf profiler

Fasten your seatbelt because you’re about to get a crash course in Python’s performance profiling!

`time`: Measure the Execution Time

In Python, the most basic form of profiling involves measuring the code execution time by calling one of the timer functions from the time module:

>>>

>>> import time

>>> def sleeper():
...     time.sleep(1.75)
...

>>> def spinlock():
...     for _ in range(100_000_000):
...         pass
...

>>> for function in sleeper, spinlock:
...     t1 = time.perf_counter(), time.process_time()
...     function()
...     t2 = time.perf_counter(), time.process_time()
...     print(f"{function.__name__}()")
...     print(f" Real time: {t2[0] - t1[0]:.2f} seconds")
...     print(f" CPU time: {t2[1] - t1[1]:.2f} seconds")
...     print()
...
sleeper()
 Real time: 1.75 seconds
 CPU time: 0.00 seconds

spinlock()
 Real time: 1.77 seconds
 CPU time: 1.77 seconds

You first define two test functions, sleeper() and spinlock(). The first function asks your operating system’s task scheduler to suspend the current thread of execution for about 1.75 seconds. During this time, the function remains dormant without occupying your computer’s CPU, allowing other threads or programs to run. In contrast, the second function performs a form of busy waiting by wasting CPU cycles without doing any useful work.

Later, you call both of your test functions. Before and after each invocation, you check the current time with time.perf_counter() to obtain the elapsed real time, or wall-clock time, and time.process_time() to get the CPU time. These will tell you how long your functions took to execute and how much of that time they spent on the processor. If a function waits for another thread or an I/O operation to finish, then it won’t use any CPU time.

Note: The performance of a computer program is typically limited by the available processing power, memory amount, input and output operations, and program latency.

If a given task predominantly does a lot of computation, then the processor’s speed will determine how long it’ll take to finish. Such a task is CPU-bound. You can sometimes run such tasks in parallel on multiple CPU cores simultaneously to reduce the overall computation time.

On the other hand, an I/O-bound task spends most of its time waiting for data to arrive from a disk, a database, or a network. Such tasks can benefit from using faster I/O channels or running them concurrently as well as asynchronously.

Read the full article at https://realpython.com/python-profiling/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #163: Python Crash Course & Learning Enough to Start Creating

2023-07-07T12:00:00+00:00

How much Python do you need to learn to start creating projects? What's a good balance of information and hands-on practice? This week on the show, Eric Matthes is here to discuss his book Python Crash Course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python 3.12 Preview: Support For the Linux perf Profiler

2023-07-05T14:00:00+00:00

The final release of Python 3.12 is scheduled for October 2023, which is growing closer. In the meantime, you can download and install its preview version to get a sneak peek at the upcoming features. One of the biggest changes announced is support for the Linux perf profiler, which is a powerful performance profiling tool.

In this tutorial, you’ll:

Install and use the Linux perf profiler with Python 3.12
Get a holistic view of your application’s performance
Explore a case study of profiling Python code with perf
Visualize the collected metrics with flame graphs and more

To fully benefit from using the perf profiler in Python 3.12, you should have a fairly good understanding of how the underlying hardware and the operating system work. In addition to that, you need to be comfortable using a Linux distribution and the build tools for compiling Python from source code.

There are many other new features and improvements coming in Python 3.12. The highlights include the following:

Ever better error messages
More powerful f-strings
Better support for subinterpreters
Improved static typing features

Check out what’s new in the changelog for more details on these features.

Free Bonus: Click here to download your sample code for a sneak peek at Python 3.12, coming in October 2023.

Seeing the Big Picture Through the Lens of `perf`

The Linux perf profiler is a truly versatile performance analysis tool. At the very least, you can use it as a statistical profiler to find hot spots in your own code, library code, and even the operating system’s code. In fact, you can hook it up to any running process, such as your web browser, and obtain its live profile, as long as you have sufficient permissions and the program was compiled with debug symbols.

The tool can also work as an event counter by measuring the exact number of low-level events occurring in both hardware and software. For example, it’s capable of counting the number of CPU cycles, instructions executed by the processor, or context switches during a program’s execution. The specific types of events may vary depending on your hardware architecture and the Linux kernel version.

Another useful feature of perf is the ability to retain the call graph of functions, which can help you understand which of potentially many calls to the same function is an actual bottleneck. With a bit of effort, you can even visualize your code paths in the form of a mathematical graph consisting of nodes and edges.

In this short section, you’ll get a basic understanding of the Linux perf profiler and its advantages over other profiling tools. If you’re already familiar with this tool, then you can jump ahead to a later section to get your hands dirty and see it in action.

What’s the Linux `perf` Profiler?

If you search online for information about the Linux perf profiler, then you may get confused by the plethora of names that people use to talk about it. To make matters worse, the profiler seems to be poorly documented. You really need to dig deep, as the corresponding article on Wikipedia and the official Wiki page don’t provide a lot of help.

The Linux perf profiler actually consists of two high-level components:

perf_events, or performance counters for Linux (PCL): A subsystem in the Linux kernel with an API that provides an abstraction layer over a hardware-specific performance monitoring unit (PMU) present in modern architectures. The PMU consists of special CPU registers or counters for collecting metrics about events such as the number of cache misses, CPU cycles, or executed instructions. This information is invaluable during performance analysis.
perf command-line tool: A user-space utility tool built on top of the perf_events API from the Linux kernel, which can help you collect and make sense of the data. It follows the same philosophy as Git by offering several specialized subcommands. You can list them all by typing perf at your command prompt.

The first component is baked right into the Linux kernel, meaning that recent versions of mainstream Linux distributions will usually have it shipped and enabled. On the other hand, you’ll most likely need to install an additional package to start using the perf command, as it’s not essential for regular users.

Note: Each processor and operating system combination supports a different set of event types. Therefore, you may sometimes need to reinstall perf to match your current kernel version after an upgrade.

While the Linux perf profiler came into existence primarily as an interface to hardware-level performance counters, it also covers numerous software performance counters. For example, you can use it to track the number of context switches made by your operating system’s task scheduler or the number of times a running thread has migrated from one CPU core to another.

Okay, but are there any other benefits of using the Linux perf profiler aside from its hardware capabilities?

How Can Python Benefit From `perf`?

For several reasons, adding support for the Linux perf profiler will greatly impact your ability to profile Python applications. There’s no better source to refer to than Pablo Galindo Salgado, who’s the primary contributor behind this new feature. In a Twitter announcement, he thoroughly explains the benefits of the Python and perf integration:

Python 3.12 will add support for the Linux perf profiler! 🔥🔥 Perf is one of the most powerful and performant profilers for Linux that allows getting a ridiculous amount of information such as CPU counters, cache misses, context switching, and much more. (Source)

The lengthy tweet thread gives examples of several new capabilities:

Read the full article at https://realpython.com/python312-perf-profiler/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Filtering Iterables With Python

2023-07-04T14:00:00+00:00

Python’s filter() is a built-in function that allows you to process an iterable and extract those items that satisfy a given condition. This process is commonly known as a filtering operation. With filter(), you can apply a filtering function to an iterable and produce a new iterable with the items that satisfy the condition at hand. In Python, filter() is one of the tools that you can use for functional programming.

In this video course, you’ll learn how to:

Use Python’s filter() in your code
Extract needed values from your iterables
Combine filter() with other functional tools
Replace filter() with more Pythonic tools

With this knowledge, you’ll be able to use filter() effectively in your code. Alternatively, you have the choice of using list comprehensions or generator expressions to write more Pythonic and readable code.

To better understand filter(), it would be helpful for you to have some previous knowledge on iterables, for loops, functions, and lambda functions. You can also refresh your memory of lambda functions through a video course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

How to Round Numbers in Python

2023-07-03T14:00:00+00:00

With many businesses turning to Python’s powerful data science ecosystem to analyze their data, understanding how to avoid introducing bias into datasets is absolutely vital. If you’ve studied some statistics, then you’re probably familiar with terms like reporting bias, selection bias, and sampling bias. There’s another type of bias that plays an important role when you’re dealing with numeric data: rounding bias.

Understanding how rounding works in Python can help you avoid biasing your dataset. This is an important skill. After all, drawing conclusions from biased data can lead to costly mistakes.

In this tutorial, you’ll learn:

Why the way you round numbers is important
How to round a number according to various rounding strategies
How to implement each strategy in pure Python
How rounding affects data and which rounding strategy minimizes this effect
How to round numbers in NumPy arrays and pandas DataFrames
When to apply different rounding strategies

Take the Quiz: Test your knowledge with our interactive “Rounding Numbers in Python” quiz. Upon completion you will receive a score so you can track your learning progress over time:

Take the Quiz »

Free Bonus: Click here to download sample code to ensure that your numbers are always well rounded!

You won’t get a treatise on numeric precision in computing, although you’ll touch briefly on the subject. Only a familiarity with the fundamentals of Python is necessary, and the math should feel familiar if you’ve had high school algebra.

You’ll start by looking at Python’s built-in rounding mechanism.

Python’s Built-in `round()` Function

Python has a built-in round() function that takes two numeric arguments, n and ndigits, and returns the number n rounded to ndigits. The ndigits argument defaults to zero, so leaving it out results in a number rounded to an integer. As you’ll see, round() may not work quite as you expect.

The way most people are taught to round a number goes something like this:

Round the number n to p decimal places by first shifting the decimal point in n by p places. To do that, multiply n by 10ᵖ (10 raised to the p power) to get a new number, m.
Then look at the digit d in the first decimal place of m. If d is less than 5, round m down to the nearest integer. Otherwise, round m up.
Finally, shift the decimal point back p places by dividing m by 10ᵖ.

It’s an algorithm! For example, the number 2.5 rounded to the nearest whole number is 3. The number 1.64 rounded to one decimal place is 1.6.

Now open up an interpreter session and round 2.5 to the nearest whole number using Python’s built-in round() function:

>>>

>>> round(2.5)
2

Gasp!

Check out how round() handles the number 1.5:

>>>

>>> round(1.5)
2

So, round() rounds 1.5 up to 2, and 2.5 down to 2!

Before you go raising an issue on the Python bug tracker, rest assured you that round(2.5) is supposed to return 2. There’s a good reason why round() behaves the way it does.

In this tutorial, you’ll learn that there are more ways to round a number than you might expect, each with unique advantages and disadvantages. round() behaves according to a particular rounding strategy—which may or may not be the one you need for a given situation.

You might be wondering, Can the way I round numbers really have that much of an impact? Next up, take a look at just how extreme the effects of rounding can be.

How Much Impact Can Rounding Have?

Suppose you have an incredibly lucky day and find $100 on the ground. Rather than spending all your money at once, you decide to play it smart and invest your money by buying some shares of different stocks.

The value of a stock depends on supply and demand. The more people there are who want to buy a stock, the more value that stock has, and vice versa. In high-volume stock markets, the value of a particular stock can fluctuate on a second-by-second basis.

Read the full article at https://realpython.com/python-rounding/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #162: Exploring the Zen of Python & pandas Features for Finance

2023-06-30T12:00:00+00:00

What advice can you extract from the Zen of Python? How can these nineteen guiding principles help you write more idiomatic Python? This week on the show, Christopher Trudeau is here, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Why Are Membership Tests So Fast for range() in Python?

2023-06-28T14:00:00+00:00

In Python, range() is most commonly used in for loops to iterate over a known range of numbers. But that’s not all it can do! There are other interesting uses of range(). For example, you can pick out elements of a range or check whether a given number belongs to a range of numbers. The latter operation is known as a membership test.

Python’s range() objects support most of the operations that you’ve come to expect from lists. For example, you can calculate the total length, pick out a number at a given index, and iterate over each element in a range. Assume that you have a range of the first ten million integers and a list with the same numbers:

>>>

>>> numbers_as_range = range(10_000_000)
>>> numbers_as_list = list(numbers_as_range)

For many operations, you can treat these objects as interchangeable. Both ranges and lists support indexing items and calculating length:

>>>

>>> numbers_as_range[123]
123
>>> numbers_as_list[123]
123

>>> len(numbers_as_range)
10000000
>>> len(numbers_as_list)
10000000

Additionally, you can iterate over each object or check if a certain element is a member. For example, you can use the in operator to check whether -1 is part of any of these containers:

>>>

>>> -1 in numbers_as_range
False

>>> -1 in numbers_as_list
False

Because the range and the list only include non-negative numbers, -1 is not a member. However, the time Python takes to figure this out is vastly different for the two data structures:

>>>

>>> from timeit import timeit

>>> timeit("-1 in numbers_as_range", globals=globals(), number=100)
4.68301093652801e-06

>>> timeit("-1 in numbers_as_list", globals=globals(), number=100)
5.704572553362310

Here, you use timeit to measure the running time of the code. The reported numbers indicate how many seconds Python spent executing each command a hundred times.

Note the e-06 suffix in the first number. It’s scientific notation that means that the number in front of the suffix represents microseconds. If you compare the numbers, then you’ll see that searching for—and not finding—an element is several orders of magnitude slower in the list than in the range object.

In this tutorial, you’ll investigate how range() works under the hood and learn why Python’s range() membership tests are so fast. If you want to brush up on how to work with range(), then you should have a look at The Python range() Function first. You can click the link below to download some example code that you’ll be exploring in this tutorial:

Free Bonus: Click here to download the sample code that you’ll use to explore how Python runs such quick membership tests on range().

In Short: A Formula Determines Membership in a Python Range

In Python, three numbers define a range: start, stop, and step. The range will begin at start and then contain every step integer that’s smaller than stop. For example, if start = 1, stop = 10, and step = 2, then the range will be the odd numbers less than ten. Namely, one, three, five, seven, and nine.

A list is much more flexible. It can contain any number of elements, and the elements can be of any type. It’s possible to have repeated elements. A list is often not sorted. In fact, it’s possible to have lists that Python can’t order, as the elements can be of mixed types that aren’t comparable.

All this flexibility has a price. To check if a list contains a particular element, Python needs to look through the list—one element at a time—until it finds a match. In principle, the operation looks something like this:

>>>

>>> def list_contains(element, list_elements):
...     for list_element in list_elements:
...         if list_element == element:
...             return True
...     return False
...

This function loops over all the list elements until it finds a match. If that match happens to be one of the first elements in the list, then it returns quickly. But if element isn’t in the list, then the function needs to inspect all the elements in the list to reach this conclusion.

If you’re working with ranges, then Python can take a shortcut and use a formula instead. With the following function, you can implement such a calculation:

>>>

>>> def range_contains(element, start, stop, step=1):
...     return start <= element < stop and (element - start) % step == 0
...

Read the full article at https://realpython.com/python-range-membership-test/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Jinja Templating

2023-06-27T14:00:00+00:00

Templates are an essential ingredient in full-stack web development. With Jinja, you can build rich templates that power the front end of your Python web applications.

Jinja is a text templating language. It allows you to process a block of text, insert values from a context dictionary, control how the text flows using conditionals and loops, modify inserted data with filters, and compose different templates together using inheritance and inclusion.

In this video course, you’ll learn how to:

Install the Jinja template engine
Create your first Jinja template
Render a Jinja template in Flask
Use for loops and conditional statements with Jinja
Nest Jinja templates
Modify variables in Jinja with filters
Use macros to add functionality to your front end

This video course is for you if you want to learn more about the Jinja template language or if you’re getting started with Flask.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #161: Resources and Advice for Building CircuitPython Projects

2023-06-23T12:00:00+00:00

Are you looking to advance your CircuitPython projects? Would you like a collection of resources and tools to help you along your path? This week on the show, Tod Kurt is here to discuss building projects with CircuitPython.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Recursion in Python

2023-06-20T14:00:00+00:00

If you’re familiar with functions in Python, then you know that it’s quite common for one function to call another. In Python, it’s also possible for a function to call itself! A function that calls itself is said to be recursive, and the technique of employing a recursive function is called recursion.

It may seem peculiar for a function to call itself, but many types of programming problems are best expressed recursively. When you bump up against such a problem, recursion is an indispensable tool for you to have in your toolkit.

By the end of this video course, you’ll understand:

What it means for a function to call itself recursively
When recursion might be your best best for solving a problem
How you can implement recursion for various use cases in Python

Along the way, you’ll study several specific programming tasks where you can use recursion in Python. You’ll also explore alternatives to recursion.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #160: Inheriting a Large Python Code Base & Building a GUI With Kivy

2023-06-16T12:00:00+00:00

What are the unique challenges of a large Python code base? What techniques can you implement to simplify the management of a big project? This week on the show, Christopher Trudeau is here, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Python Basics: Reading and Writing Files

2023-06-13T14:00:00+00:00

Files are everywhere in the modern world. They’re the medium in which data is digitally stored and transferred. Chances are, you’ve opened dozens, if not hundreds, of files just today! Now it’s time to read and write files with Python.

In this video course, you’ll learn how to:

Understand the difference between text and binary files
Learn about character encodings and line endings
Work with file objects in Python
Read and write character data in various file modes
Use open(), Path.open(), and the with statement
Take advantage of the csv module to manipulate CSV data

This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.

Note that you’ll be using IDLE to interact with Python throughout this course. If you’re just getting started, then you might want to check out Python Basics: Setting Up Python before diving into this course.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #159: Volunteering, Organizing, and Finding a Python Community

2023-06-09T12:00:00+00:00

Have you thought about getting more involved in the Python community? Are you interested in volunteering for an event or becoming an organizer? This week on the show, we speak with organizers from this year's PyCascades conference about making connections, learning new skills, and rationing your time.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Mazes in Python Part 1: Building and Visualizing

2023-06-06T14:00:00+00:00

This is the first part in a two-part series. Throughout the series, you’ll go step by step through the guided process of building a complete and working project. This will include reading a maze from a binary file, visualizing it using scalable vector graphics (SVG), and finding the shortest path from the entrance to the exit.

In part one of the series, you’ll learn how to:

Use an object-oriented approach to represent the maze in memory
Visualize the maze and its solution using scalable vector graphics (SVG)

If you’ve already completed this video course, then you can jump ahead to Mazes in Python Part 2: Storing and Solving, where you’ll complete the project by creating a binary storage format for mazes and solving mazes using NetworkX.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #158: Building Python CI With Docker & Applying for a Hacker Initiative Grant

2023-06-02T12:00:00+00:00

Do you need a refresher on using Docker with Python? Would you like to learn how to configure a continuous integration pipeline with modern tools and Docker? This week on the show, Christopher Trudeau is here, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Getting the First Match From a Python List or Iterable

2023-05-30T14:00:00+00:00

At some point in your Python journey, you may need to find the first item that matches a certain criterion in a Python iterable, such as a list or dictionary.

The simplest case is that you need to confirm that a particular item exists in the iterable. For example, you want to find a name in a list of names or a substring inside a string. In these cases, you’re best off using the in operator. However, there are many use cases when you may want to look for items with specific properties. For instance, you may need to:

Find a non-zero value in a list of numbers
Find a name of a particular length in a list of strings
Find and modify a dictionary in a list of dictionaries based on a certain attribute

In this video course, you’ll explore how best to approach all three scenarios.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

The Real Python Podcast – Episode #157: Discussing Mojo & Improving Python Object-Oriented Programming

2023-05-26T12:00:00+00:00

Would you like to speed up your Python machine-learning code dramatically? What if you only had to change a few keywords and add a couple of type hints on portions of your code? This week on the show, Christopher Trudeau is here, bringing another batch of PyCoder's Weekly articles and projects.

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Using k-Nearest Neighbors (kNN) in Python

2023-05-23T14:00:00+00:00

In this video course, you’ll get a thorough introduction to the k-Nearest Neighbors (kNN) algorithm in Python. The kNN algorithm is one of the most famous machine learning algorithms and an absolute must-have in your machine learning toolbox. Python is the go-to programming language for machine learning, so what better way to discover kNN than with Python’s famous packages NumPy and scikit-learn!

You’ll explore the kNN algorithm both in theory and in practice. It’s important to learn about the mechanics of machine learning algorithms to understand their potential and limitations. At the same time, it’s essential to understand how to use an algorithm in practice. With that in mind, you’ll also focus on the use of kNN in the Python library scikit-learn.

In this video course, you’ll learn how to:

Explain the kNN algorithm both intuitively and mathematically
Implement kNN in Python from scratch using NumPy
Use kNN in Python with scikit-learn

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

Real Python

How to Download Files From URLs With Python

Facilitating File Downloads With Python

Downloading a File From a URL in Python

Using urllib From the Standard Library

The Real Python Podcast – Episode #166: Differentiating the Versions of Python & Unlocking IPython's Magic

Python 3.12 Preview: More Intuitive and Consistent F-Strings

F-Strings Had Some Limitations Before Python 3.12

Socket Programming in Python Part 1: Handling Connections

Python & APIs: A Winning Combo for Reading Public Data

Getting to Know APIs

SOAP vs REST vs GraphQL

APIs and requests: A Match Made in Heaven

The Real Python Podcast – Episode #165: Leveraging the Features of Your Database With Postgres and Python

Python's list Data Type: A Deep Dive With Examples

Getting Started With Python’s list Data Type

Creating Web Maps From Your Data With Python Folium

Python News: What's New From June 2023

PSF Elects New Directors

The Real Python Podcast – Episode #164: Constructing Python Library APIs & Tackling Jinja Templating

Unlock IPython's Magical Toolbox for Your Coding Journey

Install and Initiate IPython on Your Machine

Improve Your Interactive Python Workflow With IPython

Mazes in Python Part 2: Storing and Solving

Profiling in Python: How to Find Performance Bottlenecks

How to Find Performance Bottlenecks in Your Python Code Through Profiling

time: Measure the Execution Time

The Real Python Podcast – Episode #163: Python Crash Course & Learning Enough to Start Creating

Python 3.12 Preview: Support For the Linux perf Profiler

Seeing the Big Picture Through the Lens of perf

What’s the Linux perf Profiler?

How Can Python Benefit From perf?

Filtering Iterables With Python

How to Round Numbers in Python

Python’s Built-in round() Function

How Much Impact Can Rounding Have?

The Real Python Podcast – Episode #162: Exploring the Zen of Python & pandas Features for Finance

Why Are Membership Tests So Fast for range() in Python?

In Short: A Formula Determines Membership in a Python Range

Jinja Templating

The Real Python Podcast – Episode #161: Resources and Advice for Building CircuitPython Projects

Recursion in Python

The Real Python Podcast – Episode #160: Inheriting a Large Python Code Base & Building a GUI With Kivy

Python Basics: Reading and Writing Files

The Real Python Podcast – Episode #159: Volunteering, Organizing, and Finding a Python Community

Mazes in Python Part 1: Building and Visualizing

The Real Python Podcast – Episode #158: Building Python CI With Docker & Applying for a Hacker Initiative Grant

Getting the First Match From a Python List or Iterable

The Real Python Podcast – Episode #157: Discussing Mojo & Improving Python Object-Oriented Programming

Using k-Nearest Neighbors (kNN) in Python

Using `urllib` From the Standard Library

APIs and `requests`: A Match Made in Heaven

Getting Started With Python’s `list` Data Type

`time`: Measure the Execution Time

Seeing the Big Picture Through the Lens of `perf`

What’s the Linux `perf` Profiler?

How Can Python Benefit From `perf`?

Python’s Built-in `round()` Function