The Python program has a useful parameter: -v When Python is launched with this parameter to run a Python program, it prints out all the Python modules and dynamic libraries that are loaded during the executing. This can be useful to see exactly which Python files and shared library files are loaded and in which order. This parameter also prints out the modules as they are cleaned up at the end of program execution.
$ python -v foo.py
Tried with: Python 2.7.3 and Ubuntu 12.04 LTS
It is really easy to profile a Python program. But, it is not straightforward to determine the different bottlenecks in the program by looking at the profiler output. A visualization of the profiler output makes it really convenient to examine the different sources of bottleneck in the program. One way to visualize the profiler output is to use RunSnakeRun.
Install RunSnakeRun from PyPI using easy_install:
$ sudo easy_install SquareMap RunSnakeRun
Open the profiler statistics file with RunSnakeRun:
$ runsnake foostats
RunSnakeRun shows a tabular view of the profiler statistics, similar to the output of cProfile. It also shows a squaremap visualization of the call tree. This makes it really convenient to drill down to the various bottlenecks in the program.
Tried with: RunSnakeRun 2.0.2b1, Python 2.7.3 and Ubuntu 12.04 LTS
cProfile is one of the many profilers that ship with Python.
To profile a function in your Python program:
# Assuming we want to profile function doMain() import cProfile cProfile.run( "doMain( a, b )" )
When a function call is executed with the profiler, it prints out detailed profiling information after the function call finishes execution.
To dump the profiler statistics to a file named foo.stats:
# Assuming we want to profile function doMain() import cProfile cProfile.run( "doMain( a, b )", filename="foo.stats" )
The stats file is not human readable, but is written in the pstats format. This can be opened by using the pstats Python module in a script or using other tools that understand this format.
To profile a Python program named foo.py:
$ python -m cProfile foo.py
To profile a Python program and dump the profiler statistics to a file named foo.stats:
$ python -m cProfile -o foo.stats foo.py
Tried with: Python 2.7.6 and Ubuntu 14.04
Python code is compiled into Python bytecode, which is then executed on the Python virtual machine. Sometimes it is interesting to see what bytecode is produced from your Python code. Thankfully, Python ships with batteries included. So, it has a disassembler module built in that is called dis.
The dis method of the dis module can be easily used to disassemble a method in your Python code to its bytecode:
| $ cat foo.py | |
| def bar(): | |
| a = 10 | |
| b = 2 | |
| c = a + b | |
| $ python | |
| >>> import dis | |
| >>> import foo | |
| >>> dis.dis(foo.bar) | |
| 2 0 LOAD_CONST 1 (10) | |
| 3 STORE_FAST 0 (a) | |
| 3 6 LOAD_CONST 2 (2) | |
| 9 STORE_FAST 1 (b) | |
| 4 12 LOAD_FAST 0 (a) | |
| 15 LOAD_FAST 1 (b) | |
| 18 BINARY_ADD | |
| 19 STORE_FAST 2 (c) | |
| 22 LOAD_CONST 0 (None) | |
| 25 RETURN_VALUE |
To see other methods of the dis module and information about the bytecode instructions, see dis in the Python documentation.
Tried with: Python 2.7.3 and Ubuntu 12.04 LTS
Python 2.x does not have an equivalent of the enum in C or C++. Python 3.x (3.4 and later) has a builtin enum type.
You can install the backported Python 3.4 enum for Python 2.x easily from here:
$ sudo pip install enum34
Usage of this enum is simple:
from enum import Enum
class Colors(Enum):
Red = 1
Green = 2
Blue = 3
c = Colors.Green
Tried with: Python 2.7.6 and Ubuntu 14.04
In some applications, you might need to implement a queue that starts off empty, but you want it to grow and be limited to a certain length. Such a queue with a maximum length can be implemented easily using deque:
| # Queue with max length of 3 | |
| from collections import deque | |
| q = deque( maxlen=3 ) | |
| # deque([]) | |
| q.append( 10 ) | |
| q.append( 20 ) | |
| q.append( 30 ) | |
| # deque([ 10, 20, 30 ]) | |
| q.append( 40 ) | |
| # deque([ 20, 30, 40 ]) |
Tried with: Python 2.7.3
The ubiquitous list can be used as a queue in Python. But, that is not efficient because of the way lists are implemented.
Using deque from the collections module is a straightforward way to use queues:
| # remove <– [….] <– insert | |
| # Queue with insertion from right and removal from left | |
| from collections import deque | |
| q = deque() | |
| # deque([]) | |
| q.append( 10 ) | |
| # deque([ 10 ]) | |
| q.append( 50 ) | |
| # deque([ 10, 50 ]) | |
| q.append( 30 ) | |
| # deque([ 10, 50, 30]) | |
| q.popleft() | |
| # 10 |
Tried with: Python 2.7.3
The argparse module makes it really easy to read, parse and use the input arguments passed to a Python program.
Usage is in 3 simple steps:
ArgumentParseradd_argumentparse_args. The arguments are returned in an object with the values available using the object members.Here is a simple program that illustrates usage of argparse with different types of input arguments:
| import argparse | |
| import sys | |
| def foo(args): | |
| print args | |
| print "Radius: ", args.rad | |
| print "Distance: ", args.d | |
| print "Name: ", args.name | |
| print "Recompute:", args.recompute | |
| print "In files: ", args.in_files | |
| def main(): | |
| # Create parser | |
| arg_parser = argparse.ArgumentParser(description="This program achieves nothing!") | |
| # Add arguments you want to handle | |
| modes = ["release", "debug"] | |
| arg_parser.add_argument("–radius", type=float, default=1.2, help="Radius of the component") | |
| arg_parser.add_argument("-d", type=int, default=9, help="Distance to the component") | |
| arg_parser.add_argument("–name", type=str, help="Name of component") | |
| arg_parser.add_argument("–mode", type=str, choices=modes, help="Mode to use. Should be either release or debug") | |
| arg_parser.add_argument("–recompute", action="store_true", default=False, help="Indicate if recomputation is needed") | |
| arg_parser.add_argument("–in_files", type=str, nargs="+", help="One or more input file paths") | |
| if len(sys.argv) == 1: | |
| arg_parser.print_help() | |
| sys.exit(1) | |
| # Parse sys.argv using parser | |
| args = arg_parser.parse_args() | |
| # Pass arguments around | |
| foo(args) | |
| if __name__ == "__main__": | |
| main() |
This call has many useful options, the only one I actually use is:
description: Specify what you want to be displayed about this program when it is invoked using --help.This call has many useful options. Many of them are needed in most scripts:
radius, then it becomes a positional argument. So, you cannot invoke its option name, but can directly pass its value. For example: ./main.py 3.14-r or double dash --radius. You can access the value of this option later in the argument object using r or radius member respectively.radius.--radius 13 or using the equals symbol --radius=13.store_true or store_false respectively.--help.-h or --help, it will list the input options available and the documentation you have provided for each option.Tried with: Python 3.5 and Ubuntu 16.04
Problem
Run ipython3.exe and it quits with this error: ImportError: No module named pkg_resources
Solution
pkg_resources is a module that ships with the setuptools package. setuptools has been adandoned now, but a fork of it named distribute provides this module. So, install distribute (see how) and this error will be gone.
Tried with: ipython 0.12.1, Python 3.2 64-bit and Windows 7 64-bit
Why distribute?
If you try to install the setuptools module for Python 3.x 64-bit on Windows, you will find that it is not available. So, you cannot use easy_install, which is provided by setuptools, on your Python 3.x!
This is because setuptools has been abandoned and it is no longer maintained. What you will need to install is distribute. This module is a fork of setuptools that offers many enhancements. But, it also strives to offer the same tools and functionality offered by setuptools. So, distribute is there whenever something or someone needs the old setuptools on new versions of Python.
How to install it?
The easiest way to install popular Python modules on Windows remains the Python binaries maintained by Christoph Gohlke. Just download the distribute package which matches your Python and Windows installation and install it.
Tried with: distribute 0.6.27, Python 3.2 64-bit and Windows 7 64-bit
Code Yarns 👨💻 