In this post I will talk about some tools and ways in which you can profile the interpreter, when running a Python script.
Just like in our previous post, the meaning of CPU profiling is the same, but now we are not targeting the Python script. Instead we want to know how the Python interpreter works and where it spends the most time when running our Python script.
We see next how you can trace the CPU usage and find the hotspots in our interpreter.
Sendmsg and recvmsg are two system calls which allow to send messages on a socket as one could by using send/recv or sendto/recvfrom with a few noticeable differences:
IPV6_PKTINFO) to be attached in the ancillary.If you read this, perhaps you already tried once to switch to from CPython to PyPy and you realized that matplotlib is not working. Or someone else sent you here to don’t waste your time :D.
In this post I will guide you, step by step to compile yourself matplotlib for CPython 2.7 and PyPy with all necessary dependencies.
In this post I will talk about some tools that can help us solve another painful problem in Python: profiling CPU usage.
CPU profiling means measuring the performance of our code by analyzing the way CPU executes the code. This translates to finding the hot spots in our code and see how we can deal with them.
We will see next how you can trace the CPU usage used by your Python scripts. We will focus on the following profilers (click them to go to the corresponding section in this blog):
In this post I will talk about some tools that can help us solve a painful problem in Python, especially when using PyPy: memory consumption.
Why are we concerned with this in the first place? Why don’t we care only about performance? The answer to these questions is rather complex, but I’ll summarize it.
PyPy is an alternative Python interpreter, that features some great advantages over CPython: speed (through it’s Just in Time compiler), compatibility (it is almost a drop in replacement of CPython) and concurrency (using stackless and greenlets).
One downside of PyPy is that in general it uses more memory than CPython, due to it’s JIT and garbage collector implementation. Nevertheless, in some cases, it is able to use less memory than CPython.
We will see next how you can measure the amount of memory used by your application.
This is the first post in the series “Hunting Performance in Python Code”. Through each post I’ll present some of the tools and profilers that exists for Python code and how each of them helps you to better find bottlenecks both in frontend (Python scripts) and/or in the backend (Python interpreter).
PyPy, compared to CPython relies more on achieving speed-up by “jitting” code as often as possible, rather than rely on its interpreter. However, jitting is not always an option, or at least not entirely. A good improvement for CPython, that we think might benefit PyPy as well, without impacting the JIT performance is Profile Guided Optimization (PGO or profopt).
I thank the PyPy developer community for their patience, kind advice and constant feedback they gave me in #pypy IRC or through email, which helped me to make this possible, especially to Carl Friedrich Bolz-Tereick and Armin Rigo.
This post contains details about a performance comparison between PyPy and CPython, when applied to OpenStack Neutron. The steps did to switch from CPython to PyPy are described in my post here.Read More »
This is a guide to deploy OpenStack Neutron with PyPy support. The underlying purpose was to determine whether using PyPy instead of CPython works for the networking service, and whether we would notice any improvements and/or drawbacks. The performance results of this can be read in my post here.
Python Files 