Debugging Python without an IDE – ipdb and web-PDB

In my previous post, I discussed how to debug python applications using the built-in python debugger, pdb. In this post I discuss alternatives to pdb that, in my opinion, provide a better interface to debugging.

pdb is limited in the sense that it can only be operated from the shell, and I repeatedly have to use the l command to see the code. And I also have to explicitly print each variable to see its value.

So, even though pdb is great, I would still prefer to have some more features. That’s where ipdb and web-pdb come along.

ipdb

ipdb is an integration between the IPython shell and pdb. That means we get features like syntax highlighting and tab completion with the same debugging interface as pdb.

To install ipdb, simply run

$ pip install ipdb

Just like last time, I’m going to use a sample program to illustrate how we would go about debugging using ipdb.

Consider the following code

def filter_even(numbers):
    for i in range(0, len(numbers)):
        if numbers[i] % 2 == 0:
            del numbers[i]

    return numbers

print(filter_even([1,2,3,4,5,6]))

At the outset this function looks very simple. It iterates over the list of numbers passed and filters out the even numbers leaving us with all of the odd numbers.

However, we get the following exception when we run this function;

$ python test.py
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    print(filter_even([1,2,3,4,5,6]))
  File "test.py", line 4, in filter_even
    if numbers[i] % 2 == 0:
IndexError: list index out of range

Okay, so what happened here? I looped from 0 to 1 less than the length of the numbers list, why did I get an IndexError?

Let’s use ipdb to debug this. Same as before, we can add a breakpoint in our code using the following statement

import ipdb; ipdb.set_trace()

Here’s how a debug session would look like using ipdb.

As you can see, I created a breakpoint inside the loop and I checked the value of i and the list numbers at each iteration. On the 5th iteration when i is 4, the numbers list has been reduced to [1, 3, 5] which causes the IndexError, since the size of the list is less than 4.

Post-mortem debugging

At this point I’ll take a slight detour to mention a really cool feature of ipdb. In the above video, you can see that I had to continue execution 4 times till I reached the iteration in which the exception occurs. This could become very tedious in some cases. Luckily, ipdb provides us with a mechanism through which we can directly start the debugger only when an exception occurs. This is known as post-mortem debugging.

Using the launch_ipdb_on_exception function, I am immediately able to go to the context in which the exception is raised. And I can see the values of the variables in that context as well. I find this to be a huge time saver in many cases.

Now then, back to the code. What we have witnessed above is that while I am iterating over the list, I am also deleting the unwanted elements from the list using the del keyword. What this does is that it modifies the list I am iterating over while I’m iterating over it. since I’m deleting elements from the list, it becomes shorter and shorter till we reach a point where the index is greater than the length of the list. Hence the IndexError exception is raised.

The way to rectify is to create a new list with the filtered elements then return the new list. The pythonic way of doing this would be

def filter_even(numbers):
    return [n for n in numbers if not n % 2 == 0]

Using web-PDB

My final tool for python debugging is web-pdb and this is my personal favorite.

pdb and ipdb are great tools but they are still limited in the sense that they can only be used from the shell. web-pdb however provides us a complete graphical interface to the debugger.

To demonstrate this, I will again debug the above code, and will be using web-pdb this time.

Just like ipdb, web-pdb has to be installed in your python environment to be usable.

$ pip install web-pdb

and the breakpoint is set using;

import web_pdb; web_pdb.set_trace()

Let’s see how a debugging session would look like using web-pdb

And just like we did with ipdb we can do post-mortem debugging and go directly to the context which raised the exception.

We can do that with the catch_post_mortem() function provided by web-PDB

We reach the context which raises the exception instantly, and we have a nice graphical interface in which we can see the currently executing code, the global variables, the local variables as well as some controls to manage the flow of the program.

We can immediately see the variables which are causing the exception on the right side of the screen.

Remote debugging with web-PDB

Since web-PDB opens up a network port on the machine (port 5555) it can be used to debug code that is remotely executing on a server or inside a docker container or a virtual machine.

I use web-PDB extensively in my workflow. And when a bug is found on a development or staging server that isn’t reproduced locally. In that case I simply SSH into the server, insert a post-mortem breakpoint on the suspected code, and run the code. Once an exception is raised, I get a complete debugging interface along with the context of execution right from the comfort of my own web browser.

Simply bliss.

Of course that requires some configuration on the DevOps end i.e. the 5555 port has to be made accessible to my IP, but that is a small price to pay for this level of convenience and productivity.

Conclusion

I’ve covered a total of 3 tools that I use to debug Python applications

• pdb
• ipdb
• web-PDB

I hope that after reading this, you guys have at least gotten a basic overview of how these tools work in python and how you can debug your code more efficiently.

Hit me up if there are any questions.

Ciao!