10.1. Debugging Tools

First create your toolbox, in this one we have:

  • Debug version of Python - great for finding out more detail of your Python code as it executes.

  • Valgrind - the goto tool for memory leaks. It is a little tricky to get working but should be in every developers toolbox.

  • OS memory monitioring - this is a quick and simple way of identifying whether memory leaks are happening or not. An example is given below: A Simple Memory Monitor

10.1.1. Build a Debug Version of Python

There are a large combination of debug builds of Python that you can create and each one will give you extra information when you either:

  • Invoke Python with a command line option.

    • Example: a Py_DEBUG build invoking Python with python -X showrefcount

  • Set an environment variable.

    • Example: a Py_DEBUG build invoking Python with PYTHONMALLOCSTATS=1 python

  • Additional functions that are added to the sys module that can give useful information.

    • Example: a Py_DEBUG build an calling sys.getobjects(...).

See here Building and Using a Debug Version of Python for instructions on how to do this.

10.1.2. Valgrind

See here Building Python for Valgrind for instructions on how to build Valgrind.

See here Using Valgrind for instructions on how to use Valgrind.

Here Finding Where the Leak is With Valgrind is an example of finding a leak with Valgrind.

10.1.3. A Simple Memory Monitor

Here is a simple process memory monitor using the psutil library:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
import sys
import time

import psutil

def memMon(pid, freq=1.0):
    proc = psutil.Process(pid)
    print(proc.memory_info_ex())
    prev_mem = None
    while True:
        try:
            mem = proc.memory_info().rss / 1e6
            if prev_mem is None:
                print('{:10.3f} [Mb]'.format(mem))
            else:
                print('{:10.3f} [Mb] {:+10.3f} [Mb]'.format(mem, mem - prev_mem))
            prev_mem = mem
            time.sleep(freq)
        except KeyboardInterrupt:
            try:
                input(' Pausing memMon, <cr> to continue, ^C to end...')
            except KeyboardInterrupt:
                print('\n')
                return

if __name__ == '__main__':
    if len(sys.argv) < 2:
        print('Usage: python pidmon.py <PID>')
        sys.exit(1)
    pid = int(sys.argv[1])
    memMon(pid)
    sys.exit(0)

Lets test it. In one shell fire up Python and find its PID:

>>> import os
>>> os.getpid()
13360

In a second shell fire up pidmon.py with this PID:

$ python3 pidmon.py 13360
pextmem(rss=7364608, vms=2526482432, pfaults=9793536, pageins=24576)
     7.365 [Mb]
     7.365 [Mb]     +0.000 [Mb]
     7.365 [Mb]     +0.000 [Mb]
...

Pause pidmon.py with Ctrl-C:

^C Pausing memMon, <cr> to continue, ^C to end...

Go back to the first shell and create a large string (1Gb):

>>> s = ' ' * 1024**3

In the second shell continue pidmon.py with <cr> and we see the memory usage:

1077.932 [Mb]  +1070.567 [Mb]
1077.932 [Mb]     +0.000 [Mb]
1077.932 [Mb]     +0.000 [Mb]
...

Go back to the first shell and delete the string:

>>> del s

In the second shell we see the memory usage drop:

1077.953 [Mb]     +0.020 [Mb]
1077.953 [Mb]     +0.000 [Mb]
 272.679 [Mb]   -805.274 [Mb]
   4.243 [Mb]   -268.435 [Mb]
   4.243 [Mb]     +0.000 [Mb]
...

In the second shell halt pidmon with two Ctrl-C commands:

^C Pausing memMon, <cr> to continue, ^C to end...^C

So we can observe the total memory usage of another process simply and cheaply. This is often the first test to do when examining processes for memory leaks.