A Twisted World: Extending Python with (open source) C++

Motivation

My purpose this evening was to write a Python module (in C++) to deal with process handling.This is related to the API spying project I mentioned earlier, where I need to be able to (at least) list and terminate processes.The solution I currently employ relies on WMI (details), and it's painfully slow.Other options are using PDH queries (also slow) or PSAPI + ctypes from Python (faster, but butt-ugly).

Since I've wanted for quite some time now to get my hands dirty and write a Python extension in C++, I figured out this would be an excellent opportunity.I also wanted to try out MinGW/Dev C++, so let's start from there.

MinGW, Dev C++ and DLLs

First, I downloaded MinGW and Dev C++. Dev C++ comes with MinGW bundled, but it hasn't been updated since 2005, so it's a rather old version of the compiler.I installed Dev C++, and then (after I ran into my first problems with Dev C++), I updated the compiler using the separately downloaded version.

Step 1. An empty DLL

In Dev C++: File -> New -> Project, check "C project", select "DLL" from the list and give it a name (in my case, vprocess); save it to an (empty) folder.

Remove the dll.h file from the project; it's not necesary (right-click it in the list on the left and click "Remove file").

Replace the contents of dllmain.c with the following (replacing vprocess with your module name):

#include <windows.h>
#include <Python.h>

__declspec(dllexport) void initvprocess();

PyMethodDef methods[] = {
 {NULL, NULL},
};

__declspec(dllexport) void initvprocess(void) {
    (void)Py_InitModule("vprocess", methods);
}
BOOL APIENTRY DllMain (HINSTANCE hInst, DWORD reason, LPVOID reserved) {
    return TRUE;
}

This is the template of a basic Python module; we'll study it in a moment. For now, notice that the DllMain always returns TRUE (regardless of the "reason" of the call): we have nothing to initialize in this particular module.If initialization operations are required, they can be performed in the init[modulename])() method.

Step 2. A Python module is born

At this point we should save the source and set up project options. In Project -> Project Options:

"Compiler" tab: turn on optimizations (not necesary, but useful);
"Build Options" tab: click "Override output filename" and enter the module name; the extension must be .pyd so as Python can find it and recognise it as a (binary) module;
"Parameters" tab: click "Add Library or Object", browse to the Python installation folder, "LIBS" subfolder, select libpython25.a.

Note: yes, it's libpython25.a, NOT python25.lib, as one might assume. At first I linked against python25.lib, and everything went fine up until I used the Py_None object, at which point I got this lovely linker error:

[Linker error] undefined reference to `_imp___Py_NoneStruct'

After a number of failed attempts to fix it (Google didn't come up with a solution/explanation, although the problem appears to be pretty common), I finally noticed something about libpython25.a in one of the results and figured out I might as well try that too. Jackpot!

The final configuration step is adding the Python "libs" and "include" paths to the compiler directory list. Tools -> Compiler Options -> Directories; in the "Libraries" tab add the "libs" subfolder of your Python installation (e.g. C:\Python25\libs), and in the "C Includes" tab add the "include" subfolder.

At this point you should have a compilable, yet useless Python module. Time to add some meat on it.

Step 3. Python objects & Windows Processes

We'll implement two methods, one that kills a process given it's PID and one that lists all running PIDs and returns a dictionary whose keys are PIDs and whose values are the executable file names of the respective PIDs.

The module methods must be declared in the call to Py_InitModule() inside the (exported) init[modulename] function as a list passed to the second argument to the function.In the above code, the methods variable contains a list of (C) tuples containing the module methods, the functions which implement them and the argument passing method (METH_VARARGS in our case will do just fine).All methods (must) return a PyObject*, and they must be decorated with __declspec(dllexport) so as the DLL will export them.

Step 3.1. killPID

First, let's look at the C code which, given a PID, obtains a handle of the corresponding process and calls TerminateProcess on it:

    HANDLE process;
    DWORD result;

    result = 0;
    process = OpenProcess(PROCESS_TERMINATE, 0, pid);
    if (process != INVALID_HANDLE_VALUE) {
       if (TerminateProcess(process, 0))
          result = 1;
       CloseHandle(process);
    }

The code is pretty straight-forward: it tries to get a hanndle (with the PROCESS_TERMINATE flag set) of the process with the given PID. If successful, it tries calling TerminateProcess on the handle and closes it.The pid variable will be obtained from the argument(s) passed to the function.If anything failed, the result variable will be set to 0; if everything worked properly, it will be set to 1.Also note that for the process-handling code to work, the tlhelp32.h header file must be included.

All that's left to do is to get the pid from the arguments using PyArg_ParseTuple and return the result variable as a Python integer (using Py_BuildValue to convert it):

__declspec(dllexport) PyObject* vprocess_killPID(PyObject *self, PyObject *args) {
    // kills a process given by PID
    // returns 1 if successful, 0 otherwise
    HANDLE process;
    DWORD pid, result;

    result = 0;
    PyArg_ParseTuple(args, "k", &pid);
    process = OpenProcess(PROCESS_TERMINATE, 0, pid);
    if (process != INVALID_HANDLE_VALUE) {
       if (TerminateProcess(process, 0))
          result = 1;
       CloseHandle(process);
    }

    return Py_BuildValue("k", result);
}

As is usually the case, error checking has been forsaken in the name of simplicity (laziness also helped).

Step 3.1. snapshot

This function will take a snapshot of the running processes and generate a Python dictionary as described above.

__declspec(dllexport) PyObject* vprocess_snapshot(PyObject *self, PyObject *args) {
    // generates a dictionary whose keys are process IDs and values are process names;
    // returns None if the snapshot fails
    PROCESSENTRY32 pe;
    HANDLE snapshot;
    PyObject* result;
    DWORD i;

    result = PyDict_New();
    snapshot = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
    if (snapshot == INVALID_HANDLE_VALUE) {
       Py_INCREF(Py_None);
       return Py_None;
    }

    pe.dwSize = sizeof(PROCESSENTRY32);
    if (Process32First(snapshot, &pe)) {
       do PyDict_SetItem(result, Py_BuildValue("k", pe.th32ProcessID), Py_BuildValue("s", pe.szExeFile));
       while (Process32Next(snapshot, &pe));
    }
    else {
       Py_INCREF(Py_None);
       return Py_None;
    }

 CloseHandle(snapshot);
 return result;
}

Interesting here is the method of building a dictionary, item by item.

Step 4. Putting it all together

The last step is to inform Python about the exported functions:

PyMethodDef methods[] = {
 {"snapshot", vprocess_snapshot, METH_VARARGS},
 {"killPID", vprocess_killPID, METH_VARARGS},
 {NULL, NULL},
};

Source code, compiled module and example .py source available here.

A Twisted World

Extending Python with (open source) C++ - Win32 Process Handling