Spawning subprocess with PyGTK using Twisted

Well, it is an age-old problem: How to schedule long-running tasks withing a GUI main loop (in our case PyGTK). There are a few ways:
  1. Use Python's subprocess module and select on the pipe with gobject's io_add_watch
  2. Use GTK's built in subprocess spawning abilities
  3. Use Twisted
1 & 2 are reasonable approaches, and they both work. Of course 1 won't work on Win32. The only problem with both 1 & 2 is that they use gobject's polling functions to achieve asynchronicity. This is nice when we are forced in a PyGTK main loop, but really not nice when the application wants to run in command line mode, and we really want to be able to share the execution code between different UIs, including perhaps other toolkits.

Enter Twisted. We need to do two things with Twisted:
  1. Make sure Twisted knows we are running with PyGTK
  2. Launch the process
Making sure Twisted knows that we are running inside PyGTK is quite easy (though I imagine the implementation was painful). To do this, you must install the gtk2reactor before importing any other reactors like so:


from twisted.internet import gtk2reactor
gtk2reactor.install()
from twisted.internet import reactor


Ok, I did say it was pretty easy. Now all that you need to remember with this is that you should now run your main loop with reactor.run, and not gtk.main.

Now we should think about spawning our subprocess. We will do this by using reactor.spawnProcess, which looks like:


twisted.internet.reactor.spawnProcess = spawnProcess(self,
    processProtocol,
    executable,
    args=(),
    env={},
    path=None,
    uid=None, gid=None, usePTY=0, childFDs=None
)

The only non-normal thing here is the processProtocol paramterer. All the other paramteres are standard things for things like subprocess.Popen. The processProtocol instance should be an instance of twisted.internet.protocol.ProcessProtocol, and defines how data is read from the pipes constructed to spawn the subprocess.

You can just use ProcessProtocol without overriding, but that will do nothing useful, not even print the results, so here is an example with our own ProcessProtocol class.


import os
# Have you remembered to install the gtk2reactor?
from twisted.internet import reactor
from twisted.internet.protocol import ProcessProtocol

class EchoingProcessProtocol(ProcessProtocol):

    # Will get called when the subprocess has data on stdout
    def outReceived(self, data):
        print 'STDOUT:', data
    
    # Will get called when the subprocess has data on stderr
    def errReceived(self, data):
        print 'STDERR:', data

    # Will get called when the subprocess starts
    def connectionMade(self):
        print 'Started running subprocess'

    # Will get called when the subprocess ends
    def processEnded(self, reason):
        print 'Completed running subprocess'

# Spawn the process and copy across the environment
reactor.spawnProcess(EchoingProcessProtocol(), 'ls', ['ls', '-al'], env=os.environ)
reactor.run()

Now the subprocess will execute, and anything written to the child's stdout will be printed to the screen.

This may seem entirely unremarkable, but this is now ready to plug into a GUI. Since the callback outReceived is called inside the gtk main loop, it won't block and it will be called when necessary, so it may as well do something like:


def outReceived(self, data):
    self.text_view.get_buffer().insert(
        self.text_view.get_buffer().get_end_iter(), data
    )

Which would add the line to a textview control (called self.text_view).

Links:
Twisted How-to Processes
Twisted How-to PYGTK

Popular posts from this blog

PyGTK, Py2exe, and Inno setup for single-file Windows installers

I am lucky, as part of my job, I get to code PyGTK. The downside is of course that I have to deploy these applications on Windows. Now these Windows users (especially at the price they are paying for this stuff) don't install dependencies, don't understand what a scripting language or interpreter is, and frankly they should not have to. Unfortunately, the list of dependencies for a PyGTK application (at the very minimum) is Python, GTK, PyGTK, PyGObject, PyCairo. 5 installers! So very early on in this project I found the way to give them exactly what they want and deserve, a single-file executable installer. There are a few guides online about how to achieve this, but none seem to work, and none are particularly new. How do we achieve this? The toolchain involved consists of two elements: 1. Py2exe website 2. Inno Setup website NOTE: The documentation for all the tools used is extensive, I will not repeat everything in there. Py2exe Py2exe is a distutils extension that searches...
Read more

Kiwi proxy widgets, a common widget API

Image
While we are on the notion of Kiwi , the PyGTK helper library, we should be discussing the proxy widgets. They offer a certain set of advantages over their plain PyGTK counterparts: They have a common signal emission API for change in content They offer a common API for setting and reading their value. They offer validation, and coercion They contain a few general improvements over the plain PyGTK widgets What does this mean? Simply it means you can make any data widget (eg Entry, CheckButton, Combo etc), you can connect its signal "content-changed", you can read the data with widget.read(), and set the value with widget.update(). If you have read any of the previous blogs about adapting Zope.schema fields to widgets, and you are alert, you will notice that this is amazingly useful. It means that we can define an interface for a general InputWidget type, something like: class IInputWidget(Interface): def read(): """Read the value in the widget"""...
Read more

Using threads in PyGTK

Well, all this thought of asynchronicity is getting to me. But since we mentioned how to use subprocesses and Twisted, I think it is only fair to talk a bit about threads. So, what will we be doing? Create an abstraction for running threads in PyGTK Use it Now the thing which we will be wanting to run is a generator, call it "take_really_long". And it will surely block, and might be a bit intensive, and well we obviously need something asynchronous. A generator is a bit nicer to use than a single plain long-running function, because we can update the user interface every time it returns, making it much much easier to do things like Progress Bars. You must remember two things when using threads with PyGTK: 1. GTK Threads must be initialised with gtk.gdk.threads_init: import gtk gtk.gdk.threads_init() 2. Any code that modifies the UI (basically any code) that is called from outside the main thread must be pushed into the main thread and called asynchronously in the main loop, w...
Read more