Using threads in PyGTK

Well, all this thought of asynchronicity is getting to me. But since we
mentioned how to use subprocesses and Twisted, I think it is only fair to talk a
bit about threads.

So, what will we be doing?

1. Create an abstraction for running threads in PyGTK
   
2. Use it
   

Now the thing which we will be wanting to run is a generator, call it
"take_really_long". And it will surely block, and might be a bit intensive, and
well we obviously need something asynchronous. A generator is a bit nicer to use
than a single plain long-running function, because we can update the user
interface every time it returns, making it much much easier to do things like
Progress Bars.

You must remember two things when using threads with PyGTK:

1. GTK Threads must be initialised with gtk.gdk.threads_init:

import gtk
gtk.gdk.threads_init()

2. Any code that modifies the UI (basically any code) that is called from
outside the main thread must be pushed into the main thread and called
asynchronously in the main loop, with gobject.idle_add:

import gobject, gtk

def set_progress_bar_fraction(fraction):
   progress_bar.set_fraction(0.2)

gobject.idle_add(set_progress_bar_fraction, 0.2)
gtk.main()

This Gobject/GTK "idle_time" is something like "as soon as the main loop can
reasonably get around to it".

So, to reiterate, when writing threaded code, I expect to see lots of calls to
gobject.idle_add.

Here is a silly example, that I urge you not to use, as we will concentrate on
abstracting out the boilerplate later.

from threading import Thread
import time
import gtk, gobject
gtk.gdk.threads_init()

class MainWindow(gtk.Window):
   def __init__(self):
       super(MainWindow, self).__init__()
       vb = gtk.VBox()
       self.add(vb)
       self.progress_bar = gtk.ProgressBar()
       vb.pack_start(self.progress_bar)
       b = gtk.Button(stock=gtk.STOCK_OK)
       vb.pack_start(b)
       b.connect('clicked', self.on_button_clicked)
       self.show_all()

   def on_button_clicked(self, button):
       self.count_in_thread(5)

   def count_in_thread(self, maximum):
       Thread(target=self.count_up, args=(maximum,)).start()

   def count_up(self, maximum):
       for i in xrange(maximum):
           fraction = (i + 1) / float(maximum)
           time.sleep(1)
           gobject.idle_add(self.set_progress_bar_fraction, fraction)

   def set_progress_bar_fraction(self, fraction):
       self.progress_bar.set_fraction(fraction)

w = MainWindow()
gtk.main()

I say this is silly, but it is not really, it's a nearly a real-world example,
and its probably how you would do it. Now imagine an application where you had
to do this a lot, it would eventually wear you down.

So we can move on to making an abstraction that allows us to forget about this
boilerplate and concentrate on integrating it smoothly with your code. This will
take the form of a class that is responsible for:

• starting a generator in a thread
  
• calling a callback with every result (yield) of the generator
  
• possibly calling a callback on completion
  

And here it is (modified from PIDA work by Tiago Cogumbrerio: Thanks Tiago!):

import threading, thread
import gobject, gtk

gtk.gdk.threads_init()


class GeneratorTask(object):

   def __init__(self, generator, loop_callback, complete_callback=None):
       self.generator = generator
       self.loop_callback = loop_callback
       self.complete_callback = complete_callback

   def _start(self, *args, **kwargs):
       self._stopped = False
       for ret in self.generator(*args, **kwargs):
           if self._stopped:
               thread.exit()
           gobject.idle_add(self._loop, ret)
       if self.complete_callback is not None:
           gobject.idle_add(self.complete_callback)

   def _loop(self, ret):
       if ret is None:
           ret = ()
       if not isinstance(ret, tuple):
           ret = (ret,)
       self.loop_callback(*ret)

   def start(self, *args, **kwargs):
       threading.Thread(target=self._start, args=args, kwargs=kwargs).start()

   def stop(self):
       self._stopped = True

So this does all of those things, you pass the generator, the callback for each
result and the callback for completion to the constructor, and the arguments to
call the generator with to the start method. This way you can call the same
GeneratorTask instance many times.

For an example of its use, lets modify our version above:

class MainWindow(gtk.Window):
   def __init__(self):
       super(MainWindow, self).__init__()
       vb = gtk.VBox()
       self.add(vb)
       self.progress_bar = gtk.ProgressBar()
       vb.pack_start(self.progress_bar)
       b = gtk.Button(stock=gtk.STOCK_OK)
       vb.pack_start(b)
       b.connect('clicked', self.on_button_clicked)
       self.show_all()

   def on_button_clicked(self, button):
       GeneratorTask(self.count_up,
                     self.set_progress_bar_fraction).start(5)

   def count_up(self, maximum):
       for i in xrange(maximum):
           fraction = (i + 1) / float(maximum)
           time.sleep(1)
           yield fraction

   def set_progress_bar_fraction(self, fraction):
       self.progress_bar.set_fraction(fraction)

w = MainWindow()
gtk.main()

One thing we have done is convert the count_up method to a generator so that we
can use it. You will notice that there is no real impact on the code that would make you
think "oh nasty, threads". The idle calls are taken care of for you, and the
generator task takes care of calling your UI function for each result.