Calling the Google Drive API and other Google APIs asynchronously with Twisted

You may know that the Google API Python Client is built on httplib2. This is a reasonable general choice, but the tight coupling is unhelpful in situations where a different HTTP library, or an entirely different approach to network programming should be used. An example of this is Twisted.

Aside: I won't be going on about how awesome Twisted is, but let's just take it for granted that it is so awesome that I could not write this application without it.

Httplib2 is blocking, and that makes it incompatible with being run inside the Twisted reactor. Fortunately we are only the latest person to have this problem, and a solution exists:

twisted.internet.threads.deferToThread

api_call = drive.files().list()

def on_list(resp):
  for item in resp['items']:
    print item['title']

d = deferToThread(api_call.execute)
d.addCallback(on_list)
A blocking call will be called in a thread and will callback on the returned deferred when it is done. I appreciate that no one* is 100% happy with this solution. "Argh, threads!", "Argh, async!" But it is a testament to the Greatness of Twisted that it has this sort of facility to play well with other, less flexible, systems.

*Did I mention? I am 100% happy.

Please comment on this post in Google+

Popular posts from this blog

PyGTK, Py2exe, and Inno setup for single-file Windows installers

I am lucky, as part of my job, I get to code PyGTK. The downside is of course that I have to deploy these applications on Windows. Now these Windows users (especially at the price they are paying for this stuff) don't install dependencies, don't understand what a scripting language or interpreter is, and frankly they should not have to. Unfortunately, the list of dependencies for a PyGTK application (at the very minimum) is Python, GTK, PyGTK, PyGObject, PyCairo. 5 installers! So very early on in this project I found the way to give them exactly what they want and deserve, a single-file executable installer. There are a few guides online about how to achieve this, but none seem to work, and none are particularly new. How do we achieve this? The toolchain involved consists of two elements: 1. Py2exe website 2. Inno Setup website NOTE: The documentation for all the tools used is extensive, I will not repeat everything in there. Py2exe Py2exe is a distutils extension that searches
Read more

ESP-IDF for Arduino Users, Tutorials Part 2: Delay

This is the second part of a series of ESP-IDF tutorials that I will complete as I learn stuff. Read part 1. One of the best things about Arduino is the ability to just block for a period with: delay(1000); // hang on a second, buddy It’s also one of the worst things. As soon as you need to do a few things at the same time, you will end up using better systems, like calling millis() or using the elapsedMillis library. Anyway, the IDF has you covered. You can delay, but amazingly it appears that once you multi-task (which you will with Free RTOS), the delay doesn’t block the other tasks from running. My Notes: Read the docs, now or later https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/freertos.html Delay doesn’t have to block like Arduino. Delay some milliseconds like this: vTaskDelay(pdMS_TO_TICKS(10)); Since vTaskDelay accepts ticks, the pdMS_TO_TICKS macro will convert for you. Help yourself out by adding your own macros if you are lazy. I am lazy. #
Read more

How to install IronPython2 with Mono on Ubuntu

This is just an instructional post with the hope that it might help others in the future. I spent a long time trying to get IronPython2 running on Mono. Now IronPython1 ships with Ubuntu, so apt-get install ironpython , if that's all that you want. Ironically, I don't really know the difference between IronPython1 and IronPython2, but "2 has got to be better than 1", right? The good news is that its very possible, but you will have to step out of your comfort zones. You will need: Mono 2.4 (sources) and IronPython 2.0.1 (binary) First build Mono (note that you need Ubuntu's Mono to bootstrap, so apt-get install mono ) but make sure you put it somewhere special so it doesn't mess with the system-installed Mono. This advice is taken from an article on the Mono wiki . $ tar xvjf mono-2.4.tar.bz2 $ cd mono $ mkdir -p ~/opt/mono $ ./configure --prefix=~/opt/mono $ make && make install So this is putting Mono in opt/mono in my home directory (which is wher
Read more