Boto : Super easy Python library to interact with amazon web services

I had first heard of Mitch Granaats boto thanks to “Monster Muck Mashup” where he used boto to build a video transcoding service using amazons compute and storage cloud EC2 and S3. Being a relative python newbie then, I remember reading the code examples and not understanding what they were doing entirely.

After a long period of dormancy I decided to resurrect and cleanup my S3 buckets. Stupid me! , I had turned on logging for the code-itch bucket ever so long ago. Result!,tens of thousands of log entries. Every S3 browser I tried, struggled to display and scroll the massive number of files. I needed a script and quick.

Initially I considered jets3t a java based super-library for all things AWS. But after reading an answer to this question on stackoverflow I decided to try boto again. This time around , I found the syntax very easy to comprehend. Within minutes I had a script that worked. The script read like pseudocode. I will reproduce it here.

I will definitely be delving more into boto to explore amazons many offerings.

Mitch Granaat also writes an excellent blog on all things aws and boto at elastician

Boto library home on googlecode

#!/usr/bin/python
import boto
from boto import *
import re
# Regexp to match the log file names
log_key = re.compile("code-itch\.[\d]{4}\-[\d]{1,2}\-[\d]{1,2}\-[\d]{1,2}\-[\d]{1,2}\-[\d]{1,2}\-[a-fA-F\d]{16}")
f = open("listbuckets_code-itch_deleted.txt" , "w")
if __name__ == '__main__':
    s3 = boto.connect_s3()
    mybucket = s3.get_bucket("code-itch")
    for key in mybucket:
        if log_key.match(key.name):
            f.write(key.name + "\n")
            mybucket.delete_key(key.name)
            print "Deleted",key.name
            f.close()

Python scripting from within Pymol

I have been using pymol for a while , but only recently did I start using python scripts to automate my tasks within pymol.

The extreme power of pymols API allows you to call python scripts from within a Pymol session as I have done in the screencast.
This text will be replaced

Additional Links and information:Scripting in Pymol
The code segment I read into the pymol session is given here:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# My list of objects in pymol session
objs = ["3org","1ots","mypdb"]
# List to keep track of pairs that have been rendered
images_done = []
for i in objs:
    for j in objs:
        if i == j :
            continue
        else:
            if (j,i) in images_done or (i,j) in images_done:
                continue
            else:
                cmd.disable("all")
                cmd.enable(i)
                cmd.enable(j)
                cmd.png("%s_%s_superpose3.png" % (i,j),640,480,72,1,1)
                cmd.disable("all")
                images_done.append((i,j))
print "all images rendered"

Crystallization reports at the RCSB or PDB : crystallization conditions for already crystallized protein

Embarking on a quest to crystallize a protein that others have already crystallized?  You need to get a detailed report of all the crystal conditions that have been used in the literature for your protein of interest.
Thanks to the many excellent features of the PDB  (RCSB)  you can use the Generate Reports drop down tab to get  a nice tabular report of all the Experimental, crystallization conditions once you have narrowed down your list of PDB files.
The above short screencast, shows you how.
Static image credit: Tabular help documentation at the RCSB/PDB

Why the ipad will change everything

I was watching Steve jobs’s ipad announcement keynote the few months back and a few minutes into it when the ipad slide first appeared ..I impulsively said ” they will sell it for $499 and this will change everything” . A few slides later I felt happy I was right with my price bet and I started thinking about what I think the device will do for us.

As we are now weeks from the device getting to market I thought I would put down my thoughts on how well the ipad will do.

The ipad IS the new “ipod” and “iphone” , a trendsetting device which will first define and then dominate the category of handheld content consumption devices.

At 499 it is the perfect internet reading device . Yes it will bring the internet to everyone!. Just as we all stopped using our CD collection once we could carry our albeit lower quality mp3 collections around with us. We will stop using our laptops and computers to consume our internet content on the ipad and other similar devices.

Multitouch is the best thing to happen to UI since windows and the mouse. Anyone who has an iphone and the kindle realizes how much more intuitive it is to slide across the screen and touch something and see it respond. It does seem like Apple with its custom silicon and ipad screen will dominate the innovations around multitouch as it gets integrated into more and more apps that work on this bigger screen. The new avataar of of iphoto and Mail.app it seems are so revolutionary in their approach to UI, that ipad apps will become the new showcases for killer apps across all platforms.

Despite the fact that it does not give you the refresh free comfort of e-ink. The fact that it can do video and the web natively will ensure that e-ink and kindles etc move away from mainstream towards the hardcore reader market. The ipad will become the reader of choice for everyone. It will be wrong to call this the kindle-killer , but for everyone who has not already bought a kindle or reads less than one book a week , they will prefer the ipad.

The rebirth of the web-app. If you have ever saved a website as an icon on your iphone , or used a site specific browser like Fluid or others , you know how much sense it makes to consume a website as an App. The ipad will ensure that javascript and native web interfaces will get comeback as the chosen way of delivering functionality to users without suffering the control of the apple app store.

Perhaps contradictorily, the app store will retain its stranglehold over the iphone-ipad ecosystem. Just as everything from Yelp to Evernote to even gmail is now “just an app” for everyone. Teams of people will develop exclusive Apps for their portals , if they chose not to re-engineer their websites to feel native on the ipad screen. App development will now make more sense for everything from board games to radiology and imaging to photo-management and sharing.

If there ever was a non computer – web browsing device..this is it! It really will change the way we consume content, watch video , listen to podcasts , watch HULU , Boxee , read Ebooks , subscribe to podcasts, watch webcasts….you name it!. I believe the ipad is the next iphone and will be a worthy successor to the ipod and iphone as trendsetting devices that will be huge successes.

The burden of python deployment

Sorry for a grouchy new years post , but I figured I will attempt to get started blogging for the year with this post. I had authored this post a while back ( October 09) when I struggled with deployment of a python app. Much of what I talk about still holds true. Indeed python deployment remains a problem with many solutions , and therefore quite painful since there is still no sure shot way to deploy a python app across platforms.

Its been almost three weeks since I released my maiden python-wxpython app. While Ive been busy doing other things , I decided to write this post to say how painful the “deployment” process of writing an application has been.
The most ironic fact is that the easiest platform to deploy for has ironically been Windows.
Once I used py2exe and innosetup I had a working setup.exe that installed the app on Windows Xp and Vista , 32 and 64 bit.
The next most fun platform to create a deplyment target for was linux. It was almost trivial to use the cxfreeze to create a binary build for Ubuntu Linux 32 bit and 64 bit . With the help of a Virtualbox VM for 64 bit SUSE, I could even release a binary build for that platform because someone asked.

But thats when the fun ended!. The mac has been an extremely painful platform to deploy python application for!.
I am using py2app to create standalone binaries. The idea is to have an app that I can click on and have it launch on any mac.
Some interesting hurdles:
1) If you create a standaone py2app build on Leopard it is not guaranteed to work on tiger
2) If you create one on Leopard it definitely does not work on snow leopard.

The only solution is to create a standalone app on tiger and then maybe it will work on Leopard.

In the end I did manage to solve my mac deployment issues which I detailed on this wiki page. Also Chris Lasher Lasher pointed me onto his series of links which talk about python deployment, Ian Bickings response is specially worth noting.