Tag Archives: Python

Pretty documentation and code snippets with snipt.net

Whether I am learning a new module or thinking up a new set of python modules to solve a problem at hand, I use google docs extensively to collect my thoughts and plan out my code.

In my google docs, I very often grab screenshots ( esp of dragged out screen area ) with ⌘-CTRL-Shift-4 and directly paste them into my Google Docs on Google Chrome as images. However, when it comes to inserting snippets of code I wrote into these documents I really like using snipt.net. I first came across snipt.net after Kenneth Love used it in his kickstarter funded getting started with django lessons.

snipt.net copy paste of html formatted syntax colored code is awesome!
snipt.net copy paste of html formatted syntax colored code is awesome!

With snipt.net , any code snippet is a copy-paste away.  I found myself using snipt.net so much that I decided to get the pro-account . To create a snipt, I copy and paste my code into the snipt.net form . Then by playing with a number of formatting options, I can change the way my code is colored and syntax highlighted . Once I am happy with the look . I save and close the snippet. Then I just copy and paste the final snippet view into google docs or make the snipt public and grab the javascript code and paste it into the Text compose view in WordPress.

For example here is some pretty,formatted python code output rendered by snipt.net javascript.

By default all snippets in paid accounts are private. Both public and private posts can be tagged, given a title and an optional description. The public posts can be directly published with a default built in template allowing for very quick “code focussed blogging”. These posts have a comments section that integrates with Facebook, Twitter, Google and Disqus logins.

A code snippet "blog post" on snipt.net.
A code snippet “blog post” on snipt.net.

Although I dont use its blogging tools much, snipt.net has made inserting syntax colored code into my Google Docs and blog posts such as this one super easy!.

Customizing you Django user admin with Inline sections

Screen Shot 2013-05-12 at 3.56.43 PM

I was using django-userena for user management on a django app. Userena is a django app that works with “django.contrib.auth.user” to provide additional features such activation, password resets over email and messaging in addition to signin and signups.

With Userena the django Auth admin site had a number of “Inline” management sections that incorporated the information coming in from the Userena app relevant to the particular user.

The sections were:
Username:Password
Personal Info
Permissions
Important Dates
Userena Registrations

I wanted to add my Custom profile to the Bottom of that list. My custom profile , otherwise also known as the one you set as the “AUTH_PROFILE_MODULE”

AUTH_PROFILE_MODULE = “accounts.MyCustomProfile”

How does one do that?

Well basically you want to extend the “custom” admin interface . The relevant django doc for all things ModelAdmin is located here.

My model which I want to add in: is

from django.db import models
from django.contrib.auth.models import User
from django.utils.translation import ugettext as _
from userena.models import UserenaBaseProfile


# Create your models here.

class MyCustomProfile(UserenaBaseProfile):
    """The Userena User profile"""
    user = models.OneToOneField(User,
        unique=True,
        verbose_name=_('user'),
        related_name='my_profile')
    bio = models.CharField(max_length=1000,default="",blank=True)
    # Make this field not editable because it controls access to uid from legacy db
    my_legacy_user_object = models.SmallIntegerField(null=True,editable=False,unique=True)

    def __unicode__(self):
        return self.bio

To accomplish the inline display of this model along with the rest of the stuff userena already put into the admin page for the user I had to add this as my admin.py for the accounts app.

__author__ = 'hari'
from django.contrib import admin
from .models import MyCustomProfile
from userena.utils import get_user_model,get_profile_model
from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.models import User
from userena.admin import UserenaAdmin,UserenaSignupInline

class MyCustomProfileAdmin(admin.ModelAdmin):
    list_display = ["user" ,"bio","my_legacy_user_object"]
    search_fields = ["user__first_name","user__last_name","my_legacy_user_object"]

#docs from https://github.com/bread-and-pepper/django-userena/blob/master/docs/faq.rst
admin.site.unregister(get_profile_model())
admin.site.register(MyCustomProfile,MyCustomProfileAdmin)

class MyCustomProfileAdminInline(admin.StackedInline):
    model = MyCustomProfile


class MyCustomProfileAddedAdmin(UserenaAdmin):
    inlines = [UserenaSignupInline,MyCustomProfileAdminInline]

admin.site.unregister(get_user_model())
admin.site.register(get_user_model(),MyCustomProfileAddedAdmin)

The result is what we get nicely here

Command line handling in Python with optparse

Command lines interfaces to programs are very empowering. I started using computers with the  Linux command line and  I have never strayed too far from programs that are predominantly command line based. Whether it was rasmol , povray , phenix or ffmpeg I always found that the command line gave the program a more  transparent interface . By that I mean it was easier ( at least for me ) to figure out how to do decipher a  manual page  than to go looking for a particular functionality in a GUI window.

No in the case of python ,  I have been writing scripts that take in user input from the command line for a while now. In these scripts , I would generally accept only one input and that would be the first in the sys.argv list. If I had more than one input I would iterate over the input list and try and figure out what the inputs were . Even worse in most cases I would  hard code the order  of inputs into my code ( terrible practise). Fortunately  for me , my discovery of the optparse module has changed all that.

The optparse module is  an object oriented ( dont let that scare you) and super-intuitive way to add command line options to any python scriptSo say you want to add an input file command line switch with the -i attribute , All you have to do is

from optparse import OptionParser

optparse_object =OptionParser()

optparser_object.add_option(“-i”,”–infile”, dest=”infile”,help=”input file for script” , metavar=”[infile.txt]”)

Once you do this you can easily have the module parse the sys.argv list and make sense of it .So you would add the following line

options_object, spillover_options = optparser_object.parse_args()

Then options_object.infile  will have the value of the input option . This is specified by the dest section in the add_option argument list) . The nice thing with the module is that all possibilities can be mapped to the same options_object.infile destination . So for eg I have mapped “-i” and “–infile” to the same destination .Even better is the option to add a help string with the help=”help text” argument . This help is then printed out if the user provides an option that the script cannot handle or if the code specifically calls  the optparser_object.print_help() function.

For a concrete example on how to use the opt_parse module consult the docs or my example code on github .

Code-itch 2009 – Progess report

Almost two years ago I first started writing the codeitch weblog about my code ambitions on wordpress ( codeitch.wordpress.com) .This blog was driven by a strong desire to get better at writing code and importantly building solutions that enabled better science. To do that I had decided to pick up python , Java , javascript and also master Excel and Matlab. Since a new year is always a time for reflections , resolutions and such, I figured I will write down this progress report.

Python: Python has clearly becoming my first choice for tackling any problem. Whether it involved analysing the many trajectories generated by  the channel finding program HOLE , or writing an http based API for Bioscreencast , or an HTTP based web service for Bioscreencastwiki, python has ensured that I can get it done . There is no problem that I dont attempt to solve using python . Python has even ensured that  I jettison a desire to pick up Excel and instead adopt Resolver one , the pythonic spreadsheet instead . My only problem with python is the learning curve to pick up a new API . The dynamic typing sure makes it hard to unravel code by just reading source , something that I was used to doing having leant to code in Java. Further , tools like netbeans and Eclipse with code completion also make it easier to familiarize yourself with a new API  in Java  . In python , I still go about it using the dir builtin function and reading the various forms of python documentation available on the web. In some ways I really cannot wait for something like nbpython to become fully useable.

Java : Since I am still developing code to solve problems for myself or web-centic problems for an apache based webserver . I have almost not written a single line of java code the last year.  Python  has ensured that java takes a backseat.

Javascript : To me javascript is  mainly a UI language , since I regard the browser as the ultimate UI . Since I have yet to get into the UI writing mode , my javascript useage over the last year was restricted to troubleshooting the few glitches we had at Bioscreencast . Again tool support ( like the amazing firebug utility or even netbeans) ensure that I dont abandon picking up javascript in my spare time and definitely plan to use javascript and a browser as my UI of choice  when the need arises.

Excel : Having got on to the python horse, I can hardly justify the headaches that mastering Excel syntaz gave me . And for most simple spreadhsheets I have gotten used to Google spreadsheets.

Matlab: My desire to master matlab was to understand the many algorihtms I encounter in X-ray crystallography. In may ways Python and MatPlotlib have ensured that I can get a lot of that done using python , without the need to learn the Matlab way of things . I have also attended a few Mathematica Seminars and am extremely impressed with the Mathemtica 6 ( and 7) platform . For understanding algotihms and the behavior of functions etc I have been turning to Mathematica and python more than Matlab.

So in summary , python and the pythonic way are what I have  embraced in 2008 and really want to start writing python code that starts to build on this foundation in 2009.I am also really starting to use Mathematica and pick up processing as a java centric visualization platform and hope to write about these in this new year.

Wanna learn python , join comp.lang.python : Regexp matching of fixed digit numbers

One of the fun parts of learning a programming language yourself is when you get stuck at solving a trivial problem . When this happens I generally try to do a few things generally involving searching . I have tabulated these activities and also the apparent value to me , both for my short-term solving of the immediate problem and for my long-term understanding of the subject in addition to improving my tacit knowledge.

Table describing merits of newsgroup memberships

In most cases the above acitivities take almost an hour. When despite all this I dont get an answer I start to get frustrated. Most of my frustrations stem from being unable to define the problem because I dont have a good enough grasp of the vocabulary required to improve my searching and querying.

This is when I resort to what I call my ultimate resource , I get outside human help and  contact a newsgroup In almost 100% of these problems I get a very satisfying answer from these forums . Forums also have the social benefit of expanding my network and importantly creating a lasting record of the solution.

Let me qualify all of this with an actual example . I had to develop a regular expression to parse text from a file and only extract sentences that contained four digit numbers

for eg

I have 1234 dollars left in the bank ( result desired:Match)

I dont have 12345 issues to deal with ( result desired : SKIP )

My initial regular expression was

[python]

p = re.compile(r’\d{4}’)

[/python]

Based on what I knew this would search for “exactly” four digit numbers . But I soon realized that this regexp matched both 1245 and 12345 i.e when you say r’\d{4}’ what you are saying is match exactly a four digit number with no constraints on whats on either side . So all of  “1234” and “12345” and “a1234abcd” or “1234asdf” were matched.

Clearly when you say “exact” in the definition of the regular expression you mean something different from “exact” to a non-savant regexp writer .

I then assumed that this had something to do with greedy vs non-greedy matching ( hence approach  in row 2 of table above)  . This too quickly proved to be a non-starter. I was clearly stuck and had no idea how to  proceed with solving the problem or even go about finding more information .

Thats when I wrote in to the comp.lang.python newsgroup . Within 15 minutes I had a reply . Although it wasnt the right answer , the promptness of the reply was well worth it. A few minutes later I had a few suggestions all of which listed multiple approaches to my problem . I quickly learnt that I had not defined the regexp fully enough.  I had to also codeify the fact that the exact match had to be a separate “word” unto itself .

Three other posts helped me arrive at the perfect solution the regexps

r’\b\d{4}\b’ ” match a four digit  number with a word boundary on either side”

OR more intricate examples that allowed me to learn more about the regexp syntax for eg. expressions containing negative look-behinds , expression notations and non-grouping parenthesis

(r'(?<!\d)\d{4}(?!\d)’) ” four digit number preceded by a non digit and  followed by a non digit)

Or

(?:\D|\b)\d{4}(?:\D|\b) ” four digit number surounded by non digit or word boundaries ”

Clearly almost 20 minutes after my post to the newsgroup , I had not only solved my particular problem but more importantly also considerably increased my knowledge of regular expression behavior and syntax. Most of the replies had not just applied their solutions to my test case , but also had tried to clear my misunderstandings of how regexps worked.  Some posts even provided whole code and novel test cases with results .

Joining an active newsgroup and posing your questions with some effort  always gives you an answer . In all of my experience on these newsgroups , the more effort you spend in definining your problem the more value you gain from the interaction.

I am quite convinced that joining a newsgroup is well worth it to start learning python or any other language or  platform