Pretty documentation and code snippets with snipt.net

Whether I am learning a new module or thinking up a new set of python modules to solve a problem at hand, I use google docs extensively to collect my thoughts and plan out my code.

In my google docs, I very often grab screenshots ( esp of dragged out screen area ) with ⌘-CTRL-Shift-4 and directly paste them into my Google Docs on Google Chrome as images. However, when it comes to inserting snippets of code I wrote into these documents I really like using snipt.net. I first came across snipt.net after Kenneth Love used it in his kickstarter funded getting started with django lessons.

snipt.net copy paste of html formatted syntax colored code is awesome!
snipt.net copy paste of html formatted syntax colored code is awesome!

With snipt.net , any code snippet is a copy-paste away.  I found myself using snipt.net so much that I decided to get the pro-account . To create a snipt, I copy and paste my code into the snipt.net form . Then by playing with a number of formatting options, I can change the way my code is colored and syntax highlighted . Once I am happy with the look . I save and close the snippet. Then I just copy and paste the final snippet view into google docs or make the snipt public and grab the javascript code and paste it into the Text compose view in WordPress.

For example here is some pretty,formatted python code output rendered by snipt.net javascript.

By default all snippets in paid accounts are private. Both public and private posts can be tagged, given a title and an optional description. The public posts can be directly published with a default built in template allowing for very quick “code focussed blogging”. These posts have a comments section that integrates with Facebook, Twitter, Google and Disqus logins.

A code snippet "blog post" on snipt.net.
A code snippet “blog post” on snipt.net.

Although I dont use its blogging tools much, snipt.net has made inserting syntax colored code into my Google Docs and blog posts such as this one super easy!.

If you spend any time on the shell command line use “z” it will save you tonnes of time!

I am not for using superlatives to attract attention. But I cannot but emphasize how amazing this shell add in has been for my daily activities.

I do spend a lot of time on the command line changing between directories. My typical directory path will read something like this

cd /home/hari/data_processing/cro_data_4_14_23/p53_p212121/compound_id_4642222/final_procesing

Now imagine if you could just cd to that directory by matching against a keyword that is unique to that directory. I could just say

z 2222 TAB

And bing! It “cd’s” you into that directory!

THAT is the amazing power of “z”

This and a few other tips like this came from a talk on Tools for Web developers from +Paul Irish

Screen shot 2013-06-23 at 5.56.36 PM

Its all coming together really well : Google I/O keynote

Alex_Washburn_Wired_cc_Larry_Page_Google_io_2013
What Google did at the I/O keynote was truly amazing. Put another way , I never really appreciated what convergence really meant till the keynote was done. I was convinced, THIS is THE most tightly knit, focussed and convergent company that exists right now. YES, Google presented a unifying vision at its I/O keynote , a vision more crystallized and consistent than Apple, Facebook, Twitter , Netflix , Microsoft, Sony or Amazon have laid out in the past 5 years.

When Larry Page took the stage at the end of the keynote. It became very clear that HE was the person behind the amazing coming-together of all of Googles disparate entities into one congruous whole. He was the person who led Google from the company that reluctantly dabbled into social networking to a company that built a powerful social layer that integrates seamlessly across its entire ecosystem.

It takes guts for a search company to keynote a section titled “The end of search”, it takes serious guts for a company to welcome competition in every sphere of its activity by continuing to build open systems, it takes even more guts for a company to invest time and energy into so many activities because they can , regardless of the profit motives behind it.

Think about it, this is a company that stitched together every LANDSAT image taken since the dawn of the program and made it available free for us all to use, This is a company that sent hikers with imager-backpacks on every trail in the US,sent underwater imagers into coral reefs and the amazon and uploaded all the data to the web, free for us to use and benefit from , because they CAN. This is a company that added a streetview layer on top of a majority of maps they publish , an image of every street they have driven down. This is a company that is creating a driverless car. This is a company that actually thinks of what it means to have an inter-planetary internet and uses the technology to bring the internet to remote reaches of our own planet.

And even though many media stories on Google , including yesterdays New York Times article follow the narrative that Google did something because of Apple, Facebook or . It was even more clear after this weeks keynote that while Google did get pushed into many areas to compete with others , it has now more than hit its stride. Every area that Google entered, it has built products or solutions infused with its DNA through and through. DNA that sets them apart from every other player in the field. A DNA which has openness at its very core!. A DNA which builds something just because it should be possible!.A DNA which embodies what JFK stated in his famous speech about going to the moon , Google chooses to tackle all these problems and more not because any of them are technologically feasible when they started , not because any of them are easy..but because they are hard!

There is something to be said for launching apps on a competitor platform simultaneously with a platform you own and control ( the Hangout app they launched last week ). I could download a books app that allowed me to load every non DRM book I own onto a Google app and it syncs automatically across all devices, regardless of which company makes that device, once again a testament to Googles openness.

While many criticize Google for building chrome , forking java , abandoning XMPP , JPG , H.264 or some other web standard, Google once more sent the message that they will do everything in their power to enhance the open web while relentlessly working to keep it moving forward , making it faster and better and YES more open! Granted many of these standards are Googles new “standards” , granted it drives a wedge in some otherwise open system used by the web..but there is no denying that the new proposal is always better and always open.

The message that came through loud and clear is that the vision that Larry and Sergey had for Google is stronger than ever. The big plans are only gettting bigger and Yes most importantly OPEN is winning. Its winning because its what makes Google, Google and allows them to make products that continue to fill ever growing technology niches. OPEN is the glue that binds it all together into one congruous whole that will continue to amaze us in the years to come.

Image credit: Alex Washburn / Wired ( creative commons )

Customizing you Django user admin with Inline sections

Screen Shot 2013-05-12 at 3.56.43 PM

I was using django-userena for user management on a django app. Userena is a django app that works with “django.contrib.auth.user” to provide additional features such activation, password resets over email and messaging in addition to signin and signups.

With Userena the django Auth admin site had a number of “Inline” management sections that incorporated the information coming in from the Userena app relevant to the particular user.

The sections were:
Username:Password
Personal Info
Permissions
Important Dates
Userena Registrations

I wanted to add my Custom profile to the Bottom of that list. My custom profile , otherwise also known as the one you set as the “AUTH_PROFILE_MODULE”

AUTH_PROFILE_MODULE = “accounts.MyCustomProfile”

How does one do that?

Well basically you want to extend the “custom” admin interface . The relevant django doc for all things ModelAdmin is located here.

My model which I want to add in: is

from django.db import models
from django.contrib.auth.models import User
from django.utils.translation import ugettext as _
from userena.models import UserenaBaseProfile


# Create your models here.

class MyCustomProfile(UserenaBaseProfile):
    """The Userena User profile"""
    user = models.OneToOneField(User,
        unique=True,
        verbose_name=_('user'),
        related_name='my_profile')
    bio = models.CharField(max_length=1000,default="",blank=True)
    # Make this field not editable because it controls access to uid from legacy db
    my_legacy_user_object = models.SmallIntegerField(null=True,editable=False,unique=True)

    def __unicode__(self):
        return self.bio

To accomplish the inline display of this model along with the rest of the stuff userena already put into the admin page for the user I had to add this as my admin.py for the accounts app.

__author__ = 'hari'
from django.contrib import admin
from .models import MyCustomProfile
from userena.utils import get_user_model,get_profile_model
from django.contrib.auth.admin import UserAdmin
from django.contrib.auth.models import User
from userena.admin import UserenaAdmin,UserenaSignupInline

class MyCustomProfileAdmin(admin.ModelAdmin):
    list_display = ["user" ,"bio","my_legacy_user_object"]
    search_fields = ["user__first_name","user__last_name","my_legacy_user_object"]

#docs from https://github.com/bread-and-pepper/django-userena/blob/master/docs/faq.rst
admin.site.unregister(get_profile_model())
admin.site.register(MyCustomProfile,MyCustomProfileAdmin)

class MyCustomProfileAdminInline(admin.StackedInline):
    model = MyCustomProfile


class MyCustomProfileAddedAdmin(UserenaAdmin):
    inlines = [UserenaSignupInline,MyCustomProfileAdminInline]

admin.site.unregister(get_user_model())
admin.site.register(get_user_model(),MyCustomProfileAddedAdmin)

The result is what we get nicely here

Getting Solr 4.0 running on Amazon EC2 for django-haystack

After almost two years this is my attempt at getting back to blogging. These instructions are not too detailed. But I spent some time trying to get Solr 4 .0 setup to serve up my django-haystack search results and figured I would document the two hurdles I faced.

I have been running apache-solr for my django-haystack search on my home Ubuntu Linux box. The whole setup was working great but the requirement to keep this box powered on all day , coupled with the noisy fans on the box made me decide to switch to hosting my solr instance on the cloud.
After much “googling” I couldnt find anything warning against running apache Solr on an Amazon micro instance so I decided to give it a try since it was going to be a dev instance anyways.

What I wanted to do : Get a full Solr 4.0 setup which would index my django dev database and serve up search results. I am using a T1.micro instance on the Amazon EC2 cloud

Step 0 : Get a new Amazon micro instance. I am using an Ubuntu 12.04 LTS 64 bit instance. I got all the required packages and java on there and installed tomcat6 ( sorry this step is deficient in details)

Step 1 : Get and install apache-solr . I used the 4.0 (dubbed Solrcloud) release which is quite different from earlier solr releases. From what I understand Solr 4.0 has a better support for distributed indexes .

wget http://apache.mirrors.tds.net/lucene/solr/4.0.0
tar -zxvf apache-solr-4.0.0.tgz
cd apache-solr-4.0.0

Step 2 : Get the schema.xml from django to play nice with the new solr 4.0.
Solr 4.0 has changed a little bit how it does things. While the start.jar is in the same place the conf and schema.xml directory are now in a few places since solr now has split up the data blocks into collection directories. For now I decided to co-opt the collection1 directory to serve up my django index.

The schema.xml from django ( gotten by running “python manage.py build_solr_schema”) now needs to be placed into the example/solr/collections/conf directory

cp schema.xml $HOME/apache-solr-4.0.0/example/solr/collections/conf

Edit this schema.xml to add the reserved __versions__ field name into the fields section.Since Solr’s updatelog is “ON” by default , the __version__ field name is required in the configuration. Alternatively you could turn that “OFF” by editing solrconfig.xml. I chose to leave that setting as it is and instead added this now required field name to the schema.xml. Look for the “fields” field in the django generated schema.xml and add the lines shown below anywhere inside that block. Here is what mine looks like after I added the “__version__ field name.

<!-- general -->
<field name="id" type="string" indexed="true" stored="true" multiValued="false" required="true"/>
<field name="django_ct" type="string" indexed="true" stored="true" multiValued="false"/>
<field name="django_id" type="string" indexed="true" stored="true" multiValued="false"/>
<!-- added this field for solrcloud , to play friendly with updatelog -->
<field name="_version_" type="long" indexed="true" stored="true"/>
<dynamicField name="*_i"  type="int"    indexed="true"  stored="true"/>
<dynamicField name="*_s"  type="string"  indexed="true"  stored="true"/>
<dynamicField name="*_l"  type="long"   indexed="true"  stored="true"/>
<dynamicField name="*_t"  type="text_en"    indexed="true"  stored="true"/>
.
.
.

Once this was done. I could easily start my solr jetty server and then ask django to rebuild the index

Step 4 : Start the solr jetty server
In directory apache-solr-4.0.0/example
java -jar start.jar

Step 5 : Rebuild the index

python manage.py rebuild_index

And everything is up and running.
Edit: I had some trouble setting up a start and stop script with ubuntu that would start the solr process and stop it on boot and add it to the default run level . Finally I followed the clear instructions at this blog entry and the script described worked just great.