Django abstracts most of the actions you would be doing with the Database. What it doesn’t abstracts, and doesn’t try to abstract is the Database modelling part. This is a quick tutorial describing to how model your data in Django models.py, and how to access and modify them.

Consider a hypothetical HR department, which wants you to build an application to track and manage their processes. They have employees who work for a department, contractors who work for multiple department. Let’s see how you you would do that in Django.

from django.db import models

class Employee(models.Model):
    name = models.CharField(max_length = 100)
    department = models.ForeignKey("Department")

class Department(models.Model):
    name = models.CharField(max_length = 100)

class EmployeeHistory(models.Model):
    employee = models.OneToOneField(Employee)
    date_joined = models.DateField()
    marital_status = models.BooleanField()

class Contactor(models.Model):
    name = models.CharField(max_length = 100)
    departments = models.ManyToManyField(Department)

Let’s see the type of relationship we created here.

An Employee has a Many-to-one relationship with Department, (i.e. One department will have many Employee, but one employee will have a single departments.)

So Employee has a field department = models.ForeignKey("Department"). See that the ForeignKey field was added on the class which has in many. In database, this creates a FK from in Employee table which refernces Departments.

A Contractor has many-to-many relationships with Department, so it has a ManyToMany field. This field can be created on either classes. At a database level this creates a new table which has a FK to both the tables.

A Employee has one-to-one relationship with EmployeeHistory. The thing to note here is that whatever we could do with OneToOneField, we can do by including the fields in the Other Class directly. However when there are fields which are only rarely needed with a given model, it is useful to separate them via a one-to-one field. The one to one field can be on either class.

Accessing objects.

Getting a specific employee.

Employee.objects.get(pk = someval)
Employee.objects.get(name= someval)

Given an department get all employees.

department.employee_set.all()

The department gets an attribute name <FKClass>_set.

Given an employee, get all colleagues.

employee.department.employee_set.objects.all()

To get siblings, get parents, and get all children.

Given an department, get number of employees.

department.employee_set.all().count()

Ok so the HR department bosses are happy with what they see, and ask you to track this data, who is the manager of each employee, who is the HOD of each department and when did each employee took leave.

from django.db import models

class Employee(models.Model):
    name = models.CharField(max_length = 100)
    manager = models.ForeignKey("self", blank = True, null = True)
    department = models.ForeignKey("Department")

class Department(models.Model):
    hod = models.ForeignKey("Employee")
    name = models.CharField(max_length = 100)

class EmployeeHistory(models.Model):
    employee = models.OneToOneField(Employee)
    date_joined = models.DateField()
    marital_status = models.BooleanField()

class Contactors(models.Model):
    name = models.CharField(max_length = 100)
    departments = models.ManyToManyField(Department)


class EmployeeLeave(models.Model):
    leave_taken = models.DateField()
    employee = models.ForeignKey(Employee)

So we now have new fields hod = models.OneToOneField("Employee") in department and manager = models.ForeignKey("self", blank = True, null = True) and a new model EmployeeLeave. Let su see the new realtions,

As one Department will have one Employee as hod, and Employee can head at max one Department, we have a one to one relationship
As many Employee will report to another Employee, so we have a FK on Employee, referencing self. As Employee will take many EmployeeLeave, EmployeeLeave has a FK to Epployee

Let us see some queries.

Get all leaves taken by an employee this year

import datetime
today = datetime.date.today()
start_of_year = datetime.datetime(today.year, 1, 1)
leaves_taken = employee.employeeleave_set.filter(leave_taken__gt=start_of_year, leave_taken__lt = today)

Get a list of all HODs.

#As there are going to be as many Employees as departments
departments = Department.objects.all()
employees = [department.hod for department in departments]

Get a list of all departments a contractor works for.

contractor.department_set.all()

Get a list of all contractor who work for a department.

department.contractor_set.all()

Note that each side of a many to many relationship get a Manager.

List of all managers in a given department.

Todo#(Can't think of any one query way to do this. If you know, let me know   )

List of all leaves taken in a given department.

EmployeeLeave.objects.filter(employee__department = department)

None that we used double underscores __ to do a filtering on a field across Entities.

List of all employees which joined a given department this year.

Employee.objects.filter(department = department, employment_history__date_joined__gte=start_of_year, )

Note that we used a double underscore __ twice, first to go across entities, and then to define the type of filter(__gte). Also we specified two filter conditions, so they were ANDed together.

List all employees which either report too a given employee, or joined before him.

Employee.objects.filter(Q(manager = employee)|Q(employee_history__date_joined__lt = employee.employee_history.date_joined))

Note the new construct Q, they are used to specify complex boolean operations. Here we used the | (or operator) to specify or condition.

Do you subscribe to our rss feed? The new feed has full text feed and many other goodness. So make sure you are subscribed to the new feed with extra fortified vitamins. Subscribe now.

Related posts:

1. Django aggregation tutorial
2. The Rails and Django models layer Rosseta stone
3. Dynamic forms with Django

There are some tools and apps which we use with almost all apps we write, and in particular which, we used for dinette. Here they are broken into useful during development, and (also) useful post development.

During Development

1. Ipython and ipdb
2. South
3. Django test utils
4. Django extensions
5. Django debug toolbar

Ipython and ipdb

Ipython is a enhanced shell for python. Ipdb similarly add extra capacity to the builtin pdb debugger. It is extremely convenient to drop into a ipython shell right where a complex piece of code is being hit.

from IPython.Shell import IPShellEmbed
ipython = IPShellEmbed()
ipython()

Of course this does not allow you to step through, so if you want to step through your code, you need to do

import ipdb
ipdb.set_trace()

Of course, you can just do pdb.set_trace(), but with tab completion and syntax highlighting, using ipdb is a no brainer.

South

South adds schema migration to Django. It has lot of advanced options, but just three command will get you along way.

Convert a normal app to use the south way.

./manage.py convert_to_south

Once you have made any changes to your apps model, write a new migration which can update the database to latest models.py

./manage.py startmigration appname nameofmigration --auto

Once you have written a migration, write a command to update the db.

./manage.py migrate appanme 

Django test utils

When you are running tests, a huge time sink is the time spent dropping and recreating tests. You might add just a single test, and the default Django testrunner will spend a huge time recreating the database. Enter manage.py quicktest which reuses databases between test runs. One of the side effects of this is that if you modify models.py between test runs you should manually recreate the database.

The tests (or whatever little of it exists) for Dinette were written using testutils’ testmaker.

Django extensions

Django extensions, (earlier Django command extensions) is a app which adds extra commands to manage.py. It has lot of goodies, but just ./manage.py shell_plus makes it worthwhile. (It imports all the models from all the apps automatically).

Get the full list at Github

Django debug toolbar

I do not personally use it but some people on our team swear by it.

After development

1. Django-pagination
2. sorl-thumbnails
3. django-compressor
4. Haystack

Django-pagination

I have decided on a standard way to build apps which need pagination. a. Build apps without pagination. b. The night before release spend an hour and add pagination to all pages. Really this is all you need

{% load pagination tags %}
{% autopaginate queryset %}
....
{% paginate %}

There is no change required to the views.

sorl-thumbnails

To thumbnail an image.

{% thumbnail image size %}

“Things should be as simple as possible, but not simpler.”

django-compressor

During development we work with multiple unminified js and css files. Django-compressor takes care of minifying and compressing it when we deploy. All we need to do is,

{% compress css %}
...
all css files
....
{% endcompress %}


{% compress js %}
...
all js files
....
{% endcompress %}

Haystack

Haystack is “modular search for Django”. It make writing search views as easy as writing the admin. You declaratively tell what fields you want to search on, and Haystack can take care of creating and updating the index, and providing a generic view to handle the search. Check a search view written using Haystack http://uswaretech.com/forum/search/?q=django

To everyone who contributed to these apps, thanks. Django development won’t be the same without these great tools. This post was inspired by Kevin Fricovsky’s post post, The apps that power Django-Mingus and is stolen about 50% from there. Thanks.

We build Amazing web apps, and we can build it for you. Contact us today.

Related posts:

1. Django for a Rails Developer
2. Doing things with Django models – aka – Django models tutorial
3. How to build a Facebook app in Django

Foss.in is without doubt India’s largest FOSS technology conference. Lakshman gave a talk today on “Python metaclasses and how Django uses them”. Here are the slides from that talk.

[Edit] Some reactions,
http://twitter.com/jaideep2588/status/6295483833
http://twitter.com/kunalbharati/status/6296572939

And the talk images, http://twitpic.com/rxhn7

You should follow us on twitter and Subscribe to our blog

Related posts:

1. The magic of metaclasses in Python
2. Beginning python
3. Better Python package management using source and version control systems

This is not yet another Django vs Rails blog post. It is a compilation of notes I made working with Django after having worked on Rails for years.

In this post I want to give a brief introduction to Django project layout from a Rails developer point of view, on what is there, what is not there and where to look for things. It should help a rails developer working on django be able to find the necessary files and underatnd the layout of the project files.

Once you have both the frameworks installed on your system you can create the projects using the commands

# creating a rails project
rails rails_project
# creating a Django project
django-admin.py startproject django_project

Lets look at the files and structure created by the respective frameworks

#rails_project
README
Rakefile
app/
    controllers/
        application_controller.rb
    helpers/
        application_helper.rb
    models/
    views/
        layouts/
config/
    boot.rb
    database.yml
    environment.rb
    environments/
        development.rb
        production.rb
        test.rb
    initializers/
        backtrace_silencers.rb
        inflections.rb
        mime_types.rb
        new_rails_defaults.rb
        session_store.rb
    locales/
        en.yml
    routes.rb
db/
    seeds.rb
doc/
    README_FOR_APP
lib/
    tasks/
log/
    development.log
    production.log
    server.log
    test.log
public/
    404.html
    422.html
    500.html
    favicon.ico
    images/
        rails.png
    index.html
    javascripts/
        application.js
        controls.js
        dragdrop.js
        effects.js
        prototype.js
    robots.txt
    stylesheets/
script/
    about
    console
    dbconsole
    destroy
    generate
    performance/
        benchmarker
        profiler
    plugin
    runner
    server
test/
    fixtures/
    functional/
    integration/
    performance/
        browsing_test.rb
    test_helper.rb
    unit/
tmp/
    cache/
    pids/
    sessions/
    sockets/
vendor/
    plugins/

that is huge….

lets look at the django project files

# django_project
__init__.py
manage.py
settings.py
urls.py

far lesser when compared to the rails project.

In fact a rails project comes with everything a web application needs. When I say everything I mean everything….. base application, routing, database configuration, development, test and production environment specific configurations and their respective log files, javascript, test, some standard html files and some helpful scripts for developing.

Why then does a Django project have so less number of files? It has got to do with the Django’s philosophy and the concept of applications. The django project is not complete without the application, so lets create a application inside the project and have a look at the structure

django-admin.py startapp app

# django_project
__init__.py
app/
    __init__.py
    models.py
    tests.py
    views.py
manage.py
settings.py
urls.py

even after including this, the number of files is still less than the rails project.

Lets break it down and relate both the frameworks.

and lets have a look at the other things in rails_project

• Logging, unders the logs directory
• Some default html files for some standard http errors, under the public directory
• Rails has very good support for testing, for that bunch of files under tests
• Vendor/Plugin, place for some third party plugins/applications.

Missing pieces in Django (for a rails developer)

• Scaffold magic
• Generate commands
• Migrations

and what is Django is providing by default? Sorry no extra files in the project; but you will get an authentication system and Django’s killer feature, admin, just by modifying your ‘INSTALLED_APPS’ in your settings. It is similar to the plugins feature in Rails, Django’s philosophy of resusable apps helps you in getting any particular functionality into your project.

Following is a list of what I like in Django (and associated apps):

Following are the questions that I keep getting,

Why not create a ‘templates’ directoy and a ‘base.html’ either in project’s directory or in the apps’s directory, because creating the same templates directory and same base.html for every project is not DRY ?

let me know via comments, if you have any answers

program used for listing the files in a directory

import os
import sys

try:
    directory = sys.argv[1]
except IndexError:
    directory = os.path.dirname(os.path.abspath(__file__))

def r_list_dir(directory, indent=0):
    dir_files = sorted([os.path.join(directory, file_name) for file_name in  os.listdir(directory)])
    for item in dir_files:
        if os.path.isdir(item):
            print " " * indent + os.path.split(item)[1] + '/'
            r_list_dir(item, indent+4)
        else:
            print " " * indent + os.path.split(item)[1]

r_list_dir(directory)

Related posts:

1. Rails and Django commands : comparison and conversion
2. The Rails and Django models layer Rosseta stone
3. Tools of Pro Django developer – aka What powers dinette and almost every app we write.

Django has three builtin template loaders which are used to get the templates for rendering.

TEMPLATE_LOADERS = (
    'django.template.loaders.filesystem.load_template_source',
    'django.template.loaders.app_directories.load_template_source',
#     'django.template.loaders.eggs.load_template_source',
)

Writing your template loader is a awfuly easy. It is a callable which

1. Returns a tuple of (openfile, filename) if it can find the template.
2. Raise TemplateDoesNotExist if the templates cannot be found.

The simplest template loader can be

from django.template import TemplateDoesNotExist
def load_template_source(template_name, template_dirs=None):
        try:
            return open(template_name).read(), template_name
        except IOError:
            raise TemplateDoesNotExist, template_name

if you put this to your template loaders directory,

TEMPLATE_LOADERS = (
    'django.template.loaders.filesystem.load_template_source',
    'django.template.loaders.app_directories.load_template_source',
    'dgrid.load_template_source'
#     'django.template.loaders.eggs.load_template_source',

You can do access a template using an absolute path.

In [6]: get_template('/home/shabda/abc.txt')
Out[6]: <django.template.Template object at 0x91c860c>

In [7]: temp = get_template('/home/shabda/abc.txt')

In [8]: temp
Out[8]: <django.template.Template object at 0x9358a0c>

In [9]: temp.render
Out[9]: <bound method Template.render of <django.template.Template object at 0x9358a0c>>

This is the first in the series of short and sweet Django posts we are going to do. You are interested, right. Do follow us on twitter or subscribe to our feed.

We build Amazing We Apps. Talk to us or email us at sales@uswaretech.com .

Related posts:

1. A response to Dropping Django

Slides and code from my talk at twincling.

Related posts:

1. Python metaclasses and how Django uses them
2. Python Wrapper on Bing API
3. Better Python package management using source and version control systems

(If you are in a hurry, here is the fun part.

A few days ago, I was working with a nullable field, which wasn’t behaving as I expected. So I started a shell, and see how nulls compare.

In [1]: None

In [2]: None > 10

Out[2]: False

In [3]: None < 10

Out[3]: True

In [4]: None == None

Out[4]: True

So python is obviously broken, next steps, see the same thing in some other language. Lets try some other language, Javascript maybe?

//#Javascript
>>> null
null
>>> null == 10
false
>>> null > 10
false
>>> null < 10
true
>>> null == null
true

Ruby?

irb(main):001:0> nil
=> nil
irb(main):002:0> nil > 10
NoMethodError: undefined method >' for nil:NilClass
  from (irb):2
  from :0
irb(main):003:0> 10 > nil
ArgumentError: comparison of Fixnum with nil failed
  from (irb):3:in>'
  from (irb):3
  from :0
irb(main):004:0> 10 < nil
ArgumentError: comparison of Fixnum with nil failed
  from (irb):4:in ` 10 == nil
=> false

Comparision of how null behaves in different languages for

• null > 10
• null < 10
• null == 10
• null == null

All these are high level languages, and with exception of Java/Haskell/Sql are dynamically typed, so why is there this much difference in how None/null/nil as a type is handled?

As you can tell, I am very new to some of these languages, so if I have got your favorite language wrong. Please let me know in the comments and I will fix it.

Related posts:

1. The Rails and Django models layer Rosseta stone

Thanks to awesome django community, there is plenty of open source django code around.

These packages get updated quite often and if you use it often like we do, you’d have possibly realized the need to manage these packages better.

Thankfully, all python ever needs is the source, and all you need to do is to place the source in the python path.

Most projects use Distributed Version Control Systems like Mercurial and Git, and they come locally with the entire history of the source which provides a lot of control to use any version of the code. For code that we use often, like django packages, using the source from a version control system seems to be the best way to manage.

In order to install django-registration, a popular django app by James Bennett, you need to:

Clone the repository:

$ hg clone https://becomingguru@bitbucket.org/ubernostrum/django-registration/ django_registration

Create a symbolic link in the python path:

$ sudo ln -fs ~/django_registration/registration /usr/lib/python2.6/dist-packages/django_registration

Thats it. django_registration is now on the python path, courtesy of symbollic link, that links the django-registration source to python path, in this case the dist-packages/ folder; thus it will be possible to import this app within the python environment.

You now have the latest version of the source in the folder ~/django_registration, which you can check, and modify. With an editor/IDE that has go to source option, you can browse the source by using a simple short cut.(which may not have been simple if the code were is some egg file in a less well known folder)

Because of cloning of the mercurial repository, you now have access to all the revisions of the application. So you can easily update, or change to other versions.

To update to a newer version when one exists:

$ hg pull -u

To move to any particular revision:

$ hg update -r200

To check the tags:

$ hg tags

tip                              221:b360801eae96
v0.7                             205:d073602dc103
v0.6                             154:e263c551ef7b
v0.5                             139:dc2bf754aa94
v0.4                             110:4d2a725d8c18
v0.3                              75:f41e8a239016
v0.2                              58:e5110bb8d48a
v0.1                              42:d28e5a770ff8

If you want to update to a recent release, say 0.7,

 $  hg update v0.7
7 files updated, 0 files merged, 0 files removed, 0 files unresolved

The same is also applicable while dealing with SVN repositories; alebit some of the revision changes go thro the network to the central server.

In order to install and work with django trunk, check out the trunk and add a symlink, as before

$ svn co http://code.djangoproject.com/svn/django/trunk/ django-trunk
$ cd django-trunk/django
$ sudo ln -fs . /usr/lib/python2.6/dist-packages/django-extensions

You will now be able to access the latest version of django:

$ python
>>> import django
>>> django.get_version()
u'1.1 beta 1 SVN-11092'

To use any other version of django you can do some SVN manipulation, as following:

#To update to a latest version
svn up
At revision 11092.

#To update to a particular revision
svn update -r11090
At revision 11090.

#To update to a last committed version
svn up -r'COMMITTED'
At revision 11079.

#Change to a tagged version - to release 1.02
svn switch http://code.djangoproject.com/svn/django/tags/releases/1.0.2/

#Change to any branch
svn switch http://code.djangoproject.com/browser/django/branches/soc2009

svn switch commands talk to the central servers and can be time consuming.

Possible shortcomings of using source and symbollic links:

Setuptools distribution adds to the python standard library Distutils module, by adding a lot of functionality, among others that stores the package meta data like the version, location of download. If you perform

easy_install django

after installing it by source as above, easy_install again downloads the latest released version of django 1.02, listed in the Python Package Index, as it hasn’t been informed of django installation.

If you are not going to use easy_install at all, this isn’t a problem; but it is good to also update setup tools that a version of this application exists, if you at all need. You can then use the pkg_resources module installed with setup tools distibution, to query to find all the installed applications, from python.

If the application contains a setup.py file, you could manually inform setup tools to update using the following command

python setup.py develop

The workflow

To install packages, in a way with high control following needs to be done:

* Checkout the repository
* Add a symbollic link to the repository
* Update setup tools of this installation.

Enter PIP

PIP is a package management tool that does the workflow we need, well. It automatically checks out from the repos and runs python manage.py develop and updates few other resources, so that python package management tools in the system are well aware of the installed packages.

If you are checking out the source and creating symbolic links, it is kind of awkward to have to create symbolic links all the time; Pip automates that. Pip also updates the setuptools with the metadata of the application, so that next time if you ask the same application to be installed, via easy_install or pip, it doesn’t download and install again.

So the following command:

pip install -e hg+http://bitbucket.org/ubernostrum/django-registration/#egg=django_registration

• Clones the mercurial repository django-registration locally, into Env/src/django_registration
• Creates the required symbollic links in the python path, in this case in the site-packages folder
• Updates the local package management tools by setup tools of the new package.

-e option tells pip to install the package as editable, that is to leave the repository undeleted. The hg+ and egg=django_registration indicate the repository and the local folder to install to.

Pip works well with git, SVN, Hg an Bzr. Provided you inform the pip of the repository and the folder name, pip completes the work-flow.

Since the source is present in the Env/src folder, along with the SVN, Git or Hg data, one can modify the versions using version control systems, as illustrated in some of the commands above

Multiple environments

Most often the requirement will be just to update the package into the latest version, for which simple revision control update commands seem good enough. Its good to also mess around with the code changing into different versions and trying different things.

But if you seem to be changing of the versions too often, for compatibility between different environments, you might want to using virtualenv, that isolates a python working environment. virtualenvwrapper adds a bash wrapper to use virtualenv conviniently.

Do you also want your app development managed well? Look us up

Related posts:

1. Python metaclasses and how Django uses them
2. Python Wrapper on Bing API
3. Beginning python

If you used Django for any length of time, you would have come across the login_required decorator. You write @login_required before a view, and it magically becomes accessible only to authenticated users.

Decorators were introduced in python 2.4. PEP 318 is the PEP describing it. At the simplest decorators are nothing but callables returning other callables, and the decorator syntax @decorator is nothing but foo = bar(foo), where both bar and foo are callables.

Let us see some decorators in action.

In [1]: def good_function():
   ...:     print 'I am a good function'
   ...:     
   ...:     

In [2]: def decorator(orig_func):
   ...:     def bad_func():
   ...:         print 'I am a bad function'
   ...:     return bad_func
   ...: 

In [3]: good_function = decorator(good_function)

In [4]: good_function()
I am a bad function

In [5]: @decorator
   ....: def good_function2():
   ....:     print 'I am a good function'
   ....:     
   ....:     

In [6]: good_function2()
I am a bad function

So you can see that the decorated function depended only on the value of the function returned by the decorator.

Here we discarded the function we were passed. Any useful decorator, decorates the original function, so it would use the original function. Let us see an actual decorator which may be useful.

def is_val_positive_deco(orig_func):
        def temp_func(val):
                if val < 0:
                        return 0
                else:
                        return orig_func(val)
        return temp_func

@is_val_positive_deco
def sqrt(val):
        import math
        return math.pow(val, (1.0/2))

print sqrt(-1)
print sqrt(4)

Here we defined an decorator is_val_positive_deco which will make functions return 0, if the argument passed is negative. We can use this decorator to guard against MathErrors.

Class based decorators

Decorators are just callables, and hence can be a class which has __call__ method. Sometimes they are easier to understand and reason about(than decorators written as functions). Lets see one,

class LogArgumentsDecorator(object):
        def __init__(self, orig_func):
                self.orig_func = orig_func
                print 'started logging: %s' % orig_func.__name__

        def __call__(self,  *args, **kwargs):
                print 'args: %s' % args
                print 'kwargs:%s'% kwargs
                return self.orig_func(*args, **kwargs)

@LogArgumentsDecorator      
def sum_of_squares(a, b):
        return a*a + b*b

print sum_of_squares(3, b=4)

This outputs,

started logging: sum_of_squares
args: 3
kwargs:{'b': 4}
25

Lets see what happens when we do,

@LogArgumentsDecorator      
def sum_of_squares(a, b):

This is equivalent to

sum_of_squares = LogArgumentsDecorator(sum_of_squares)

At this point, we created a new LogArgumentsDecorator object, so LogArgumentsDecorator.__init__(self, sum_of_squares) gets called. Hence print 'started logging: %s' % orig_func.__name__ line is executed.

When we do sum_of_squares(...)

LogArgumentsDecorator.__call__ gets called with the passed values, where we print the arguments and call the original function.

Here is one more example of a class based decorator. Here we rewrite(sort of, anyway) Django’s login_required decorator as a class based decorator.

class LoginRequiredDecorator(object):
def __init__(self, orig_func):
    self.orig_func = orig_func

def __call__(self, request, *args, **kwargs):
    if request.user.is_authenticated():
        self.orig_func(request, *args, **kwargs)
    else:
        return HttpResponseRedirect(reverse('...'))

Similar to above, it takes the view function, during __init__. When the request comes, __call__ makes sure that user is authenticated, and takes action appropriately.

Resources * Decorator Library * Pep 318

Subscribe to The Uswaretech Blog where we will be writing the next part of this tutorial explaining more complex decorators, decorators which take arguments, decorator factories and more. Or follow us on twitter.

Related posts:

1. Understanding DateTime, tzinfo, timedelta & TimeZone Conversions in python
2. Dynamic forms with Django
3. Aren’t django signals a little like comefrom?

Adrian Holovaty is the co-creator of Django, author of the Django book and is currently the BDFL of Django along with Jacob Kaplan-Moss. He studied journalism at University of Missouri–Columbia and has worked with many news sites including Lawrence Journal World and Washington Post. He currently works at EveryBlock, a startup he founded.

Shabda: Would you tell a little about yourself? You majored in Journalism, how did you move to Programming?

Adrian: Sure! I have a dual background in journalism and computer science. I intended to get a degree in journalism and a minor in computer science, but things got a little off track. I heard from a professional colleague that a certain Web site would probably hire me if I graduated a semester early, so I hurriedly dropped my CS minor and got special permission to graduate early — but once the time came to ask for that job, the company was in the middle of a hiring freeze and didn’t have a position for me. So, in hindsight, I should’ve stuck with the minor, but things ended up working out OK.

Because journalism and computer science don’t normally go together, I’ve had some success in this silly little niche of employing Web development in news organizations — “journalism via computer programming.” Professionally, I’ve always worked as a Web developer at news organizations, up until my current gig, which is to run EveryBlock. But in its own way, it’s a news organization as well.

Aside from all of that, I’m quite into music, particularly gypsy jazz, which is the music of Django Reinhardt (hence the name of the Web framework). I just recently attended a week-long gypsy jazz guitar camp in Massachusetts and had a tremendous time staying up until early hours of the morning jamming with great musicians from various parts of the world. If money were no object, I would love to play music full time. In the meantime, I’ll settle for posting YouTube videos of my guitar playing, for an audience that seems to be comprised mostly of 16-year-old boys who constantly pester me for transcriptions of my arrangements.

I live with my wife in the beautiful city of Chicago.

Shabda: What are your current roles and responsibilities at Django?

Adrian: Well, I co-created Django back in 2003 with my friend Simon Willison while we were working together. Now, along with Jacob Kaplan-Moss, I’m Benevolent Dictator For Life of the project — a title we shamelessly ripped off from Guido/Python. This means that I have a hand in all sorts of things, from high-level API design to checking in patches and fixing bugs. I’ve touched probably every bit of the framework over the years, including implementing the initial ORM (back when it was a code generator!), pair-programming the original Django template system and URLconf/view framework with Simon and building the original admin application with design help from Wilson Miner.

One of the things I enjoy the most is writing documentation — both because I like technical writing and because I cannot stand bad open-source documentation.

Shabda: Can you describe the philosophy behind Django. What are its overarching goals?

Adrian: A couple of years ago, I wrote the “Design philosophies” document, which sums up the main points nicely.

Generally, the goal is to make Web development fast, fun and easy for the developer, while keeping performance as fast as possible and code as easy to understand as possible.

Shabda: How did the move at WorldOnline from PHP to Python happen? Why did you start from scratch (which finally lead to creation of Django), instead of using something like Zope?

Adrian: Simon and I had been big fans of Mark Pilgrim (and still are!), and we’d read his online book “Dive Into Python.” This was around the same time that the PHP “infrastructure” I’d developed had begun to feel really, really crufty, so…one day we just decided to start using Python! It’s quite nice working at a small organization with a very loose management structure; our boss, Rob Curley, was cool enough to let the developers themselves decide which technologies to use, as long as the work got done. “I don’t care how the sausage is made,” he always used to say.

We looked at the existing Python Web frameworks/libraries, but none of them felt 100% right to us. Simon and I are both quite into best practices, such as clean URLs, proper use of GET vs. POST, etc., so we were very picky in our analysis of those existing tools.

Shabda: Can you tell us about Everyblock? Why did you choose to create your own mapping engine instead of using something like Google Maps? How hard has it been creating a new mapping engine?

Adrian: EveryBlock is the project I lead for my day job. It’s funded by a two-year grant from the Knight Foundation, and the goal is to experiment with block-specific news — that is, the site lets you enter an address (currently only in Chicago, New York City and San Francisco) and view recent news within a very tight radius of that address. I’m pretty confident it’s the most granular attempt at local news ever attempted on the Web. Nobody else is crazy enough to do it, I guess!

We do a TON of work collecting local information, normalizing it and pulling it together for people in one place. A large part of what we publish is government data such as crime reports, restaurant inspections, building permits, business licenses…and even local movie filmings. Much of this stuff either is buried in “deep Web” government databases or has never before been available online. A second part of what we do is detecting addresses and locations in news articles and blog entries. Plus, we pull in various other geographic Web stuff like Flickr photos and Yelp reviews.

All in all, our goal is to show you recent, geographically relevant stuff that you might not have heard about. Say the local newspaper wrote something about your neighborhood on page B-23 — would you really have noticed that article? Say there was an aggravated assault on your block — would you really have remembered to check your police department’s crime-reports Web site on a daily basis? That’s the basic philosophy.

Regarding our custom mapping engine…Let’s face it: Google Maps is so passe. As one of the original Google Maps mashup guys (I developed chicagocrime.org by reverse-engineering Google’s JavaScript, before the API was released), I have all the respect in the world for what Google has done to invigorate the world of Web maps. But it’s time to take the next step. The Google Maps API doesn’t give you any control over the colors of the map tiles, or change the fonts in street labels, or disable building footprints, or hide one-way street markers or subway stations or bus stops or any of the other stuff that’s essentially hard-coded in the map. So, at EveryBlock, we rolled our own maps so we could have much, much more fine-grained control over all of these things. Not to mention it’s a great way to differentiate ourselves from the 2.5 million boring, same-old-yellow-blue-orange Google Maps mashups out there.

Paul Smith from the EveryBlock team has written an article at A List Apart with more of the technical specifics.

Shabda: As per the Knight grant rules, you would be releasing the code for Everyblock next year. Would you not be giving away your secret sauce? How do you plan to maintain competitive advantage once that code is freely avaliable under a permissive license?

Adrian: Our competitive advantage is that we’re an incredible team, and I’m sure we’ll come up with a way to feed our families.

Shabda: What problems would you like Django to tackle after 1.0? What big features would you be most interested in having in Django, after it hits 1.0?

Adrian: Generally I’d like to add higher and higher levels of abstraction to the framework. The Django admin application is a good example of that – it’s not just an abstraction of an HTTP request; it’s an abstraction of an entire Web application! We should do more of those.

Shabda: What is one thing about Django which you absolutely love, and one thing which you think Django should do differently?

Adrian: I love the way URLconfs work — like a table of contents for your Web app. I also love template inheritance. I don’t love the fact that we’re generally slow in keeping up with tickets and feature requests.

Shabda: A lot of people these days have to evaluate between Django and ROR. How can they make this decision? When should Adrian Holovaty use ROR? When should DHH use Django?

Adrian: My answer is simple: Try out both frameworks and see which one you like better.

Shabda: Does Django need to market itself differently? What can Django community do for this?

Adrian: I’m comfortable with the amount of attention (or lack thereof, depending on your perspective) that our project gets. I’m comfortable with the size of the community. I’m comfortable with the fact that the right people have found out about it through word of mouth, books, blogs, Google App Engine or wherever else. If things continue at their current rate, we’ll continue to do just fine, as a healthy open-source project.

The one thing I’d ask the community to do is to continue staying civil, polite and approachable.

Shabda: What does the phrase ‘journalism via computer programming’ mean? How can these two divergent fields be tied together?

Adrian: “Journalism via computer programming,” in my opinion, is when a journalist writes code to tell a story. Instead of talking on the phone with sources to gather facts, this could involve screen-scraping Web sites to gather raw data. Instead of writing a newspaper article, this could involve building a database-driven Web site.

Journalism has several subdisciplines — photography, information graphics, video. I advocate that computer programming should be another one of those subdisciplines. Just as a newspaper employs photographers and graphic artists, it should employ programmers who help gather information and tell stories with it.

Shabda: How can traditional journalists do more ‘journalism via computer programming’? How can programmers do more ‘journalism via computer programming’? Is it easier for Programmers to move to this field, or for Journalists?

Adrian: I believe it’s easier for programmers to become journalists than it is for journalists to become programmers, but both sides need to gain an appreciation for the other in order for this sort of thing to happen more often. Fortunately, some news organizations are starting to hire developers with this in mind, and some geeks are realizing journalism is a great, (mostly) pure field that lets you improve the world through information.

One concrete thing programmers can do is to look for jobs at news organizations, which desperately need technical talent. Developers, you will be loved, you will be treated like geniuses, and your non-techie coworkers will be very easily impressed!

Another route programmers can take is to get training in journalism – in fact, Northwestern University’s journalism school is giving out full-tuition scholarships for programmers who want to learn journalism.

Shabda: Thanks Adrian.

This was Adrian’s interview, the last in django-interviews series. But we have a lot of interesting things coming up, so stay tuned.

Related posts:

1. An Interview with Jacob Kaplan-Moss – Creator of Django
2. Interview with James Bennett – Django release manager
3. An interview with Russell Keith-Magee – Django core contributor