Django abstracts most of the actions you would be doing with the Database. What it doesn’t abstracts, and doesn’t try to abstract is the Database modelling part. This is a quick tutorial describing to how model your data in Django models.py, and how to access and modify them.

Consider a hypothetical HR department, which wants you to build an application to track and manage their processes. They have employees who work for a department, contractors who work for multiple department. Let’s see how you you would do that in Django.

from django.db import models

class Employee(models.Model):
    name = models.CharField(max_length = 100)
    department = models.ForeignKey("Department")

class Department(models.Model):
    name = models.CharField(max_length = 100)

class EmployeeHistory(models.Model):
    employee = models.OneToOneField(Employee)
    date_joined = models.DateField()
    marital_status = models.BooleanField()

class Contactor(models.Model):
    name = models.CharField(max_length = 100)
    departments = models.ManyToManyField(Department)

Let’s see the type of relationship we created here.

An Employee has a Many-to-one relationship with Department, (i.e. One department will have many Employee, but one employee will have a single departments.)

So Employee has a field department = models.ForeignKey("Department"). See that the ForeignKey field was added on the class which has in many. In database, this creates a FK from in Employee table which refernces Departments.

A Contractor has many-to-many relationships with Department, so it has a ManyToMany field. This field can be created on either classes. At a database level this creates a new table which has a FK to both the tables.

A Employee has one-to-one relationship with EmployeeHistory. The thing to note here is that whatever we could do with OneToOneField, we can do by including the fields in the Other Class directly. However when there are fields which are only rarely needed with a given model, it is useful to separate them via a one-to-one field. The one to one field can be on either class.

Accessing objects.

Getting a specific employee.
Employee.objects.get(pk = someval)
Employee.objects.get(name= someval)
Given an department get all employees.
department.employee_set.all()

The department gets an attribute name <FKClass>_set.

Given an employee, get all colleagues.

employee.department.employee_set.objects.all()

To get siblings, get parents, and get all children.

Given an department, get number of employees.

department.employee_set.all().count()

Ok so the HR department bosses are happy with what they see, and ask you to track this data, who is the manager of each employee, who is the HOD of each department and when did each employee took leave.

from django.db import models

class Employee(models.Model):
    name = models.CharField(max_length = 100)
    manager = models.ForeignKey("self", blank = True, null = True)
    department = models.ForeignKey("Department")

class Department(models.Model):
    hod = models.ForeignKey("Employee")
    name = models.CharField(max_length = 100)

class EmployeeHistory(models.Model):
    employee = models.OneToOneField(Employee)
    date_joined = models.DateField()
    marital_status = models.BooleanField()

class Contactors(models.Model):
    name = models.CharField(max_length = 100)
    departments = models.ManyToManyField(Department)


class EmployeeLeave(models.Model):
    leave_taken = models.DateField()
    employee = models.ForeignKey(Employee)

So we now have new fields hod = models.OneToOneField("Employee") in department and manager = models.ForeignKey("self", blank = True, null = True) and a new model EmployeeLeave. Let su see the new realtions,

As one Department will have one Employee as hod, and Employee can head at max one Department, we have a one to one relationship
As many Employee will report to another Employee, so we have a FK on Employee, referencing self. As Employee will take many EmployeeLeave, EmployeeLeave has a FK to Epployee

Let us see some queries.

Get all leaves taken by an employee this year

import datetime
today = datetime.date.today()
start_of_year = datetime.datetime(today.year, 1, 1)
leaves_taken = employee.employeeleave_set.filter(leave_taken__gt=start_of_year, leave_taken__lt = today)

Get a list of all HODs.

#As there are going to be as many Employees as departments
departments = Department.objects.all()
employees = [department.hod for department in departments]

Get a list of all departments a contractor works for.

contractor.department_set.all()

Get a list of all contractor who work for a department.

department.contractor_set.all()

Note that each side of a many to many relationship get a Manager.

List of all managers in a given department.

Todo#(Can't think of any one query way to do this. If you know, let me know :)  )

List of all leaves taken in a given department.

EmployeeLeave.objects.filter(employee__department = department)

None that we used double underscores __ to do a filtering on a field across Entities.

List of all employees which joined a given department this year.

Employee.objects.filter(department = department, employment_history__date_joined__gte=start_of_year, )

Note that we used a double underscore __ twice, first to go across entities, and then to define the type of filter(__gte). Also we specified two filter conditions, so they were ANDed together.

List all employees which either report too a given employee, or joined before him.

Employee.objects.filter(Q(manager = employee)|Q(employee_history__date_joined__lt = employee.employee_history.date_joined))

Note the new construct Q, they are used to specify complex boolean operations. Here we used the | (or operator) to specify or condition.


Do you subscribe to our rss feed? The new feed has full text feed and many other goodness. So make sure you are subscribed to the new feed with extra fortified vitamins. Subscribe now.

{ 13 comments }

Summary: How to integrate a non Django database system in your Django code, using Wordpress as example. The completed code is available at github or you can see some screnshots


Though there are quite a few good Django blog applications, our blog is based on Wordpress. A number of plugin’s make moving to a Django based app a bad decision for us, and not in the spirit of “best tools for the job”.

We moved the other way, and decided to use Django to admin the Wordpress database. The completed code is available on Github

It is not too hard, with the the builtin Django commands. Django provides the inspectdb command which allows you to build your models.py from an existing non Django database.

Here we will see the steps followed for Wordpress, but it would be about the same for all systems.

Take a back up of wordpress
mysqldump -u wordpress_user -p --database wordpress_database > data.sql
Create a new project, and set its settings to use the Wordpress database.
    DATABASE_ENGINE = 'mysql'           # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
    DATABASE_NAME = ''             # Or path to database file if using sqlite3.
    DATABASE_USER = ''             # Not used with sqlite3.
    DATABASE_PASSWORD = ''         # Not used with sqlite3.
    DATABASE_HOST = ''             # Set to empty string for localhost. Not used with sqlite3.
    DATABASE_PORT = ''             # Set to empty string for default. Not used with sqlite3.
Get the initial models.py
./manage.py inspectdb > models.py

This will create all the database tables in a form Django can understand. Here is what this command creates for a my Wordpress installation with the YARPP plugin. http://gist.github.com/278962

Create a new app and put this models.py there.

With this done, you can treat the non Django part as a standalone application. Since Wordpress appends all its table with wp_ prefix, we name this applications wp to maintain table name compatibility with Django naming conventions.

You will notice that all models have the db_table populated, so we can rename tables, without changes to the database.

Differences between Wordpress and Django style naming.

At this point you will notice some differences in how Django names things (in a best practice sort of way), and how Wordpress does it.

a. Django table and model class name are (generally) singular. eg class Post(models.Models) leads to table app_post. Wordpress tables are (most of them) named plural eg wp_posts.

b. Django attributes are generally named without the table name part. Eg

class Comment(models.Model):
    author_name = models.TextField()
    content = models.TextField()

Wordpress is explicit here and includes the table prefix with attributes.

mysql> desc wp_comments;
+----------------------+---------------------+------+-----+---------------------+----------------+
| Field                | Type                | Null | Key | Default             | Extra          |
+----------------------+---------------------+------+-----+---------------------+----------------+
| comment_ID           | bigint(20) unsigned | NO   | PRI | NULL                | auto_increment | 
| comment_post_ID      | bigint(20) unsigned | NO   | MUL | 0                   |                | 
| comment_author       | tinytext            | NO   |     | NULL                |                | 
| comment_author_email | varchar(100)        | NO   |     |                     |                | 

.....

I believe this is due to the way you would generally be using the code. In Django you would do comment.author where being explicit doesn’t add any value, while in Wordpress, you would use, select comment_author, post_title ... from wp_comment, wp_post ... where join, where being explicit is useful.

You can decouple the Django and database names by using the db_table and db_column attributes. We choose to rename the Class names to match Django conventions while we let the column names remain the same.

Add Admin and other Django niceties.

Wordpress doesn’t (seem to) have foreign key constraints setup correctly, and uses bigint(20) unsigned without foreign key constraints to refer to referred entities. This means Django creates all ForeignKeys as IntegerFields.

Modify them to use ForeignKey instead. Also add __unicode__, to your classes.

Add an admin.py to register all your classes.

And you are done! Now you can access, and work with your Wordpress data inside Django and Django admin.


There are a few more things which will allow a easier Wordpress setup.

Create template tags to show the latest posts and comments.
@register.inclusion_tag("wp/recent_posts.html")
def show_posts(num_comments):
    return {"posts": Post.objects.filter(post_type="post", post_status="publish").order_by("-post_date")[:num_comments]}

So you can see that there is nothing Wordpress specific we need too do here.

Create a better admin.
Add ModelAdmin to generally used models.
Allows accessing attributes via the Django style names.

If you override __getattr__, you can access the attributes via other names. Eg in the current setup you need to do comment.comment_content, comment.comment_author etc, while we would like to do comment.content and comment.author as a shortcut.

class WordPressModel(object):    
    def __getattr__(self, v):
        if v in self.__dict__:
            return self.__dict__[v]
        else:
            new_v = "%s_%s" % (self.__class__.__name__.lower(),  v)
            if new_v in self.__dict__:
                return self.__dict__[new_v]
            else:
                raise AttributeError

It is highly debatable whether this is a good idea :) , but it is too convenient right now not to test this method out.

Here are some screenshots.


Do you subscribe to our feed? We recently made a full text feed available, so if you are using the old feed, you should change it. Subscribe now.

{ 37 comments }

Doing things with Django forms

January 14, 2010

Forms are one of the best features of Django. (After models, admin, url routing etc ). Here is a quick tutorial describing how to do things with Django forms.

Basic form

Prob. You want to show a form, validate it and display it.

Ans. Create a simple form.

class UserForm(forms.Form): username = forms.CharField() [...]

Read the full article →

Tools of Pro Django developer – aka What powers dinette and almost every app we write.

January 13, 2010

There are some tools and apps which we use with almost all apps we write, and in particular which, we used for dinette. Here they are broken into useful during development, and (also) useful post development.

During Development

Ipython and ipdb South Django test utils Django extensions Django debug toolbar

Ipython and ipdb

Ipython is a enhanced shell for python. Ipdb similarly add extra capacity [...]

Read the full article →

django-forum

January 7, 2010

twitter ready version:

We have released a Django forum application, with some cool features not in any other Django based forum. You can get it here or see it in action.

blog version

There are quite a few Django based forum applications, so why another? Its a bit of a rhetorical question, as the answer is “none [...]

Read the full article →

Django-buzz

December 12, 2009

We have updated the Django popular stories widget, which powers the widget in the side bar. It is now on Github.

Earlier the code used was hardcoded to only find Django stories, but now you can create arbitary topics from admin.

Get the code and create the widgets on your servers, (or let us know, and we [...]

Read the full article →

Using bpython shell with django (and some Ipython features you should know)

December 12, 2009

What is bpython?

bpython is a fancy interface to the Python interpreter for Unix-like operating system.

says the bpython home page. It provides syntax highlighting, auto completion, auto-indentation and such stuff.

Unlike iPython, which implements then entire shell functions and emulates the standard python shell, and adds enhancements, bpython just adds features on top of the [...]

Read the full article →

Foss.in 2009: The best foss.in. Ever.

December 10, 2009

There are technology conferences, and then there are gatherings of passionate technology enthusiasts. You will most likely not know what I am talking about, unless you saw that someone wrote a software for controlling a toy-car, using it’s blue tooth module, from an Nokia phone, at the event. You will most likely not know what [...]

Read the full article →

Python metaclasses and how Django uses them

December 3, 2009

Foss.in is without doubt India’s largest FOSS technology conference. Lakshman gave a talk today on “Python metaclasses and how Django uses them”. Here are the slides from that talk.

Doing magic with python metaclassesView more documents from Usware Technologies.

[Edit] Some reactions, http://twitter.com/jaideep2588/status/6295483833 http://twitter.com/kunalbharati/status/6296572939

And the talk images, http://twitpic.com/rxhn7

You should follow us on twitter and Subscribe to our blog

Read the full article →

Django quiz

December 3, 2009

A quick django quiz. Answers available tomorrow. Get it as a text file (django-quiz) or on google docs or read below.

### Easy

1. You have a class defined as

class Post(models.Model): name = models.CharField(max_length=100) is_active = models.BooleanField(default=False)

You create multiple [...]

Read the full article →