José Padilla

TL;DR – I came up with a very simple solution to handle WebSockets in Django apps. You just install django-websocket-request, run a script and you have WebSockets for your Django app. The cool thing is that this solution makes Django believe its getting a (somewhat) normal HTTP request so you can reuse almost all of you application’s code. Plays nicely with Django REST Framework and with plain function-based views and class-based views. Check out the demo and it’s source code on GitHub.

The Details

We’re building Blimp 2 which means a lot of changes in our current application, infrastructure, and how we’d tackle old and new problems. One of the decisions we made regarding our application was how our frontend and backend would be organized and how they would interact.

Blimp’s backend is currently built on Django. It serves our frontend’s HTML, handles our private and public API, and all our business logic. That’s how most web apps are usually built, but Blimp is already a client-side JavaScript heavy application. Right now a normal request goes something like: You request a URL, there’s some backend work, queries to the database, caching, some more logic, render an HTML response, load third party CSS and JavaScript libraries, load our JavaScript application, some more logic, and finally render it.

After months of usage and growth, we’ve noticed a few key improvements we can build on. The new version will have a backend app that will only serve JSON, no HTML. Our frontend app, served from another location, will then consume data from that API. We want the frontend app to use WebSockets when possible, falling back to XHR.

Web frameworks like Django are built for the HTTP request/response life cycle, so everything from middleware, authentication, and views takes in an HTTP request, and at the end is an HTTP response. On the other hand, a WebSockets server implementation knows nothing about that kind of lifecycle and what it means.

Our main objectives were:

1. Reuse the same serialization and deserialization logic between the HTTP and WebSockets API.
2. Reuse all business logic for all available resources.

First ideas that came to mind were to abstract all of what we wanted to reuse by writing them as methods for our models. That way we would write what we needed once, sharing it between the two implementations. This seemed like the right answer at the moment, but after a couple of hours, it proved not to be. Since we are building on top of the awesome Django REST framework, we were going to have to subclass tons of their Generic Views and Mixins, which didn’t sound as bad at the moment, but still being very skeptical we decided to look for other possible solutions.

By then we knew that we wanted a single REST API available via two different transports, HTTP and WebSockets. The best case scenario for me was to avoid rewriting anything of what we have already working for the new HTTP API, just to have it work via WebSockets. Then it clicked. I remembered that Sails.js does something similar to what we wanted to achieve.

Sails supports transport agnostic routing, which allows your controllers/policies to automatically handle Socket.io / WebSocket messages. In the past, you’d have to maintain a separate code base to make that happen.

So in more formal terms, we wanted to support transport agnostic routing, allowing us to use everything in Django’s request/response lifecycle to automatically handle WebSocket messages.

Solution

WebSocketRequest is the surprisingly simple solution I came up with to solve this.

WebSocketRequest is a simple class that requires a JSON string containing the following keys: method, url, data, and token. The method key can be any HTTP method: GET, POST, PUT, DELETE, PATCH, HEAD, or OPTIONS. The url key is an absolute URL without the domain name. The data key is an optional dictionary in which the key-value pairs in used to create the method’s data payload. The token key is also optional, and used to recreate an HTTP Authorization header, Authorization: JWT YOUR_TOKEN_HERE. You might want to check out my last blog post to learn more about authentication with JSON Web Tokens and if you’re using Django REST framework, you might like to check out django-rest-framework-jwt.

WebSocketRequest works as following:

1. Validates JSON string payload.
2. Creates an instance of Django’s RequestFactory.
3. Dynamically calls one of RequestFactory’s methods, which returns a WSGIRequest object
4. Resolves the given URL
5. Instantiates the resolved view passing along the request, any positional and keyword arguments from the resolved URL.

Yeah, RequestFactory, you read that right. You might be familiar with it if you’ve written tests in Django, but if you haven’t, the RequestFactory provides a way to generate a request instance that can be used as the first argument for any view. The only down side to this is that it does not support middleware, which might be a problem for some.

You can install WebSocketRequest via pip.

pip install django-websocket-request

As always source is available on GitHub.

Demo time!

I’ve setup a demo application running on Heroku. Source code is also available on GitHub.

http://dwr-example.herokuapp.com/

Note that Django is not running at all. Tornado is serving a static HTML file and is routing the websocket requests. django-websocket-request then does the magic.

Throttling is enabled and snippets are erased periodically. If you run into any errors feel free to bother me on Twitter about it.

I definitely would like to hear about possible problems of this solution. Where do you think that this fails? How about for a production environment? Where can it be improved?

Photo Credit: Alice Bartlett / Martin Börjesson