In my previous post I complained about annoying flickering. Well, I’ve upgraded to TinyMCE 2.0 RC2 and it doesn’t flicker anymore! Run, don’t walk, and upgrade immediatelly!

Finally I bit the bullet and decided to implement OpenWrt web-based user interface using Felix Fietkau ’s code. Well, MIPS processors used by OpenWrt units are not race horses and GUI is slow to my taste. Yes, I know that stock firmware is not faster. "Slow" is absolute category for me.

How do we combat latencies and slow network speed? Exactly. We can apply the same for OpenWrt, where "slow network speed" is replaced with "slow processor on local network". So I decided to do The Right Thing^(tm) and implement it using AJAX approach. I will use Dojo .

Hurricane Rita was donwgraded to Category 3 storm (125mph/200kph winds). Here in Dallas we expect heavy rains for 2+ days.

Sunset was very colorful today. I tried to capture it in this picture:

You can find bigger picture on Flickr .

Update: "The Simple Way" part of this tutorial is obsolete now. I am going to recreate examples using new improved RSS framework. Stay tuned!

I was asked several times to explain how I did RSS for my site. Django has RSS framework, which is not documented. Most probably I am not the right guy to explain it but I’ll try.

There are three ways to implement RSS with Django:

1. The Simple: using Django’s RSS framework.
2. The Smart: using django.utils.feedgenerator.
3. The Hard: write a view and output XML manually or using standard xml.sax.saxutils. If you want Django to do everything for you, then you should use "The Simple Way". If you want some custom object selection, you should use "The Smart Way". For obsessed workaholics, S&M adepts, and guys-with-really-convoluted-needs the only way is "The Hard Way". Being lazy I prefer #1 and #2. If you want #3, I suggest you to study files mentioned in "The Simple Way" subsection below.

The Simple Way

If you look into Django’s code, you will find several files related to RSS:

After some requests I decided to publish my code for categories. It’s very simple. It was inspired by following articles: A "category" Data Model (note: this article uses old-style model format, it doesn’t work anymore) and Relating an object to itself, many-to-one .

from django.core import meta

class Category(meta.Model):
    """
    Category defines following fields:

    name        - simple name of category, e.g., 'C++'
    full_name   - full name of category, which includes names of all parents,
                  e.g. 'Development::C++'
    parent      - reference to parent category or null
    description - HTML description of category, or null

    Notes:

    1) full_name is not editable. It is calculated automatically
       by calculate_full_name() method during save phase (hook _pre_save).
    2) Separator specified by get_separator() method. It can be
       overridden in subclasses.
    """
    name        = meta.CharField(maxlength=50)
    full_name   = meta.CharField(maxlength=250, unique=True,
                                 editable=False)
    parent      = meta.ForeignKey('self', blank=True, null=True,
                                  related_name='child')
    description = meta.TextField(blank=True, null=True,
                                 help_text="Use raw HTML.")

    class META:
        verbose_name_plural = 'Categories'
        module_name = verbose_name_plural.lower()
        admin = meta.Admin(
            list_display  = ('full_name',),
            search_fields = ['full_name',],
            js = (
                '/tiny_mce/tiny_mce.js',
                '/appmedia/admin/js/textareas.js',
            ),
        )
        ordering = ('full_name',)

    def __repr__(self):
        return self.full_name

    def _pre_save(self):
        self.full_name = self.calculate_full_name()

    def get_separator(self):
        return '::'

    # note: used in templates
    def get_all_children(self):
        "get list of all children of the category"
        output = []
        children = self.get_child_list()
        for c in children:
            output.append(c)
            output.extend(c.get_all_children())
        return output

    # note: the last object of the list is the category itself
    # note: used in templates
    def get_all_parents(self):
        "get list of all parents of the category from top level parent down"
        id_list = []
        object = self
        while True:
            id_list.append(object)
            if not object.parent_id:
                break
            object = object.get_parent()
        id_list.reverse()
        return id_list

    # note: used in templates
    def calculate_full_name(self):
        "calculate full name from parent list"
        return self.get_separator().join([x.name for x in self.get_all_parents()])

    # note: used in templates
    def get_path(self):
        "get relative path of the category"
        return 'categories/%d/' % self.id

As you can see it is very simple. This model defines full_name field, which is auto-populated. I was toying with an idea to keep parent_name and produce full_name by concatenating parent_name with name. But after some trials I decided against it: it introduced a lot of complexities and runtime overhead by saving a few bytes in database. In my opinion it doesn’t worth it. This model defines some convenience methods as well. They are meant to be used in templates.

Finally we have official Django badges . Wilson Miner did a great job putting them together. I decided to use this one:

How do you like it?

If you are a fan of 80 by 15 pixels badges, don’t fret — Dave Hodder has you covered with his rendition of "Made with Django" theme. Collection of his small badges are attached to ticket #509 . Check it out too.

Today is the day of new releases: OpenWrt just released new White Russian RC3 .

BTW, contrary to popular belief, it’s not just for Linksys WRT54G . You can run it too, if you have supported hardware . I run it on Buffalo WBR2-G54 .

Wow, they move fast! Go and grab Tiny MCE 2.0RC2 .

Finally I added RSS 2.0 using Django’s facilities.

Default RSS feed is for my blog. Categories have two versions of RSS feed: exclusively for documents of the category, and documents for the category and its children. (I hope it is not news anymore: I have hierarchical categories.)

Update: InnoDB is in the house !

In my previous post I already complained about lack of transaction support by DreamHost. The only database option is MySQL with InnoDB engine, which doesn’t support transactions. Everybody knows that transactions are nice. So what about MySQL/InnoDB or PostgreSQL?

So I hit DreamHost forums and found several posts from DreamHost people about PostgreSQL and InnoDB:

• Re: PostgreSQL rant on 06/02/05 by Jeff @ DreamHost (DH Discussion Forum Admin)
• Re: PostgreSQL rant on 06/01/05 by Dallas (DreamHost Head Honcho/Founder)
• Re: mod_proxy support on 04/07/05 by Jeff @ DreamHost (DH Discussion Forum Admin)
• Re: PostgreSQL on 12/03/04 by will
• Re: PostgreSQL rant on 10/15/04 by nate
• Re: PostgreSQL rant on 06/23/04 by Jeff @ DreamHost (DH Discussion Forum Admin)
• Re: MySQL 4, PostgreSQL on 07/15/03 by will
• Re: MySQL 4, PostgreSQL on 04/07/03 by Jeff @ DreamHost (DH Discussion Forum Admin)
• Re: transactions in mysql on 02/11/03 by Jeff @ DreamHost (DH Discussion Forum Admin)

So I’ve got the impression that PostgreSQL is out for now, but MySQL/InnoDB is really close to be implemented and offered to customers. I decided to ask the support directly and got two firm answers: MySQL/InnoDB — No, PostgreSQL — No. They don’t have any plans to introduce them.Why? No reasons were given. Obviously "it could change" over time. But for the last several years it didn’t.

Now this blog runs on sql:. I use MySQL as my backend. Before that I tested it locally with SQLite (so easy to install!) and MySQL.

I learned today that MySQL, which is provided by my host (DreamHost ), uses MyISAM for all tables. MyISAM doesn’t support transactions. AT home I use InnoDB, which supports transactions. However the hope is not lost — DreamHost mulls over InnoDB support. But as far as I know, they don’t have an implementation schedule yet.

This site was running using locmem: caching option, which is fine for small web site. Now it runs using file:. Please let me know, if you spot any issues.

I am planning to implement sql: next.

TinyMCE 2.0RC1 is out. Take it while it’s hot! I’ve upgraded my admin pages and it looks better than previous one (duh!).

Nevertheless I see annoying flickering in FireFox. But it flickers less now.

I am not talking about multithreading problems. locmem: doesn’t have them. But it does have problems with gzip. Sometimes I see binary stuff on my screen. It looks like a gzipped web page without proper gzip encoding flag. I don’t do anything related to compression in my cache.

Is it a bug in Django’s caching? Hmm… Apparently it works properly with memcache. Can it be some bug in simple: implementation, which was a base for locmem:? Weird.

Being a little bit frustrated by the lack of caching options for my web site I’ve added locmem: cache to Django. It is fully multithreaded local memory-based cache, which uses reader-writer lock to do its magic. It seems to work well but I need some time to test it more.

I think file: is going to be next.

It looks like all kinks with Django on Dreamhost were worked out. Of course, I am referring to MySQL problem (see my previous post ) and strange stalling of long pages, which I reported to Django’s mailing list . It turned out that other people had these problem too. Jason McBrayer shared his experience . He pointed out to possible flup /fcgi.py problem. I contacted Allan Saddi — the author of flup and other Python libraries and utilities . He came to help immediately. We were able to isolate the problematic piece of software and in no time Allan made a fix.

It looks like the problem is solved. Django creates one (1) connection to database and reuses it to handle requests. Multithreaded FastCGI creates threads to handle requests. These threads make requests to MySQL, which gets confused and loses its cool^H^Honnection. This process is random and compounded with multiple web requests at the same time. For example, if Google Web Accelerator decided to prefetch some pages, your web site would be hosed.

I decided to try how well Opera works with my site. It mostly works. You can go and see stuff but all Dojo-related code doesn’t work. Interesting that Google Maps works properly.

When I switched to admin mode to edit this article, it turned out that TinyMCE doesn’t work either.

But good news is Dojo and TinyMCE failed gracefully.

I complained about instability of this site in my previous post . Now, after intensive googling, it looks like it may be related to multiple threads or forked processes using the same connection to MySQL. Apparently it can confuse MySQL. Children should create their own connections after spawning.

If Django does reuse connections, it may be possible to fix it. Otherwise, CGI is the way to go. The latter solution would be slow.

I am totally at loss. I am getting weird errors like this:

OperationalError: (2013, 'Lost connection to MySQL server during query')

They are pretty much random but fairly frequent. Apparently it may be caused by different MySQL timeouts, which I cannot control. It seems that in order to combat those you have to call connection.ping() periodically to reconnect. I added them to Django’s MySQL handler and … now I am getting these errors during execution of connection.ping()! Bizarre!