Working on OSS @ Work: Dozer profiling Pylons

It's been a fun little bit working on helping speed up a Pylons app at work. Performance needed improvement, and while we knew a couple of big places, I also wanted to look at getting some profiling in place. Fortunately I recently ran across Marius's post on profiling Pylons applications with Dozer. Now Dozer was originally meant for viewing app logs and memory checking, but it seemed that the in dev work added some profiling ability. So away I went checking out his code and seeing if I could get it to run. Once I realized that you had to setup the middleware for each type of activity you wanted to perform (cProfile, memory monitor, log view) I got things running. Very cool!

Right off the bat I realized I might be able to help a bit. The links were dark blue on black, there seemed to be a some issues with the ui on the profiling detail screen. Since we were using this for work I took it a chance to help improve things upstream. I moved the css images to get the arrow indicators working right, cycled the background rows vs hard coding them, and did some small ui tweaks I liked. I also coded up a set of "show/hide all" links to quickly expand things out and back.

Of course, it's not all roses. The show all pretty much times out in anything but Chrome. There's still more ui bits I think that could be improved.

Now that I had tinkered though it was time to add in some features we could use. First up, log view for json requests. We have some timer code in our SqlAlchemy logs and so I want to be able to view those in browser, but also for json requests. So I tweaked it up so that on json requests it adds a _dozer_logview object to the response. This then shows the various log entries along with the time diff that the html view shows.

Once that was going we decided this would be great to put on a staging version of our web application that the client has access to. The nice thing is that the staging is as close to production as possible, so some profiling information there would be very helpful. We don't want the html logview information visible to the client though. It detracts from the ui. To help with this I added a ipfilter config option for the LogView feature. In this way we can limit it to a couple of testing dev boxes and not worry about the client getting access.

I've pushed the code up into a bitbucket fork for Marius's repository. Hopefully this is useful and it's awesome that I got to spend some work time working on code that I can hopefully give back. I love this stuff.