testing

testing the future of Juju with snaps

Juju 2.3 is under heavy development and one thing we all want when we're working on the next big release of our software product is to get feedback from users. Are you solving the problems your user has? Are there bugs in the corner cases that a user can find before the release? Are the performance improvements you made working for everyone like you expect? The more folks that test the software before it's out the better off your software will be!

With the recent calls for testing out the Cross Model Relations and Storage Improvements coming in Juju 2.3 I think it'd be good to point out how we can leverage the power of channels in snaps to test out the upcoming features in Juju. 

To get Juju via snaps you can search the snap store and install it like so:

$ snap find juju
$ sudo snap install --classic juju

This then drops the Juju binary in the /snap/bin directory. 

$ /snap/bin/juju --version
2.2.2-zesty-amd64

That's great that we've got the latest stable version of Juju. Let's see what other versions we can get access to. 

Let's try to use the new storage flag on the deploy command that Andrew points out in his blog post.

$ /snap/bin/juju deploy --attach-storage
ERROR flag provided but not defined: --attach-storage

Bummer! That isn't in the stable release of Juju yet. Note that it calls out the flag as not being defined. Let's see if we can get access to a more bleeding edge Juju.

$ snap info juju
name:      juju
summary:   "juju client"
publisher: canonical
contact:   http://jujucharms.com
description: |
  Through the use of charms, juju provides you with shareable, re-usable, and
  repeatable expressions of devops best practices.
commands:
  - juju
tracking:    stable
installed:   2.2.2 (2142) 25MB classic
refreshed:   2017-07-13 16:20:52 -0400 EDT
channels:                                      
  stable:    2.2.2                      (2142) 25MB classic
  candidate: 2.2.2                      (2142) 25MB classic
  beta:      2.2.3+2.2-9909aa4          (2180) 43MB classic
  edge:      2.3-alpha1+develop-1f3f66e (2187) 43MB classic

There we can see that in the edge channel has an upcoming 2.3-alpha release in there. Let's switch to it and test out what's coming in Juju 2.3. 

$ sudo snap refresh --edge juju
juju (edge) 2.3-alpha1+develop-1f3f66e from 'canonical' refreshed

$ /snap/bin/juju --version
2.3-alpha1-zesty-amd64

Now let's check out that command Andrew was talking about with the storage feature in Juju 2.3. 

$ /snap/bin/juju deploy --attach-storage
ERROR flag needs an argument: --attach-storage

There we go, now we've got access to the upcoming storage features in Juju 2.3 and we can provide great feedback to the dev team. 

After we're done testing and providing that feedback we can easily switch back to using the stable release for our normal work. 

$ sudo snap refresh --stable juju
juju 2.2.2 from 'canonical' refreshed

Give it a try, check out the latest in the upcoming 2.3 work and file bugs, send feedback, and be ready to leverage the great work being faster. 

Bookie PyCon 2012 Sprint Report

http://www.flickr.com/photos/66176388@N00/4592858614

So PyCon sprints, what can you say? You go, you hack...and hack...and some some point you take a break for a beverage...and hack some more.

Last year I got the real movement behind Bookie. So this really marks the one year anniversary for Bookie. I've had it as a side project to hack on in my spare time for the last whole year. In that time, honestly it's not crazy different from where it started. However, it's gone through multiple JS rewrites, two different UI designs, and a whole lot more. I've really learned a lot about development, testing, and making some hard choices over the last year. I hope by this time next year I'll have announced Bookie on some big site (reddit/hacker news?) and survived.

This sprint though wasn't the time. So what did I get done?

tl;dr

Bookie got improvements

  • Better JS tests
  • Better PY tests
  • Start of HTML5 history
  • Invite system
  • Threaded content fetcher
  • Start of celery background runner

First, I started out by working on getting the html5 history stuff going. It's not perfect yet, but it's started and I really realized I needed to have a better way to do JS tests, so...

Next I redid the JS tests. I don't want to have to fire up the application in order to run my JS tests. I also don't want to have to hit the database and such. This means I had to change the API tests. Rather than making real requests/responses, I test that the classes build the right type of requests. I verify the url, data payload, etc are correct.

Once the JS tests were redone, I realized that I hated how I had yXXX.js as the filenames and redid those as well. While I was cleaning up I dumped a bunch of old code we no longer needed. Basically tons of gardening cleaning out the weeds.

With that out of the way, the next day of sprints was all about getting an invite system underway. I originally wanted to do a throttled signup process, so anyone could sign up, but then I realized that really, invites will work better. The people using Bookie now will know who'd be interested in testing and if someone really wants to get in, they'll contact me in some fashion.

With that up, I got to spend the next day fixing bugs in everything. Wheeee! What was cool was that I managed to get a few people at the sprints curious about Bookie and testing things out. Nothing exposes bugs like new users. During this process I spent some time cleaning house on the Python side of things and making tests easier to run/write.

Finally, I've started work on the background processing using Celery. I've got a big hurdle in that, but my cron'd stats processes are working and I've almost got imports running as out of process celery children. That should really help with new users. You know at some point they'll come flooding in right? :-)

Overall, while I didn't get a ton of new user facing features going, I did a TON of clean up and maintenance. As one person expressed "Wow, there's a lot more in here than I expected when you said it was a bookmark application". Bookie has really grown over the last year and she needs me to spend some time giving things some love before moving forward too fast. The sprints really gave me a chance to do that, all while hanging out and chatting with really smart people. What more can I ask for?

Bookie status report: Aug 22 2011 (0.3 yay!)

Updates this week: 0.3 out the door

We had to do some hacking of dying limbs to get there, but 0.3 is out the door. This was the biggest release cycle since things started up. However, the features of an API and multi user auth were worth it. There were a ton of small tweaks long the way as well. Here's hoping that 0.4 is a bit faster in the moving and releasing

You'll notice that we're backtracking on having a Firefox extension and a mobile site. Both are horribly broken at the moment. I really want to get this going though and they will be specific todo items in up coming releases. Well, except the Firefox extension part. Honestly, every time I try to get started with the Firefox extension I feel like the team there is just damn hostile to developers. It's beyond ridiculous how unable to get a handle on things there it's been. Confusing docs all over the place, an add on sdk that tries to run a virtualenv, but you don't built it from any of your local python's so your fubar if you have the wrong one, the fact that it's all getting redone again in JS and not Python...Sorry. I'm just not willing to be a fulltime firefox dev to make the dippy thing work.

Links

Next up

  • Work on more of the live api tests. It's running via the latest build of phantom.js and working well
  • Start work on the 0.4 todo list
  • Especially the save/edit ui and a bookmarklet for those.

Alpha Testing

We have a signup for if you're interested in alpha testing the hosted install at https://bmark.us. If you'd like to try it out fill out the signup form here.

Taking Part

If you care to help or follow along make sure to follow the project on github: http://github.com/mitechie/Bookie

  • Current Chrome Extension: 0.3.16
  • Most recent code updates: develop branch
  • Current working branch: develop

Bookie status report: Aug 14th 2011

Updates this week

Finally, the new and fancy Bookie API is reworked and better than ever. The live running site is updated to use the new api and there's a new chrome extension that's update for it as well. Make sure you get the latest dev version of the Chrome extension.

With the api updated, it's time to get to work on finishing up 0.3 for release. This means that we need to finish updating the mobile site and some rework of how the readable content is stored. Since we pull the content out of the page based on actual page you see, we should store these separated from user to user. We don't want to leak logged in info from one user to another.

Farewell Firefox

With the api update, the Firefox is so far out of date as to be unusable. There's some work going on to help catch it up, but I think I'm going to remove it as a feature in the docs and such. We just can't claim it at this time.

New testing

With the new api we hit some testing issues. Since there's no good way I've found to functional test the extension, the tests aren't great and reliable. I also don't have tests setup on some shared code like the bookie.api.js code. I'm thinking of setting up a running instance with some live data with the sole purpose of running tests on the JS code against it. We might even go so far as to setup the extension html files as an actual website and write some functional tests around that. After all, I can load popup.html and do some testing of that as a normal web page. We'll be missing the interactions with background.html and such, but might be worth some attempt.

Any ideas and help with this is most welcome.

Alpha Testing

We have a signup for if you're interested in alpha testing the hosted install at https://bmark.us. If you'd like to try it out fill out the signup form here.

Taking Part

If you care to help or follow along make sure to follow the project on github: http://github.com/mitechie/Bookie

  • Current Chrome Extension: 0.3.11
  • Most recent code updates: develop branch
  • Current working branch: develop

SqlAlchemy Migrations for mysql and sqlite and testing

I really want to be a unit tester, I really do. Unfortunately it's one of those things I can't seem to get going. I start and end up falling short before I get over the initial setup hurdle. Or I get a couple of tests working, but then as I have a hard time trying to test parts of things I fade. So here goes my latest attempt. It's for a web app I'm working on at work and I REALLY want to have this under at least basic unit tests. Since it's a database talking web application my first step is to get a test db up and runnging to run my tests against. At least with that up I can start some actual web tests that add and alter objects via some ajax api calls.

In order to get a test db I first had to figure out how to setup a database for the tests. For speed and ease purposes I'd rather be able to use sqlite. This way I don't need to setup/reset a mysql db on each host I end up trying to run tests on.

Of course this is complicated because I'm using sqllchemy-migrate for my application. This means part of the testing should be to init a new sqlite db and then bring it up to the latest version. In order to do this I had to convert my existing migrations to work in both MySQL and Sqlite.

Step 1: I need a way to tell the migrations code to use the sqlite db vs the live mysql db. I've setup a manage.py script in my project root so I hacked it up to check for a --sqlite flag. Not that great, but it works. [sourcecode lang="python"] """ In order to support unit testing with sqlite we need to add a flag for specifying that db

python manage.py version_control --sqlite python manage.py upgrade --sqlite

Otherwise it will default to using the mysql connection string

""" from migrate.versioning.shell import main

import sys if '--sqlite' in sys.argv: main(url='sqlite:///apptesting.db',repository='app/migrations') else: main(url='mysql://connection_stringl',repository='app/migrations') [/sourcecode]

Step 2: Not all of my existing migrations were sqlite friendly. I had cheated and added some columns by straight sql like

[sourcecode lang="python"] from sqlalchemy import * from migrate import *

def upgrade(): sql = "ALTER TABLE jobs ADD COLUMN created TIMESTAMP DEFAULT CURRENT_TIMESTAMP;" migrate_engine.execute(sql);

def downgrade(): sql = = "ALTER TABLE jobs DROP created;" migrate_engine.execute(sql); [/sourcecode]

This worked great with MySQL, but sqlite didn't like it. In order to get things to work both ways I moved to using the changeset tools to make these more SA happy.

[sourcecode lang="python"] from sqlalchemy import * from migrate import * from migrate.changeset import * from datetime import datetime

meta = MetaData(migrate_engine) jobs_table = Table('jobs', meta)

def upgrade(): col = Column('created', DateTime, default=datetime.now) col.create(jobs_table)

def downgrade(): sql = "ALTER TABLE jobs DROP created;" migrate_engine.execute(sql); [/sourcecode]

A couple of notes. This abstracted the column creation so that sqlite and mysql would take it. Notice I did NOT update the drop command. Sqlite won't drop columns, and I honestly didn't care because the goal is for my unit tests to be able to bring up a database instance for testing, I'm not going to run through the downgrade commands in the sqlite database.

Step 3. So with all of the migrations moved to do everything via SA abstracted code vs SQL strings, I was in business. My final problem was one migration in particular. I had changed a field from a varchar to a int field. Sqlite won't let you do simple 'ALTER TABLE...' and even when I had the command turned into SA based changeset code my db upgrade failed due to sqlite tossing an exception.

What did I do? I cheated. First, I updated the migration with the original column definition to an Integer field. I mean, any new installs could walk that migration up just fine. I happen to know that the two deployments right now have already made the change from varchar to int. So for them, the change won't break anything for upgrade/downgrade.

I then kept the migration with the change, but tossed it in a try/except block so that I could trap it nice and just output a message "If this fails, it's probably sqlite choking.". It's hackish, but works for all my use cases I need.

Now I can create a new test database with the commands [code] python manage.py version_control --sqlite python manage.py upgrade --sqlite [/code]

Now I can start building my test suite to use this as the bootstrap to create a test db. I'll have to them remove the file on teardown so that I don't get any errors, but that'll be part of the testing setup. Not in memory, but oh well...it works.