Time for some fun!

The first thing I did was break the plastic housing off the mic and exposed the sensor, I snipped off the headphone socket and taped up the rest. It’s not the prettiest bit of electrical tape you’re likely to see, but it does the job:

I then taped the mic to the airlock roughly at the area where the bubbles will be, then taped the whole thing up to make it a bit more water-resistant.

Okay, now onto the fun bit: writing the software

The easiest way to get sound from the mic port is to use ALSA‘s arecord, which comes as part of alsa-utils and should already be installed on your distro. Run amixer from the command line to find the correct input and unmute it if required, but in most cases just running arecord will work. I set my volume levels to be 100% (unamplified) as I want to be the only person messing with the data.

I’m after a raw stream of data without any of that WAV header nonsense, so throwing a stream of raw longs to stdout looks a little like this:

gaz@beast:~$ arecord --format=S32_LE --file-type=raw

The next task is detecting the bubbles in this raw data. The mic is touching the airlock so the bubbles are very loud and high pitched, so plugging in a high pass filter might help get rid of any background noise. I found that using Audacity’s high pass filter managed to keep the bubble track intact while filtering out most of the the sound of a vacuum cleaner, which is pretty impressive. I didn’t write my own high pass filter as it seems to work okay without one, maybe I’ll make an FFT-based one in the future.

For the moment though, I simply defined a bubble-part as a volume reading that is two standard deviations away from the mean, where the mean is taken by summing the distance of each sample from the base line; waves always add up to 0, so we need an absolute measurement from the base line or the average will always be 0. If this bubble part is more than t milliseconds from the last one, then we have a new bubble.

My limited testing shows that the bubble count appears to be roughly right even when I’m playing music in the background, but it will probably need some tweaks before it works in all cases. So far it’s only been tested with my 25 litre plastic wine fermenter, so I have no idea how it will perform on demijohns. I can think of a number of things that are wrong with the algorithm too.

For a start it’s adaptive, so if you simply unplug the cable then the odd bubble will be detected. A minimum bubble length may help here. Secondly, the averages are taken from the last listening period so it’s all dependent on the length of this window, moving averages might be better, but would be more processor intensive (as if Python wasn’t processor intensive enough) and I have no idea what that does statistically, but I imagine it’s not good from a purist perspective. Finally, it’s using average volume of each |sample value| but it actually uses peak levels to detect bubbles, so I guess that it ought to be using the maximum |sample value| over N samples where N is equivalent to some minimum frequency.

Ignoring all that it appears to work okay. Here’s a graph that shows it in action along with my thermometer, the dip is caused by me dismantling it while taking the photos above.

You can download or fork the the code over github.

Happy brewing!

Thankfully, a search for the device ID (0c45:7401 Microdia) finds ruby bindings for Temper.c by Michitaka Ohno, which conveniently includes the reverse-engineered USB protocol. So I’ve forked this on GitHub, removed the Ruby bits, edited the unit test wrapped it up in a little script that logs the time and temperature to a CSV file. It currently only supports one device, which will change if I ever get another one of these devices (spoiler: I probably will)

To start logging your TEMPer1′s data in Ubuntu just

 # install dependencies
 $ sudo apt-get install git build-essential libusb-0.1.4 libusb-0.1.4-dev
 # clone the logger
 $ git clone git://github.com/bitplane/temper.git
 # build it
 $ cd temper
 $ make clean
 $ make
 # start logging!
 $ sudo ./log.sh > temperature.log

Now I’m just waiting for my USB clip-on-mic to arrive. Once it does I can log the bubbles from the airlock on my beverages along with the temperature, and make some pretty graphs.

Watch this space!

My last experience with a printer in Ubuntu was when I attempted to use an old Lexmark Z-something, there were no Linux drivers and after much struggling I had to reboot into Windows to use it. The Windows drivers were close to a hundred megabytes, sluggish, always running and came complete with an intrusive systray app to further sap system resources. The app had a non-standard and ugly UI plus annoying American voice that begged me to “please insert paper in the auto sheet-feeder” each time I tried to print something. I remember Canon doing a similar thing with their Windows software too which is one reason why I’ve avoided printers for all these years. That sort of blatant disregard for system resources and desktop space makes the overhead of attaching a printer weigh in heavier than the reward it offers.

I don’t know whether it’s HP’s, Canonical’s or Debian’s idea but the experience with my OfficeJet 4500 in Ubuntu has been great. Rather than having to download drivers I just plugged it in. I installed gLabels from the Software Center and printed out some labels within minutes. Pressing meta and typing “print” takes me to Gnome’s Printing app which lists my printer and its fax machine counterpart (remember those?!), and I can right click and set quality settings and so on in a standard UI:

Keying “scan” into Unity’s search bar takes me to Canonical’s “Simple Scan” which really is as easy as π, it just scans. Before I could scan through Gimp I had to install XSane, which a bit of a mess. It opens four windows and somehow transfers to Gimp as JPEG by default, leaving 16×16 JPEG compression artefacts in my scanned images. These look bad even if I choose 100% quality in the menu, so it looks like the quality settings are ignored:

edit: A bit of investigation showed that this is due to Sane’s XP driver using 10% JPEG compression by default. This can be disabled via the Window -> Show advanced options dialog.

Expect lots more activity there in the future

Sorry for not writing to you of late, but this year has been a busy one.  I’ll make up for my lack of output with this mammoth post reviewing a small selection of interesting things I got up to this year…

Did far too much work

Feels like I’ve been busting my balls for the last two years, so I decided to stick all my invoices into Google Docs and graph the output. The answer is yes, I’ve been working like a full-time employee.

2009 was a good year!

I love consulting and Microsoft have a great working environment, but a rolling contract at a huge corporation means living out of a hotel for months at a time, which gets old rather fast. I learned a lot about BizTalk, Visual Studio Load Test and T-SQL, I’ll miss the new friends I’ve made, but I certainly won’t miss the travel. So I’m taking a couple of months off to doss around, learn some new technologies and work on some free software before looking for another contract.

Rediscovered Bitcoin during the bubble

I initially flirted with Bitcoin in late 2010 and was completely fascinated by the idea of a P2P currency, specially the proof-of-work method and the anonymity. There was no GPU mining software available and doing work units on the CPU had pretty low returns, but in December I decided to get a decent PC for experimenting with this new currency. After a bunch of cooling problems and losing my wallet.dat (containing a mere 5 BTC, worth about 30 US cents each; £1 total at the time) I fell out of love with Bitcoin and forgot all about it…

Fast forward 6 months and everyone is talking about this amazing new currency, demand outstrips supply, prices soar and the bubble finally bursts at $28 a piece. Why, oh why did I spend £2,000 on a top of the range computer then not even bother mining any BTC, rather than £1000 on BTC in December and sell out at £95,0000 in June?

What could have been

On top of the new PC the Bitcoin thing ended up costing me about £50 in electricity, and my ATI 5970 hasn’t been feeling well since. I did play online poker with Americans after they closed all the poker sites, and made about $200 trading on Mt Gox. Not enough BTC to have someone assassinated but the option to buy 5 wraps of heroin from Silk Road is always open (INSTANT ADDICTION, TORE FAMILY APART. A++++ WOULD BUY AGAIN)

Read Gödel, Escher, Bach

With all the travel I’ve had a fair bit of time for reading, which is nice. Along with The Design of Everyday Things and Zen and the art of Motorcycle Maintenance, Godel, Escher, Bach (that’s Girdle, Esher, Bark to us infidels) changed the way that I think about a lot of things.

This year I’m going to try to set more time aside for reading.

Got into home brewing

A few months back I decided to start brewing mead, yeah, before Skyrim came out. I’m now seven batches in and I think I’m getting the hang of it, my latest batches taste a bit less like sweaty feet and a bit more like something you might want to drink. I started out using a 5L bottle of water with 3 jars of honey, some raisins and bread yeast added to it and a balloon stuck over the top, and now I’ve progressed to demijohns with real air-locks and exotic ingredients like.. uh.. orange flesh, cloves and vanilla pods. I even made some labels:

How's the abdomen?

I figure that this whole home brewing thing is a long game, it’s worth starting while I’m in my 30s if I’m going to actually master it before I’m old and grey. That is if the state of the kitchen doesn’t cause me to die of death by angry woman first.

Replaced my MBW-150

After the LEDs burned out on my Sony Ericsson MBW-150 I replaced it with a shiny new Metawatch. Apparently it looks like a cheap, tacky Casio (so some random guy in a bar tells me anyway), but it’s not really supposed to be bling; the firmware and Android software is open source, making the device infinitely programmable.

The down-side is that the firmware is too large to be compiled using gcc for MSP430, so there’s currently no free development environment. I sketched out an analogue face for it in C, but if I wanted to spend £2000 on an analogue watch face then I would have got a Rolex instead!

Neglected Free Software

I’ve been lazy when it comes to developing this year. I’ve done no work on my Android apps, pushed a single commit to Irrlicht and wrote about 100 lines of my web-based VNC client. I feel like I’ve cheated on the world by spending most of the year writing T-SQL and C# for Microsoft, so my new year’s resolution is to get off my backside and write some code.

And blog more. Yeah, I’ll do that too.

If you fix any bugs, please send a pull request and I’ll do my best to merge them in, but keep in mind this is also my first attempt at using git; expect schoolboy errors there too!

import sys, time
 
sys.stdout.write("Date,Time,"
                 "Proc Run,Proc sleep,"
                 "Mem swap use,Mem free,Mem buffered,Mem Cache,"
                 "Swap in,Swap out,Blocks in,Blocks out,"
                 "Interrupts,Context switches,"
                 "CPU %User,CPU %System,CPU %Idle,CPU %Wait,CPU %Stolen\n")
 
for line in iter(sys.stdin.readline, ""):
    if line[0] != 'p' and line[0:2] != ' r':
        t = time.strftime("%d/%m/%Y,%H:%M:%S", time.localtime(time.time()))
        line = ' '.join(line.split())
        line = line.replace(" ", ",")
        sys.stdout.write("%s,%s\n" % (t, line))
        sys.stdout.flush()

(Thanks to fabrizoM for the help!)

Run it from a shell script like so:

#!/bin/bash
vmstat 5 | python vmstat2csv.py >> servername-vmstat.log

Then start the job from the shell, press CTRL+Z and nohup it (so it doesn’t die if you disconnect) and tail it (so you can keep an eye on it) like so:

[1]+  Stopped                 ./log.sh
[gaz@box ~]$ bg
[1]+ ./log.sh &
[gaz@box ~]$ disown
[gaz@box ~]$ tail -f servername-vmstat.log
Date,Time,Proc Run,Proc sleep,Mem swap use,Mem free,Mem buffered,Mem Cache,Swap in,Swap out,Blocks in,Blocks out,Interrupts,Context switches,CPU %User,CPU %System,CPU %Idle,CPU %Wait,CPU %Stolen
23/11/2010,12:59:47,1,0,0,1266452,465576,2074012,0,0,2,15,0,36,0,0,99,0,0
23/11/2010,13:00:11,0,0,0,1264460,465584,2074024,0,0,0,168,581,577,1,1,94,4,0

When you’re done, import it into the Analysis tool via the Tools -> External Monitors menu. Make sure you set the correct date for the custom format and remember to keep a close eye on the timezones. If you make a mistake while importing there’s no going back, so save a backup of your analysis files before you attempt this.

I guess I’d better make a promotional image for Market and update the screenshots so they actually look like recent versions of the app!

• a) Using Float.parseFloat(String): 1516.449ms
• b) My own parseFloat(String): 654.882ms
• c) …and my own isDigit(char): 449.622
• d) …and isDigit written inline: 295.865ms
• e) …copying the string into a char buffer and avoiding all string access and method calls: 254.527ms (112ms spent copying the characters!)

So in theory, if the method took a character buffer and length it could be as short as 135ms, over ten times faster than Float.parseFloat. I’m currently using option d as gives a nice performance gain without breaking my design, but when performance tuning the entire API later this may see some huge changes. Here’s the function I cooked up:

public static float parseFloat(String f) {
	final int len   = f.length();
	float     ret   = 0f;         // return value
	int       pos   = 0;          // read pointer position
	int       part  = 0;          // the current part (int, float and sci parts of the number)
	boolean   neg   = false;      // true if part is a negative number
 
	// find start
	while (pos < len && (f.charAt(pos) < '0' || f.charAt(pos) > '9') && f.charAt(pos) != '-' && f.charAt(pos) != '.')
		pos++;
 
	// sign
	if (f.charAt(pos) == '-') { 
		neg = true; 
		pos++; 
	}
 
	// integer part
	while (pos < len && !(f.charAt(pos) > '9' || f.charAt(pos) < '0'))
		part = part*10 + (f.charAt(pos++) - '0');
	ret = neg ? (float)(part*-1) : (float)part;
 
	// float part
	if (pos < len && f.charAt(pos) == '.') {
		pos++;
		int mul = 1;
		part = 0;
		while (pos < len && !(f.charAt(pos) > '9' || f.charAt(pos) < '0')) {
			part = part*10 + (f.charAt(pos) - '0'); 
			mul*=10; pos++;
		}
		ret = neg ? ret - (float)part / (float)mul : ret + (float)part / (float)mul;
	}
 
	// scientific part
	if (pos < len && (f.charAt(pos) == 'e' || f.charAt(pos) == 'E')) {
		pos++;
		neg = (f.charAt(pos) == '-'); pos++;
		part = 0;
		while (pos < len && !(f.charAt(pos) > '9' || f.charAt(pos) < '0')) {
			part = part*10 + (f.charAt(pos++) - '0'); 
		}
		if (neg)
			ret = ret / (float)Math.pow(10, part);
		else
			ret = ret * (float)Math.pow(10, part);
	}	
	return ret;
}

This is of course a very crude method, it requires a well-formed number and will return 0 or partial numbers instead of raising exceptions, it doesn’t work with hex floats or parse NaNs or Infinity and it doesn’t use doubles internally so the accuracy is not perfect. It does however work for my purposes, so I thought I’d share it with the Internet.