<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>bitplane.net &#187; statistics</title>
	<atom:link href="http://bitplane.net/tag/statistics/feed/" rel="self" type="application/rss+xml" />
	<link>http://bitplane.net</link>
	<description>Rants, ramblings, free software</description>
	<lastBuildDate>Tue, 08 May 2012 13:57:38 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.2</generator>
		<item>
		<title>Bubbler: Monitoring homebrew CO2 production using arecord and Python</title>
		<link>http://bitplane.net/2012/02/airlock-mic-project/</link>
		<comments>http://bitplane.net/2012/02/airlock-mic-project/#comments</comments>
		<pubDate>Wed, 08 Feb 2012 21:14:48 +0000</pubDate>
		<dc:creator>Gaz Davidson</dc:creator>
				<category><![CDATA[Development]]></category>
		<category><![CDATA[Hardware]]></category>
		<category><![CDATA[audio]]></category>
		<category><![CDATA[home-brewing]]></category>
		<category><![CDATA[linux]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[sensors]]></category>
		<category><![CDATA[statistics]]></category>

		<guid isPermaLink="false">http://bitplane.net/?p=830</guid>
		<description><![CDATA[A couple of weeks back I was thinking about monitors and sensors and had a crazy idea: it ought to be possible to gather stats on how well my plonk is fermenting by attaching a mic to the airlock and counting the bubbles. So in a moment of enthusiasm I ordered a cheap lapel microphone [...]]]></description>
			<content:encoded><![CDATA[<p>A couple of weeks back I was thinking about monitors and sensors and had a crazy idea: it ought to be possible to gather stats on how well my plonk is fermenting by attaching a mic to the airlock and counting the bubbles. So in a moment of enthusiasm I ordered a cheap lapel microphone from eBay and forgot all about it&#8230; until Saturday morning when it finally arrived in the post.</p>
<p>Time for some fun!</p>
<p><span id="more-830"></span></p>
<p>The first thing I did was break the plastic housing off the mic and exposed the sensor, I snipped off the headphone socket and taped up the rest. It&#8217;s not the prettiest bit of electrical tape you&#8217;re likely to see, but it does the job:</p>
<p><a href="http://bitplane.net/wp-content/uploads/2012/02/butchered-mic.jpg"><img class="size-medium wp-image-832 alignnone" title="Butchering the mic" src="http://bitplane.net/wp-content/uploads/2012/02/butchered-mic-210x300.jpg" alt="" width="210" height="300" /></a> <a href="http://bitplane.net/wp-content/uploads/2012/02/butchered-mic-1.jpg"><img class="size-medium wp-image-831 alignnone" title="Butchered mic front" src="http://bitplane.net/wp-content/uploads/2012/02/butchered-mic-1-223x300.jpg" alt="" width="223" height="300" /></a></p>
<p>I then taped the mic to the airlock roughly at the area where the bubbles will be, then taped the whole thing up to make it a bit more water-resistant.</p>
<p><a href="http://bitplane.net/wp-content/uploads/2012/02/airlock.jpg"><img class="alignnone size-medium wp-image-833" title="airlock" src="http://bitplane.net/wp-content/uploads/2012/02/airlock-160x300.jpg" alt="" width="160" height="300" /></a> <a href="http://bitplane.net/wp-content/uploads/2012/02/mic-attached.jpg"><img class="size-medium wp-image-834 alignnone" title="Mic attached" src="http://bitplane.net/wp-content/uploads/2012/02/mic-attached-300x264.jpg" alt="" width="300" height="264" /></a></p>
<p>Okay, now onto the fun bit: writing the software <img src='http://bitplane.net/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> </p>
<p>The easiest way to get sound from the mic port is to use <abbr title="Advanced Linux Sound Architecture">ALSA</abbr>&#8216;s arecord, which comes as part of <a href="http://www.alsa-project.org/main/index.php/Download">alsa-utils</a> and should already be installed on your distro. Run amixer from the command line to find the correct input and unmute it if required, but in most cases just running arecord will work. I set my volume levels to be 100% (unamplified) as I want to be the only person messing with the data.</p>
<p>I&#8217;m after a raw stream of data without any of that WAV header nonsense, so throwing a stream of raw longs to stdout looks a little like this:</p>

<div class="wp_syntax"><div class="code"><pre class="sh" style="font-family:monospace;">gaz@beast:~$ arecord --format=S32_LE --file-type=raw</pre></div></div>

<p>The next task is detecting the bubbles in this raw data. The mic is touching the airlock so the bubbles are very loud and high pitched, so plugging in a high pass filter might help get rid of any background noise. I found that using Audacity&#8217;s high pass filter managed to keep the bubble track intact while filtering out most of the the sound of a vacuum cleaner, which is pretty impressive. I didn&#8217;t write my own high pass filter as it seems to work okay without one, maybe I&#8217;ll make an <abbr title="Fast Fourier Transform">FFT</abbr>-based one in the future.</p>
<p>For the moment though, I simply defined a bubble-part as a volume reading that is two standard deviations away from the mean, where the mean is taken by summing the distance of each sample from the base line; waves always add up to 0, so we need an absolute measurement from the base line or the average will always be 0. If this bubble part is more than t milliseconds from the last one, then we have a new bubble.</p>
<p>My limited testing shows that the bubble count appears to be roughly right even when I&#8217;m playing music in the background, but it will probably need some tweaks before it works in all cases. So far it&#8217;s only been tested with my 25 litre plastic wine fermenter, so I have no idea how it will perform on demijohns. I can think of a number of things that are wrong with the algorithm too.</p>
<p>For a start it&#8217;s adaptive, so if you simply unplug the cable then the odd bubble will be detected. A minimum bubble length may help here. Secondly, the averages are taken from the last listening period so it&#8217;s all dependent on the length of this window, moving averages might be better, but would be more processor intensive (as if Python wasn&#8217;t processor intensive enough) and I have no idea what that does statistically, but I imagine it&#8217;s not good from a purist perspective. Finally, it&#8217;s using average volume of each |sample value| but it actually uses peak levels to detect bubbles, so I guess that it ought to be using the maximum |sample value| over N samples where N is equivalent to some minimum frequency.</p>
<p>Ignoring all that it appears to work okay. Here&#8217;s a graph that shows it in action along with my <a href="http://bitplane.net/2012/01/temper1-ubuntu/">thermometer</a>, the dip is caused by me dismantling it while taking the photos above.</p>
<p><a href="http://bitplane.net/wp-content/uploads/2012/02/BPM-chart.png"><img class="wp-image-841 alignnone" title="Graphing the data" src="http://bitplane.net/wp-content/uploads/2012/02/BPM-chart.png" alt="" width="494" height="292" /></a></p>
<p>You can download or fork the the code over <a href="https://github.com/bitplane/bubbler">github</a>.</p>
<p>Happy brewing!</p>
]]></content:encoded>
			<wfw:commentRss>http://bitplane.net/2012/02/airlock-mic-project/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Interpolated list class for Python</title>
		<link>http://bitplane.net/2009/12/interpolist/</link>
		<comments>http://bitplane.net/2009/12/interpolist/#comments</comments>
		<pubDate>Sat, 12 Dec 2009 03:25:25 +0000</pubDate>
		<dc:creator>Gaz Davidson</dc:creator>
				<category><![CDATA[Development]]></category>
		<category><![CDATA[python]]></category>
		<category><![CDATA[statistics]]></category>

		<guid isPermaLink="false">http://bitplane.net/?p=362</guid>
		<description><![CDATA[While messing with Python to output some better graphs for my Facebook group scraper I stumbled upon an interesting problem. What happens if you have missing samples, or want to change the sampling frequency half way through your log? Well, you could use a proper math library and have extra dependencies or generate the missing [...]]]></description>
			<content:encoded><![CDATA[<p>While messing with Python to output some better graphs for my <a href="http://bitplane.net/2009/12/bash-for-xmas-number-one/">Facebook group scraper</a> I stumbled upon an interesting problem. What happens if you have missing samples, or want to change the sampling frequency half way through your log? Well, you could use a proper math library and have extra dependencies or generate the missing data manually, but the simplest answer I could think of was to write a list class which interpolates between missing values, so you can do something like this:</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #66cc66;">&gt;&gt;&gt;</span> a = InterpoList<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> a<span style="color: black;">&#91;</span><span style="color: #ff4500;">0</span><span style="color: black;">&#93;</span>   = <span style="color: #ff4500;">0</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> a<span style="color: black;">&#91;</span><span style="color: #ff4500;">100</span><span style="color: black;">&#93;</span> = <span style="color: #ff4500;">200</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> a<span style="color: black;">&#91;</span><span style="color: #ff4500;">200</span><span style="color: black;">&#93;</span> = <span style="color: #ff4500;">0</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> a<span style="color: black;">&#91;</span><span style="color: #ff4500;">50</span><span style="color: black;">&#93;</span>
<span style="color: #ff4500;">100.0</span>
<span style="color: #66cc66;">&gt;&gt;&gt;</span> a<span style="color: black;">&#91;</span><span style="color: #ff4500;">125</span><span style="color: black;">&#93;</span>
<span style="color: #ff4500;">150.0</span></pre></div></div>

<p>And then store the time values as seconds past the epoch. Now I can inject data from multiple sources taken at different frequencies and retrieve an interpolated value for any time, which means more nice graphs like this:</p>
<p><img src="http://chart.apis.google.com/chart?chtt=Members+per+minute&amp;chs=500x150&amp;cht=ls&amp;chxt=x,y&amp;chg=25.000,20&amp;chxl=0:|11:10|12:10|13:10|14:10|15:10|1:|0|25|50|75|100|125&amp;chds=0,125&amp;chd=t:51.20,48.00,48.00,50.60,50.60,57.40,57.40,55.00,55.00,46.00,46.00,49.60,49.60,28.60,28.60,43.80,43.80,45.40,45.40,45.65,45.65,46.50,46.60,50.01,51.20,48.70,46.96,50.25,54.20,55.05,57.00,56.31,52.43,52.43,39.93,39.93,8.80,8.80,14.60,14.60,66.00,66.00,58.80,58.80,50.83,50.83,16.25,16.25,43.26,43.26,60.20,60.20,2.80,2.80,1.71,1.60,5.82,7.18,4.60,2.81,2.62,2.40,2.58,3.00,4.55,13.20,13.20,3.20,3.20,3.60,3.60,7.00,7.00,4.78,4.78,3.61,3.61,8.40,8.40,9.40,9.40,13.40,13.40,20.60,20.60,5.40,5.40,34.13,37.00,11.70,3.60,14.99,22.40,36.77,54.02,52.05,47.16,48.28,54.60,54.60" alt="" /></p>
<p>As usual I also decided to share it with the Internet, because I&#8217;m nice like that! You can get a copy of the InterpoList module <a href="http://svn.bitplane.net/misc/trunk/py/InterpoList.py">here</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://bitplane.net/2009/12/interpolist/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Piwik in, Google Analytics out</title>
		<link>http://bitplane.net/2009/08/piwik-in-google-analytics-out/</link>
		<comments>http://bitplane.net/2009/08/piwik-in-google-analytics-out/#comments</comments>
		<pubDate>Thu, 27 Aug 2009 03:47:29 +0000</pubDate>
		<dc:creator>Gaz Davidson</dc:creator>
				<category><![CDATA[bitplane.net]]></category>
		<category><![CDATA[gallery2]]></category>
		<category><![CDATA[piwik]]></category>
		<category><![CDATA[statistics]]></category>

		<guid isPermaLink="false">http://bitplane.net/?p=51</guid>
		<description><![CDATA[I've stopped using Google Analytics and have now moved over to Piwik...]]></description>
			<content:encoded><![CDATA[<p>I&#8217;ve stopped using Google Analytics and have now moved over to <a href="http://piwik.org/">Piwik</a>. I don&#8217;t have that many visitors, but there are still lots of <a href="http://piwik.org/blog/2009/06/7-reasons-why-your-commercial-site-needs-piwik/">good reasons</a> to use Piwik. It&#8217;s better for everyone if I keep hold of my own stats instead of giving it all to Google and I may even do some Piwik hacking in the future.</p>
<p>Getting it working in Gallery2 was a bit of a pain, so I <a href="http://codex.gallery2.org/Gallery2:How_To_Add_Piwik_Analytics_to_my_Gallery_2">updated their wiki</a> for them, but other than that it&#8217;s about the same as installing Google Analytics.</p>
<p>Subscribers are welcome to see my laughable stats <img src='http://bitplane.net/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> </p>
]]></content:encoded>
			<wfw:commentRss>http://bitplane.net/2009/08/piwik-in-google-analytics-out/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

