Tag: python

Import vmstat logs into LoadRunner

Posted by Gaz Davidson – 2010/11/15

Here’s a little script to format vmstat’s output into something you can import into LoadRunner’s analysis tool:

import sys, time
 
sys.stdout.write("Date,Time,"
                 "Proc Run,Proc sleep,"
                 "Mem swap use,Mem free,Mem buffered,Mem Cache,"
                 "Swap in,Swap out,Blocks in,Blocks out,"
                 "Interrupts,Context switches,"
                 "CPU %User,CPU %System,CPU %Idle,CPU %Wait,CPU %Stolen\n")
 
for line in iter(sys.stdin.readline, ""):
    if line[0] != 'p' and line[0:2] != ' r':
        t = time.strftime("%d/%m/%Y,%H:%M:%S", time.localtime(time.time()))
        line = ' '.join(line.split())
        line = line.replace(" ", ",")
        sys.stdout.write("%s,%s\n" % (t, line))
        sys.stdout.flush()

(Thanks to fabrizoM for the help!)

Run it from a shell script like so:

#!/bin/bash
vmstat 5 | python vmstat2csv.py >> servername-vmstat.log

Then start the job from the shell, press CTRL+Z and nohup it (so it doesn’t die if you disconnect) and tail it (so you can keep an eye on it) like so:

[1]+  Stopped                 ./log.sh
[gaz@box ~]$ bg
[1]+ ./log.sh &
[gaz@box ~]$ disown
[gaz@box ~]$ tail -f servername-vmstat.log
Date,Time,Proc Run,Proc sleep,Mem swap use,Mem free,Mem buffered,Mem Cache,Swap in,Swap out,Blocks in,Blocks out,Interrupts,Context switches,CPU %User,CPU %System,CPU %Idle,CPU %Wait,CPU %Stolen
23/11/2010,12:59:47,1,0,0,1266452,465576,2074012,0,0,2,15,0,36,0,0,99,0,0
23/11/2010,13:00:11,0,0,0,1264460,465584,2074024,0,0,0,168,581,577,1,1,94,4,0

When you’re done, import it into the Analysis tool via the Tools -> External Monitors menu. Make sure you set the correct date for the custom format and remember to keep a close eye on the timezones. If you make a mistake while importing there’s no going back, so save a backup of your analysis files before you attempt this.

Automatic favicon to PNG script

Posted by Gaz Davidson – 2010/05/16

Ever want to add a link to a site with the favourites icon in the page along with the link? I did, so I made this cool little toy:

http://favicon.bitplane.net/

read more…

Interpolated list class for Python

Posted by Gaz Davidson – 2009/12/12

While messing with Python to output some better graphs for my Facebook group scraper I stumbled upon an interesting problem. What happens if you have missing samples, or want to change the sampling frequency half way through your log? Well, you could use a proper math library and have extra dependencies or generate the missing data manually, but the simplest answer I could think of was to write a list class which interpolates between missing values, so you can do something like this:

>>> a = InterpoList()
>>> a[0]   = 0
>>> a[100] = 200
>>> a[200] = 0
>>> a[50]
100.0
>>> a[125]
150.0

And then store the time values as seconds past the epoch. Now I can inject data from multiple sources taken at different frequencies and retrieve an interpolated value for any time, which means more nice graphs like this:

As usual I also decided to share it with the Internet, because I’m nice like that! You can get a copy of the InterpoList module here.