examples

Python & Gnuplot & AHK & Processing

Posted by admin on April 24, 2010
Processing, Python, examples / Comments Off

Gnuplot is a great command line tool that creates very quickly graphs from data files.

The official site of Gnuplot is: http://www.gnuplot.info/ . To see examples of graphs produced by Gnuplot, you should have a look here: http://gnuplot.sourceforge.net/demo_4.4/.

Using python and gnuplot, I produced approx 80 graphs similar to the one below, that fitted nicely on 5 A4 pages. gnuplot_sample

Python scans a very long file with all the historical data and passes the required data for each graph to gnuplot. The result is 80 graphs as png files in less than a minute. Then another script in processing puts everything together in a 5 page pdf file (2 cols x 8 rows = 16 per page). AutoHotKey is what “glues” everything together and makes this a “single click” solution.

Tags: , , ,

Playing with Python in the wild Amazon (elastic cloud)

Posted by admin on April 05, 2010
Python, article, examples / Comments Off

This Easter bank holiday, I spent some time working on a little problem I have at work with a simulation I’m writing in Python.

It’s basically a Monte-Carlo simulation which is performing billions of calculations. To be more exact, the simulation is a script that performs about half a million of calculations and it has to run a few hundreds of thousands of times. In other words, a lot of number crunching… And as my bosses are a bit impatient, it has to be done quickly, i.e. in hours as opposed to days.

I was running benchmarks at work with various machines (an old G5, a new intel duo laptop and and old P3 desktop). Based on my benchmarks, the old P3 desktop would need approx 10 days to complete the required calculations and it became clear that, if I want to complete these calculations within a reasonable amount of time, I will need a significant amount of processing power.

Over the weekend, I decided to try and quantify/benchmark the performance of Amazon’s EC2 service. I used the m2.4xlarge option (”High-Memory Quadruple Extra Large Instance 68.4 GB of memory, 26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each” per Amazon’s description), 1690 GB of local instance storage, 64-bit platform”) which gives the highest computing power. To my surprise, the python script was running on this virtual machine just slightly faster than when it was running on my laptop. After having a look at the monitoring service Amazon offers, I realised that less than 20% of the computing power of this VM was used while running the model…

After a bit of searching/googling/reading, I found that
- My python script was essentially only using one of the cores of the VM machine, in other words 1/8th of it’s capacity.
- To use many processors efficiently one could use the multiprocessor package which is available in version 2.6, but this really complicates things and introduces new levels of complexity on my project.
- I also tried Parallel Python, but I didn’t see any significant changes in the CPU usage.
- A very interesting video on this issue (Python GIL) from David Beazley can be found here.

After reading all this and spending a lot of time experimenting, I took a step back and tried to think this through. As far as my project was concerned and due to the way a Monte Carlo simulation works, my 50 billion calculations were actually 100,000 repetitions of half a million calculations,but these repetitions were independent of each other. In other words, they could run concurrently.

In the end, a simple bash script solved this problem:

#!/bin/bash

for i in {0..1000}
do
python 50M.pyx &
done

In the above script, I was running 1,000 times a simple script (50M.pyx) which contains a loop of 100 iterations of my main set of calculations (half a million calculations). In other words, (1,000 x 100 x 500,000 = ) 50 billion calculations.
The ampersand at the end of the python line indicates that the shell should not wait for the command to finish before it moves on. As a result, it practically launches 1,000 simultaneous instances of the 50M.pyx script. With this simple method, the powerful VM server is working to the max:

max_cpu_usage

It ended up taking 9 hours 13 mins to run 50 billion calculations on this machine. On an old P3 Dell desktop this would take approx 10 days, so the lesson is that, if we use many python instances that run concurrently and a (much much) better machine, there is a significant improvement in performance. Am I stating the obvious? :)

When my simulation is ready, I will probably use 10 instances to get the calculations finished in less than 1 hour.

Tags: , , , ,

Multiple computers on a desk…

Posted by arkadian on March 18, 2010
article, examples / 2 Comments

At work, I have been working with two computers over the past few years. That involved having two screens, two mice and two keyboards on my desk. I wasn’t very pleased but I could live with it.

When I realised I would have to get a third computer, it became apparent that I needed a better solution to manage them all while maintaining my sanity…

I got a stand from allcam.biz (http://www.allcam.biz). I got the longer side arms (optional extra) and attached two 17″ on the two sides (portrait) and a 19″ in the middle (landscape).

TFT stand

By effectively removing two monitor stands, I saved a lot of desk real estate. This stand does not swivel freely (you need to unscrew the bolts if you want to adjust the monitors), but this is not a big issue for me.

Then I had to solve the problem with the keyboards & mice. At first, I thought I will have to go for a hardware solution (either a kvm box from which I will only use the key & mouse inputs/outputs) or a simple usb switch that allows you to share usb devices across different computers.

But, after a bit of searching, I found that there is a free software solution to this problem: Synergy. This is a fantastic little program, that allows you to seamlessly use the same mouse and keyboard across as many computers as you want. What’s even better, you can move from one computer screen to another by just moving the mouse past the side border of one screen and into the other screen. In other words, in the same way as if the two screens are attached to the same graphics card! This is achieved by sending the keystrokes and mouse coordinates via ethernet. It works so well that, unless you were told, you wouldn’t know that the mouse is not physically connected to all three machines. And to make things even better, you can mix-and-match windows, linux and osx machines and you can copy and paste text across different machines! Amazing!

At that point, I had to get a wireless keyboard and mouse. I got the Logitech Cordless Desktop S520 from Amazon for about £30:
Logitech S520
This is a great solid keyboard: it has a standard shape (I can’t work with curvy keyboards), an armrest that is actually part of the keyboard, a low battery indicator both on the mouse and the keyboard and the Caps Lock and Num Lock lights on the usb receiver (which means you don’t waste any battery for them). The laser mouse is one of the best I’ve used.

By removing the multiple keyboards, mice, cables and monitor stands, I have saved a lot of space. I even have space for a couple more computers now! :)

Tags: , ,

PIR sensor + Arduino + Processing + Skype…

Posted by admin on November 23, 2009
Arduino, Processing, examples / Comments Off

This post is about a simple and interesting combination of software and hardware.

Last week, I picked up a couple of PIR sensors from Ebay.

pir2_w500

PIR sensors are (usually) easy to interface with an arduino board.
Red: 5V, Black: Ground, Yellow: Analog In. Could it be easier than this?

PIR_w500

After a bit of experimentation, I found that the board sends a pulse as soon as it detects some sort of movement. Also, it sends a pulse at the very beginning, when it sort of “boots”.

My ultimate aim was to start Skype as soon as movement was detected.

I loaded the standard firmata sketch on my arduino, see: http://arduino.cc/en/Reference/Firmata. Basically, the arduino simply sends all the analog port readings to Processing.

The following Processing code is simply a proof of concept. The sketch starts, looks for an arduino at the second serial port of my PC, waits for 5 secs for the PIR to send its first pulse and then, if movement is detected calls a number, using Skype.

At any moment, in the middle of the screen, you see the value of the PIR sensor.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
import cc.arduino.*;
import java.io.*;
import processing.serial.*;
 
int analogPin = 5;
int myV; // PIR sensor value
 
PFont font;
Arduino arduino;
 
//////////////////////////////////////////////////////////////////////////////
void setup(){
  size(400, 300);
  font = createFont("ArialUnicodeMS-12.vlw",22, false); 
 
  //my board is the 2nd serial device, hence Arduino.list()[1]
  println(arduino.list());
  arduino = new Arduino(this, Arduino.list()[1], 57600);
}
 
//////////////////////////////////////////////////////////////////////////////
void draw(){
    background(0);
    fill(255);
 
  // the PIR sensor is sending out HIGH signal in the first 5 secs
  // so we will ignore all the early signals
  if(millis() < 5000){
 
  }
 
  else{
    // myV is the scaled value of the PIR sensor
    myV = arduino.analogRead(analogPin);
 
    textFont(font, 22); 
    textAlign(CENTER,CENTER);
    text("PIR sensor value: " + myV,width/2,height/2);
    text("Active for " + round(millis()/1000) + " seconds",width/2,height*3/4);
 
    callme();
  }
}
 
//////////////////////////////////////////////////////////////////////////////
void callme(){
 
  // you should be logged in on Skype and have some credit
  // if your are dialling an external line
  String mySkype = "C:\\Program Files\\Skype\\Phone\\skype.exe /nosplash /callto:+44798000000";
 
  if(myV>100){
 
    print(" MyV: " + myV + " ");
 
    try {
      String line;
      Process p = Runtime.getRuntime().exec(mySkype);
      p.waitFor();
      System.out.println("EXIT: " + p.exitValue());
    }
 
    catch (Exception err) {
      err.printStackTrace();
    }
 
      delay(3000); 
  }
 
 
 
}
//////////////////////////////////////////////////////////////////////////////

Here is a very short video of this test in action:

PIR Sensor – Test from Arkadian.Eu on Vimeo.

Tags: , , , ,

Simple Analog Signal Meter

Posted by admin on October 25, 2009
Arduino, Projects, examples / Comments Off

I bought a couple of these signal meters from Maplin a few months ago and today I found some time to try them on an an arduino.

Maplin sell these for £4 each, which is probably quite expensive for what they really are, given the cheap plastic material and the overall quality: just to give you an idea: the transparent front cover is attached to the rest of the unit with a piece of tape…

Still, if you want to add a simple meter on your arduino, without messing around with lcds and their libraries, this is by far the easiest way.

In my test, I simply used a potentiometer and I attached the meter like this:

Simple Analog Signal Meter

Simple Analog Signal Meter

(btw, the battery underneath the meter has just a “supporting” role)

Here is a simplified version of Tom Igoe’s “AnalogueWriteMega.pde” example, that comes with Arduino 0017:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
const int thisPin = 5;
 
void setup() {
  pinMode(thisPin, OUTPUT); 
}
 
void loop() {
  for (int x = 0; x < 255; x++) {
    analogWrite(thisPin, x);
    delay(50);
  } 
 
  for (int x = 255; x >= 0; x--) {
    analogWrite(thisPin, x);
    delay(50);
  } 
 
  delay(100);
}

… and here is a video of how it works (apologies for the poor quality):

Simple Signal Meter from Arkadian.Eu on Vimeo.

Tags: ,

Arduino-based keyboard for Windows

Posted by admin on October 03, 2009
Arduino, Projects, examples / 3 Comments

Today, on the arduino.cc forums, a user had the following problem: he wanted to start video in quicktime by pressing a custom button on an arduino. Basically, the arduino board should act as a simple keyboard for the computer.

I found this problem interesting and I ended up solving it. Here is the link to the forum.

Basically this problem has three parts.
1. An arduino sketch which, when the button is pressed, sends a character on the serial.
2. A serial-to-keystrokes program, which takes the character from the serial connection and converts it to a real keystroke
3. A program on the windows machine that translates the keystroke to whatever we want to do.

Sounds tricky? It’s actually easier than it sounds.

1. The Arduino Sketch
This is quite straight forward. The simple arduino keyboard circuit looks like this:

arduino-keyboard

Here is a modified version of the Button.pde example that comes with the latest Arduino release. I have only added the lines required for the serial connection and a very long (2 sec) delay after the key is pressed; I used this 2 sec delay to keep things simple, but you may want to adapt the longer and more appropriate “Debounce.pde” sketch (also available as an example in the latest Arduino release) which takes care of the debouncing more efficiently.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
const int buttonPin = 2;     
const int ledPin =  13;      
int buttonState = 0;         
 
void setup() {
  Serial.begin(9600); 
  pinMode(ledPin, OUTPUT);      
  pinMode(buttonPin, INPUT);     
}
 
void loop(){
  buttonState = digitalRead(buttonPin);
  if (buttonState == HIGH) {     
    digitalWrite(ledPin, HIGH);  
    Serial.print("a");
    delay(2000);
  }  
  else {
    digitalWrite(ledPin, LOW); 
  }
}

2. From Serial to Keystrokes
I found a very nice program, AAC Keys, which converts any characters the arduino sends to the serial port to actual keystrokes. This is available for both windows and Mac and it’s free to download; the download is a single executable file, which, when activated, stays in your system tray and monitors the serial port. The only thing you need to configure here is the port and the Baud rate:
aac_keys

Note that if AAC Keys is activated and configured, the arduino environment won’t be able to communicate with the arduino board, as the port cannot be shared.

At this stage, with the AAC Keys activated and configured, pressing the button on the arduino will create the character “a” on the computer.

3. Windows Shortcuts
What we need to do now is to somehow start playing a video with Quicktime. To play a video with QuickTime, all we need to do is to run the following command in a DOS window:

"C:\Program Files\QuickTime\QuickTimePlayer.exe" "C:\your video path\Video_A.avi"

If the video doesn’t start directly, then go to QuickTime >> Preferences >> Player Preferences… and select “automatically play movies when opened”.

My favourite windows program for these tasks is AutoHotKey. When AutoHotKey is installed, textfiles with the .AHK extension are executed by AutoHotKey.

The following AHK script, when executed, will stay in the system tray waiting for the “a” key (either from the keyboard or from the arduino board) to be pressed.

1
2
3
a:: 
Run "C:\Program Files\QuickTime\QuickTimePlayer.exe" "C:\your video path\Video_A.avi"
return

…and that’s it! A simple (1-key) Arduino keyboard, which, when pressed, launches the video we want in quicktime.

There are many ways to extend this idea. More keys can be added (many can be added on a single analog port with resistors), and, with a bluetooth adaptor on the arduino board and the computer, we could even design a bluetooth keyboard for the computer! :)

Tags: , , , ,