Arkadian.eu

Brian Evans has published a book on Arduino on Lulu.com, which is available either as a free pdf download or a real paperback for £3.72 + postage. Arduino Programming Notebook – Direct link

This is book (or “notebook” as Brian calls it, as it is only 36 pages long) is a very short introduction to Adruino. It covers all the basics and it’s really all you need in order to get started if you know another programming language.

Most of the information on this book is also available online on the arduino.cc website, but it’s always worth having a printout handy, in case you want to look something up pretty quickly.

AWS (Amazon Web Services) has been around for a while, but I never thought I was the target audience for it, so I never really looked into that.

Last weekend, I had the following problem to solve: I’m working on a model, that realistically needs approx 1,000 “normal” PCs in order to run within an hour. This is a simulation running on Processing I’ve been working on.

Realistically, I won’t be able to persuade my boss to buy me a cluster of servers with the processing power of 1,000 machines for this.

Here is where AWS saves me. AWS allows you to rent any number of machines for any number of hours. Well, sort of… You don’t actually rent real machines, you get virtual machines, runnning on clusters I suppose. The beauty of this service is that you can select either a pre-defined image or build and use your own. There are many pre-defined images to select from, including Fedora, Ubuntu, Suse, Debian and of course all the Windows Servers.

You have full control of the firewall and you can attach IPs and storage (at a price) if you like on your loaded image.

I created an account and started playing around and I was actually pretty impressed with the service.

You are getting charged by the hour; if, for example, you are using a smallish linux server for 1 hour, you will pay $0.10 which is not actually that bad. In my case, if I need to use 1,000 machines for 1 hour, I’ll have to pay $100, which is pretty acceptable. There are other charges for storage and traffic, but again these are pretty low too.

Even if you don’t need 1,000 machines, it’s always great to be able to have a virtual linux machine ready to be used as a sandbox for just 10 cents an hour.

I think Google offers a similar service, but, Google’s service is more about running a program on their servers rather than a program on a virtual machine that you have full (root) control, like AWS

Link: Amazon Web Services

I’m currently working on a simulation model in Processing, which creates in every cycle, 350,000 lines of temporary data in chunks of 1,000 lines.

I decided to save the temporary data in a MySQL table, as this allows me to easily add an extra query at the end that summarises everything and saves the final results on a different table.

As the 350,000 lines need only to be stored temporarily, I’m using the Memory storage engine, instead of the default (MyISAM). The advantage of the Memory storage engine for temporary storage is that it saves the data on the RAM rather than the hard drive, is (or should be) faster and when you reboot the machine, the temp data is gone. The structure of the table is stored on the hard drive, so you can use the table again and again.

Here is a test sketch I wrote to get an estimate of how long it takes to create 350,000 records.

If you want to try this, you need to download and install the SQLibrary (http://bezier.de/processing/libs/sql/). You also need to install a mySQL database locally. If you don’t have one installed already, I suggest you go to apachefriends.org and download the latest version of XAMPP.

First, to create a simple table in RAM, run the following mysql snippet:

1
2
3
4
5
6

CREATE TABLE IF NOT EXISTS `arkadian_test` (
  `myID` int(11) NOT NULL AUTO_INCREMENT,
  `myRecord` int(15) NOT NULL,
  `myTimestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`myID`)
) ENGINE=MEMORY  DEFAULT CHARSET=latin1;

This is a simple table, that is stored in ram (ENGINE=MEMORY).

The following sketch creates 350,000 records in chunks of 1,000 records.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

 
/* //////////////////////////////////////////////////////////////////////
Date: 1 Aug 2009
From: Arkadian.eu
Desc: A quick sketch that creates 350K records in MySQL
*///////////////////////////////////////////////////////////////////////
 
import de.bezier.data.sql.*;
MySQL mx;
 
public int maxRecords = 350000;
public int currentRecord = 0;
 
// adjust these for your configuration
public String user     = "root";
public String pass     = ""; //blank
public String database = "test";
public String dbhost = "localhost";
/////////////////////////////////////////////////////////////////////////
void setup() {
  size(100, 100);
  mx = new MySQL( this, dbhost, database, user, pass );
  while(mx.connect()==false){delay(1000);}
  mx.execute("TRUNCATE TABLE  `arkadian_test`"); 
}
/////////////////////////////////////////////////////////////////////////
void draw() {
 if(currentRecord<maxRecords){
    // let's check to see if the connection is still up
    while(mx.connect()==false){
      mx = new MySQL( this, "localhost", database, user, pass );
    }
 
    if ( mx.connect()==true ){  
 
      for(int i=0; i<1000; i++){
        currentRecord++;
        mx.execute("INSERT INTO arkadian_test(myRecord) VALUES ('" + currentRecord+ "')"); 
      }
 
    }
 
  }
 
  else{
    mx.close();
    exit();
  }
}
/////////////////////////////////////////////////////////////////////////

The above sketch takes about 45-50 seconds to run on my 2GHz Intel Core2 Duo laptop with 3GB RAM.

To see how long it takes on your machine, just look at the timestamps of the first and last record on your database.

For more information on MySQL tables on RAM, visit this site.