Robotic Xylophone – The Java Side

Filed under: development, java — digitaljoel @ 4:54 pm

About a year ago a good friend called and asked if I would be interested in a little hobby program with him.  He wanted to create a robotic xylophone using an old bell kit he found in the local classifieds.  He wanted to control it using and Arduino Mega board.  His proposition was that he could do all the hardware work, but he didn’t know how to do the software. That’s where I came in.

He fabricated a mount for the bell kit.  It would hold the bell kit above solenoids mounted on a board.  There was one solenoid for each note in the bell kit.  He would then put neodymium magnets on top of the solenoid.  The magnets would be reversed from the polarity of the solenoid so that when current was applied to the solenoid it would shoot the pin up and so it would strike the bottom of the bell.  I thought it was pretty ingenious.

This was my first (and only) Arduino project, so I had a lot to learn.  We went through several iterations of ideas on how to get the music into the Arduino.  Maybe a web interface that would allow the user to “write” the music.  Sounded like a pain.

I have a child that I believe is talented musically (yeah, every dad will say that about their child) and I thought it would be fun for her to play a duet with herself.  The robotic bell kit replaying something she had already played, and then her playing the other part on her own bell kit.  We have a digital piano that allows us to record a track as it’s played and write it to USB in midi format.  So I decided that would be the way to get the music into the Arduino.

That meant I had to figure out how to parse the midi file on the Arduino.  It’s been a VERY long time since I have written any C code.  And I didn’t find any suitably easy libraries I could use to do it.  Finally, I didn’t want to learn the ins and outs of the midi format, so I looked to see if there was a midi parsing library for Java, which I’m very comfortable in.  Sure enough there was.  And even better, it was super easy to use and didn’t even require any other libraries, it’s part of core Java (maybe not once 9 comes out huh?)

So I decided what I would do is write a java program that would translate a midi file into a custom format that I could more easily read on the Arduino.  I would then write that file to an SD card which I would read from on the Arduino.  It took me longer to come to that design than it did to write all the code.

Speaking of code, here’s the Java side of things. I added a bunch of comments, so I won’t be doing any further explanation of it.

import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Queue;

import javax.sound.midi.InvalidMidiDataException;
import javax.sound.midi.MidiEvent;
import javax.sound.midi.MidiMessage;
import javax.sound.midi.MidiSystem;
import javax.sound.midi.Sequence;
import javax.sound.midi.ShortMessage;
import javax.sound.midi.Track;

 * Class that reads a midi file and outputs a custom format that is then read in my arduino code.
 * The customer format consists for an event stream, where each event is 3 data members constituting
 *   a total of 4 bytes per event.
 * An event is [outPin, hi/lo, duration(ms)]
 * outPin : if -1 then command is to delay, otherwise it contains the pin to change hi/low value for
 * hi/lo : if not delay then set out pin to hi on 1, low on 0. Otherwise ignore.
 * duration: if delay, then this is duration, otherwise ignore
 * So an event of [8,1,0] says to turn pin 8 high.
 *    an event of [-1,1,180] says to sleep for 180ms before handling the next event in the stream.
public class MidiToArduino {

    public static void main( String... args ) {
        if ( args.length <= 0 ) {
            System.out.println( "usage: java MidiToArduino <file1> <file2> ..." );
        for ( String filename : args ) {
            MidiToArduino instance = new MidiToArduino( filename );
            System.out.println( (instance.convert() ? "Converted: " : "Failed: " ) + filename );

    // arduino is expecting short, short, int (1 bytes, 1 bytes, 2 bytes)
    // pin, value, duration.

    private static Map<Short, Short> pinMap = new HashMap<>();

    private static final short LOWEST = 55;
    private static final short HIGHEST = 84;
    private static final short OCTAVE = 13;

    // we need to map notes from the midi stream to out pins on the arduino mega.
    // We only map pins 55 through 84 because that's all the notes that were available on
    // the bell kit we were using.  We map them to pins 2-13, and 22-38 becuase those
    // are the pins we are going to be using to ouptut signale to the solenoid.
        // 53 = F
        // 84 = C
        // middle C = 60
        pinMap.put((short)55, (short)2);
        pinMap.put((short)56, (short)3);
        pinMap.put((short)57, (short)4);
        pinMap.put((short)58, (short)5);
        pinMap.put((short)59, (short)6);
        pinMap.put((short)60, (short)7);
        pinMap.put((short)61, (short)8);
        pinMap.put((short)62, (short)9);
        pinMap.put((short)63, (short)10);
        pinMap.put((short)64, (short)11);
        pinMap.put((short)65, (short)12);
        pinMap.put((short)66, (short)13);
        pinMap.put((short)67, (short)22);
        pinMap.put((short)68, (short)23);
        pinMap.put((short)69, (short)24);
        pinMap.put((short)70, (short)25);
        pinMap.put((short)71, (short)26);
        pinMap.put((short)72, (short)27);
        pinMap.put((short)73, (short)28);
        pinMap.put((short)74, (short)29);
        pinMap.put((short)75, (short)30);
        pinMap.put((short)76, (short)31);
        pinMap.put((short)77, (short)32);
        pinMap.put((short)78, (short)33);
        pinMap.put((short)79, (short)34);
        pinMap.put((short)80, (short)35);
        pinMap.put((short)81, (short)36);
        pinMap.put((short)82, (short)37);
        pinMap.put((short)83, (short)38);
        pinMap.put((short)84, (short)39);


    // These are the commands within the midi stream that we are interested in.
    private static final int NOTE_ON = 0x90;
    private static final int NOTE_OFF = 0x80;
    // This is how long we want to allow the out pin on the arduino to remain high in order to play a note.
    private static final int DELAY_MS = 10;
    // This is the key for a DELAY event in the feed to the arduino
    private static final short DELAY = -1;

    private final String inputFileName;
    private final String outputFileName;

    public MidiToArduino( String filename ) {
        inputFileName = filename;
        outputFileName = getOutputFilename( inputFileName );

     * This is where the work happens.  Not thread safe. For each conversion you must create a new instance of MidiToArduino.
    public boolean convert() {
        // when a note is played we will then add an event to this queue so that we stop applying a high
        // signal to that pin after the appropriate amount of time, as specified in DELAY_MS.
        Queue<Event> liftEvents = new LinkedList<>();
        // This list keeps the final event stream that will be sent to the arduino.  It will be a merging of the
        // midi events and our newly created lift events.
        List<Event> allEvents = new ArrayList<>();
        try {
            // most of this is boilerplate to read the midi file.
            Sequence sequence = MidiSystem.getSequence(new File(inputFileName));

            // We need to map from the 'tick' in midi, to milliseconds, since that's what our delay is on arduino.
            long microseconds = sequence.getMicrosecondLength();
            long tickLength = sequence.getTickLength();

            long msPerTick = (microseconds/(tickLength*1000));

            System.out.println( "msPerTick = " + msPerTick );

            // choose the longest track of any multitrack midi file, assuming it is the one with the music notes.
            Track track = getLongestTrack( sequence );

            int lastTick = 0;
            for ( int i = 0; i < track.size(); i++ ) {
                // iterate through and output each event, but not key offs.
                MidiEvent event = track.get(i);
                MidiMessage message = event.getMessage();
                // all this message stuff is from the midi parsing library.
                if ( message instanceof ShortMessage )
                    long tick = event.getTick()*msPerTick;
                    // before we process the next event from the midi stream, we have to see if there are any
                    // notes that we are currently playing that need to be lifted.
                    while ( liftEvents.peek() != null && liftEvents.peek().tick < tick ) {                         lastTick = addEvent( allEvents, liftEvents.poll(), lastTick );                     }                     ShortMessage sm = (ShortMessage)message;                     int command = sm.getCommand();                     int velocity = sm.getData2();                     if ( command == NOTE_ON && velocity > 0 ) {
                        // we only want to handle this event if it's where the player played a note.
                        int key = sm.getData1();
                        Event e = new Event( (int)tick, (short)key, (short)1 );
                        lastTick = addEvent( allEvents, e, lastTick );
                        // make sure we insert a new event to lift this note at the appropriate time.
                        liftEvents.add(new Event((int)tick + DELAY_MS, (short)key, (short)0));
        } catch (InvalidMidiDataException | IOException e) {
            // bail
            return false;

        writeToFile( allEvents );

        return true;

    private void writeToFile( List<Event> events ) {
        try (FileOutputStream out = new FileOutputStream( outputFileName )) {
            // we write just the bytes because it was easy to read on the arduino
        } catch (IOException e) {

    private byte[] getAllBytes(List<Event> allEvents) {
        // we know that each event will take 4 bytes, so we can easily create an array of the appropriate size.
        byte[] allBytes = new byte[allEvents.size()*4];
        int pos = 0;
        for ( Event e : allEvents ) {
            System.out.println( e + "," );
            for ( byte b : e.getBytes()) {
                allBytes[pos++] = b;
        return allBytes;

     * Because I don't know which track contains the actual music, I just pick the track with the most events and assume.
    private Track getLongestTrack( Sequence sequence ) {
        Track result = null;
        Track[] tracks = sequence.getTracks();
        for ( int i = 0; i < tracks.length; i++ ) {
            if ( result == null ) {
                result = tracks[i];
            else if ( tracks[i].size() > result.size() ) {
                result = tracks[i];
        return result;

     * Add an event and return the time of the last event.
     * @param events
     * @param newEvent
     * @return
    private int addEvent( List<Event> events, Event newEvent, int lastTick ) {
        int duration = newEvent.tick - lastTick;
        if ( duration > 0 ) {
            events.add( new Event(duration, DELAY, (short)1 ));
        events.add(new Event( 0, getPin( ), newEvent.value));
        return lastTick + duration;

     * Get the output pin that will be used to play a note.  If the note from the midi stream is too high
     * or too low, it will be adjusted by an octave in the right direction until it is within the range
     * that can be played by our bell kit.
    private short getPin( short note ) {
      while ( note < LOWEST ) {
          System.out.println( "raising " + note );
          note += OCTAVE;
      while ( note > HIGHEST ) {
          System.out.println( "lowering " + note );
          note -= OCTAVE;
      return pinMap.get(note);

    private String getOutputFilename( String input ) {
        String base = input;
        int index = input.lastIndexOf(".");
        if ( index > 0 ) {
            base = input.substring(0, index + 1 );
        return base + "jwf";

     * Simple representation of an event within our event stream.
    private class Event {
        public final short pin;
        public final short value;
        public final int tick;

        public Event( int tick, short pin, short value) {
 = pin;
          this.value = value;
          this.tick = tick;

         * Return the bytes as they should be written to the file.
         * [ pin (1 byte), value (1 byte), duration (2 bytes) ]
         * @return
        public byte[] getBytes() {
            return new byte[] { (byte)pin, (byte)value, (byte)(tick >> 8), (byte)tick };

         * Output the note for debugging in a format that is easy to copy and paste into an array in the arduino code
         * for testing a static event stream.
        public String toString() {
            return "{ " + (pin != DELAY ? pin : "DELAY") + ", " + value + ", " + tick + "}";



Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at