Making Things + AS3 – Part 1 and 2 LED Sound Visualizer
Posted: September 4, 2007 at 10:27 pmIn the obnoxious spirit of having a ton of "Part 1″ posts going on here, I decided to put up this initial experiment with my new making things controller. There's a lot to cover as this has become one of the more challenging things I have been tinkering with lately. Before going any further let me give you a brief introduction to what this is. If you are familiar with Making Things and flosc then skip to the next paragraph. The fine folks at Making Things have made a hardware controller kit that is capable of interfacing with the Flash environment. Basically what this means is that you can interact with a swf electronically and mechanically using this hardware interface. Things like stepper and servo motors, proximity sensors, and other electronic devices can be controlled and monitored through flash. Flash is able to do this by way of a java server application that is run in the background called flosc. It's based on the system of OSC that is typically used to interface MIDI applications and hardware. You can learn about OSC here and you can read more about flosc here.
Also before going any further I highly recommend you read through John Barton's progress before proceeding with this post as he has provided the insight and basis for this experiment and has gone through a few more trials and tribulations over this then myself. Check out his blog here. Make sure you go through all his posts first as there is a progression and you will have a good sense of where things stand.
That being said, AS3 integration of the controller seems to be a relatively new thing. I know John, as you saw in his posts, hit some snags with the communication of data and flosc. I know I came across issues on my own trying to get his examples working but hopefully this will be one more resource to help get something going. Just so you know, the code used to communicate with flosc is in "beta" version if you will. John mentioned planning on updating the code per the recent developments he encountered in his latest post. Since he's a busy man like the rest of us, I tried making a couple modifications myself in the meantime. Ideally it would be great to do a re-write using the Binary Socket class in AS3.
I have included what seems to be the most recent version of flosc in the source files. I reccomend you check on the version and with John's progress as well. Be patient and prepared for frustration as there are a lot of elements in this scenario that can create issues and make things NOT work.
Before starting up flosc you will need to disable your internet settings. I hope to have an example of the controller working within a network even a utility class that can configure the controller in a network using DHCP, but for the time being… Go to your system preferences. Make sure both the ethernet and the usb connection are plugged into the computer (I have the external power supply connected to mine). You might get a message saying that the Network has detected the Making Things etc.. click ok and enable that along with your ethernet port. That message simply indicates that the computer can connect to the board via the USB cable. Because the board only communicates via ethernet, the USB is only important for powering the board and using mchelper.
At this point you could fire up the mchelper application that I included with the source. Be sure to check the version of this on the making things site. Test if the board is connected by clicking on the Ethernet tab and in the command line type in "/appled/0/state 1″ and click send. Hopfully your number 0 LED will light up! Make sure you quit the mchelper program as it will block connections made using flosc.
Next we need to start up the flosc server application. First, make note of your computers IP address as you will need it in the class source. To start flosc using a mac:
Open up the terminal. Type "cd " (be sure to include the space after cd)
Then drag the flosc folder into the terminal window. This will paste the path into the terminal window.
Hit return and you should be in the flosc directory.
Type "java Gateway 10000 10000″ and you should see "OscServer created…" and "TcpServer created…"
This will tell you that the flosc server is running. Gateway is the java program in the flosc directory and the 10000 numbers designate the incoming and outgoing ports respectively that flosc and the board talk to each other on.
Next open up the MakeLEDTest.fla and the MakeMusicLED.as file. In the class file you will see const variables for the computer and controller IP address you should really only need to add the computers IP address. Publish the file and hopefully you will see a successful connection in the output/debugger window. If you do, click on the green button and the music should be playing along with the LEDs on the controller and the visual representation in the swf. If they don't, don't fret. It took me quite a while to get things working. I noticed things as silly as adding a private class variable to the as file would prevent the LED's from working. Retrace your steps and check the data flow. Make sure the IP addresses match up.
Before getting to the LED code, be aware that I have made a couple minor changes to the OSCConnection.as file that John did. I added two private methods to reorder the OSC attributes in the XML data in regards to Johns discoveries. I also changed the type to "i" instead of "s" and things worked well. I'm sure this is a band-aid fix for the time being.
So now for the code actual LED code. Here we are going to load a mp3 and have the LED's on the board reflect the amplitude level of the music. Check out Lee Brimelow's example here for a refresher on getting the amplitude value of a sound channel. The only change I made is averaging the two channels since we have one set of LED's on the board.
Here's the code:
package com.hydrotik.make { import flash.display.Shape; import flash.display.Sprite; import flash.events.MouseEvent; import flash.utils.Timer; import flash.events.Event; import flash.events.TimerEvent; import flash.net.URLRequest; import flash.media.Sound; import flash.media.SoundChannel; import org.fwiidom.osc.*; public class MakeMusicLED extends Sprite { private static const STR_LOCAL_IP:String = "169.254.73.165"; private static const STR_REMOTE_IP:String = "192.168.0.200"; private static const NUM_PORT:Number = 10000; private static const LOOP:String = "jens_buchert.mp3"; private var oscConn:OSCConnection; private var ledArray:Array; private var _ledShapeArray:Array; private var sc:SoundChannel; private var s:Sound; public function MakeMusicLED() { //Initialize connection to the FLOSC server ledArray = []; oscConn = new OSCConnection(STR_LOCAL_IP, NUM_PORT); oscConn.addEventListener(OSCConnectionEvent.ON_CONNECT, onConnect); oscConn.addEventListener(OSCConnectionEvent.ON_CONNECT_ERROR, onConnectError); oscConn.addEventListener(OSCConnectionEvent.ON_PACKET_IN, onPacketIn); oscConn.addEventListener(OSCConnectionEvent.ON_PACKET_OUT, onPacketOut); oscConn.addEventListener(OSCConnectionEvent.ON_CLOSE, onClose); oscConn.connect(); } private function onConnect(evtEvent:OSCConnectionEvent):void { trace("FLOSC Connection established"); var button:Sprite = new Sprite(); button.graphics.beginFill(0x66CC66); button.graphics.drawRect(20, 50, 40, 20); button.buttonMode = true; button.addEventListener(MouseEvent.CLICK, onClickHandler); addChild(button); s = new Sound(); s.load(new URLRequest(LOOP)); _ledShapeArray = []; var startX:int = 20; for(var i:int = 0; i<4; i++){ _ledShapeArray[i] = new Shape(); _ledShapeArray[i].graphics.beginFill(0x00FF00); _ledShapeArray[i].graphics.drawRect(0, 0, 8, 3); _ledShapeArray[i].x = startX; _ledShapeArray[i].y = 20; _ledShapeArray[i].alpha = .1; addChild(_ledShapeArray[i]); startX = startX + 10; } } private function onConnectError(evtEvent:OSCConnectionEvent):void { trace("FLOSC Connection error"); } private function onClose(evtEvent:OSCConnectionEvent):void { trace("FLOSC Connection closed"); } private function onPacketIn(evtOSC:OSCConnectionEvent):void { trace("\t>> data received: " + evtOSC.data.name + " " + evtOSC.data.data); } private function onPacketOut(evtOSC:OSCConnectionEvent):void { trace("\t>> data sent: " + evtOSC.data.name + " " + evtOSC.data.data); } private function onClickHandler(event:MouseEvent):void { this.addEventListener(Event.ENTER_FRAME, showAmplitude); sc = s.play(0, 1000); } private function showAmplitude(eventArgs:Event) { var v:Number = 4 * ((sc.leftPeak + sc.rightPeak)/2); for(var i:int = 0; i<4; i++){ if(v > i){ _ledShapeArray[i].alpha = 1; oscConn.sendOSCPacket( new OSCPacket("/appled/"+i.toString()+"/state", [1], STR_REMOTE_IP, NUM_PORT) ); }else{ _ledShapeArray[i].alpha = .1; oscConn.sendOSCPacket( new OSCPacket("/appled/"+i.toString()+"/state", [0], STR_REMOTE_IP, NUM_PORT) ); } } } } }
The music is the song "Mélange Eléctrique" from Jens Buchert's album Spa Lounge. Great downtempo album.
Making Things Music LED Source
The Discussion
see what everyone is saying
Hi I really need to use these codes in order to output the data from (HI)Max MSP to Flash. I just want to use my dance mate as a control in flash. Do I still need Flosc? Any suggestions?
I haven't used Max MSP, but flosc is what is used to communicate. Since this code is old I'm not sure what changes have been made. Hopefully it's easier to get this working as I had a tedious time with it.