Concurrency Everywhere

Jon and Carl have really pulled out all the stops on this one. It could make you weep it’s so beautiful. What you see here is a screenshot of the typical jEdit interface. We use jEdit because it is open source, lightweight, and runs on Windows, Mac, and Linux without too much fuss. The code in the window is actually a near-copy of the original Surveyor firmware, translated into occam-pi. (Click on the images to see big versions…)

The first thing we need to do is fire up the occPlug. This is our plugin for compiling and running occam programs on a number of different platforms. Currently, the Windows release supports the desktop, the Mindstorms RCX, and the Surveyor SRV-1.

Once the occPlug is started, we can hit the compile button, and as you can see, our code is compiled for the Surveyor SRV-1 target. This isn’t the real magic, though…

When I run this code after compiling it, it is uploaded to the Surveyor and kicked off. Note that this is how all user programs are treated; we’ve turned what used to be the only firmware on the Surveyor into just another program. This is important, because it means it is easy for users to extend the default firmware (so to speak) to work with existing tools on the market, as well as incorporate their own algorithms for processing images, or navigation, and so on.

When I hit run, I get connected up to the Surveyor automatically. You see that “Command” area down below? I can type messages in and send them to my SRV-1 with no fuss whatsoever. Type, hit return (or the “Send” button), and my characters are whisked away into the aether through the magic of 802.11g. One of the commands I can send is the I command (meaning, I typed the letter I and hit return). Why “I”? I don’t know; we’d have to ask Howard. The important thing is that the I command sends me an image. And Jon and Carl have done a wonderful job here; when the terminal catches the header for the JPEG-compressed image coming back, it opens up an image pane, and displays what the Surveyor sees. We don’t have to do anything!

Here, I’ve sent the a command followed by I, and the image pane automatically resizes. Because the a command is interpreted to mean to set the image size to 160 x 128, I get a smaller image back. I then sent an A followed by an I…

And you can’t tell exactly, but that’s a really big picture of my kitchen floor in the background. It took a little longer to send than the tiny image, but we love those big pictures for their general awesomeness. The last thing I did was dial the size of the image back, and fired up my laser cannons. Those things are awesome.

How long does it take to compile that firmware? Let me see… I’m running Windows as a VMWare virtual machine on my MacBook Amateur (2×2GHz, 2GB RAM), and it takes roughly “one Mississippi.” How long does it take to upload the program? About as long. That alone has made developing on the Surveyor under occam-pi an absolute joy: no JTAG, no long compilation, just edit, compile, run, explore, tweak, try again… nice and easy.

Now that things are falling into place, we can start doing some very cool things, I believe. Earlier today, I was exploring the YUV colorspace, and wondering how hard it was to see an orange. Well, my first attempt was pretty lame:

As you can see, I’m highlighting pixels that I believe should be orange-colored. Oops. Well, given that I was squinting at a two-dimensional YUV colorspace plot, it wasn’t a bad guestimate. Note that I apparently have a better orange detector than a baseball detector:

Don’t read too much into this, though. I mean, I was mostly figuring out what Carl and Jon had done, and taking the occasional screenshot.

So, what’s the punchline? Jon and Carl have assembled an excellent IDE for exploring occam-pi and robotics on the Surveyor SRV-1. We’re going to be releasing this to students this weekend, and it goes live for laboratories in the Robotics class at Olin on Monday. We’re using the Surveyor to explore challenges of vision-based navigation in conjunction with simple state machines and behavior engines. What’s more, they’ll be learning how to architect these solutions in parallel-safe ways from day one. I think this laboratory (this is only one 2-week lab of four) is part of what makes Olin’s robotics class completely unique in the world today.

And yeah, I’m pretty jazzed that the software that we’ve all worked on for so long is being put to use for the purposes it was intended. That, really, is the awesome part.

Update: Does it work over the wireless? Yes. This is part of what we like about the Blackfin Surveyor—no wires. You write your code, hit compile, hit run, and the code goes through the air to your robot. You can then talk back and forth completely over the WiFi. And to answer the second qusetion: does it work on the Mac?

You tell me? Because everything in the IDE is WiFi and Java, there’s no real platform dependency. We’ve done a Windows build that will be distributed to Olin students momentarily, and want to discover any obvious problems. Once we’ve done that, we’ll do Mac and Linux builds. Doing releases is not a process that we’ve sufficiently streamlined—something that perhaps needs to be improved.

Regardless, yes, the Mac is a full-peer in this enterprise. Given that the Transterpreter was written entirely on Macs, we’re happy to support it.

(That’s a shot from Jon’s Surveyor, which lives in England and gets nice views out over Canterbury.)

I’ve discovered that programming while helping students and working with colleagues (both in the room and across the ocean) is actually distracting enough to prevent you from writing good code.

By coming in around 6AM this morning, I got some good quiet time that I was able to use productively, and in a short amount of time wrapped my head around YUV 4:22 (UYVY), and got some drawing routines ported into our Surveyor package. I have some questions for Carl and Jon about this, but for the moment, I’m glad my explorations are (mostly) over.

Last night, I got as far as wondering why I was able to make my images green:

This morning, I quickly isolated the luma component (Y) of the image:

Then, it was on to drawing. I think he sub-sampling for JPEG conversion is confusing me, but we can reliably draw shapes over the image:

Now, when students start doing image processing, they can circle or box in regions they are interested in, and see if their algorithms are returning values that make sense. (They can also print debugging information back to the PC, which is useful as well.) Between these two modes of interaction, I’d say we have a reasonably rich platform/API developing upon which students can develop parallel-safe robotics applications on the SRV-1 using occam-pi.

NOTE: We will be unleashing the things you read about here on students at Olin College very soon. We’ll bundle up a release for general consumption (as well as the documentation that we alpha test on the Olin students) Real Soon Now(TM).

I’ve been experimenting with the new firmware for the Surveyor SRV-1b that Carl and Jon have done a lot of excellent work on. Here’s something wacky: what used to be the “firmware” can now be executed as “user code”.

I’ll get there in a few steps. Consider this piece of test code that Jon wrote:

PROC tests (CHAN BYTE in?, out!, CHAN P.LASER lasers!,
            CHAN P.LED leds!, CHAN P.MOTOR motors!)
  SEQ
    out.string("SRV-1 Test Program (of Doom)*n", 0, out!)
    out.string("Testing death lasers*n", 0, out!)
    test.lasers(lasers!)
    out.string("Testing less deadly LED*'s*n", 0, out!)
    test.leds(leds!)
    out.string("Testing harmless motors*n", 0, out!)
    test.motors(motors!)
:

This is is the main process from Jon’s test program. The parameters coming into the process header are channels out to the environment. In this case, the environment is the Surveyor, so we have channels for the serial communications over the WiFi radio (those are the channels `in’ and `out’), and there are output channels for the lasers, the LEDs, and the motors. Each of these are named as you might expect. To run this on the Surveyor, we compile it, upload the bytecode, and things just go.

But what’s great is that this program, although it doesn’t do much, is not really different than the firmware that used to be written in C. The original firmware would handle commands from the SRV Console, and then spit images back, drive around, and do whatever else you commanded your mobile wireless camera platform to do. That is, all the default firmware did was respond to commands over a textual protocol. (There was a C interpreter, to. But my point is that you used a textual protocol to initiate all kinds of things.) Given that we have a channel of bytes representing the textual input coming over the radio, it seems like we could implement the old protocol completely in occam-pi.

And that’s what Carl and Jon have done. They’ve implemented what used to be firmware as a program that any user can now write and upload to the Surveyor. The program srv1.occ can be compiled, uploaded to the Surveyor, and executed as a user program, even though it is implementing the entire SRV-1 protocol. It is now a “user program” that implements what I’ve referred to as the “old firmware.” If we want to kill this “new firmware,” we issue a ‘!’, and it is shut down cleanly. Now, we can upload a newer version, or perhaps a completely different program.

This has its tradeoffs. For example, it means that my SRV-1 doesn’t wake up ready to send me images. On the other hand, it does mean that I can easily modify and extend the firmware, including extending the protocol or (because occam-pi is a parallel-safe language) running my own additional code along with the original firmware. I can filter the channel carrying the commands (perhaps ignoring every other request for an image, or drawing on the images to add data to them), and so on. Over time, we’ll probably end up refactoring the “firmware” into a bunch of reusable components that program authors can selectively include in their programs to get parts of the original firmware’s behavior in their programs. (We’ll see… I haven’t given this much thought yet, but perhaps Carl and Jon have.)

This is still evolving rapidly, but it is an absolute joy to be able to easily modify my occam-pi code, send it over the WiFi to the Surveyor, and get completely new, low-level behavior without having to go through a lengthy reflashing of the bot over JTAG (or, worse, WiFi).

As can be seen above (sorta), I’ve begun to explore drawing on the images before shipping them from the SRV-1 back to the user. I believe this is important for students exploring robotic vision, as they need a way to indicate what they are looking for; drawing onto the image strikes me as a very simple way for their code to communicate that “this looks important!”. It might be that they’re doing edge detection, or blob finding, or any of a host of other things. My code doesn’t do anything exciting yet, but tomorrow is another day; more excitement will ensue.

Nicely done to the UK team! Wootness.

I just wanted to say that Roborealm looks like a very nicely done set of tools for exploring robotics and vision. I haven’t had a chance to play with it yet, but it is something I’d like to do.

The team is nearing completion of a new firmware for the SRV-1 that provides low-level, low-latency parallelism on the Blackfin-based Surveyor. Some excellent work has been done by Carl and Jon on this port, and I’m very excited to see it in the hands of students this semester.

When we release that documentation, I’ll also release the tutorial booklet that students used on the LEGO Mindstorms at the start of the term. It provided a questions-first introduction to occam-pi on the Mindstorms RCX. These documents, taken together (and iterated/revised a bit) will provide, I think, an excellent introduction to the design of concurrent and parallel algorithms for robotic control on some very cool platforms.

Watch this space!

The last post on this blog was in October. Since then, things have been busy. Over the next few posts, I’ll try and catch up a bit.

For the last three months, we’ve been working hard on the new Blackfin Surveyor SRV-1.

Blackfin SRV-1, image stolen from the Surveyor webpages

We first encountered the Surveyor at AAAI 2007. When Howard “Surveyor Daddy” Gordon updated the Surveyor from a 60MHz ARM with a few KB of RAM to a 500MHz Blackfin with 32MB of RAM, we took notice. And, even better, when it sprouted that oh-so-excellent WiFi tail, we really took notice. (It is so much easier to work with WiFi than the ZigBee radio that was there. At least for us it is.)

This semester, I’m co-teaching ENGR 3390: Robotics with Dave Barrett at Olin College, and we’re using the Surveyor as the platform for the mobile portion of the course. We’ll be using the Surveyor as a platform as a demonstrator where we can introduce students to architectures for cognition on robotic platforms in a parallel-safe way using the Transterpreter and occam-pi. And what we realized very quickly is that the SRV-1 looks like it is just begging to be beaten all to hell in the classroom.

See that processor on top, all nice and exposed? After a team of students shuffles in from the dry, Boston winter—BZZAPT!. No more Blackfin. One good static discharge, and the processor or RAM are gone. Or, those little laser diode wires? Oops! A careless gesture, and they’re yanked out. Or what about a drop from the counter-top? I’m not confident the board would handle the fall very gracefully.

Dave threw together a quick Solidworks model, and dumped it out to Olin’s rapid prototyping machine. This thing is incredible: it prints in 3D in ABS plastic. Send the model, step back, and 12 hours later you have this:

That is a battle-ready SRV-1. The laser mount was removed, and the diodes inserted into the body, which was simply printed with the diode mounts in the right place. The camera hole is oversize, and easily gives the camera a full field of view.

As you can see, the rapid prototyping machine has enough resolution to render depressed text in the body of the bot. This one clearly declares itself as property of Olin’s ENGR 3390: Robotics course.

What you may or may not be able to tell is that the entire top of the box has mounting holes at 1/2″ spacing perforating it; it is amazing what the RP machine can do. This means we can easily screw-mount sensor towers or anything else we might like to the top of the bot quickly and easily, running wires straight down through the body to the SRV-1 board.

Generally, the pictures are low quality because they were taken with the webcam built into my Macbook. I’ll get better pictures at a later point.

The first thing we’re going to have the students in the course do is take the basic Solidworks model and design bodies for their team’s bots. This way, each bot will have a unique body, and the students will have some ownership of the process. However, we already have a few things we want to improve in the basic body before we turn them loose:

• The body doesn’t have anywhere to grab on to in the back; as you can see, it is just resting on the SRV-1’s aluminum body. Our plan is to wrap around inside of the treads, where there is just enough clearance for us to sneak the body down. This way, we provide more protection for the boards from grit and the like, and can attach across the base of the bot on the underside.
• We need blinkenlight mounting holes build into the body. This way, you can just push an LED up from underneath, and it will sit flush with the housing. We can wire them into the GPIO pins inside, and have all the wires hidden from view, and safe from snags on obstacles.
• A view hole needs to be left for viewing the state of the three LEDs on the Blackfin’s motherboard.
• A mounting point for a header board could be built into the body, so sensors could be attached and detached outside; this depends on whether we expect to be doing much of this kind of prototyping with different sensors or not.
• Mount points flush with the body wall for IR and other sensors on the front/back/sides could be very useful. We have a number of interesting sensors for the students to characterize and use, and mounting them safely inside the body would be a good idea.
• The antenna should probably be above the ground plane; we could run a cable from the current point, under the body, and up to the top, where we could easily grow an antenna mount.
• We have blue and yellow plastic coming in, which will produce a lasting color. Otherwise, we have to paint, which is less durable and a mess to work with. (Well, as messy as spray paint ever is, anyway.)
• I want my robot’s body covered in dimples (and maybe some through mounting holes) that are compatible with a certain plastic building toy that rhymes with Tierra del Fuego.

Those were the immediate improvements that we saw could be made.

We’re really excited about the course, and I think both Dave and I are pretty excited about how cool the bodies for the Surveyorcould be. What’s really awesome is that they are printed in ABS plastic, and have a wall thickness of roughly 3/16″ of an inch. (I haven’t measured; that’s just my eyeball estimate from memory.) I pounded on it with my fist (off the Surveyor), and I hurt my knuckles. In short, the body is tough. The ultimate goal is to make it possible for a student, in their excitement, to give the Surveyor a kick on the field of combat (or drive it off a table), and have the bot be in a good condition to survive the fall.

Over the next two weeks, this body will evolve a lot. We’re going to be playing with it, and then 24 of Olin’s energetic, creative engineers-to-be are going to rapid-fire evolve it to their hearts content. I think we’ll end up with something very, very cool. I’ll post pictures and Solidworks models as the body evolves.

Some time ago (but only a few posts ago), I ranted about the state of embedded systems development under Linux. If we break the rant down, it basically read like this:

How come I don’t have carrier-grade compilers for every single chip under the sun for free? This is ridiculous!

The truth is, these tools have a cost. GCC is a great resource, but the number of platforms that can be realistically supported through gratis development is actually, quite likely, small. One way that it could be done is if every hardware manufacturer employed one or more GCC developers to maintain back-ends for their CPUs. Perhaps some manufacturers do just that.

However, they’d also have to maintain drivers for the programmers that connect to their chips, and make sure that you could debug the hardware from GDB on Windows, Mac, and Linux—which is a non-trivial amount of work. So, now you’re maintaining a toolchain that goes from the compiler, to the linker, to the driver for the programmer… and there’s probably a standard library to maintain as well.

What I’m starting to see (from being in an engineering school) is that people don’t always have the mentality that they can build things themselves… when it comes to software. Because I wrote a linker for a weird little instruction set, and helped write a bytecode interpreter for that instruction set, II have a different perspective regarding the authoring of software than, say, your typical mechanical engineer. I have no real way to compare, but I suspect I sit down and think about the design and implementation of compilers or distributed systems in much the same way they think about… drivetrains and … moving… things. Or something.

I’ve been wrestling with getting the Transterpreter compiled for a particular MSP430 part. You see, there are a number of parts in the MSP430 family that GCC supports very well—and we have compiled and executed occam-pi programs on those CPUs just fine. But the development board I purchased for $120 has a CPU that is… less-well supported. As in, I need to build a patched version of binutils to get things working. I spent the better part of six hours last weekend debugging what was going wrong and attempting to get things running, and finally realized that my time was worth more than the cost of a tool.

Rowley Associates produce the Crossworks line of compilers for a number of embedded processors, including the MSP430 family of devices. Lets look at their prices:

Right now, I’m too busy to chase after broken builds of binutils. Six hours of my time, valued at $25/hour, is the cost of a compiler that handles the CPU I’m trying to build for, as well as many others. And not only that, but the IDE and tools will work on Windows and Linux… and soon my Mac.

In other words, I get the IDE on Windows, Mac, and Linux for $149. My problems of getting my software to compile on the MSP430FG4619 simply go away. Does this help the next person who comes along and wants to run the Transterpreter on this chip? No. However, they’ll have the same options I currently have:

1. Get the GCC-based tools to work
2. Buy the compiler

If I had infinite time, I would get the GCC toolchain working. But I don’t. I just want the devboard to work, so I can test and demo our VM on a small platform. And because other projects I’m involved in leverage the MSP430, the compiler won’t hurt there, either.

So, as soon as I get another hour or two free, I’m buying the Crossworks tools. I’m no longer a grad student with infinite time—and this is a battle I can afford to win with cash.

I thought it might be worth mentioning (in light of my rants) that it is possible to use the Softbaugh USBP in a Windows XP VM running under VMWare Fusion—that is, on a MacBook.

I was running into problems with VMWare Fusion 1.0beta4. However, I missed the recent upgrade to v1.0 “Gold Master”, or (if you prefer), the final release of version one. In upgrading, Windows XP (running in the VM) correctly found the USBP, and as a result, I can now get on with programming MSP430-based devices on my MacBook. (I think; I have’t actually tried loading code to a device yet, but Windows sees the programming dongle, and I think that’s the important step. I expect things to work at this point.)

The picture above is the Softbaugh PrgUSBP application running in VMWare Fusion’s “Unity” mode. In this mode, the Windows app plays along with OSX as if it were a full-fledged Mac application. This way, I can Apple-TAB back-and-forth between my editor and the programmer, and not have to keep flipping back to the full VM.

As it stands, I think it is sad that embedded programming with chips like the MSP430, ARM9, and other interesting devices is so damn expensive. The programmer was $150, my dev board was $115, VMWare Fusion is $80, and XP… I can’t remember. Granted, many of these things were (in my case) paid by grants, but for a hobbyist, $300+ is rather pricey.

I’d love it if we had a way of programming these chips cheaply ($50 or less), and it worked on every platform. Perhaps, someday, I’ll make that a mission. It would have to be a mission of love, because my guess is that there isn’t enough money in these tools to make it up in volume.

I continue to be frustrated by the state of embedded systems programming.

If I want to program an ARM9, I need a special programmer. If I want to program an MSP430, it will require a special programmer. The list goes on and on, for each new chip that I think I’d like to develop code for; the cost is prohibitive, and in each case, it is likely that the tools:

1. Only work under Windows, or
2. Work under Linux if you have a parallel port.

I don’t know about you, but my MacBook doesn’t have a parallel port. And Windows running under VMWare doesn’t always recognize these little programming dongles. And really, I don’t want to use Windows or Linux at all. I want to use my Mac.

I’m thinking that I need a sub-$300 device (sub $100 would be ideal) that can program any small device, whether it is the new Blackfin processor or a crappy little PIC. This implies a software solution, with some mix-n-match cabling coming out of the box.

I’m going to start thinking about this a bit more, because it is a genuine problem in the embedded space. It represents the worst kind of vendor lock-in, and makes my life as an educator and hobbyist a complete nightmare. I need a solution that:

1. Works with any embedded device (via software update, if necessary)
2. Works with any computing platform (Mac, Linux, Windows)
3. Is cheap and portable

In the end, the solution is probably called a “LEGO Mindstorms NXT”, and I should just give up on embedded platforms. But I’m not prepared for that yet.

Along with sketching out an explicit group development process, we discussed several other aspects of our project’s organization. As these were relatively straight-forward things, I’ve grouped them all here.

Wrappers

A ‘wrapper’ is what makes it possible to run the Transterpreter on a new platform. That is, cross-compiling the interpreter for a new architecture is boring and easy. Connecting up the language to the hardware (or underlying operating system) is the interesting bit, and the wrapper is where we do that work.

Up until now, we have typically done wrapper development in branches. However, this is difficult, as it means each wrapper developer must constantly be merging the trunk back into their branch. This is silly, as most of our wrappers we want to remain current, and take advantage of any improvements in the trunk.

So, moving forward, we’re going to try and keep wrapper development in the trunk. Because wrappers are mutually exclusive, we don’t have to worry about one adversely influencing the other, and it will make it easier for us to make sure that changes that help one branch doesn’t hurt another (eg. using buildbot and cross compilers).

This requires us to be a bit more vigilant about…

Tagging

Up until now, we’ve been rather lazy about tagging. As a group, we need to start tagging more often. This might be by deciding on features that should go into milestones, or perhaps through an ad-hoc democratic process, where someone suggests that we tag, and the group agrees (or disagrees) that it is a good time to do so.

This is an area that requires more discussion, but it is clear that the project could do with more versioning and tagging, providing snapshots of known good/stable points.

Libraries

Adam has been doing a lot of work at making libraries for occam-pi cross-runtime buildable. By this, I mean that we have a situation where we have two runtimes (CCSP and the Transterpreter), each which are capable of different things. For example, the interpreter will always be smaller, and therefore more portable to more (embedded) platforms. Unfortunately, if I had developed a C library with occam-pi bindings, it was virtually impossible to use it from both the Transterpreter and the CCSP (KRoC) runtime.

Adam’s work has brought us into a world where we can easily build such libraries for either runtime from the same source with minimal effort. This is excellent, and we’ll be converting all of our libraries to use this new build system over time. Which, brings me to the last bit…

Build systems

Some time ago, I explored the use of Scons as a build system. It represents a spike, and can build the Transterpreter for Linux, Mac (PPC and Intel), the MSP430, and Windows.

However, it’s a bit ugly. It represents my first time working with Scons, and it could be better. Also, we discovered that mingw provides an environment that lets us easily build everything using our Autotools-based environment. So, whereas we thought the Scons build would improve things across the board, it’s not clear that it’s a 100% necessary improvement.

Either way, we’re going to bring Scons into the trunk, and play with it. Because it is completely orthogonal to the Autoconf/Automake build system, it can’t hurt. However, we’re going to be looking at this closely over the next few months, as one of them has to go, and we definitely want our build system to be cleaner/better organized/simplified greatly. If that means using Scons, great. If it means revamping our existing system, great.

Up next…

Those are reasonably straight-forward changes coming down the pipe; as a side effect of these things, we’ll likely do some branch cleanup and repos reorganization. As the time comes for these things, they’ll be discussed on the development list.

The last thing I’ll talk about is a project mission statement and project goals. That’ll come later this week.

So, the first of our diaspora updates involves making our development process more explicit. While our group has, largely, worked this way already, it hasn’t been explicit. This had a lot to do with the fact that the entire team was housed in the same suite of offices at the University of Kent. Now, things have changed.

The first document I want to introduce you to is our development process. We’ve attempted to keep this as short and simple as possible. We want just enough process to keep us moving forward harmoniously, but not so much that developing on the Transterpreter is notfun. Given that we hope that students interested in concurrency, embedded systems, and other cool stuff like that will want to get involved, we also wanted the process to be heavy enough to provide some structure and be pedagogically valuable, but light enough so as not to get in the way of (say) an enjoyable final-year project.

So take a look, and join the development mailing list if you want to discuss it.