The Arboretum is an experiment in developing scaleable paradigms for high-level, parallel, multi-media applications using open-source tools and commodity-off-the-shelf (COTS) hardware.
The Arboretum is a network of 12 Linux-based computers, connected by a network. It's that simple. Like your school's computer lab or a bunch of laptops in a LAN party. Or... a microcosm of the internet?
Let's break it down. In our context "Scaleable" means that we should be able to add resources both vertically and horizontally (faster computers and more computers, respectively) to our network and get increased throughput correlated to those added resources. Or, more simply, our system shouldn't break if we make it bigger or faster.
"High-level" means that we shouldn't have to all have computer programming degrees from Carnegie Mellon or MIT in order to use the system. Parallel computing platforms like MPI or PVM require either or both a deep understanding of the C programming language and a long, complicated setup. We don't expect our users to have either because, frankly, neither do we. We're not going to breaking any performance records here, but the entry point is more accessable and the learning curve is much gentler. In other words, "super-computing made easy."
I've already used the word "parallel" a few time on this page. I felt you getting nervous, so I'll explain. What The Arboretum is designed to do is to use many computers to do one job. The idea (without getting technical) is this: some tasks (but not all of them) can be accomplished in less time if you divide the work among multiple workers (or computers in our case). I said that not all tasks are like this; here is an example illustrating the difference: I can make 100 paper airplanes in 100 minutes at 1 plane/minute. But if I give 10 people 10 planes to make, we can get the job done in (about) a tenth of the time it would take me to do it alone. This task is "parallelizable." It is not the case, however, that 9 women can make 1 baby in 1 month. This task cannot be accomplished in a parallel fashion due to the fact that it takes a baby nine months or so in the womb and can only be in one womb at a time. (Don't think about that last point too hard.)
The first task--the paper airplanes--is actually called an embarassingly parallel problem because it can be done completely in parallel, since no part of the process depends on another part to be finished before it can begin. There are many types of tasks that are somewhere in between the two examples, however, the embarassingly parallel ones work best on systems like The Arboretum.
The one term that is actually a bit of stretch is "multi-media," believe it or not. In order padding-left: 1px;to not overlap with other work being done at ITP, we're only focusing on rendering sound in this prototype stage. The system we're building is, technically speaking, equally suited for graphics work, however, there are many projects like this dealing with graphics and very few designed to work with sound alone. We feel that this is a space that needs further exploration and so we are limiting the scope of The Aboretum to sound at this stage.
The Arboretum is being designed to explore the use of open source audio programming platforms like Chuck and SuperCollider on dedicated networks (or clusters) using Open Sound Control (OSC) as a communication protocol between running processes. The use of OSC is a key distinguishing feature of this system. From an engineer's perspective this is an unnecessarily heavy and bloated way to communicate over a network and is a bit like using a truck to deliver a letter to a friend when you could have just called them. Nevertheless, the benefits of using a OSC are analagous to the idea that the truck can reach places where telephone lines don't exist and the letter is written in a language that everyone understands. OSC is implemented in numerous languages and embedded in many software and hardware applications and is sent over standard UDP network ports. This means that I can send OSC packets from a keyboard in New York to synthesizer in Tokyo--if there's internet, it will get there.
The Arboretum is essentially a network of machines that provide audio synthesis services on a local network which communicate using the OSC protocol (where it makes sense to do so). There are 5 nodes on the network whose main job is make sound--either pumping it in as close to realtime as possible out their sound cards, or rendering uncompressed wave files in a non-realtime process. These are the strongest of the client nodes and accomplish this task using the SuperCollider and Chuck audio programming languages. Another 7 client nodes keep track of mathematics or other distributed tasks not directly related to audio synthesis. These tasks may range from setting up of datasets or number-crunching tasks to running processes that manage the core synthesis servers. There is one node that is dedicated to providing data services for the rest of the cluster. The data node is both a database server and a file server (NFS). Finally, there is the head node, which is the command center for the entire cluster. When you sit down to work with The Arboretum, you are working from the head node. The idea is to make the head node the locus of abstraction for the system as a whole. What that means is that a user should not have to worry about communication with the other nodes on the cluster because the head node should take care of that communication for the user.
Our plan is to implement a series of interfaces that will allow musicians to play the Arboretum like one giant instrument. The musical metaphor appropriate here is one of treating the code (and the network itself) like a Gamelan instrumentl; many people will play the same instrument--many people interacting to create one "music."
You want to make your own cluster? Great! Keep reading our blog. There we will post not only news about our progress, but source code, instructions, and tips for building your own cluster. We're all about "free" in all senses of the word--as in "free speech", "free beer", and "freedom". Feel free to email us as well: arboretum.info (at) gmail (dot) com.