I ended up in graphics through an interest in artificial intelligence and computer learning. My first major AI project was based on the local traditions at the University I went to back in Canada. AI and teamwork between autonomous virtual characters featured heavily in the game, and that led me to doing graduate work at the Media Lab, where we were making synthetic characters – again, a healthy mix of graphics and AI.

It’s a research lab in Dublin, Ireland affiliated with the MIT Media Lab in Boston. MindGames is one of the research groups at MLE, and they focus on intelligent biofeedback. The goal of the group is to add hefty doses of AI to the biofeedback loop, with the aim of designing new types of interfaces, and teaching skills in the virtual world that people can transfer to the real world.

When I joined the MindGames group, the team was brimming with ideas for projects they wanted to implement – everything from brain-computer interfaces to giant, wall-sized “magic mirrors.” But there were only a few of us on the team with development backgrounds – much less graphics development experience, which is pretty niche. So Symphony grew out of a need to empower the team, and give people with diverse backgrounds the platform and tools they needed to rapidly develop and deploy the different projects, sometimes to clinical, physical rehab and even performing arts environments.

Ease of development, ease of integration with the rest of our architecture which was written in C#

Less risk is arguable. Graphics programming is still a bit of a dark art, and discovering the interfaces available on an object like Direct3D.Device is a lot easier using Managed DX. Working with the CLR (Common Language Runtime) reduced the risk that developers who hadn’t previously dabbled with 3D were going to introduce problems like memory leaks. And a lot of the resource management could get tucked away under the hood of Symphony. And the C# syntax is (in my opinion) a lot more elegant and intuitive than the C++ syntax.

The first version of Symphony’s graphics engine actually used the very excellent CsGL library. We ended up migrating to Managed DirectX for a number of reasons, mostly around the pixel and vertex shader support.

In addition to the above, there was also the interop story. It was very easy to integrate with legacy signal acquisition code.

We also did some work with the Compact Framework using mobile devices (such as the iPaq). We were able to leverage a lot of the code and create a 2D graphics engine that could talk to the full-blown Symphony.

And speaking of which – having access to the .Net Framework libraries made things like network communications a breeze. We used technologies ranging from Bluetooth wireless to Web services. And bear in mind we never had a dedicated networking programmer. .Net made everything incredibly straightforward for us.

For a more thorough answer to this question, check out the symphony website and the CodeZone magazine article about Symphony.

No. Not at all. I don’t think anyone (myself included) expected the managed engine to work as well as it did. The stat they give is that ManagedDX offers 98% of the speed for 50% of the code. It’s impossible for me to know at this point if it really was 50% less code, but we sure had some full-throttle speed.

As a specific example, we were consistently pulling off 60fps with a scene of about 40,000 polys that included a skinned mesh character. It was fun to be able to tell our artists to "go nuts" and really mean it.

One thing I have to say: Managed DirectX is pretty nascent, and breaking changes to the API (of some significance!) have been made with every release I’ve worked with. It was frustrating to be continually updating the version of Managed DirectX on every PC – and updating my skinned mesh code, and so on.

Yes and no. Still Life, a project that featured a particle system, would create and destroy thousands of objects a second. The CLR handled this egregious abuse of memory management without any hiccups.

However, it wasn’t always clear when some reference was being kept to an object, somewhere under the hood of Managed DirectX, and therefore that object wasn’t being garbage collected when we hoped it would. That was the biggest problem we had: trying to be sure that objects we wanted to be collected were being collected.

In the academic environment we were working in, I didn’t worry too much about these issues. We did have a specific incident when one member at the lab distributed a solution to an outside party, and I insisted that the Assemblies be obfuscated. I am aware of the issues involved but not particularly worried. I think obfuscation goes a long way. The benefits of using a Managed solution really are (1) improved end-user experience through (2) increased developer productivity. Someone using a decompiled Assembly isn’t likely to be more productive unless they also have documentation, and aren’t wasting their time reverse-compiling something they’re not meant to.

I would perhaps be expressing more concern if we had particular algorithms that represented valuable, unpublished IP. In an academic environment, we’re meant to publish these, rather than have them hacked out of our code.

I don’t have any complaint about installing the .Net framework and DirectX. That’s not a hassle. Most people have .Net installed now anyway.

I look forward to the ClickOnce deployment technique in the next release of Visual Studio (2005). It’s like something straight off of my wishlist from Symphony days. It will make it very easy to manage multiple deployments of software. It also handles updates and dependencies (like the .Net framework).

I don’t think so. It’s just that it was easier to implement all those boxes and lines that we drew on the architecture diagrams ;)

The base objects for key data structures are shared across very different systems (signal processing, scenegraph, and DX resource management, for example). We got huge wins from being able to use reflection and metadata (esp. Attribute tags) to create visualizers that would work across Symphony.

Rather than share a snippet I’ll tell you about a cool technique. Reflection, as a technique, is likely to be too slow to use a lot during your main update loop. But when initializing objects at start-up time, it’s a whole different story.

A little reflection went a long way toward making it possible to assign vertex information to vertices that had an arbitrary VertexFormat. When initializing a VertexBuffer, we had a provider that was able to give position, normal, UV coord info, etc., for each vertex. But we wanted to be able to decide at run-time the specifics of the vertex format. By reflecting on the vertex type, we were able to figure out what information we needed to pull out of the provider by looking at the fields in the VertexFormat Type.

This was very cool stuff – instead of having to special-case every possible kind of vertex, it would work with whatever we threw at it. Which was extremely useful when we were experimenting with shaders.

The first Symphony continues to be used by several research groups at MediaLabEurope.

After leaving MediaLabEurope, I spent the summer building a sequel (from scratch) to Symphony. It represented a significant improvement, and it learned (a lot) from the lessons of the first Symphony. I thought seriously about releasing it for use by other developers. However, my career path has taken a slightly different turn at this point, and Second Symphony is on the shelf for now. I’d love to see it used by other developers, so… we’ll see what happens :)