For anyone who is interested in what LMAX is up to, and is still following my blog, have a look at this post about the latest tool they’ve open sourced: the Coalescing Ring Buffer.

If you see anything about LMAX - the Disruptor, Continuous Delivery, or even the selection criteria for hiring developers, you’ll see that LMAX is pretty keen on Agile. However, no-one’s documented the Agile process there, as far as I know. Although I personally had it on my todo list, I never had the motivation, the hook to do it. And I realised eventually that’s because I’m not sure it’s a process that would work very well for another team, in another company, working in another business.

Devoxx 2012 from Roy van Rijn on Vimeo.

Devoxx topped off a crazy two months of conferences. I’ve heard people talk about the conference season in the past, and been slightly (OK, very) jealous of all that jet-setting. I’ll admit, however, to a slight feeling of relief that my focus until Christmas is pretty much going to be coding. I hope.

I’m late with my write-up of QCon, and what’s worse, it will be partial - “sadly” I was in Lanzarote on a training week with the running club from the Thursday (8th) so I missed most of it. A sacrifice I had to make for 7 days in the sunshine…

When you’re applying for a job at LMAX, your CV (or résumé, for our American readers) usually comes through me and I decide whether to call you for a technical phone screen.

I’m going to let you into a secret.

“What… exactly… were you guys looking to get out of today’s event? Because…” “Because we’re girls?” “Um… yes…” Kim impetuously opts for The Truth: “We’re here to meet men.” Our interrogator looks round dubiously. “No, really, why are you here?” Phew. My reputation is intact1 Kim eloquently describes what her situation is as Product Manager and the criteria she’s measuring third party products against. I explain how LMAX aims to be the fastest retail exchange in the world, and therefore low latency is a tiny bit important to us.

Here’s a photo of me representing LMAX as I pick up the Duke Award we won for the Disruptor (Thanks to Martijn for taking the photo)

So I’ve been at JavaOne for the better part of three days, it’s time to record some of my observations so far:

So, I’m off to JavaOne next week! This is an unexpected and very pleasant surprise. I’ll be there with Martin (of the Disruptor fame), and Martijn (that’s not going to get confusing at all). Martin will be talking about the Disruptor on Thursday, and Martijn is busy talking about… everything. Most importantly for the LJC he’s representing us in our shiny new JCP Executive Committee role. I’m really looking forward to meeting pretty much anyone and everyone who’ll talk to me.

My recent slow-down in posting is because I’ve been trying to write a post explaining memory barriers and their applicability in the Disruptor. The problem is, no matter how much I read and no matter how many times I ask the ever-patient Martin and Mike questions trying to clarify some point, I just don’t intuitively grasp the subject. I guess I don’t have the deep background knowledge required to fully understand.

So, rather than make an idiot of myself trying to explain something I don’t really get, I’m going to try and cover, at an abstract / massive-simplification level, what I do understand in the area. Martin has written a post going into memory barriers in some detail, so hopefully I can get away with skimming the subject.

We mention the phrase Mechanical Sympathy quite a lot, in fact it’s even Martin’s blog title. It’s about understanding how the underlying hardware operates and programming in a way that works with that, not against it.

We get a number of comments and questions about the mysterious cache line padding in the RingBuffer, and I referred to it in the last post. Since this lends itself to pretty pictures, it’s the next thing I thought I would tackle.

Martin Fowler has written a really good article describing not only the Disruptor, but also how it fits into the architecture at LMAX. This gives some of the context that has been missing so far, but the most frequently asked question is still “What is the Disruptor?”.

I’m working up to answering that. I’m currently on question number two: “Why is it so fast?”.

So now I’ve covered the ring buffer itself, reading from it and writing to it.

Logically the next thing to do is to wire everything up together.

I talked about multiple producers - they have the producer barrier to keep them in order and under control. I’ve talked about consumers in a simple situation. Multiple consumers can get a little more involved. We’ve done some clever stuff to allow the consumers to be dependent on each other and the ring buffer. Like a lot of applications, we have a pipeline of things that need to happen before we can actually get on with the business logic - for example, we need to make sure the messages have been journalled to disk before we can do anything.

The Disruptor paper and the performance tests cover some basic configurations that you might want. I’m going to go over the most interesting one, mostly because I needed the practice with the graphics tablet.

This is the missing piece in the end-to-end view of the Disruptor. Brace yourselves, it’s quite long. But I decided to keep it in a single blog so you could have the context in one place.

The important areas are: not wrapping the ring; informing the consumers; batching for producers; and how multiple producers work.

The next in the series of understanding the Disruptor pattern developed at LMAX.

After the last post we all understand ring buffers and how awesome they are. Unfortunately for you, I have not said anything about how to actually populate them or read from them when you’re using the Disruptor.

Recently we open sourced the LMAX Disruptor, the key to what makes our exchange so fast. Why did we open source it? Well, we’ve realised that conventional wisdom around high performance programming is… a bit wrong. We’ve come up with a better, faster way to share data between threads, and it would be selfish not to share it with the world. Plus it makes us look dead clever.

On the site you can download a technical article explaining what the Disruptor is and why it’s so clever and fast. I even get a writing credit on it, which is gratifying when all I really did is insert commas and re-phrase sentences I didn’t understand.

However, I find the whole thing a bit much to digest all at once, so I’m going to explain it in smaller pieces, as suits my NADD audience.

First up - the ring buffer. Initially I was under the impression the Disruptor was just the ring buffer. But I’ve come to realise that while this data structure is at the heart of the pattern, the clever bit about the Disruptor is controlling access to it.

On Wednesday I tagged along to the STAC London Summit to provide backup for Mike, who was on the “The Future of Messaging Middleware” panel.

The panel consisted of two messaging providers, one hardware (Solace Systems) and one software (29West/Informatica), and two “users”, Citihub and LMAX. Obviously both providers were arguing that theirs was the best solution. But what I found interesting is that I came away with the impression that everyone was really on the same side - everyone wants to use or to provide the best system, but there are different approaches. Which one you adopt is likely to be influenced by how your team work and the hardware you have (or can obtain).

I’m honoured to now be listed on three aggregated blog feeds: The new and somewhat embryonic blogs.lmax.com. The London Java Community Aggrity. ThoughtWorks Alumni, along with a whole host of distinguished names. The pressure’s on to try and write useful stuff. Oh OK, I’ll settle for “entertaining”.

I have been struggling with .NET. Actually, I have been fighting pitched battles with it.

All I want to do is take our existing Java client example code and write an equivalent in C#. Easy, right?

Trisha’s Guide to Converting Java to C#

Turns out writing the actual C# is relatively straightforward. Putting to one side the question of writing optimal code (these are very basic samples after all), to get the examples to compile and run was a simple process:

I attended TradeTech last week, an annual event about Equities and Derivatives trading. I assumed from the title that there would be a reasonable focus on technology, but I found it was more “Trade” and less “Tech”.

At LMAX we adopted Google Web Toolkit pretty early on. One of the motivations for using it was so we only had to worry about recruiting Java guys, and then we could all work on every part of the application including the web UI. Sure, you can learn a bunch of different skills if you want to, but it reduced context-switching and kept the skill set we were hiring for to a nice short list.

We went live with ‘real’ customers this week just gone. It’s the culmination of nearly two years work for me personally, and three years for our company. It’s really nice to be live at last, and to have our name out there. It might (in fact, should) change the focus of our work. Without paying customers it’s much more difficult to prioritise work based on what they might need or want.