Featuring...


So, your team has worked hard on designing a high-performance software system. One of the key components is a thread or service designed to buffer and protect the application from slow operations by caching the response of lookups from slower or more distant systems (like frequently used credential data or permissions, for example that might be coming from database servers).

On the test-bench, everything looks good and then the code goes into production. Everything scales nicely up to peak loads, and then the whole system begins to stall and stutter. Performance ratchets in a staccato sinusoid, working just fine here, and coming to an almost complete halt there. After a while, as overall load falls and end-users become frustrated, it all sorts itself out and runs smoothly, until just after the next peak load.

You direct the team to investigate, to profile code, to monitor logs and performance. Nobody can find a link. Your engineers come up with plans to optimize the cache service and eke every microsecond of performance out of it.

Mysteriously, the problem just gets worse, not better despite thousands of hours and weeks or months of investigation, and tuning. Someone discovers that disabling hyperthreading on the server mitigates the problem significantly, but nobody’s closer to a solution.

Congratulations, you’ve hit a common, but very rarely understood problem in high-performance systems design. It’s bitten almost every high-performance systems shop, yet almost nobody has truly solved the problem, because hardly anyone has understood the true cause at the time. Most engineers end up working around it, because they never quite know where to look.

Let’s save the day and tell your team where to look. It might not be the source of your particular problem, but ruling it out early can save you tens of thousands of dollars in development, and far more than that in frustrated customers.

Read the rest of this entry »


Jump to the new comic, or new readers can click the banner to begin at the rather rough beginning:

Spirit

By: Tateru Nino

The “Spirit” rover on Mars was designed to last just 90 days in some of the harshest conditions we’ve ever dumped equipment into. After 2,213 days Spirit has finally had to be declared as a stationary research station, and is expected to maybe last a few more months until the accretion of sand prevents it getting power from its solar panels.

It’s both a sad and a proud moment.

Read the rest of this entry »

An interesting shift

By: Tateru Nino

From some perspectives, every gadget, tool, and device has a narrative; a story of how it fits (or fails to fit) into your hand, pocket, purse, shed and/or life.

What I find interesting is the narrative shift in the last few years to gadgets whose computing power must be continually paid for.

Read the rest of this entry »

New comic: Espionage

By: Tateru Nino

Jump to the new comic, or new readers can click the banner to begin at the rather rough beginning:

The bomb

By: Tateru Nino

Earlier this month Paul Chambers was arrested, banned from Robin Hood Airport in Doncaster and suspended from his job. Why? He jokingly twittered a bomb threat.

Has the world gone mad? Apparently some of it has.

Read the rest of this entry »

Template refit

By: Tateru Nino

Well, it’s been a bit of a difficult slog, but I’ve made a slew of changes to the blog template to allow for the site to be better-rendered by a variety of browsers.

Read the rest of this entry »

The golden handshake

By: Tateru Nino

This is great. Really great stuff.

Read the rest of this entry »

A moment of clarity

By: Tateru Nino

It’s going back a few years, but the company I was working for back then sent me to a Windows NT 4.0 launch event. A bunch of us went, though it didn’t really have a lot to do with us, but there were invitations, plus free food and swag at the venue.

Read the rest of this entry »


Your Ad Here
  • Support us

    Writing is my day job. Site advertising pays for the hosting, but nothing else. Help keep us in coffee and keyboards

    ... or donate in Second Life at this location.

  • NCI - free education and information for new Second Life users