CODEWITHSUNDEEP
fault-tolerancehp-nonstoptandem
ddimitrov
ddimitrov

Reputation: 3343

How does HP/Tandem NonStop achieve single failure FT without spares?

As far as I could gather from Wikipedia and the mindboggling HPE website, the claim to fame of the NonStop system architecture is that it can achieve a single-failure FT without having to allocate excessive amounts of spare capacity (i.e. in lockstepped architecture you would typically need to overprovision by 3x).

This seems a desirable property, yet I couldn't find more details about the approach they use and the caveats. I.e. what are the assumptions they make about the network, the kind of failures they tolerate, assumed client behavior, the acceptable time to recover, the workflows they run, etc.

Could anybody describe in brief how does the NonStop system solve the typical problems with failure detection and failure correction? Is it a generic magical solution on system level, or does it require that the applications are written to use certain transaction facilities and checkpoint data and communications?

Thanks a lot!

Upvotes: 3

Views: 787

Answers (4)

Tristan Fabrini
Tristan Fabrini

Reputation: 68

I developed apps on Tandem systems in 1980s and early 1990s. The Tandem NonStop hardware was essentially tightly multiplexed processors and storage in a single chassis. This could be (and often was) scaled up by adding more chassis. The HW approach is quite interesting in itself, see for example https://www.hpl.hp.com/techreports/tandem/TR-86.2.pdf

However, application-level fault tolerance relied on an API for certain Guardian operating system services. This had to be configured and invoked from application code. Roughly, when an app started in the primary machine, it would request creation of a hot stand-by back-up process on the second machine. Once all this was spun up, the primary app would call the API to take a checkpoint, which would copy the entire state of the primary's process space to the secondary process. Disk was shared and replicated - a forerunner of now commonplace RAIDs. The secondary's OS instance looked for a heartbeat from the primary - if it didn't hear that, it would take over, assuming the either or both the primary's HW or SW was dead. This thumbnail isn't the whole story of course. For details see "Software" in https://pages.cs.wisc.edu/~remzi/Classes/739/Fall2015/Papers/tandem-TR-90.pdf

This was how ultra-high availability and scalability was achieved for apps running on Tandem stacks.

But, using Guardian services was not easy. In addition to developing app feature code, it required understanding how each Guardian API call supported the fault-tolerance strategy, and then thinking through when and what to checkpoint, and how to deal with restart edge cases. Devising adequate test suites was a puzzle. All this increased the time, cost, and difficulty of development and testing. A lot of shops thought they were buying all this out of the box (Tandem systems were significantly more expensive than other mid-range systems), and then realized it was too heavy a lift to actually develop NonStop code. As a result, only about 20% of Tandem app code used these capabilities (my best recollection of a study of that.) The rest simply used the dual stacks in simplex mode.

I haven't yet seen anything that reaches elegance and effectiveness of the Tandem HW/SW architecture, including their container approach ;-) Tandem Coffee Mug

Upvotes: 2

Maytham Fahmi
Maytham Fahmi

Reputation: 33397

This paper from HP conceptually cover your questions:

https://www.hpl.hp.com/techreports/tandem/TR-86.2.pdf

Upvotes: 6

Mark Poriss
Mark Poriss

Reputation: 31

Definitely not similar to IBM architecture, the T/16 CPU was a proprietary design. It was greatly influenced by the HP 3000 minicomputer. The original Tandem team were mostly from HP and Burroughs.

The wiki states most everything at a high level: https://en.wikipedia.org/wiki/Tandem_Computers

Upvotes: 1

ZhijieWang
ZhijieWang

Reputation: 453

i think it is similar to IBM architecture, shared nothing structure. Lots of redundancy, but nothing is shared or provisioned/dedicated -- based on my previous reading on IBM z/OS and mainframes.

Normally this type of system uses proprietary OS and modified kernel and special FS/driver to leverage the underlying hardware. In some cases, yes applications needs to be modified to leverage some special transaction libraries, but just like you need to have transaction locks for RDBMS when you scale it horizontally.

A lots of this HA/FT may be achieved in Kernel level, abstract away from applications.

Notice the chip in HPE Non-Stop systems, it is Itanium architecture, not regular Xeon chips. Just like IBM had its own proprietary enterprise class CPU for a while https://en.wikipedia.org/wiki/Z/Architecture

Upvotes: 0

Related Questions