Unix in the Data-Center: How To Fail by Succeeding

What would your answer be if a selection team charged with hiring a new CIO to develop and implement an organization-wide “strategic systems architecture” were to ask you what management considerations most differentiate use of Windows from use of Linux?

The right answer, I think, is that the more fully the organization implements the Unix Business Architecture (explained below), the smaller and more outward-facing the systems organization can be. Conversely, the more fully it implements Microsoft’s client-server architecture, the larger and more inwardly focused the IT organization has to be. Explaining that answer, and the terms used, is what this column is about.

First, let’s look at the terminology. I use the term “UBA,” or Unix Business Architecture, to mean a Unix “pure-play” with centralized processing, desktop smart displays (like Sun’s Sunray 1g), simplified networking and formal management support for people who use any Unix variant, including Mac OS X, Linux and BSD, at home.

In this environment, all processing is on the server, with the desktop device limited to handling user interaction. As a result, things always work. What shows up on a user’s desktop depends on the user, not the display or its location. In addition, application support is handled by lead users, there is no help desk, and security issues never reach users.

As a result, the CIO in a pure Unix environment can metaphorically “face outward,” looking beyond day-to-day operations in the data center to focus on business issues and user services while needing to commit only a minimal number of people to housekeeping tasks like server and applications management.

In Distinct Contrast

In contrast, Microsoft’s client-server architecture puts a powerful computer on the user’s desk but uses it for little more than storing the user’s home environment and managing the interface to many small servers run by the data center. Notice that client-server is client-server whether those desktops run Windows, Linux or Mac OS X. The critical differences between the two architectures derive from the presence of a desktop computer capable of stand-alone use, regardless of what operating system that desktop machine runs.

In practice, of course, most Microsoft client-server implementations rely on Microsoft software running on Intel-based gear, but an implementation built using Linux servers and desktops still would be client-server — and thus ultimately would fall heir to the same problems.

Originally, the idea behind client-server was that desktop cycles would be used to process data stored on servers. In practice, however, coping with the realities of data backup, network failures, the improbability of transaction serialization in a highly distributed client-server network, and client software control have turned implementations of this idea into something of a sham because service centralization scales easily but true client-server doesn’t.

Morphing the Client-Server Model

As early as 1992, therefore, smart-display maker NCD recognized the value of the Web browser as a kind of universal display client and started offering Mosaic in either server-downloadable or ROM-loadable forms to augment its standard X/Postscript interfaces. NCD was well ahead of its time on this. But by 2000, the impact of repeated attempts to make PC-based client-server computing work was clear enough that a statistical analysis of various client-server management strategies carried out by Julie Smith David, David Schuff and Robert St. Louis [“Managing Your IT Total Cost of Ownership,” Communications of the ACM, January 2002] showed that costs vary inversely with control intensity and management centralization.

This means that the more controls on the PC desktop are tightened, effectively morphing client-server back into the mainframe architecture it replaced, the cheaper and more reliable it gets.

Of course, building a mainframe data-center using Wintel isn’t particularly efficient and has organizational costs that go beyond simple systems issues. For example, use of the Microsoft client-server environment typically means users have to carry their computers with them when they leave their desks.

It also means systems are considered so untrustworthy and transient that users compromise the value of large-scale software like enterprise resource planning packages by limiting usage to the most basic parts of the package; the network software stack needed gets to be so complex that the reboot-reload cycle replaces failure analysis and remediation. In addition to all of this, the expectation that security will fail more or less at random is integrated into daily operations, and everything related to IT exists in a perpetual state of rollout or upgrade. Finally, systems management has to provide a well-staffed help desk simply to ensure that users can get their clients started up and connected to the central service.

As a result, the CIO in a Microsoft client-server environment has to be “inwardly focused,” concentrating on firefighting in the data-center and totally committed to the daily struggle to provide the most basic of all forms of user support: simply keeping things running in the face of whatever today’s security, software, budget or staffing crisis might be.

Driving Toward Resolution

Of course, both the UBA and the fully implemented Microsoft architecture represent extremes. In reality, most organizational systems deployments fall somewhere between these, and it’s the need to drive that in-between world to a resolution in one direction or another that should inform your answer to the hiring committee. In formulating that answer, what you need to talk about most are the managerial consequences of each decision — not in terms of technology but in terms of how that technology affects daily IT and business operations.

You need to pick one architecture and stick with it for all servers and upward of 95 percent of your desktops, because the most common situation in which the business relies on a few Unix servers to provide truly critical services in an otherwise predominantly Microsoft environment is also the worst possible one.

What happens in this situation follows a well-understood pattern: At first, the Unix systems carry most of the real workload, but the staff involved become increasingly disconnected from the Wintel majority around them and eventually leave. The Microsoft Certified Systems Engineers (MCSEs) put in charge of the Unix servers then “administer” them as if they were Windows servers, using Windows certainties and Windows telnet or some other third-party Unix management software for Windows, with the result that the cheapest and most reliable part of the IT operation is quickly transformed into its most expensive and least-reliable piece.

It’s not unusual, in the late stages of this transformation, to see rock-solid combinations, such as an older Oracle release on Sparc or PostGreSQL on Linux, going down daily while a cloud of MCSEs badmouth Unix and struggle to rescue the company from it by porting the database to Microsoft’s SQL Server. Look at this process from a CIO perspective, and the inevitability with which the numerical majority of Wintel people drives out the Unix gurus means such mixed environments should be seen up-front as virtual Microsoft pure-plays with rapidly metastasizing cost and reliability cancers at their hearts.

Decision One Way or Another

Thus, a decision one way or the other has to be made — and the decision to go with Microsoft is by far the easiest because users won’t question it and you won’t have to lay off most of the IT staff or rebuild the relationship between IT and its users. Doing the right thing is much harder — and a lot less rewarding in terms of your corporate visibility, career and long-term salary expectations. Remember, this kind of data-center disaster doesn’t grow out of any Wintel technical advantage or peer pressure from Windows advocates: What kills data-center Unix is success.

IT management, like all management, focuses where the squeak is — but only for the duration of the emergency. As a result, Unix people and technology brought in to solve an urgent software, technology or cost problem become invisible to senior IT managers as soon the crisis ends. The rule is simple: no squeal, no grease. IT managers inundated by the daily cacophony of Wintel support grow that side of their business while ignoring what works simply because it works.

Eventually, of course, the Unix people move on, leaving the MCSE crowd in control of the systems. The consequent eruption of reliability, performance and systems recovery crises refocuses IT management’s attention on Unix, but by then it’s usually too late to save the data-center without major changes in staffing and direction.

Biggest Long-Term Problem

That’s also the biggest long-term problem facing the Unix CIO: It’s not easy to transition an organization to the UBA successfully. But once you succeed, senior management will forget all about IT, unconsciously deleting it (and you) from their agenda, meaning that the most effective organizational CIOs also tend to be the least visible, least appreciated and least promoted.

With that image in mind, you can look the committee in its collective eye and say that the real consequence of the architecture decision isn’t whether they should continue with a mixed environment — if that’s what they have, it has to go — but what happens afterward.

That, you can tell them, boils down to whether you’ll be getting in their faces just to remind them of your existence or at meetings they’ve had to call to discuss the budgetary or functional compromises needed to cope with the latest Wintel crisis.