TABLE 1. Data center design conditions.


A data center the size of a football field and an owner with sustainable mandates made it hard for this project team to just take the conventional wisdom and run with it. In part one of two, the author winds up eating some crow but conquers the problem. Get ready for an old-school system comparison, but focus on rack entering conditions instead of average space conditions, and check those design expectations at the door.

Have you ever been flat-out wrong about something? Perhaps, in the middle of a discussion, you blustered, “No way!” … only to hear the oft-heard retort, “Dude, way.”

Well, it hasn’t happened that often to me, but not because I’m always right. Rather just the opposite. When I don’t completely understand something, I’m usually bright enough not to be critical. You know the old adage that it’s better to keep silent and be thought an idiot than to open your mouth and remove all doubt? I had that tattooed on my chest just so I wouldn’t forget.

But one day, I stood in the project war room on a mission critical facility, and pontificated, “Of the two schemes we are evaluating, I guarantee you that one just won’t fly.” And what was the concept? I didn’t know it at the time, but it was the very concept I would be passionately advocating for just four weeks later.



TABLE 2. Energy and utility use matrix.

Some Background

My firm was fortunate enough to be chosen to be on the team for a significant data center facility with a 70,000-sq-ft raised floor area (RFA). For context, that’s a data hall the size of a football field, with a load around 100 W/sq ft. The estimated cooling load for the facility was nominally 2,000 tons with N+1 redundancy.

The concept designers who had been working on the job prior to our arrival had two cooling schemes in mind. Both were variable primary, but one was a water cooled centrifugal plant with variable-speed chillers and evaporative cooling towers. The second was a glycol condenser loop utilizing dry coolers and compressorized computer room air conditioning (CRAC) units.

The owner’s operating staff was aghast at the thought of having hundreds of compressors throughout the building whose eventual replacement and maintenance could very well become overwhelming. On the other hand, the concept designer made a very eloquent case for the dry coolers based on their simplicity and the fact that the significant water use associated with evaporative towers would be eliminated.

As a guy who considers himself pretty savvy when it comes to sustainable design, I admit my curiosity was piqued at the possibility of eliminating all of that water evaporating or going down the drain, along with its associated chemicals and cost. But the idea of compressorized equipment throughout the building was a non-starter.



TABLE 3. Life-cycle cost matrix.

Air and Water

So what about using air cooled chillers? You eliminate the compressors and the wasted water, but again, our conventional wisdom tells us that air cooled chillers are significantly less efficient than water cooled, especially if your water-based system has a waterside economizer. On top of that, there was a strong possibility that this project might be sited in the American Northeast, and air cooled chillers won’t operate at temperatures at or near 0°F. So it would seem that the only option was traditional water cooled chillers and towers.

But the issue of water conservation wouldn’t go away. Not for the client, because they had a corporate mandate to use the fewest natural resources possible. Not for the concept designer, because he was steadfast in his insistence that water conservation and equipment simplicity out weighed the possibility of lower operating costs. And not for me, because I was in the unique position, as the engineer of record, of having to carry a design forward that would satisfy both parties while still meeting my own professional responsibility to do no harm to man or the earth.

So the concept engineer and I were sequestered in a room and given a whole hour to come up with a design that met the various demands. And it should go without saying, but cost had to stay within the project budget.

So there we were: The big idea concept guy with his dry cooler paradigm and yours truly with my own mechanical blinders on, advocating for the traditional plant. By the way, I hate the term “traditional.” It’s right up there with “industry standard” and “standard practice” in the lexicon of phrases that convey sound design, but belie the fact that they provide cover for the unimaginative designer who will sacrifice innovation for convenience. Regardless of how I feel about those terms, the reality was that I was in that room using them.

Now the concept guy was crazy - like a fox. He knew better than I that it made no sense to fall on his sword for a concept that the owner had clearly squelched, but he was principled enough to stick by his guns. Especially because he knew deep down that he was right about the water issue. So together we schemed.
And then, what I came to call the “elegant solution” revealed itself.



The Elegant Solution

So let’s recap the marching orders. We had to limit water use but keep simplicity integral to the design. We had to do this while using as little energy as possible and keeping operating costs in line. And we had to do all this at a reasonable cost in a redundant and reliable configuration that could operate 24/7/365 with absolutely no down time.

My first suggestion was to use air cooled chillers (how remarkably courageous). But while this addressed water and simplicity, it was a letdown in the energy and operating cost arena. Further, it was hopeless due to the low ambient operating limit.

So what if we added a dry cooler sub loop to provide cooling when the temperatures required the chillers to be shut down at low ambient? Basically, since dry coolers are simply coils with fans on them, if the ambient outdoor air provided a sufficient drybulb approach, then you could chill the water using air only, eliminating the chillers from the equation. This would be a kind of low ambient economizer. Economizer? Did I just say economizer?

That’s when it hit me like a ton of air cooled bricks. If we could provide sufficient capacity with dry coolers to handle the entire load at 0°, couldn’t we get partial economizer operation at say 15°? And, if 15° worked, couldn’t we go higher? Heck, if my chilled water return temperature is between 50° and 55°, couldn’t I, theoretically, start getting some free precooling at a 5° approach when the outdoor temperature was in the range of 45°?

Eureka! In spite of myself, I (actually we) had found it.


Proof and the Proverbial Pudding

So at first, this looked like an interesting idea that would satisfy this particular client and the design team’s preferences and concerns. It was a compromise that I thought would probably cost more to operate, but we could feel better about ourselves because we had struck a bargain with each other and Mother Nature.
But there it is. Another one of those kiss your sister kind of words: compromise.

When we emerged from our little conclave, those who had not witnessed our genius simply scoffed at the proposed notion. Many saw the compromise as simply a short cut to a predisposed conclusion. A settling, if you will, as in settling for second best. The consequence of which is not very satisfying ... just ask my lovely bride.

So upon being challenged, I naturally did what any designer with conviction would do: I folded like a cheap tent at the end of a rainy Boy Scout Jamboree. That is to say, I backed down quickly. But the concept engineer didn’t. His gut told him that this was the answer and he would be delighted to allow me to disprove the idea. A life-cycle cost analysis (LCCA) was commissioned by the owner and I was asked to compare the air cooled and water cooled options.

When cornered, I uttered the aforementioned pronouncement. Specifically, I stated with confidence that while the air cooled option had merit on the water conservation front, there was no way it could win a head-to-head LCCA shoot out. Unbeknownst to me at the time, was that at that precise moment I had placed an advance order for a healthy serving of crow.


FIGURE 1. Variable primary chilled water plant with dry cooler economizer.

Data Center Environmental Optimization

In a concerted effort to optimize the HVAC design while maintaining an appropriate data center environment, the environmental conditions were reviewed.

Per the Design Conditions for Datacom Equipment Centers design guide from the ASHRAE Datacom Series, the recommended cold aisle (rack inlet) conditions for a Class 1 data center are as listed in the first two columns of Table 1. After polling the owner’s operations personnel (a varied lot, I might add), the consensus was to use the criteria listed in the third column of Table 1. It is worth noting that the table reflects the more relevant rack entering conditions, not the more commonly assumed (and frankly useless), average space conditions.

Based on a nominal 70° supply air temperature at the rack and an airside differential of 25°, the hot aisle temperature would be about 95°. While this is well within the design guide parameters and equipment tolerances (it’s not called the hot aisle for nothing), it presented a massive shift in expectations on the part of the endusers.

A shift, however, was warranted since this change in the RFA environmental parameters created a cascading effect that ultimately carried back to the condensing mediums. Specifically, higher air temperatures allow higher chilled water temperatures, which increase chiller compressor efficiencies. But more importantly, it would increase the number of available free cooling hours. Ultimately, the chilled water parameters used in the study were 57° supply with a 12° differential.

You and I both know that you couldn’t maintain acceptable occupied space conditions with an entering water temperature that high. So it was understood that the non-critical occupied areas would have to be addressed with either a low-temperature chilled water system operating at a more traditional temperature, or with heat pump AHUs using the higher chilled water temperature loop as a condenser medium.

Since the building required N+1 redundancy, it was decided that we would study two plants providing 1,000 tons each. It was also assumed that the secondary pumping energy would be equal in both cases studied. And since I didn’t have enough options to study, I was asked to consider two locations, one in Pennsylvania and the other in North Carolina.



The Plants Compared

Old school first: A straightforward variable primary chilled water plant was modeled with three nominal 500-ton centrifugal VFD chillers in parallel, three equally sized cooling towers in parallel, one nominal 1,000 ton plate-and-frame heat exchanger, and three condenser pumps running at about .07kW/ton.

The model reflected the following sequence: When the outdoor wetbulb temperature was above 47°, the chillers would handle the entire load. Below 47° wetbulb, the chillers would be shut down and essentially free cooling would be provided via the plate-and-frame heat exchanger.

We recognized that in actual operation it would not be possible to make the transition from chillers to free cooling every hour the temperature drops below the wetbulb threshold. But to keep our analysis as objective as possible (and simple), each hour was counted toward the free cooling total. In turn, the energy usage estimated may have been optimistically low, but in our opinion, not enough to unfairly skew the results.

Our elegant solution consisted of two plants with three nominal 500-ton air cooled VFD chillers in parallel and five nominal 170-ton dry fluid coolers in series with the air cooled chillers. You can see in Figure 1 the final layout we actually applied in the final design.

The following sequence was modeled: When the outdoor drybulb temperature was 57°, the air cooled chillers would handle the entire load. Between approximately 47° and 57° drybulb, a portion of the load would be handled by the dry coolers, which, as you may recall, would be in series with the air cooled chillers. As the drybulb temperatures drops, the proportion of essentially free cooling to cooling requiring the compressors would increase. Below 47° drybulb, the dry coolers would handle the entire load.

Now, the caveat of “essentially free” cooling is applied here since in actuality there will always be a minimal amount of dry cooler or cooling tower fan energy that is required. But this is drastically less than the otherwise requisite compressor energy.



The Bottom Line

Looking at Table 2, the higher energy usage seen in the South in all cases could be attributed to fewer available economizer hours. But notice that while the two concepts were essentially even in the North, the air cooled plant proved significantly less attractive in the relatively warmer Carolina locale. This observation was validated in a recent article that appeared in HPAC Magazine, which applied the same concept in a data center using chillers with integrated dry coolers. In that same article, there is a fascinating graphic showing an approximate air cooled Mason-Dixon line that I strongly encourage you to take a look at.

But energy use is only part of the equation. We then ran the 15-year LCCA using the following approximate utility rates: Electricity at both locations $0.10 kWh. Water at $2.50/100 cu ft in PA and $2.90/100 cu ft in NC.
Sewage at $1.70/100 cu ft in PA and $2.60/100 cu ft in NC. Refer to Table 3 for the results.

Now, the dramatic impact of cutting out water usage floated to the top (pun painfully intended). In the North, the air cooled system’s favorable spread was what we expected based on the energy use. But we were surprised that even with higher energy use in the South, when you factored in the relatively high local water and sewer rates, the air cooled option defied conventional wisdom and won out again.



Conclusion

Ultimately, after all of my feverish machinations, the impartial computer proved the mad concept designer’s hunch. He would have told you that you could have saved yourself the trouble, but to his credit he was magnanimous. He was a little insufferable, but hey, he was a pain even before we ran the numbers (I kid because I love).

So lesson learned. The less obvious choice of an air cooled chilled water system ultimately was less costly in both locations. This could primarily be attributed to the lower first costs, the relatively high number of available economizer hours, and to a lesser but no less surprising extent, the relatively high water and sewer costs at both sites.

Not only was it cheaper, but the system would be easier to operate and maintain. The simplicity of the automatic economizer changeover was worth the price of admission alone. Since it would basically be automatic, it would certainly be preferable to a waterside economizer and/or a system with a changeover sequence that requires a weather station, a pair of unloaded dice, and access to a Magic 8-Ball just to determine when conditions will be just right for just long enough to commit the system to the whims of Mother Nature.

As the project evolved, a number of other interesting opportunities and equally interesting solutions were realized. But more on that next month. In the end, it was, to paraphrase my Russian counterpart, a freaking thing of beauty (FToB).

Of course I would be remiss if I didn’t identify my tormenter/comrade. The guy with the sixth sense and unwavering conviction was and is Alex Balsam. Alex is a principal with Sigma7 Design Group, a New York-based consulting firm that specializes in mission critical design. You can contact him for your next data center project atwww.sigma7design.com.

Or you can contact me, but don’t expect any guarantees.ES

Bibliography

American Society of Heating, Refrigerating, and Air-Conditioning Engineers.Thermal Guidelines for Data Processing Environments. Atlanta.2004.


Cited Works

1. Whitmore, Graham. HPAC Magazine. “Green HVAC Designs for Data Centers.” December 2007: 20-24.