Selecting the Optimal Data Center Cooling Solution

A better way to think about data center cooling is to forget the notion of adding ‘cold’ to a room. Rather, think about air conditioning as removing heat from the room. To that end, there are two components that make up any effective cooling solution. They are first, the Heat Removal section, often referred to as the CRAC, CRAH, or the evaporator, and the second, the Heat Rejection section which is named according to the cooling approach executed and referred to as the condenser, dry-cooler, fluid-cooler, cooling tower, etc.

As such, the first step in selecting an appropriate data center cooling solution always starts with establishing suitable design criteria. Further, selecting the optimal cooling solution involves a deep understanding and comparison between the performance characteristics, capital expense (CAPEX), and operational expense (OPEX) of each potential configuration.

Establish Suitable Design Criteria

Key design criteria definition establishes the critical load, site availability requirements, and IT equipment deployment density for a specific computer room improvement, expansion, or new construction project. These three (3) high-level decisions serve as variables in a cost model matrix to identify their affect on overall project costs across multiple scenarios, thus allowing a narrowing of the scope of the project to one that is affordable, yet meets the business’ needs. Following this the narrowed scope serves as a basis of engineering and design, as well as the subsequent construction.

Key Design Criteria for Load

The load design criteria is defined as two (2) separate parameters:

1. The Day-One Load, and
2. The Maximum Permissible Load after all phases, upgrades, expansions, and improvements.

Understanding the day-one load is important is because air conditioners cannot operate effective with too little load as much they cannot handle too much load. Conversely, understanding the maximum load is vital for overall space and power budgeting.

Key Design Criteria for Availability

Availability is a measure of the probability that a data center’s site and supporting infrastructure will stay active and usable, but more importantly an estimate on the amount of downtime that may be realized in making different design decisions. Design characteristics such as reliability, redundancy, non-disruptive maintainability, and fault tolerance are all availability improvement drivers as well as impacting on cost. The first organization that made a meaningful contribution has been The Uptime Institute with their paper, ‘Tier Classifications Define Site Infrastructure Performance’. However, most experienced data center design firms have their own rating guidelines to estimate availability. Typically, these ratings have been developed using a much broader sample of data center designs, across a wider definition of infrastructure configurations, and provide more detailed CAPEX and OPEX impacts.

Key Design Criteria for Density

The equipment deployment density is a statement of the watts-per-square foot (W/SF), on a per-rack, cabinet, or enclosure basis, that the IT load will be deployed. While this approach is divergent from its predecessor, describing the W/SF of the computer room itself, it is a far better measure since today’s load density diversity can stretch from 0 W/SF for a cable infrastructure rack to 25+ kW/SF for an ultra high-density blade cabinet – all in the same computer room. Therefore, this more granular statement of load geometry allows the designer to better adapt cooling systems approach to the needs of the load with respect to supply and return air control.

Computer Room Air Conditioning Approaches

Room-Based vs. Row-Based vs. Rack-Based Heat Removal

Regardless of the cooling system’s heat rejection method, the air side configuration is 50% of the cooling equation. The guiding principal for the air side of the system is heat removal. This is the process of using the ‘cold’ coil to absorb heat from the room (as created by the server) by passing the room air across the coil via a fan or blower.

Regardless of the approach, the main focus to effective heat removal is to remove it as close to the source it is generated as is practical. This means that Rack-Based Cooling will win, as compared to the other approaches, every time. This is because in a rack-based cooling scenario the heat is absorbed within the rack, and sometime even right at the chip set! However, the cost for achieving these efficiencies is often high because each load source has to have its own heat removal equipment.

As such, the next best options are Row-Based cooling solutions. Row-Based cooling equipment intersperses the air conditioners in between rows of cabinets that are the heat load. Therefore, the heat removal, as well as the cold air supply, is closely coupled to their intended targets. The benefit to this approach is that one air conditioner unit can serve multiple load cabinets in a given area or zone. In addition, this allows adaptation to varying deployment densities.

Finally, the traditional standby approach has been perimeter located down-flow air conditioners pumping cold air into a raised floor plenum. In this approach, the heat is removed via the top of the air conditioner. This is traditionally been done via free air, however in later years this approach has been augmented by utilizing the space above the suspended ceiling in the room as a return air plenum.

In all cases, the ‘secret’ is to reduce air mixing. All techniques and approaches work toward achieving that goal from the various air conditioner configurations, to the bevy of air containment systems available, even the adaptation of the hot aisle / cold aisle strategy itself.

Containment vs. Non-Containment

In keeping with the ‘air mixing is bad’ theme, it was inevitable that containment solutions evolved. Containment is the execution of containing the hot air and/or cold air to improve the effectiveness and efficiency of whatever cooling solution is deployed. In fact, the fight has been downright contentious over which is better: cold aisle containment (CAC) or hot aisle containment (HAC).

I contend that the problem isn’t one of containment, rather one of air flow balancing. Picture a fish tank being filled by a hose, but with a spigot that can let water out. If you fill the tank faster than you let water out, then the tank overflows. This is analogous to the hot aisle. If the IT load pumps heat into the hot aisle faster than the CRAC/CRAH can remove the heat then the excess heat billow out and mixes with the cold air targeted for the IT equipment. Conversely, if you let water out faster than the hose can fill, the opposite is true. In this instance, the hot aisle will draw in cold air from the surrounding to make up for the lack heat available to be rejected. In either case, no degree of non-perfect sealed containment will prevent air mixing. Hence, the problem is air flow, not containment.

Direct Expansion (DX)-Based vs. Chilled Water-Based Heat Rejection

As stated earlier, the other half of the cooling approach is heat rejection. This is the process of rejecting the heat from the room where it was located and transferring that energy to the ambient environment preferably outside of the building.

The two main mediums for transferring heat from one location to another are via refrigerant or water. While water is a far better for transferring heat as it can hold far more heat per its volume than refrigerant can, refrigerant is lighter than water and therefore makes for a less complicated albeit limited method.

Further, the characteristic that makes refrigerant-based air conditioners attractive is that it can exist in both a liquid and gaseous state at much lower temperatures than water which can be controlled by pressure, or by the act of compressing the liquid into a gas which in turn adds heat. This compressed gas can be moved to the outdoor condenser section where a fan blowing ambient air across the coil can reduce the temperature of the refrigerant. Thus, when it makes its way back to the indoor air conditioner the pressure can be released by allowing the refrigerant to expand and thereby get cold. This condition is called the refrigeration process and is the basis of all direct expansion (DX)-based air conditioner solutions.

To further complicate matters, the DX process can be accomplished in a few different ways including, air cooled, glycol cooled (which uses a water / glycol mixture), and water cooled (which utilizes a cooling tower). Common to all of them is that a compressor is utilized in the air conditioning system to compress refrigerant. Often the indoor portion of this system is referred to as a Computer Room Air Conditioner (CRAC) unit.

A chilled water cooling approach utilizes a Computer Room Air Handling (CRAH) unit. This means, that the CRAH does not rely on a compressor within to create the refrigeration process. Rather, a chiller creates cold water via a number approaches outside the scope of this paper. In turn, the chilled water is pumped to the CRAH and a blower passes the room air across the cold water coil which absorbs the heat from the room and carries it to the heat rejection equipment.

Each of these approaches carries with it a unique CAPEX / OPEX condition. Choosing the appropriate cooling solution for any data center project therefore requires an analysis by a design professional familiar with comparing the different approaches. However, in general DX air cooled solutions can accommodate the majority of computer room cooling application given its relative low cost, ease of installation and operation, its scalability due to the 1:1 nature of its indoor to outdoor components.

Energy Usage Improvements Using Economization and/or Evaporative Cooling Techniques

Ask any data center manager for a list of priorities for effective operations and on the top of the list will be availability. In fact, availability trumps energy efficiency every time. That said there are methods that can be employed that do not overly affect availability while at the same time able to provide energy usage efficiency. One such method is the use of economizers in concert with the heat rejection approach. There are two distinct ways to employ economization air-side and water-side. Both leverage leveraging the ambient condition to offset using mechanical-based cooling.

Air-side economizers accomplish this by actually bringing in outside air into the facility while at the same time exhausting heat out of the facility in balance with the amount coming in. Typically, the outside air must be tempered, meaning filtered of particulates and adjusted for humidity content prior to introducing it into the critical environment. The danger is that no amount of tempering is failsafe in preventing air infiltration when the air outside becomes toxic or worse.

Water-side economizers provide less efficiency gains, but are far less risky to data center operations. This approach is typically utilized with chilled water operation. The ambient condition is used to reject the heat from the ‘warmed’ water, thereby making ‘cold’ water in lieu of having to use a mechanical process to do so. The result is energy saving from not having to run the chillers all the time.

A third technique to produce cooling with very low energy cost is evaporative cooling. An evaporative cooler is a device that cools air through the evaporation of water. Evaporative cooling differs from typical air conditioning systems which use vapor-compression or absorption refrigeration cycles. Evaporative cooling works by employing water’s large enthalpy of vaporization, or the amount of energy it takes to convert a substance into a gas. The temperature of dry air can be dropped significantly through the phase transition of liquid water to water vapor, which requires much less energy than refrigeration. In extremely dry climates, it also has the added benefit of conditioning the air with more moisture to the benefit of the electronics. However, unlike refrigeration, it requires a water source, and must continually consume water to operate.

There are a number of approaches to evaporative cooling however usually one is applicable to data center applications. Traditional evaporative coolers use only a fraction of the energy of vapor-compression or absorption air conditioning systems. Unfortunately, except in very dry climates they increase humidity to high levels. Two-stage, or indirect-direct evaporative coolers do not produce humidity levels as high as that produced by traditional single-stage evaporative coolers. In the first stage of a two-stage cooler, warm air is pre-cooled indirectly without adding humidity (by passing inside a heat exchanger that is cooled by evaporation on the outside). In the direct stage, the pre-cooled air passes through a water-soaked pad and picks up humidity as it cools. Since the air supply is pre-cooled in the first stage, less humidity is needed in the direct stage to reach the desired cooling temperatures. The result is cooler air with a relative humidity between 50 and 70 percent, depending on the climate, compared to a traditional system that produces about 70–80 percent relative humidity air. Since ASHRAE’s TC 9.9 standard widened the allowable operating humidity limits indirect-direct evaporative cooling has become more commonplace in drier climates.

In general air conditioning will account for approximately 60% of the facility construction budget and potentially more energy consumption than the load itself.

The Role of Consultants in the Process

Sufficient Planning Prior to Detailed Engineering and/or Construction

Given the variety of approaches to architect a data center cooling solution, each with its pluses and minuses, it is imperative to develop an overall game plan prior to detailed engineering design or construction. Qualified Data Center design consultants:

• leverage proven processes to ascertain client near- and long-term requirements for data center power, space, and cooling requirements,
• utilized design tools such as measurement solutions, monitoring tools, and Computational Fluid Dynamics (CFD) design software to accurately depict existing and new load requirements; and,
• call upon experiential designs with similar requirements and scenarios to help clients design a cooling approach which will support client IT loads requirements.

CFD Tools to Analyze Steady-State and Failure Conditions in Comparing Designs

Consultants utilize powerful 3-D CFD software for the design, operational analysis, and maintenance for data center and computer rooms of all types and sizes. The intent is to provide better designs and management of mission critical facilities throughout their lifetime. The goal is to minimize the risk of cooling problems as well as increase the overall manageability and performance of the site.

CFD modeling gives engineers the luxury to consider several design options in the minimum amount of time. As a result, the final design is not based on a tentative approach, but is a result of a professional design process considering several options and selecting the optimum solution. This can save on capital and operational costs as well as save time by avoiding mistakes and during commissioning.

A typical CFD modeling process includes:

• Evaluation of the Architectural Shell and Room Type (I.e. raised floor, ducted supply)
• Maps Obstructions
• Raised Floor & Suspended Ceiling Layouts
• Computer Room Air Conditioner (CRAC) Methodology
• Development of CFD Model(s)
• Model Analysis & Update
• Site Engineering & Construction Documents

Analysis of CAPEX/OPEX Ramifications of Cooling Design Decisions

Using experiential-based modeling, experienced Data Center Design Consultants provide clients with an analysis of CAPEX/OPEX ramifications related to a new data center cooling design. By looking at data centers with similar size, IT infrastructure, and potential cooling designs, consultants add valuable experience in terms of the real costs associated with building a data center cooling system.

In addition, consultants can build operating cost models to assist in final decision making regarding design approaches. These models are then validated against real data center case scenarios to confirm accuracy.

Summary

To select the optimal cooling system for a data center there is no panacea. Each approach to data center cooling design (I.e. Hot Aisle Containment vs. Cold Aisle Containment or Room vs. Row vs. Rack cooling approaches) has its merits. Ultimately, key design criteria for a specific facility and IT environment as well as pressures related to CAPEX versus OPEX budgets drive the proper approach for a given company.

By selecting and leveraging experienced data center design consultants, a company can avoid costly mistakes and, worse, potential and expensive design changes down the road. By leveraging proven design approaches, CFD design tools, and significant prior experiences, data center cooling design consultants should be considered as an integral part of the data center design team long before orders are placed, floors are installed, and cooling systems provisioned.

To learn more about PTS recommended Air Conditioning Equipment & Systems, contact us or visit:

• Air Flow Management
• Rack-Based Cooling Solutions
• Row-Based Cooling Solutions
• Room-Based Cooling Solutions
• Economizer Cooling Solutions
• Heat Rejection Solutions

Pete Sacco, President & CEO, of PTS Data Center Solutions authored an interesting article on the most effective way to select an Optimal Data Center Cooling Solution.

A better way to think about data center cooling is to forget the notion of adding ‘cold’ to a room. Rather, think about air conditioning as removing heat from the room. Selecting the optimal cooling solution involves a deep understanding and comparison between the performance characteristics, capital expense (CAPEX), and operational expense (OPEX) of each potential configuration.

The article provides a deep dive into:

Read the entire article by clicking here or learn more by contacting PTS.

To learn more about PTS services and solutions for data center cooling needs, click here.