Data center cooling is a critical concern in modern IT infrastructure, driving the development of innovative systems and solutions, involving air conditioning, water cooling techniques, and other liquid-based mechanisms to ensure optimal performance and energy efficiency. Since cooling systems account for approximately 40% of a data center’s total energy consumption, cost is also a crucial consideration.
There are several data center cooling systems used across the industry, including Computer Room Air Conditioning (CRAC) and Computer Room Air Handler (CRAH) units, hot and cold aisle containment, in-row and in-rack cooling, chilled water, liquid cooling, free cooling, evaporative cooling, and more.
Data center cooling plays a crucial role in maintaining system performance, and in this article, Dgtl Infra covers both traditional and emerging cooling methods. Continue reading to understand what the purpose of data center cooling is, why it is important, and how it works.
What is Data Center Cooling?
Data center cooling refers to the equipment, systems, methods, and technologies used to regulate the temperature, humidity, and airflow within a data center facility. Since data centers often house thousands of servers, IT devices, and other electronic equipment that generate significant amounts of heat, proper cooling is essential to maintain optimal performance and prevent overheating.
What is the Purpose of Data Center Cooling?
Cooling systems in data centers are designed to dissipate the heat generated by the operation of servers, storage devices, networking hardware, and various other equipment. This heat, measured in terms of temperature, is produced as electrical energy is converted into thermal energy, a process that occurs due to inefficiencies in the electronic components.
Along with managing heat, data center cooling systems also maintain proper humidity levels within the facility. By doing so, they prevent the buildup of static electricity and condensation, which can lead to corrosion. Both of these factors can cause significant damage to electronic equipment.
The primary purpose of data center cooling is to maintain environmental conditions that are suitable for the operation of IT equipment. The industry organization ASHRAE recommends that the temperature in data centers be maintained within the range of 18°C to 27°C (64°F to 81°F). ASHRAE also advises that humidity levels for data centers should fall within the range of 40% to 60%, depending on the specific equipment and configuration.
To achieve the necessary temperature and humidity levels within a data center, proper airflow and ventilation are essential. These elements work together to dissipate heat and regulate environmental conditions in the facility.
Why is Data Center Cooling Important?
Data center cooling is important for a number of reasons, including performance, preventing downtime, equipment longevity, energy efficiency, and worker comfort. Here is a detailed examination of each of these factors:
- Performance: Excessive heat in a data center can create ‘hot spots,’ leading to malfunctioning server components such as processors and memory. When overheated, computer systems may slow down, freeze, or even suffer permanent hardware damage. Proper cooling allows systems to maintain optimal performance and the onboard logic of the server
- Preventing Downtime: Overheating can cause system failures and result in downtime. In data centers, downtime is not only disruptive but also incredibly costly. Implementing appropriate cooling techniques helps in preventing such issues
- Equipment Longevity: Prolonged exposure to high temperatures can reduce the lifespan of hardware. Conversely, maintaining a cool environment extends the life of the equipment. It is also important to note that too much humidity can be harmful to equipment. When hygrometric dust particles stick to electrical components, it reduces heat transfer and can even lead to corrosion
- Energy Efficiency: Efficient cooling systems consume less energy and water, making them more environmentally friendly and cost-effective. Traditional “mechanical” data center cooling methods, such as air conditioning, can be very energy-intensive and consume significant amounts of water. Thus, adopting modern cooling solutions, that use less energy or water, is particularly beneficial regarding operational power costs
- Worker Comfort: While comfort cooling is not the main purpose of data center cooling systems, it is essential to maintain a safe environment for human operators working within the data center, particularly for short-term periods
How Does Data Center Cooling Work?
Data center cooling works by employing a number of key components and strategies for maintaining optimal conditions in the facility. Generally, these include server cooling, computer room cooling (or server room cooling), airflow management, heat rejection, fluid conditioning, and humidity control.
Server cooling refers to lowering the temperature of IT equipment by removing the heat generated by individual server components such as processors and memory. These components are highly localized and intense heat sources that are responsible for effectively changing the state of energy from electricity to heat, thereby requiring cooling.
Computer Room Cooling
Computer room cooling (or server room cooling) involves maintaining the ambient temperature and humidity of the computer room(s) within data centers by removing the heat and moisture generated from the space housing the IT equipment, including servers, storage devices, and networking hardware. Here, the focus is on ensuring that there is consistent and uniform cooling, so that hot spots are avoided in the room.
Airflow management is the control of airflow direction, velocity, and quality through HVAC systems to ensure the efficient distribution of cool air and removal of excess hot air within the data center. This process is crucial for efficiently supporting both server cooling and computer room cooling by keeping hot and cold air separate using containment strategies, raised floors, ceiling return plenums, blanking panels, and specialized ventilation systems.
Heat rejection is the process of transferring heat generated by servers and other IT equipment from inside the data center to an outside location. This is usually accomplished through a heat exchanger, a device that transfers heat energy from one fluid to another, such as from air to water. The chosen fluid serves to absorb and dissipate the heat, efficiently cooling the equipment. Next, the heat from the data center is transferred to a location outside, typically released into the atmosphere, but it may also be sent to nearby bodies of water or the ground.
Fluid conditioning involves the maintenance and treatment of cooling fluids, such as water, refrigerants, or glycol, used in the data center. It includes tempering, filtering, treating, and managing the fluid to ensure that it remains effective in absorbing and transferring heat within the data center.
Humidity control means maintaining optimal humidity levels within a data center by using humidifiers or dehumidifiers that add or remove moisture from the air, respectively. In data centers, humidity levels are monitored using hygrometers and sensors that measure the moisture content in the air.
Traditional Data Center Cooling Systems and Methods
Data centers require effective cooling systems and methods to manage temperature and humidity, ensuring the proper functioning and durability of their equipment. These cooling systems transport heat energy from the indoor IT spaces to the outdoor environment. Below is an illustration of a typical data center layout showing some of these cooling system components:
This section delves into the traditional “mechanical” cooling systems and methods utilized in data centers. These encompass air conditioning systems, namely Computer Room Air Conditioning (CRAC) units and Computer Room Air Handler (CRAH) units; airflow management systems, namely hot aisle and cold aisle containment, as well as in-row and in-rack cooling; and water cooling systems using chilled water.
Air Conditioning Systems in Data Centers
Computer Room Air Conditioning (CRAC) Units
Computer Room Air Conditioning (CRAC) units are commonly used in many data centers. These specialized air conditioners are designed to cool the air surrounding the servers, and they often come equipped with humidifiers to ensure specific humidity levels within the data center.
The distribution of cool air can be achieved through raised floor systems or directly by using ductwork. In these systems, pressure serves a critical role in managing air flow. Higher-pressure areas force cool air into equipment inlets, such as those on servers, while lower-pressure regions draw the warm air back into the CRAC system. This drawn-in warm air is then cooled and recirculated.
The interplay between air conditioners and fans creates a pressure differential that regulates proper airflow throughout the data center. This differential ensures that cool air reaches every part of the facility, absorbing heat from the equipment. This absorbed heat is subsequently re-cooled through a heat exchange process within the CRAC system.
Inside CRAC units, coiled heat exchangers are used to absorb warm air from the data center, which allows for efficient heat transfer with different cooling mediums, such as refrigerant, glycol, or water. CRAC systems can be categorized into several types based on these mediums:
- Air-Cooled CRAC: Uses fluid refrigerant to absorb heat from the air inside the data center. The absorbed heat is then transferred outside through a condenser, where it dissipates, and the refrigerant is cycled back into the room. This can also be referred to as Direct Expansion (DX) cooling
- Glycol-Cooled CRAC: Employs glycol as the cooling medium to absorb excess heat from the data center. The heat is then transferred to a dry cooler outside the facility, where it is cooled by ambient air, and the glycol is recirculated to continue cooling
- Water-Cooled CRAC: Utilizes water to absorb heat from the data center. The heated water is then pumped to a cooling tower where the heat is released into the atmosphere through evaporation. The cooled water is subsequently sent back to the data center to absorb more heat
Computer Room Air Handler (CRAH) Units
Computer Room Air Handler (CRAH) units are used in data centers to regulate temperature and humidity, with chilled water serving as the cooling medium. These units distribute cool air either through raised floor systems or ductwork.
By strategically placing CRAH units and controlling air pressure, the effective distribution of cool air is achieved. High-pressure areas guide the cool air into equipment inlets, such as those on servers, while low-pressure areas draw warm air back into the CRAH system.
Within the CRAH system, warm air is pulled into the unit and passes over coils filled with circulating chilled water. This water absorbs the heat from the air, cooling it before it is recirculated back into the room. The chilled water circulates in a loop between the CRAH and a chiller, facilitating a continuous cooling process.
Please refer to the “Chilled Water System” section below for details about the outdoor heat exchange process through the chiller.
CRAC vs CRAH in Data Centers
The key differences between CRAC units and CRAH units lie primarily in their cooling mediums, with CRAC units often using a refrigerant or glycol, resulting in less energy efficiency and more complexity. In contrast, CRAH units draw outside air in and cool it using chilled water, typically offering better energy efficiency and a simpler design. Below is a table comparing these two data center cooling devices:
|Feature||CRAC Units||CRAH Units|
|Cooling Medium||Refrigerant, glycol||Chilled water|
|Energy Efficiency||Generally less efficient||Generally more efficient|
|Temperature Control||Direct control of temperature||Depends on chilled water temperature|
|Humidity Control||Better control||Less control|
|Complexity||More complicated system||Simpler system|
|Cost||Generally more expensive||Generally less expensive|
|Applicability||Smaller data centers||Larger data centers|
Airflow Management Systems in Data Centers
Hot Aisle and Cold Aisle Containment
Hot aisle and cold aisle containment is an airflow management technique used to enhance the efficiency of air conditioning systems within data centers. This system arranges server racks into parallel rows, with one row designated as the “hot aisle” and the adjacent row as the “cold aisle.” Each row is enclosed and faces the opposite direction to its neighboring row, allowing for the separation of hot exhaust air and cold intake air.
In this configuration, hot air vents and cold air intakes face each other, creating alternating aisles of hot and cold air. Hot air is expelled through the vents in the hot aisles, while cold air is directed into the cold aisles. Specifically, the cold air is channeled towards the front of the servers in the cold aisles, and hot air is exhausted through vents at the back of the servers into the hot aisles.
This containment strategy allows the hot air to be routed efficiently to the cooling units. It minimizes the mixing of hot and cold air, allowing cooling systems to concentrate on cooling the hot aisles, which enhances overall efficiency. If needed, doors and walls can be incorporated into the layout to guide airflow even more precisely.
In-Row and In-Rack Cooling
In-row and in-rack cooling are distinct methods used to deliver localized cooling in a data center. These techniques vary based on the placement of the cooling unit, which directly supplies cold air to the IT equipment. Both methods are designed to provide cooling precisely where heat is being generated, as follows:
- In-Row Cooling: Cooling units are placed between server racks, in the same row. By situating the cooling units close to the servers, in-row cooling minimizes the path that the cool air must travel, reducing the risk of mixing with hot air. It can also be customized to handle specific hot spots
- In-Rack Cooling: Cooling units are integrated directly inside individual server racks, providing precise temperature control at the server level. This method can be used for high-density applications where traditional cooling methods may struggle
Water Cooling Systems in Data Centers
Chilled Water System
Chilled water systems in data centers utilize chilled water as the cooling medium to transfer heat from inside the facility to the outside environment. This chilled water circulates through pipes, where it absorbs heat from servers and other devices.
Once the water has absorbed this heat, it becomes warm and is then sent to a chiller. The chiller, which may be located either indoors or outdoors, is a device that cools the water by employing either a vapor-compression or absorption refrigeration cycle. There are three main types of chillers:
- Water-Cooled Chillers: Utilizes a condenser water loop to carry the heat, with a cooling tower that dispels the heat into the atmosphere
- Glycol Chillers: Uses glycol (a heat-absorbing substance with a lower freezing point than water) which flows through pipes to a dry cooler (or fluid cooler) to release the heat into the atmosphere
- Air-Cooled Chillers: Involves a refrigerant that undergoes a phase change to absorb heat, with a condenser cooling the refrigerant and discharging the heat into the atmosphere
These types of chillers all chill the water once again, thereby forming a continuous cooling loop. The method is efficient and scalable, relying on water’s excellent heat transport properties to maintain optimal temperature levels within the data center.
Overall, chilled water cooling systems in data centers can be an integral part of a broader solution, including Computer Room Air Handler (CRAH) units. These CRAH units employ the chilled water to effectively regulate both the temperature and humidity levels within data centers.
New Data Center Cooling Systems and Methods
New data center cooling systems are being used to increase energy efficiency and cooling effectiveness, with the goal of reducing costs and environmental impact compared to traditional “mechanical” methods. This section delves into emerging and alternative cooling systems and methods utilized in data centers. These encompass liquid cooling, free cooling (economization), evaporative cooling, adiabatic cooling, and geothermal cooling.
Liquid cooling is a server-level cooling technology designed to dissipate excess heat from electronic devices in data centers, especially within high-performance computing (HPC) environments. Unlike traditional methods that rely on air, liquid cooling employs a liquid medium, such as water or a specialized coolant, to absorb and carry away heat. This technique offers more precise temperature control and is more efficient for certain applications, such as the training of deep learning models.
Liquid cooling can be categorized into two main types:
- Immersion Cooling: This method involves submerging servers in containers filled with a dielectric (non-conductive) fluid within a closed system. The fluid absorbs the heat produced by the device, converts it into vapor, and then condenses it back into liquid, enabling the device to cool down much more efficiently than it would with air cooling
- Direct Liquid Cooling (or Direct-to-Chip Liquid Cooling): This approach utilizes flexible tubes to channel dielectric fluid – commonly water, ethylene glycol, or a combination of the two – directly to a cold plate situated above a processing chip or other heat-generating component, like the CPU or GPU. The fluid absorbs the heat, turning into vapor, which is then carried out of the equipment through the same tube, effectively removing the heat
In both of these server-level cooling techniques, the extracted heat is transferred to a chilled water loop, where it is then transported back to the data center’s cooling plant. From there, the heat is expelled into the outside atmosphere, completing the cooling process.
Free Cooling (Economization)
Free cooling, also referred to as economization, is a technique employed in HVAC systems that leverages external environmental conditions to cool spaces, thereby minimizing the reliance on “mechanical” cooling systems and refrigerants. This energy-efficient method takes advantage of the ambient temperature and is primarily implemented in two ways:
- Airside Economization: This process uses outside air to cool data centers when the outdoor ambient temperature is lower. It operates by drawing in cooler outside air and expelling the warmer indoor air. This mixing of air creates the desired conditions before the air enters the data center
- Water-Side Economization: This method employs water or a mixture of water and glycol to transfer heat away from the data center. Cooling towers or other heat exchangers are often used for this purpose. The technique leverages cooler outdoor temperatures to chill the water, assisting in the cooling process
Both approaches contribute to energy efficiency by making use of natural temperature differences. It is also worth noting that both Computer Room Air Conditioning (CRAC) units and Computer Room Air Handler (CRAH) units can be configured to operate in economizer mode. The economizer mode takes advantage of outside air temperatures to cool the data center, reducing the reliance on traditional “mechanical” cooling methods.
Evaporative cooling, also known as “swamp cooling” or “wet air cooling,” is a method of removing heat from data centers. This technique involves spraying water onto a surface or into the airstream, where it evaporates and absorbs a significant amount of heat in the process of transitioning from a liquid to a gas. The resulting cooling effect lowers the temperature of the air passing across the sprayed surface or stream, subsequently cooling the equipment within the data center.
This cooling method is especially effective in arid or dry climates with low humidity. Dry air can hold more water vapor, which enhances the evaporation process and makes the cooling more efficient.
Evaporative cooling can be categorized into two main types:
- Direct Evaporative Cooling: In this method, the air being cooled through evaporation is the same air circulated within the data center. Water is brought into direct contact with the air, either by spraying into the air stream or by passing the air through pads soaked with water. As the water evaporates, it absorbs heat, lowering the air’s temperature. The result is cooler, humidified air that is then distributed throughout the data center
- Indirect Evaporative Cooling: This method cools a secondary stream of air or water, which subsequently cools the primary air stream without adding moisture. The secondary stream is cooled by water and then exchanges heat with the primary air stream through a heat exchanger. This cools the primary air stream without adding moisture, and the cooled primary air stream is then used to regulate the temperature of the data center
Adiabatic cooling in data centers refers to a process in which the air is cooled without the exchange of heat with the external environment. In this method, the pressure change of the air within the system leads to a temperature change. When the air is allowed to expand, its pressure decreases, and consequently, its temperature also decreases. This cooled air is then used to cool the equipment within the data center.
Geothermal cooling systems in data centers take advantage of the Earth’s consistent subsurface temperature to cool equipment. These systems circulate a heat-absorbing fluid through a closed-loop pipe system that is buried deep underground. Since temperatures at this depth remain relatively stable, the system can effectively and efficiently transfer heat away from the data center.