Why a Data Center Hotspot Escapes the Average Temperature

A room or rack face whose average temperature sits inside ASHRAE’s recommended range is widely read as safe, and many computer room air conditioners treat the return air temperature as the temperature of the space, a practice the 2006 PG&E design guide flags and a 2009 Lawrence Berkeley National Laboratory report calls the most common control method. That reading does not hold, because ASHRAE judges compliance at each equipment air intake, and one intake can run hot while every average stays in range. This piece covers air-cooled, open raised-floor rooms with hot and cold aisles, where the evidence below comes from, and leaves out contained aisles and liquid-cooled racks.

Compliance measured at each air intake

ASHRAE TC 9.9 defines its thermal classes by the air entering IT equipment. The recommended range is 18 to 27 °C for classes A1 through A4.

For class A1 the allowable range is 15 to 32 °C, so its upper limit sits 5 K above the recommended one. Those two upper limits are the thresholds the rest of this piece measures against.

Source: ASHRAE TC 9.9, Thermal Guidelines for Data Processing Environments, 4th edition reference card (2015 table)

The measurement point follows from that definition. In a 2020 report for the Federal Energy Management Program, LBNL describes ASHRAE’s placement as sensors 2 inches in front of the equipment, at the geometric center of the air intake of the top, middle and bottom units in a rack. The same report says most ambient temperatures have very little to do with actual IT equipment intake temperatures.

A room average, a return-air reading or a rack-face mean is therefore a proxy for the compliance metric, not the metric. The proxy works only when the air in front of the racks is well mixed. Schneider Electric’s White Paper 199 notes that hot aisle and cold aisle layouts purposely create areas of hot and cold temperatures, so the proxy fails by design in the rooms that rely on it most.

Recirculated exhaust over the top shelf

Herrlin’s 2005 ASHRAE Transactions paper on the Rack Cooling Index modeled the mechanism with computational fluid dynamics, supplying 16 °C air from a raised floor at a load of 150 W/ft².

In the paper’s Figure 3, the bottom and middle intakes of each rack received 16 °C air and the top received 31 °C.

Source: M. K. Herrlin, ASHRAE Transactions 111(2), 2005, Figure 3

The paper attributes the step to recirculation at the top of the racks. The supply air did not submerge the top shelves, partly because cold air was lost at the ends of the cold aisles, so the upper equipment drew exhaust that had looped over the rack. An overhead, top-down system at the same density gave intakes of 22 to 25 °C at all three heights.

LBNL’s 2020 report describes the same temperature distribution as typical. In an open raised-floor room the transitional and hot layers generally sit in the upper third of the cold aisle, and the vertical gradient is large next to the gradient along the aisle. Most data centers, it adds, have a combination of bypass and recirculation. This is the same tile-and-airflow constraint that sets the per-rack air cooling limit, seen at the scale of one rack face.

A measured case shows it too. White Paper 199 assessed a legacy room in which 7.2% of the total IT airflow recirculated, and its Figure 1 labels the lower front of a rack at 21.5 °C and the upper front at 27.5 °C.

Source: P. Lin, Schneider Electric, How to Fix Hot Spots in the Data Center, White Paper 199 (2014)

A data center hotspot in these rooms is a layer problem, not a room problem. The layer sits where a rack has the fewest sensors and where the cold supply air arrives last.

Averages that favor the wrong rack face

Treat the three labeled heights in Herrlin’s Figure 3 as three sensors on one rack face and take the plain mean of the readings.

Bottom-up rack face: (31 + 16 + 16) ÷ 3 = 21 °C.

Top-down rack face, left lineup: (24 + 22 + 23) ÷ 3 = 23 °C.

Both results sit inside the recommended range, and the mean ranks the bottom-up face as the cooler one. It is the only face with an intake above 27 °C. A mean lets over-cooled positions offset a hot one, a trade no server at the top of a rack can make. The calculation uses only the three labeled points per face, so it illustrates the averaging rather than reporting a measured rack mean.

Herrlin built the Rack Cooling Index to remove that offset. The high-side index is RCI-HI = (1 − Σ(Tx − Tmax-rec) ÷ ((Tmax-all − Tmax-rec) × n)) × 100%, where the sum runs only over intakes warmer than the maximum recommended temperature and n is the number of intakes. Over-cooled intakes earn no credit, so local overheating cannot be averaged away.

The DOE Air-Management Tool rates an RCI of 100% as ideal, 95% as good and 90% as acceptable. It appends an asterisk when at least one intake is above the maximum allowable temperature.

Source: LBNL and ANCIS, DOE Data Center Air-Management Tool: Engineering Reference, v3 (2023)

Now apply the current A1 limits, Tmax-rec = 27 °C and Tmax-all = 32 °C, to an assumed row of 10 racks with 10 intakes each, so n = 100.

Case 1, the top intake of every rack reads 31 °C and all others stay at or below 27 °C: the excess is 10 × 4 = 40 K, and RCI-HI = (1 − 40 ÷ (5 × 100)) × 100% = 92%.

Case 2, only one rack has the hot top intake: RCI-HI = (1 − 4 ÷ (5 × 100)) × 100% = 99.2%.

These are worked examples with assumed intake counts, not values from the paper, which used the older 2004 limits. No asterisk appears in either case, because 31 °C is below the allowable maximum. The index is a clear improvement on the mean, since nothing cancels the excess, but it still divides by n, and one server drawing 31 °C air in that row scores close to ideal. The maximum intake temperature is the number that names the hotspot, and RCI says how widespread it is.

The top third of the rack face

Return air is the weakest of the proxies. LBNL-3137E describes return-air control as a blended air temperature regime that masks the actual needs of the servers, and calls the return the worst location for maintaining temperatures at the IT inlet. In the White Paper 199 room, the return duct is labeled 23.2 °C while the upper front of a rack reads 27.5 °C.

Where the intake sensors sit decides whether the hotspot shows up at all. LBNL’s 2020 simulation compared two 18-sensor schemes with a 144-sensor reference that read every intake, in a room with a shallow hot layer.

Table 1. What an 18-sensor scheme reports against a 144-sensor reference (°F as published, °C converted)

* Computer simulation, shallow hot layer, 16 racks × 9 IT units. LBNL compares schemes; absolute RCI values not the point. °C = (°F − 32) × 5/9.

Source: Lawrence Berkeley National Laboratory (M. Herrlin), “Thermal Guidelines and Temperature Measurements in Data Centers” (15 September 2020), Table 2

The knee, hip and head scheme came closest to the reference index but reported a maximum 3 °F below the true peak, because the hot layer sat above its top sensor. LBNL’s remedy is to add a few sensors per row near the top of the racks. The DOE FEMP best practices guide, revised in 2024, goes further and recommends controlling supply temperature and airflow on IT equipment intake temperatures and not on the return temperature.

Moving control points or setpoints in an operating room is a design change that needs qualified engineering review. Before trusting a temperature that says a room is fine, check these:

• Whether the sensor sits at an equipment air intake, or in the return, the aisle or the room
• Whether any sensor reads the top unit of the rack face, including end-of-row racks
• The maximum intake temperature, reported next to any average or index
• Whether the cooling units control on intake temperature or on return air
• Whether an RCI figure carries the asterisk that marks an intake above the allowable limit

References

• Thermal Guidelines for Data Processing Environments, 4th Edition, Reference Card / ASHRAE TC 9.9 / 2015 / https://xp20.ashrae.org/datacom1_4th/ReferenceCard.pdf
• Thermal Guidelines and Temperature Measurements in Data Centers / Lawrence Berkeley National Laboratory for U.S. DOE FEMP (M. Herrlin) / September 15, 2020 / https://datacenters.lbl.gov/sites/default/files/FINAL%20Thermal%20Guidelines%20and%20Temp%20Measurements%209-15-2020.pdf
• Rack Cooling Effectiveness in Data Centers and Telecom Central Offices: The Rack Cooling Index (RCI) / ASHRAE Transactions vol. 111, part 2 (M. K. Herrlin) / 2005 / https://www.e3tnw.org/images/RCI.pdf
• DOE Data Center Air-Management (AM) Tool: Engineering Reference, v3 / Lawrence Berkeley National Laboratory and ANCIS / March 15, 2023 / https://datacenters.lbl.gov/sites/default/files/2023-03/DOE%20AM%20Tool–Eng%20Reference%20v3%20%283-15-2023%29.pdf
• Control of Computer Room Air Conditioning using IT Equipment Sensors, LBNL-3137E / Lawrence Berkeley National Laboratory (G. C. Bell, B. Storey, M. K. Patterson) / September 2009 / https://eta-publications.lbl.gov/sites/default/files/lbnl-3137e.pdf
• How to Fix Hot Spots in the Data Center, White Paper 199 Rev 0 / Schneider Electric (P. Lin) / 2014 / https://www.se.com/au/en/download/document/SPD_VAVR-9GNNGR_EN/
• Best Practices Guide for Energy-Efficient Data Center Design / U.S. Department of Energy, Federal Energy Management Program / July 2024 / https://www.energy.gov/sites/default/files/2024-07/best-practice-guide-data-center-design.pdf
• High Performance Data Centers: A Design Guidelines Sourcebook / Pacific Gas and Electric Company / January 2006 / https://datacenters.lbl.gov/sites/default/files/DCdesignPGE2006.pdf