[This continues from  Airflow Management Considerations for a New Data Center: Part 6: Server Reliability versus Inlet Temperature]

In case you missed the first six parts of this seven-part series, I will take just a moment to clarify that this will not be a discussion on the criticality of plugging holes with filler panels and floor grommets, separating hot aisles from cold aisles, minimizing or eliminating bypass and recirculation, deploying variable air volume fans, intelligently locating perforated floor tiles and measuring temperature at server inlets. I do not consider any of those practices to be “considerations”; rather, those practices are what I call the minimum price of admission. None of these practices fall into the state of the art or leading edge categories of data center design, but are firmly established as best practices. By all established industry standards and guidelines, these airflow management tactics are the minimum starting point before you can start benefiting from being able to control airflow volume and temperature – the activity of airflow management, and the key to exploiting both efficiency and effectiveness opportunities in the data center.

Airflow management considerations will inform the degree to which we can take advantage of our excellent airflow management practices to drive down the operating cost of our data center. In previous installments of this seven-part series, I demonstrated that data centers could be run warmer than conventional wisdom would suggest before increased server fan energy reversed mechanical plant savings before server performance was adversely affected and before server price premiums consumed mechanical plant savings. I then suggested chiller-free data centers are much more realistic than conventional wisdom might purport and provided evidence that IT equipment OEM’s tend to generally allow for wider humidity ranges than mainstream standards and industry guidelines. The first five parts of this series provided evidence from manufacturers’ product information, independent lab research results and math models that together make a rather compelling argument for the efficacy of designing, building and operating data centers without chiller plants or refrigerant cooling. In the last piece, I demonstrated how the ASHRAE “X” factor can be used to predict the effect on server life and reliability operating in different temperature environments. Up to this point, our focus has been on the health and well-being of our computer equipment and controlling the expense of managing the environment for that equipment. Today, I depart from my previous focus in two important ways: we’ll discuss the effect of all this on our staffs and I’ll have more questions than answers.

The final consideration has to do with the effect of operating the data center at a higher server inlet temperature on fan noise produced by the servers in response to those temperatures. The ASHRAE handbook provides some general estimates for increased noise exposure at higher server inlet temperatures, summarized in Table 1 below.

These noise level increments are based on fan laws that describe sound power levels of fans increasing with the fifth power of rotational speed, meaning that a 20% increase in fan speed will result in a 4dB increase in noise level.² However, my own first pass at this baseline assessment produced slightly different results. I looked at the server energy increases at different elevated inlet temperatures compiled by the ASHRAE Technical Committee 9.9 IT Sub-committee³ and backed into a fan energy estimate per the methodology I used in part 1 of this series.⁴ In short, I assumed an 800-watt server with a nominal fan energy budget of 80 watts at 68˚F inlet temperature and applied the total server energy increase at each temperature increment to the fans. Those results are reported in the Energy Δ column in Table 2 below. The Increase column is merely the percent increase in fan energy at each temperature increment. Then I applied the affinity fan law of (Q₁/Q₂)³ = P₁/P₂ (changes in fan energy are the cube of changes in fan rpm), by taking the cube root of each increase in fan energy to calculate the RPM ratio against the baseline, in this case 68˚ and 72˚, e.g. ∛1.1 = 1.032, or a 10% increase in energy equated to a 3.2% increase in fan speed. The noise Δ is calculated by the following equation:

Lw_a = Lw_b + 70log₁₀ (d_a/d_b) + 50log₁₀ (n_a/n_b)
Where:
Lw = Sound power level in decibels
d = Rotor diameter
n = Rotor speed
And additional subscripts:
_a = Data at required performance condition
_b = Data at base performance condition

Since we don’t plan to open up our servers and change any fan sizes, d_a/d_b will always be 1 and 70 times the base ten log of 1 is 0, so we don’t need to concern ourselves with anything but the last addend. Therefore, for example, at 86˚F server inlet temperature, our fan speeds have increased 9.3% from our base condition, so multiply 50 times the base 10 log of 1.093 to get 1.931 dB as the increased server fan noise at 86˚F versus 68˚F. This 1.931 dB is significantly less than the 4.7 dB expected increase in sound power level from ASHRAE’s Table 1 above, as are all the calculated values in Table 2, especially since decibels are a base 10 logarithmic scale. Double checking my methodology against the example from the ASHRAE passage of a 20% fan speed increase producing a 4 dB sound increase, 50 times the base ten log of 1.2 is 3.959, close enough to “4” to be a reasonable confirmation, so the differences are likely attributable to different assumptions about server fan speeds at different inlet temperatures. On closer examination, the ASHRAE calculations appear to be based on legacy Class A1 servers, rather than on current generation Class A3 servers. When comparing the different scenarios in Table 6, readers should consider whether their experience will be with newer servers or legacy equipment.

A very thorough study of data center noise was presented at last year’s MIPROS Conference and included noise level measurements from an actual case study data center. One of their conclusions was that noise levels were typically higher in hot aisles than in cold aisles, which is only partially demonstrated by their data summarized in Table 3 below. While this data is useful as contributing to the data set for helping us understand noise levels in data centers, I am not sure how much useful information it will contribute to our discussion on the effects of higher temperatures on data center noise. For example, the site plan from which this data was collected showed various numbers of in row coolers in different rows and the data suggests the in row coolers may have been more responsible for noise levels than the difference between hot aisles and cold aisles.

Table 4 below is from the manufacturer’s documentation for the row-based coolers used in the study reported in Table 3 and application of these sound specifications show how much they contribute to the overall noise data collected.

What is particularly interesting in this product data is that it specifies sound measurements at six feet from the sound source, and the data center study sound data was all recorded at head level in the center of either a hot aisle or cold aisle, which puts those data points all within two feet of sound sources, whether that be a server cabinet or a row-based cooling unit. The equation for determining the loss of sound pressure over distance is 20 times log base 10 of the distance, so at six feet that loss would be 15.56 dB. If we use the 72.4 dB at 70% fan speed from the manufacturer’s documentation and add that 15.56 loss at six feet, we calculate 88 dB at the source. At two feet from the source (20log₁₀2) we get 3.5 dB, which we subtract from the source sound pressure level and get 85.5 dB in the center of a hot aisle right behind one of those coolers. I suspect the data from the Miljkovic study would be quite a bit different if the space was using perimeter precision cooling or economizer air movers outside of the white space.

In fact, server fans are obviously not the only source of noise in the data center and when cooling units are on the floor, they are typically the largest contributor to the overall noise level. However, may I remind the reader, our subject today is the effect on noise levels of raising temperatures, which means our objective here will typically work in our favor with cooling equipment. Table 5 summarizes the noise characteristics of different fan types and in a broad brush stroke, we will see axial fans in servers and centrifugal fans in cooling equipment, with a variety of notable exceptions.

The noise Δ equation we used earlier to calculate noise increases in servers at higher temperatures works in our favor with our cooling equipment as we take advantage of excellent airflow management and resultant higher temperatures to ratchet down our cooling unit fans. In fact, I have seen examples of data centers increasing their cooling unit redundancy from N+1 to 2N merely by reducing demand through good airflow management. When we can reduce our air movers from 80% RPM to 50% RPM, we not only cash a check for a 75% energy reduction, but we enjoy a 10.2 dB reduction in noise, or 1/10 the sound power or approximately 1/3 sound amplitude. Given the logarithmic reduction in noise over distance, the degree to which mechanical plant noise can either be mitigated or essentially removed from the data center space, the effect of server fan noise sources at higher power levels can be space-contained to minimize the hazard in perimeter work areas.

To further illustrate our lack of precision and resolution on the issue of noise in the data center at higher temperatures, I have compiled the different scenarios I have discussed in Table 6 below to offer some comparisons of these different data points. The 70 dBA and 80 dBA baselines are from multiple sources as the typical range for noise levels in data centers, which will include all the IT equipment, as well as electrical and mechanical equipment, i.e., cooling. The 75.5 dBA baseline is the average of all the hot and cold aisle measurements in the study reported on by Miljkovic and the 84 dBA baseline is from an ASHRAE assumption⁸ that may be somewhat alarmist or perhaps based on projected densities not yet realized in most of the data center marketplace. The left-hand column under each of those four baseline scenarios shows the incremental increase in server fan noise at the different temperature increments for ASHRAE Class A3 servers and the right-hand column under each of those scenarios shows those noise level predictions for ASHRAE Class A1 Servers.

To determine if these estimated incremental increases in sound power level are problematic, they need to be considered in terms of the total environment into which they are being introduced. The baselines could be adjusted up or down by the type of cooling deployed, the location of deployment and level of concurrently running redundancy. Hearing protection programs are mandated beginning at exposure to 85 dBA and Table 7 shows maximum exposure time requirements and recommendations for different hazard levels. Prudence would suggest the efficacy of at least doing a noise level check in any data center. Moreover, the data reported on here clearly points to the need to involve an acoustics engineer for any plan to allow server inlet temperatures to creep up into the 95˚F range for the development of a monitoring and protection program, as well as developing mitigation architectural elements into the total space design for absorption and redirection strategies.

In conclusion, in addition to protecting data center workers and visitors from suffering hearing loss and complying with relevant safety regulations, there is a body of research indicating a relationship between noise exposure and such non-auditory conditions as a distraction, endocrine responses, psychiatric disorders and cardiovascular disease.¹⁰ It obviously makes sense to pay attention to this element of our data center design and operation. I have reported on relevant research and basic engineering principles to show where auditory hazards in the data center can be expected. Nevertheless, there is a shortage of extensive research and our inevitable movement toward higher densities and increased energy conservation awareness, accompanied with likely higher operating temperatures, suggest it would be useful to more precisely understand the impact of moving all cooling equipment outside of the white space. It will also be useful to understand how much noise exposure is reduced by aisle containment structures and cabinets with closed loop exhaust chimneys, and what practical approaches might eliminate all air movement for the data center from people-occupied space.

¹ From Table 2.7, “Expected Increase in A-Weighted Sound Power Level”, Thermal Guidelines for Data Processing Environments, 4th edition, ASHRAE TC9.9, page 27
² Thermal Guidelines for Data Processing Environments, page 26
³ Figure 2.7, Thermal Guidelines for Data Processing Environments, page27
⁴ “Airflow Management Considerations for a New Data Center – Part 1: Server Power versus Inlet Temperature,” Upsite Technologies Blog, May 3, 2017
⁵ “Noise Within a Data Center,” Dubravko Miljković, MIPROS 2016 Conference Proceedings, Croatia, May 2016, page 1354
⁶ Manufacturer’s technical product bulletin
⁷ “Fan Noise Prediction,” Course handout and notes for ME458: Engineering Noise Control, John Lamancusa, Pennsylvania State University, fall 2000
^7a Axial fan blade passing tone can be high, depending on flow obstructions
⁸ Thermal Guidelines for Data Processing Environments, page27
⁹ Miljkovic, page 1352
¹⁰ “Noise Pollution: Non-auditory Effects on Health,” Stephen A Stansfeld and Mark P Matheson, British Health Bulletin, Volume 68, Issue 1, December 1, 2003