Data center that point of the cold plate liquid cooling
Date：2023-11-03 | Hits：613
introduction
With the rapid development of new generation information technologies such as cloud computing, big data and artificial intelligence, data has witnessed explosive growth. As the key infrastructure and important guarantee to promote the rapid development of financial technology and the digital transformation of the financial industry, the construction scale of data centers is also expanding, and the proportion of energy consumption is getting higher and higher. At the same time, the continuous promotion of the "dual carbon" goal and the continuous tightening of energy-saving regulatory policies at the national level have also put on the "straitjacket" for the development of data centers, and relevant departments such as the National Development and Reform Commission, the Ministry of Industry and Information Technology, the Government Offices Administration, and the Energy Administration have frequently issued joint documents to put forward clear restrictions on PUE. The average power utilization efficiency of new large and super-large data centers in the country has dropped below 1.3, and the "East Number and West Count" project has also strengthened the green development requirements for 10 data center clusters of eight computing power hub nodes, controlling the PUE index within 1.25. In addition, the proliferation of compute-intensive application scenarios and the trend of cloud on the business have led to a significant increase in the power consumption of the server devices carrying the load of these applications, the wattage of heat generated per unit space in the data center is rising, and the power density of the single rack is getting higher and higher. Therefore, new cooling and cooling technologies are needed to match the increase in equipment power density brought about by the increase in demand and meet the policy requirements brought about by energy-saving supervision.
Under the background of internal and external demands such as policy pulling and data center energy saving and efficiency reduction, liquid cooling as an advanced and efficient refrigeration solution that can support higher power density came into being, among which cold plate liquid cooling has advantages of mature technology, high heat dissipation efficiency, strong adaptability, low system noise and low operation TCO cost. It provides a new idea of energy-saving innovation management for the development of green and low-carbon computing power center.
02
Technical analysis of cold plate liquid cooling system
2.1 Cooling principle of cold plate liquid cooling system
Cold plate liquid cooling is mainly through the closed cavity composed of high thermal conductivity metals such as copper and aluminum, the heat of high heat density components such as chip, CPU/GPU, memory is indirectly transferred to the cooling liquid closed in the circulation pipeline, and then the cooling liquid is used to take the heat away, and the intermediate heat is transported to the back end for cooling according to the transmission characteristics of the working fluid. Through the combination of the primary side and the secondary side, the whole liquid circulation of the cold plate liquid cooling system is realized, so as to achieve the goal of heat dissipation for IT equipment.
Figure 1 Architecture diagram of cold plate liquid cooling system
2.2 Advantages of cold plate liquid cooling system
(1) Good material compatibility. Because the coolant flow in the pipeline is not in direct contact with the motherboard and chip module, the material compatibility is strong. When selecting the coolant, you can only consider the compatibility between the coolant and the circulation line and the cold plate
(2) Strong adaptability and convenient maintenance. Cold plate liquid cooling does not change the original form of the server motherboard, and can be directly compatible with the original refrigeration system (conventional chilled water system), which has better feasibility in technology, industry and large-scale production. And the server installation and maintenance is basically the same as that of conventional air-cooled servers.
(3) The TCO cost is low, and the operating PUE can be as low as 1.2. For traditional rack-mounted air-cooled servers, resource utilization has been significantly improved, while the use of high specific heat of the liquid working medium, less cycle energy consumption and simplified heat transfer process, so that limited energy can be allocated to more computing power, thereby reducing operating costs and increasing computing power output.
2.3 Cold plate liquid cooling system design
The cold plate liquid cooling system consists of two parts: the primary side (outdoor) and the secondary side (indoor). The primary side refers to the loop that drains the heat from the secondary side to the outdoor environment or other heat recovery units, including the outdoor dry cooler and chiller and other cold sources. The secondary side refers to the loop that takes heat from the server and dissipates heat in the primary side, including the liquid supply loop and the internal flow channel of the server. Wall heat transfer occurs between the two parts through the plate heat exchanger in the CDU (cold volume distribution unit), and the working medium is not mixed.
The secondary solution is fixed and configured using the CDU and the subsequent system architecture. The primary side mainly has the following four solutions according to the usage conditions and usage scenarios:
(1) Water-cooled chiller + cooling tower (open) + plate change
In high-heat and high-humidity areas, the environment requirements of the machine room are high, and the direct use of closed cooling tower/dry cooler can not directly meet the cooling requirements, and auxiliary mechanical refrigeration devices are needed, and the cold source is usually the combined cooling mode of chiller + cooling tower. The system has two operating modes:
Mode 1: The outdoor temperature is low, the cold tower + plate can meet the refrigeration requirements, without the need to open the cold machine.
Mode 2: The cold tower outlet water temperature is higher than the CDU requirement, and the cold tower and chiller need to be mechanically supplemented for cooling.
FIG. 2 Schematic diagram of cooling tower + water cooler + board changing system
(2) Air cooled chiller
The air-cooled chiller integrates the condenser, water pump, compressor and other components as a whole, and is usually equipped with free cold source modules (such as dry coolers) to make full use of outdoor natural cold source, high integration, suitable for small systems and water shortage areas. The system also has two operating modes:
Mode 1: When the outdoor temperature is low, the free cold source module can meet the cooling requirements without the need to open the chiller.
Mode 2: The free cold source module cannot meet the temperature requirements of the CDU and directly uses the air cooler mode.
FIG. 3 Schematic diagram of air-cooled chiller system
(3) Closed cooling tower/dry cooler For the local temperature is low throughout the year, the use of closed cooling tower/dry cooler can meet the CDU temperature requirements, no mechanical refrigeration throughout the year. Closed cooling tower mainly evaporative heat dissipation, can output lower temperature, circulation system water quality is good, for CDU and other heat exchange equipment friendly, the disadvantage of large water consumption; The dry cooler is large in volume and the cooling capacity of a single machine is small, but it is easy to deploy, and evaporative cooling can also be partially used after the configuration of wet film.
FIG. 4 Schematic diagram of closed cooling tower system
(4) Open cooling tower
Open cooling tower and closed cooling tower cooling mode is exactly the same, the disadvantage is that the water and the atmosphere are connected, resulting in poor water quality.
FIG. 5 Schematic diagram of open cooling tower system
The preceding solution takes the liquid cooling side as the main factor. In actual operation, the equipment room needs to be equipped with a small number of air conditioners to meet the heat dissipation requirements of low-heat density components (such as hard disks and interface cards) of the server.
that the main liquid-cooled equipment room (including cabinets and IT devices), CDU, and secondary pipes be provided by the same manufacturer to ensure delivery integrity.
(2) The design method of data center using cold plate liquid cooling technology is different from the traditional data center using air cooling, and the energy-saving transformation of the stock data center, such as the load-bearing load of the machine room, the height of the machine room, and the pipeline route under the floor need to be re-accounted. If you hastily replace the cooling method, in addition to affecting the space utilization rate, personnel maintenance and reconstruction costs will also be a large expenditure.
(3) At present, some enterprises and associations have launched standards in the field of liquid cooling segmentation, which can provide framework guidance, but the detailed design of key components such as CDU, quick plug joints, liquid supply and return pipes, cold plates and other specifications have led to compatibility problems of various manufacturers' products, and the limitations of use from the perspective of customers are not conducive to large-scale deployment of mass deployment.
(4) Regarding the size issue, if the liquid cooling manufacturer provides cabinets, CDU and inter-column air conditioners at the same time, the size will be considered in a unified planning, and there will generally be no problem. However, if the cabinet is a complete set of IT equipment manufacturers, the size is not uniform after the arrival of difficult to deal with, re-unified size will face the problem of delivery time and cost increase, the implementation of non-uniform size can not be neat and beautiful, affecting the sensory effect.
Under the background of the current wave of information creation and digital transformation, the cloud migration of information systems in the financial industry and the popularity of Internet financial products have increased the demand for agile response and immediate change of business in the financial industry, and the demand for computing power will further increase, which will bring about the increase of heat dissipation of high-density server deployment and the power density of single cabinet. Cold plate liquid cooling can be regarded as a reference path.