The ARM Data Management Facility (DMF) is the initial collection point for raw data from all the ARM sites. In June, the DMF completed installing and testing $643,000 of new computers and equipment purchased through the American Recovery and Reinvestment Act of 2009. These upgrades increase the DMF processing and storage capacity by 6-10 times to support the current and future demand for ARM computing power.
Data arriving at the DMF are processed once an hour, every day, 365 days a year. This processing activity includes routine monitoring for readability and usability before the data are transferred to the ARM Data Archive. In addition, the DMF provides computational resources for scientists to develop “value-added products”—physical models that use ARM data as inputs to fulfill unmet measurement needs and to improve the quality of existing ARM measurements.
With the influx of funds from the Recovery Act, new and upgraded instruments throughout the user facility will result in a large increase in data flowing through the DMF, as well as increased downstream processing for data products. The recent upgrades and new equipment improve not only the performance and reliability of the DMF services, but also provide increased computing capability for scientists to develop and run algorithms for new data products. Matt Macduff, engineer for the DMF at Pacific Northwest National Laboratory, explains more in this YouTube video.