Regional Data Centers

Case study

Background

The processing power and storage capacity of computer systems are rising dramatically, and these capabilities are being provided in ever‐smaller physical devices. These dense, high capacity devices result in exponentially greater server‐ room power and cooling requirements. For current systems, the power and cooling costs of a compute‐cluster surpass the equipment costs during the lifetime of the equipment, normally within three years. As a result, during the next decade, the overall cost to the University of California associated with locating electronic systems within its existing high power costs / low power efficiency data centers will increase significantly given the aforementioned increase in higher density computing. Compounding the problem is the fact that many systems are located in substandard or inadequate spaces designated originally for instructional or light office use. To varying degrees, all University of California campuses and medical centers have a vested interest in ensuring that appropriate system‐wide colocation facilities are available to cost‐effectively support the growing cyberinfrastructure needs of the University of California collectively.

Goal

As a result, the UC Information Technology Leadership Council (ITLC) has taken this opportunity to explore utilization of the San Diego Supercomputer Center (SDSC) data center as a regional colocation facility with the objective of meeting the needs of campuses and medical centers throughout the system.

Successes

Approximately 18 months ago, UCSD Chancellor Mary Anne Fox committed space for 225 racks at SDSC for use as a UC regional colocation facility. Currently, 115 have been used by campuses to date.

  1. All UC campuses (and several medical centers) have
    created 18 month plans for placing computer systems /
    racks at the SDSC facility

  2. All UC campuses (and several medical centers) have
    created plans to place at least one rack (and in many cases
    several racks) at SDSC during the next six months.

  3. The development of regional colocation “communications plans” is underway at all UC campuses; these communications plans will facilitate campus efforts to promote and more fully utilize regional colocation facilities
    (and SDSC in particular).

Challenges

As ITLC has explored the use of SDSC, the issue of “hidden” power costs has slowed overall utilization of the facility for systemwide colocation needs. Campus power costs are in many cases unknown to administrators and principal investigators; in other cases, they may be known to some extent, but cannot be charged to individual grants or programs. Because of this, PIs and administrators locate computer equipment on campuses despite higher total costs both to the campus and the university as a whole versus placing the equipment in a regional data center.

This is the fundamental conundrum facing the system. The method of acquiring and paying for electricity within UC is both inefficient and costly, but there is no formal mechanism to utilize savings in (campus) expenditures on electrical costs in support of a more effective and efficient solution, namely, locating computers/systems in a regional data center.

Initial investment

The University of California is actively reviewing options relating to the investments and incentives that will be necessary to locate additional systems, servers, and racks at the SDSC regional colocation facility. Additionally, in time, UC may consider investments associated with constructing and maintaining a new colocation facility that will address UC’s collective needs once the SDSC space has been exhausted.

Fiscal results, current and anticipated

UC is reviewing cost savings associated with the colocation initiative. However, a recent study indicates that, in general, campuses electricity rates exceed SDSC’s by 35%‐50%, and this electricity is less effectively utilized (again, by 35%‐50%).

Current action and next steps

While continuing with migration of racks to SDSC, and development and dissemination of campus communication plans, ITLC will develop for review by the UC senior management a set of options for efficient, state‐of‐the‐art facilities that meet campus and medical center research, clinical, and general business needs in ways that bring financial savings and efficiencies in the face of exponentially growing power and cooling demands.

Concluding statement

Modern colocation facilities hold great promise as tools to contain costs and reduce UC’s demand on scarce resources. UC should therefore create the policies, processes, and procedures that will enable campuses and medical centers to recognize the real costs of inefficient local facilities and allow campuses to re‐allocate funds that would otherwise be spent on local power costs in support of more efficient (and green) solutions within regional colocation facilities.