Green IT means many things to different people, says Kenneth Brill, executive director at the Uptime Institute in Santa Fe, New Mexico. "For some, it means not printing on paper through automation, not traveling to meetings through videoconferencing, or changing procurement and waste management procedures to responsibly dispose of obsolete equipment," he says. "However, for me, energy consumption is by far the biggest green issue for IT."
IT energy consumption should be of concern to all CIOs and senior technologists. "Global carbon dioxide emissions from datacenters are actually similar to emissions from global air travel," says Killey. "But for every 100 watts datacenters consume, only 2.5 watts result in useful computing." He says that the rest is consumed on ancillaries such as server power supply, fans and infrastructure hardware that cool servers, uninterruptible power supply (UPS), lighting, and central cooling. Indeed, when these overheads are included, datacenter IT spending is growing annually at 20 percent compared with average IT growth of 6 percent. "The case for green IT is clear," says Killey.
The Uptime Institute draws a blunt comparison: "The manager of a manufacturing plant with a $100 million investment would be fired for single-digit asset utilization, or the plant would be closed," says Brill. "IT has single-digit hardware asset utilization on similar investments and nobody even thinks much about it. I don't think this is very green."
Nevertheless, many senior technologists are failing to grasp the challenge. "Banks are mainly relying on the improving carbon credentials of the equipment vendors," says Excelian's Vile. "Multi-core technology is dramatically reducing the power draw. Where they used to need 1,000 blades they now do with 250. People are consolidating workload on the new servers."
New technology will help but it takes time to have an impact. Meanwhile, demand is still growing and the high heat dissipation and energy demands of dense server racks often require changes to the datacenter. "We're seeing a much more rapid obsolescence of facilities now from the old 25-year cycle," says Liam Newcombe, secretary of the Data Center Specialist Group at the British Computer Society. He adds that technology is evolving quickly. "By the time you fill up your datacenter, you may have to re-engineer the power and cooling systems. A modular crop rotation approach is much better, where you buy or refresh the mechanical and electrical components together with the computer hardware at your refresh interval." Such a strategy is justified because, as Newcombe says, "The energy cost and the datacenter infrastructure cost both exceed the cost of commodity servers, so now the increased compute performance of new processors can drive early replacement, particularly in blade systems."
Power Efficiency Puzzles
What are firms trying to optimize-carbon emissions? Energy usage? For Citi the broad strategy is to optimize its datacenters by closing down and consolidating smaller, less efficient facilities. "We're going to two major sites with good performance efficiencies in each region," says Killey. "In Europe, we completed a new facility in Frankfurt with a design power usage effectiveness (PUE) of 1.2 last year and are currently transferring work onto it."
PUE measures the ratio of total facility power to that required by the IT equipment. Average PUE levels in the US are reported to be more than 2.0, with many greater than 3.0. It measures what facility managers can control, up to a point.
"PUE is a great idea, but doesn't go far enough," says Brill, "because it only addresses facilities' power and cooling efficiency. The biggest lever for constraining datacenter energy consumption is in IT hardware buying practices, virtualization, and the killing of comatose servers." However, Brill warns that success by IT in implementing these solutions will make a facility's PUE worse. "IT plug load will drop," explains Brill. "As a result, the utilization of the power and cooling plant will fall, making the plant less efficient. If all you look at is PUE, the facility manager's performance will look worse when in fact total energy consumption has dropped and the enterprise is saving money."
Killey admits that PUE design calculations are always based on full load, which is rarely the case. "The challenge is heightened by the current fall in business activity," he adds. "Although in some cases we're actually seeing increases in load."
"Datacenters have been built to scale up, not down," says Newcombe. "There are no processes or design options. People are really struggling to reduce their costs."
Killey explains that Citi models the thermodynamics of the datacenter rigorously using a variety of techniques to modularize the power and services: hot aisle, cold aisle, for example, cooling only those areas that are in use. "With 1,000 watts per square meter, we're talking megawatts for each datacenter, so serious money is at stake. Everything is subject to a rigid control process," he says.
The modeling is indeed serious. "Our new datacenter effectiveness metric, which we are developing with McKinsey and Co., is called CADE, which stands for corporate average datacenter efficiency," says Brill at the Uptime Institute. After launching CADE last year, Brill will release case studies to show what firms have achieved. For an IT intensive company with a 4,000-square-meter, 5-megawatt datacenter, the projected savings for a 15 percent annual growth rate in new applications are 100 million ($129 million) over four years, a 112,000-ton carbon reduction, and a 6 megawatt reduction in utility load growth, he reports.
Modular design is key to achieving such savings. "Some large operators are using containerized' datacenters where the container includes all the power distribution, air conditioning and IT equipment in one integrated unit," says Newcombe. "These units are best suited to the likes of Google who have large, homogenous environments but can be useful to corporates on a temporary basis." He argues that in a corporate facility you can achieve much of the modularity and greater flexibility in a traditional datacenter with modern design approaches.
A more radical strategy would be to consolidate on new, offshore datacenters, which would be located where energy costs are low. "Some US banks are looking seriously at rotating their heavy portfolio and risk calculations around the world, using grid computing to exploit cheaper and renewable sources of electricity while using office machines that are spare," says Michael Mainelli, executive chairman of Z/Yen Group, a technology consultancy. "Others are looking at locating server farms in cheaper renewable areas, such as Iceland with its geothermal power."
Parallel Virtual Worlds
The roadmap for many of these initiatives often begins with server virtualization. "Cloud computing, especially internal clouds, is driving this consolidation by using virtual machines to move applications around like a mainframe," says Vile.
Brill gives an example of what can be achieved. "One European bank reduced its server count from 3,100 down to 150 just through virtualization," says Brill. "Green IT was not the objective, but the result was a terrific 92 percent reduction in power demands. That's the way to go."
The Royal Bank of Canada reaped similar rewards by going green. "Early in 2007 we realized we were running out of power and space in our datacenters," explains Rob Muroff, director of distributed systems integration at the bank. "We had been adding servers in the traditional way for many years, and decided to go for a virtualization strategy rather than continue to compound the problem. After migrating to an alternative virtualization platform and following a small pilot, we began rolling out in earnest in the spring of 2008."
Muroff reports that, to date, 1,500 small to mid-range servers have been virtualized, with a consolidation ratio of 30:1. "We reckon that might increase to 35:1 or 40:1 with further optimizations," he adds. "It's had an enormous impact on space and power. RBC's users have been very positive and especially like the improved resilience and in some cases the performance improvements."
Given the bank's Microsoft Windows footprint of almost 8,000 servers, Muroff expects that it will take another couple of years to complete the process, which includes Solaris, Linux and all SQL databases as well.
Virtualization is well suited to multi-core chip technology. "For existing applications, virtualization is one way to access the additional performance of the extra cores," says Newcombe. "But individual applications will not run faster."
Speed Demons
As market data volumes explode and complexity increases, many firms have looked seriously at specialist silicon accelerators to replace serried ranks of servers. "People looking at high-end terascale or now even petascale requirements just can't solve their problems with conventional hardware architectures," says Michal Harasimiuk, co-founder of Petapath, a high-performance computing provider. "They look at specialist systems like ClearSpeed that offer up to a 10-time advantage in gigaflops per watt over x86 architectures, and with a much smaller footprint."
The benefits are attractive, he says. "In many cases we've shown banks how to speed up their code by up to 10 times using software enhancements alone and by up to 40 times using the combination of software enhancements and the latest acceleration hardware."
Multi-core chips run slower and therefore much cooler, and observers predict that eight-core chips will arrive to the market this year and 16 cores next year. "This is slowing down adoption of specialist silicon, where the software impact is much greater, even if gigaflop-per-watt ratios are much higher. The re-training and people costs are just too high," says Vile. Nevertheless, he agrees there are still niche opportunities for specialist silicon like general-purpose computation on graphics processing units (GPGPU) or field-programmable gate arrays (FPGAs), but it needs a major redesign. "Just porting conventional financial libraries still leaves you streaming a lot of data on and off the chip," he says.
Harasimiuk, however, sees potential solutions. "The new Open Computing Language (OpenCL), published at the end of last year, will allow quants to implement a model once and then run and optimize it on different hardware architectures without re-coding," he says. "We think this will make a big difference in adoption of heterogeneous grids mixing multi-core x86 and hardware accelerators with outstanding carbon savings."
Thinking Outside the Box
"The real problem, of course, is the overweening importance of latency with its knock-on implications for market data rates and gross power," says Z/Yen's Mainelli. He notes how, currently, banks are co-locating their servers near the exchanges with huge power increments to deal with market fragmentation, but where does it end? "We really need to rethink," says Mainelli. "We could potentially break loose of latency if the exchanges went over to periodic auctions every few seconds instead of this continuous first-in-first-out (FIFO) matching model. That would allow firms to locate their processing where it is cheap and reduce unnecessary data rates. It might also enable more efficient matching for block trades."
We may soon need such radical thinking, according to Mainelli. Environmental impact models like IPAT-where impact is a function of population, affluence and technology-suggest that by 2050 demand is likely to rise by 450 percent. "If then we can't support a 450 percent increase in environmental impact, the implication is that we have to reduce our environmental footprint by 80 percent," says Mainelli.
Reductions of 15 to 20 percent and not 80 percent are what most non-environmentalists think is required. Says Mainelli, "This is only the beginning of a long migration, where we start to internalize the previous externalities of free' natural resources."
Bob Giffords is an independent banking and technology analyst. He can be reached at .
-->