source:
computerworld.com.au/index.php/id;101366230;fp;16;fpid;0
--- copy & paste ---
Case Study: Exchange makes Linux call
Patrick Thibodeau
05/01/2005 07:30:04
The Chicago Mercantile Exchange credits its migration to commodity Intel-based servers and Linux with cutting costs and shaving 100 milliseconds off the time required to complete a trade.
In a quiet corner office of Chicago Mercantile Exchange, high above the steady roar of shouting traders on the exchange floor, Joseph Panfil, director of distributed computing, is focused on the smallest increments of time.
There is a direct relationship between system performance and trading volume at the CME, also known as the Merc. By reducing the amount of time it takes to process a trade, trading volume can increase. The more trades there are, the more trading fees the exchange collects.
Time is the bottom-line metric of Panfil's world. "Everything in the trading world is a matter of split seconds," he says.
Panfil is part of a team implementing system changes that have so far reduced round-trip trading times from about 1,800 milliseconds in 1998 to 350 milliseconds today. Five trades can now be conducted in the time it used to take to complete one.
To cut times further and reduce IT costs, the CME is pursuing a major IT overhaul that involves a gradual adoption of the Linux operating system on Intel-based servers while moving off Sun Microsystems' Solaris and Sparc hardware. The conversion began about a year ago and is now 30 percent complete.
Complex Architecture
The CME's architecture consists of three major technologies: Unix and Linux systems handle the input paths for orders and output paths for quotes. Hewlett-Packard NonStop servers send out order confirmations and quotes. And IBM mainframes are responsible for "clearing," or processing, of all trades.
Traders still work shoulder to shoulder on the CME's floor, but since June, the majority of trades have been handled by the Merc's electronic trading system, Globex. The Merc began electronic trading in the early 1990s, using a third-party system to support after-hours trading. In 1998, it introduced its own system.
Customers want reliability and speed in electronic trading environments, says James Krause, the Merc's CIO. Balancing those two priorities means that the exchange, while interested in performance-improving technologies, isn't likely to be the first adopter of a new system. "Whatever we do, we have to make it work right and make sure it's fast," Krause says.
Trading times have been improved by making applications more responsive, upgrading server CPUs and undertaking architectural changes to reduce the number of hops in the path of a trade. For instance, the process previously used to include a disk write, but that has been moved out of the direct path of a trade.
The decision to use Linux and move to fast Intel chips also led to a reduction in trading time. The Merc credits its Linux deployment with cutting the time it takes to process a trade by about 100 milliseconds to the current 350 milliseconds. The goal is to reduce that to 100 milliseconds. Exchange officials say they believe that using faster Intel chips will bring them halfway toward that goal and that application optimization on the NonStop servers will take care of the rest.
But the Merc's interest in Linux was also sparked by a desire to save money. Leveraging competition among vendors of Intel-based commodity hardware is an important element of this strategy. "One of the things that we were trying to do with Linux is to be totally agnostic and not get tied into a vendor," says Panfil.
Even before the Merc deployed any Linux systems, it had already reduced some costs. "We started using that cost goal against Sun to get them to lower their price and be more competitive," Panfil says. In one case, Sun reduced the price for an US$18,000 two-way UltraSparc server to less than $10,000. That reduction came "just by having that competitive threat," says Panfil. However, a comparable two-way Intel-based server cost just $3,000, he notes.
Preparing for Linux
The Merc had to address several issues before it could deploy Linux. Third-party software had to be approved to run on the operating system and tested for speed and stability. And the Merc's technical staff also needed training.
There are 19 people in its Unix area, and some were already well versed in Linux. "They were doing this stuff in the basement of their homes. It was good that we had some people on the leading edge," says Panfil. When some Solaris administrators resisted the change, Panfil told them Linux knowledge would broaden their skills. "Why not get that other skill?" he told them.
Testing also showed that the Merc could move some applications off a four-CPU Sparc-based server, replace it with a two-way Intel-based commodity server and not lose any performance. Moreover, the leaner Linux code helped speed up performance on some applications.
When the Merc began considering its Linux deployment more than a year ago, Sun knew it wasn't in a strong position to compete. "We didn't have a good answer for them," acknowledges Glenn Weinberg, vice president of the operating platforms group at Sun.
At the time, Sun didn't have its x86 product line or Solaris running on x86 in "any serious way," Weinberg says. But today, he notes, "those things have both changed." The vendor has made a "tremendous investment" in getting Solaris to run on x86-based machines, and Solaris is outperforming Linux on some financial services applications, Weinberg claims. Sun's x86 line includes Advanced Micro Devices' Opteron processors and Intel's Xeon. "We think we're in an increasingly good position there," he says.
CME's goal is to convert 45 percent of its Solaris systems to Linux by year's end. Originally, the exchange planned to migrate all of its servers off Solaris, but applications on many of the servers are written specifically for Solaris, and it now has no plans to migrate those.
Dan Kusnetzky, an analyst at IDC, says the Solaris kernel may be better tested, more reliable and more scalable than Linux. But he sees Linux, and the freedom it represents, as having powerful appeal to businesses. Whether Sun's late response can counter the threat remains to be seen. "It's going to be difficult to come back and recover that territory," he says.
Still, the Merc isn't closing the door on Solaris. Panfil will continue to evaluate Sun's products, and he says the vendor's planned year-end rollout of Solaris 10, which will run on Opteron processors, could be a viable alternative to Linux. Panfil isn't worried about supporting both operating systems. "We have proven we can support Solaris and Linux in parallel," he says, citing the expertise available on his staff.
The move to commodity-based systems has also made the Merc more aggressive in seeking the best deals. HP is supplying the x86 servers the exchange uses, but Panfil says it could just as easily buy from IBM.
The Merc is unlikely to return to RISC-based servers because Intel boxes have the investment edge, says Chief Technology Officer Charlie Troxel. "There are huge leaps in performance ability on that side of the space," he says. "The world has gone to commodity hardware, and we might as well be there too."
Support Headaches
A key issue with the Merc's use of Linux is support. With Sun, Panfil says, the Merc deals with a mature and responsive support organization that will immediately fly out a kernel expert if needed. But he says he thinks the Merc's Linux vendor, Red Hat, needs to improve its support. Currently, he says, Red Hat emphasizes purchasing more products as a way to fix problems. "When there are issues, they need to step up better," Panfil says.
Michael Tiemann, Red Hat's vice president of open-source affairs, says he understands Panfil's concerns; he acknowledges that his company is still learning and says it is making changes.
Tiemann says that Red Hat's goal is to sell products upfront and that the important thing is that when the Merc had problems, they were solved. "Ultimately, Red Hat was able to dig into its technical knowledge and expertise . . . and help that customer get to the place that they wanted," he says.
Panfil says the Merc's hands aren't tied by any vendor, and he uses competition to get the terms and support he wants. In particular, he points to Novell Inc.'s acquisition of SUSE Linux, which took place after the Merc decided to use Red Hat Linux. The Merc is evaluating SUSE Linux in its labs. "Our threat to a vendor is we can fire you at any time," he says.
The Merc's push also involves building data centers. Last year, it opened a remote facility with sophisticated cooling facilities capable of handling very dense blade servers, if a decision is made to deploy them. The Merc is also using code control and configuration management software from BladeLogic Inc. in Waltham, Mass. The vendor provides the tools needed to create audit trails for compliance with federal laws, says Panfil.
Whether electronic trading replaces noisy traders, or "open outcry," as it's called, remains to be seen, but the infrastructure changes will ensure that the Merc is ready. "If we weren't reliable and weren't as fast as our competitors, we wouldn't see the explosive growth that we have seen in the transition from an open-outcry environment to an electronic trading environment," says Krause. "Our future is electronic trading."
Migration Milestones:
The Merc's Transition to Intel/Linux Servers
First quarter, 2002
The CME begins considering Intel/Linux x86 machines as Solaris/Sparc server replacements.
Fourth quarter, 2002
Testing begins on HP servers and Red Hat Linux.
Third/fourth quarter, 2003
Implementation begins. Trading time is about 500 milliseconds.
July 2004
Conversion of systems to Intel/Linux servers is 30 percent completed. Faster servers shave 100 milliseconds off trading times.
December 2004
About 45 percent of all Solaris/Sparc servers are scheduled to be replaced with Intel/Linux servers. The Merc hasn't established a final conversion goal but says it isn't aiming for 100 percent conversion. Some applications will continue to run on Solaris/Sparc systems.
computerworld.com.au/index.php/id;101366230;fp;16;fpid;0
--- copy & paste ---
Case Study: Exchange makes Linux call
Patrick Thibodeau
05/01/2005 07:30:04
The Chicago Mercantile Exchange credits its migration to commodity Intel-based servers and Linux with cutting costs and shaving 100 milliseconds off the time required to complete a trade.
In a quiet corner office of Chicago Mercantile Exchange, high above the steady roar of shouting traders on the exchange floor, Joseph Panfil, director of distributed computing, is focused on the smallest increments of time.
There is a direct relationship between system performance and trading volume at the CME, also known as the Merc. By reducing the amount of time it takes to process a trade, trading volume can increase. The more trades there are, the more trading fees the exchange collects.
Time is the bottom-line metric of Panfil's world. "Everything in the trading world is a matter of split seconds," he says.
Panfil is part of a team implementing system changes that have so far reduced round-trip trading times from about 1,800 milliseconds in 1998 to 350 milliseconds today. Five trades can now be conducted in the time it used to take to complete one.
To cut times further and reduce IT costs, the CME is pursuing a major IT overhaul that involves a gradual adoption of the Linux operating system on Intel-based servers while moving off Sun Microsystems' Solaris and Sparc hardware. The conversion began about a year ago and is now 30 percent complete.
Complex Architecture
The CME's architecture consists of three major technologies: Unix and Linux systems handle the input paths for orders and output paths for quotes. Hewlett-Packard NonStop servers send out order confirmations and quotes. And IBM mainframes are responsible for "clearing," or processing, of all trades.
Traders still work shoulder to shoulder on the CME's floor, but since June, the majority of trades have been handled by the Merc's electronic trading system, Globex. The Merc began electronic trading in the early 1990s, using a third-party system to support after-hours trading. In 1998, it introduced its own system.
Customers want reliability and speed in electronic trading environments, says James Krause, the Merc's CIO. Balancing those two priorities means that the exchange, while interested in performance-improving technologies, isn't likely to be the first adopter of a new system. "Whatever we do, we have to make it work right and make sure it's fast," Krause says.
Trading times have been improved by making applications more responsive, upgrading server CPUs and undertaking architectural changes to reduce the number of hops in the path of a trade. For instance, the process previously used to include a disk write, but that has been moved out of the direct path of a trade.
The decision to use Linux and move to fast Intel chips also led to a reduction in trading time. The Merc credits its Linux deployment with cutting the time it takes to process a trade by about 100 milliseconds to the current 350 milliseconds. The goal is to reduce that to 100 milliseconds. Exchange officials say they believe that using faster Intel chips will bring them halfway toward that goal and that application optimization on the NonStop servers will take care of the rest.
But the Merc's interest in Linux was also sparked by a desire to save money. Leveraging competition among vendors of Intel-based commodity hardware is an important element of this strategy. "One of the things that we were trying to do with Linux is to be totally agnostic and not get tied into a vendor," says Panfil.
Even before the Merc deployed any Linux systems, it had already reduced some costs. "We started using that cost goal against Sun to get them to lower their price and be more competitive," Panfil says. In one case, Sun reduced the price for an US$18,000 two-way UltraSparc server to less than $10,000. That reduction came "just by having that competitive threat," says Panfil. However, a comparable two-way Intel-based server cost just $3,000, he notes.
Preparing for Linux
The Merc had to address several issues before it could deploy Linux. Third-party software had to be approved to run on the operating system and tested for speed and stability. And the Merc's technical staff also needed training.
There are 19 people in its Unix area, and some were already well versed in Linux. "They were doing this stuff in the basement of their homes. It was good that we had some people on the leading edge," says Panfil. When some Solaris administrators resisted the change, Panfil told them Linux knowledge would broaden their skills. "Why not get that other skill?" he told them.
Testing also showed that the Merc could move some applications off a four-CPU Sparc-based server, replace it with a two-way Intel-based commodity server and not lose any performance. Moreover, the leaner Linux code helped speed up performance on some applications.
When the Merc began considering its Linux deployment more than a year ago, Sun knew it wasn't in a strong position to compete. "We didn't have a good answer for them," acknowledges Glenn Weinberg, vice president of the operating platforms group at Sun.
At the time, Sun didn't have its x86 product line or Solaris running on x86 in "any serious way," Weinberg says. But today, he notes, "those things have both changed." The vendor has made a "tremendous investment" in getting Solaris to run on x86-based machines, and Solaris is outperforming Linux on some financial services applications, Weinberg claims. Sun's x86 line includes Advanced Micro Devices' Opteron processors and Intel's Xeon. "We think we're in an increasingly good position there," he says.
CME's goal is to convert 45 percent of its Solaris systems to Linux by year's end. Originally, the exchange planned to migrate all of its servers off Solaris, but applications on many of the servers are written specifically for Solaris, and it now has no plans to migrate those.
Dan Kusnetzky, an analyst at IDC, says the Solaris kernel may be better tested, more reliable and more scalable than Linux. But he sees Linux, and the freedom it represents, as having powerful appeal to businesses. Whether Sun's late response can counter the threat remains to be seen. "It's going to be difficult to come back and recover that territory," he says.
Still, the Merc isn't closing the door on Solaris. Panfil will continue to evaluate Sun's products, and he says the vendor's planned year-end rollout of Solaris 10, which will run on Opteron processors, could be a viable alternative to Linux. Panfil isn't worried about supporting both operating systems. "We have proven we can support Solaris and Linux in parallel," he says, citing the expertise available on his staff.
The move to commodity-based systems has also made the Merc more aggressive in seeking the best deals. HP is supplying the x86 servers the exchange uses, but Panfil says it could just as easily buy from IBM.
The Merc is unlikely to return to RISC-based servers because Intel boxes have the investment edge, says Chief Technology Officer Charlie Troxel. "There are huge leaps in performance ability on that side of the space," he says. "The world has gone to commodity hardware, and we might as well be there too."
Support Headaches
A key issue with the Merc's use of Linux is support. With Sun, Panfil says, the Merc deals with a mature and responsive support organization that will immediately fly out a kernel expert if needed. But he says he thinks the Merc's Linux vendor, Red Hat, needs to improve its support. Currently, he says, Red Hat emphasizes purchasing more products as a way to fix problems. "When there are issues, they need to step up better," Panfil says.
Michael Tiemann, Red Hat's vice president of open-source affairs, says he understands Panfil's concerns; he acknowledges that his company is still learning and says it is making changes.
Tiemann says that Red Hat's goal is to sell products upfront and that the important thing is that when the Merc had problems, they were solved. "Ultimately, Red Hat was able to dig into its technical knowledge and expertise . . . and help that customer get to the place that they wanted," he says.
Panfil says the Merc's hands aren't tied by any vendor, and he uses competition to get the terms and support he wants. In particular, he points to Novell Inc.'s acquisition of SUSE Linux, which took place after the Merc decided to use Red Hat Linux. The Merc is evaluating SUSE Linux in its labs. "Our threat to a vendor is we can fire you at any time," he says.
The Merc's push also involves building data centers. Last year, it opened a remote facility with sophisticated cooling facilities capable of handling very dense blade servers, if a decision is made to deploy them. The Merc is also using code control and configuration management software from BladeLogic Inc. in Waltham, Mass. The vendor provides the tools needed to create audit trails for compliance with federal laws, says Panfil.
Whether electronic trading replaces noisy traders, or "open outcry," as it's called, remains to be seen, but the infrastructure changes will ensure that the Merc is ready. "If we weren't reliable and weren't as fast as our competitors, we wouldn't see the explosive growth that we have seen in the transition from an open-outcry environment to an electronic trading environment," says Krause. "Our future is electronic trading."
Migration Milestones:
The Merc's Transition to Intel/Linux Servers
First quarter, 2002
The CME begins considering Intel/Linux x86 machines as Solaris/Sparc server replacements.
Fourth quarter, 2002
Testing begins on HP servers and Red Hat Linux.
Third/fourth quarter, 2003
Implementation begins. Trading time is about 500 milliseconds.
July 2004
Conversion of systems to Intel/Linux servers is 30 percent completed. Faster servers shave 100 milliseconds off trading times.
December 2004
About 45 percent of all Solaris/Sparc servers are scheduled to be replaced with Intel/Linux servers. The Merc hasn't established a final conversion goal but says it isn't aiming for 100 percent conversion. Some applications will continue to run on Solaris/Sparc systems.
