CyVerse, CloudLab Collaboration Accelerates R&D


By Shelley Littin, CyVerse

CyVerse teams are accelerating product development and testing through a collaboration with CloudLab, an infrastructure project designed to advance scientific research on the future of cloud computing.

CloudLab technology allows researchers to do repeatable experiments on computer networks by launching virtual computing clouds and testing new software code and algorithms in these clouds without risk of interfering with other networks.

CyVerse software engineer Julian Pistorius develops new software code for Atmosphere, CyVerse’s cloud computing platform. Atmosphere provides computing resources for life scientists to repeat experiments virtually and is built upon a computing technology called OpenStack, an open source platform for creating individual computing clouds.

“We initially became interested in CloudLab because they provide a way to very quickly set up a known configuration of OpenStack,” Pistorius said. “I can set up CloudLab to use OpenStack for Atmosphere and then I can experiment on the Atmosphere code. CloudLab gives me an advantage in that I don’t need to worry about breaking the public OpenStack clouds. If I mess up a virtual cloud while testing or developing new tools, then I can just blow it away and start again. This technology helps me develop new products faster.”

Amit Juneja, also a CyVerse software engineer, uses CloudLab to solve problems that require a functioning OpenStack cluster.

“The first problem I solved was writing a script for OpenStack. With access to CloudLab, all I had to do was activate my cluster on CloudLab and link it to my script with just a few clicks of the mouse.” When not using CloudLab, it could take Juneja two days just to complete setup before even beginning to test code.

“CloudLab has provided me a platform to test OpenStack-based solutions quickly, and it has helped accelerate the learning curve required for OpenStack,” Juneja said.

The CyVerse Quality Assurance (QA) team also could make use of CloudLab’s technology. The QA team tests the robustness of new tools, apps, and wokflows by attempting to dismantle them.

Using CloudLab, the team can run more advanced performance tests of how platforms behave without worrying that their experiments will adversely impact the OpenStack resource and thus other Atmosphere users.

“As more of our users are bringing their infrastructure to us we need an easy way to integrate that with Atmosphere,” said Nirav Merchant, co-principal investigator for CyVerse. “The Atmosphere team needs access to testbeds where different combinations of cloud infrastructure can be made available to us very quickly. CloudLab represents that style of infrastructure.”

“CloudLab's mission is to support work that furthers fundamental cloud architectures, helping to advance our notion of what the cloud is, and what it can be used for,” noted Robert Ricci, principal investigator for CloudLab. “An important part of that mission is to support cyberinfrastructure projects such as CyVerse as they evolve to meet the changing needs of science.”

“CloudLab could become a crucial part of how we run tests for our infrastructure,” said Pistorius, who is developing ways to track computing resources for Atmosphere to ensure that CyVerse resources are allocated appropriately for all users.

“I’ll be creating a system that ensures three aspects of allocation: transparency, reliability, and accuracy,” he explained. “For the end user, this will provide improved insight into the amount of resources they use so they can better plan their projects. And it will be good for everybody because it will mean that the system is fair.”