Unidata uses Jetstream to make geoscience data available to science community

For three years, the services and software of the Unidata Program Center at the University Corporation for Atmospheric Research (UCAR) have run on Jetstream virtual machines. With these Jetstream resources, Unidata provides a variety of geoscience data to university faculty, researchers, and students. Alongside those short-term archive and near real-time data holdings, Unidata also provides visualization and analysis tools. The tools and data are accessible through the Unidata Science Gateway, which is also hosted on Jetstream.

Julien Chastang, a Unidata software engineer, says the Unidata Science Gateway brings together a number of data services and pre-configured software tools to achieve scientific and instructional objectives. “Science students and professionals spend too much time distracted by software that is difficult to access, install, and use. Cloud computing accelerates scientific discoveries and collaborations by reducing research and data friction,” said Chastang. By streamlining access to data and software, “We aim to improve ‘time to science’ for atmospheric science researchers and students with the NSF-funded XSEDE Jetstream cloud,” said Chastang.

As part of the Unidata Science Gateway, Unidata has deployed a JupyterHub server pre-configured for atmospheric science analysis and visualization. Jupyter and JupyterHub technology allow users to run interactive code in a web browser. This technology has gained popularity for scientific research and instruction. “A JupyterHub server compliments and enhances other science gateway offerings such as collections of data which may be large or unwieldy in size,” said Chastang. “Users can develop and run their analysis and visualization codes proximate to these data holdings which may be difficult and expensive to download.” The Unidata JupyterHub was developed in partnership with XSEDE Extended Collaborative Support Services (ECSS) program. This collaboration reached a milestone when it won Best Poster at the PEARC18 conference for a poster entitled “Scaling JupyterHub Using Kubernetes on Jetstream Cloud: Platform as a Service for Research and Educational Initiatives.”

The Unidata JupyterHub can access co-located data services such as the Unidata THREDDS data server, also running on Unidata’s Jetstream allocation. This data server delivers forecast models, as well as satellite and radar data to proximate and remote clients such as the Unidata Integrated Data Viewer (IDV). In addition to THREDDS, Unidata also runs on an  Advanced Weather Interactive Processing System (AWIPS) data server for the benefit of AWIPS users in the Unidata university community. And as part of the NOAA Big Data Project, Unidata maintains a THREDDS data server on the Jetstream cloud serving Nexrad data from Amazon S3. 

In alignment with open access research objectives, Unidata’s “Science as a Service” model aims to provide a straightforward avenue to data services and software tools across a wide range of geoscientific disciplines. These goals strongly overlap with Jetstream’s mission led by the Indiana University Pervasive Technology Institute (PTI) and funded by the National Science Foundation. Like the tools and services provided by the Unidata Program Center, Jetstream’s resources are accessible to researchers and educators free of charge. “Working with the Indiana University Jetstream team has brought us to where we are today,” said Chastang. “It would have been difficult or impossible to achieve these goals if the NSF Jetstream cloud did not exist.”