System Overview
Floodwater is a forecast model controller that has been developed by The Water Institute for use in coastal and riverine flooding applications. It is designed to be robust, flexible, and extensible so that it can rapidly adapt to changing needs of clients, novel model applications, and an ever-changing landscape of available data. Floodwater has been tested and run on a variety of high performance computing systems, including the Pittsburgh Supercomputing Center’s Bridges-2 system, the RENCI Hatteras system, the University of Georgia’s Sapelo2 system, and on virtual infrastructure within the Amazon Web Services (AWS) cloud. The system is architecture agnostic and can be run on any system where the required dependencies are available, including x86 and ARM.
The Floodwater system is designed to be modular and extensible. Floodwater can control multiple types of models and has a variety of external services that can be used for data visualization, storage, and archival.
The basic schematic of the forecast system components is shown below:
Specifically, within the Floodwater system, a number of scripts and components are linked together using the ecFlow controller driven by the Floodwater system. The schematic below shows the different services that are linked together to form the Floodwater system.
External Components
The Floodwater system is used to generate the model simulations in high performance computing environments, however, preprocessing of the data and postprocessing of the data for visualization of the model results takes place in external services.
Upstream Data Providers
External to the system, meteorological data is provided from an external source. The Floodwater system uses MetGet as its external data provider. Using a single source of meteorological data allows for minimal, if any, changes within the Floodwater system when changing meteorological data sources and allows the system to remain flexible and extensible. The single source of meteorological data for the Floodwater system is MetGet.
The lateral inflow provider are upstream services that provide boundary condition information which are not simulated within any component of the Floodwater system. These can be things like upstream rivers and discharges. Presently, Floodwater can retrieve lateral inflow information from the United States Geological Survey (USGS) and the River Forecast Center (RFC). We intent to extend the lateral inflow provider to include other sources in the future.
Downstream Processing Services
Presently, Floodwater has been configured to work with the Renaissance Computing Institute (RENCI) APSViz system and The Water Institute’s FloodID system as its downstream processing services. These services allow for users to visualize the model results in a web browser for digest by stakeholders, decision makers, and the general public if desired.
Model Controller
From the diagram above, Floodwater acts as the model controller. Floodwater is responsible for monitoring available data, set up and execution up each simulation, monitoring progress, and publishing the model results to downstream processing services. Floodwater utilizes the ecFlow framework developed by the European Centre for Medium-Range Weather Forecasts (ECMWF). The ecFlow framework allows for the Floodwater system to utilize a robust backbone for monitoring and scheduling jobs as well as tracking the dependency tree. Th ecFlow system is utilized not only by ECMWF, but also by the the National Oceanic and Atmospheric Administration (NOAA) National Centers for Environmental Prediction (NCEP) for all production workflows (ref) as part of their implementation standards. By starting from a robust backbone that has managed some of the most critical workflows in this field, Floodwater is able to focus on the specifics of implementing the models which it implements.
Floodwater is mostly written as a Python wrapper around the ecFlow framework. The framework is configured such that the system is generally model agnostic and it is relatively straightforward to implement new types of models within the system. Presently, Floodwater has the capability to run:
Development is currently underway to include the SFINCS model from Deltares.
Forecast Cycles
Forecasting workflows assume that the model is run in a cyclical fashion. The Floodwater system is designed to support this workflow and makes use of the MetGet system to act as a throttle, telling the system when it can proceed and when it should wait for more data.
Each forecast scenario is referred to as a “suite”. Suites consist of a series of cycles that are run continually and dependent upon the type of meteorological forcing that the user has requested be used. Cycles consist of a nowcast, which steps the current time forward to the next forecast cycle, and a forecast, which uses forecast meteorological data generate model predictions. A schematic of the workflow is shown below:
The nowcast portion of the forecast workflow is common to each forecast, and therefore when ensemble based forecasts are enabled, a single nowcast simulation can span multiple forecast simulations as shown below:
Within each nowcast and forecast simulation, each of the models enabled for a particular forecast suite are run and their results are published as desired.