Command Line Interface
The command line interface allows the user to manage most aspects of the system without the GUI. This can be useful in a pinch when the GUI is not available or in the case that the user wants to automate some of the tasks.
The CLI has a help screen that shows the list of sub-commands that are available. Each sub-command allows the user to access different functionalities of the CLI.
$ floodwater --help
usage: floodwater [-h] [--version] {load,run,begin,resume,suspend,delete,write,status,server,gui,ping,model,data,system-info} ...
Floodwater Simulation System
positional arguments:
{load,run,begin,resume,suspend,delete,write,status,server,gui,ping,model,data,system-info}
Sub-command help
load Load a suite to the server
run Load and run a suite
begin Begin a suite which has been loaded
resume Resume a suspended suite
suspend Suspend a suite
delete Delete a suite from the server
write Write a suite definition to a file
status Get the status of a suite
server Start/Stop the ecFlow server
gui Launch the ecFlow GUI
ping Ping the ecFlow server
model Model specific commands
data Get data from a data source
system-info Get information about the system
options:
-h, --help show this help message and exit
--version show program's version number and exit
The help screen for each sub-command can be accessed by using the --help option after the sub-command.
Sub-commands
The command line interface has a series of sub-commands which can be run. Each is described below.
load
$ floodwater load --help
usage: floodwater load [-h] [--write FILE] [--host HOST] [--port PORT] config
positional arguments:
config Name of Floodwater suite configuration file
options:
-h, --help show this help message and exit
--write FILE Write the definitions to a file
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The load sub-command loads a suite to the ecFlow server. The suite is loaded but not run. This can be useful
to stage suites within the system, check that the configuration is what you expected, or during development to ensure
that the suite is valid and the YAML configuration files were able to parse without error. If you wish to run the command
after the suite is loaded, use the begin sub-command or interact with the suite in the GUI.
run
$ floodwater run --help
usage: floodwater run [-h] [--write FILE] [--host HOST] [--port PORT] config
positional arguments:
config Name of Floodwater suite configuration file
options:
-h, --help show this help message and exit
--write FILE Write the definitions to a file
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The run sub-command loads and runs a suite on the ecFlow server. This is the same as running the load
sub-command followed by the begin sub-command.
begin
$ floodwater begin --help
usage: floodwater begin [-h] [--write FILE] [--host HOST] [--port PORT] config
positional arguments:
config Name of Floodwater suite configuration file
options:
-h, --help show this help message and exit
--write FILE Write the definitions to a file
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The begin sub-command begins a suite which has been previously loaded to the ecFlow server. This is different
than the resume command which resumes a suite which has been suspended. The begin command is used to
start a suite for the first time.
resume
$ floodwater resume --help
usage: floodwater resume [-h] [--host HOST] [--port PORT] config
positional arguments:
config Name of Floodwater suite configuration file or suite name
options:
-h, --help show this help message and exit
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The resume sub-command resumes a suite which has been previously suspended. This is different than the
begin command which starts a suite for the first time.
suspend
$ floodwater suspend --help
usage: floodwater suspend [-h] [--host HOST] [--port PORT] config
positional arguments:
config Name of Floodwater suite configuration file or suite name
options:
-h, --help show this help message and exit
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The suspend sub-command suspends a suite which is currently running. This is different than the delete
command which removes a suite from the ecFlow server. Suspend is usually a good first step before deleting a suite as
it will let the suite spin-down. Deleting a suite will attempt to immediately remove it from the server which can cause
zombie processes.
delete
$ floodwater delete --help
usage: floodwater delete [-h] [--host HOST] [--port PORT] config
positional arguments:
config Name of Floodwater suite configuration file or suite name
options:
-h, --help show this help message and exit
--force Force delete the simulation
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The delete sub-command deletes a suite from the ecFlow server. This is different than the suspend
command which suspends a suite which is currently running. Suspend is usually a good first step before deleting a suite
as it will let the suite spin-down. Deleting a suite will attempt to immediately remove it from the server which can
cause zombie processes. If the suite currently has processes running, the suite will not be deleted.
The --force option can be used to force the deletion of a suite which is currently running, but should
be used with caution.
write
$ floodwater write --help
usage: floodwater write [-h] config filename
positional arguments:
config Name of Floodwater suite configuration file or suite name
filename Name of the ecflow definitions file to write
options:
-h, --help show this help message and exit
The write sub-command writes the ecFlow definitions for a suite to a file. This can be useful for debugging
or for understanding how the suite is constructed.
status
$ floodwater status --help
usage: floodwater status [-h] [--host HOST] [--port PORT] [--verbose] [suite]
positional arguments:
suite A suite configuration file or a suite name. 'all' for all suites on the current server
options:
-h, --help show this help message and exit
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
--verbose Print the status of all tasks within a suite
The Floodwater status command allows a user to query the status of running suites from the terminal. The status command can be used to query the status of a single suite or all suites on the current server. The status command can also be used to query the status of all tasks within a suite.
An example output of the status command is shown below:
$ floodwater status
Suite: adcirc_gfs_hec_example: active
+--------+----------+--------------------------------------------------------------------------------------------------------+
| Status | Progress | Task |
+--------+----------+--------------------------------------------------------------------------------------------------------+
| active | | forecast_ensemble/forecast_base/HECRAS/preprocessing/analysis_wait/retry/hecras_forecast_wait_analysis |
+--------+----------+--------------------------------------------------------------------------------------------------------+
or, more verbosely:
$ floodwater status --verbose
Suite: adcirc_gfs_hec_example: queued
+----------+----------+-----------------------------------------------------------------------------------------------------------------------------------+
| Status | Progress | Task |
+----------+----------+-----------------------------------------------------------------------------------------------------------------------------------+
| queued | | alerts/push_alerts |
| complete | | analysis/initialization/cycle_data_wait/data |
| complete | | analysis/initialization/cycle_data_wait/retry/wait_next_cycle |
| complete | | analysis/initialization/cycle_initialization_analysis |
| complete | | analysis/ADCIRC/preprocessing/get_meteorological_forcing/wait_meteorological_forcing/data |
| complete | | analysis/ADCIRC/preprocessing/get_meteorological_forcing/wait_meteorological_forcing/retry/metget_wait_data |
| complete | | analysis/ADCIRC/preprocessing/get_meteorological_forcing/metget_get_forcing |
| complete | | analysis/ADCIRC/simulation/adcirc_approximate_simulation_wallclock |
| complete | | analysis/ADCIRC/simulation/adcirc_prep_simulation |
| complete | | analysis/ADCIRC/simulation/adcirc_simulation |
| complete | | analysis/ADCIRC/postprocessing/adcirc_generate_figures |
| complete | | analysis/ADCIRC/archive/apsviz_archive/adcirc_archive_data_apsviz |
| complete | | analysis/ADCIRC/archive/adcirc_archive_data_local |
| queued | | analysis/ADCIRC/apsviz_messaging/messaging/send_initialization_message_preprocessing/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_running_message_preprocessing/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_complete_message_preprocessing/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_initialization_message_simulation/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_wait_message_simulation/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_running_message_simulation/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_complete_message_simulation/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_initialization_message_postprocessing/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_running_message_postprocessing/rmq_send_message |
| complete | | analysis/ADCIRC/apsviz_messaging/messaging/send_complete_message_postprocessing/rmq_send_message |
| complete | | analysis/HECRAS/preprocessing/get_wind_forcing/wait_meteorological_forcing/data |
| complete | | analysis/HECRAS/preprocessing/get_wind_forcing/wait_meteorological_forcing/retry/metget_wait_data |
| complete | | analysis/HECRAS/preprocessing/get_wind_forcing/metget_get_forcing |
| complete | | analysis/HECRAS/preprocessing/get_precipitation_forcing/wait_meteorological_forcing/data |
| complete | | analysis/HECRAS/preprocessing/get_precipitation_forcing/wait_meteorological_forcing/retry/metget_wait_data |
| complete | | analysis/HECRAS/preprocessing/get_precipitation_forcing/metget_get_forcing |
| queued | | analysis/HECRAS/preprocessing/generate_offshore_boundary/hecras_extract_offshore_boundary |
| complete | | analysis/HECRAS/preprocessing/generate_upstream_boundary/hecras_get_flow_boundary |
| queued | | analysis/HECRAS/simulation/hecras_approximate_simulation_wallclock |
| queued | | analysis/HECRAS/simulation/hecras_prep_simulation |
| queued | | analysis/HECRAS/simulation/hecras_simulation |
| queued | | analysis/HECRAS/postprocessing/hecras_generate_raster |
| queued | | analysis/HECRAS/archive/apsviz_archive/hecras_archive_data_apsviz |
| queued | | analysis/HECRAS/archive/hecras_archive_data_local |
| queued | | analysis/HECRAS/apsviz_messaging/messaging/always |
| complete | | forecast_ensemble/forecast_base/initialization/analysis_wait/data |
| complete | | forecast_ensemble/forecast_base/initialization/analysis_wait/retry/wait_analysis |
| complete | | forecast_ensemble/forecast_base/initialization/cycle_initialization_forecast |
| complete | | forecast_ensemble/forecast_base/ADCIRC/preprocessing/get_meteorological_forcing/wait_meteorological_forcing/data |
| complete | | forecast_ensemble/forecast_base/ADCIRC/preprocessing/get_meteorological_forcing/wait_meteorological_forcing/retry/metget_wait_data |
| complete | | forecast_ensemble/forecast_base/ADCIRC/preprocessing/get_meteorological_forcing/metget_get_forcing |
| complete | | forecast_ensemble/forecast_base/ADCIRC/simulation/adcirc_approximate_simulation_wallclock |
| complete | | forecast_ensemble/forecast_base/ADCIRC/simulation/adcirc_prep_simulation |
| complete | | forecast_ensemble/forecast_base/ADCIRC/simulation/adcirc_simulation |
| complete | | forecast_ensemble/forecast_base/ADCIRC/postprocessing/adcirc_generate_figures |
| complete | | forecast_ensemble/forecast_base/ADCIRC/archive/apsviz_archive/adcirc_archive_data_apsviz |
| complete | | forecast_ensemble/forecast_base/ADCIRC/archive/adcirc_archive_data_local |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_initialization_message_preprocessing/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_running_message_preprocessing/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_complete_message_preprocessing/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_initialization_message_simulation/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_wait_message_simulation/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_running_message_simulation/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_complete_message_simulation/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_initialization_message_postprocessing/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_running_message_postprocessing/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_complete_message_postprocessing/rmq_send_message |
| complete | | forecast_ensemble/forecast_base/ADCIRC/apsviz_messaging/messaging/send_run_properties_message_simulation/rmq_send_run_properties |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/get_wind_forcing/wait_meteorological_forcing/data |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/get_wind_forcing/wait_meteorological_forcing/retry/metget_wait_data |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/get_wind_forcing/metget_get_forcing |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/get_precipitation_forcing/wait_meteorological_forcing/data |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/get_precipitation_forcing/wait_meteorological_forcing/retry/metget_wait_data |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/get_precipitation_forcing/metget_get_forcing |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/generate_offshore_boundary/hecras_extract_offshore_boundary |
| complete | | forecast_ensemble/forecast_base/HECRAS/preprocessing/generate_upstream_boundary/hecras_get_flow_boundary |
| queued | | forecast_ensemble/forecast_base/HECRAS/preprocessing/analysis_wait/data |
| queued | | forecast_ensemble/forecast_base/HECRAS/preprocessing/analysis_wait/retry/hecras_forecast_wait_analysis |
| queued | | forecast_ensemble/forecast_base/HECRAS/simulation/hecras_approximate_simulation_wallclock |
| queued | | forecast_ensemble/forecast_base/HECRAS/simulation/hecras_prep_simulation |
| queued | | forecast_ensemble/forecast_base/HECRAS/simulation/hecras_simulation |
| queued | | forecast_ensemble/forecast_base/HECRAS/postprocessing/hecras_generate_raster |
| queued | | forecast_ensemble/forecast_base/HECRAS/archive/apsviz_archive/hecras_archive_data_apsviz |
| queued | | forecast_ensemble/forecast_base/HECRAS/archive/hecras_archive_data_local |
| queued | | forecast_ensemble/forecast_base/HECRAS/apsviz_messaging/messaging/send_run_properties_message_simulation/rmq_send_run_properties |
+----------+----------+-----------------------------------------------------------------------------------------------------------------------------------+
Simulation jobs which are currently running will show a percent complete in the progress column and jobs which have been deemed late will appear with a late status flag.
server
$ floodwater server --help
usage: floodwater server [-h] [--port PORT] [--verbose] {start,stop}
positional arguments:
{start,stop} Start or stop an ecFlow server instance
options:
-h, --help show this help message and exit
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
--verbose Log all messages to file
The server sub-command allows a user to start or stop an ecFlow server instance. The server daemon must be
running in order for the Floodwater system to be utilized. Also, when the server is stopped it will create a checkpoint
file which will allow it to resume from the same state when it is restarted. This is useful for when the server is
rebooted or undergoes maintenance of some kind.
gui
$ floodwater gui --help
usage: floodwater gui [-h] [--verbose]
options:
-h, --help show this help message and exit
--verbose Log all messages to the console. Otherwise, only errors are logged
The GUI command will allow the user to start the ecFlow GUI. While not strictly necessary, this is a nice shorthand
way to start the GUI without having to remember the ecFlow command line options. Additionally, if there are issues
starting the GUI, the --verbose option can be used to print all messages to a log file in the user’s home
directory.
ping
$ floodwater ping --help
usage: floodwater ping [-h] [--host HOST] [--port PORT]
options:
-h, --help show this help message and exit
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The ping command will attempt to connect to the ecFlow server and return the time it took to connect. This can be useful for debugging connection issues or for testing the connection to the server.
model
The model sub-command groups a series of model-specific utility commands which are not tied to a running suite,
such as preparing an ADCIRC mesh decomposition or locating the nearest model node/cell to a set of observation stations.
It is organized as floodwater model <adcirc|hecras|sfincs> <command>.
model adcirc prep
$ floodwater model adcirc prep --help
usage: floodwater model adcirc prep [-h] --mesh MESH --config CONFIG --attributes ATTRIBUTES --ndomain NDOMAIN --adcprep ADCPREP
[--sal-file SAL_FILE] [--station-file STATION_FILE] [--verbose]
options:
-h, --help show this help message and exit
--mesh MESH Path to mesh for decomposition
--config CONFIG Path to mesh configuration json file
--attributes ATTRIBUTES
Path to nodal attributes file
--ndomain NDOMAIN Number of ADCIRC PE domains
--adcprep ADCPREP Path to adcprep executable
--sal-file SAL_FILE Name of the self attraction and loading (SAL, fort.24) file
--station-file STATION_FILE
Name of the station file
--verbose Verbose output
The model adcirc prep sub-command generates a pre-packaged ADCIRC mesh decomposition archive (see the
Model Geometry Configuration Files section for more information). The --mesh, --config, --attributes,
--ndomain, and --adcprep options are all required.
model adcirc find-station / model hecras find-station / model sfincs find-station
$ floodwater model adcirc find-station --help
usage: floodwater model adcirc find-station [-h] --mesh MESH [--station-file STATION_FILE] --output OUTPUT
[--min-depth MIN_DEPTH] [--max-distance MAX_DISTANCE]
options:
-h, --help show this help message and exit
--mesh MESH Path to mesh file
--station-file STATION_FILE
Name of the station yaml file (or None to use all stations)
--output OUTPUT Name of the output file
--min-depth MIN_DEPTH
Minimum depth for station
--max-distance MAX_DISTANCE
Maximum distance a station can be shifted in meters (default=10,000)
Each of the three model types (adcirc, hecras, sfincs) provides a find-station sub-command
which locates the nearest valid model node/cell for each station in a station file, optionally shifting station locations
that fall on dry or excessively shallow ground. The --mesh option accepts a model mesh (ADCIRC/SFINCS), while the
HEC-RAS variant instead takes a --plan option (path to a HEC-RAS plan file). --mesh/--plan and
--output are required; --station-file defaults to using all available stations if not provided.
--min-depth defaults to 0.0. The default for --max-distance is 10,000 meters for ADCIRC
and 1,000 meters for HEC-RAS and SFINCS.
data
The data sub-command groups utilities for working with data sources outside of a running suite. Presently it
provides a single station command.
data station
$ floodwater data station --help
usage: floodwater data station [-h] --start-date START_DATE --end-date END_DATE [--interval INTERVAL] [--show]
[--save-plot SAVE_PLOT] [--save-data SAVE_DATA] input_file
positional arguments:
input_file The input file
options:
-h, --help show this help message and exit
--start-date START_DATE
The start date for the data
--end-date END_DATE The end date for the data
--interval INTERVAL The output interval for the data (in minutes) [default=10]
--show Show the plot
--save-plot SAVE_PLOT
Save the plot to a file
--save-data SAVE_DATA
Save the data to a file
The data station sub-command allows a flow boundary station input file (see the Flow Boundary Schema section)
to be tested independently of a running suite. It reads the boundary/station definitions from input_file and
retrieves the aggregated flow data for the given --start-date/--end-date window, which can then be
inspected with --show, or saved to disk with --save-plot/--save-data.
system-info
$ floodwater system-info --help
usage: floodwater system-info [-h] [--pretty] [--host HOST] [--port PORT]
options:
-h, --help show this help message and exit
--pretty Pretty print the output
--host HOST ecFlow Server Host. Determined from environment variable 'ECF_HOST' if not provided
--port PORT ecFlow Server Port. Determined from environment variable 'ECF_PORT' if not provided
The system-info command will return information about the current system. This can be useful for debugging issues and is useful to developers when filing bug reports. The output of the command is shown below:
$ floodwater system-info
{
"floodwater_version": "48fcbc5",
"system": {
"platform": "Linux-5.15.0-88-generic-x86_64-with-glibc2.35",
"architecture": [
"64bit",
"ELF"
],
"machine": "x86_64",
"processor": "x86_64"
},
"python": {
"python_version": "3.10.12",
"python_build": [
"main",
"Jun 23 2023 22:40:32"
],
"python_compiler": "GCC 12.3.0",
"conda": "23.7.3",
"pip": "23.2.1"
},
"job_scheduler": {
"slurm": "22.05.2"
},
"dependencies": {
"ecflow": {
"client": "5.11.3",
"server": "5.11.3"
},
"numpy": "1.24.4",
"scipy": "1.11.2",
"pandas": "2.1.0",
"matplotlib": "3.7.2",
"netCDF4": "1.6.4",
"utide": "0.3.0",
"boto3": "1.28.44",
"pika": "1.3.1",
"paramiko": "3.3.1",
"colorama": "0.4.6",
"numba": "0.57.1",
"shapely": "2.0.1",
"yaml": "6.0.1",
"prettytable": "3.8.0",
"tqdm": "4.66.1",
"schema": "0.7.5",
"requests": "2.31.0",
"xarray": "2023.8.0",
"psycopg2": "2.9.7 (dt dec pq3 ext lo64)",
"sqlalchemy": "2.0.20",
"geopandas": "0.13.2",
"pyproj": "3.5.0",
"h5py": "3.9.0",
"geocube": "0.4.2",
"mpl_toolkits.basemap": "1.3.7"
}
}