Introduction#
ECCO Modeling Utilities (EMU) is a collection of menu-driven tools for analyzing the ECCO model itself, in addition to its state (flux-forced version of ECCO Version 4 Release 4). A detailed description of the EMU tools is available in this presentation slide deck. The tools (and their abbreviations) include the following:
Sampling (samp); Evaluates time-series of the model state (example plot).
Forward Gradient (fgrd); Computes state’s forward gradient (example plot).
Adjoint (adj); Computes state’s adjoint gradient (example plot).
Convolution (conv); Evaluates adjoint gradient decomposition (example plot).
Tracer (trc); Computes passive tracer evolution (example plot).
Budget (budg); Evaluates contributions to a quantity’s budget (example plot).
Modified Simulation (msim); Runs V4r4 with modified input (example plot).
Attribution (atrb); Evaluates state time-series by control type (example plot).
The EMU code is accessible at ECCO-GROUP/ECCO-EIS. There is a main README file for the EMU tools, as well as individual README files (README_***) for each tool. Additionally, EMU provides programs for interactively reading and plotting in different programming languages. For a detailed description, refer to the EMU User Guide.
The EMU tools are menu-driven and configured with user-specified input parameters. Once configured, the tools will either run as a normal shell script or be submitted as a batch job using SLURM on the P-Cluster. Among these tools, the Sampling and Attribution Tools are set up to run as normal shell scripts. However, please do not run these scripts on the head node, as explained in Getting Started with the P-Cluster. You must first request an interactive node using salloc instead of running directly on the head node to avoid overloading it, as it has limited resources (see table below). The rest of the EMU tools are designed to be automatically submitted as SLURM batch jobs, so there is no need to use salloc to request an interactive node.
Table: EMU Tools and Whether They Require an Interactive Node
Tool |
Request Interactive Node or Not |
|---|---|
Sampling (samp) |
Yes |
Forward Gradient (fgrd) |
No |
Adjoint (adj) |
No |
Convolution (conv) |
No |
Tracer (trc) |
No |
Budget (budg) |
No |
Modified Simulation (msim) |
No |
Attribution (atrb) |
Yes |
We first create a master working directory and change into it:
mkdir -p /efs_ecco/USERNAME/ECCO/EMU/tryout
cd /efs_ecco/USERNAME/ECCO/EMU/tryout
When running each tool, EMU will create a subdirectory in the current directory named emu_**TOOL_ABBREVIATION**_*/, where **TOOL_ABBREVIATION** is the abbreviation of each tool (as specified in the parentheses after each tool’s name in the table above). The rest of the directory name (indicated by *) depends on the input parameters (see an example in Sampling Tool). Inside emu_**TOOL_ABBREVIATION**_*/, there are two subdirectories: temp and output. The temp directory is for execution, and the output directory is the archive directory where, after successful execution, useful output files, along with the configuration information, are stored for plotting and analysis.
After generating EMU results on the P-Cluster, we will switch to OSS to visualize and analyze the EMU results using Jupyter Notebook. We use OSS for visualization and analysis because it has a better Python and Jupyter Notebook configuration than the P-Cluster.
Attention
EMU tools will be run on the P-Cluster for computation, while the visualization and analysis of results will be conducted on OSS.
EMU also has built-in plotting tools using Python and IDL (Interactive Data Language). We will provide examples on how to use the Python plotting script on OSS to visualize the results generated by EMU. Plotting the results using IDL can be done similarly (see /efs_ecco/ECCO/EMU/emu_userinterface_dir/idl/README_idl).
The following sections are organized as follows. Section Installing EMU provides instructions on how to install EMU on the P-Cluster. Readers can skip this section because EMU has already been pre-installed on the P-Cluster for ECCO Hackathon participants. The subsequent sections are organized by EMU tools, with one tool per section. In each section, we describe how to run an EMU tool as well as how to use the built-in Python plotting tool to visualize the results. The final section, Create Mask, provides examples for creating user-specified masking for the EMU tools.