Modified Simulation Tool#
The Modified Simulation (msim) Tool runs V4r4 with modified input. Examining the model state of a modified simulation or comparing it with that of the original V4r4 can provide important scientific insights into the impact of model input, such as climatological versus non-climatological forcings or local versus remote forcings.
Attention
Each modified simulation will integrate over the entire V4r4 period from 1992 through 2017, which will take more than 12 hours to complete. The total size of the output files for each simulation amounts to several hundred GBs when outputting 3D temperature and salt fluxes (see below). Be mindful when running multiple modified simulations, and contact one of the Hackathon organizers if in doubt.
EMU has a set of pre-modified input files in the directory /efs_ecco/ECCO/EMU/emu_input_dir/emu_msim/ that replace one or more surface ocean and initial conditions with V4r4’s 1992-2017 mean. The subdirectory names, along with descriptions of the modifications, are listed in the table below. These subdirectory names, including the full path if needed, can be used as input parameters for the Modified Simulation Tool (see below).
Forcing and/or Initial Conditions Replaced with V4r4’s 1992-2017 Means
Directory Name |
Forcing and/or Initial Conditions Replaced with V4r4’s 1992-2017 Means |
|---|---|
mean_ALL |
All forcings and initial conditions |
mean_IC |
Initial conditions |
mean_TFLUX_oceQsw |
Shortwave heat flux |
mean_oceFWflx |
Freshwater flux |
mean_oceSflux_oceSPflx |
Salt plume flux |
mean_oceTAUX_oceTAUY |
Zonal and meridional wind stress |
mean_sIceLoadPatmPload_nopabar |
Surface pressure load |
In this tutorial, we demonstrate how to run a modified simulation that is the same as V4r4, except that the zonal and meridional wind stress is replaced with V4r4’s 1992-2017 mean (in mean_oceTAUX_oceTAUY). We then show how to use EMU’s built-in plotting tool to plot some of the results.
Run Modified Simulation Tool#
We start EMU (see, e.g., Adjoint Tool) and choose the Modified Simulation Tool by entering 7.
7
choice is 7) Modified Simulation Tool (msim)
See /efs_ecco/ECCO/EMU/emu_userinterface_dir/README_msim
************************************
EMU Modified Simulation Tool (singularity)
************************************
**** Steps 1 & 2: Setup & Specification
NB: V4r4's forcing is in
/efs_ecco/ECCO/EMU/emu_input_dir/forcing/other/flux-forced/forcing
Enter directory name with user replacement files ... ?
We then the directory name with the full path of the time-mean wind stress /efs_ecco/ECCO/EMU/emu_input_dir/emu_msim/mean_oceTAUX_oceTAUY.
Replacement files will be read from
/efs_ecco/ECCO/EMU/emu_input_dir/emu_msim/mean_oceTAUX_oceTAUY
Output directory will be /efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY
1) Set up files for MITgcm
Output budget (fluxes) ... (YES/NO)?
We are now prompted to choose whether we want to output 3D temperature and salt fluxes, which can be used as input files for the Budget Tool. We enter YES to output these terms. Choosing NO will significantly reduce the total size of the output files.
... outputting budget
diags exists
diags exists
diags exists
Replacement file ... diags
Replacement file ... msim.dir_out
Total # of files to be replaced ... 2
Proceed to replace and run the model ... (Y/N)?
We choose Y to confirm.
The tool then asks for a short description about the replacement files:
Optionally, enter short description about the replacement files … ?
(Will be copied in output file msim.info for reference.)
(Skip if not needed.)
After we enter a description Zonal and meridional wind stress is replaced with V4r4 1992-2017 mean., we complete the configuration and the run is submitted as a batch job.
Replacing ... diags
rm: cannot remove '/efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY/diags': Is a directory
cp: -r not specified; omitting directory '/efs_ecco/ECCO/EMU/emu_input_dir/emu_msim/mean_oceTAUX_oceTAUY/diags'
Replacing ... msim.dir_out
Replacing ... msim.info
Total # of files replaced ... 3
3) Run MITgcm
... Running batch job pbs_msim.sh
to compute the model's response to modified input.
Estimated wallclock time:
#SBATCH --ntasks-per-node=36
********************************************
Results will be in emu_msim_tau
********************************************
Progress of the computation can be monitored by
ls -l emu_msim_tau/diags/*2d*day*data | wc -l
which counts the number of days the model has integrated.
(As standard output, the model saves daily mean files of
sea level and ocean bottom pressure, unless changed in
file data.diagnostics.)
Submitted batch job 829
EMU interactive execution complete. Sat Oct 5 01:20:01 UTC 2024
The run (and also directory) name is emu_msim_mean_oceTAUX_oceTAUY, with the run itself taking more than 12 hours to complete. Also, note that the directory structure inside the run directory is different from that of other tools, which have output and temp subdirectories (see EMU Introduction). This run directory does not have these two subdirectories but is more similar to an MITgcm run directory, such as V4r4’s run directory, which contains both input and output files and directories.
Visualize Adjoint Tool Results#
We now show how to use EMU’s built-in plotting tool to plot a map of monthly SSH at month 12. Plotting the results of Modified Simulation Tool is the same as plotting Sampling Tool results, except that we now enter or specify the run directory /efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY. Again, we will show two methods of using the visualization tool.
Tip
A Jupyter Notebook, msim_viz (after downloading, rename it to msim_viz.ipynb), is provided for users’ convenience. It reproduces the steps and figures described in this visualization tutorial. Users can also add more sophisticated analysis on top of this notebook.
Same as in Sampling Tool, we first log into OSS and start a Jupyter Notebook session.
Method 1: Menu-driven Input#
First create a new Jupyter Notebook, import the runpy module, and then call the Python plotting script located at /efs_ecco/ECCO/EMU/emu_userinterface_dir/python/emu_plot.py. The plotting script asks for the directory name of the EMU run (in this case, /efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY), and then it generates the plot.
import runpy
runpy.run_path('/efs_ecco/ECCO/EMU/emu_userinterface_dir/python/emu_plot.py');
Found file: /efs_ecco/ECCO/EMU/emu_userinterface_dir/emu_env.singularity
EMU Input Files directory: /efs_ecco/ECCO/EMU/emu_input_dir
Enter directory of EMU run to examine; e.g., emu_samp_m_2_45_585_1 ... ? emu_msim_mean_oceTAUX_oceTAUY
Reading /efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY
Reading Modified Simulation Tool output ...
Checking EMU standard model state output ...
Detected
391 files of state_2d_set1_day.*.data
12 files of state_2d_set1_mon.*.data
12 files of state_3d_set1_mon.*.data
Choose variable to plot ...
1) SSH
2) OBP
3) THETA
4) SALT
5) U
6) V
Select monthly or daily mean ... (m/d)?
(NOTE: daily mean available for SSH and OBP only.):
We want to plot the monthly mean SSH, so we enter m for the monthly mean fields and 1 for SSH.
==> Reading and plotting monthly means ...
Enter variable # to plot ... (1-6)? 1
-------------------
Plotting ... SSH
==> Reading and plotting 2d monthly means ...
Plotting ... SSH
Enter file # to read ... (1-12)?
Since this run has only integrated for 12 model months, we have just 12 monthly mean files for SSH. We choose to plot month 12 by entering 12. The map of the monthly mean SSH for month 12 is shown below:
We enter 0 to end plotting SSH and N when asked if we want to plot another field:
Enter file # to read ... (1-12)? 0
*********************************************
Returning variable
fld2d: last plotted 2d field
Plot another file ... (Y/N)? N
The plotting tool exits after displaying the following message that users can use the Sampling Tool and the Budget Tool to analyze the output of this modified simulation.
*********************************************
EMU's standard model state output can be sampled using EMU's
Sampling Tool, specifying the diag subdirectory of this run
in response to the Tool's prompt;
/efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY/diags
*********************************************
Checking Budget output ...
Total number of subdirectories: 27
1) ADVr_SLT_mon_mean has 13 files
2) ADVr_TH_mon_mean has 13 files
3) ADVx_SLT_mon_mean has 13 files
4) ADVx_TH_mon_mean has 13 files
5) ADVy_SLT_mon_mean has 13 files
6) ADVy_TH_mon_mean has 13 files
7) DFrE_SLT_mon_mean has 13 files
8) DFrE_TH_mon_mean has 13 files
9) DFrI_SLT_mon_mean has 13 files
10) DFrI_TH_mon_mean has 13 files
11) DFxE_SLT_mon_mean has 13 files
12) DFxE_TH_mon_mean has 13 files
13) DFyE_SLT_mon_mean has 13 files
14) DFyE_TH_mon_mean has 13 files
15) ETAN_mon_inst has 13 files
16) ETAN_mon_mean has 13 files
17) GGL90TKE_mon_mean has 13 files
18) SALT_mon_inst has 13 files
19) SFLUX_mon_mean has 13 files
20) TFLUX_mon_mean has 13 files
21) THETA_mon_inst has 13 files
22) UVELMASS_mon_mean has 13 files
23) VVELMASS_mon_mean has 13 files
24) WVELMASS_mon_mean has 13 files
25) oceFWflx_mon_mean has 13 files
26) oceQsw_mon_mean has 13 files
27) oceSPtnd_mon_mean has 13 files
*********************************************
Budget output of this run can be analyzed using
EMU's Budget Tool, specifying the diag subdirectory of this run
in response to the Tool's prompt;
/efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY/diags
Method 2: Argument-based Input#
The detailed steps for Method 2 can be found in msim_viz.ipynb (see Tip above).
Load modules
import sys
sys.path.append('/efs_ecco/ECCO/EMU/emu_userinterface_dir/')
import emu_plot_arg_py as ept
import lib_python
Invoke Method 2
# Modified Simulation Tool
globals_dict = ept.emu_plot(run_name="/efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY",
frequency='m',
pfile_beg=0, pfile_end=1000000,
pvar=1);
Found file: /efs_ecco/ECCO/EMU/emu_userinterface_dir/emu_env.singularity
EMU Input Files directory: /efs/owang/ECCO/EMU_test/emu_input_dir
Specified directory of EMU run to examine: /efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY
Reading /efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY
Reading Modified Simulation Tool output ...
Checking EMU standard model state output ...
Detected
9392 files of state_2d_set1_day.*.data
308 files of state_2d_set1_mon.*.data
308 files of state_3d_set1_mon.*.data
Specified start and end files #: 0 and 1000000
Choose variable to plot ...
1) SSH
2) OBP
3) THETA
4) SALT
5) U
6) V
Specified frequency 'm' for monthly or 'd' for daily: m
==> Reading and plotting monthly means ...
Specified variable # to extract: 1
-------------------
Plotting ... SSH
==> Reading and extracting 2d monthly means ...
Extrating ... SSH
*********************************************
Returning variable
fld2d: last plotted 2d field
*********************************************
EMU's standard model state output can be sampled using EMU's
Sampling Tool, specifying the diag subdirectory of this run
in response to the Tool's prompt;
/efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY/diags
*********************************************
Checking Budget output ...
Total number of subdirectories: 27
1) ADVr_SLT_mon_mean has 308 files
2) ADVr_TH_mon_mean has 308 files
3) ADVx_SLT_mon_mean has 308 files
4) ADVx_TH_mon_mean has 308 files
5) ADVy_SLT_mon_mean has 308 files
6) ADVy_TH_mon_mean has 308 files
7) DFrE_SLT_mon_mean has 308 files
8) DFrE_TH_mon_mean has 308 files
9) DFrI_SLT_mon_mean has 308 files
10) DFrI_TH_mon_mean has 308 files
11) DFxE_SLT_mon_mean has 308 files
12) DFxE_TH_mon_mean has 308 files
13) DFyE_SLT_mon_mean has 308 files
14) DFyE_TH_mon_mean has 308 files
15) ETAN_mon_inst has 308 files
16) ETAN_mon_mean has 308 files
17) GGL90TKE_mon_mean has 308 files
18) SALT_mon_inst has 308 files
19) SFLUX_mon_mean has 308 files
20) TFLUX_mon_mean has 308 files
21) THETA_mon_inst has 308 files
22) UVELMASS_mon_mean has 308 files
23) VVELMASS_mon_mean has 308 files
24) WVELMASS_mon_mean has 308 files
25) oceFWflx_mon_mean has 308 files
26) oceQsw_mon_mean has 308 files
27) oceSPtnd_mon_mean has 308 files
*********************************************
Budget output of this run can be analyzed using
EMU's Budget Tool, specifying the diag subdirectory of this run
in response to the Tool's prompt;
/efs_ecco/owang/ECCO/EMU/tryout/emu_msim_mean_oceTAUX_oceTAUY/diags
***********************
EMU variables read as global variables in module global_emu_var (emu); e.g., emu.nx
***********************
cs drc drf dvol3d
dxc dxg dyc dyg
fld2d hfacc hfacs hfacw
nr nx ny rac
ras raw raz rc
rf sn xc xg
yc yg
Extract Data from Return Dictionary
globals_dict.get('return_vars').keys()
Make plot
# Extract data and set title for the plot
fld_all = return_vars_dict['variable']
title = 'SSH 12 state_2d_set1_mon.0000008772.data'
# Create plot
lib_python.plt_state2d(fld_all[12], title)
Method 2 generates the same figure (not shown; see the figure embedded in msim_viz.ipynb) as that generated by Method 1.