DECKv1 AMIP

Back to Overview of v1 DECK simulations (low-res, water cycle)

Description

DECKv1 AMIP

Log

Mar 16, 2018 : simulation started
Apr 4, 2018 : simulation complete.
Jun 22, 2018 : started A2 simulation. Use a different code base, since Edison updated modules.
Jun 25, 2018 : rerun A2 with the code base that is BFB as A1.
Jul 5, 2018 : need to edit env_archive.xml to fix the short-term archiving problem (https://github.com/E3SM-Project/E3SM/issues/2422). Solved by (https://github.com/E3SM-Project/E3SM/pull/2459) on Jul 27, 2018
Jul 19, 2018 : started A3 simulation.
Jul 26, 2018 : cray-mpich/7.6.0 cannot be loaded after Edison maintenance on Jul 25, 2018 which prevents submitting jobs and short-term archiving.
Aug 1, 2018 : cray-mpich/7.6.0 can now be loaded by editing env_mach_specific.xml. A PR (https://github.com/E3SM-Project/E3SM/pull/2473) has been submitted to fix this issue on maint-1.0 branch.

20180316.DECKv1b_A1.ne30_oEC.edison

Edison jobs

#	Simulation period	Years	Nodes	BFBFLAG	Time stamp	Duration (hr)	Cost per year	Effective SYPD	Notes
1	1870-01-01 - 1882-01-01	12	114	true	180316-210850	33.24	3756.85	8.66
2	1882-01-01 - 1898-05-21	16	114	true	180318-062752	48.00			Died exceeding time limit
3	1898-01-01 - 1904-01-01	6	114	true	180320-101507	17.61	3981.42	8.17
4	1904-01-01 - 1920-01-01	16	114	true	180321-124503	46.67	3955.96	8.23
5	1920-01-01 - 1935-01-01	15	114	true	180323-113012	42.12	3807.55	8.55
6	1935-01-01 - 1950-01-01	15	114	true	180325-054149	42.23	3817.20	8.53
7	1950-01-01 - 1965-01-01	15	114	true	180327-000527	42.37	3830.37	8.50
8	1965-01-01 - 1980-01-01	15	113	true	180328-183404	41.83	3781.25	8.61
9	1980-01-01 -1993-12-15	13	113	true	180330-122807				Died with error: ADIOI_CRAY_READCONTIG(266): filename='/project/projectdirs/acme/inputdata/atm/cam/volc/CMIP_DOE-ACME_radiation_1850-2014_v3_c20171205.nc' error='Input/output error' errno=5 PE=00084 R_rec=23896 off=0370151580 len=0001042084 See MPICH_MPIIO_ABORT_ON_RW_ERROR. pio_support::pio_die:: myrank= -1 : ERROR: pionfget_mod.F90: 84: 289 : Unknow error occurs in reading file
10	1993-01-01 -		113	true	180401-031116				0: ADIOI_CRAY_READCONTIG(266): filename='/project/projectdirs/acme/inputdata/atm/cam/physprops/mam4_mode1_rrtmg_c130628.nc' error='Input/output error' errno=5 PE=00000 R_rec=04145 off=0000000000 len=0000262144 See MPICH_MPIIO_ABORT_ON_RW_ERROR. 0: NetCDF: Not a valid ID 0: pio_support::pio_die:: myrank= -1 : ERROR: ionf_mod.F90: 235 :
11	1993-01-01 - 2008-01-01	15	113	true	180401-134107	42.59	3850.50	8.45
12	2008-01-01 - 2015-01-01	7	113	true	180403-090936	19.58	3794.25	8.58

20180622.DECKv1b_A2.ne30_oEC.edison

Edison jobs

#	Simulation period	Years	Nodes	BFBFLAG	Time stamp	Duration (hr)	Raw core hours	Cost per year	Effective SYPD	Notes

#	Simulation period	Years	Nodes	BFBFLAG	Time stamp	Duration (hr)	Cost per year	Effective SYPD	Notes
1	1870-01-01 - 1881-03-09	11	113	true	180627-175550				Died exceeding time limit
2	1881-01-01 - 1891-01-01	10	113	true	180629-203503	27.50	3728.83	8.73
3	1891-01-01 - 1906-01-01	15	113	true	180701-001001	40.01	3616.97	9.00
4	1906-01-01 - 1921-01-01	15	113	true	180702-161447	42.32	3826.07	8.51
5	1921-01-01 - 1936-01-01	15	113	true	180704-112405	47.78	4319.59	7.53
6	1936-01-01 - 1936-01-06		113	true	180706-111600				forrtl: error (78): process killed (SIGTERM)
7	1936-01-01 - 1937-03-26	1	113	true	180706-190934				ADIOI_CRAY_WRITECONTIG(261): filename='20180622.DECKv1b_A2.ne30_oEC.edison.cam.h3.1937-02-25-00000.nc' error='Input/output error' errn o=5 PE=00000 W_rec=29548 off=0000000004 len=0000000004 See MPICH_MPIIO_ABORT_ON_RW_ERROR. 0: size=3987.0 MB rss=1770.6 MB share=33.2 M pio_support::pio_die:: myrank= -1 : ERROR: pionfput_mod.F90: 0: 834 : Unknow error occurs in writting file The error seems due to unstable scratch file system
8	1937-01-01 -		113	true	180707-174257				NetCDF: Not a valid ID 792: pio_support::pio_die:: myrank= -1 : ERROR: ionf_mod.F90: 235 :
9	1937-01-01 - 1952-01-01	15	113	true	180708-103928	47.10	4257.81	7.64
10	1952-01-01 - 1959-06-05	7	113	true	180710-100600				Scratch disk maintenance
11	1959-01-01 - 1974-01-01	15	113	true	180712-135848	40.65	3674.61	8.86
12	1974-01-01 - 1989-01-01	15	113	true	180714-095812	40.96	3703.19	8.79
13	1989-01-01 - 2004-01-01	15	113	true	180716-030013	40.92	3699.12	8.80
14	2004-01-01 - 2015-01-01	11	113	true	180717-195924	29.21	3601.35	9.04

20180716.DECKv1b_A3.ne30_oEC.edison

Edison jobs

#	Simulation period	Years	Nodes	BFBFLAG	Time stamp	Duration (hr)	Cost per year	Effective SYPD
1	1870-01-01 - 1885-01-01	15	113	true	180719-112429	42.03	3799.10	8.57
2	1885-01-01 - 1900-01-01	15	113	true	180721-114019	39.84	3601.71	9.04
3	1900-01-01 - 1915-01-01	15	113	true	180723-033512	40.67	3676.32	8.85
4	1915-01-01 - 1930-01-01	15	113	true	180801-113647	42.89	3876.87	8.39
5	1930-01-01 - 1945-01-01	15	113	true	180803-063611	42.37	3830.19	8.50
6	1945-01-01 - 1960-01-01	15	113	true	180805-185821	40.79	3687.03	8.83
7	1960-01-01 - 1975-01-01	15	113	true	180807-115000	39.91	3608.20	9.02
8	1975-01-01 - 1990-01-01	15	113	true	180809-034854	42.08	3803.71	8.56
9	1990-01-01 - 2005-01-01	15	113	true	180810-220437	41.22	3726.02	8.73
10	2005-01-01 - 2015-01-01	10	113	true	180812-152313	27.21	3689.07	8.82

Output files on Edison scratch:

/global/cscratch1/sd/tang30/ACME_simulations/20180316.DECKv1b_A1.ne30_oEC.edison
/global/cscratch1/sd/tang30/ACME_simulations/20180622.DECKv1b_A2.ne30_oEC.edison
/global/cscratch1/sd/tang30/ACME_simulations/20180716.DECKv1b_A3.ne30_oEC.edison