<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
241 H= 241:00 +CPU= 30.669
23:24:34 STEP 242 H= 242:00 +CPU= 32.537
23:25:22 STEP 243 H= 243:00 +CPU= 48.294
23:25:55 STEP 244 H= 244:00 +CPU= 33.186
23:26:27 STEP 245 H= 245:00 +CPU= 31.701
23:27:19 STEP 246 H= 246:00 +CPU= 51.922
23:27:50 STEP 247 H= 247:00 +CPU= 30.472
23:28:22 STEP 248 H= 248:00 +CPU= 32.357
23:29:12 STEP 249 H= 249:00 +CPU= 49.898
23:29:45 STEP 250 H= 250:00 +CPU= 32.169
23:30:17 STEP 251 H= 251:00 +CPU= 32.180
23:31:10 STEP 252 H= 252:00 +CPU= 52.682
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000252
23:31:43 STEP 253 H= 253:00 +CPU= 33.035
23:32:15 STEP 254 H= 254:00 +CPU= 31.271
23:33:07 STEP 255 H= 255:00 +CPU= 51.817
23:33:38 STEP 256 H= 256:00 +CPU= 30.796
23:34:11 STEP 257 H= 257:00 +CPU= 33.039
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1982050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0964
(argv4) batchid: 951
(argv5) wuid: 12168608
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4256fbc6f66cfb208b03a1938fbc280e to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
ic_ancil_file: ic_ancil_12168608
ifsdata_file: ifsdata_12168608
climate_data_file: clim_data_12168608
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6104672c324e9967055c5958b195a98b to: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_855cf5fe80f0152c7ecb040f82df0610 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12168608.xml
-- Model is restarting --
Adjusting last_iter, 258, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
last_cpu_time: 9667
upload_file_number: 8
last_iter: 239
last_upload: 860400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934
The child process has been launched with process id: 968666
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:968666:968666] [20221222:233603:1671748563:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
23:37:16 STEP 240 H= 240:00 +CPU= 59.603
23:37:48 STEP 241 H= 241:00 +CPU= 31.419
23:38:22 STEP 242 H= 242:00 +CPU= 33.678
23:39:14 STEP 243 H= 243:00 +CPU= 51.576
23:39:48 STEP 244 H= 244:00 +CPU= 33.726
23:40:21 STEP 245 H= 245:00 +CPU= 33.159
23:41:15 STEP 246 H= 246:00 +CPU= 53.181
23:41:48 STEP 247 H= 247:00 +CPU= 33.429
23:42:21 STEP 248 H= 248:00 +CPU= 32.864
23:43:13 STEP 249 H= 249:00 +CPU= 51.240
23:43:45 STEP 250 H= 250:00 +CPU= 31.988
23:44:20 STEP 251 H= 251:00 +CPU= 35.451
23:45:18 STEP 252 H= 252:00 +CPU= 55.553
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000252
23:46:00 STEP 253 H= 253:00 +CPU= 41.730
23:46:29 STEP 254 H= 254:00 +CPU= 28.229
23:47:22 STEP 255 H= 255:00 +CPU= 52.739
23:47:53 STEP 256 H= 256:00 +CPU= 31.411
23:48:29 STEP 257 H= 257:00 +CPU= 35.545
23:49:20 STEP 258 H= 258:00 +CPU= 51.010
23:49:54 STEP 259 H= 259:00 +CPU= 34.159
23:50:25 STEP 260 H= 260:00 +CPU= 30.416
23:51:20 STEP 261 H= 261:00 +CPU= 54.891
23:51:51 STEP 262 H= 262:00 +CPU= 30.813
23:52:30 STEP 263 H= 263:00 +CPU= 36.239
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000252
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_8.zip
Uploading the intermediate file: upload_file_8.zip
23:55:10 STEP 264 H= 264:00 +CPU= 56.165
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000264
23:55:43 STEP 265 H= 265:00 +CPU= 27.942
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1982050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0964
(argv4) batchid: 951
(argv5) wuid: 12168608
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4256fbc6f66cfb208b03a1938fbc280e to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
ic_ancil_file: ic_ancil_12168608
ifsdata_file: ifsdata_12168608
climate_data_file: clim_data_12168608
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6104672c324e9967055c5958b195a98b to: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_855cf5fe80f0152c7ecb040f82df0610 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12168608.xml
-- Model is restarting --
Adjusting last_iter, 266, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
last_cpu_time: 10753
upload_file_number: 9
last_iter: 263
last_upload: 946800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934
The child process has been launched with process id: 972559
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:972559:972559] [20221223:010136:1671753696:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
01:04:01 STEP 264 H= 264:00 +CPU= 59.837
01:04:33 STEP 265 H= 265:00 +CPU= 31.319
01:05:05 STEP 266 H= 266:00 +CPU= 31.608
01:05:56 STEP 267 H= 267:00 +CPU= 50.221
01:06:28 STEP 268 H= 268:00 +CPU= 31.693
01:06:59 STEP 269 H= 269:00 +CPU= 30.973
01:07:51 STEP 270 H= 270:00 +CPU= 50.906
01:08:23 STEP 271 H= 271:00 +CPU= 31.687
01:08:57 STEP 272 H= 272:00 +CPU= 34.117
01:09:48 STEP 273 H= 273:00 +CPU= 50.177
01:10:23 STEP 274 H= 274:00 +CPU= 34.361
01:10:55 STEP 275 H= 275:00 +CPU= 31.487
01:11:53 STEP 276 H= 276:00 +CPU= 56.271
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000276
01:12:25 STEP 277 H= 277:00 +CPU= 31.406
01:12:58 STEP 278 H= 278:00 +CPU= 32.920
01:13:50 STEP 279 H= 279:00 +CPU= 51.025
01:14:22 STEP 280 H= 280:00 +CPU= 32.545
01:14:55 STEP 281 H= 281:00 +CPU= 32.522
01:15:47 STEP 282 H= 282:00 +CPU= 50.999
01:16:18 STEP 283 H= 283:00 +CPU= 31.189
01:16:51 STEP 284 H= 284:00 +CPU= 33.436
01:17:40 STEP 285 H= 285:00 +CPU= 48.228
01:18:13 STEP 286 H= 286:00 +CPU= 33.406
01:18:51 STEP 287 H= 287:00 +CPU= 32.188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_9.zip
Uploading the intermediate file: upload_file_9.zip
01:20:14 STEP 288 H= 288:00 +CPU= 56.799
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000288
01:20:46 STEP 289 H= 289:00 +CPU= 31.401
01:21:18 STEP 290 H= 290:00 +CPU= 32.543
01:22:09 STEP 291 H= 291:00 +CPU= 50.345
01:22:42 STEP 292 H= 292:00 +CPU= 32.944
01:23:13 STEP 293 H= 293:00 +CPU= 31.641
01:24:05 STEP 294 H= 294:00 +CPU= 51.695
01:24:37 STEP 295 H= 295:00 +CPU= 31.970
01:25:10 STEP 296 H= 296:00 +CPU= 32.703
01:26:02 STEP 297 H= 297:00 +CPU= 51.124
01:26:35 STEP 298 H= 298:00 +CPU= 32.927
01:27:06 STEP 299 H= 299:00 +CPU= 30.779
01:28:05 STEP 300 H= 300:00 +CPU= 57.292
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000300
01:28:36 STEP 301 H= 301:00 +CPU= 30.816
01:29:09 STEP 302 H= 302:00 +CPU= 32.302
01:29:59 STEP 303 H= 303:00 +CPU= 50.181
01:30:32 STEP 304 H= 304:00 +CPU= 33.326
01:31:04 STEP 305 H= 305:00 +CPU= 31.708
01:31:59 STEP 306 H= 306:00 +CPU= 52.116
01:32:31 STEP 307 H= 307:00 +CPU= 31.479
01:33:02 STEP 308 H= 308:00 +CPU= 31.099
01:33:53 STEP 309 H= 309:00 +CPU= 50.097
01:34:28 STEP 310 H= 310:00 +CPU= 33.838
01:34:59 STEP 311 H= 311:00 +CPU= 30.776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_10.zip
Uploading the intermediate file: upload_file_10.zip
01:36:07 STEP 312 H= 312:00 +CPU= 57.558
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000312
01:36:40 STEP 313 H= 313:00 +CPU= 31.664
01:37:12 STEP 314 H= 314:00 +CPU= 32.380
01:38:01 STEP 315 H= 315:00 +CPU= 48.812
01:38:35 STEP 316 H= 316:00 +CPU= 33.430
01:39:07 STEP 317 H= 317:00 +CPU= 31.753
01:39:59 STEP 318 H= 318:00 +CPU= 52.155
01:40:33 STEP 319 H= 319:00 +CPU= 33.215
01:41:06 STEP 320 H= 320:00 +CPU= 32.657
01:41:57 STEP 321 H= 321:00 +CPU= 51.162
01:42:31 STEP 322 H= 322:00 +CPU= 33.650
01:43:01 STEP 323 H= 323:00 +CPU= 29.982
01:44:06 STEP 324 H= 324:00 +CPU= 57.848
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000324
01:44:37 STEP 325 H= 325:00 +CPU= 31.018
01:45:10 STEP 326 H= 326:00 +CPU= 33.224
01:46:01 STEP 327 H= 327:00 +CPU= 51.005
01:46:34 STEP 328 H= 328:00 +CPU= 32.583
01:47:06 STEP 329 H= 329:00 +CPU= 31.456
01:47:59 STEP 330 H= 330:00 +CPU= 52.784
01:48:30 STEP 331 H= 331:00 +CPU= 31.345
01:49:03 STEP 332 H= 332:00 +CPU= 32.631
01:49:53 STEP 333 H= 333:00 +CPU= 49.719
01:50:25 STEP 334 H= 334:00 +CPU= 32.490
01:51:02 STEP 335 H= 335:00 +CPU= 33.328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000324
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000324
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_11.zip
Uploading the intermediate file: upload_file_11.zip
01:52:04 STEP 336 H= 336:00 +CPU= 54.802
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000336
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000336
01:52:36 STEP 337 H= 337:00 +CPU= 31.717
01:53:08 STEP 338 H= 338:00 +CPU= 31.608
01:53:58 STEP 339 H= 339:00 +CPU= 49.695
01:54:30 STEP 340 H= 340:00 +CPU= 32.349
01:55:02 STEP 341 H= 341:00 +CPU= 31.975
01:55:53 STEP 342 H= 342:00 +CPU= 50.676
01:56:26 STEP 343 H= 343:00 +CPU= 32.455
01:56:59 STEP 344 H= 344:00 +CPU= 32.512
01:57:49 STEP 345 H= 345:00 +CPU= 49.696
01:58:23 STEP 346 H= 346:00 +CPU= 33.813
01:58:54 STEP 347 H= 347:00 +CPU= 30.821
01:59:54 STEP 348 H= 348:00 +CPU= 58.191
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000348
02:00:29 STEP 349 H= 349:00 +CPU= 34.158
02:01:00 STEP 350 H= 350:00 +CPU= 31.704
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1982050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0964
(argv4) batchid: 951
(argv5) wuid: 12168608
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4256fbc6f66cfb208b03a1938fbc280e to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
ic_ancil_file: ic_ancil_12168608
ifsdata_file: ifsdata_12168608
climate_data_file: clim_data_12168608
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6104672c324e9967055c5958b195a98b to: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_855cf5fe80f0152c7ecb040f82df0610 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12168608.xml
-- Model is restarting --
Adjusting last_iter, 351, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
last_cpu_time: 14174
upload_file_number: 12
last_iter: 335
last_upload: 1206000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934
The child process has been launched with process id: 975998
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:975998:975998] [20221223:020249:1671757369:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
02:04:52 STEP 336 H= 336:00 +CPU= 57.082
02:05:23 STEP 337 H= 337:00 +CPU= 31.073
02:05:57 STEP 338 H= 338:00 +CPU= 33.432
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
02:07:01 STEP 339 H= 339:00 +CPU= 51.952
02:07:34 STEP 340 H= 340:00 +CPU= 32.618
02:08:05 STEP 341 H= 341:00 +CPU= 30.923
02:08:58 STEP 342 H= 342:00 +CPU= 52.982
02:09:31 STEP 343 H= 343:00 +CPU= 32.111
02:10:02 STEP 344 H= 344:00 +CPU= 31.082
02:10:52 STEP 345 H= 345:00 +CPU= 50.141
02:11:25 STEP 346 H= 346:00 +CPU= 32.521
02:11:55 STEP 347 H= 347:00 +CPU= 30.066
02:12:52 STEP 348 H= 348:00 +CPU= 51.226
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000348
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000348
02:13:19 STEP 349 H= 349:00 +CPU= 27.346
02:13:53 STEP 350 H= 350:00 +CPU= 33.559
02:14:42 STEP 351 H= 351:00 +CPU= 48.983
02:15:16 STEP 352 H= 352:00 +CPU= 33.809
02:15:48 STEP 353 H= 353:00 +CPU= 32.178
02:16:41 STEP 354 H= 354:00 +CPU= 52.230
02:17:13 STEP 355 H= 355:00 +CPU= 31.882
02:17:47 STEP 356 H= 356:00 +CPU= 33.743
02:18:37 STEP 357 H= 357:00 +CPU= 50.151
02:19:08 STEP 358 H= 358:00 +CPU= 30.807
02:19:41 STEP 359 H= 359:00 +CPU= 28.047
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000336
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000348
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000348
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_12.zip
Uploading the intermediate file: upload_file_12.zip
02:20:44 STEP 360 H= 360:00 +CPU= 55.107
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000360
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000360
02:21:17 STEP 361 H= 361:00 +CPU= 32.938
02:21:49 STEP 362 H= 362:00 +CPU= 31.865
02:22:41 STEP 363 H= 363:00 +CPU= 51.888
02:23:15 STEP 364 H= 364:00 +CPU= 33.061
02:23:49 STEP 365 H= 365:00 +CPU= 33.893
02:24:40 STEP 366 H= 366:00 +CPU= 50.779
02:25:12 STEP 367 H= 367:00 +CPU= 32.107
02:25:44 STEP 368 H= 368:00 +CPU= 32.067
02:26:37 STEP 369 H= 369:00 +CPU= 52.508
02:27:10 STEP 370 H= 370:00 +CPU= 33.304
02:27:43 STEP 371 H= 371:00 +CPU= 32.334
02:28:41 STEP 372 H= 372:00 +CPU= 57.680
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000372
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000372
02:29:13 STEP 373 H= 373:00 +CPU= 31.510
02:29:46 STEP 374 H= 374:00 +CPU= 33.679
02:30:38 STEP 375 H= 375:00 +CPU= 50.970
02:31:11 STEP 376 H= 376:00 +CPU= 32.819
02:31:43 STEP 377 H= 377:00 +CPU= 31.745
02:32:33 STEP 378 H= 378:00 +CPU= 50.317
02:33:05 STEP 379 H= 379:00 +CPU= 31.387
02:33:38 STEP 380 H= 380:00 +CPU= 33.028
02:34:28 STEP 381 H= 381:00 +CPU= 49.612
02:35:00 STEP 382 H= 382:00 +CPU= 31.113
02:35:39 STEP 383 H= 383:00 +CPU= 32.372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000360
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000372
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000372
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_13.zip
Uploading the intermediate file: upload_file_13.zip
02:36:44 STEP 384 H= 384:00 +CPU= 59.512
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000384
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000384
02:37:16 STEP 385 H= 385:00 +CPU= 31.044
02:37:49 STEP 386 H= 386:00 +CPU= 33.597
02:38:38 STEP 387 H= 387:00 +CPU= 48.984
02:39:12 STEP 388 H= 388:00 +CPU= 33.093
02:39:44 STEP 389 H= 389:00 +CPU= 31.816
02:40:36 STEP 390 H= 390:00 +CPU= 51.773
02:41:08 STEP 391 H= 391:00 +CPU= 31.309
02:41:39 STEP 392 H= 392:00 +CPU= 31.162
02:42:28 STEP 393 H= 393:00 +CPU= 49.456
02:43:00 STEP 394 H= 394:00 +CPU= 31.463
02:43:35 STEP 395 H= 395:00 +CPU= 34.396
02:44:37 STEP 396 H= 396:00 +CPU= 56.823
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000396
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000396
02:45:07 STEP 397 H= 397:00 +CPU= 30.221
02:45:41 STEP 398 H= 398:00 +CPU= 33.675
02:46:32 STEP 399 H= 399:00 +CPU= 51.498
02:47:04 STEP 400 H= 400:00 +CPU= 31.699
02:47:36 STEP 401 H= 401:00 +CPU= 32.113
02:48:28 STEP 402 H= 402:00 +CPU= 51.711
02:49:00 STEP 403 H= 403:00 +CPU= 31.379
02:49:33 STEP 404 H= 404:00 +CPU= 33.131
02:50:24 STEP 405 H= 405:00 +CPU= 50.655
02:50:58 STEP 406 H= 406:00 +CPU= 34.336
02:51:32 STEP 407 H= 407:00 +CPU= 31.253
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000384
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000396
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000396
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_14.zip
Uploading the intermediate file: upload_file_14.zip
02:52:37 STEP 408 H= 408:00 +CPU= 58.683
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000408
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000408
02:53:08 STEP 409 H= 409:00 +CPU= 30.622
02:53:42 STEP 410 H= 410:00 +CPU= 34.341
02:54:32 STEP 411 H= 411:00 +CPU= 49.672
02:55:04 STEP 412 H= 412:00 +CPU= 31.836
02:55:36 STEP 413 H= 413:00 +CPU= 31.165
02:56:29 STEP 414 H= 414:00 +CPU= 52.762
02:57:01 STEP 415 H= 415:00 +CPU= 31.984
02:57:34 STEP 416 H= 416:00 +CPU= 32.331
02:58:24 STEP 417 H= 417:00 +CPU= 50.766
02:58:58 STEP 418 H= 418:00 +CPU= 33.150
02:59:29 STEP 419 H= 419:00 +CPU= 30.891
03:00:28 STEP 420 H= 420:00 +CPU= 53.768
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000420
03:01:00 STEP 421 H= 421:00 +CPU= 32.004
03:01:34 STEP 422 H= 422:00 +CPU= 34.237
03:02:26 STEP 423 H= 423:00 +CPU= 51.621
03:02:59 STEP 424 H= 424:00 +CPU= 33.243
03:03:32 STEP 425 H= 425:00 +CPU= 32.923
03:04:25 STEP 426 H= 426:00 +CPU= 52.326
03:04:58 STEP 427 H= 427:00 +CPU= 32.873
03:05:32 STEP 428 H= 428:00 +CPU= 33.730
03:06:24 STEP 429 H= 429:00 +CPU= 51.803
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1982050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0964
(argv4) batchid: 951
(argv5) wuid: 12168608
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/2
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_4256fbc6f66cfb208b03a1938fbc280e to: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/2/oifs_43r3_ps_0964_1982050100_123_951_12168608.zip
ic_ancil_file: ic_ancil_12168608
ifsdata_file: ifsdata_12168608
climate_data_file: clim_data_12168608
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6104672c324e9967055c5958b195a98b to: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/2/ic_ancil_12168608.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_855cf5fe80f0152c7ecb040f82df0610 to: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/2/ifsdata/ifsdata_12168608.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/2/159l_2/clim_data_12168608.zip
Checking for progress XML file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
Opened progress file ok : /var/lib/boinc-client/slots/2/progress_file_12168608.xml
-- Model is restarting --
Adjusting last_iter, 430, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/2/progress_file_12168608.xml
last_cpu_time: 17853
upload_file_number: 15
last_iter: 407
last_upload: 1465200
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934
The child process has been launched with process id: 980440
Executing the command: /var/lib/boinc-client/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:thomas-HP-ENVY-dv7-Notebook-PC:1:1:980440:980440] [20221223:030758:1671761278:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
03:10:11 STEP 408 H= 408:00 +CPU= 51.584
03:10:44 STEP 409 H= 409:00 +CPU= 32.768
03:11:15 STEP 410 H= 410:00 +CPU= 30.677
03:12:05 STEP 411 H= 411:00 +CPU= 49.862
03:12:38 STEP 412 H= 412:00 +CPU= 32.448
03:13:09 STEP 413 H= 413:00 +CPU= 30.702
03:14:02 STEP 414 H= 414:00 +CPU= 52.971
03:14:34 STEP 415 H= 415:00 +CPU= 31.512
03:15:07 STEP 416 H= 416:00 +CPU= 32.618
03:15:56 STEP 417 H= 417:00 +CPU= 49.181
03:16:31 STEP 418 H= 418:00 +CPU= 33.732
03:17:01 STEP 419 H= 419:00 +CPU= 30.044
03:18:00 STEP 420 H= 420:00 +CPU= 53.851
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000420
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000420
03:18:31 STEP 421 H= 421:00 +CPU= 31.538
03:19:03 STEP 422 H= 422:00 +CPU= 31.419
03:19:53 STEP 423 H= 423:00 +CPU= 50.195
03:20:25 STEP 424 H= 424:00 +CPU= 32.032
03:20:56 STEP 425 H= 425:00 +CPU= 30.902
03:21:48 STEP 426 H= 426:00 +CPU= 51.264
03:22:19 STEP 427 H= 427:00 +CPU= 31.603
03:22:51 STEP 428 H= 428:00 +CPU= 32.104
03:23:41 STEP 429 H= 429:00 +CPU= 49.759
03:24:14 STEP 430 H= 430:00 +CPU= 32.933
03:24:47 STEP 431 H= 431:00 +CPU= 31.764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000408
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMGGhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMSHhq0f+000420
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12168608/ICMUAhq0f+000420
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0964_1982050100_123_951_12168608_0_r1648322934_15.zip
Uploading the intermediate file: upload_file_15.zip
03:25:49 STEP 432 H= 432:00 +CPU= 57.060
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+000432
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+000432
03:26:20 STEP 433 H= 433:00 +CPU= 31.182
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
