<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
+CPU= 39.404
16:03:39 STEP 979 H= 244:45 +CPU= 28.376
16:04:16 STEP 980 H= 245:00 +CPU= 35.232
16:04:52 STEP 981 H= 245:15 +CPU= 33.894
16:05:30 STEP 982 H= 245:30 +CPU= 35.915
16:06:12 STEP 983 H= 245:45 +CPU= 39.645
16:07:01 STEP 984 H= 246:00 +CPU= 46.091
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+000984
16:07:37 STEP 985 H= 246:15 +CPU= 34.636
16:08:16 STEP 986 H= 246:30 +CPU= 37.137
16:08:48 STEP 987 H= 246:45 +CPU= 31.211
16:09:22 STEP 988 H= 247:00 +CPU= 33.420
16:10:03 STEP 989 H= 247:15 +CPU= 38.711
16:10:43 STEP 990 H= 247:30 +CPU= 37.135
16:11:20 STEP 991 H= 247:45 +CPU= 35.191
16:11:57 STEP 992 H= 248:00 +CPU= 36.194
16:12:32 STEP 993 H= 248:15 +CPU= 33.132
16:13:10 STEP 994 H= 248:30 +CPU= 35.993
16:13:47 STEP 995 H= 248:45 +CPU= 34.740
16:14:39 STEP 996 H= 249:00 +CPU= 50.002
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+000996
16:15:21 STEP 997 H= 249:15 +CPU= 39.027
16:16:01 STEP 998 H= 249:30 +CPU= 38.119
16:16:41 STEP 999 H= 249:45 +CPU= 38.720
16:17:20 STEP 1000 H= 250:00 +CPU= 38.186
16:18:02 STEP 1001 H= 250:15 +CPU= 39.095
16:18:45 STEP 1002 H= 250:30 +CPU= 40.676
16:19:18 STEP 1003 H= 250:45 +CPU= 32.412
16:20:03 STEP 1004 H= 251:00 +CPU= 41.584
16:20:42 STEP 1005 H= 251:15 +CPU= 36.812
16:21:25 STEP 1006 H= 251:30 +CPU= 41.019
16:21:58 STEP 1007 H= 251:45 +CPU= 31.870
16:23:00 STEP 1008 H= 252:00 +CPU= 59.593
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001008
16:23:43 STEP 1009 H= 252:15 +CPU= 41.557
16:24:27 STEP 1010 H= 252:30 +CPU= 41.574
16:25:15 STEP 1011 H= 252:45 +CPU= 45.362
16:25:55 STEP 1012 H= 253:00 +CPU= 39.206
16:26:35 STEP 1013 H= 253:15 +CPU= 39.013
16:27:17 STEP 1014 H= 253:30 +CPU= 39.682
16:28:04 STEP 1015 H= 253:45 +CPU= 45.565
16:28:46 STEP 1016 H= 254:00 +CPU= 39.538
16:29:27 STEP 1017 H= 254:15 +CPU= 38.664
16:30:08 STEP 1018 H= 254:30 +CPU= 39.151
16:30:48 STEP 1019 H= 254:45 +CPU= 37.329
16:31:37 STEP 1020 H= 255:00 +CPU= 46.137
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001020
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a007
(argv4) batchid: 949
(argv5) wuid: 12166401
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7ce91203b1b08443f4e24d55a126930 to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a007_2016092300_15_949_12166401.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a007_2016092300_15_949_12166401.zip
ic_ancil_file: ic_ancil_12166401
ifsdata_file: ifsdata_12166401
climate_data_file: clim_data_12166401
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/10/ic_ancil_12166401.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12166401.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_450dacd8598235ff373223ea04e30a3e to: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166401.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166401.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166401.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166401.zip
Checking for progress XML file: /var/lib/boinc-client/slots/10/progress_file_12166401.xml
Opened progress file ok : /var/lib/boinc-client/slots/10/progress_file_12166401.xml
-- Model is restarting --
Adjusting last_iter, 1021, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/10/progress_file_12166401.xml
last_cpu_time: 36166
upload_file_number: 10
last_iter: 1007
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a007_2016092300_15_949_12166401_0_r583304937
The child process has been launched with process id: 187368
Executing the command: /var/lib/boinc-client/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187368:187368] [20221215:163406:1671082446:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
16:35:03 STEP 1008 H= 252:00 +CPU= 45.857
16:35:55 STEP 1009 H= 252:15 +CPU= 50.139
16:36:51 STEP 1010 H= 252:30 +CPU= 52.420
16:37:41 STEP 1011 H= 252:45 +CPU= 48.569
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:48:07 STEP 1012 H= 253:00 +CPU= 47.856
16:49:00 STEP 1013 H= 253:15 +CPU= 49.812
16:49:49 STEP 1014 H= 253:30 +CPU= 46.374
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:59:05 STEP 1015 H= 253:45 +CPU= 50.727
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:12:32 STEP 1016 H= 254:00 +CPU= 47.912
17:13:31 STEP 1017 H= 254:15 +CPU= 49.939
17:14:31 STEP 1018 H= 254:30 +CPU= 51.779
17:15:29 STEP 1019 H= 254:45 +CPU= 50.262
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:20:37 STEP 1020 H= 255:00 +CPU= 58.120
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:27:18 STEP 1021 H= 255:15 +CPU= 42.496
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:36:15 STEP 1022 H= 255:30 +CPU= 46.583
17:36:50 STEP 1023 H= 255:45 +CPU= 30.258
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:49:23 STEP 1024 H= 256:00 +CPU= 31.545
17:50:02 STEP 1025 H= 256:15 +CPU= 32.387
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:56:46 STEP 1026 H= 256:30 +CPU= 25.492
17:57:14 STEP 1027 H= 256:45 +CPU= 26.415
17:57:50 STEP 1028 H= 257:00 +CPU= 34.129
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:00:24 STEP 1029 H= 257:15 +CPU= 30.595
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:02:54 STEP 1030 H= 257:30 +CPU= 26.729
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:06:39 STEP 1031 H= 257:45 +CPU= 27.861
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:20:53 STEP 1032 H= 258:00 +CPU= 33.498
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001032
18:21:27 STEP 1033 H= 258:15 +CPU= 30.876
18:22:03 STEP 1034 H= 258:30 +CPU= 32.239
18:22:38 STEP 1035 H= 258:45 +CPU= 31.605
18:23:15 STEP 1036 H= 259:00 +CPU= 33.551
18:23:51 STEP 1037 H= 259:15 +CPU= 31.805
18:24:25 STEP 1038 H= 259:30 +CPU= 30.968
18:25:05 STEP 1039 H= 259:45 +CPU= 35.223
18:25:48 STEP 1040 H= 260:00 +CPU= 38.288
18:26:26 STEP 1041 H= 260:15 +CPU= 34.829
18:27:05 STEP 1042 H= 260:30 +CPU= 36.078
18:27:44 STEP 1043 H= 260:45 +CPU= 35.591
18:28:34 STEP 1044 H= 261:00 +CPU= 44.228
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001044
18:29:15 STEP 1045 H= 261:15 +CPU= 37.385
18:29:58 STEP 1046 H= 261:30 +CPU= 38.810
18:30:37 STEP 1047 H= 261:45 +CPU= 34.919
18:31:19 STEP 1048 H= 262:00 +CPU= 38.083
18:31:58 STEP 1049 H= 262:15 +CPU= 35.522
18:32:36 STEP 1050 H= 262:30 +CPU= 35.068
18:33:15 STEP 1051 H= 262:45 +CPU= 35.133
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:36:02 STEP 1052 H= 263:00 +CPU= 36.773
18:36:43 STEP 1053 H= 263:15 +CPU= 36.024
18:37:26 STEP 1054 H= 263:30 +CPU= 38.012
18:38:07 STEP 1055 H= 263:45 +CPU= 36.171
18:39:02 STEP 1056 H= 264:00 +CPU= 50.608
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a007_2016092300_15_949_12166401_0_r583304937_10.zip
Uploading the intermediate file: upload_file_10.zip
18:42:31 STEP 1057 H= 264:15 +CPU= 16.780
18:42:40 STEP 1058 H= 264:30 +CPU= 8.776
18:42:49 STEP 1059 H= 264:45 +CPU= 8.710
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:50:09 STEP 1060 H= 265:00 +CPU= 35.875
18:50:49 STEP 1061 H= 265:15 +CPU= 38.698
18:51:29 STEP 1062 H= 265:30 +CPU= 36.080
18:52:10 STEP 1063 H= 265:45 +CPU= 38.069
18:52:53 STEP 1064 H= 266:00 +CPU= 40.446
18:53:35 STEP 1065 H= 266:15 +CPU= 38.004
18:54:17 STEP 1066 H= 266:30 +CPU= 39.174
18:55:01 STEP 1067 H= 266:45 +CPU= 40.177
18:55:50 STEP 1068 H= 267:00 +CPU= 46.351
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001068
18:56:25 STEP 1069 H= 267:15 +CPU= 32.414
18:56:59 STEP 1070 H= 267:30 +CPU= 32.421
18:57:39 STEP 1071 H= 267:45 +CPU= 37.704
18:58:20 STEP 1072 H= 268:00 +CPU= 37.356
18:59:03 STEP 1073 H= 268:15 +CPU= 39.700
18:59:40 STEP 1074 H= 268:30 +CPU= 35.276
19:00:20 STEP 1075 H= 268:45 +CPU= 36.103
19:00:59 STEP 1076 H= 269:00 +CPU= 36.149
19:01:36 STEP 1077 H= 269:15 +CPU= 35.349
19:02:15 STEP 1078 H= 269:30 +CPU= 36.505
19:02:52 STEP 1079 H= 269:45 +CPU= 34.031
19:03:38 STEP 1080 H= 270:00 +CPU= 43.519
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001080
19:04:14 STEP 1081 H= 270:15 +CPU= 34.019
19:04:45 STEP 1082 H= 270:30 +CPU= 29.958
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a007
(argv4) batchid: 949
(argv5) wuid: 12166401
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7ce91203b1b08443f4e24d55a126930 to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a007_2016092300_15_949_12166401.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a007_2016092300_15_949_12166401.zip
ic_ancil_file: ic_ancil_12166401
ifsdata_file: ifsdata_12166401
climate_data_file: clim_data_12166401
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/10/ic_ancil_12166401.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12166401.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_450dacd8598235ff373223ea04e30a3e to: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166401.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166401.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166401.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166401.zip
Checking for progress XML file: /var/lib/boinc-client/slots/10/progress_file_12166401.xml
Opened progress file ok : /var/lib/boinc-client/slots/10/progress_file_12166401.xml
-- Model is restarting --
Adjusting last_iter, 1083, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/10/progress_file_12166401.xml
last_cpu_time: 38995
upload_file_number: 11
last_iter: 1055
last_upload: 950400
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a007_2016092300_15_949_12166401_0_r583304937
The child process has been launched with process id: 220590
Executing the command: /var/lib/boinc-client/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:191143:1671091903:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
19:12:24 STEP 1056 H= 264:00 +CPU= 32.675
19:12:56 STEP 1057 H= 264:15 +CPU= 29.654
19:13:30 STEP 1058 H= 264:30 +CPU= 32.387
19:14:06 STEP 1059 H= 264:45 +CPU= 35.349
19:14:34 STEP 1060 H= 265:00 +CPU= 26.600
19:14:58 STEP 1061 H= 265:15 +CPU= 23.316
19:15:23 STEP 1062 H= 265:30 +CPU= 24.958
19:15:48 STEP 1063 H= 265:45 +CPU= 24.590
19:16:12 STEP 1064 H= 266:00 +CPU= 22.950
19:16:34 STEP 1065 H= 266:15 +CPU= 22.184
19:16:57 STEP 1066 H= 266:30 +CPU= 22.181
19:17:20 STEP 1067 H= 266:45 +CPU= 22.346
19:17:50 STEP 1068 H= 267:00 +CPU= 29.541
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001068
19:18:15 STEP 1069 H= 267:15 +CPU= 23.414
19:18:39 STEP 1070 H= 267:30 +CPU= 23.123
19:19:03 STEP 1071 H= 267:45 +CPU= 23.446
19:19:24 STEP 1072 H= 268:00 +CPU= 20.657
19:19:47 STEP 1073 H= 268:15 +CPU= 22.014
19:20:09 STEP 1074 H= 268:30 +CPU= 21.956
19:20:35 STEP 1075 H= 268:45 +CPU= 24.366
19:21:00 STEP 1076 H= 269:00 +CPU= 24.433
19:21:26 STEP 1077 H= 269:15 +CPU= 24.984
19:21:49 STEP 1078 H= 269:30 +CPU= 22.396
19:22:12 STEP 1079 H= 269:45 +CPU= 22.379
19:22:42 STEP 1080 H= 270:00 +CPU= 28.804
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001080
19:23:02 STEP 1081 H= 270:15 +CPU= 19.666
19:23:25 STEP 1082 H= 270:30 +CPU= 22.004
19:23:48 STEP 1083 H= 270:45 +CPU= 21.983
19:24:15 STEP 1084 H= 271:00 +CPU= 24.958
19:24:37 STEP 1085 H= 271:15 +CPU= 21.843
19:24:57 STEP 1086 H= 271:30 +CPU= 19.703
19:25:19 STEP 1087 H= 271:45 +CPU= 21.541
19:25:40 STEP 1088 H= 272:00 +CPU= 21.443
19:26:06 STEP 1089 H= 272:15 +CPU= 25.122
19:26:26 STEP 1090 H= 272:30 +CPU= 19.854
19:26:47 STEP 1091 H= 272:45 +CPU= 19.880
19:27:13 STEP 1092 H= 273:00 +CPU= 24.900
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001092
19:27:34 STEP 1093 H= 273:15 +CPU= 18.983
19:27:53 STEP 1094 H= 273:30 +CPU= 19.209
19:28:13 STEP 1095 H= 273:45 +CPU= 19.237
19:28:33 STEP 1096 H= 274:00 +CPU= 19.780
19:28:55 STEP 1097 H= 274:15 +CPU= 21.036
19:29:16 STEP 1098 H= 274:30 +CPU= 20.788
19:29:42 STEP 1099 H= 274:45 +CPU= 25.386
19:30:16 STEP 1100 H= 275:00 +CPU= 31.613
19:30:46 STEP 1101 H= 275:15 +CPU= 28.602
19:31:16 STEP 1102 H= 275:30 +CPU= 28.281
19:31:40 STEP 1103 H= 275:45 +CPU= 23.114
19:32:10 STEP 1104 H= 276:00 +CPU= 29.262
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001104
19:32:34 STEP 1105 H= 276:15 +CPU= 23.840
19:33:00 STEP 1106 H= 276:30 +CPU= 25.393
19:33:28 STEP 1107 H= 276:45 +CPU= 27.637
19:33:53 STEP 1108 H= 277:00 +CPU= 24.217
19:34:17 STEP 1109 H= 277:15 +CPU= 23.846
19:34:45 STEP 1110 H= 277:30 +CPU= 27.190
19:35:11 STEP 1111 H= 277:45 +CPU= 25.504
19:35:37 STEP 1112 H= 278:00 +CPU= 25.755
19:36:02 STEP 1113 H= 278:15 +CPU= 23.574
19:36:33 STEP 1114 H= 278:30 +CPU= 29.882
19:37:04 STEP 1115 H= 278:45 +CPU= 28.811
19:37:33 STEP 1116 H= 279:00 +CPU= 28.264
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001116
19:37:57 STEP 1117 H= 279:15 +CPU= 23.526
19:38:24 STEP 1118 H= 279:30 +CPU= 26.042
19:38:53 STEP 1119 H= 279:45 +CPU= 26.609
19:39:24 STEP 1120 H= 280:00 +CPU= 29.875
19:39:50 STEP 1121 H= 280:15 +CPU= 24.400
19:40:16 STEP 1122 H= 280:30 +CPU= 24.932
19:40:45 STEP 1123 H= 280:45 +CPU= 27.061
19:41:15 STEP 1124 H= 281:00 +CPU= 29.242
19:41:42 STEP 1125 H= 281:15 +CPU= 26.417
19:42:12 STEP 1126 H= 281:30 +CPU= 29.317
19:42:40 STEP 1127 H= 281:45 +CPU= 27.608
19:43:12 STEP 1128 H= 282:00 +CPU= 29.743
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001128
19:43:35 STEP 1129 H= 282:15 +CPU= 21.385
19:44:05 STEP 1130 H= 282:30 +CPU= 29.309
19:44:32 STEP 1131 H= 282:45 +CPU= 25.861
19:45:01 STEP 1132 H= 283:00 +CPU= 28.457
19:45:29 STEP 1133 H= 283:15 +CPU= 26.347
19:45:55 STEP 1134 H= 283:30 +CPU= 24.835
19:46:23 STEP 1135 H= 283:45 +CPU= 27.252
19:46:54 STEP 1136 H= 284:00 +CPU= 28.813
19:47:23 STEP 1137 H= 284:15 +CPU= 28.234
19:47:50 STEP 1138 H= 284:30 +CPU= 26.193
19:48:14 STEP 1139 H= 284:45 +CPU= 22.746
19:48:51 STEP 1140 H= 285:00 +CPU= 35.119
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001140
19:49:19 STEP 1141 H= 285:15 +CPU= 26.184
19:49:47 STEP 1142 H= 285:30 +CPU= 27.501
19:50:14 STEP 1143 H= 285:45 +CPU= 24.826
19:50:40 STEP 1144 H= 286:00 +CPU= 24.364
19:51:03 STEP 1145 H= 286:15 +CPU= 22.375
19:51:29 STEP 1146 H= 286:30 +CPU= 24.102
19:51:58 STEP 1147 H= 286:45 +CPU= 27.485
19:52:27 STEP 1148 H= 287:00 +CPU= 26.824
19:52:53 STEP 1149 H= 287:15 +CPU= 24.775
19:53:20 STEP 1150 H= 287:30 +CPU= 25.416
19:53:47 STEP 1151 H= 287:45 +CPU= 26.398
19:54:27 STEP 1152 H= 288:00 +CPU= 37.897
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a007_2016092300_15_949_12166401_0_r583304937_11.zip
Uploading the intermediate file: upload_file_11.zip
19:54:51 STEP 1153 H= 288:15 +CPU= 22.968
19:55:17 STEP 1154 H= 288:30 +CPU= 25.238
19:55:43 STEP 1155 H= 288:45 +CPU= 25.602
19:56:11 STEP 1156 H= 289:00 +CPU= 27.132
19:56:38 STEP 1157 H= 289:15 +CPU= 26.241
19:57:05 STEP 1158 H= 289:30 +CPU= 25.368
19:57:35 STEP 1159 H= 289:45 +CPU= 27.985
19:58:03 STEP 1160 H= 290:00 +CPU= 25.908
19:58:29 STEP 1161 H= 290:15 +CPU= 25.866
19:59:01 STEP 1162 H= 290:30 +CPU= 31.067
19:59:29 STEP 1163 H= 290:45 +CPU= 27.254
20:00:03 STEP 1164 H= 291:00 +CPU= 32.481
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001164
20:00:33 STEP 1165 H= 291:15 +CPU= 28.452
20:01:00 STEP 1166 H= 291:30 +CPU= 26.311
20:01:26 STEP 1167 H= 291:45 +CPU= 24.564
20:01:50 STEP 1168 H= 292:00 +CPU= 23.678
20:02:15 STEP 1169 H= 292:15 +CPU= 24.181
20:02:39 STEP 1170 H= 292:30 +CPU= 23.382
20:03:04 STEP 1171 H= 292:45 +CPU= 23.990
20:03:30 STEP 1172 H= 293:00 +CPU= 25.081
20:03:53 STEP 1173 H= 293:15 +CPU= 21.569
20:04:14 STEP 1174 H= 293:30 +CPU= 20.041
20:04:39 STEP 1175 H= 293:45 +CPU= 24.841
20:05:10 STEP 1176 H= 294:00 +CPU= 29.591
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001176
20:05:36 STEP 1177 H= 294:15 +CPU= 25.196
20:06:01 STEP 1178 H= 294:30 +CPU= 25.184
20:06:24 STEP 1179 H= 294:45 +CPU= 22.476
20:06:50 STEP 1180 H= 295:00 +CPU= 24.216
20:07:17 STEP 1181 H= 295:15 +CPU= 25.364
20:07:41 STEP 1182 H= 295:30 +CPU= 23.432
20:08:07 STEP 1183 H= 295:45 +CPU= 24.824
20:08:33 STEP 1184 H= 296:00 +CPU= 24.732
20:08:57 STEP 1185 H= 296:15 +CPU= 22.794
20:09:22 STEP 1186 H= 296:30 +CPU= 23.701
20:09:45 STEP 1187 H= 296:45 +CPU= 21.143
20:10:14 STEP 1188 H= 297:00 +CPU= 28.410
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001188
20:10:37 STEP 1189 H= 297:15 +CPU= 21.305
20:11:00 STEP 1190 H= 297:30 +CPU= 21.982
20:11:24 STEP 1191 H= 297:45 +CPU= 22.258
20:11:49 STEP 1192 H= 298:00 +CPU= 23.828
20:12:13 STEP 1193 H= 298:15 +CPU= 22.574
20:12:43 STEP 1194 H= 298:30 +CPU= 28.348
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 5397MB (heap), 4897MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x80f380 (old at (nil))
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3843] 5397 MB (maxheap), 4897 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CALL_SL
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 1 (1004)
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LAPINEA
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201453:1671095693:3789.961] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LARMES
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201454:1671095694:3790.962] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:roo:1:1:220590:220590] [20221215:201454:1671095694:3790.962] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 000000000080FFB1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001D8CDC0 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001D5CDD8 Unknown Unknown Unknown
oifs_43r3_model.e 00000000011C7B12 larmes_ 543 larmes.F90
oifs_43r3_model.e 00000000011C319D lapinea_ 270 lapinea.F90
oifs_43r3_model.e 00000000011BB1CC call_sl_ 300 call_sl.F90
oifs_43r3_model.e 000000000139E65D call_sl_heap_ 94 call_sl_heap.F90
oifs_43r3_model.e 0000000000D53278 gp_model_ 533 gp_model.F90
oifs_43r3_model.e 0000000000D3FB32 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 00000000014A1C3E scan2m_ 535 scan2m.F90
oifs_43r3_model.e 00000000006AD494 stepo_ 327 stepo.F90
oifs_43r3_model.e 00000000005750DC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 000000000055E289 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 000000000055D452 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000546AC8 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 00000000004078AD cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040276F MAIN__ 96 master.F90
oifs_43r3_model.e 000000000040270D Unknown Unknown Unknown
oifs_43r3_model.e 0000000001D83B1A Unknown Unknown Unknown
oifs_43r3_model.e 0000000001D85377 Unknown Unknown Unknown
oifs_43r3_model.e 00000000004025E5 Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a007
(argv4) batchid: 949
(argv5) wuid: 12166401
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166401
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c7ce91203b1b08443f4e24d55a126930 to: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a007_2016092300_15_949_12166401.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_bl_a007_2016092300_15_949_12166401.zip
ic_ancil_file: ic_ancil_12166401
ifsdata_file: ifsdata_12166401
climate_data_file: clim_data_12166401
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/10/ic_ancil_12166401.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12166401.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_450dacd8598235ff373223ea04e30a3e to: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166401.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166401.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166401.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166401.zip
Checking for progress XML file: /var/lib/boinc-client/slots/10/progress_file_12166401.xml
Opened progress file ok : /var/lib/boinc-client/slots/10/progress_file_12166401.xml
-- Model is restarting --
Adjusting last_iter, 1195, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/10/progress_file_12166401.xml
last_cpu_time: 42515
upload_file_number: 12
last_iter: 1151
last_upload: 1036800
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a007_2016092300_15_949_12166401_0_r583304937
The child process has been launched with process id: 6290
Executing the command: /var/lib/boinc-client/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6290:6290] [20221216:112012:1671150012:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
11:20:39 STEP 1152 H= 288:00 +CPU= 23.043
11:21:17 STEP 1153 H= 288:15 +CPU= 36.690
11:21:51 STEP 1154 H= 288:30 +CPU= 32.667
11:22:28 STEP 1155 H= 288:45 +CPU= 34.972
11:23:00 STEP 1156 H= 289:00 +CPU= 30.823
11:23:38 STEP 1157 H= 289:15 +CPU= 36.842
11:24:09 STEP 1158 H= 289:30 +CPU= 30.843
11:24:39 STEP 1159 H= 289:45 +CPU= 28.958
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:27:14 STEP 1160 H= 290:00 +CPU= 27.405
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:32:21 STEP 1161 H= 290:15 +CPU= 33.853
11:32:55 STEP 1162 H= 290:30 +CPU= 31.514
11:33:25 STEP 1163 H= 290:45 +CPU= 28.651
11:34:07 STEP 1164 H= 291:00 +CPU= 38.861
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001164
11:34:43 STEP 1165 H= 291:15 +CPU= 33.253
11:35:19 STEP 1166 H= 291:30 +CPU= 33.408
11:35:55 STEP 1167 H= 291:45 +CPU= 33.381
11:36:25 STEP 1168 H= 292:00 +CPU= 27.862
11:36:59 STEP 1169 H= 292:15 +CPU= 31.183
11:37:37 STEP 1170 H= 292:30 +CPU= 34.725
11:38:15 STEP 1171 H= 292:45 +CPU= 36.045
11:38:52 STEP 1172 H= 293:00 +CPU= 34.167
11:39:27 STEP 1173 H= 293:15 +CPU= 32.733
11:40:05 STEP 1174 H= 293:30 +CPU= 35.284
11:40:39 STEP 1175 H= 293:45 +CPU= 32.162
11:41:22 STEP 1176 H= 294:00 +CPU= 40.509
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001176
11:41:58 STEP 1177 H= 294:15 +CPU= 33.632
11:42:34 STEP 1178 H= 294:30 +CPU= 34.605
11:43:09 STEP 1179 H= 294:45 +CPU= 34.381
11:43:44 STEP 1180 H= 295:00 +CPU= 34.075
11:44:20 STEP 1181 H= 295:15 +CPU= 34.439
11:44:53 STEP 1182 H= 295:30 +CPU= 31.441
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:47:43 STEP 1183 H= 295:45 +CPU= 31.381
11:48:19 STEP 1184 H= 296:00 +CPU= 32.991
11:48:53 STEP 1185 H= 296:15 +CPU= 31.913
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:51:29 STEP 1186 H= 296:30 +CPU= 32.637
11:52:05 STEP 1187 H= 296:45 +CPU= 32.959
11:52:50 STEP 1188 H= 297:00 +CPU= 41.488
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001188
11:53:29 STEP 1189 H= 297:15 +CPU= 36.267
11:54:03 STEP 1190 H= 297:30 +CPU= 30.675
11:54:41 STEP 1191 H= 297:45 +CPU= 35.377
11:55:20 STEP 1192 H= 298:00 +CPU= 36.282
11:55:54 STEP 1193 H= 298:15 +CPU= 32.194
11:56:29 STEP 1194 H= 298:30 +CPU= 32.076
11:57:06 STEP 1195 H= 298:45 +CPU= 34.159
11:57:42 STEP 1196 H= 299:00 +CPU= 34.745
11:58:18 STEP 1197 H= 299:15 +CPU= 33.926
11:58:58 STEP 1198 H= 299:30 +CPU= 36.188
11:59:34 STEP 1199 H= 299:45 +CPU= 34.071
12:00:22 STEP 1200 H= 300:00 +CPU= 45.344
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAh7zg+001200
12:01:02 STEP 1201 H= 300:15 +CPU= 37.206
12:01:38 STEP 1202 H= 300:30 +CPU= 33.011
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
