<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
n: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1cac76e188de84702b879efd14204636 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip
ic_ancil_file: ic_ancil_12166402
ifsdata_file: ifsdata_12166402
climate_data_file: clim_data_12166402
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_30f947d694b92b83f2821852fc0c9fec to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12166402.xml
-- Model is restarting --
Adjusting last_iter, 1010, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml
last_cpu_time: 35720
upload_file_number: 10
last_iter: 1007
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904
The child process has been launched with process id: 187185
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:187185:187185] [20221215:163143:1671082303:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
16:32:26 STEP 1008 H= 252:00 +CPU= 36.416
16:33:15 STEP 1009 H= 252:15 +CPU= 46.352
16:34:04 STEP 1010 H= 252:30 +CPU= 47.020
16:34:51 STEP 1011 H= 252:45 +CPU= 45.055
16:35:43 STEP 1012 H= 253:00 +CPU= 49.732
16:36:36 STEP 1013 H= 253:15 +CPU= 49.812
16:37:28 STEP 1014 H= 253:30 +CPU= 50.926
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:47:53 STEP 1015 H= 253:45 +CPU= 48.297
16:48:51 STEP 1016 H= 254:00 +CPU= 53.513
16:49:39 STEP 1017 H= 254:15 +CPU= 46.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
16:58:51 STEP 1018 H= 254:30 +CPU= 49.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:12:23 STEP 1019 H= 254:45 +CPU= 51.237
17:13:37 STEP 1020 H= 255:00 +CPU= 62.792
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001020
17:14:38 STEP 1021 H= 255:15 +CPU= 52.881
17:15:36 STEP 1022 H= 255:30 +CPU= 51.228
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:20:37 STEP 1023 H= 255:45 +CPU= 50.107
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:27:23 STEP 1024 H= 256:00 +CPU= 46.475
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:36:18 STEP 1025 H= 256:15 +CPU= 44.773
17:36:54 STEP 1026 H= 256:30 +CPU= 30.196
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:49:26 STEP 1027 H= 256:45 +CPU= 30.050
17:50:03 STEP 1028 H= 257:00 +CPU= 31.744
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
17:56:54 STEP 1029 H= 257:15 +CPU= 30.563
17:57:31 STEP 1030 H= 257:30 +CPU= 34.956
17:57:58 STEP 1031 H= 257:45 +CPU= 26.439
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:00:36 STEP 1032 H= 258:00 +CPU= 34.927
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001032
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:06:30 STEP 1033 H= 258:15 +CPU= 33.637
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:20:40 STEP 1034 H= 258:30 +CPU= 29.706
18:21:15 STEP 1035 H= 258:45 +CPU= 31.989
18:21:52 STEP 1036 H= 259:00 +CPU= 33.872
18:22:28 STEP 1037 H= 259:15 +CPU= 32.003
18:23:01 STEP 1038 H= 259:30 +CPU= 31.253
18:23:40 STEP 1039 H= 259:45 +CPU= 34.279
18:24:16 STEP 1040 H= 260:00 +CPU= 32.858
18:24:51 STEP 1041 H= 260:15 +CPU= 31.399
18:25:30 STEP 1042 H= 260:30 +CPU= 35.976
18:26:10 STEP 1043 H= 260:45 +CPU= 36.070
18:26:59 STEP 1044 H= 261:00 +CPU= 45.400
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001044
18:27:36 STEP 1045 H= 261:15 +CPU= 34.628
18:28:18 STEP 1046 H= 261:30 +CPU= 38.237
18:29:01 STEP 1047 H= 261:45 +CPU= 38.090
18:29:44 STEP 1048 H= 262:00 +CPU= 38.646
18:30:23 STEP 1049 H= 262:15 +CPU= 35.717
18:31:04 STEP 1050 H= 262:30 +CPU= 36.131
18:31:46 STEP 1051 H= 262:45 +CPU= 39.254
18:32:31 STEP 1052 H= 263:00 +CPU= 39.858
18:33:12 STEP 1053 H= 263:15 +CPU= 36.541
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:35:56 STEP 1054 H= 263:30 +CPU= 35.263
18:36:42 STEP 1055 H= 263:45 +CPU= 40.312
18:37:37 STEP 1056 H= 264:00 +CPU= 50.133
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904_10.zip
Uploading the intermediate file: upload_file_10.zip
18:38:17 STEP 1057 H= 264:15 +CPU= 35.476
18:38:54 STEP 1058 H= 264:30 +CPU= 34.090
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
18:49:54 STEP 1059 H= 264:45 +CPU= 39.229
18:50:39 STEP 1060 H= 265:00 +CPU= 41.500
18:51:17 STEP 1061 H= 265:15 +CPU= 35.331
18:51:55 STEP 1062 H= 265:30 +CPU= 35.425
18:52:34 STEP 1063 H= 265:45 +CPU= 36.086
18:53:13 STEP 1064 H= 266:00 +CPU= 36.277
18:53:53 STEP 1065 H= 266:15 +CPU= 37.718
18:54:35 STEP 1066 H= 266:30 +CPU= 39.196
18:55:13 STEP 1067 H= 266:45 +CPU= 35.156
18:55:58 STEP 1068 H= 267:00 +CPU= 41.956
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001068
18:56:41 STEP 1069 H= 267:15 +CPU= 40.242
18:57:20 STEP 1070 H= 267:30 +CPU= 37.710
18:58:03 STEP 1071 H= 267:45 +CPU= 40.159
18:58:43 STEP 1072 H= 268:00 +CPU= 37.856
18:59:24 STEP 1073 H= 268:15 +CPU= 38.339
19:00:07 STEP 1074 H= 268:30 +CPU= 41.378
19:00:46 STEP 1075 H= 268:45 +CPU= 36.402
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a008
(argv4) batchid: 949
(argv5) wuid: 12166402
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1cac76e188de84702b879efd14204636 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip
ic_ancil_file: ic_ancil_12166402
ifsdata_file: ifsdata_12166402
climate_data_file: clim_data_12166402
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_30f947d694b92b83f2821852fc0c9fec to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12166402.xml
-- Model is restarting --
Adjusting last_iter, 1076, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml
last_cpu_time: 38437
upload_file_number: 11
last_iter: 1055
last_upload: 950400
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904
The child process has been launched with process id: 219518
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:190217:1671091337:0.001] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
19:03:05 STEP 1056 H= 264:00 +CPU= 37.603
19:03:43 STEP 1057 H= 264:15 +CPU= 35.370
19:04:19 STEP 1058 H= 264:30 +CPU= 35.362
19:04:52 STEP 1059 H= 264:45 +CPU= 31.617
19:05:28 STEP 1060 H= 265:00 +CPU= 34.301
19:06:00 STEP 1061 H= 265:15 +CPU= 30.511
19:06:40 STEP 1062 H= 265:30 +CPU= 38.444
19:07:14 STEP 1063 H= 265:45 +CPU= 32.350
19:07:50 STEP 1064 H= 266:00 +CPU= 33.931
19:08:24 STEP 1065 H= 266:15 +CPU= 32.182
19:09:04 STEP 1066 H= 266:30 +CPU= 37.531
19:09:35 STEP 1067 H= 266:45 +CPU= 29.264
19:10:17 STEP 1068 H= 267:00 +CPU= 39.307
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001068
19:10:52 STEP 1069 H= 267:15 +CPU= 33.840
19:11:29 STEP 1070 H= 267:30 +CPU= 35.318
19:12:00 STEP 1071 H= 267:45 +CPU= 30.048
19:12:28 STEP 1072 H= 268:00 +CPU= 27.664
19:13:02 STEP 1073 H= 268:15 +CPU= 31.973
19:13:32 STEP 1074 H= 268:30 +CPU= 28.005
19:14:07 STEP 1075 H= 268:45 +CPU= 33.745
19:14:35 STEP 1076 H= 269:00 +CPU= 26.518
19:15:02 STEP 1077 H= 269:15 +CPU= 25.226
19:15:27 STEP 1078 H= 269:30 +CPU= 23.679
19:15:51 STEP 1079 H= 269:45 +CPU= 23.365
19:16:21 STEP 1080 H= 270:00 +CPU= 28.869
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001080
19:16:44 STEP 1081 H= 270:15 +CPU= 23.018
19:17:07 STEP 1082 H= 270:30 +CPU= 21.875
19:17:27 STEP 1083 H= 270:45 +CPU= 19.757
19:17:50 STEP 1084 H= 271:00 +CPU= 23.352
19:18:13 STEP 1085 H= 271:15 +CPU= 22.534
19:18:39 STEP 1086 H= 271:30 +CPU= 23.827
19:19:01 STEP 1087 H= 271:45 +CPU= 22.286
19:19:28 STEP 1088 H= 272:00 +CPU= 25.286
19:19:51 STEP 1089 H= 272:15 +CPU= 22.232
19:20:19 STEP 1090 H= 272:30 +CPU= 25.991
19:20:44 STEP 1091 H= 272:45 +CPU= 24.328
19:21:11 STEP 1092 H= 273:00 +CPU= 26.456
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001092
19:21:36 STEP 1093 H= 273:15 +CPU= 24.960
19:21:57 STEP 1094 H= 273:30 +CPU= 20.102
19:22:20 STEP 1095 H= 273:45 +CPU= 21.939
19:22:43 STEP 1096 H= 274:00 +CPU= 21.951
19:23:09 STEP 1097 H= 274:15 +CPU= 23.530
19:23:33 STEP 1098 H= 274:30 +CPU= 22.770
19:24:00 STEP 1099 H= 274:45 +CPU= 24.606
19:24:22 STEP 1100 H= 275:00 +CPU= 22.046
19:24:47 STEP 1101 H= 275:15 +CPU= 23.635
19:25:09 STEP 1102 H= 275:30 +CPU= 21.640
19:25:29 STEP 1103 H= 275:45 +CPU= 20.270
19:25:58 STEP 1104 H= 276:00 +CPU= 28.423
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001104
19:26:21 STEP 1105 H= 276:15 +CPU= 21.993
19:26:43 STEP 1106 H= 276:30 +CPU= 21.062
19:27:03 STEP 1107 H= 276:45 +CPU= 20.109
19:27:26 STEP 1108 H= 277:00 +CPU= 22.208
19:27:47 STEP 1109 H= 277:15 +CPU= 21.112
19:28:09 STEP 1110 H= 277:30 +CPU= 20.962
19:28:30 STEP 1111 H= 277:45 +CPU= 19.897
19:28:52 STEP 1112 H= 278:00 +CPU= 21.512
19:29:13 STEP 1113 H= 278:15 +CPU= 21.302
19:29:39 STEP 1114 H= 278:30 +CPU= 24.702
19:30:10 STEP 1115 H= 278:45 +CPU= 28.683
19:30:40 STEP 1116 H= 279:00 +CPU= 30.151
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001116
19:31:10 STEP 1117 H= 279:15 +CPU= 28.324
19:31:36 STEP 1118 H= 279:30 +CPU= 25.078
19:31:58 STEP 1119 H= 279:45 +CPU= 21.917
19:32:20 STEP 1120 H= 280:00 +CPU= 22.178
19:32:49 STEP 1121 H= 280:15 +CPU= 28.621
19:33:15 STEP 1122 H= 280:30 +CPU= 25.494
19:33:41 STEP 1123 H= 280:45 +CPU= 25.778
19:34:07 STEP 1124 H= 281:00 +CPU= 25.450
19:34:31 STEP 1125 H= 281:15 +CPU= 23.630
19:34:55 STEP 1126 H= 281:30 +CPU= 23.572
19:35:18 STEP 1127 H= 281:45 +CPU= 23.263
19:35:56 STEP 1128 H= 282:00 +CPU= 35.671
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001128
19:36:24 STEP 1129 H= 282:15 +CPU= 26.441
19:36:56 STEP 1130 H= 282:30 +CPU= 31.054
19:37:25 STEP 1131 H= 282:45 +CPU= 27.181
19:37:54 STEP 1132 H= 283:00 +CPU= 27.775
19:38:19 STEP 1133 H= 283:15 +CPU= 25.302
19:38:44 STEP 1134 H= 283:30 +CPU= 24.235
19:39:15 STEP 1135 H= 283:45 +CPU= 30.780
19:39:44 STEP 1136 H= 284:00 +CPU= 27.912
19:40:12 STEP 1137 H= 284:15 +CPU= 26.938
19:40:38 STEP 1138 H= 284:30 +CPU= 26.704
19:41:02 STEP 1139 H= 284:45 +CPU= 23.178
19:41:35 STEP 1140 H= 285:00 +CPU= 32.572
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001140
19:42:06 STEP 1141 H= 285:15 +CPU= 30.367
19:42:35 STEP 1142 H= 285:30 +CPU= 28.185
19:43:03 STEP 1143 H= 285:45 +CPU= 26.619
19:43:35 STEP 1144 H= 286:00 +CPU= 30.954
19:44:03 STEP 1145 H= 286:15 +CPU= 27.920
19:44:30 STEP 1146 H= 286:30 +CPU= 26.499
19:44:58 STEP 1147 H= 286:45 +CPU= 27.891
19:45:29 STEP 1148 H= 287:00 +CPU= 30.128
19:45:57 STEP 1149 H= 287:15 +CPU= 27.971
19:46:25 STEP 1150 H= 287:30 +CPU= 26.921
19:46:51 STEP 1151 H= 287:45 +CPU= 26.014
19:47:34 STEP 1152 H= 288:00 +CPU= 39.684
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904_11.zip
Uploading the intermediate file: upload_file_11.zip
19:48:01 STEP 1153 H= 288:15 +CPU= 25.080
19:48:24 STEP 1154 H= 288:30 +CPU= 22.347
19:48:47 STEP 1155 H= 288:45 +CPU= 22.582
19:49:16 STEP 1156 H= 289:00 +CPU= 26.877
19:49:41 STEP 1157 H= 289:15 +CPU= 23.725
19:50:11 STEP 1158 H= 289:30 +CPU= 28.334
19:50:38 STEP 1159 H= 289:45 +CPU= 26.421
19:51:07 STEP 1160 H= 290:00 +CPU= 28.888
19:51:35 STEP 1161 H= 290:15 +CPU= 25.890
19:52:02 STEP 1162 H= 290:30 +CPU= 26.053
19:52:31 STEP 1163 H= 290:45 +CPU= 27.335
19:53:08 STEP 1164 H= 291:00 +CPU= 36.416
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001164
19:53:36 STEP 1165 H= 291:15 +CPU= 26.980
19:53:59 STEP 1166 H= 291:30 +CPU= 22.545
19:54:29 STEP 1167 H= 291:45 +CPU= 28.477
19:54:56 STEP 1168 H= 292:00 +CPU= 25.582
19:55:23 STEP 1169 H= 292:15 +CPU= 25.863
19:55:51 STEP 1170 H= 292:30 +CPU= 26.960
19:56:19 STEP 1171 H= 292:45 +CPU= 26.339
19:56:46 STEP 1172 H= 293:00 +CPU= 25.902
19:57:16 STEP 1173 H= 293:15 +CPU= 28.525
19:57:45 STEP 1174 H= 293:30 +CPU= 27.383
19:58:12 STEP 1175 H= 293:45 +CPU= 26.244
19:58:52 STEP 1176 H= 294:00 +CPU= 37.871
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001176
19:59:21 STEP 1177 H= 294:15 +CPU= 28.198
19:59:48 STEP 1178 H= 294:30 +CPU= 25.592
20:00:19 STEP 1179 H= 294:45 +CPU= 29.990
20:00:45 STEP 1180 H= 295:00 +CPU= 25.977
20:01:20 STEP 1181 H= 295:15 +CPU= 35.180
20:01:49 STEP 1182 H= 295:30 +CPU= 27.236
20:02:15 STEP 1183 H= 295:45 +CPU= 25.332
20:02:40 STEP 1184 H= 296:00 +CPU= 24.082
20:03:04 STEP 1185 H= 296:15 +CPU= 23.008
20:03:30 STEP 1186 H= 296:30 +CPU= 25.009
20:03:55 STEP 1187 H= 296:45 +CPU= 24.104
20:04:26 STEP 1188 H= 297:00 +CPU= 29.009
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001188
20:04:49 STEP 1189 H= 297:15 +CPU= 22.031
20:05:12 STEP 1190 H= 297:30 +CPU= 22.081
20:05:38 STEP 1191 H= 297:45 +CPU= 26.369
20:06:05 STEP 1192 H= 298:00 +CPU= 25.223
20:06:30 STEP 1193 H= 298:15 +CPU= 24.024
20:06:55 STEP 1194 H= 298:30 +CPU= 24.053
20:07:21 STEP 1195 H= 298:45 +CPU= 24.215
20:07:48 STEP 1196 H= 299:00 +CPU= 25.540
20:08:13 STEP 1197 H= 299:15 +CPU= 23.387
20:08:40 STEP 1198 H= 299:30 +CPU= 25.916
20:09:04 STEP 1199 H= 299:45 +CPU= 23.634
20:09:40 STEP 1200 H= 300:00 +CPU= 33.804
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001200
20:10:05 STEP 1201 H= 300:15 +CPU= 24.364
20:10:31 STEP 1202 H= 300:30 +CPU= 24.549
20:10:56 STEP 1203 H= 300:45 +CPU= 24.690
20:11:22 STEP 1204 H= 301:00 +CPU= 25.743
20:11:48 STEP 1205 H= 301:15 +CPU= 25.018
20:12:15 STEP 1206 H= 301:30 +CPU= 25.210
20:12:42 STEP 1207 H= 301:45 +CPU= 26.296
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 5397MB (heap), 4897MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x80f380 (old at (nil))
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3843] 5397 MB (maxheap), 4897 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CALL_SL_HEAP
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : CALL_SL
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : >OMP-CALL_SL 1 (1004)
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LAPINEA
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LARCINA
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201453:1671095693:4356.564] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:3897] : LASCAW
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201454:1671095694:4357.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:roo:1:1:219518:219518] [20221215:201454:1671095694:4357.564] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 000000000080FFB1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001D8CDC0 Unknown Unknown Unknown
oifs_43r3_model.e 00000000011D2146 lascaw_ 611 lascaw.F90
oifs_43r3_model.e 00000000011CB5C5 larcina_ 319 larcina.F90
oifs_43r3_model.e 00000000011C416D lapinea_ 325 lapinea.F90
oifs_43r3_model.e 00000000011BB1CC call_sl_ 300 call_sl.F90
oifs_43r3_model.e 000000000139E65D call_sl_heap_ 94 call_sl_heap.F90
oifs_43r3_model.e 0000000000D53278 gp_model_ 533 gp_model.F90
oifs_43r3_model.e 0000000000D3FB32 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 00000000014A1C3E scan2m_ 535 scan2m.F90
oifs_43r3_model.e 00000000006AD494 stepo_ 327 stepo.F90
oifs_43r3_model.e 00000000005750DC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 000000000055E289 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 000000000055D452 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000546AC8 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 00000000004078AD cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040276F MAIN__ 96 master.F90
oifs_43r3_model.e 000000000040270D Unknown Unknown Unknown
oifs_43r3_model.e 0000000001D83B1A Unknown Unknown Unknown
oifs_43r3_model.e 0000000001D85377 Unknown Unknown Unknown
oifs_43r3_model.e 00000000004025E5 Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.07_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a008
(argv4) batchid: 949
(argv5) wuid: 12166402
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/14
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.07
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_app_1.07_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1cac76e188de84702b879efd14204636 to: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/14/oifs_43r3_bl_a008_2016092300_15_949_12166402.zip
ic_ancil_file: ic_ancil_12166402
ifsdata_file: ifsdata_12166402
climate_data_file: clim_data_12166402
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/14/ic_ancil_12166402.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_30f947d694b92b83f2821852fc0c9fec to: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/14/ifsdata/ifsdata_12166402.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/14/159l_2/clim_data_12166402.zip
Checking for progress XML file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml
Opened progress file ok : /var/lib/boinc-client/slots/14/progress_file_12166402.xml
-- Model is restarting --
Adjusting last_iter, 1208, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/14/progress_file_12166402.xml
last_cpu_time: 42506
upload_file_number: 12
last_iter: 1199
last_upload: 1036800
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904
The child process has been launched with process id: 6291
Executing the command: /var/lib/boinc-client/slots/14/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2091] fp = 0x30eaa80
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x80f7e0 (old at 0x401140)
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x80f7e0 (old at (nil))
[EC_DRHOOK:roo:1:1:6291:6291] [20221216:112012:1671150012:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
11:20:40 STEP 1200 H= 300:00 +CPU= 23.917
11:21:19 STEP 1201 H= 300:15 +CPU= 36.826
11:21:51 STEP 1202 H= 300:30 +CPU= 31.651
11:22:29 STEP 1203 H= 300:45 +CPU= 36.471
11:23:02 STEP 1204 H= 301:00 +CPU= 30.760
11:23:27 STEP 1205 H= 301:15 +CPU= 24.166
11:23:53 STEP 1206 H= 301:30 +CPU= 25.251
11:24:26 STEP 1207 H= 301:45 +CPU= 32.429
11:25:01 STEP 1208 H= 302:00 +CPU= 34.462
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:32:09 STEP 1209 H= 302:15 +CPU= 27.921
11:32:46 STEP 1210 H= 302:30 +CPU= 33.579
11:33:22 STEP 1211 H= 302:45 +CPU= 32.640
11:34:07 STEP 1212 H= 303:00 +CPU= 40.928
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001212
11:34:46 STEP 1213 H= 303:15 +CPU= 35.514
11:35:22 STEP 1214 H= 303:30 +CPU= 33.107
11:35:57 STEP 1215 H= 303:45 +CPU= 33.149
11:36:36 STEP 1216 H= 304:00 +CPU= 35.608
11:37:10 STEP 1217 H= 304:15 +CPU= 31.033
11:37:45 STEP 1218 H= 304:30 +CPU= 32.983
11:38:22 STEP 1219 H= 304:45 +CPU= 34.865
11:39:04 STEP 1220 H= 305:00 +CPU= 37.816
11:39:38 STEP 1221 H= 305:15 +CPU= 31.693
11:40:15 STEP 1222 H= 305:30 +CPU= 34.608
11:40:53 STEP 1223 H= 305:45 +CPU= 35.185
11:41:35 STEP 1224 H= 306:00 +CPU= 39.223
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001224
11:42:12 STEP 1225 H= 306:15 +CPU= 35.144
11:42:48 STEP 1226 H= 306:30 +CPU= 34.565
11:43:24 STEP 1227 H= 306:45 +CPU= 34.730
11:44:01 STEP 1228 H= 307:00 +CPU= 35.589
11:44:37 STEP 1229 H= 307:15 +CPU= 34.759
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:47:25 STEP 1230 H= 307:30 +CPU= 30.013
11:47:58 STEP 1231 H= 307:45 +CPU= 30.191
11:48:36 STEP 1232 H= 308:00 +CPU= 35.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
11:51:12 STEP 1233 H= 308:15 +CPU= 33.937
11:51:46 STEP 1234 H= 308:30 +CPU= 31.864
11:52:22 STEP 1235 H= 308:45 +CPU= 34.058
11:53:07 STEP 1236 H= 309:00 +CPU= 41.385
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001236
11:53:45 STEP 1237 H= 309:15 +CPU= 35.418
11:54:19 STEP 1238 H= 309:30 +CPU= 31.224
11:54:55 STEP 1239 H= 309:45 +CPU= 33.714
11:55:32 STEP 1240 H= 310:00 +CPU= 34.576
11:56:07 STEP 1241 H= 310:15 +CPU= 32.961
11:56:45 STEP 1242 H= 310:30 +CPU= 35.197
11:57:25 STEP 1243 H= 310:45 +CPU= 36.260
11:58:01 STEP 1244 H= 311:00 +CPU= 32.523
11:58:39 STEP 1245 H= 311:15 +CPU= 34.647
11:59:16 STEP 1246 H= 311:30 +CPU= 34.262
11:59:50 STEP 1247 H= 311:45 +CPU= 32.474
12:00:38 STEP 1248 H= 312:00 +CPU= 43.586
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/14/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12166402/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a008_2016092300_15_949_12166402_0_r1649076904_12.zip
Uploading the intermediate file: upload_file_12.zip
12:01:19 STEP 1249 H= 312:15 +CPU= 36.569
12:01:54 STEP 1250 H= 312:30 +CPU= 33.945
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully
</stderr_txt>
]]>
