<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
H= 476:00 +CPU= 21.150
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:47:48 STEP 477 H= 477:00 +CPU= 32.800
13:48:10 STEP 478 H= 478:00 +CPU= 21.675
13:48:31 STEP 479 H= 479:00 +CPU= 20.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:50:21 STEP 480 H= 480:00 +CPU= 39.116
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000480
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000456
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000468
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000468
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018_19.zip
Uploading the intermediate file: upload_file_19.zip
13:50:40 STEP 481 H= 481:00 +CPU= 19.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:52:16 STEP 482 H= 482:00 +CPU= 21.791
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:54:10 STEP 483 H= 483:00 +CPU= 32.646
13:54:31 STEP 484 H= 484:00 +CPU= 21.048
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:56:04 STEP 485 H= 485:00 +CPU= 20.798
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:57:49 STEP 486 H= 486:00 +CPU= 33.592
13:58:10 STEP 487 H= 487:00 +CPU= 21.162
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
13:59:52 STEP 488 H= 488:00 +CPU= 21.579
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:01:26 STEP 489 H= 489:00 +CPU= 32.778
14:01:48 STEP 490 H= 490:00 +CPU= 21.428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
14:03:19 STEP 491 H= 491:00 +CPU= 20.883
14:03:55 STEP 492 H= 492:00 +CPU= 36.172
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.302] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 3547MB (heap), 4419MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 3547 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : MASTER
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT0
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT1
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT2
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT3
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CNT4
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : STEPO
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : SCAN2M
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL_HEAP
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GP_MODEL
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG_DRV
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : >OMP-CPG (1025)
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : CPG_GP
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153853:1674110333:8247.306] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] : GPGRGEO
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153854:1674110334:8248.306] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:p500:1:1:3285920:3285920] [20230119:153854:1674110334:8248.306] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
oifs_43r3_model.e 0000000001CE916B Unknown Unknown Unknown
oifs_43r3_model.e 000000000081FFF1 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DC9090 Unknown Unknown Unknown
oifs_43r3_model.e 0000000000C5C6ED gpgrgeo_ 227 gpgrgeo.F90
oifs_43r3_model.e 0000000000C399CD cpg_gp_ 832 cpg_gp.F90
oifs_43r3_model.e 0000000000C2E72E cpg_ 627 cpg.F90
oifs_43r3_model.e 0000000000C2C638 cpg_drv_ 351 cpg_drv.F90
oifs_43r3_model.e 0000000000BF17C8 gp_model_ 419 gp_model.F90
oifs_43r3_model.e 00000000012E9482 gp_model_heap_ 74 gp_model_heap.F90
oifs_43r3_model.e 0000000000BBC1CE scan2m_ 535 scan2m.F90
oifs_43r3_model.e 000000000057A664 stepo_ 327 stepo.F90
oifs_43r3_model.e 000000000055DEEC cnt4_ 1133 cnt4.F90
oifs_43r3_model.e 00000000005471C9 cnt3_ 267 cnt3.F90
oifs_43r3_model.e 0000000000546412 cnt2_ 88 cnt2.F90
oifs_43r3_model.e 0000000000545E28 cnt1_ 92 cnt1.F90
oifs_43r3_model.e 000000000040708D cnt0_ 146 cnt0.F90
oifs_43r3_model.e 000000000040220F MAIN__ 96 master.F90
oifs_43r3_model.e 00000000004021A2 Unknown Unknown Unknown
oifs_43r3_model.e 0000000001DCA390 Unknown Unknown Unknown
oifs_43r3_model.e 000000000040206E Unknown Unknown Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2016050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0293
(argv4) batchid: 985
(argv5) wuid: 12201937
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2070d7e6834a0cc8305e1783101e78ad to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0293_2016050100_123_985_12201937.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0293_2016050100_123_985_12201937.zip
ic_ancil_file: ic_ancil_12201937
ifsdata_file: ifsdata_12201937
climate_data_file: clim_data_12201937
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8e466483447d435415fd2a687b5597f6 to: /var/lib/boinc-client/slots/3/ic_ancil_12201937.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12201937.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f6f4f299b41bde418d5a6a3eb9fa418c to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12201937.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12201937.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12201937.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12201937.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12201937.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12201937.xml
-- Model is restarting --
Adjusting last_iter, 492, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12201937.xml
last_cpu_time: 14037
upload_file_number: 20
last_iter: 479
last_upload: 1728000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018
The child process has been launched with process id: 52432
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:52432:52432] [20230119:195022:1674125422:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:28:20 STEP 480 H= 480:00 +CPU= 39.613
20:28:44 STEP 481 H= 481:00 +CPU= 23.398
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:31:29 STEP 482 H= 482:00 +CPU= 23.251
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:47:58 STEP 483 H= 483:00 +CPU= 35.251
20:48:20 STEP 484 H= 484:00 +CPU= 21.542
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
20:50:30 STEP 485 H= 485:00 +CPU= 20.547
20:51:07 STEP 486 H= 486:00 +CPU= 35.893
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:12:21 STEP 487 H= 487:00 +CPU= 21.353
21:12:43 STEP 488 H= 488:00 +CPU= 21.339
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:14:47 STEP 489 H= 489:00 +CPU= 32.340
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:16:30 STEP 490 H= 490:00 +CPU= 21.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:18:02 STEP 491 H= 491:00 +CPU= 20.494
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:19:37 STEP 492 H= 492:00 +CPU= 34.576
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000492
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000492
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:20:58 STEP 493 H= 493:00 +CPU= 20.015
21:21:18 STEP 494 H= 494:00 +CPU= 20.276
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:24:12 STEP 495 H= 495:00 +CPU= 31.696
21:24:34 STEP 496 H= 496:00 +CPU= 21.698
21:24:54 STEP 497 H= 497:00 +CPU= 20.152
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:26:47 STEP 498 H= 498:00 +CPU= 31.891
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:28:40 STEP 499 H= 499:00 +CPU= 20.465
21:29:01 STEP 500 H= 500:00 +CPU= 21.131
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:30:54 STEP 501 H= 501:00 +CPU= 31.718
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:32:24 STEP 502 H= 502:00 +CPU= 20.186
21:32:45 STEP 503 H= 503:00 +CPU= 21.345
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:35:44 STEP 504 H= 504:00 +CPU= 37.575
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000504
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000480
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000492
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000492
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018_20.zip
Uploading the intermediate file: upload_file_20.zip
21:36:05 STEP 505 H= 505:00 +CPU= 20.944
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:37:36 STEP 506 H= 506:00 +CPU= 20.938
21:38:09 STEP 507 H= 507:00 +CPU= 32.172
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:40:01 STEP 508 H= 508:00 +CPU= 21.326
21:40:21 STEP 509 H= 509:00 +CPU= 20.142
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:43:26 STEP 510 H= 510:00 +CPU= 33.104
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:44:48 STEP 511 H= 511:00 +CPU= 20.870
21:45:08 STEP 512 H= 512:00 +CPU= 20.507
21:45:39 STEP 513 H= 513:00 +CPU= 30.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:47:31 STEP 514 H= 514:00 +CPU= 20.820
21:47:52 STEP 515 H= 515:00 +CPU= 21.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:49:38 STEP 516 H= 516:00 +CPU= 34.830
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:51:11 STEP 517 H= 517:00 +CPU= 20.180
21:51:32 STEP 518 H= 518:00 +CPU= 21.050
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:54:15 STEP 519 H= 519:00 +CPU= 31.906
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2016050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0293
(argv4) batchid: 985
(argv5) wuid: 12201937
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2070d7e6834a0cc8305e1783101e78ad to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0293_2016050100_123_985_12201937.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0293_2016050100_123_985_12201937.zip
ic_ancil_file: ic_ancil_12201937
ifsdata_file: ifsdata_12201937
climate_data_file: clim_data_12201937
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8e466483447d435415fd2a687b5597f6 to: /var/lib/boinc-client/slots/3/ic_ancil_12201937.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12201937.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f6f4f299b41bde418d5a6a3eb9fa418c to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12201937.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12201937.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12201937.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12201937.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12201937.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12201937.xml
-- Model is restarting --
Adjusting last_iter, 520, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12201937.xml
last_cpu_time: 15071
upload_file_number: 21
last_iter: 503
last_upload: 1814400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018
The child process has been launched with process id: 73907
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:73907:73907] [20230119:215557:1674132957:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:57:28 STEP 504 H= 504:00 +CPU= 36.055
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
21:58:50 STEP 505 H= 505:00 +CPU= 20.721
21:59:11 STEP 506 H= 506:00 +CPU= 20.716
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:01:04 STEP 507 H= 507:00 +CPU= 31.960
22:01:25 STEP 508 H= 508:00 +CPU= 21.080
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:03:05 STEP 509 H= 509:00 +CPU= 20.416
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:04:40 STEP 510 H= 510:00 +CPU= 33.276
22:05:00 STEP 511 H= 511:00 +CPU= 19.801
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:06:32 STEP 512 H= 512:00 +CPU= 20.662
22:07:04 STEP 513 H= 513:00 +CPU= 32.034
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:08:57 STEP 514 H= 514:00 +CPU= 20.636
22:09:17 STEP 515 H= 515:00 +CPU= 20.160
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:11:24 STEP 516 H= 516:00 +CPU= 35.486
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000516
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000516
22:11:44 STEP 517 H= 517:00 +CPU= 19.789
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:13:57 STEP 518 H= 518:00 +CPU= 21.954
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:15:41 STEP 519 H= 519:00 +CPU= 32.732
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:17:03 STEP 520 H= 520:00 +CPU= 21.310
22:17:24 STEP 521 H= 521:00 +CPU= 20.531
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:19:08 STEP 522 H= 522:00 +CPU= 32.969
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:20:59 STEP 523 H= 523:00 +CPU= 20.205
22:21:21 STEP 524 H= 524:00 +CPU= 21.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:23:04 STEP 525 H= 525:00 +CPU= 31.810
22:23:24 STEP 526 H= 526:00 +CPU= 20.253
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:24:57 STEP 527 H= 527:00 +CPU= 20.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:27:15 STEP 528 H= 528:00 +CPU= 37.165
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000528
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000504
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018_21.zip
Uploading the intermediate file: upload_file_21.zip
22:27:36 STEP 529 H= 529:00 +CPU= 21.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:28:58 STEP 530 H= 530:00 +CPU= 21.626
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:30:51 STEP 531 H= 531:00 +CPU= 31.309
22:31:12 STEP 532 H= 532:00 +CPU= 21.037
22:31:32 STEP 533 H= 533:00 +CPU= 19.805
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:33:16 STEP 534 H= 534:00 +CPU= 32.798
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:35:07 STEP 535 H= 535:00 +CPU= 20.036
22:35:27 STEP 536 H= 536:00 +CPU= 20.515
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:37:31 STEP 537 H= 537:00 +CPU= 32.732
22:37:52 STEP 538 H= 538:00 +CPU= 20.925
22:38:12 STEP 539 H= 539:00 +CPU= 19.684
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:40:18 STEP 540 H= 540:00 +CPU= 35.740
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000540
22:41:51 STEP 541 H= 541:00 +CPU= 20.461
22:42:11 STEP 542 H= 542:00 +CPU= 20.219
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:44:04 STEP 543 H= 543:00 +CPU= 32.149
22:44:25 STEP 544 H= 544:00 +CPU= 20.633
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:46:26 STEP 545 H= 545:00 +CPU= 20.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:48:11 STEP 546 H= 546:00 +CPU= 32.255
22:48:31 STEP 547 H= 547:00 +CPU= 20.181
22:48:52 STEP 548 H= 548:00 +CPU= 20.666
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:51:05 STEP 549 H= 549:00 +CPU= 32.138
22:51:26 STEP 550 H= 550:00 +CPU= 21.060
22:51:46 STEP 551 H= 551:00 +CPU= 20.093
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:53:46 STEP 552 H= 552:00 +CPU= 38.339
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018_22.zip
Uploading the intermediate file: upload_file_22.zip
22:55:18 STEP 553 H= 553:00 +CPU= 20.740
22:55:38 STEP 554 H= 554:00 +CPU= 20.413
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:57:42 STEP 555 H= 555:00 +CPU= 32.734
22:58:02 STEP 556 H= 556:00 +CPU= 19.978
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
22:59:53 STEP 557 H= 557:00 +CPU= 19.474
23:00:26 STEP 558 H= 558:00 +CPU= 33.187
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:01:59 STEP 559 H= 559:00 +CPU= 20.651
23:02:19 STEP 560 H= 560:00 +CPU= 20.528
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:04:43 STEP 561 H= 561:00 +CPU= 32.857
23:05:04 STEP 562 H= 562:00 +CPU= 20.570
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:06:36 STEP 563 H= 563:00 +CPU= 20.707
23:07:11 STEP 564 H= 564:00 +CPU= 34.881
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000564
23:08:52 STEP 565 H= 565:00 +CPU= 20.392
23:09:13 STEP 566 H= 566:00 +CPU= 20.942
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:11:07 STEP 567 H= 567:00 +CPU= 32.272
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:12:49 STEP 568 H= 568:00 +CPU= 20.892
23:13:10 STEP 569 H= 569:00 +CPU= 21.020
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:14:54 STEP 570 H= 570:00 +CPU= 32.932
23:15:14 STEP 571 H= 571:00 +CPU= 19.871
23:15:34 STEP 572 H= 572:00 +CPU= 20.412
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:17:37 STEP 573 H= 573:00 +CPU= 31.946
23:17:58 STEP 574 H= 574:00 +CPU= 20.611
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:19:30 STEP 575 H= 575:00 +CPU= 20.630
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:21:30 STEP 576 H= 576:00 +CPU= 38.298
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018_23.zip
Uploading the intermediate file: upload_file_23.zip
23:21:50 STEP 577 H= 577:00 +CPU= 20.128
23:22:11 STEP 578 H= 578:00 +CPU= 20.566
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:24:05 STEP 579 H= 579:00 +CPU= 32.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:25:29 STEP 580 H= 580:00 +CPU= 22.081
23:25:49 STEP 581 H= 581:00 +CPU= 20.536
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:27:42 STEP 582 H= 582:00 +CPU= 32.495
23:28:02 STEP 583 H= 583:00 +CPU= 19.513
23:28:22 STEP 584 H= 584:00 +CPU= 20.488
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:30:24 STEP 585 H= 585:00 +CPU= 31.550
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:31:45 STEP 586 H= 586:00 +CPU= 20.027
23:32:04 STEP 587 H= 587:00 +CPU= 19.557
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:34:43 STEP 588 H= 588:00 +CPU= 35.758
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000588
23:35:03 STEP 589 H= 589:00 +CPU= 19.860
23:35:24 STEP 590 H= 590:00 +CPU= 20.206
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:37:17 STEP 591 H= 591:00 +CPU= 32.351
23:37:37 STEP 592 H= 592:00 +CPU= 20.424
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:39:10 STEP 593 H= 593:00 +CPU= 20.059
23:39:42 STEP 594 H= 594:00 +CPU= 31.930
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:41:22 STEP 595 H= 595:00 +CPU= 20.278
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:42:44 STEP 596 H= 596:00 +CPU= 20.761
23:43:15 STEP 597 H= 597:00 +CPU= 31.508
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:44:58 STEP 598 H= 598:00 +CPU= 21.452
23:45:19 STEP 599 H= 599:00 +CPU= 20.652
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:46:48 STEP 600 H= 600:00 +CPU= 37.606
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018_24.zip
Uploading the intermediate file: upload_file_24.zip
23:47:07 STEP 601 H= 601:00 +CPU= 19.718
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:49:19 STEP 602 H= 602:00 +CPU= 20.726
23:49:51 STEP 603 H= 603:00 +CPU= 31.875
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:51:14 STEP 604 H= 604:00 +CPU= 21.086
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:52:15 STEP 605 H= 605:00 +CPU= 19.598
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:53:59 STEP 606 H= 606:00 +CPU= 33.273
23:54:20 STEP 607 H= 607:00 +CPU= 20.594
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
23:55:52 STEP 608 H= 608:00 +CPU= 21.070
Suspend request received from the BOINC client, suspending the child process
Quit request received from the BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2016050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0293
(argv4) batchid: 985
(argv5) wuid: 12201937
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2070d7e6834a0cc8305e1783101e78ad to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0293_2016050100_123_985_12201937.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0293_2016050100_123_985_12201937.zip
ic_ancil_file: ic_ancil_12201937
ifsdata_file: ifsdata_12201937
climate_data_file: clim_data_12201937
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8e466483447d435415fd2a687b5597f6 to: /var/lib/boinc-client/slots/3/ic_ancil_12201937.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12201937.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_f6f4f299b41bde418d5a6a3eb9fa418c to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12201937.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12201937.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12201937.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12201937.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12201937.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12201937.xml
-- Model is restarting --
Adjusting last_iter, 609, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12201937.xml
last_cpu_time: 17699
upload_file_number: 25
last_iter: 599
last_upload: 2160000
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018
The child process has been launched with process id: 91901
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0 (RAND_MAX=2147483647)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:p500:1:1:91901:91901] [20230119:235931:1674140371:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:01:12 STEP 600 H= 600:00 +CPU= 36.325
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:02:34 STEP 601 H= 601:00 +CPU= 21.526
00:02:55 STEP 602 H= 602:00 +CPU= 20.055
00:03:26 STEP 603 H= 603:00 +CPU= 31.326
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:05:18 STEP 604 H= 604:00 +CPU= 20.707
00:05:37 STEP 605 H= 605:00 +CPU= 19.268
00:06:09 STEP 606 H= 606:00 +CPU= 31.733
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:08:01 STEP 607 H= 607:00 +CPU= 20.630
00:08:21 STEP 608 H= 608:00 +CPU= 20.665
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:10:05 STEP 609 H= 609:00 +CPU= 32.390
00:10:26 STEP 610 H= 610:00 +CPU= 21.091
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:11:57 STEP 611 H= 611:00 +CPU= 21.011
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:13:52 STEP 612 H= 612:00 +CPU= 34.534
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000612
00:14:14 STEP 613 H= 613:00 +CPU= 21.831
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:15:57 STEP 614 H= 614:00 +CPU= 21.134
00:16:29 STEP 615 H= 615:00 +CPU= 32.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:18:00 STEP 616 H= 616:00 +CPU= 20.354
00:18:20 STEP 617 H= 617:00 +CPU= 20.599
00:18:53 STEP 618 H= 618:00 +CPU= 32.505
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:20:35 STEP 619 H= 619:00 +CPU= 20.908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:22:08 STEP 620 H= 620:00 +CPU= 21.153
00:22:39 STEP 621 H= 621:00 +CPU= 31.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:24:22 STEP 622 H= 622:00 +CPU= 21.308
00:24:43 STEP 623 H= 623:00 +CPU= 21.004
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
00:26:51 STEP 624 H= 624:00 +CPU= 37.029
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12201937/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0293_2016050100_123_985_12201937_0_r2095794018_25.zip
Uploading the intermediate file: upload_file_25.zip
00:27:11 STEP 625 H= 625:00 +CPU= 20.195
Abort request received from BOINC client, ending the child process
</stderr_txt>
]]>
