climateprediction.net home page
Task 22266383

Task 22266383

Name oifs_43r3_ps_0710_1995050100_123_964_12181354_0
Workunit 12181354
Created 22 Dec 2022, 9:03:52 UTC
Sent 15 Jan 2023, 4:51:15 UTC
Report deadline 14 Feb 2023, 4:51:15 UTC
Received 15 Jan 2023, 7:20:14 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1538137
Run time 46 min 44 sec
CPU time 44 min 42 sec
Validate state Invalid
Credit 1,209.60
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,253.94 MB
Peak swap size 4,974.12 MB
Peak disk usage 1,229.38 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1995050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0710
(argv4) batchid: 964
(argv5) wuid: 12181354
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_968aef63e15676e599e0d28a14120fb3 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0710_1995050100_123_964_12181354.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0710_1995050100_123_964_12181354.zip
ic_ancil_file: ic_ancil_12181354
ifsdata_file: ifsdata_12181354
climate_data_file: clim_data_12181354
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6c20fbeee0958e98de0795957fd6b497 to: /var/lib/boinc-client/slots/1/ic_ancil_12181354.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12181354.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1967d5effac51e0d7a96ba97232e1366 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12181354.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12181354.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12181354.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12181354.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12181354.xml
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12181354.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809
The child process has been launched with process id: 2131
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2131:2131] [20230114:230804:1673759284:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1995050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0710
(argv4) batchid: 964
(argv5) wuid: 12181354
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_968aef63e15676e599e0d28a14120fb3 to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0710_1995050100_123_964_12181354.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0710_1995050100_123_964_12181354.zip
ic_ancil_file: ic_ancil_12181354
ifsdata_file: ifsdata_12181354
climate_data_file: clim_data_12181354
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_6c20fbeee0958e98de0795957fd6b497 to: /var/lib/boinc-client/slots/1/ic_ancil_12181354.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12181354.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_1967d5effac51e0d7a96ba97232e1366 to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12181354.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12181354.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12181354.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12181354.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12181354.xml
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12181354.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809
The child process has been launched with process id: 2138
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:moi-desktop:1:1:2138:2138] [20230114:230907:1673759347:0.001] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  23:09:27 STEP    0 H=   0:00 +CPU= 12.455
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000000
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000000
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000000
  23:09:35 STEP    1 H=   1:00 +CPU=  7.654
  23:09:44 STEP    2 H=   2:00 +CPU=  7.133
  23:09:55 STEP    3 H=   3:00 +CPU= 11.204
  23:10:02 STEP    4 H=   4:00 +CPU=  6.661
  23:10:09 STEP    5 H=   5:00 +CPU=  6.495
  23:10:20 STEP    6 H=   6:00 +CPU= 10.972
  23:10:27 STEP    7 H=   7:00 +CPU=  6.490
  23:10:34 STEP    8 H=   8:00 +CPU=  7.257
  23:10:44 STEP    9 H=   9:00 +CPU= 10.574
  23:10:51 STEP   10 H=  10:00 +CPU=  6.583
  23:10:58 STEP   11 H=  11:00 +CPU=  6.860
  23:11:10 STEP   12 H=  12:00 +CPU= 11.767
  23:11:17 STEP   13 H=  13:00 +CPU=  6.674
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000012
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000012
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000012
  23:11:24 STEP   14 H=  14:00 +CPU=  6.700
  23:11:34 STEP   15 H=  15:00 +CPU= 10.626
  23:11:41 STEP   16 H=  16:00 +CPU=  7.107
  23:11:48 STEP   17 H=  17:00 +CPU=  6.413
  23:11:59 STEP   18 H=  18:00 +CPU= 10.952
  23:12:06 STEP   19 H=  19:00 +CPU=  6.858
  23:12:12 STEP   20 H=  20:00 +CPU=  6.646
  23:12:23 STEP   21 H=  21:00 +CPU= 10.679
  23:12:30 STEP   22 H=  22:00 +CPU=  6.954
  23:12:38 STEP   23 H=  23:00 +CPU=  7.235
  23:12:50 STEP   24 H=  24:00 +CPU= 12.575
  23:12:57 STEP   25 H=  25:00 +CPU=  6.785
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000024
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000024
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000012
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_0.zip
Uploading the intermediate file: upload_file_0.zip
  23:13:04 STEP   26 H=  26:00 +CPU=  6.756
  23:13:15 STEP   27 H=  27:00 +CPU= 11.034
  23:13:22 STEP   28 H=  28:00 +CPU=  6.728
  23:13:29 STEP   29 H=  29:00 +CPU=  6.862
  23:13:39 STEP   30 H=  30:00 +CPU= 10.642
  23:13:46 STEP   31 H=  31:00 +CPU=  6.437
  23:13:53 STEP   32 H=  32:00 +CPU=  6.909
  23:14:03 STEP   33 H=  33:00 +CPU= 10.546
  23:14:10 STEP   34 H=  34:00 +CPU=  6.865
  23:14:16 STEP   35 H=  35:00 +CPU=  6.466
  23:14:28 STEP   36 H=  36:00 +CPU= 11.756
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000036
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000036
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000036
  23:14:35 STEP   37 H=  37:00 +CPU=  6.819
  23:14:42 STEP   38 H=  38:00 +CPU=  6.564
  23:14:53 STEP   39 H=  39:00 +CPU= 10.929
  23:15:00 STEP   40 H=  40:00 +CPU=  6.962
  23:15:06 STEP   41 H=  41:00 +CPU=  6.627
  23:15:17 STEP   42 H=  42:00 +CPU= 10.618
  23:15:24 STEP   43 H=  43:00 +CPU=  6.841
  23:15:31 STEP   44 H=  44:00 +CPU=  6.758
  23:15:41 STEP   45 H=  45:00 +CPU= 10.589
  23:15:49 STEP   46 H=  46:00 +CPU=  7.083
  23:15:55 STEP   47 H=  47:00 +CPU=  6.867
  23:16:09 STEP   48 H=  48:00 +CPU= 12.762
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000048
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000048
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000036
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_1.zip
Uploading the intermediate file: upload_file_1.zip
  23:16:16 STEP   49 H=  49:00 +CPU=  6.701
  23:16:23 STEP   50 H=  50:00 +CPU=  6.648
  23:16:34 STEP   51 H=  51:00 +CPU= 10.795
  23:16:41 STEP   52 H=  52:00 +CPU=  6.600
  23:16:47 STEP   53 H=  53:00 +CPU=  6.837
  23:16:59 STEP   54 H=  54:00 +CPU= 10.924
  23:17:06 STEP   55 H=  55:00 +CPU=  6.591
  23:17:13 STEP   56 H=  56:00 +CPU=  6.754
  23:17:23 STEP   57 H=  57:00 +CPU= 10.517
  23:17:30 STEP   58 H=  58:00 +CPU=  6.788
  23:17:37 STEP   59 H=  59:00 +CPU=  6.540
  23:17:49 STEP   60 H=  60:00 +CPU= 12.178
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000060
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000060
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000060
  23:17:56 STEP   61 H=  61:00 +CPU=  6.663
  23:18:03 STEP   62 H=  62:00 +CPU=  6.732
  23:18:14 STEP   63 H=  63:00 +CPU= 10.423
  23:18:21 STEP   64 H=  64:00 +CPU=  7.045
  23:18:29 STEP   65 H=  65:00 +CPU=  6.553
  23:18:39 STEP   66 H=  66:00 +CPU= 10.677
  23:18:46 STEP   67 H=  67:00 +CPU=  6.387
  23:18:52 STEP   68 H=  68:00 +CPU=  6.720
  23:19:03 STEP   69 H=  69:00 +CPU= 10.558
  23:19:10 STEP   70 H=  70:00 +CPU=  6.876
  23:19:17 STEP   71 H=  71:00 +CPU=  6.963
  23:19:30 STEP   72 H=  72:00 +CPU= 12.804
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000072
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000072
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000060
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_2.zip
Uploading the intermediate file: upload_file_2.zip
  23:19:37 STEP   73 H=  73:00 +CPU=  6.660
  23:19:43 STEP   74 H=  74:00 +CPU=  6.773
  23:19:54 STEP   75 H=  75:00 +CPU= 10.337
  23:20:00 STEP   76 H=  76:00 +CPU=  6.642
  23:20:07 STEP   77 H=  77:00 +CPU=  6.721
  23:20:18 STEP   78 H=  78:00 +CPU= 10.858
  23:20:25 STEP   79 H=  79:00 +CPU=  6.572
  23:20:31 STEP   80 H=  80:00 +CPU=  6.714
  23:20:42 STEP   81 H=  81:00 +CPU= 10.440
  23:20:49 STEP   82 H=  82:00 +CPU=  6.983
  23:20:56 STEP   83 H=  83:00 +CPU=  6.725
  23:21:07 STEP   84 H=  84:00 +CPU= 11.608
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000084
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000084
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000084
  23:21:14 STEP   85 H=  85:00 +CPU=  6.546
  23:21:21 STEP   86 H=  86:00 +CPU=  6.782
  23:21:31 STEP   87 H=  87:00 +CPU= 10.333
  23:21:37 STEP   88 H=  88:00 +CPU=  6.542
  23:21:44 STEP   89 H=  89:00 +CPU=  6.660
  23:21:55 STEP   90 H=  90:00 +CPU= 10.694
  23:22:01 STEP   91 H=  91:00 +CPU=  6.358
  23:22:08 STEP   92 H=  92:00 +CPU=  6.789
  23:22:19 STEP   93 H=  93:00 +CPU= 10.445
  23:22:26 STEP   94 H=  94:00 +CPU=  6.995
  23:22:33 STEP   95 H=  95:00 +CPU=  6.893
  23:22:46 STEP   96 H=  96:00 +CPU= 12.242
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000096
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000096
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000084
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_3.zip
Uploading the intermediate file: upload_file_3.zip
  23:22:52 STEP   97 H=  97:00 +CPU=  6.359
  23:22:59 STEP   98 H=  98:00 +CPU=  6.586
  23:23:09 STEP   99 H=  99:00 +CPU= 10.299
  23:23:16 STEP  100 H= 100:00 +CPU=  6.530
  23:23:22 STEP  101 H= 101:00 +CPU=  6.569
  23:23:33 STEP  102 H= 102:00 +CPU= 10.659
  23:23:39 STEP  103 H= 103:00 +CPU=  6.591
  23:23:46 STEP  104 H= 104:00 +CPU=  6.913
  23:23:57 STEP  105 H= 105:00 +CPU= 10.594
  23:24:04 STEP  106 H= 106:00 +CPU=  7.099
  23:24:11 STEP  107 H= 107:00 +CPU=  6.811
  23:24:23 STEP  108 H= 108:00 +CPU= 11.553
  23:24:29 STEP  109 H= 109:00 +CPU=  6.582
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000108
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000108
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000108
  23:24:36 STEP  110 H= 110:00 +CPU=  6.985
  23:24:47 STEP  111 H= 111:00 +CPU= 10.473
  23:24:54 STEP  112 H= 112:00 +CPU=  6.828
  23:25:01 STEP  113 H= 113:00 +CPU=  6.878
  23:25:12 STEP  114 H= 114:00 +CPU= 10.816
  23:25:19 STEP  115 H= 115:00 +CPU=  6.608
  23:25:26 STEP  116 H= 116:00 +CPU=  7.020
  23:25:36 STEP  117 H= 117:00 +CPU= 10.559
  23:25:43 STEP  118 H= 118:00 +CPU=  6.844
  23:25:50 STEP  119 H= 119:00 +CPU=  6.735
  23:26:04 STEP  120 H= 120:00 +CPU= 12.736
  23:26:11 STEP  121 H= 121:00 +CPU=  6.554
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000120
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000120
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000108
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_4.zip
Uploading the intermediate file: upload_file_4.zip
  23:26:18 STEP  122 H= 122:00 +CPU=  7.237
  23:26:29 STEP  123 H= 123:00 +CPU= 10.583
  23:26:36 STEP  124 H= 124:00 +CPU=  7.080
  23:26:42 STEP  125 H= 125:00 +CPU=  6.489
  23:26:53 STEP  126 H= 126:00 +CPU= 10.875
  23:27:00 STEP  127 H= 127:00 +CPU=  6.553
  23:27:07 STEP  128 H= 128:00 +CPU=  6.977
  23:27:18 STEP  129 H= 129:00 +CPU= 10.729
  23:27:24 STEP  130 H= 130:00 +CPU=  6.626
  23:27:31 STEP  131 H= 131:00 +CPU=  6.747
  23:27:42 STEP  132 H= 132:00 +CPU= 11.440
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000132
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000132
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000132
  23:27:49 STEP  133 H= 133:00 +CPU=  6.724
  23:27:56 STEP  134 H= 134:00 +CPU=  6.694
  23:28:07 STEP  135 H= 135:00 +CPU= 10.733
  23:28:13 STEP  136 H= 136:00 +CPU=  6.682
  23:28:20 STEP  137 H= 137:00 +CPU=  6.550
  23:28:31 STEP  138 H= 138:00 +CPU= 11.147
  23:28:38 STEP  139 H= 139:00 +CPU=  6.551
  23:28:45 STEP  140 H= 140:00 +CPU=  7.033
  23:28:55 STEP  141 H= 141:00 +CPU= 10.806
  23:29:02 STEP  142 H= 142:00 +CPU=  7.067
  23:29:10 STEP  143 H= 143:00 +CPU=  6.998
  23:29:23 STEP  144 H= 144:00 +CPU= 12.544
  23:29:30 STEP  145 H= 145:00 +CPU=  6.515
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000144
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000144
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000132
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_5.zip
Uploading the intermediate file: upload_file_5.zip
  23:29:37 STEP  146 H= 146:00 +CPU=  6.979
  23:29:48 STEP  147 H= 147:00 +CPU= 10.650
  23:29:54 STEP  148 H= 148:00 +CPU=  6.616
  23:30:01 STEP  149 H= 149:00 +CPU=  6.467
  23:30:12 STEP  150 H= 150:00 +CPU= 10.717
  23:30:18 STEP  151 H= 151:00 +CPU=  6.621
  23:30:25 STEP  152 H= 152:00 +CPU=  6.638
  23:30:36 STEP  153 H= 153:00 +CPU= 10.431
  23:30:42 STEP  154 H= 154:00 +CPU=  6.696
  23:30:49 STEP  155 H= 155:00 +CPU=  6.682
  23:31:01 STEP  156 H= 156:00 +CPU= 11.333
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000156
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000156
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000156
  23:31:07 STEP  157 H= 157:00 +CPU=  6.669
  23:31:14 STEP  158 H= 158:00 +CPU=  6.554
  23:31:24 STEP  159 H= 159:00 +CPU= 10.459
  23:31:31 STEP  160 H= 160:00 +CPU=  6.626
  23:31:37 STEP  161 H= 161:00 +CPU=  6.606
  23:31:48 STEP  162 H= 162:00 +CPU= 10.612
  23:31:55 STEP  163 H= 163:00 +CPU=  6.612
  23:32:01 STEP  164 H= 164:00 +CPU=  6.657
  23:32:12 STEP  165 H= 165:00 +CPU= 10.336
  23:32:19 STEP  166 H= 166:00 +CPU=  6.984
  23:32:25 STEP  167 H= 167:00 +CPU=  6.636
  23:32:39 STEP  168 H= 168:00 +CPU= 12.536
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000168
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000168
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000156
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_6.zip
Uploading the intermediate file: upload_file_6.zip
  23:32:46 STEP  169 H= 169:00 +CPU=  6.649
  23:32:52 STEP  170 H= 170:00 +CPU=  6.824
  23:33:03 STEP  171 H= 171:00 +CPU= 10.217
  23:33:10 STEP  172 H= 172:00 +CPU=  7.001
  23:33:16 STEP  173 H= 173:00 +CPU=  6.318
  23:33:26 STEP  174 H= 174:00 +CPU= 10.575
  23:33:33 STEP  175 H= 175:00 +CPU=  6.320
  23:33:40 STEP  176 H= 176:00 +CPU=  6.854
  23:33:50 STEP  177 H= 177:00 +CPU= 10.098
  23:33:56 STEP  178 H= 178:00 +CPU=  6.594
  23:34:03 STEP  179 H= 179:00 +CPU=  6.257
  23:34:15 STEP  180 H= 180:00 +CPU= 11.860
  23:34:21 STEP  181 H= 181:00 +CPU=  6.422
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000180
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000180
  23:34:28 STEP  182 H= 182:00 +CPU=  6.591
  23:34:38 STEP  183 H= 183:00 +CPU= 10.660
  23:34:45 STEP  184 H= 184:00 +CPU=  6.675
  23:34:52 STEP  185 H= 185:00 +CPU=  6.628
  23:35:02 STEP  186 H= 186:00 +CPU= 10.505
  23:35:09 STEP  187 H= 187:00 +CPU=  6.693
  23:35:15 STEP  188 H= 188:00 +CPU=  6.568
  23:35:26 STEP  189 H= 189:00 +CPU= 10.645
  23:35:33 STEP  190 H= 190:00 +CPU=  6.969
  23:35:40 STEP  191 H= 191:00 +CPU=  6.771
  23:35:53 STEP  192 H= 192:00 +CPU= 12.562
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000192
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000192
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000180
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_7.zip
Uploading the intermediate file: upload_file_7.zip
  23:36:00 STEP  193 H= 193:00 +CPU=  6.596
  23:36:07 STEP  194 H= 194:00 +CPU=  6.689
  23:36:17 STEP  195 H= 195:00 +CPU= 10.697
  23:36:24 STEP  196 H= 196:00 +CPU=  6.993
  23:36:31 STEP  197 H= 197:00 +CPU=  6.533
  23:36:42 STEP  198 H= 198:00 +CPU= 11.096
  23:36:49 STEP  199 H= 199:00 +CPU=  6.615
  23:36:56 STEP  200 H= 200:00 +CPU=  6.802
  23:37:06 STEP  201 H= 201:00 +CPU= 10.489
  23:37:13 STEP  202 H= 202:00 +CPU=  6.834
  23:37:19 STEP  203 H= 203:00 +CPU=  6.372
  23:37:31 STEP  204 H= 204:00 +CPU= 11.593
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000204
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000204
  23:37:38 STEP  205 H= 205:00 +CPU=  6.878
  23:37:45 STEP  206 H= 206:00 +CPU=  6.643
  23:37:56 STEP  207 H= 207:00 +CPU= 10.432
  23:38:02 STEP  208 H= 208:00 +CPU=  6.750
  23:38:09 STEP  209 H= 209:00 +CPU=  6.967
  23:38:20 STEP  210 H= 210:00 +CPU= 10.843
  23:38:27 STEP  211 H= 211:00 +CPU=  6.721
  23:38:34 STEP  212 H= 212:00 +CPU=  6.644
  23:38:44 STEP  213 H= 213:00 +CPU= 10.795
  23:38:51 STEP  214 H= 214:00 +CPU=  6.834
  23:38:58 STEP  215 H= 215:00 +CPU=  6.596
  23:39:13 STEP  216 H= 216:00 +CPU= 13.003
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000216
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000204
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000204
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_8.zip
Uploading the intermediate file: upload_file_8.zip
  23:39:20 STEP  217 H= 217:00 +CPU=  6.567
  23:39:27 STEP  218 H= 218:00 +CPU=  6.970
  23:39:37 STEP  219 H= 219:00 +CPU= 10.420
  23:39:44 STEP  220 H= 220:00 +CPU=  6.711
  23:39:51 STEP  221 H= 221:00 +CPU=  6.571
  23:40:01 STEP  222 H= 222:00 +CPU= 10.850
  23:40:08 STEP  223 H= 223:00 +CPU=  6.435
  23:40:15 STEP  224 H= 224:00 +CPU=  7.009
  23:40:26 STEP  225 H= 225:00 +CPU= 10.370
  23:40:33 STEP  226 H= 226:00 +CPU=  6.808
  23:40:39 STEP  227 H= 227:00 +CPU=  6.657
  23:40:52 STEP  228 H= 228:00 +CPU= 11.999
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000228
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000228
  23:40:58 STEP  229 H= 229:00 +CPU=  6.441
  23:41:06 STEP  230 H= 230:00 +CPU=  7.066
  23:41:17 STEP  231 H= 231:00 +CPU= 10.809
  23:41:23 STEP  232 H= 232:00 +CPU=  6.933
  23:41:30 STEP  233 H= 233:00 +CPU=  6.638
  23:41:42 STEP  234 H= 234:00 +CPU= 10.959
  23:41:48 STEP  235 H= 235:00 +CPU=  6.548
  23:41:55 STEP  236 H= 236:00 +CPU=  6.836
  23:42:06 STEP  237 H= 237:00 +CPU= 10.725
  23:42:13 STEP  238 H= 238:00 +CPU=  6.965
  23:42:20 STEP  239 H= 239:00 +CPU=  6.955
  23:42:34 STEP  240 H= 240:00 +CPU= 12.914
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000240
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000216
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000228
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000228
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_9.zip
Uploading the intermediate file: upload_file_9.zip
  23:42:41 STEP  241 H= 241:00 +CPU=  6.682
  23:42:48 STEP  242 H= 242:00 +CPU=  7.113
Uploading trickle at timestep: 864000
  23:42:59 STEP  243 H= 243:00 +CPU= 10.594
  23:43:06 STEP  244 H= 244:00 +CPU=  7.340
  23:43:13 STEP  245 H= 245:00 +CPU=  6.736
  23:43:27 STEP  246 H= 246:00 +CPU= 11.713
  23:43:33 STEP  247 H= 247:00 +CPU=  6.492
  23:43:40 STEP  248 H= 248:00 +CPU=  6.909
  23:43:51 STEP  249 H= 249:00 +CPU= 10.797
  23:43:58 STEP  250 H= 250:00 +CPU=  6.809
  23:44:05 STEP  251 H= 251:00 +CPU=  6.753
  23:44:16 STEP  252 H= 252:00 +CPU= 11.863
  23:44:23 STEP  253 H= 253:00 +CPU=  6.922
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000252
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000252
  23:44:30 STEP  254 H= 254:00 +CPU=  6.776
  23:44:41 STEP  255 H= 255:00 +CPU= 11.027
  23:44:48 STEP  256 H= 256:00 +CPU=  6.808
  23:44:55 STEP  257 H= 257:00 +CPU=  6.887
  23:45:06 STEP  258 H= 258:00 +CPU= 10.993
  23:45:13 STEP  259 H= 259:00 +CPU=  6.817
  23:45:20 STEP  260 H= 260:00 +CPU=  7.038
  23:45:31 STEP  261 H= 261:00 +CPU= 10.909
  23:45:37 STEP  262 H= 262:00 +CPU=  6.762
  23:45:44 STEP  263 H= 263:00 +CPU=  6.966
  23:45:58 STEP  264 H= 264:00 +CPU= 12.766
  23:46:04 STEP  265 H= 265:00 +CPU=  6.615
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000264
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000240
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000252
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000252
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_10.zip
Uploading the intermediate file: upload_file_10.zip
  23:46:11 STEP  266 H= 266:00 +CPU=  6.634
  23:46:22 STEP  267 H= 267:00 +CPU= 10.781
  23:47:30 STEP  268 H= 268:00 +CPU=  8.603
  23:47:36 STEP  269 H= 269:00 +CPU=  6.565
  23:47:47 STEP  270 H= 270:00 +CPU= 10.663
  23:47:53 STEP  271 H= 271:00 +CPU=  6.465
  23:48:00 STEP  272 H= 272:00 +CPU=  6.819
  23:48:11 STEP  273 H= 273:00 +CPU= 10.414
  23:48:18 STEP  274 H= 274:00 +CPU=  6.823
  23:48:24 STEP  275 H= 275:00 +CPU=  6.778
  23:48:39 STEP  276 H= 276:00 +CPU= 12.064
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000276
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000276
  23:48:45 STEP  277 H= 277:00 +CPU=  6.496
  23:48:53 STEP  278 H= 278:00 +CPU=  7.470
  23:49:04 STEP  279 H= 279:00 +CPU= 10.759
  23:49:11 STEP  280 H= 280:00 +CPU=  7.023
  23:49:18 STEP  281 H= 281:00 +CPU=  6.969
  23:49:29 STEP  282 H= 282:00 +CPU= 11.364
  23:49:36 STEP  283 H= 283:00 +CPU=  6.541
  23:49:43 STEP  284 H= 284:00 +CPU=  6.954
  23:49:54 STEP  285 H= 285:00 +CPU= 11.228
  23:50:01 STEP  286 H= 286:00 +CPU=  6.972
  23:50:08 STEP  287 H= 287:00 +CPU=  6.950
  23:50:22 STEP  288 H= 288:00 +CPU= 12.557
  23:50:28 STEP  289 H= 289:00 +CPU=  6.673
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000288
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000264
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000276
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000276
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_11.zip
Uploading the intermediate file: upload_file_11.zip
  23:50:36 STEP  290 H= 290:00 +CPU=  7.322
  23:50:47 STEP  291 H= 291:00 +CPU= 11.182
  23:50:54 STEP  292 H= 292:00 +CPU=  7.019
  23:51:01 STEP  293 H= 293:00 +CPU=  6.866
  23:51:13 STEP  294 H= 294:00 +CPU= 11.114
  23:51:20 STEP  295 H= 295:00 +CPU=  6.804
  23:51:27 STEP  296 H= 296:00 +CPU=  7.111
  23:51:38 STEP  297 H= 297:00 +CPU= 10.657
  23:51:45 STEP  298 H= 298:00 +CPU=  6.847
  23:51:51 STEP  299 H= 299:00 +CPU=  6.899
  23:52:04 STEP  300 H= 300:00 +CPU= 11.887
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000300
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000300
  23:52:10 STEP  301 H= 301:00 +CPU=  6.795
  23:52:17 STEP  302 H= 302:00 +CPU=  6.904
  23:52:28 STEP  303 H= 303:00 +CPU= 10.911
  23:52:35 STEP  304 H= 304:00 +CPU=  6.989
  23:52:42 STEP  305 H= 305:00 +CPU=  6.891
  23:52:53 STEP  306 H= 306:00 +CPU= 10.884
  23:53:00 STEP  307 H= 307:00 +CPU=  6.700
  23:53:07 STEP  308 H= 308:00 +CPU=  6.888
  23:53:17 STEP  309 H= 309:00 +CPU= 10.575
  23:53:25 STEP  310 H= 310:00 +CPU=  6.829
  23:53:32 STEP  311 H= 311:00 +CPU=  6.789
  23:53:46 STEP  312 H= 312:00 +CPU= 12.761
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000312
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000312
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000288
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMGGhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMSHhq0f+000300
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12181354/ICMUAhq0f+000300
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0710_1995050100_123_964_12181354_0_r1562766809_12.zip
Uploading the intermediate file: upload_file_12.zip
  23:53:53 STEP  313 H= 313:00 +CPU=  6.862
  23:54:00 STEP  314 H= 314:00 +CPU=  7.116
  23:54:11 STEP  315 H= 315:00 +CPU= 10.712
  23:54:19 STEP  316 H= 316:00 +CPU=  7.178
  23:54:26 STEP  317 H= 317:00 +CPU=  7.071
  23:54:37 STEP  318 H= 318:00 +CPU= 10.495
  23:54:43 STEP  319 H= 319:00 +CPU=  6.604
  23:54:50 STEP  320 H= 320:00 +CPU=  6.540
  23:55:01 STEP  321 H= 321:00 +CPU= 10.868
  23:55:08 STEP  322 H= 322:00 +CPU=  6.763
  23:55:14 STEP  323 H= 323:00 +CPU=  6.606
  23:55:26 STEP  324 H= 324:00 +CPU= 11.731
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+000324
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+000324
  23:55:39 STEP  325 H= 325:00 +CPU=  7.167
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:20:22 1538137 22266383 oifs_43r3_ps_0710_1995050100_123_964_12181354_0 864,000 1,973 0.0023


©2024 climateprediction.net