Task 21745315

Name	hadcm3s_hz15_190012_240_835_011892513_2
Workunit	11892513
Created	25 Sep 2019, 19:36:20 UTC
Sent	25 Sep 2019, 23:51:32 UTC
Report deadline	7 Sep 2020, 5:11:32 UTC
Received	1 Nov 2019, 5:37:42 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1489844
Run time	20 days 6 hours 57 min 52 sec
CPU time	20 days 1 hours 44 min 49 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.86 GFLOPS
Application version	UK Met Office HadCM3 short v8.36 i686-pc-linux-gnu
Peak working set size	166.72 MB
Peak swap size	215.34 MB
Peak disk usage	98.97 MB
Stderr	<core_client_version>7.16.3</core_client_version> <![CDATA[ <stderr_txt> Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 10:41:49 (10233): called boinc_finish(193) Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (21 frames): ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bc177] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb9950] /lib/i386-linux-gnu/libc.so.6(getenv+0x99)[0xf7ad80c9] /lib/i386-linux-gnu/libc.so.6(+0xb538d)[0xf7b5838d] /lib/i386-linux-gnu/libc.so.6(+0xb571b)[0xf7b5871b] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7b56857] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7632] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7d80] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7e71] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb9950] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7a00] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fb9950] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fb9939] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4b)[0xf7b6861b] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7b9bcd1] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80cead5] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804fe8b] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80515bd] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80517bd] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0xf7ac1751] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804c7a1] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... MainError: 02:58:07 PM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... MainError: 02:58:07 PM No files match the supplied pattern. MainError: 04:20:08 AM No files match the supplied pattern. MainError: 04:20:08 AM No files match the supplied pattern. MainError: 09:35:54 PM No files match the supplied pattern. MainError: 09:35:54 PM No files match the supplied pattern. MainError: 09:02:16 PM No files match the supplied pattern. MainError: 09:02:17 PM No files match the supplied pattern. MainError: 11:01:37 AM No files match the supplied pattern. MainError: 11:01:37 AM No files match the supplied pattern. MainError: 08:09:18 AM No files match the supplied pattern. MainError: 08:09:18 AM No files match the supplied pattern. MainError: 04:44:15 PM No files match the supplied pattern. MainError: 04:44:15 PM No files match the supplied pattern. MainError: 03:11:58 AM No files match the supplied pattern. MainError: 03:11:58 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 04:30:46 PM No files match the supplied pattern. MainError: 04:30:46 PM No files match the supplied pattern. MainError: 06:42:54 PM No files match the supplied pattern. MainError: 06:42:54 PM No files match the supplied pattern. Processing restart Year 1920 Month 12 Day 1 MainError: 04:36:44 AM No files match the supplied pattern. MainError: 04:36:44 AM No files match the supplied pattern. 23:37:14 (1836): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_1.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_2.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_8.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz15_190012_240_835_011892513_2_r1556603350_9.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Nov 2019 04:38:12	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	518,400	1,734,273	3.3454
30 Oct 2019 19:42:18	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	492,480	1,616,262	3.2819
30 Oct 2019 10:57:20	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	466,560	1,522,431	3.2631
26 Oct 2019 04:06:31	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	440,640	1,390,524	3.1557
24 Oct 2019 16:45:39	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	414,720	1,267,289	3.0558
23 Oct 2019 08:13:43	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	388,800	1,151,274	2.9611
22 Oct 2019 11:02:32	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	362,880	1,076,507	2.9666
21 Oct 2019 21:31:28	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	336,960	1,026,493	3.0463
20 Oct 2019 22:15:20	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	311,040	942,511	3.0302
20 Oct 2019 05:11:59	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	285,120	880,752	3.0891
19 Oct 2019 14:32:16	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	267,840	829,195	3.0959
19 Oct 2019 11:31:01	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	265,680	818,400	3.0804
19 Oct 2019 08:30:04	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	263,520	807,568	3.0645
19 Oct 2019 05:29:09	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	261,360	796,875	3.0490
19 Oct 2019 02:23:01	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	259,200	786,319	3.0336
18 Oct 2019 23:21:44	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	257,040	776,878	3.0224
18 Oct 2019 21:22:54	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	254,880	768,468	3.0150
18 Oct 2019 19:18:43	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	252,720	759,966	3.0071
18 Oct 2019 16:17:48	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	250,560	751,551	2.9995
18 Oct 2019 14:13:51	1489844	21745315	hadcm3s_hz15_190012_240_835_011892513_2	248,400	743,860	2.9946