Task 21744241

Name	hadcm3s_hz35_190012_240_835_011892533_0
Workunit	11892533
Created	25 Sep 2019, 10:41:10 UTC
Sent	25 Sep 2019, 21:53:11 UTC
Report deadline	7 Sep 2020, 3:13:11 UTC
Received	5 Nov 2019, 13:03:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	0 (0x00000000)
Computer ID	1489851
Run time	23 days 9 hours 20 min 58 sec
CPU time	23 days 4 hours 15 min 2 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.85 GFLOPS
Application version	UK Met Office HadCM3 short v8.36 i686-pc-linux-gnu
Peak working set size	166.75 MB
Peak swap size	215.61 MB
Peak disk usage	4.52 MB
Stderr	<core_client_version>7.16.3</core_client_version> <![CDATA[ <stderr_txt> error: cannot delete old /var/lib/boinc-client/projects/climateprediction.net/hadcm3s_se_8.36_i686-pc-linux-gnu.so No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... 06:06:40 (28545): called boinc_finish(193) Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=28545, selfPID=28545, iMonCtr=1 Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3s_se_8.36_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Signal 15 received: Software termination signal from kill Signal 15 received: Abnormal termination triggered by abort call Signal 15 received, exiting... SIGSEGV: segmentation violation Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80bc177] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f09950] /lib/i386-linux-gnu/libc.so.6(getenv+0x99)[0xf7a280c9] /lib/i386-linux-gnu/libc.so.6(+0xb538d)[0xf7aa838d] /lib/i386-linux-gnu/libc.so.6(+0xb571b)[0xf7aa871b] /lib/i386-linux-gnu/libc.so.6(localtime_r+0x17)[0xf7aa6857] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7632] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7d80] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80b7e71] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f09950] linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f09939] /lib/i386-linux-gnu/libc.so.6(nanosleep+0x4b)[0xf7ab861b] /lib/i386-linux-gnu/libc.so.6(usleep+0x41)[0xf7aebcd1] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80cead5] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804fe8b] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80515bd] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x80517bd] /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0xf7a11751] ../../projects/climateprediction.net/hadcm3s_8.36_i686-pc-linux-gnu[0x804c7a1] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... MainError: 03:00:32 PM No files match the supplied pattern. Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... MainError: 03:00:32 PM No files match the supplied pattern. MainError: 11:14:29 PM No files match the supplied pattern. MainError: 11:14:29 PM No files match the supplied pattern. MainError: 01:18:37 AM No files match the supplied pattern. MainError: 01:18:37 AM No files match the supplied pattern. MainError: 12:04:16 AM No files match the supplied pattern. MainError: 12:04:16 AM No files match the supplied pattern. MainError: 06:29:41 AM No files match the supplied pattern. MainError: 06:29:41 AM No files match the supplied pattern. MainError: 05:22:58 PM No files match the supplied pattern. MainError: 05:22:58 PM No files match the supplied pattern. MainError: 07:33:38 AM No files match the supplied pattern. MainError: 07:33:38 AM No files match the supplied pattern. MainError: 07:20:43 PM No files match the supplied pattern. MainError: 07:20:43 PM No files match the supplied pattern. MainError: 06:32:00 AM No files match the supplied pattern. MainError: 06:32:00 AM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 08:53:57 PM No files match the supplied pattern. MainError: 08:53:57 PM No files match the supplied pattern. Suspended CPDN Monitor - Suspend request from BOINC... MainError: 10:19:43 PM No files match the supplied pattern. MainError: 10:19:43 PM No files match the supplied pattern. MainError: 04:09:04 PM No files match the supplied pattern. MainError: 04:09:04 PM No files match the supplied pattern. Processing restart Year 1920 Month 12 Day 1 MainError: 12:17:10 AM No files match the supplied pattern. MainError: 12:17:10 AM No files match the supplied pattern. 06:17:36 (1843): called boinc_finish(0) </stderr_txt> <message> upload failure: <file_xfer_error> <file_name>hadcm3s_hz35_190012_240_835_011892533_0_r1468866189_1.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz35_190012_240_835_011892533_0_r1468866189_2.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz35_190012_240_835_011892533_0_r1468866189_3.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz35_190012_240_835_011892533_0_r1468866189_4.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz35_190012_240_835_011892533_0_r1468866189_5.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz35_190012_240_835_011892533_0_r1468866189_6.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> <file_xfer_error> <file_name>hadcm3s_hz35_190012_240_835_011892533_0_r1468866189_7.zip</file_name> <error_code>-161 (not found)</error_code> </file_xfer_error> </message> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Nov 2019 13:08:15	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	518,400	2,002,490	3.8628
04 Nov 2019 16:09:28	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	492,480	1,931,993	3.9230
03 Nov 2019 22:24:20	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	466,560	1,869,221	4.0064
01 Nov 2019 21:08:58	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	440,640	1,769,258	4.0152
30 Oct 2019 10:56:47	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	414,720	1,649,407	3.9772
27 Oct 2019 19:23:08	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	388,800	1,523,229	3.9178
26 Oct 2019 07:38:24	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	362,880	1,395,789	3.8464
24 Oct 2019 17:27:00	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	336,960	1,258,893	3.7360
23 Oct 2019 06:30:26	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	311,040	1,133,846	3.6453
22 Oct 2019 00:06:29	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	285,120	1,025,670	3.5973
21 Oct 2019 01:21:17	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	259,200	946,238	3.6506
19 Oct 2019 23:51:39	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	233,280	853,828	3.6601
19 Oct 2019 14:16:46	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	224,640	819,798	3.6494
19 Oct 2019 11:15:30	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	222,480	808,701	3.6349
19 Oct 2019 08:14:35	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	220,320	797,649	3.6204
19 Oct 2019 05:13:40	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	218,160	786,458	3.6050
19 Oct 2019 02:12:43	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	216,000	778,410	3.6038
19 Oct 2019 01:10:29	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	213,840	772,731	3.6136
18 Oct 2019 23:06:13	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	211,680	767,117	3.6239
18 Oct 2019 22:09:20	1489851	21744241	hadcm3s_hz35_190012_240_835_011892533_0	209,520	761,522	3.6346