Task 13644873

Name	hadcm3n_t130_1940_40_007536028_2
Workunit	7733260
Created	18 Nov 2011, 19:05:54 UTC
Sent	18 Nov 2011, 19:13:51 UTC
Report deadline	18 Feb 2012, 2:41:02 UTC
Received	18 Dec 2011, 17:42:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1043044
Run time	17 days 18 hours 57 min 44 sec
CPU time	15 days 12 hours 42 min 41 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	2.42 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> g, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 0, checkPID=0, selfPID=0, iMonCtr=0 Model crash detected, will try to restart... 03:55:04 (5392): No heartbeat from core client for 30 sec - exiting 03:55:05 (5392): No heartbeat from core client for 30 sec - exiting 03:55:06 (5392): No heartbeat from core client for 30 sec - exiting 03:55:07 (5392): No heartbeat from core client for 30 sec - exiting 03:55:08 (5392): No heartbeat from core client for 30 sec - exiting 03:55:09 (5392): No heartbeat from core client for 30 sec - exiting 03:55:10 (5392): No heartbeat from core client for 30 sec - exiting 03:55:11 (5392): No heartbeat from core client for 30 sec - exiting 03:55:12 (5392): No heartbeat from core client for 30 sec - exiting 03:55:13 (5392): No heartbeat from core client for 30 sec - exiting 03:55:14 (5392): No heartbeat from core client for 30 sec - exiting 03:55:15 (5392): No heartbeat from core client for 30 sec - exiting 03:55:16 (5392): No heartbeat from core client for 30 sec - exiting 03:55:17 (5392): No heartbeat from core client for 30 sec - exiting 03:55:18 (5392): No heartbeat from core client for 30 sec - exiting 03:55:19 (5392): No heartbeat from core client for 30 sec - exiting 03:55:20 (5392): No heartbeat from core client for 30 sec - exiting 03:55:21 (5392): No heartbeat from core client for 30 sec - exiting 03:55:22 (5392): No heartbeat from core client for 30 sec - exiting 03:55:23 (5392): No heartbeat from core client for 30 sec - exiting 03:55:24 (5392): No heartbeat from core client for 30 sec - exiting 03:55:25 (5392): No heartbeat from core client for 30 sec - exiting 03:55:27 (5392): No heartbeat from core client for 30 sec - exiting 03:55:28 (5392): No heartbeat from core client for 30 sec - exiting 03:55:29 (5392): No heartbeat from core client for 30 sec - exiting 03:55:30 (5392): No heartbeat from core client for 30 sec - exiting 03:55:31 (5392): No heartbeat from core client for 30 sec - exiting 03:55:32 (5392): No heartbeat from core client for 30 sec - exiting 03:55:33 (5392): No heartbeat from core client for 30 sec - exiting 03:55:34 (5392): No heartbeat from core client for 30 sec - exiting 03:55:35 (5392): No heartbeat from core client for 30 sec - exiting 03:55:36 (5392): No heartbeat from core client for 30 sec - exiting 03:55:37 (5392): No heartbeat from core client for 30 sec - exiting 03:55:38 (5392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:19 (6792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:38:44 (7844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:38:45 (7844): No heartbeat from core client for 30 sec - exiting 20:38:46 (7844): No heartbeat from core client for 30 sec - exiting 20:38:47 (7844): No heartbeat from core client for 30 sec - exiting 20:38:48 (7844): No heartbeat from core client for 30 sec - exiting 20:38:49 (7844): No heartbeat from core client for 30 sec - exiting 20:38:50 (7844): No heartbeat from core client for 30 sec - exiting 03:08:47 (1532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:59:02 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:24:17 (3628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 02:11:42 (3608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:43 (3608): No heartbeat from core client for 30 sec - exiting 02:12:39 (4620): No heartbeat from core client for 30 sec - exiting 02:12:40 (4620): No heartbeat from core client for 30 sec - exiting 02:12:41 (4620): No heartbeat from core client for 30 sec - exiting 02:12:42 (4620): No heartbeat from core client for 30 sec - exiting 02:12:43 (4620): No heartbeat from core client for 30 sec - exiting 02:12:44 (4620): No heartbeat from core client for 30 sec - exiting 02:12:45 (4620): No heartbeat from core client for 30 sec - exiting 02:12:46 (4620): No heartbeat from core client for 30 sec - exiting 02:12:47 (4620): No heartbeat from core client for 30 sec - exiting 02:12:48 (4620): No heartbeat from core client for 30 sec - exiting 02:12:49 (4620): No heartbeat from core client for 30 sec - exiting 02:12:50 (4620): No heartbeat from core client for 30 sec - exiting 02:12:51 (4620): No heartbeat from core client for 30 sec - exiting 02:12:52 (4620): No heartbeat from core client for 30 sec - exiting 02:12:53 (4620): No heartbeat from core client for 30 sec - exiting 02:12:54 (4620): No heartbeat from core client for 30 sec - exiting 02:12:55 (4620): No heartbeat from core client for 30 sec - exiting 02:12:56 (4620): No heartbeat from core client for 30 sec - exiting 02:12:58 (4620): No heartbeat from core client for 30 sec - exiting 02:12:59 (4620): No heartbeat from core client for 30 sec - exiting 02:13:00 (4620): No heartbeat from core client for 30 sec - exiting 02:13:01 (4620): No heartbeat from core client for 30 sec - exiting 02:13:02 (4620): No heartbeat from core client for 30 sec - exiting 02:13:03 (4620): No heartbeat from core client for 30 sec - exiting 02:13:04 (4620): No heartbeat from core client for 30 sec - exiting 02:13:05 (4620): No heartbeat from core client for 30 sec - exiting 02:13:06 (4620): No heartbeat from core client for 30 sec - exiting 02:13:07 (4620): No heartbeat from core client for 30 sec - exiting 02:13:08 (4620): No heartbeat from core client for 30 sec - exiting 02:13:09 (4620): No heartbeat from core client for 30 sec - exiting 02:13:10 (4620): No heartbeat from core client for 30 sec - exiting 02:13:11 (4620): No heartbeat from core client for 30 sec - exiting 02:13:12 (4620): No heartbeat from core client for 30 sec - exiting 02:13:13 (4620): No heartbeat from core client for 30 sec - exiting 02:13:14 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:13:15 (4620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 16:16:03 (7248): No heartbeat from core client for 30 sec - exiting 16:16:04 (7248): No heartbeat from core client for 30 sec - exiting 16:16:05 (7248): No heartbeat from core client for 30 sec - exiting 16:16:06 (7248): No heartbeat from core client for 30 sec - exiting 16:16:07 (7248): No heartbeat from core client for 30 sec - exiting 16:16:08 (7248): No heartbeat from core client for 30 sec - exiting 16:16:09 (7248): No heartbeat from core client for 30 sec - exiting 16:16:10 (7248): No heartbeat from core client for 30 sec - exiting 16:16:11 (7248): No heartbeat from core client for 30 sec - exiting 16:16:12 (7248): No heartbeat from core client for 30 sec - exiting 16:16:13 (7248): No heartbeat from core client for 30 sec - exiting 16:16:14 (7248): No heartbeat from core client for 30 sec - exiting 16:16:15 (7248): No heartbeat from core client for 30 sec - exiting 16:16:16 (7248): No heartbeat from core client for 30 sec - exiting 16:16:17 (7248): No heartbeat from core client for 30 sec - exiting 16:16:18 (7248): No heartbeat from core client for 30 sec - exiting 16:16:19 (7248): No heartbeat from core client for 30 sec - exiting 16:16:20 (7248): No heartbeat from core client for 30 sec - exiting 16:16:21 (7248): No heartbeat from core client for 30 sec - exiting 16:16:22 (7248): No heartbeat from core client for 30 sec - exiting 16:16:23 (7248): No heartbeat from core client for 30 sec - exiting 16:16:24 (7248): No heartbeat from core client for 30 sec - exiting 16:16:25 (7248): No heartbeat from core client for 30 sec - exiting 16:16:26 (7248): No heartbeat from core client for 30 sec - exiting 16:16:27 (7248): No heartbeat from core client for 30 sec - exiting 16:16:28 (7248): No heartbeat from core client for 30 sec - exiting 16:16:29 (7248): No heartbeat from core client for 30 sec - exiting 16:16:30 (7248): No heartbeat from core client for 30 sec - exiting 16:16:31 (7248): No heartbeat from core client for 30 sec - exiting 16:16:32 (7248): No heartbeat from core client for 30 sec - exiting 16:16:33 (7248): No heartbeat from core client for 30 sec - exiting 16:16:34 (7248): No heartbeat from core client for 30 sec - exiting 16:16:35 (7248): No heartbeat from core client for 30 sec - exiting 16:16:36 (7248): No heartbeat from core client for 30 sec - exiting 16:16:37 (7248): No heartbeat from core client for 30 sec - exiting 16:16:38 (7248): No heartbeat from core client for 30 sec - exiting 16:16:39 (7248): No heartbeat from core client for 30 sec - exiting 16:16:40 (7248): No heartbeat from core client for 30 sec - exiting 16:16:41 (7248): No heartbeat from core client for 30 sec - exiting 16:16:42 (7248): No heartbeat from core client for 30 sec - exiting 16:16:43 (7248): No heartbeat from core client for 30 sec - exiting 16:16:44 (7248): No heartbeat from core client for 30 sec - exiting 16:16:45 (7248): No heartbeat from core client for 30 sec - exiting 16:16:46 (7248): No heartbeat from core client for 30 sec - exiting 16:16:47 (7248): No heartbeat from core client for 30 sec - exiting 16:16:48 (7248): No heartbeat from core client for 30 sec - exiting 16:16:49 (7248): No heartbeat from core client for 30 sec - exiting 16:16:50 (7248): No heartbeat from core client for 30 sec - exiting 16:16:51 (7248): No heartbeat from core client for 30 sec - exiting 16:16:52 (7248): No heartbeat from core client for 30 sec - exiting 16:16:53 (7248): No heartbeat from core client for 30 sec - exiting 16:16:54 (7248): No heartbeat from core client for 30 sec - exiting 16:16:55 (7248): No heartbeat from core client for 30 sec - exiting 16:16:56 (7248): No heartbeat from core client for 30 sec - exiting 16:16:57 (7248): No heartbeat from core client for 30 sec - exiting 16:16:58 (7248): No heartbeat from core client for 30 sec - exiting 16:16:59 (7248): No heartbeat from core client for 30 sec - exiting 16:17:00 (7248): No heartbeat from core client for 30 sec - exiting 16:17:01 (7248): No heartbeat from core client for 30 sec - exiting 16:17:02 (7248): No heartbeat from core client for 30 sec - exiting 16:17:03 (7248): No heartbeat from core client for 30 sec - exiting 16:17:04 (7248): No heartbeat from core client for 30 sec - exiting 16:17:05 (7248): No heartbeat from core client for 30 sec - exiting 16:17:06 (7248): No heartbeat from core client for 30 sec - exiting 16:17:07 (7248): No heartbeat from core client for 30 sec - exiting 16:17:08 (7248): No heartbeat from core client for 30 sec - exiting 16:17:09 (7248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:17:10 (7248): No heartbeat from core client for 30 sec - exiting 16:59:18 (7860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:03 (6244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:35:46 (6232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:34:23 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:36:21 (2432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:00:41 (6696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:05:39 (444): No heartbeat from core client for 30 sec - exiting 19:05:40 (444): No heartbeat from core client for 30 sec - exiting 19:05:41 (444): No heartbeat from core client for 30 sec - exiting 19:05:42 (444): No heartbeat from core client for 30 sec - exiting 19:05:43 (444): No heartbeat from core client for 30 sec - exiting 19:05:44 (444): No heartbeat from core client for 30 sec - exiting 19:05:45 (444): No heartbeat from core client for 30 sec - exiting 19:05:46 (444): No heartbeat from core client for 30 sec - exiting 19:05:47 (444): No heartbeat from core client for 30 sec - exiting 19:05:48 (444): No heartbeat from core client for 30 sec - exiting 19:05:49 (444): No heartbeat from core client for 30 sec - exiting 19:05:50 (444): No heartbeat from core client for 30 sec - exiting 19:05:51 (444): No heartbeat from core client for 30 sec - exiting 19:05:52 (444): No heartbeat from core client for 30 sec - exiting 19:05:54 (444): No heartbeat from core client for 30 sec - exiting 19:05:55 (444): No heartbeat from core client for 30 sec - exiting 19:05:56 (444): No heartbeat from core client for 30 sec - exiting 19:05:58 (444): No heartbeat from core client for 30 sec - exiting 19:05:59 (444): No heartbeat from core client for 30 sec - exiting 19:06:02 (444): No heartbeat from core client for 30 sec - exiting 19:06:04 (444): No heartbeat from core client for 30 sec - exiting 19:06:05 (444): No heartbeat from core client for 30 sec - exiting 19:06:08 (444): No heartbeat from core client for 30 sec - exiting 19:06:14 (444): No heartbeat from core client for 30 sec - exiting 19:06:15 (444): No heartbeat from core client for 30 sec - exiting 19:06:16 (444): No heartbeat from core client for 30 sec - exiting 19:06:17 (444): No heartbeat from core client for 30 sec - exiting 19:06:20 (444): No heartbeat from core client for 30 sec - exiting 19:06:21 (444): No heartbeat from core client for 30 sec - exiting 19:06:22 (444): No heartbeat from core client for 30 sec - exiting 19:06:23 (444): No heartbeat from core client for 30 sec - exiting 19:06:25 (444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:26 (444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:24:52 (7392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:59:37 (4632): No heartbeat from core client for 30 sec - exiting 19:59:38 (4632): No heartbeat from core client for 30 sec - exiting 19:59:39 (4632): No heartbeat from core client for 30 sec - exiting 19:59:40 (4632): No heartbeat from core client for 30 sec - exiting 19:59:41 (4632): No heartbeat from core client for 30 sec - exiting 19:59:42 (4632): No heartbeat from core client for 30 sec - exiting 19:59:43 (4632): No heartbeat from core client for 30 sec - exiting 19:59:44 (4632): No heartbeat from core client for 30 sec - exiting 19:59:45 (4632): No heartbeat from core client for 30 sec - exiting 19:59:46 (4632): No heartbeat from core client for 30 sec - exiting 19:59:47 (4632): No heartbeat from core client for 30 sec - exiting 19:59:48 (4632): No heartbeat from core client for 30 sec - exiting 19:59:49 (4632): No heartbeat from core client for 30 sec - exiting 19:59:50 (4632): No heartbeat from core client for 30 sec - exiting 19:59:51 (4632): No heartbeat from core client for 30 sec - exiting 19:59:52 (4632): No heartbeat from core client for 30 sec - exiting 19:59:53 (4632): No heartbeat from core client for 30 sec - exiting 19:59:54 (4632): No heartbeat from core client for 30 sec - exiting 19:59:55 (4632): No heartbeat from core client for 30 sec - exiting 19:59:56 (4632): No heartbeat from core client for 30 sec - exiting 19:59:57 (4632): No heartbeat from core client for 30 sec - exiting 19:59:58 (4632): No heartbeat from core client for 30 sec - exiting 19:59:59 (4632): No heartbeat from core client for 30 sec - exiting 20:00:00 (4632): No heartbeat from core client for 30 sec - exiting 20:00:01 (4632): No heartbeat from core client for 30 sec - exiting 20:00:02 (4632): No heartbeat from core client for 30 sec - exiting 20:00:03 (4632): No heartbeat from core client for 30 sec - exiting 20:00:04 (4632): No heartbeat from core client for 30 sec - exiting 20:00:05 (4632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:15:41 (5764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:46:48 (7904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:48:22 (5556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:21:34 (7192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:33:22 (6340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:35:15 (6760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:35:16 (6760): No heartbeat from core client for 30 sec - exiting 23:24:52 (6652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:13:26 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:49:25 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:00:49 (336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:36:33 (876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:38:37 (6380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:10:03 (6972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:42:26 (6376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 06:15:22 (8024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:46:57 (1652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:58 (6700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:59 (6700): No heartbeat from core client for 30 sec - exiting 18:48:00 (6700): No heartbeat from core client for 30 sec - exiting 18:48:01 (6700): No heartbeat from core client for 30 sec - exiting 18:48:02 (6700): No heartbeat from core client for 30 sec - exiting 18:48:03 (6700): No heartbeat from core client for 30 sec - exiting 18:48:04 (6700): No heartbeat from core client for 30 sec - exiting 19:07:17 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:38:38 (2312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:35:21 (5396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:36:52 (7420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:30:26 (6344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: C I/O Error feof - Unit 62 - Return code = 16 BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t130ko.pjg0c10 Error converting file to netcdf: dataout/t130ko.pig0c10 Error converting file to netcdf: dataout/t130ko.pfg0c10 Error converting file to netcdf: dataout/t130ko.pcg0c10 Error converting file to netcdf: dataout/t130ko.pbg0c10 Error converting file to netcdf: dataout/t130ko.pag0c10 Error converting file to netcdf: dataout/t130ka.phg0c10 Error converting file to netcdf: dataout/t130ka.pgg0c10 Error converting file to netcdf: dataout/t130ka.peg0c10 Error converting file to netcdf: dataout/t130ka.pdg0c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:01:24 (4132): No heartbeat from core client for 30 sec - exiting 13:01:25 (4132): No heartbeat from core client for 30 sec - exiting 13:01:26 (4132): No heartbeat from core client for 30 sec - exiting 13:01:27 (4132): No heartbeat from core client for 30 sec - exiting 13:01:28 (4132): No heartbeat from core client for 30 sec - exiting 13:01:29 (4132): No heartbeat from core client for 30 sec - exiting 13:01:30 (4132): No heartbeat from core client for 30 sec - exiting 13:01:31 (4132): No heartbeat from core client for 30 sec - exiting 13:01:32 (4132): No heartbeat from core client for 30 sec - exiting 13:01:33 (4132): No heartbeat from core client for 30 sec - exiting 13:01:34 (4132): No heartbeat from core client for 30 sec - exiting 13:01:35 (4132): No heartbeat from core client for 30 sec - exiting 13:01:36 (4132): No heartbeat from core client for 30 sec - exiting 13:01:37 (4132): No heartbeat from core client for 30 sec - exiting 13:01:38 (4132): No heartbeat from core client for 30 sec - exiting 13:01:39 (4132): No heartbeat from core client for 30 sec - exiting 13:01:40 (4132): No heartbeat from core client for 30 sec - exiting 13:01:41 (4132): No heartbeat from core client for 30 sec - exiting 13:01:42 (4132): No heartbeat from core client for 30 sec - exiting 13:01:43 (4132): No heartbeat from core client for 30 sec - exiting 13:01:44 (4132): No heartbeat from core client for 30 sec - exiting 13:01:45 (4132): No heartbeat from core client for 30 sec - exiting 13:01:46 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:54:48 (8620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:54:49 (8620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:33:43 (7736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:33:44 (7736): No heartbeat from core client for 30 sec - exiting 16:33:45 (7736): No heartbeat from core client for 30 sec - exiting 16:33:46 (7736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:52:56 (7444): No heartbeat from core client for 30 sec - exiting 02:52:58 (7444): No heartbeat from core client for 30 sec - exiting 02:52:59 (7444): No heartbeat from core client for 30 sec - exiting 02:53:00 (7444): No heartbeat from core client for 30 sec - exiting 02:53:01 (7444): No heartbeat from core client for 30 sec - exiting 02:53:02 (7444): No heartbeat from core client for 30 sec - exiting 02:53:03 (7444): No heartbeat from core client for 30 sec - exiting 02:53:04 (7444): No heartbeat from core client for 30 sec - exiting 02:53:05 (7444): No heartbeat from core client for 30 sec - exiting 02:53:06 (7444): No heartbeat from core client for 30 sec - exiting 02:53:07 (7444): No heartbeat from core client for 30 sec - exiting 02:53:08 (7444): No heartbeat from core client for 30 sec - exiting 02:53:09 (7444): No heartbeat from core client for 30 sec - exiting 02:53:10 (7444): No heartbeat from core client for 30 sec - exiting 02:53:11 (7444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:53:12 (7444): No heartbeat from core client for 30 sec - exiting 03:01:34 (8420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:01:35 (8420): No heartbeat from core client for 30 sec - exiting 03:01:36 (8420): No heartbeat from core client for 30 sec - exiting 03:01:37 (8420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 03:25:25 (8316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:29:25 (3504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:29:26 (3504): No heartbeat from core client for 30 sec - exiting 03:29:27 (3504): No heartbeat from core client for 30 sec - exiting 03:29:28 (3504): No heartbeat from core client for 30 sec - exiting 03:29:29 (3504): No heartbeat from core client for 30 sec - exiting 03:29:30 (3504): No heartbeat from core client for 30 sec - exiting 03:29:31 (3504): No heartbeat from core client for 30 sec - exiting 03:29:32 (3504): No heartbeat from core client for 30 sec - exiting 03:29:33 (3504): No heartbeat from core client for 30 sec - exiting 03:29:34 (3504): No heartbeat from core client for 30 sec - exiting 03:29:35 (3504): No heartbeat from core client for 30 sec - exiting 03:37:08 (8876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:41:46 (9348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:34:53 (2960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:38:34 (6372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:42:54 (3204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:42:55 (3204): No heartbeat from core client for 30 sec - exiting 03:57:27 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:57:28 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:40:55 (6832): No heartbeat from core client for 30 sec - exiting 04:40:56 (6832): No heartbeat from core client for 30 sec - exiting 04:40:57 (6832): No heartbeat from core client for 30 sec - exiting 04:40:58 (6832): No heartbeat from core client for 30 sec - exiting 04:40:59 (6832): No heartbeat from core client for 30 sec - exiting 04:41:00 (6832): No heartbeat from core client for 30 sec - exiting 04:41:01 (6832): No heartbeat from core client for 30 sec - exiting 04:41:02 (6832): No heartbeat from core client for 30 sec - exiting 04:41:03 (6832): No heartbeat from core client for 30 sec - exiting 04:41:04 (6832): No heartbeat from core client for 30 sec - exiting 04:41:05 (6832): No heartbeat from core client for 30 sec - exiting 04:41:06 (6832): No heartbeat from core client for 30 sec - exiting 04:41:07 (6832): No heartbeat from core client for 30 sec - exiting 04:41:08 (6832): No heartbeat from core client for 30 sec - exiting 04:41:09 (6832): No heartbeat from core client for 30 sec - exiting 04:41:10 (6832): No heartbeat from core client for 30 sec - exiting 04:41:11 (6832): No heartbeat from core client for 30 sec - exiting 04:41:12 (6832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:05:45 (1692): No heartbeat from core client for 30 sec - exiting 18:05:46 (1692): No heartbeat from core client for 30 sec - exiting 18:05:47 (1692): No heartbeat from core client for 30 sec - exiting 18:05:48 (1692): No heartbeat from core client for 30 sec - exiting 18:05:49 (1692): No heartbeat from core client for 30 sec - exiting 18:05:50 (1692): No heartbeat from core client for 30 sec - exiting 18:05:52 (1692): No heartbeat from core client for 30 sec - exiting 18:05:53 (1692): No heartbeat from core client for 30 sec - exiting 18:05:54 (1692): No heartbeat from core client for 30 sec - exiting 18:05:55 (1692): No heartbeat from core client for 30 sec - exiting 18:05:56 (1692): No heartbeat from core client for 30 sec - exiting 18:05:57 (1692): No heartbeat from core client for 30 sec - exiting 18:05:58 (1692): No heartbeat from core client for 30 sec - exiting 18:05:59 (1692): No heartbeat from core client for 30 sec - exiting 18:06:00 (1692): No heartbeat from core client for 30 sec - exiting 18:06:01 (1692): No heartbeat from core client for 30 sec - exiting 18:06:02 (1692): No heartbeat from core client for 30 sec - exiting 18:06:03 (1692): No heartbeat from core client for 30 sec - exiting 18:06:04 (1692): No heartbeat from core client for 30 sec - exiting 18:06:05 (1692):CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6336, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6336, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6336, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6336, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6336, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6336, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Dec 2011 12:09:36	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	648,000	1,301,467	2.0084
16 Dec 2011 14:58:27	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	622,080	1,251,050	2.0111
12 Dec 2011 21:05:10	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	596,160	1,199,637	2.0123
09 Dec 2011 02:36:10	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	570,240	1,148,576	2.0142
08 Dec 2011 11:33:55	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	544,320	1,097,594	2.0164
07 Dec 2011 16:19:27	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	518,400	1,046,887	2.0195
07 Dec 2011 00:48:04	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	492,480	997,308	2.0251
06 Dec 2011 07:59:39	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	466,560	946,071	2.0278
01 Dec 2011 07:10:16	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	440,640	896,629	2.0348
30 Nov 2011 15:13:35	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	414,720	847,570	2.0437
30 Nov 2011 00:34:03	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	388,800	798,569	2.0539
29 Nov 2011 09:11:48	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	362,880	748,767	2.0634
28 Nov 2011 18:18:49	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	336,960	698,183	2.0720
28 Nov 2011 02:53:50	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	311,040	643,761	2.0697
27 Nov 2011 03:49:18	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	285,120	588,758	2.0649
26 Nov 2011 11:55:21	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	259,200	534,514	2.0622
25 Nov 2011 19:50:35	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	233,280	479,110	2.0538
25 Nov 2011 03:14:34	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	207,360	421,599	2.0332
24 Nov 2011 09:56:54	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	181,440	364,644	2.0097
22 Nov 2011 17:11:02	1043044	13644873	hadcm3n_t130_1940_40_007536028_2	155,520	310,292	1.9952