Task 13370443

Name	hadcm3n_t4a6_1940_40_007451943_0
Workunit	7649446
Created	10 Sep 2011, 12:17:22 UTC
Sent	11 Sep 2011, 19:07:10 UTC
Report deadline	12 Dec 2011, 2:34:21 UTC
Received	15 Nov 2011, 17:33:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1045266
Run time	18 days 9 hours 47 min 12 sec
CPU time	17 days 18 hours 48 min 22 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.67 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4132, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10376, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=824, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4168, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3080, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4128, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4860, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4860, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4860, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4036, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2656, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1276, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3804, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4104, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2928, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2928, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2928, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3696, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3468, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4176, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3192, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3192, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3236, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3236, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3596, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3596, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1192, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1272, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4120, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4120, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2988, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2988, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2988, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2988, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4060, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2860, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4168, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3832, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:26:33 (2180): No heartbeat from core client for 30 sec - exiting 18:26:34 (2180): No heartbeat from core client for 30 sec - exiting 18:26:35 (2180): No heartbeat from core client for 30 sec - exiting 18:26:36 (2180): No heartbeat from core client for 30 sec - exiting 18:26:37 (2180): No heartbeat from core client for 30 sec - exiting 18:26:39 (2180): No heartbeat from core client for 30 sec - exiting 18:26:40 (2180): No heartbeat from core client for 30 sec - exiting 18:26:41 (2180): No heartbeat from core client for 30 sec - exiting 18:26:42 (2180): No heartbeat from core client for 30 sec - exiting 18:26:43 (2180): No heartbeat from core client for 30 sec - exiting 18:26:44 (2180): No heartbeat from core client for 30 sec - exiting 18:26:45 (2180): No heartbeat from core client for 30 sec - exiting 18:26:46 (2180): No heartbeat from core client for 30 sec - exiting 18:26:47 (2180): No heartbeat from core client for 30 sec - exiting 18:26:48 (2180): No heartbeat from core client for 30 sec - exiting 18:26:49 (2180): No heartbeat from core client for 30 sec - exiting 18:26:51 (2180): No heartbeat from core client for 30 sec - exiting 18:26:52 (2180): No heartbeat from core client for 30 sec - exiting 18:26:53 (2180): No heartbeat from core client for 30 sec - exiting 18:26:54 (2180): No heartbeat from core client for 30 sec - exiting 18:26:55 (2180): No heartbeat from core client for 30 sec - exiting 18:26:56 (2180): No heartbeat from core client for 30 sec - exiting 18:26:57 (2180): No heartbeat from core client for 30 sec - exiting 18:26:58 (2180): No heartbeat from core client for 30 sec - exiting 18:26:59 (2180): No heartbeat from core client for 30 sec - exiting 18:27:00 (2180): No heartbeat from core client for 30 sec - exiting 18:27:01 (2180): No heartbeat from core client for 30 sec - exiting 18:27:03 (2180): No heartbeat from core client for 30 sec - exiting 18:27:04 (2180): No heartbeat from core client for 30 sec - exiting 18:27:05 (2180): No heartbeat from core client for 30 sec - exiting 18:27:06 (2180): No heartbeat from core client for 30 sec - exiting 18:27:07 (2180): No heartbeat from core client for 30 sec - exiting 18:27:08 (2180): No heartbeat from core client for 30 sec - exiting 18:27:09 (2180): No heartbeat from core client for 30 sec - exiting 18:27:10 (2180): No heartbeat from core client for 30 sec - exiting 18:27:11 (2180): No heartbeat from core client for 30 sec - exiting 18:27:12 (2180): No heartbeat from core client for 30 sec - exiting 18:27:13 (2180): No heartbeat from core client for 30 sec - exiting 18:27:15 (2180): No heartbeat from core client for 30 sec - exiting 18:27:16 (2180): No heartbeat from core client for 30 sec - exiting 18:27:17 (2180): No heartbeat from core client for 30 sec - exiting 18:27:18 (2180): No heartbeat from core client for 30 sec - exiting 18:27:19 (2180): No heartbeat from core client for 30 sec - exiting 18:27:20 (2180): No heartbeat from core client for 30 sec - exiting 18:27:21 (2180): No heartbeat from core client for 30 sec - exiting 18:27:22 (2180): No heartbeat from core client for 30 sec - exiting 18:27:23 (2180): No heartbeat from core client for 30 sec - exiting 18:27:24 (2180): No heartbeat from core client for 30 sec - exiting 18:27:26 (2180): No heartbeat from core client for 30 sec - exiting 18:27:27 (2180): No heartbeat from core client for 30 sec - exiting 18:27:28 (2180): No heartbeat from core client for 30 sec - exiting 18:27:29 (2180): No heartbeat from core client for 30 sec - exiting 18:27:30 (2180): No heartbeat from core client for 30 sec - exiting 18:27:31 (2180): No heartbeat from core client for 30 sec - exiting 18:27:32 (2180): No heartbeat from core client for 30 sec - exiting 18:27:33 (2180): No heartbeat from core client for 30 sec - exiting 18:27:34 (2180): No heartbeat from core client for 30 sec - exiting 18:27:35 (2180): No heartbeat from core client for 30 sec - exiting 18:27:36 (2180): No heartbeat from core client for 30 sec - exiting 18:27:38 (2180): No heartbeat from core client for 30 sec - exiting 18:27:39 (2180): No heartbeat from core client for 30 sec - exiting 18:27:40 (2180): No heartbeat from core client for 30 sec - exiting 18:27:41 (2180): No heartbeat from core client for 30 sec - exiting 18:27:42 (2180): No heartbeat from core client for 30 sec - exiting 18:27:43 (2180): No heartbeat from core client for 30 sec - exiting 18:27:44 (2180): No heartbeat from core client for 30 sec - exiting 18:27:45 (2180): No heartbeat from core client for 30 sec - exiting 18:27:46 (2180): No heartbeat from core client for 30 sec - exiting 18:27:47 (2180): No heartbeat from core client for 30 sec - exiting 18:27:48 (2180): No heartbeat from core client for 30 sec - exiting 18:27:50 (2180): No heartbeat from core client for 30 sec - exiting 18:27:51 (2180): No heartbeat from core client for 30 sec - exiting 18:27:52 (2180): No heartbeat from core client for 30 sec - exiting 18:27:53 (2180): No heartbeat from core client for 30 sec - exiting 18:27:54 (2180): No heartbeat from core client for 30 sec - exiting 18:27:55 (2180): No heartbeat from core client for 30 sec - exiting 18:27:56 (2180): No heartbeat from core client for 30 sec - exiting 18:27:57 (2180): No heartbeat from core client for 30 sec - exiting 18:27:58 (2180): No heartbeat from core client for 30 sec - exiting 18:27:59 (2180): No heartbeat from core client for 30 sec - exiting 18:28:00 (2180): No heartbeat from core client for 30 sec - exiting 18:28:02 (2180): No heartbeat from core client for 30 sec - exiting 18:28:03 (2180): No heartbeat from core client for 30 sec - exiting 18:28:04 (2180): No heartbeat from core client for 30 sec - exiting 18:28:05 (2180): No heartbeat from core client for 30 sec - exiting 18:28:06 (2180): No heartbeat from core client for 30 sec - exiting 18:28:07 (2180): No heartbeat from core client for 30 sec - exiting 18:28:08 (2180): No heartbeat from core client for 30 sec - exiting 18:28:09 (2180): No heartbeat from core client for 30 sec - exiting 18:28:10 (2180): No heartbeat from core client for 30 sec - exiting 18:28:11 (2180): No heartbeat from core client for 30 sec - exiting 18:28:12 (2180): No heartbeat from core client for 30 sec - exiting 18:28:14 (2180): No heartbeat from core client for 30 sec - exiting 18:28:15 (2180): No heartbeat from core client for 30 sec - exiting 18:28:16 (2180): No heartbeat from core client for 30 sec - exiting 18:28:17 (2180): No heartbeat from core client for 30 sec - exiting 18:28:18 (2180): No heartbeat from core client for 30 sec - exiting 18:28:19 (2180): No heartbeat from core client for 30 sec - exiting 18:28:20 (2180): No heartbeat from core client for 30 sec - exiting 18:28:21 (2180): No heartbeat from core client for 30 sec - exiting 18:28:22 (2180): No heartbeat from core client for 30 sec - exiting 18:28:23 (2180): No heartbeat from core client for 30 sec - exiting 18:28:24 (2180): No heartbeat from core client for 30 sec - exiting 18:28:26 (2180): No heartbeat from core client for 30 sec - exiting 18:28:27 (2180): No heartbeat from core client for 30 sec - exiting 18:28:28 (2180): No heartbeat from core client for 30 sec - exiting 18:28:29 (2180): No heartbeat from core client for 30 sec - exiting 18:28:30 (2180): No heartbeat from core client for 30 sec - exiting 18:28:31 (2180): No heartbeat from core client for 30 sec - exiting 18:28:32 (2180): No heartbeat from core client for 30 sec - exiting 18:28:33 (2180): No heartbeat from core client for 30 sec - exiting 18:28:34 (2180): No heartbeat from core client for 30 sec - exiting 18:28:35 (2180): No heartbeat from core client for 30 sec - exiting 18:28:36 (2180): No heartbeat from core client for 30 sec - exiting 18:28:38 (2180): No heartbeat from core client for 30 sec - exiting 18:28:39 (2180): No heartbeat from core client for 30 sec - exiting 18:28:40 (2180): No heartbeat from core client for 30 sec - exiting 18:28:41 (2180): No heartbeat from core client for 30 sec - exiting 18:28:42 (2180): No heartbeat from core client for 30 sec - exiting 18:28:43 (2180): No heartbeat from core client for 30 sec - exiting 18:28:44 (2180): No heartbeat from core client for 30 sec - exiting 18:28:45 (2180): No heartbeat from core client for 30 sec - exiting 18:28:46 (2180): No heartbeat from core client for 30 sec - exiting 18:28:47 (2180): No heartbeat from core client for 30 sec - exiting 18:28:48 (2180): No heartbeat from core client for 30 sec - exiting 18:28:50 (2180): No heartbeat from core client for 30 sec - exiting 18:28:51 (2180): No heartbeat from core client for 30 sec - exiting 18:28:52 (2180): No heartbeat from core client for 30 sec - exiting 18:28:53 (2180): No heartbeat from core client for 30 sec - exiting 18:28:54 (2180): No heartbeat from core client for 30 sec - exiting 18:28:55 (2180): No heartbeat from core client for 30 sec - exiting 18:28:56 (2180): No heartbeat from core client for 30 sec - exiting 18:28:57 (2180): No heartbeat from core client for 30 sec - exiting 18:28:58 (2180): No heartbeat from core client for 30 sec - exiting 18:28:59 (2180): No heartbeat from core client for 30 sec - exiting 18:29:01 (2180): No heartbeat from core client for 30 sec - exiting 18:29:02 (2180): No heartbeat from core client for 30 sec - exiting 18:29:03 (2180): No heartbeat from core client for 30 sec - exiting 18:29:04 (2180): No heartbeat from core client for 30 sec - exiting 18:29:05 (2180): No heartbeat from core client for 30 sec - exiting 18:29:06 (2180): No heartbeat from core client for 30 sec - exiting 18:29:07 (2180): No heartbeat from core client for 30 sec - exiting 18:29:08 (2180): No heartbeat from core client for 30 sec - exiting 18:29:09 (2180): No heartbeat from core client for 30 sec - exiting 18:29:10 (2180): No heartbeat from core client for 30 sec - exiting 18:29:11 (2180): No heartbeat from core client for 30 sec - exiting 18:29:13 (2180): No heartbeat from core client for 30 sec - exiting 18:29:14 (2180): No heartbeat from core client for 30 sec - exiting 18:29:15 (2180): No heartbeat from core client for 30 sec - exiting 18:29:16 (2180): No heartbeat from core client for 30 sec - exiting 18:29:17 (2180): No heartbeat from core client for 30 sec - exiting 18:29:18 (2180): No heartbeat from core client for 30 sec - exiting 18:29:19 (2180): No heartbeat from core client for 30 sec - exiting 18:29:20 (2180): No heartbeat from core client for 30 sec - exiting 18:29:21 (2180): No heartbeat from core client for 30 sec - exiting 18:29:22 (2180): No heartbeat from core client for 30 sec - exiting 18:29:23 (2180): No heartbeat from core client for 30 sec - exiting 18:29:25 (2180): No heartbeat from core client for 30 sec - exiting 18:29:26 (2180): No heartbeat from core client for 30 sec - exiting 18:29:27 (2180): No heartbeat from core client for 30 sec - exiting 18:29:28 (2180): No heartbeat from core client for 30 sec - exiting 18:29:29 (2180): No heartbeat from core client for 30 sec - exiting 18:29:30 (2180): No heartbeat from core client for 30 sec - exiting 18:29:31 (2180): No heartbeat from core client for 30 sec - exiting 18:29:32 (2180): No heartbeat from core client for 30 sec - exiting 18:29:33 (2180): No heartbeat from core client for 30 sec - exiting 18:29:34 (2180): No heartbeat from core client for 30 sec - exiting 18:29:35 (2180): No heartbeat from core client for 30 sec - exiting 18:29:37 (2180): No heartbeat from core client for 30 sec - exiting 18:29:38 (2180): No heartbeat from core client for 30 sec - exiting 18:29:39 (2180): No heartbeat from core client for 30 sec - exiting 18:29:40 (2180): No heartbeat from core client for 30 sec - exiting 18:29:41 (2180): No heartbeat from core client for 30 sec - exiting 18:29:42 (2180): No heartbeat from core client for 30 sec - exiting 18:29:43 (2180): No heartbeat from core client for 30 sec - exiting 18:29:44 (2180): No heartbeat from core client for 30 sec - exiting 18:29:45 (2180): No heartbeat from core client for 30 sec - exiting 18:29:46 (2180): No heartbeat from core client for 30 sec - exiting 18:29:47 (2180): No heartbeat from core client for 30 sec - exiting 18:29:49 (2180): No heartbeat from core client for 30 sec - exiting 18:29:50 (2180): No heartbeat from core client for 30 sec - exiting 18:29:51 (2180): No heartbeat from core client for 30 sec - exiting 18:29:52 (2180): No heartbeat from core client for 30 sec - exiting 18:29:53 (2180): No heartbeat from core client for 30 sec - exiting 18:29:54 (2180): No heartbeat from core client for 30 sec - exiting 18:29:55 (2180): No heartbeat from core client for 30 sec - exiting 18:29:56 (2180): No heartbeat from core client for 30 sec - exiting 18:29:57 (2180): No heartbeat from core client for 30 sec - exiting 18:29:58 (2180): No heartbeat from core client for 30 sec - exiting 18:29:59 (2180): No heartbeat from core client for 30 sec - exiting 18:30:01 (2180): No heartbeat from core client for 30 sec - exiting 18:30:02 (2180): No heartbeat from core client for 30 sec - exiting 18:30:03 (2180): No heartbeat from core client for 30 sec - exiting 18:30:04 (2180): No heartbeat from core client for 30 sec - exiting 18:30:05 (2180): No heartbeat from core client for 30 sec - exiting 18:30:06 (2180): No heartbeat from core client for 30 sec - exiting 18:30:07 (2180): No heartbeat from core client for 30 sec - exiting 18:30:08 (2180): No heartbeat from core client for 30 sec - exiting 18:30:09 (2180): No heartbeat from core client for 30 sec - exiting 18:30:10 (2180): No heartbeat from core client for 30 sec - exiting 18:30:11 (2180): No heartbeat from core client for 30 sec - exiting 18:30:13 (2180): No heartbeat from core client for 30 sec - exiting 18:30:14 (2180): No heartbeat from core client for 30 sec - exiting 18:30:15 (2180): No heartbeat from core client for 30 sec - exiting 18:30:16 (2180): No heartbeat from core client for 30 sec - exiting 18:30:17 (2180): No heartbeat from core client for 30 sec - exiting 18:30:18 (2180): No heartbeat from core client for 30 sec - exiting 18:30:19 (2180): No heartbeat from core client for 30 sec - exiting 18:30:20 (2180): No heartbeat from core client for 30 sec - exiting 18:30:21 (2180): No heartbeat from core client for 30 sec - exiting 18:30:22 (2180): No heartbeat from core client for 30 sec - exiting 18:30:23 (2180): No heartbeat from core client for 30 sec - exiting 18:30:25 (2180): No heartbeat from core client for 30 sec - exiting 18:30:26 (2180): No heartbeat from core client for 30 sec - exiting 18:30:27 (2180): No heartbeat from core client for 30 sec - exiting 18:30:28 (2180): No heartbeat from core client for 30 sec - exiting 18:30:29 (2180): No heartbeat from core client for 30 sec - exiting 18:30:30 (2180): No heartbeat from core client for 30 sec - exiting 18:30:31 (2180): No heartbeat from core client for 30 sec - exiting 18:30:32 (2180): No heartbeat from core client for 30 sec - exiting 18:30:33 (2180): No heartbeat from core client for 30 sec - exiting 18:30:34 (2180): No heartbeat from core client for 30 sec - exiting 18:30:36 (2180): No heartbeat from core client for 30 sec - exiting 18:30:37 (2180): No heartbeat from core client for 30 sec - exiting 18:30:38 (2180): No heartbeat from core client for 30 sec - exiting 18:30:39 (2180): No heartbeat from core client for 30 sec - exiting 18:30:40 (2180): No heartbeat from core client for 30 sec - exiting 18:30:41 (2180): No heartbeat from core client for 30 sec - exiting 18:30:42 (2180): No heartbeat from core client for 30 sec - exiting 18:30:43 (2180): No heartbeat from core client for 30 sec - exiting 18:30:44 (2180): No heartbeat from core client for 30 sec - exiting 18:30:45 (2180): No heartbeat from core client for 30 sec - exiting 18:30:46 (2180): No heartbeat from core client for 30 sec - exiting 18:30:48 (2180): No heartbeat from core client for 30 sec - exiting 18:30:49 (2180): No heartbeat from core client for 30 sec - exiting 18:30:50 (2180): No heartbeat from core client for 30 sec - exiting 18:30:51 (2180): No heartbeat from core client for 30 sec - exiting 18:30:52 (2180): No heartbeat from core client for 30 sec - exiting 18:30:53 (2180): No heartbeat from core client for 30 sec - exiting 18:30:54 (2180): No heartbeat from core client for 30 sec - exiting 18:30:55 (2180): No heartbeat from core client for 30 sec - exiting 18:30:56 (2180): No heartbeat from core client for 30 sec - exiting 18:30:57 (2180): No heartbeat from core client for 30 sec - exiting 18:30:58 (2180): No heartbeat from core client for 30 sec - exiting 18:31:00 (2180): No heartbeat from core client for 30 sec - exiting 18:31:01 (2180): No heartbeat from core client for 30 sec - exiting 18:31:02 (2180): No heartbeat from core client for 30 sec - exiting 18:31:03 (2180): No heartbeat from core client for 30 sec - exiting 18:31:04 (2180): No heartbeat from core client for 30 sec - exiting 18:31:05 (2180): No heartbeat from core client for 30 sec - exiting 18:31:06 (2180): No heartbeat from core client for 30 sec - exiting 18:31:07 (2180): No heartbeat from core client for 30 sec - exiting 18:31:08 (2180): No heartbeat from core client for 30 sec - exiting 18:31:09 (2180): No heartbeat from core client for 30 sec - exiting 18:31:10 (2180): No heartbeat from core client for 30 sec - exiting 18:31:12 (2180): No heartbeat from core client for 30 sec - exiting 18:31:13 (2180): No heartbeat from core client for 30 sec - exiting 18:31:14 (2180): No heartbeat from core client for 30 sec - exiting 18:31:15 (2180): No heartbeat from core client for 30 sec - exiting 18:31:16 (2180): No heartbeat from core client for 30 sec - exiting 18:31:17 (2180): No heartbeat from core client for 30 sec - exiting 18:31:18 (2180): No heartbeat from core client for 30 sec - exiting 18:31:19 (2180): No heartbeat from core client for 30 sec - exiting 18:31:20 (2180): No heartbeat from core client for 30 sec - exiting 18:31:21 (2180): No heartbeat from core client for 30 sec - exiting 18:31:22 (2180): No heartbeat from core client for 30 sec - exiting 18:31:24 (2180): No heartbeat from core client for 30 sec - exiting 18:31:25 (2180): No heartbeat from core client for 30 sec - exiting 18:31:26 (2180): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4276, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4132, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Nov 2011 18:07:10	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	777,600	1,536,496	1.9759
15 Nov 2011 18:07:10	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	751,680	1,483,954	1.9742
15 Nov 2011 18:07:09	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	725,760	1,430,572	1.9711
08 Nov 2011 12:28:17	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	699,840	1,380,534	1.9726
06 Nov 2011 12:24:16	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	673,920	1,328,771	1.9717
05 Nov 2011 21:49:20	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	648,000	1,275,767	1.9688
03 Nov 2011 20:06:16	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	622,080	1,223,224	1.9663
01 Nov 2011 19:52:35	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	596,160	1,168,738	1.9604
31 Oct 2011 19:41:56	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	570,240	1,115,542	1.9563
31 Oct 2011 19:06:49	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	544,320	1,062,683	1.9523
31 Oct 2011 17:15:24	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	518,400	1,010,951	1.9501
31 Oct 2011 14:05:38	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	492,480	959,393	1.9481
31 Oct 2011 14:05:38	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	466,560	909,406	1.9492
31 Oct 2011 14:05:38	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	440,640	858,000	1.9472
31 Oct 2011 14:05:38	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	414,720	805,509	1.9423
16 Oct 2011 13:55:57	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	388,800	752,616	1.9357
15 Oct 2011 11:45:22	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	362,880	700,874	1.9314
13 Oct 2011 18:09:37	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	336,960	650,962	1.9319
13 Oct 2011 05:56:14	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	311,040	609,757	1.9604
10 Oct 2011 18:13:41	1045266	13370443	hadcm3n_t4a6_1940_40_007451943_0	285,120	563,343	1.9758