Task 15607303

Name	hadcm3n_4l1g_1940_40_008309705_2
Workunit	8460840
Created	13 Feb 2013, 17:19:22 UTC
Sent	13 Feb 2013, 17:19:38 UTC
Report deadline	16 May 2013, 0:46:49 UTC
Received	24 Apr 2013, 4:16:07 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-226 (0xFFFFFF1E) ERR_TOO_MANY_EXITS
Computer ID	1233640
Run time	22 days 20 hours 14 min 44 sec
CPU time	18 days 1 hours 30 min 34 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	2.12 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> too many exit(0)s </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7220, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:50:40 (3976): No heartbeat from core client for 30 sec - exiting 07:50:41 (3976): No heartbeat from core client for 30 sec - exiting 07:50:42 (3976): No heartbeat from core client for 30 sec - exiting 07:50:44 (3976): No heartbeat from core client for 30 sec - exiting 07:50:45 (3976): No heartbeat from core client for 30 sec - exiting 07:50:46 (3976): No heartbeat from core client for 30 sec - exiting 07:50:47 (3976): No heartbeat from core client for 30 sec - exiting 07:50:48 (3976): No heartbeat from core client for 30 sec - exiting 07:50:49 (3976): No heartbeat from core client for 30 sec - exiting 07:50:50 (3976): No heartbeat from core client for 30 sec - exiting 07:50:51 (3976): No heartbeat from core client for 30 sec - exiting 07:50:52 (3976): No heartbeat from core client for 30 sec - exiting 07:50:53 (3976): No heartbeat from core client for 30 sec - exiting 07:50:54 (3976): No heartbeat from core client for 30 sec - exiting 07:50:56 (3976): No heartbeat from core client for 30 sec - exiting 07:50:57 (3976): No heartbeat from core client for 30 sec - exiting 07:50:58 (3976): No heartbeat from core client for 30 sec - exiting 07:50:59 (3976): No heartbeat from core client for 30 sec - exiting 07:51:00 (3976): No heartbeat from core client for 30 sec - exiting 07:51:01 (3976): No heartbeat from core client for 30 sec - exiting 07:51:02 (3976): No heartbeat from core client for 30 sec - exiting 07:51:03 (3976): No heartbeat from core client for 30 sec - exiting 07:51:04 (3976): No heartbeat from core client for 30 sec - exiting 07:51:05 (3976): No heartbeat from core client for 30 sec - exiting 07:51:07 (3976): No heartbeat from core client for 30 sec - exiting 07:51:08 (3976): No heartbeat from core client for 30 sec - exiting 07:51:09 (3976): No heartbeat from core client for 30 sec - exiting 07:51:10 (3976): No heartbeat from core client for 30 sec - exiting 07:51:11 (3976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3860, iMonCtr=1 Model crash detected, will try to restart... 23:13:35 (3420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4840, iMonCtr=1 Model crash detected, will try to restart... 07:22:13 (3424): No heartbeat from core client for 30 sec - exiting 07:22:14 (3424): No heartbeat from core client for 30 sec - exiting 07:22:15 (3424): No heartbeat from core client for 30 sec - exiting 07:22:16 (3424): No heartbeat from core client for 30 sec - exiting 07:22:17 (3424): No heartbeat from core client for 30 sec - exiting 07:22:18 (3424): No heartbeat from core client for 30 sec - exiting 07:22:19 (3424): No heartbeat from core client for 30 sec - exiting 07:22:20 (3424): No heartbeat from core client for 30 sec - exiting 07:22:21 (3424): No heartbeat from core client for 30 sec - exiting 07:22:22 (3424): No heartbeat from core client for 30 sec - exiting 07:22:23 (3424): No heartbeat from core client for 30 sec - exiting 07:22:24 (3424): No heartbeat from core client for 30 sec - exiting 07:22:25 (3424): No heartbeat from core client for 30 sec - exiting 07:22:26 (3424): No heartbeat from core client for 30 sec - exiting 07:22:27 (3424): No heartbeat from core client for 30 sec - exiting 07:22:28 (3424): No heartbeat from core client for 30 sec - exiting 07:22:29 (3424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13916, iMonCtr=1 Model crash detected, will try to restart... 06:44:32 (5832): No heartbeat from core client for 30 sec - exiting 06:44:33 (5832): No heartbeat from core client for 30 sec - exiting 06:44:34 (5832): No heartbeat from core client for 30 sec - exiting 06:44:35 (5832): No heartbeat from core client for 30 sec - exiting 06:44:37 (5832): No heartbeat from core client for 30 sec - exiting 06:44:38 (5832): No heartbeat from core client for 30 sec - exiting 06:44:39 (5832): No heartbeat from core client for 30 sec - exiting 06:44:40 (5832): No heartbeat from core client for 30 sec - exiting 06:44:41 (5832): No heartbeat from core client for 30 sec - exiting 06:44:42 (5832): No heartbeat from core client for 30 sec - exiting 06:44:43 (5832): No heartbeat from core client for 30 sec - exiting 06:44:44 (5832): No heartbeat from core client for 30 sec - exiting 06:44:45 (5832): No heartbeat from core client for 30 sec - exiting 06:44:46 (5832): No heartbeat from core client for 30 sec - exiting 06:44:47 (5832): No heartbeat from core client for 30 sec - exiting 06:44:48 (5832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:47:55 (4812): No heartbeat from core client for 30 sec - exiting 21:47:57 (4812): No heartbeat from core client for 30 sec - exiting 21:47:58 (4812): No heartbeat from core client for 30 sec - exiting 21:47:59 (4812): No heartbeat from core client for 30 sec - exiting 21:48:00 (4812): No heartbeat from core client for 30 sec - exiting 21:48:01 (4812): No heartbeat from core client for 30 sec - exiting 21:48:02 (4812): No heartbeat from core client for 30 sec - exiting 21:48:03 (4812): No heartbeat from core client for 30 sec - exiting 21:48:04 (4812): No heartbeat from core client for 30 sec - exiting 21:48:05 (4812): No heartbeat from core client for 30 sec - exiting 21:48:06 (4812): No heartbeat from core client for 30 sec - exiting 21:48:07 (4812): No heartbeat from core client for 30 sec - exiting 21:48:09 (4812): No heartbeat from core client for 30 sec - exiting 21:48:10 (4812): No heartbeat from core client for 30 sec - exiting 21:48:11 (4812): No heartbeat from core client for 30 sec - exiting 21:48:12 (4812): No heartbeat from core client for 30 sec - exiting 21:48:13 (4812): No heartbeat from core client for 30 sec - exiting 21:48:14 (4812): No heartbeat from core client for 30 sec - exiting 21:48:15 (4812): No heartbeat from core client for 30 sec - exiting 21:48:16 (4812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7648, iMonCtr=1 Model crash detected, will try to restart... 15:43:10 (5444): No heartbeat from core client for 30 sec - exiting 15:43:11 (5444): No heartbeat from core client for 30 sec - exiting 15:43:12 (5444): No heartbeat from core client for 30 sec - exiting 15:43:13 (5444): No heartbeat from core client for 30 sec - exiting 15:43:14 (5444): No heartbeat from core client for 30 sec - exiting 15:43:15 (5444): No heartbeat from core client for 30 sec - exiting 15:43:16 (5444): No heartbeat from core client for 30 sec - exiting 15:43:17 (5444): No heartbeat from core client for 30 sec - exiting 15:43:18 (5444): No heartbeat from core client for 30 sec - exiting 15:43:19 (5444): No heartbeat from core client for 30 sec - exiting 15:43:20 (5444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:38:55 (1444): No heartbeat from core client for 30 sec - exiting 22:38:57 (1444): No heartbeat from core client for 30 sec - exiting 22:38:58 (1444): No heartbeat from core client for 30 sec - exiting 22:38:59 (1444): No heartbeat from core client for 30 sec - exiting 22:39:00 (1444): No heartbeat from core client for 30 sec - exiting 22:39:01 (1444): No heartbeat from core client for 30 sec - exiting 22:39:02 (1444): No heartbeat from core client for 30 sec - exiting 22:39:03 (1444): No heartbeat from core client for 30 sec - exiting 22:39:04 (1444): No heartbeat from core client for 30 sec - exiting 22:39:05 (1444): No heartbeat from core client for 30 sec - exiting 22:39:07 (1444): No heartbeat from core client for 30 sec - exiting 22:39:08 (1444): No heartbeat from core client for 30 sec - exiting 22:39:09 (1444): No heartbeat from core client for 30 sec - exiting 22:39:10 (1444): No heartbeat from core client for 30 sec - exiting 22:39:11 (1444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:59:12 (4120): No heartbeat from core client for 30 sec - exiting 06:59:13 (4120): No heartbeat from core client for 30 sec - exiting 06:59:14 (4120): No heartbeat from core client for 30 sec - exiting 06:59:15 (4120): No heartbeat from core client for 30 sec - exiting 06:59:16 (4120): No heartbeat from core client for 30 sec - exiting 06:59:17 (4120): No heartbeat from core client for 30 sec - exiting 06:59:18 (4120): No heartbeat from core client for 30 sec - exiting 06:59:19 (4120): No heartbeat from core client for 30 sec - exiting 06:59:20 (4120): No heartbeat from core client for 30 sec - exiting 06:59:21 (4120): No heartbeat from core client for 30 sec - exiting 06:59:22 (4120): No heartbeat from core client for 30 sec - exiting 06:59:23 (4120): No heartbeat from core client for 30 sec - exiting 06:59:24 (4120): No heartbeat from core client for 30 sec - exiting 06:59:25 (4120): No heartbeat from core client for 30 sec - exiting 06:59:26 (4120): No heartbeat from core client for 30 sec - exiting 06:59:27 (4120): No heartbeat from core client for 30 sec - exiting 06:59:28 (4120): No heartbeat from core client for 30 sec - exiting 06:59:29 (4120): No heartbeat from core client for 30 sec - exiting 06:59:30 (4120): No heartbeat from core client for 30 sec - exiting 06:59:31 (4120): No heartbeat from core client for 30 sec - exiting 06:59:32 (4120): No heartbeat from core client for 30 sec - exiting 06:59:33 (4120): No heartbeat from core client for 30 sec - exiting 06:59:34 (4120): No heartbeat from core client for 30 sec - exiting 06:59:35 (4120): No heartbeat from core client for 30 sec - exiting 06:59:36 (4120): No heartbeat from core client for 30 sec - exiting 06:59:37 (4120): No heartbeat from core client for 30 sec - exiting 06:59:38 (4120): No heartbeat from core client for 30 sec - exiting 06:59:39 (4120): No heartbeat from core client for 30 sec - exiting 06:59:40 (4120): No heartbeat from core client for 30 sec - exiting 06:59:41 (4120): No heartbeat from core client for 30 sec - exiting 06:59:42 (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Apr 2013 09:14:12	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	855,360	1,540,882	1.8014
22 Apr 2013 09:34:12	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	829,440	1,493,978	1.8012
14 Apr 2013 10:32:10	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	803,520	1,447,791	1.8018
13 Apr 2013 14:25:36	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	777,600	1,401,936	1.8029
12 Apr 2013 21:03:52	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	751,680	1,355,749	1.8036
12 Apr 2013 00:56:34	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	725,760	1,309,403	1.8042
11 Apr 2013 09:19:12	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	699,840	1,263,062	1.8048
10 Apr 2013 10:18:45	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	673,920	1,216,515	1.8051
09 Apr 2013 16:31:42	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	648,000	1,169,548	1.8049
08 Apr 2013 05:56:03	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	622,080	1,123,128	1.8054
21 Mar 2013 17:56:16	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	596,160	1,076,638	1.8060
20 Mar 2013 20:39:46	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	570,240	1,030,745	1.8076
19 Mar 2013 21:43:56	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	544,320	984,340	1.8084
18 Mar 2013 00:51:36	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	518,400	937,390	1.8082
12 Mar 2013 22:27:13	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	492,480	890,413	1.8080
11 Mar 2013 23:18:41	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	466,560	843,786	1.8085
11 Mar 2013 05:21:44	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	440,640	797,081	1.8089
10 Mar 2013 13:20:07	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	414,720	750,452	1.8095
09 Mar 2013 20:16:43	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	388,800	703,510	1.8094
08 Mar 2013 23:19:52	1233640	15607303	hadcm3n_4l1g_1940_40_008309705_2	362,880	656,483	1.8091