Task 15615837

Name	hadcm3n_zh8j_1880_40_008248187_2
Workunit	8403311
Created	20 Feb 2013, 19:51:04 UTC
Sent	20 Feb 2013, 19:51:11 UTC
Report deadline	23 May 2013, 3:18:22 UTC
Received	26 May 2013, 0:46:29 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1223920
Run time	42 days 23 hours 0 min 29 sec
CPU time	31 days 4 hours 54 min 44 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	1.34 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:39:10 (107624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:26:36 (83616): No heartbeat from core client for 30 sec - exiting 05:26:37 (83616): No heartbeat from core client for 30 sec - exiting 05:26:38 (83616): No heartbeat from core client for 30 sec - exiting 05:26:39 (83616): No heartbeat from core client for 30 sec - exiting 05:26:40 (83616): No heartbeat from core client for 30 sec - exiting 05:26:41 (83616): No heartbeat from core client for 30 sec - exiting 05:26:42 (83616): No heartbeat from core client for 30 sec - exiting 05:26:43 (83616): No heartbeat from core client for 30 sec - exiting 05:26:44 (83616): No heartbeat from core client for 30 sec - exiting 05:26:45 (83616): No heartbeat from core client for 30 sec - exiting 05:26:46 (83616): No heartbeat from core client for 30 sec - exiting 05:26:47 (83616): No heartbeat from core client for 30 sec - exiting 05:26:48 (83616): No heartbeat from core client for 30 sec - exiting 05:26:49 (83616): No heartbeat from core client for 30 sec - exiting 05:26:50 (83616): No heartbeat from core client for 30 sec - exiting 05:26:51 (83616): No heartbeat from core client for 30 sec - exiting 05:26:52 (83616): No heartbeat from core client for 30 sec - exiting 05:26:53 (83616): No heartbeat from core client for 30 sec - exiting 05:26:54 (83616): No heartbeat from core client for 30 sec - exiting 05:26:55 (83616): No heartbeat from core client for 30 sec - exiting 05:26:56 (83616): No heartbeat from core client for 30 sec - exiting 05:26:57 (83616): No heartbeat from core client for 30 sec - exiting 05:26:58 (83616): No heartbeat from core client for 30 sec - exiting 05:26:59 (83616): No heartbeat from core client for 30 sec - exiting 05:27:00 (83616): No heartbeat from core client for 30 sec - exiting 05:27:01 (83616): No heartbeat from core client for 30 sec - exiting 05:27:02 (83616): No heartbeat from core client for 30 sec - exiting 05:27:03 (83616): No heartbeat from core client for 30 sec - exiting 05:27:04 (83616): No heartbeat from core client for 30 sec - exiting 05:27:05 (83616): No heartbeat from core client for 30 sec - exiting 05:27:06 (83616): No heartbeat from core client for 30 sec - exiting 05:27:07 (83616): No heartbeat from core client for 30 sec - exiting 05:27:08 (83616): No heartbeat from core client for 30 sec - exiting 05:27:09 (83616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:28:08 (69680): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:28:09 (69680): No heartbeat from core client for 30 sec - exiting 06:28:10 (69680): No heartbeat from core client for 30 sec - exiting 06:28:12 (69680): No heartbeat from core client for 30 sec - exiting 06:28:13 (69680): No heartbeat from core client for 30 sec - exiting 06:29:27 (85308): No heartbeat from core client for 30 sec - exiting 06:29:29 (85308): No heartbeat from core client for 30 sec - exiting 06:29:30 (85308): No heartbeat from core client for 30 sec - exiting 06:29:31 (85308): No heartbeat from core client for 30 sec - exiting 06:29:32 (85308): No heartbeat from core client for 30 sec - exiting 06:29:33 (85308): No heartbeat from core client for 30 sec - exiting 06:29:34 (85308): No heartbeat from core client for 30 sec - exiting 06:29:35 (85308): No heartbeat from core client for 30 sec - exiting 06:29:36 (85308): No heartbeat from core client for 30 sec - exiting 06:29:37 (85308): No heartbeat from core client for 30 sec - exiting 06:29:38 (85308): No heartbeat from core client for 30 sec - exiting 06:29:39 (85308): No heartbeat from core client for 30 sec - exiting 06:29:41 (85308): No heartbeat from core client for 30 sec - exiting 06:29:42 (85308): No heartbeat from core client for 30 sec - exiting 06:29:43 (85308): No heartbeat from core client for 30 sec - exiting 06:29:44 (85308): No heartbeat from core client for 30 sec - exiting 06:29:46 (85308): No heartbeat from core client for 30 sec - exiting 06:29:47 (85308): No heartbeat from core client for 30 sec - exiting 06:29:48 (85308): No heartbeat from core client for 30 sec - exiting 06:29:50 (85308): No heartbeat from core client for 30 sec - exiting 06:29:51 (85308): No heartbeat from core client for 30 sec - exiting 06:29:52 (85308): No heartbeat from core client for 30 sec - exiting 06:29:53 (85308): No heartbeat from core client for 30 sec - exiting 06:29:54 (85308): No heartbeat from core client for 30 sec - exiting 06:29:55 (85308): No heartbeat from core client for 30 sec - exiting 06:29:56 (85308): No heartbeat from core client for 30 sec - exiting 06:29:57 (85308): No heartbeat from core client for 30 sec - exiting 06:29:58 (85308): No heartbeat from core client for 30 sec - exiting 06:29:59 (85308): No heartbeat from core client for 30 sec - exiting 06:30:01 (85308): No heartbeat from core client for 30 sec - exiting 06:30:02 (85308): No heartbeat from core client for 30 sec - exiting 06:30:03 (85308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:08:33 (54256): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:08:38 (54256): No heartbeat from core client for 30 sec - exiting 07:08:39 (54256): No heartbeat from core client for 30 sec - exiting 07:08:40 (54256): No heartbeat from core client for 30 sec - exiting 07:08:41 (54256): No heartbeat from core client for 30 sec - exiting 07:08:42 (54256): No heartbeat from core client for 30 sec - exiting 07:08:43 (54256): No heartbeat from core client for 30 sec - exiting 07:08:44 (54256): No heartbeat from core client for 30 sec - exiting 07:08:45 (54256): No heartbeat from core client for 30 sec - exiting 07:08:46 (54256): No heartbeat from core client for 30 sec - exiting 07:08:47 (54256): No heartbeat from core client for 30 sec - exiting 07:13:16 (54940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:15:11 (41192): No heartbeat from core client for 30 sec - exiting 07:15:16 (41192): No heartbeat from core client for 30 sec - exiting 07:15:17 (41192): No heartbeat from core client for 30 sec - exiting 07:15:18 (41192): No heartbeat from core client for 30 sec - exiting 07:15:19 (41192): No heartbeat from core client for 30 sec - exiting 07:15:20 (41192): No heartbeat from core client for 30 sec - exiting 07:15:21 (41192): No heartbeat from core client for 30 sec - exiting 07:15:22 (41192): No heartbeat from core client for 30 sec - exiting 07:15:23 (41192): No heartbeat from core client for 30 sec - exiting 07:15:24 (41192): No heartbeat from core client for 30 sec - exiting 07:15:25 (41192): No heartbeat from core client for 30 sec - exiting 07:15:26 (41192): No heartbeat from core client for 30 sec - exiting 07:15:27 (41192): No heartbeat from core client for 30 sec - exiting 07:15:28 (41192): No heartbeat from core client for 30 sec - exiting 07:15:29 (41192): No heartbeat from core client for 30 sec - exiting 07:15:30 (41192): No heartbeat from core client for 30 sec - exiting 07:15:31 (41192): No heartbeat from core client for 30 sec - exiting 07:15:32 (41192): No heartbeat from core client for 30 sec - exiting 07:15:33 (41192): No heartbeat from core client for 30 sec - exiting 07:15:34 (41192): No heartbeat from core client for 30 sec - exiting 07:15:35 (41192): No heartbeat from core client for 30 sec - exiting 07:15:36 (41192): No heartbeat from core client for 30 sec - exiting 07:15:37 (41192): No heartbeat from core client for 30 sec - exiting 07:15:38 (41192): No heartbeat from core client for 30 sec - exiting 07:15:39 (41192): No heartbeat from core client for 30 sec - exiting 07:15:40 (41192): No heartbeat from core client for 30 sec - exiting 07:15:41 (41192): No heartbeat from core client for 30 sec - exiting 07:15:42 (41192): No heartbeat from core client for 30 sec - exiting 07:15:43 (41192): No heartbeat from core client for 30 sec - exiting 07:15:44 (41192): No heartbeat from core client for 30 sec - exiting 07:15:45 (41192): No heartbeat from core client for 30 sec - exiting 07:15:46 (41192): No heartbeat from core client for 30 sec - exiting 07:16:17 (41192): No heartbeat from core client for 30 sec - exiting 07:16:18 (41192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:18:06 (28992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:23:02 (102012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:55:35 (7492): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:26:01 (97144): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:38:31 (23116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:23:03 (5004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 04:32:51 (115632): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:13:51 (15204): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:13:53 (15204): No heartbeat from core client for 30 sec - exiting 10:13:54 (15204): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:15:45 (8688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:15:47 (8688): No heartbeat from core client for 30 sec - exiting 21:59:49 (8236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:10:28 (14556): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 02:10:33 (14556): No heartbeat from core client for 30 sec - exiting 02:10:34 (14556): No heartbeat from core client for 30 sec - exiting 02:10:36 (14556): No heartbeat from core client for 30 sec - exiting 02:10:37 (14556): No heartbeat from core client for 30 sec - exiting 02:10:38 (14556): No heartbeat from core client for 30 sec - exiting 02:10:39 (14556): No heartbeat from core client for 30 sec - exiting 02:10:40 (14556): No heartbeat from core client for 30 sec - exiting 02:10:41 (14556): No heartbeat from core client for 30 sec - exiting 02:10:42 (14556): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:55:37 (9528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 03:56:03 (9528): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:19:48 (18964): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:19:52 (18964): No heartbeat from core client for 30 sec - exiting 10:19:53 (18964): No heartbeat from core client for 30 sec - exiting 10:19:54 (18964): No heartbeat from core client for 30 sec - exiting 10:19:55 (18964): No heartbeat from core client for 30 sec - exiting 10:19:56 (18964): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:25:04 (8656): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:17:11 (10432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:36:51 (19000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:22:20 (9008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:04:02 (182480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:04:03 (182480): No heartbeat from core client for 30 sec - exiting 20:04:04 (182480): No heartbeat from core client for 30 sec - exiting 20:04:46 (186440): No heartbeat from core client for 30 sec - exiting 20:04:52 (186440): No heartbeat from core client for 30 sec - exiting 20:04:53 (186440): No heartbeat from core client for 30 sec - exiting 20:04:54 (186440): No heartbeat from core client for 30 sec - exiting 20:04:55 (186440): No heartbeat from core client for 30 sec - exiting 20:04:56 (186440): No heartbeat from core client for 30 sec - exiting 20:04:57 (186440): No heartbeat from core client for 30 sec - exiting 20:04:58 (186440): No heartbeat from core client for 30 sec - exiting 20:05:00 (186440): No heartbeat from core client for 30 sec - exiting 20:05:01 (186440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:05:02 (186440): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:09:54 (9964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:09:55 (9964): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 02:21:19 (10204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:21:22 (10204): No heartbeat from core client for 30 sec - exiting 02:21:23 (10204): No heartbeat from core client for 30 sec - exiting 02:27:49 (21456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:27:53 (21456): No heartbeat from core client for 30 sec - exiting 02:54:43 (23544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:54:44 (23544): No heartbeat from core client for 30 sec - exiting 03:25:23 (24548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 22:49:47 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:47:21 (7636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:47:24 (7636): No heartbeat from core client for 30 sec - exiting 23:47:25 (7636): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 11:37:49 (4948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:37:50 (4948): No heartbeat from core client for 30 sec - exiting 11:37:51 (4948): No heartbeat from core client for 30 sec - exiting 11:37:52 (4948): No heartbeat from core client for 30 sec - exiting 11:37:53 (4948): No heartbeat from core client for 30 sec - exiting 11:37:54 (4948): No heartbeat from core client for 30 sec - exiting 13:10:06 (369316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:10:09 (369316): No heartbeat from core client for 30 sec - exiting 13:10:11 (369316): No heartbeat from core client for 30 sec - exiting 13:10:12 (369316): No heartbeat from core client for 30 sec - exiting 13:10:13 (369316): No heartbeat from core client for 30 sec - exiting 13:10:14 (369316): No heartbeat from core client for 30 sec - exiting 13:19:15 (370076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:19:16 (370076): No heartbeat from core client for 30 sec - exiting 13:28:40 (369228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:28:49 (369228): No heartbeat from core client for 30 sec - exiting 13:28:51 (369228): No heartbeat from core client for 30 sec - exiting 13:28:52 (369228): No heartbeat from core client for 30 sec - exiting 13:28:53 (369228): No heartbeat from core client for 30 sec - exiting 13:28:54 (369228): No heartbeat from core client for 30 sec - exiting 13:28:55 (369228): No heartbeat from core client for 30 sec - exiting 13:28:56 (369228): No heartbeat from core client for 30 sec - exiting 13:28:57 (369228): No heartbeat from core client for 30 sec - exiting 13:28:58 (369228): No heartbeat from core client for 30 sec - exiting 13:28:59 (369228): No heartbeat from core client for 30 sec - exiting 13:29:00 (369228): No heartbeat from core client for 30 sec - exiting 13:33:45 (370396): No heartbeat from core client for 30 sec - exiting 13:33:46 (370396): No heartbeat from core client for 30 sec - exiting 13:33:47 (370396): No heartbeat from core client for 30 sec - exiting 13:33:48 (370396): No heartbeat from core client for 30 sec - exiting 13:33:49 (370396): No heartbeat from core client for 30 sec - exiting 13:33:50 (370396): No heartbeat from core client for 30 sec - exiting 13:33:51 (370396): No heartbeat from core client for 30 sec - exiting 13:33:52 (370396): No heartbeat from core client for 30 sec - exiting 13:33:53 (370396): No heartbeat from core client for 30 sec - exiting 13:33:54 (370396): No heartbeat from core client for 30 sec - exiting 13:33:55 (370396): No heartbeat from core client for 30 sec - exiting 13:33:56 (370396): No heartbeat from core client for 30 sec - exiting 13:33:57 (370396): No heartbeat from core client for 30 sec - exiting 13:33:58 (370396): No heartbeat from core client for 30 sec - exiting 13:33:59 (370396): No heartbeat from core client for 30 sec - exiting 13:34:00 (370396): No heartbeat from core client for 30 sec - exiting 13:34:01 (370396): No heartbeat from core client for 30 sec - exiting 13:34:02 (370396): No heartbeat from core client for 30 sec - exiting 13:34:03 (370396): No heartbeat from core client for 30 sec - exiting 13:34:04 (370396): No heartbeat from core client for 30 sec - exiting 13:34:05 (370396): No heartbeat from core client for 30 sec - exiting 13:34:06 (370396): No heartbeat from core client for 30 sec - exiting 13:34:07 (370396): No heartbeat from core client for 30 sec - exiting 13:34:08 (370396): No heartbeat from core client for 30 sec - exiting 13:34:09 (370396): No heartbeat from core client for 30 sec - exiting 13:34:10 (370396): No heartbeat from core client for 30 sec - exiting 13:34:11 (370396): No heartbeat from core client for 30 sec - exiting 13:34:12 (370396): No heartbeat from core client for 30 sec - exiting 13:34:13 (370396): No heartbeat from core client for 30 sec - exiting 13:34:14 (370396): No heartbeat from core client for 30 sec - exiting 13:34:15 (370396): No heartbeat from core client for 30 sec - exiting 13:34:16 (370396): No heartbeat from core client for 30 sec - exiting 13:34:17 (370396): No heartbeat from core client for 30 sec - exiting 13:34:18 (370396): No heartbeat from core client for 30 sec - exiting 13:34:19 (370396): No heartbeat from core client for 30 sec - exiting 13:34:20 (370396): No heartbeat from core client for 30 sec - exiting 13:34:21 (370396): No heartbeat from core client for 30 sec - exiting 13:34:22 (370396): No heartbeat from core client for 30 sec - exiting 13:34:23 (370396): No heartbeat from core client for 30 sec - exiting 13:34:24 (370396): No heartbeat from core client for 30 sec - exiting 13:34:25 (370396): No heartbeat from core client for 30 sec - exiting 13:34:26 (370396): No heartbeat from core client for 30 sec - exiting 13:34:27 (370396): No heartbeat from core client for 30 sec - exiting 13:34:28 (370396): No heartbeat from core client for 30 sec - exiting 13:34:29 (370396): No heartbeat from core client for 30 sec - exiting 13:34:30 (370396): No heartbeat from core client for 30 sec - exiting 13:34:31 (370396): No heartbeat from core client for 30 sec - exiting 13:34:32 (370396): No heartbeat from core client for 30 sec - exiting 13:34:33 (370396): No heartbeat from core client for 30 sec - exiting 13:34:34 (370396): No heartbeat from core client for 30 sec - exiting 13:34:35 (370396): No heartbeat from core client for 30 sec - exiting 13:34:36 (370396): No heartbeat from core client for 30 sec - exiting 13:34:37 (370396): No heartbeat from core client for 30 sec - exiting 13:34:38 (370396): No heartbeat from core client for 30 sec - exiting 13:34:39 (370396): No heartbeat from core client for 30 sec - exiting 13:34:40 (370396): No heartbeat from core client for 30 sec - exiting 13:34:41 (370396): No heartbeat from core client for 30 sec - exiting 13:34:42 (370396): No heartbeat from core client for 30 sec - exiting 13:34:43 (370396): No heartbeat from core client for 30 sec - exiting 13:34:44 (370396): No heartbeat from core client for 30 sec - exiting 13:34:45 (370396): No heartbeat from core client for 30 sec - exiting 13:34:46 (370396): No heartbeat from core client for 30 sec - exiting 13:34:47 (370396): No heartbeat from core client for 30 sec - exiting 13:34:48 (370396): No heartbeat from core client for 30 sec - exiting 13:34:49 (370396): No heartbeat from core client for 30 sec - exiting 13:34:50 (370396): No heartbeat from core client for 30 sec - exiting 13:34:51 (370396): No heartbeat from core client for 30 sec - exiting 13:34:52 (370396): No heartbeat from core client for 30 sec - exiting 13:34:53 (370396): No heartbeat from core client for 30 sec - exiting 13:34:54 (370396): No heartbeat from core client for 30 sec - exiting 13:34:55 (370396): No heartbeat from core client for 30 sec - exiting 13:34:56 (370396): No heartbeat from core client for 30 sec - exiting 13:34:57 (370396): No heartbeat from core client for 30 sec - exiting 13:34:58 (370396): No heartbeat from core client for 30 sec - exiting 13:34:59 (370396): No heartbeat from core client for 30 sec - exiting 13:35:00 (370396): No heartbeat from core client for 30 sec - exiting 13:35:01 (370396): No heartbeat from core client for 30 sec - exiting 13:35:02 (370396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:32:26 (184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:40:46 (14180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13728, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 May 2013 15:39:21	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	959,040	2,673,439	2.7876
24 May 2013 11:38:49	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	933,120	2,600,365	2.7867
23 May 2013 00:55:16	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	907,200	2,525,687	2.7840
21 May 2013 14:00:00	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	881,280	2,454,126	2.7847
20 May 2013 06:55:15	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	855,360	2,383,429	2.7865
19 May 2013 03:10:12	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	829,440	2,311,233	2.7865
18 May 2013 00:30:29	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	803,520	2,239,227	2.7868
16 May 2013 20:39:21	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	777,600	2,165,216	2.7845
15 May 2013 16:18:18	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	751,680	2,092,273	2.7835
14 May 2013 04:01:53	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	725,760	2,021,261	2.7850
08 May 2013 07:28:50	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	699,840	1,952,531	2.7900
06 May 2013 11:59:23	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	673,920	1,884,815	2.7968
05 May 2013 15:45:38	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	648,000	1,813,372	2.7984
28 Apr 2013 11:41:58	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	622,080	1,739,693	2.7966
26 Apr 2013 19:00:27	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	596,160	1,667,655	2.7973
21 Apr 2013 16:15:51	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	570,240	1,594,601	2.7964
17 Apr 2013 13:23:33	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	544,320	1,518,318	2.7894
14 Apr 2013 19:03:50	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	518,400	1,440,346	2.7784
12 Apr 2013 10:44:57	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	492,480	1,366,879	2.7755
11 Apr 2013 05:07:54	1223920	15615837	hadcm3n_zh8j_1880_40_008248187_2	466,560	1,294,661	2.7749