Task 14847580

Name	hadcm3n_zioe_1880_40_008026197_1
Workunit	8181311
Created	29 Jun 2012, 15:17:55 UTC
Sent	29 Jun 2012, 15:17:56 UTC
Report deadline	28 Sep 2012, 22:45:07 UTC
Received	2 Jul 2012, 18:28:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1060946
Run time	1 days 21 hours 14 min 52 sec
CPU time	1 days 15 hours 24 min 8 sec
Validate state	Invalid
Credit	933.12
Device peak FLOPS	3.25 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> 08:26:57 (496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:28:29 (530): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 08:53:31 (552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:43 (2479): No heartbeat from core client for 30 sec - exiting 08:55:44 (2479): No heartbeat from core client for 30 sec - exiting 08:55:45 (2479): No heartbeat from core client for 30 sec - exiting 08:55:46 (2479): No heartbeat from core client for 30 sec - exiting 08:55:47 (2479): No heartbeat from core client for 30 sec - exiting 08:55:48 (2479): No heartbeat from core client for 30 sec - exiting 08:55:49 (2479): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:02:30 (2490): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:07:27 (2555): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 11:32:12 (5326): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:35:06 (5489): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:18:10 (5495): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:59:11 (7959): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:02:38 (9932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:43:17 (11674): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:45:01 (13592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:26:56 (13598): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:28:29 (15560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:00:21 (15566): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:22:27 (17864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:15:13 (19778): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:16:23 (21689): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:17:20 (21709): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:58:28 (23364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:58:30 (23364): No heartbeat from core client for 30 sec - exiting 18:58:31 (23364): No heartbeat from core client for 30 sec - exiting 18:58:32 (23364): No heartbeat from core client for 30 sec - exiting 18:58:33 (23364): No heartbeat from core client for 30 sec - exiting 18:58:34 (23364): No heartbeat from core client for 30 sec - exiting 18:58:35 (23364): No heartbeat from core client for 30 sec - exiting 18:58:36 (23364): No heartbeat from core client for 30 sec - exiting 19:03:49 (23687): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:06:54 (255): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:16:56 (572): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:16:57 (572): No heartbeat from core client for 30 sec - exiting 09:16:58 (572): No heartbeat from core client for 30 sec - exiting 09:16:59 (572): No heartbeat from core client for 30 sec - exiting 09:17:00 (572): No heartbeat from core client for 30 sec - exiting 09:17:01 (572): No heartbeat from core client for 30 sec - exiting 09:17:02 (572): No heartbeat from core client for 30 sec - exiting 09:17:03 (572): No heartbeat from core client for 30 sec - exiting 09:17:04 (572): No heartbeat from core client for 30 sec - exiting 09:17:05 (572): No heartbeat from core client for 30 sec - exiting 09:17:06 (572): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 09:26:57 (632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:26:59 (632): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:36:41 (763): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:39:20 (832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:42:23 (853): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:42:27 (853): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 09:44:13 (871): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:25 (890): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:57:29 (890): No heartbeat from core client for 30 sec - exiting 09:57:30 (890): No heartbeat from core client for 30 sec - exiting 09:58:51 (1012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:59:50 (1017): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:59:51 (1017): No heartbeat from core client for 30 sec - exiting 09:59:52 (1017): No heartbeat from core client for 30 sec - exiting 09:59:53 (1017): No heartbeat from core client for 30 sec - exiting 09:59:54 (1017): No heartbeat from core client for 30 sec - exiting 09:59:55 (1017): No heartbeat from core client for 30 sec - exiting 09:59:56 (1017): No heartbeat from core client for 30 sec - exiting 09:59:57 (1017): No heartbeat from core client for 30 sec - exiting 09:59:58 (1017): No heartbeat from core client for 30 sec - exiting 09:59:59 (1017): No heartbeat from core client for 30 sec - exiting 10:00:00 (1017): No heartbeat from core client for 30 sec - exiting 10:00:01 (1017): No heartbeat from core client for 30 sec - exiting 10:00:02 (1017): No heartbeat from core client for 30 sec - exiting 10:00:03 (1017): No heartbeat from core client for 30 sec - exiting 10:00:04 (1017): No heartbeat from core client for 30 sec - exiting 10:00:05 (1017): No heartbeat from core client for 30 sec - exiting 10:00:06 (1017): No heartbeat from core client for 30 sec - exiting 10:00:07 (1017): No heartbeat from core client for 30 sec - exiting 10:00:08 (1017): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:12:27 (1030): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:12:28 (1030): No heartbeat from core client for 30 sec - exiting 10:12:29 (1030): No heartbeat from core client for 30 sec - exiting 10:12:30 (1030): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:56:23 (274): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:56:25 (274): No heartbeat from core client for 30 sec - exiting 12:08:16 (411): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:35 (2686): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:31:39 (2949): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:36:41 (3415): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:36:42 (3415): No heartbeat from core client for 30 sec - exiting 13:36:43 (3415): No heartbeat from core client for 30 sec - exiting 13:36:44 (3415): No heartbeat from core client for 30 sec - exiting 13:36:45 (3415): No heartbeat from core client for 30 sec - exiting 13:36:46 (3415): No heartbeat from core client for 30 sec - exiting 13:36:47 (3415): No heartbeat from core client for 30 sec - exiting 13:36:48 (3415): No heartbeat from core client for 30 sec - exiting 13:36:49 (3415): No heartbeat from core client for 30 sec - exiting 13:36:50 (3415): No heartbeat from core client for 30 sec - exiting 13:36:51 (3415): No heartbeat from core client for 30 sec - exiting 13:46:40 (3454): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:46:41 (3454): No heartbeat from core client for 30 sec - exiting 13:46:42 (3454): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:14:38 (231): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:14:41 (231): No heartbeat from core client for 30 sec - exiting 23:14:42 (231): No heartbeat from core client for 30 sec - exiting 23:54:47 (473): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:08 (2482): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:56:09 (2482): No heartbeat from core client for 30 sec - exiting 23:56:10 (2482): No heartbeat from core client for 30 sec - exiting 23:56:11 (2482): No heartbeat from core client for 30 sec - exiting 23:56:12 (2482): No heartbeat from core client for 30 sec - exiting 23:56:13 (2482): No heartbeat from core client for 30 sec - exiting 23:56:14 (2482): No heartbeat from core client for 30 sec - exiting 23:56:15 (2482): No heartbeat from core client for 30 sec - exiting 23:56:16 (2482): No heartbeat from core client for 30 sec - exiting 23:56:17 (2482): No heartbeat from core client for 30 sec - exiting 01:00:43 (2508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:13 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:12:03 (7355): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:31:13 (9246): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:46:43 (11658): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:25:26 (14036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:04:36 (16046): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:27:52 (18604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:30:56 (20986): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:29:26 (22689): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:42:49 (25387): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:56:25 (27704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:27:30 (30038): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:22:37 (32499): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:25:01 (35721): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:13:08 (37408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:35:10 (39503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:37:12 (42445): No heartbeat from core client for 30 sec - exiting 21:37:13 (42445): No heartbeat from core client for 30 sec - exiting 21:37:14 (42445): No heartbeat from core client for 30 sec - exiting 21:37:15 (42445): No heartbeat from core client for 30 sec - exiting 21:37:16 (42445): No heartbeat from core client for 30 sec - exiting 21:37:17 (42445): No heartbeat from core client for 30 sec - exiting 21:37:18 (42445): No heartbeat from core client for 30 sec - exiting 21:37:19 (42445): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:52:41 (42452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:52:42 (42452): No heartbeat from core client for 30 sec - exiting 22:52:43 (42452): No heartbeat from core client for 30 sec - exiting 22:52:44 (42452): No heartbeat from core client for 30 sec - exiting 00:27:54 (44744): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:43:44 (47252): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:43:45 (47252): No heartbeat from core client for 30 sec - exiting 01:43:46 (47252): No heartbeat from core client for 30 sec - exiting 01:47:13 (49602): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:11:34 (51292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:58:09 (53706): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:02:39 (55851): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:22:16 (58086): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:35:11 (59929): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 07:28:34 (62226): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:28:02 (250): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:36:09 (396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:38:43 (471): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:38:44 (471): No heartbeat from core client for 30 sec - exiting 08:38:45 (471): No heartbeat from core client for 30 sec - exiting 08:38:47 (471): No heartbeat from core client for 30 sec - exiting 08:38:48 (471): No heartbeat from core client for 30 sec - exiting 08:38:49 (471): No heartbeat from core client for 30 sec - exiting 08:38:50 (471): No heartbeat from core client for 30 sec - exiting 08:38:51 (471): No heartbeat from core client for 30 sec - exiting 08:38:52 (471): No heartbeat from core client for 30 sec - exiting 08:38:53 (471): No heartbeat from core client for 30 sec - exiting 08:38:54 (471): No heartbeat from core client for 30 sec - exiting 08:38:55 (471): No heartbeat from core client for 30 sec - exiting 08:38:56 (471): No heartbeat from core client for 30 sec - exiting 08:43:07 (506): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:43:10 (506): No heartbeat from core client for 30 sec - exiting 08:43:11 (506): No heartbeat from core client for 30 sec - exiting 08:43:12 (506): No heartbeat from core client for 30 sec - exiting 08:43:13 (506): No heartbeat from core client for 30 sec - exiting 08:43:14 (506): No heartbeat from core client for 30 sec - exiting 08:43:15 (506): No heartbeat from core client for 30 sec - exiting 08:43:16 (506): No heartbeat from core client for 30 sec - exiting 08:43:17 (506): No heartbeat from core client for 30 sec - exiting 08:43:18 (506): No heartbeat from core client for 30 sec - exiting 08:43:19 (506): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 08:44:49 (539): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:08 (556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:10 (556): No heartbeat from core client for 30 sec - exiting 08:48:11 (556): No heartbeat from core client for 30 sec - exiting 08:48:12 (556): No heartbeat from core client for 30 sec - exiting 08:48:13 (556): No heartbeat from core client for 30 sec - exiting 08:48:14 (556): No heartbeat from core client for 30 sec - exiting 08:48:15 (556): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:57:22 (585): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:42 (661): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:46 (661): No heartbeat from core client for 30 sec - exiting 09:07:30 (699): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:07:31 (699): No heartbeat from core client for 30 sec - exiting 09:07:32 (699): No heartbeat from core client for 30 sec - exiting 09:08:27 (731): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:14:07 (745): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:15:04 (783): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:19:18 (786): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:24:28 (823): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:27:38 (860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:27:43 (860): No heartbeat from core client for 30 sec - exiting 09:27:44 (860): No heartbeat from core client for 30 sec - exiting 09:27:45 (860): No heartbeat from core client for 30 sec - exiting 09:34:24 (892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:39:27 (941): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:28 (978): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:30 (1012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:31 (1012): No heartbeat from core client for 30 sec - exiting 09:52:48 (1069): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:51 (1069): No heartbeat from core client for 30 sec - exiting 09:52:52 (1069): No heartbeat from core client for 30 sec - exiting 09:52:53 (1069): No heartbeat from core client for 30 sec - exiting 09:52:54 (1069): No heartbeat from core client for 30 sec - exiting 09:56:49 (1096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:34:41 (1144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:36:21 (1515): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:49:43 (1546): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:54:46 (1652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:59:43 (1692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:04:44 (1733): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:04:45 (1733): No heartbeat from core client for 30 sec - exiting 11:04:46 (1733): No heartbeat from core client for 30 sec - exiting 11:04:47 (1733): No heartbeat from core client for 30 sec - exiting 11:04:48 (1733): No heartbeat from core client for 30 sec - exiting 11:04:49 (1733): No heartbeat from core client for 30 sec - exiting 11:09:44 (1793): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:47 (1846): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:16:49 (3553): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:19:56 (6885): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Mon Jul 2 11:21:07 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9041f000 - 0x90422fff /usr/lib/system/libcompiler_rt.dylib 0x90773000 - 0x90777fff /usr/lib/system/libsystem_network.dylib 0x91574000 - 0x9157cfff /usr/lib/system/libcopyfile.dylib 0x9157d000 - 0x9157efff /usr/lib/system/libsystem_sandbox.dylib 0x91ffb000 - 0x92003fff /usr/lib/system/libunwind.dylib 0x92583000 - 0x9258afff /usr/lib/system/libsystem_dnssd.dylib 0x9260d000 - 0x92611fff /usr/lib/system/libcache.dylib 0x92cd5000 - 0x92cebfff /usr/lib/system/libxpc.dylib 0x93c9c000 - 0x93c9dfff /usr/lib/system/libremovefile.dylib 0x93d72000 - 0x93d72fff /usr/lib/system/libkeymgr.dylib 0x93fa7000 - 0x93fa9fff /usr/lib/system/libdyld.dylib 0x93ffc000 - 0x94004fff /usr/lib/system/liblaunch.dylib 0x94008000 - 0x94009fff /usr/lib/system/libunc.dylib 0x962f6000 - 0x96339fff /usr/lib/system/libcommonCrypto.dylib 0x96392000 - 0x963a0fff /usr/lib/libz.1.dylib 0x97b0d000 - 0x97b10fff /usr/lib/system/libmathCommon.A.dylib 0x988c1000 - 0x988c2fff /usr/lib/system/libsystem_blocks.dylib 0x988d9000 - 0x988d9fff /usr/lib/system/libdnsinfo.dylib 0x98f73000 - 0x98f7afff /usr/lib/system/libsystem_notify.dylib 0x996b0000 - 0x996b5fff /usr/lib/system/libmacho.dylib 0x999f2000 - 0x99a20fff /usr/lib/libSystem.B.dylib 0x9a086000 - 0x9a087fff /usr/lib/system/libquarantine.dylib 0x9aa51000 - 0x9aa6ffff /usr/lib/system/libsystem_kernel.dylib 0x9ac47000 - 0x9ac76fff /usr/lib/system/libsystem_info.dylib 0x9b8d2000 - 0x9b99dfff /usr/lib/system/libsystem_c.dylib 0x9b9e3000 - 0x9b9f1fff /usr/lib/system/libdispatch.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8564, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Mon Jul 2 11:21:10 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9041f000 - 0x90422fff /usr/lib/system/libcompiler_rt.dylib 0x90773000 - 0x90777fff /usr/lib/system/libsystem_network.dylib 0x91574000 - 0x9157cfff /usr/lib/system/libcopyfile.dylib 0x9157d000 - 0x9157efff /usr/lib/system/libsystem_sandbox.dylib 0x91ffb000 - 0x92003fff /usr/lib/system/libunwind.dylib 0x92583000 - 0x9258afff /usr/lib/system/libsystem_dnssd.dylib 0x9260d000 - 0x92611fff /usr/lib/system/libcache.dylib 0x92cd5000 - 0x92cebfff /usr/lib/system/libxpc.dylib 0x93c9c000 - 0x93c9dfff /usr/lib/system/libremovefile.dylib 0x93d72000 - 0x93d72fff /usr/lib/system/libkeymgr.dylib 0x93fa7000 - 0x93fa9fff /usr/lib/system/libdyld.dylib 0x93ffc000 - 0x94004fff /usr/lib/system/liblaunch.dylib 0x94008000 - 0x94009fff /usr/lib/system/libunc.dylib 0x962f6000 - 0x96339fff /usr/lib/system/libcommonCrypto.dylib 0x96392000 - 0x963a0fff /usr/lib/libz.1.dylib 0x97b0d000 - 0x97b10fff /usr/lib/system/libmathCommon.A.dylib 0x988c1000 - 0x988c2fff /usr/lib/system/libsystem_blocks.dylib 0x988d9000 - 0x988d9fff /usr/lib/system/libdnsinfo.dylib 0x98f73000 - 0x98f7afff /usr/lib/system/libsystem_notify.dylib 0x996b0000 - 0x996b5fff /usr/lib/system/libmacho.dylib 0x999f2000 - 0x99a20fff /usr/lib/libSystem.B.dylib 0x9a086000 - 0x9a087fff /usr/lib/system/libquarantine.dylib 0x9aa51000 - 0x9aa6ffff /usr/lib/system/libsystem_kernel.dylib 0x9ac47000 - 0x9ac76fff /usr/lib/system/libsystem_info.dylib 0x9b8d2000 - 0x9b99dfff /usr/lib/system/libsystem_c.dylib 0x9b9e3000 - 0x9b9f1fff /usr/lib/system/libdispatch.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8564, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Mon Jul 2 11:21:13 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9041f000 - 0x90422fff /usr/lib/system/libcompiler_rt.dylib 0x90773000 - 0x90777fff /usr/lib/system/libsystem_network.dylib 0x91574000 - 0x9157cfff /usr/lib/system/libcopyfile.dylib 0x9157d000 - 0x9157efff /usr/lib/system/libsystem_sandbox.dylib 0x91ffb000 - 0x92003fff /usr/lib/system/libunwind.dylib 0x92583000 - 0x9258afff /usr/lib/system/libsystem_dnssd.dylib 0x9260d000 - 0x92611fff /usr/lib/system/libcache.dylib 0x92cd5000 - 0x92cebfff /usr/lib/system/libxpc.dylib 0x93c9c000 - 0x93c9dfff /usr/lib/system/libremovefile.dylib 0x93d72000 - 0x93d72fff /usr/lib/system/libkeymgr.dylib 0x93fa7000 - 0x93fa9fff /usr/lib/system/libdyld.dylib 0x93ffc000 - 0x94004fff /usr/lib/system/liblaunch.dylib 0x94008000 - 0x94009fff /usr/lib/system/libunc.dylib 0x962f6000 - 0x96339fff /usr/lib/system/libcommonCrypto.dylib 0x96392000 - 0x963a0fff /usr/lib/libz.1.dylib 0x97b0d000 - 0x97b10fff /usr/lib/system/libmathCommon.A.dylib 0x988c1000 - 0x988c2fff /usr/lib/system/libsystem_blocks.dylib 0x988d9000 - 0x988d9fff /usr/lib/system/libdnsinfo.dylib 0x98f73000 - 0x98f7afff /usr/lib/system/libsystem_notify.dylib 0x996b0000 - 0x996b5fff /usr/lib/system/libmacho.dylib 0x999f2000 - 0x99a20fff /usr/lib/libSystem.B.dylib 0x9a086000 - 0x9a087fff /usr/lib/system/libquarantine.dylib 0x9aa51000 - 0x9aa6ffff /usr/lib/system/libsystem_kernel.dylib 0x9ac47000 - 0x9ac76fff /usr/lib/system/libsystem_info.dylib 0x9b8d2000 - 0x9b99dfff /usr/lib/system/libsystem_c.dylib 0x9b9e3000 - 0x9b9f1fff /usr/lib/system/libdispatch.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8564, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Mon Jul 2 11:21:16 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9041f000 - 0x90422fff /usr/lib/system/libcompiler_rt.dylib 0x90773000 - 0x90777fff /usr/lib/system/libsystem_network.dylib 0x91574000 - 0x9157cfff /usr/lib/system/libcopyfile.dylib 0x9157d000 - 0x9157efff /usr/lib/system/libsystem_sandbox.dylib 0x91ffb000 - 0x92003fff /usr/lib/system/libunwind.dylib 0x92583000 - 0x9258afff /usr/lib/system/libsystem_dnssd.dylib 0x9260d000 - 0x92611fff /usr/lib/system/libcache.dylib 0x92cd5000 - 0x92cebfff /usr/lib/system/libxpc.dylib 0x93c9c000 - 0x93c9dfff /usr/lib/system/libremovefile.dylib 0x93d72000 - 0x93d72fff /usr/lib/system/libkeymgr.dylib 0x93fa7000 - 0x93fa9fff /usr/lib/system/libdyld.dylib 0x93ffc000 - 0x94004fff /usr/lib/system/liblaunch.dylib 0x94008000 - 0x94009fff /usr/lib/system/libunc.dylib 0x962f6000 - 0x96339fff /usr/lib/system/libcommonCrypto.dylib 0x96392000 - 0x963a0fff /usr/lib/libz.1.dylib 0x97b0d000 - 0x97b10fff /usr/lib/system/libmathCommon.A.dylib 0x988c1000 - 0x988c2fff /usr/lib/system/libsystem_blocks.dylib 0x988d9000 - 0x988d9fff /usr/lib/system/libdnsinfo.dylib 0x98f73000 - 0x98f7afff /usr/lib/system/libsystem_notify.dylib 0x996b0000 - 0x996b5fff /usr/lib/system/libmacho.dylib 0x999f2000 - 0x99a20fff /usr/lib/libSystem.B.dylib 0x9a086000 - 0x9a087fff /usr/lib/system/libquarantine.dylib 0x9aa51000 - 0x9aa6ffff /usr/lib/system/libsystem_kernel.dylib 0x9ac47000 - 0x9ac76fff /usr/lib/system/libsystem_info.dylib 0x9b8d2000 - 0x9b99dfff /usr/lib/system/libsystem_c.dylib 0x9b9e3000 - 0x9b9f1fff /usr/lib/system/libdispatch.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8564, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Mon Jul 2 11:21:20 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9041f000 - 0x90422fff /usr/lib/system/libcompiler_rt.dylib 0x90773000 - 0x90777fff /usr/lib/system/libsystem_network.dylib 0x91574000 - 0x9157cfff /usr/lib/system/libcopyfile.dylib 0x9157d000 - 0x9157efff /usr/lib/system/libsystem_sandbox.dylib 0x91ffb000 - 0x92003fff /usr/lib/system/libunwind.dylib 0x92583000 - 0x9258afff /usr/lib/system/libsystem_dnssd.dylib 0x9260d000 - 0x92611fff /usr/lib/system/libcache.dylib 0x92cd5000 - 0x92cebfff /usr/lib/system/libxpc.dylib 0x93c9c000 - 0x93c9dfff /usr/lib/system/libremovefile.dylib 0x93d72000 - 0x93d72fff /usr/lib/system/libkeymgr.dylib 0x93fa7000 - 0x93fa9fff /usr/lib/system/libdyld.dylib 0x93ffc000 - 0x94004fff /usr/lib/system/liblaunch.dylib 0x94008000 - 0x94009fff /usr/lib/system/libunc.dylib 0x962f6000 - 0x96339fff /usr/lib/system/libcommonCrypto.dylib 0x96392000 - 0x963a0fff /usr/lib/libz.1.dylib 0x97b0d000 - 0x97b10fff /usr/lib/system/libmathCommon.A.dylib 0x988c1000 - 0x988c2fff /usr/lib/system/libsystem_blocks.dylib 0x988d9000 - 0x988d9fff /usr/lib/system/libdnsinfo.dylib 0x98f73000 - 0x98f7afff /usr/lib/system/libsystem_notify.dylib 0x996b0000 - 0x996b5fff /usr/lib/system/libmacho.dylib 0x999f2000 - 0x99a20fff /usr/lib/libSystem.B.dylib 0x9a086000 - 0x9a087fff /usr/lib/system/libquarantine.dylib 0x9aa51000 - 0x9aa6ffff /usr/lib/system/libsystem_kernel.dylib 0x9ac47000 - 0x9ac76fff /usr/lib/system/libsystem_info.dylib 0x9b8d2000 - 0x9b99dfff /usr/lib/system/libsystem_c.dylib 0x9b9e3000 - 0x9b9f1fff /usr/lib/system/libdispatch.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8564, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Mon Jul 2 11:21:24 2012 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x9041f000 - 0x90422fff /usr/lib/system/libcompiler_rt.dylib 0x90773000 - 0x90777fff /usr/lib/system/libsystem_network.dylib 0x91574000 - 0x9157cfff /usr/lib/system/libcopyfile.dylib 0x9157d000 - 0x9157efff /usr/lib/system/libsystem_sandbox.dylib 0x91ffb000 - 0x92003fff /usr/lib/system/libunwind.dylib 0x92583000 - 0x9258afff /usr/lib/system/libsystem_dnssd.dylib 0x9260d000 - 0x92611fff /usr/lib/system/libcache.dylib 0x92cd5000 - 0x92cebfff /usr/lib/system/libxpc.dylib 0x93c9c000 - 0x93c9dfff /usr/lib/system/libremovefile.dylib 0x93d72000 - 0x93d72fff /usr/lib/system/libkeymgr.dylib 0x93fa7000 - 0x93fa9fff /usr/lib/system/libdyld.dylib 0x93ffc000 - 0x94004fff /usr/lib/system/liblaunch.dylib 0x94008000 - 0x94009fff /usr/lib/system/libunc.dylib 0x962f6000 - 0x96339fff /usr/lib/system/libcommonCrypto.dylib 0x96392000 - 0x963a0fff /usr/lib/libz.1.dylib 0x97b0d000 - 0x97b10fff /usr/lib/system/libmathCommon.A.dylib 0x988c1000 - 0x988c2fff /usr/lib/system/libsystem_blocks.dylib 0x988d9000 - 0x988d9fff /usr/lib/system/libdnsinfo.dylib 0x98f73000 - 0x98f7afff /usr/lib/system/libsystem_notify.dylib 0x996b0000 - 0x996b5fff /usr/lib/system/libmacho.dylib 0x999f2000 - 0x99a20fff /usr/lib/libSystem.B.dylib 0x9a086000 - 0x9a087fff /usr/lib/system/libquarantine.dylib 0x9aa51000 - 0x9aa6ffff /usr/lib/system/libsystem_kernel.dylib 0x9ac47000 - 0x9ac76fff /usr/lib/system/libsystem_info.dylib 0x9b8d2000 - 0x9b99dfff /usr/lib/system/libsystem_c.dylib 0x9b9e3000 - 0x9b9f1fff /usr/lib/system/libdispatch.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8564, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Jul 2012 11:07:27	1060946	14847580	hadcm3n_zioe_1880_40_008026197_1	77,760	122,979	1.5815
02 Jul 2012 11:07:27	1060946	14847580	hadcm3n_zioe_1880_40_008026197_1	51,840	83,462	1.6100
02 Jul 2012 11:07:27	1060946	14847580	hadcm3n_zioe_1880_40_008026197_1	25,920	45,124	1.7409