Task 16099110

Name	hadcm3n_o6o8_2020_40_008373174_2
Workunit	8524033
Created	28 Nov 2013, 9:27:15 UTC
Sent	28 Nov 2013, 9:27:19 UTC
Report deadline	27 Feb 2014, 16:54:30 UTC
Received	18 Jan 2014, 9:23:47 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1300556
Run time	11 days 21 hours 1 min 49 sec
CPU time	11 days 15 hours 6 min 12 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	3.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> Das Gerät erkennt den Befehl nicht. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:37:40 (1000): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 10:37:42 (1000): No heartbeat from core client for 30 sec - exiting 10:37:43 (1000): No heartbeat from core client for 30 sec - exiting 10:37:44 (1000): No heartbeat from core client for 30 sec - exiting 10:37:45 (1000): No heartbeat from core client for 30 sec - exiting 10:37:46 (1000): No heartbeat from core client for 30 sec - exiting 10:37:47 (1000): No heartbeat from core client for 30 sec - exiting 10:37:48 (1000): No heartbeat from core client for 30 sec - exiting 10:37:49 (1000): No heartbeat from core client for 30 sec - exiting 10:37:50 (1000): No heartbeat from core client for 30 sec - exiting 10:37:51 (1000): No heartbeat from core client for 30 sec - exiting 10:37:53 (1000): No heartbeat from core client for 30 sec - exiting 10:37:54 (1000): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 23:59:41 (2904): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... 00:00:46 (2904): No heartbeat from core client for 30 sec - exiting 00:00:47 (2904): No heartbeat from core client for 30 sec - exiting 00:00:48 (2904): No heartbeat from core client for 30 sec - exiting 00:00:49 (2904): No heartbeat from core client for 30 sec - exiting 00:00:50 (2904): No heartbeat from core client for 30 sec - exiting 00:00:52 (2904): No heartbeat from core client for 30 sec - exiting 00:00:53 (2904): No heartbeat from core client for 30 sec - exiting 00:00:54 (2904): No heartbeat from core client for 30 sec - exiting 00:00:55 (2904): No heartbeat from core client for 30 sec - exiting 00:00:56 (2904): No heartbeat from core client for 30 sec - exiting 00:00:57 (2904): No heartbeat from core client for 30 sec - exiting 00:00:58 (2904): No heartbeat from core client for 30 sec - exiting 00:00:59 (2904): No heartbeat from core client for 30 sec - exiting 00:01:00 (2904): No heartbeat from core client for 30 sec - exiting 00:01:01 (2904): No heartbeat from core client for 30 sec - exiting 00:01:02 (2904): No heartbeat from core client for 30 sec - exiting 00:01:04 (2904): No heartbeat from core client for 30 sec - exiting 00:01:05 (2904): No heartbeat from core client for 30 sec - exiting 00:01:06 (2904): No heartbeat from core client for 30 sec - exiting 00:01:07 (2904): No heartbeat from core client for 30 sec - exiting 00:01:08 (2904): No heartbeat from core client for 30 sec - exiting 00:01:09 (2904): No heartbeat from core client for 30 sec - exiting 00:01:10 (2904): No heartbeat from core client for 30 sec - exiting 00:01:11 (2904): No heartbeat from core client for 30 sec - exiting 00:01:12 (2904): No heartbeat from core client for 30 sec - exiting 00:01:13 (2904): No heartbeat from core client for 30 sec - exiting 00:01:15 (2904): No heartbeat from core client for 30 sec - exiting 00:01:16 (2904): No heartbeat from core client for 30 sec - exiting 00:01:17 (2904): No heartbeat from core client for 30 sec - exiting 00:01:18 (2904): No heartbeat from core client for 30 sec - exiting 00:01:19 (2904): No heartbeat from core client for 30 sec - exiting 00:01:20 (2904): No heartbeat from core client for 30 sec - exiting 00:01:21 (2904): No heartbeat from core client for 30 sec - exiting 00:01:22 (2904): No heartbeat from core client for 30 sec - exiting 00:01:23 (2904): No heartbeat from core client for 30 sec - exiting 00:01:24 (2904): No heartbeat from core client for 30 sec - exiting 00:01:25 (2904): No heartbeat from core client for 30 sec - exiting 00:01:27 (2904): No heartbeat from core client for 30 sec - exiting 00:01:28 (2904): No heartbeat from core client for 30 sec - exiting 00:01:29 (2904): No heartbeat from core client for 30 sec - exiting 00:01:30 (2904): No heartbeat from core client for 30 sec - exiting 00:01:31 (2904): No heartbeat from core client for 30 sec - exiting 00:01:32 (2904): No heartbeat from core client for 30 sec - exiting 00:01:33 (2904): No heartbeat from core client for 30 sec - exiting 00:01:34 (2904): No heartbeat from core client for 30 sec - exiting 00:01:35 (2904): No heartbeat from core client for 30 sec - exiting 00:01:36 (2904): No heartbeat from core client for 30 sec - exiting 00:01:37 (2904): No heartbeat from core client for 30 sec - exiting 00:01:39 (2904): No heartbeat from core client for 30 sec - exiting 00:01:40 (2904): No heartbeat from core client for 30 sec - exiting 00:01:41 (2904): No heartbeat from core client for 30 sec - exiting 00:01:42 (2904): No heartbeat from core client for 30 sec - exiting 00:01:43 (2904): No heartbeat from core client for 30 sec - exiting 00:01:44 (2904): No heartbeat from core client for 30 sec - exiting 00:01:45 (2904): No heartbeat from core client for 30 sec - exiting 00:01:46 (2904): No heartbeat from core client for 30 sec - exiting 00:01:47 (2904): No heartbeat from core client for 30 sec - exiting 00:01:48 (2904): No heartbeat from core client for 30 sec - exiting 00:01:49 (2904): No heartbeat from core client for 30 sec - exiting 09:27:48 (3504): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:17:40 (2824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:36:49 (2436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:56:09 (2436): No heartbeat from core client for 30 sec - exiting 12:56:10 (2436): No heartbeat from core client for 30 sec - exiting 12:56:11 (2436): No heartbeat from core client for 30 sec - exiting 12:56:12 (2436): No heartbeat from core client for 30 sec - exiting 12:56:14 (2436): No heartbeat from core client for 30 sec - exiting 13:01:42 (2076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:13:22 (2400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:13:54 (2400): No heartbeat from core client for 30 sec - exiting 13:13:55 (2400): No heartbeat from core client for 30 sec - exiting 13:13:56 (2400): No heartbeat from core client for 30 sec - exiting 13:13:57 (2400): No heartbeat from core client for 30 sec - exiting 13:13:58 (2400): No heartbeat from core client for 30 sec - exiting 13:13:59 (2400): No heartbeat from core client for 30 sec - exiting 13:14:01 (2400): No heartbeat from core client for 30 sec - exiting 13:14:02 (2400): No heartbeat from core client for 30 sec - exiting 13:14:03 (2400): No heartbeat from core client for 30 sec - exiting 13:14:04 (2400): No heartbeat from core client for 30 sec - exiting 13:14:41 (1928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:30:17 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:30:26 (3848): No heartbeat from core client for 30 sec - exiting 13:30:27 (3848): No heartbeat from core client for 30 sec - exiting 13:30:28 (3848): No heartbeat from core client for 30 sec - exiting 13:30:30 (3848): No heartbeat from core client for 30 sec - exiting 13:31:17 (776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:32:23 (3892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:32:26 (3892): No heartbeat from core client for 30 sec - exiting 13:32:27 (3892): No heartbeat from core client for 30 sec - exiting 13:32:28 (3892): No heartbeat from core client for 30 sec - exiting 13:32:29 (3892): No heartbeat from core client for 30 sec - exiting 13:32:30 (3892): No heartbeat from core client for 30 sec - exiting 13:32:31 (3892): No heartbeat from core client for 30 sec - exiting 13:32:32 (3892): No heartbeat from core client for 30 sec - exiting 13:32:34 (3892): No heartbeat from core client for 30 sec - exiting 13:33:12 (3988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:31:56 (3060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:32:08 (3060): No heartbeat from core client for 30 sec - exiting 00:32:09 (3060): No heartbeat from core client for 30 sec - exiting 00:32:10 (3060): No heartbeat from core client for 30 sec - exiting 02:07:34 (924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:38 (924): No heartbeat from core client for 30 sec - exiting 02:07:39 (924): No heartbeat from core client for 30 sec - exiting 02:07:40 (924): No heartbeat from core client for 30 sec - exiting 02:07:41 (924): No heartbeat from core client for 30 sec - exiting 02:07:42 (924): No heartbeat from core client for 30 sec - exiting 02:07:43 (924): No heartbeat from core client for 30 sec - exiting 02:07:44 (924): No heartbeat from core client for 30 sec - exiting 02:07:46 (924): No heartbeat from core client for 30 sec - exiting 02:07:47 (924): No heartbeat from core client for 30 sec - exiting 02:07:48 (924): No heartbeat from core client for 30 sec - exiting 02:08:51 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:09:19 (3432): No heartbeat from core client for 30 sec - exiting 02:09:20 (3432): No heartbeat from core client for 30 sec - exiting 02:10:41 (2628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:13:50 (2628): No heartbeat from core client for 30 sec - exiting 02:13:51 (2628): No heartbeat from core client for 30 sec - exiting 02:13:52 (2628): No heartbeat from core client for 30 sec - exiting 02:13:53 (2628): No heartbeat from core client for 30 sec - exiting 02:13:54 (2628): No heartbeat from core client for 30 sec - exiting 02:13:55 (2628): No heartbeat from core client for 30 sec - exiting 02:13:56 (2628): No heartbeat from core client for 30 sec - exiting 02:13:57 (2628): No heartbeat from core client for 30 sec - exiting 02:15:06 (3600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:21:16 (3564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:03:22 (3548): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 02:05:10 (3824): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:31:35 (3112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:31:49 (3112): No heartbeat from core client for 30 sec - exiting 00:31:50 (3112): No heartbeat from core client for 30 sec - exiting 00:31:51 (3112): No heartbeat from core client for 30 sec - exiting 00:31:52 (3112): No heartbeat from core client for 30 sec - exiting 00:31:54 (3112): No heartbeat from core client for 30 sec - exiting 00:31:55 (3112): No heartbeat from core client for 30 sec - exiting 00:31:56 (3112): No heartbeat from core client for 30 sec - exiting 00:31:57 (3112): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 02:10:57 (3840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:01 (3840): No heartbeat from core client for 30 sec - exiting 02:11:02 (3840): No heartbeat from core client for 30 sec - exiting 02:11:03 (3840): No heartbeat from core client for 30 sec - exiting 02:11:04 (3840): No heartbeat from core client for 30 sec - exiting 02:11:05 (3840): No heartbeat from core client for 30 sec - exiting 02:11:06 (3840): No heartbeat from core client for 30 sec - exiting 02:11:07 (3840): No heartbeat from core client for 30 sec - exiting 02:11:09 (3840): No heartbeat from core client for 30 sec - exiting 02:11:10 (3840): No heartbeat from core client for 30 sec - exiting 02:11:11 (3840): No heartbeat from core client for 30 sec - exiting 02:15:51 (5728): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:31 (5728): No heartbeat from core client for 30 sec - exiting 02:16:32 (5728): No heartbeat from core client for 30 sec - exiting 02:16:33 (5728): No heartbeat from core client for 30 sec - exiting 02:16:35 (5728): No heartbeat from core client for 30 sec - exiting 02:16:36 (5728): No heartbeat from core client for 30 sec - exiting 02:18:52 (3820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:38:18 (5656): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:38:38 (5656): No heartbeat from core client for 30 sec - exiting 02:38:39 (5656): No heartbeat from core client for 30 sec - exiting 02:38:40 (5656): No heartbeat from core client for 30 sec - exiting 02:38:41 (5656): No heartbeat from core client for 30 sec - exiting 02:38:42 (5656): No heartbeat from core client for 30 sec - exiting 02:38:43 (5656): No heartbeat from core client for 30 sec - exiting 02:38:44 (5656): No heartbeat from core client for 30 sec - exiting 02:38:45 (5656): No heartbeat from core client for 30 sec - exiting 02:38:46 (5656): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 22:07:17 (3884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:00:05 (3536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:01:43 (2352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:04:41 (2352): No heartbeat from core client for 30 sec - exiting 06:04:42 (2352): No heartbeat from core client for 30 sec - exiting 06:04:43 (2352): No heartbeat from core client for 30 sec - exiting 06:04:44 (2352): No heartbeat from core client for 30 sec - exiting 06:04:45 (2352): No heartbeat from core client for 30 sec - exiting 06:04:46 (2352): No heartbeat from core client for 30 sec - exiting 06:04:47 (2352): No heartbeat from core client for 30 sec - exiting 06:04:48 (2352): No heartbeat from core client for 30 sec - exiting 06:05:56 (2396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:06:00 (2396): No heartbeat from core client for 30 sec - exiting 06:06:01 (2396): No heartbeat from core client for 30 sec - exiting 06:06:02 (2396): No heartbeat from core client for 30 sec - exiting 06:06:03 (2396): No heartbeat from core client for 30 sec - exiting 06:06:04 (2396): No heartbeat from core client for 30 sec - exiting 06:07:58 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:08:12 (3900): No heartbeat from core client for 30 sec - exiting 06:08:13 (3900): No heartbeat from core client for 30 sec - exiting 06:08:14 (3900): No heartbeat from core client for 30 sec - exiting 06:08:15 (3900): No heartbeat from core client for 30 sec - exiting 06:08:16 (3900): No heartbeat from core client for 30 sec - exiting 06:08:17 (3900): No heartbeat from core client for 30 sec - exiting 06:08:18 (3900): No heartbeat from core client for 30 sec - exiting 06:08:19 (3900): No heartbeat from core client for 30 sec - exiting 06:10:44 (4068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:47 (3392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:15:12 (3392): No heartbeat from core client for 30 sec - exiting 06:15:13 (3392): No heartbeat from core client for 30 sec - exiting 06:15:14 (3392): No heartbeat from core client for 30 sec - exiting 06:15:18 (3392): No heartbeat from core client for 30 sec - exiting 06:15:19 (3392): No heartbeat from core client for 30 sec - exiting 06:16:50 (2816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:18:28 (2144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:04:00 (2368): No heartbeat from core client for 30 sec - exiting 12:04:01 (2368): No heartbeat from core client for 30 sec - exiting 12:04:02 (2368): No heartbeat from core client for 30 sec - exiting 12:04:04 (2368): No heartbeat from core client for 30 sec - exiting 12:04:05 (2368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:21 (1956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:28:58 (3356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:17 (3356): No heartbeat from core client for 30 sec - exiting 14:29:18 (3356): No heartbeat from core client for 30 sec - exiting 14:29:20 (3356): No heartbeat from core client for 30 sec - exiting 14:29:21 (3356): No heartbeat from core client for 30 sec - exiting 14:29:22 (3356): No heartbeat from core client for 30 sec - exiting 14:29:23 (3356): No heartbeat from core client for 30 sec - exiting 14:29:24 (3356): No heartbeat from core client for 30 sec - exiting 14:34:21 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:34:31 (3576): No heartbeat from core client for 30 sec - exiting 14:34:33 (3576): No heartbeat from core client for 30 sec - exiting 14:34:34 (3576): No heartbeat from core client for 30 sec - exiting 14:34:35 (3576): No heartbeat from core client for 30 sec - exiting 14:34:36 (3576): No heartbeat from core client for 30 sec - exiting 14:34:37 (3576): No heartbeat from core client for 30 sec - exiting 14:34:38 (3576): No heartbeat from core client for 30 sec - exiting 14:34:39 (3576): No heartbeat from core client for 30 sec - exiting 14:34:40 (3576): No heartbeat from core client for 30 sec - exiting 14:34:41 (3576): No heartbeat from core client for 30 sec - exiting 14:44:22 (1000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:45:01 (720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:53:17 (276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:54:34 (1064): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:55:08 (3412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:02:34 (3112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:04:20 (1552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 15:11:27 (3528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:28 (3528): No heartbeat from core client for 30 sec - exiting 15:12:42 (3848): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:43 (3848): No heartbeat from core client for 30 sec - exiting 15:12:44 (3848): No heartbeat from core client for 30 sec - exiting 15:12:45 (3848): No heartbeat from core client for 30 sec - exiting 15:12:47 (3848): No heartbeat from core client for 30 sec - exiting 15:12:48 (3848): No heartbeat from core client for 30 sec - exiting 15:12:49 (3848): No heartbeat from core client for 30 sec - exiting 15:13:25 (3684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:13:31 (3684): No heartbeat from core client for 30 sec - exiting 15:13:32 (3684): No heartbeat from core client for 30 sec - exiting 15:13:33 (3684): No heartbeat from core client for 30 sec - exiting 15:13:34 (3684): No heartbeat from core client for 30 sec - exiting 15:13:35 (3684): No heartbeat from core client for 30 sec - exiting 15:14:31 (2348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:21:15 (1312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:23:04 (3368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:29:23 (3924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:30:05 (3284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:08 (2348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:10 (2348): No heartbeat from core client for 30 sec - exiting 15:31:11 (2348): No heartbeat from core client for 30 sec - exiting 15:31:12 (2348): No heartbeat from core client for 30 sec - exiting 15:31:13 (2348): No heartbeat from core client for 30 sec - exiting 15:31:14 (2348): No heartbeat from core client for 30 sec - exiting 15:31:15 (2348): No heartbeat from core client for 30 sec - exiting 15:31:16 (2348): No heartbeat from core client for 30 sec - exiting 15:31:17 (2348): No heartbeat from core client for 30 sec - exiting 15:31:18 (2348): No heartbeat from core client for 30 sec - exiting 15:31:19 (2348): No heartbeat from core client for 30 sec - exiting 15:36:22 (1432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:36:26 (1432): No heartbeat from core client for 30 sec - exiting 15:36:27 (1432): No heartbeat from core client for 30 sec - exiting 15:36:28 (1432): No heartbeat from core client for 30 sec - exiting 15:36:29 (1432): No heartbeat from core client for 30 sec - exiting 15:36:30 (1432): No heartbeat from core client for 30 sec - exiting 15:37:06 (3492): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:39:26 (3588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:00:41 (3088): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:00:50 (3088): No heartbeat from core client for 30 sec - exiting 05:00:51 (3088): No heartbeat from core client for 30 sec - exiting 05:00:52 (3088): No heartbeat from core client for 30 sec - exiting 05:00:53 (3088): No heartbeat from core client for 30 sec - exiting 05:00:54 (3088): No heartbeat from core client for 30 sec - exiting 05:00:56 (3088): No heartbeat from core client for 30 sec - exiting 05:00:57 (3088): No heartbeat from core client for 30 sec - exiting 05:00:58 (3088): No heartbeat from core client for 30 sec - exiting 05:00:59 (3088): No heartbeat from core client for 30 sec - exiting 05:01:00 (3088): No heartbeat from core client for 30 sec - exiting 05:02:28 (2624): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:06:32 (2624): No heartbeat from core client for 30 sec - exiting 05:06:34 (2624): No heartbeat from core client for 30 sec - exiting 05:06:35 (2624): No heartbeat from core client for 30 sec - exiting 05:06:36 (2624): No heartbeat from core client for 30 sec - exiting 05:06:37 (2624): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 05:08:28 (3404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:08:29 (3404): No heartbeat from core client for 30 sec - exiting 05:08:30 (3404): No heartbeat from core client for 30 sec - exiting 05:10:00 (3952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:12:03 (2324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:13:26 (2036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:14:40 (3052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:16:47 (2828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:18:04 (3816): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Jan 2014 00:50:15	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	855,360	993,377	1.1614
17 Jan 2014 16:37:00	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	829,440	963,799	1.1620
17 Jan 2014 08:22:05	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	803,520	934,318	1.1628
17 Jan 2014 00:09:54	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	777,600	904,824	1.1636
16 Jan 2014 16:01:09	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	751,680	875,543	1.1648
15 Jan 2014 22:25:58	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	725,760	846,015	1.1657
15 Jan 2014 13:35:21	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	699,840	816,884	1.1672
14 Jan 2014 21:26:04	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	673,920	792,913	1.1766
14 Jan 2014 14:42:26	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	648,000	768,815	1.1864
14 Jan 2014 07:47:49	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	622,080	743,996	1.1960
14 Jan 2014 01:51:52	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	596,160	724,123	1.2146
25 Dec 2013 00:08:31	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	570,240	697,876	1.2238
24 Dec 2013 16:40:19	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	544,320	671,248	1.2332
24 Dec 2013 09:13:44	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	518,400	644,269	1.2428
24 Dec 2013 01:51:56	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	492,480	617,755	1.2544
23 Dec 2013 18:19:24	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	466,560	591,051	1.2668
23 Dec 2013 10:51:41	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	440,640	564,021	1.2800
19 Dec 2013 07:25:05	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	414,720	536,710	1.2942
19 Dec 2013 00:42:52	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	388,800	513,854	1.3216
18 Dec 2013 17:19:10	1300556	16099110	hadcm3n_o6o8_2020_40_008373174_2	362,880	486,058	1.3394