Task 15995990

Name	hadcm3n_834i_1980_40_008461989_0
Workunit	8612845
Created	30 Aug 2013, 22:28:05 UTC
Sent	1 Sep 2013, 10:49:49 UTC
Report deadline	1 Dec 2013, 18:17:00 UTC
Received	18 Oct 2013, 11:59:00 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	980519
Run time	24 days 22 hours 33 min 39 sec
CPU time	21 days 23 hours 34 min 38 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> 02:18:36 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:48 (4776): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5600, iMonCtr=1 Model crash detected, will try to restart... 09:02:41 (5684): No heartbeat from core client for 30 sec - exiting 09:02:42 (5684): No heartbeat from core client for 30 sec - exiting 09:02:43 (5684): No heartbeat from core client for 30 sec - exiting 09:02:44 (5684): No heartbeat from core client for 30 sec - exiting 09:02:45 (5684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:05:18 (4660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3352, iMonCtr=1 Model crash detected, will try to restart... 19:06:40 (4516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:44:30 (4320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:52:14 (5932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:52:15 (5932): No heartbeat from core client for 30 sec - exiting 19:52:16 (5932): No heartbeat from core client for 30 sec - exiting 19:52:17 (5932): No heartbeat from core client for 30 sec - exiting 19:52:18 (5932): No heartbeat from core client for 30 sec - exiting 19:52:19 (5932): No heartbeat from core client for 30 sec - exiting 19:52:20 (5932): No heartbeat from core client for 30 sec - exiting 19:52:21 (5932): No heartbeat from core client for 30 sec - exiting 19:52:22 (5932): No heartbeat from core client for 30 sec - exiting 19:52:23 (5932): No heartbeat from core client for 30 sec - exiting 19:52:24 (5932): No heartbeat from core client for 30 sec - exiting 20:32:45 (3180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:17 (3180): No heartbeat from core client for 30 sec - exiting 02:21:11 (5360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6120, iMonCtr=1 Model crash detected, will try to restart... 20:00:26 (6136): No heartbeat from core client for 30 sec - exiting 20:00:28 (6136): No heartbeat from core client for 30 sec - exiting 20:00:29 (6136): No heartbeat from core client for 30 sec - exiting 20:00:30 (6136): No heartbeat from core client for 30 sec - exiting 20:00:31 (6136): No heartbeat from core client for 30 sec - exiting 20:00:32 (6136): No heartbeat from core client for 30 sec - exiting 20:00:33 (6136): No heartbeat from core client for 30 sec - exiting 20:00:34 (6136): No heartbeat from core client for 30 sec - exiting 20:00:35 (6136): No heartbeat from core client for 30 sec - exiting 20:00:36 (6136): No heartbeat from core client for 30 sec - exiting 20:00:37 (6136): No heartbeat from core client for 30 sec - exiting 20:00:38 (6136): No heartbeat from core client for 30 sec - exiting 20:00:40 (6136): No heartbeat from core client for 30 sec - exiting 20:00:41 (6136): No heartbeat from core client for 30 sec - exiting 20:00:42 (6136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:29 (3804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:48:26 (4324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1588, iMonCtr=1 Model crash detected, will try to restart... 08:49:15 (1216): No heartbeat from core client for 30 sec - exiting 08:49:16 (1216): No heartbeat from core client for 30 sec - exiting 08:49:17 (1216): No heartbeat from core client for 30 sec - exiting 08:49:18 (1216): No heartbeat from core client for 30 sec - exiting 08:49:19 (1216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:22:39 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:03:17 (6084): No heartbeat from core client for 30 sec - exiting 20:03:19 (6084): No heartbeat from core client for 30 sec - exiting 20:03:20 (6084): No heartbeat from core client for 30 sec - exiting 20:03:21 (6084): No heartbeat from core client for 30 sec - exiting 20:03:22 (6084): No heartbeat from core client for 30 sec - exiting 20:03:23 (6084): No heartbeat from core client for 30 sec - exiting 20:03:24 (6084): No heartbeat from core client for 30 sec - exiting 20:03:25 (6084): No heartbeat from core client for 30 sec - exiting 20:03:26 (6084): No heartbeat from core client for 30 sec - exiting 20:03:27 (6084): No heartbeat from core client for 30 sec - exiting 20:03:28 (6084): No heartbeat from core client for 30 sec - exiting 20:03:30 (6084): No heartbeat from core client for 30 sec - exiting 20:03:31 (6084): No heartbeat from core client for 30 sec - exiting 20:03:32 (6084): No heartbeat from core client for 30 sec - exiting 20:03:33 (6084): No heartbeat from core client for 30 sec - exiting 20:03:34 (6084): No heartbeat from core client for 30 sec - exiting 20:03:35 (6084): No heartbeat from core client for 30 sec - exiting 20:03:36 (6084): No heartbeat from core client for 30 sec - exiting 20:03:37 (6084): No heartbeat from core client for 30 sec - exiting 20:03:38 (6084): No heartbeat from core client for 30 sec - exiting 20:03:39 (6084): No heartbeat from core client for 30 sec - exiting 20:03:40 (6084): No heartbeat from core client for 30 sec - exiting 20:03:42 (6084): No heartbeat from core client for 30 sec - exiting 20:03:43 (6084): No heartbeat from core client for 30 sec - exiting 20:03:44 (6084): No heartbeat from core client for 30 sec - exiting 20:03:45 (6084): No heartbeat from core client for 30 sec - exiting 20:03:46 (6084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:03:47 (6084): No heartbeat from core client for 30 sec - exiting 20:03:48 (6084): No heartbeat from core client for 30 sec - exiting 20:03:49 (6084): No heartbeat from core client for 30 sec - exiting 20:03:50 (6084): No heartbeat from core client for 30 sec - exiting 20:03:51 (6084): No heartbeat from core client for 30 sec - exiting 20:03:52 (6084): No heartbeat from core client for 30 sec - exiting 20:03:54 (6084): No heartbeat from core client for 30 sec - exiting 20:03:55 (6084): No heartbeat from core client for 30 sec - exiting 20:03:56 (6084): No heartbeat from core client for 30 sec - exiting 20:10:30 (4576): No heartbeat from core client for 30 sec - exiting 20:10:31 (4576): No heartbeat from core client for 30 sec - exiting 20:10:32 (4576): No heartbeat from core client for 30 sec - exiting 20:10:33 (4576): No heartbeat from core client for 30 sec - exiting 20:10:34 (4576): No heartbeat from core client for 30 sec - exiting 20:10:35 (4576): No heartbeat from core client for 30 sec - exiting 20:10:36 (4576): No heartbeat from core client for 30 sec - exiting 20:10:37 (4576): No heartbeat from core client for 30 sec - exiting 20:10:38 (4576): No heartbeat from core client for 30 sec - exiting 20:10:39 (4576): No heartbeat from core client for 30 sec - exiting 20:10:40 (4576): No heartbeat from core client for 30 sec - exiting 20:10:41 (4576): No heartbeat from core client for 30 sec - exiting 20:10:42 (4576): No heartbeat from core client for 30 sec - exiting 20:10:43 (4576): No heartbeat from core client for 30 sec - exiting 20:10:44 (4576): No heartbeat from core client for 30 sec - exiting 20:10:45 (4576): No heartbeat from core client for 30 sec - exiting 20:10:46 (4576): No heartbeat from core client for 30 sec - exiting 20:10:47 (4576): No heartbeat from core client for 30 sec - exiting 20:10:48 (4576): No heartbeat from core client for 30 sec - exiting 20:10:49 (4576): No heartbeat from core client for 30 sec - exiting 20:10:50 (4576): No heartbeat from core client for 30 sec - exiting 20:10:51 (4576): No heartbeat from core client for 30 sec - exiting 20:10:52 (4576): No heartbeat from core client for 30 sec - exiting 20:10:53 (4576): No heartbeat from core client for 30 sec - exiting 20:10:54 (4576): No heartbeat from core client for 30 sec - exiting 20:10:55 (4576): No heartbeat from core client for 30 sec - exiting 20:10:56 (4576): No heartbeat from core client for 30 sec - exiting 20:10:57 (4576): No heartbeat from core client for 30 sec - exiting 20:10:58 (4576): No heartbeat from core client for 30 sec - exiting 20:10:59 (4576): No heartbeat from core client for 30 sec - exiting 20:11:00 (4576): No heartbeat from core client for 30 sec - exiting 20:11:01 (4576): No heartbeat from core client for 30 sec - exiting 20:11:02 (4576): No heartbeat from core client for 30 sec - exiting 20:11:03 (4576): No heartbeat from core client for 30 sec - exiting 20:11:04 (4576): No heartbeat from core client for 30 sec - exiting 20:11:05 (4576): No heartbeat from core client for 30 sec - exiting 20:11:06 (4576): No heartbeat from core client for 30 sec - exiting 20:11:07 (4576): No heartbeat from core client for 30 sec - exiting 20:11:08 (4576): No heartbeat from core client for 30 sec - exiting 20:11:09 (4576): No heartbeat from core client for 30 sec - exiting 20:11:10 (4576): No heartbeat from core client for 30 sec - exiting 20:11:11 (4576): No heartbeat from core client for 30 sec - exiting 20:11:12 (4576): No heartbeat from core client for 30 sec - exiting 20:11:13 (4576): No heartbeat from core client for 30 sec - exiting 20:11:14 (4576): No heartbeat from core client for 30 sec - exiting 20:11:15 (4576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:17:33 (7380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:17:35 (7380): No heartbeat from core client for 30 sec - exiting 02:18:50 (7244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:06:22 (6236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:07:01 (6632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9452, iMonCtr=1 Model crash detected, will try to restart... 12:44:44 (4288): No heartbeat from core client for 30 sec - exiting 12:44:45 (4288): No heartbeat from core client for 30 sec - exiting 12:44:46 (4288): No heartbeat from core client for 30 sec - exiting 12:44:47 (4288): No heartbeat from core client for 30 sec - exiting 12:44:48 (4288): No heartbeat from core client for 30 sec - exiting 12:44:49 (4288): No heartbeat from core client for 30 sec - exiting 12:44:50 (4288): No heartbeat from core client for 30 sec - exiting 12:44:51 (4288): No heartbeat from core client for 30 sec - exiting 12:44:52 (4288): No heartbeat from core client for 30 sec - exiting 12:44:53 (4288): No heartbeat from core client for 30 sec - exiting 12:44:54 (4288): No heartbeat from core client for 30 sec - exiting 12:44:56 (4288): No heartbeat from core client for 30 sec - exiting 12:44:57 (4288): No heartbeat from core client for 30 sec - exiting 12:44:58 (4288): No heartbeat from core client for 30 sec - exiting 12:44:59 (4288): No heartbeat from core client for 30 sec - exiting 12:45:00 (4288): No heartbeat from core client for 30 sec - exiting 12:45:01 (4288): No heartbeat from core client for 30 sec - exiting 12:45:02 (4288): No heartbeat from core client for 30 sec - exiting 12:45:03 (4288): No heartbeat from core client for 30 sec - exiting 12:45:04 (4288): No heartbeat from core client for 30 sec - exiting 12:45:05 (4288): No heartbeat from core client for 30 sec - exiting 12:45:07 (4288): No heartbeat from core client for 30 sec - exiting 12:45:08 (4288): No heartbeat from core client for 30 sec - exiting 12:45:09 (4288): No heartbeat from core client for 30 sec - exiting 12:45:10 (4288): No heartbeat from core client for 30 sec - exiting 12:45:11 (4288): No heartbeat from core client for 30 sec - exiting 12:45:12 (4288): No heartbeat from core client for 30 sec - exiting 12:45:13 (4288): No heartbeat from core client for 30 sec - exiting 12:45:14 (4288): No heartbeat from core client for 30 sec - exiting 12:45:15 (4288): No heartbeat from core client for 30 sec - exiting 12:45:16 (4288): No heartbeat from core client for 30 sec - exiting 12:45:17 (4288): No heartbeat from core client for 30 sec - exiting 12:45:18 (4288): No heartbeat from core client for 30 sec - exiting 12:45:19 (4288): No heartbeat from core client for 30 sec - exiting 12:45:20 (4288): No heartbeat from core client for 30 sec - exiting 12:45:21 (4288): No heartbeat from core client for 30 sec - exiting 12:45:22 (4288): No heartbeat from core client for 30 sec - exiting 12:45:23 (4288): No heartbeat from core client for 30 sec - exiting 12:45:24 (4288): No heartbeat from core client for 30 sec - exiting 12:45:25 (4288): No heartbeat from core client for 30 sec - exiting 12:45:26 (4288): No heartbeat from core client for 30 sec - exiting 12:45:27 (4288): No heartbeat from core client for 30 sec - exiting 12:45:28 (4288): No heartbeat from core client for 30 sec - exiting 12:45:29 (4288): No heartbeat from core client for 30 sec - exiting 12:45:30 (4288): No heartbeat from core client for 30 sec - exiting 12:45:31 (4288): No heartbeat from core client for 30 sec - exiting 12:45:32 (4288): No heartbeat from core client for 30 sec - exiting 12:45:33 (4288): No heartbeat from core client for 30 sec - exiting 12:45:34 (4288): No heartbeat from core client for 30 sec - exiting 12:45:35 (4288): No heartbeat from core client for 30 sec - exiting 12:45:36 (4288): No heartbeat from core client for 30 sec - exiting 12:45:37 (4288): No heartbeat from core client for 30 sec - exiting 12:45:38 (4288): No heartbeat from core client for 30 sec - exiting 12:45:39 (4288): No heartbeat from core client for 30 sec - exiting 12:45:40 (4288): No heartbeat from core client for 30 sec - exiting 12:45:41 (4288): No heartbeat from core client for 30 sec - exiting 12:45:42 (4288): No heartbeat from core client for 30 sec - exiting 12:45:43 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:46:34 (1388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3844, iMonCtr=1 Model crash detected, will try to restart... 15:07:00 (4608): No heartbeat from core client for 30 sec - exiting 15:07:01 (4608): No heartbeat from core client for 30 sec - exiting 15:07:02 (4608): No heartbeat from core client for 30 sec - exiting 15:07:03 (4608): No heartbeat from core client for 30 sec - exiting 15:07:04 (4608): No heartbeat from core client for 30 sec - exiting 15:07:05 (4608): No heartbeat from core client for 30 sec - exiting 15:07:06 (4608): No heartbeat from core client for 30 sec - exiting 15:07:07 (4608): No heartbeat from core client for 30 sec - exiting 15:07:09 (4608): No heartbeat from core client for 30 sec - exiting 15:07:10 (4608): No heartbeat from core client for 30 sec - exiting 15:07:11 (4608): No heartbeat from core client for 30 sec - exiting 15:07:12 (4608): No heartbeat from core client for 30 sec - exiting 15:07:13 (4608): No heartbeat from core client for 30 sec - exiting 15:07:14 (4608): No heartbeat from core client for 30 sec - exiting 15:07:15 (4608): No heartbeat from core client for 30 sec - exiting 15:07:16 (4608): No heartbeat from core client for 30 sec - exiting 15:07:17 (4608): No heartbeat from core client for 30 sec - exiting 15:07:18 (4608): No heartbeat from core client for 30 sec - exiting 15:07:19 (4608): No heartbeat from core client for 30 sec - exiting 15:07:21 (4608): No heartbeat from core client for 30 sec - exiting 15:07:22 (4608): No heartbeat from core client for 30 sec - exiting 15:07:23 (4608): No heartbeat from core client for 30 sec - exiting 15:07:24 (4608): No heartbeat from core client for 30 sec - exiting 15:07:25 (4608): No heartbeat from core client for 30 sec - exiting 15:07:26 (4608): No heartbeat from core client for 30 sec - exiting 15:07:27 (4608): No heartbeat from core client for 30 sec - exiting 15:07:28 (4608): No heartbeat from core client for 30 sec - exiting 15:07:29 (4608): No heartbeat from core client for 30 sec - exiting 15:07:30 (4608): No heartbeat from core client for 30 sec - exiting 15:07:31 (4608): No heartbeat from core client for 30 sec - exiting 15:07:32 (4608): No heartbeat from core client for 30 sec - exiting 15:07:33 (4608): No heartbeat from core client for 30 sec - exiting 15:07:34 (4608): No heartbeat from core client for 30 sec - exiting 15:07:35 (4608): No heartbeat from core client for 30 sec - exiting 15:07:36 (4608): No heartbeat from core client for 30 sec - exiting 15:07:37 (4608): No heartbeat from core client for 30 sec - exiting 15:07:38 (4608): No heartbeat from core client for 30 sec - exiting 15:07:39 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:57 (4048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:16:59 (4048): No heartbeat from core client for 30 sec - exiting 15:17:00 (4048): No heartbeat from core client for 30 sec - exiting 15:17:01 (4048): No heartbeat from core client for 30 sec - exiting 15:18:10 (4616): No heartbeat from core client for 30 sec - exiting 15:18:11 (4616): No heartbeat from core client for 30 sec - exiting 15:18:12 (4616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:31:59 (6096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:32:00 (6096): No heartbeat from core client for 30 sec - exiting 15:32:01 (6096): No heartbeat from core client for 30 sec - exiting 15:32:46 (2636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:41:59 (168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:42:01 (168): No heartbeat from core client for 30 sec - exiting 02:11:58 (3512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:11:59 (3512): No heartbeat from core client for 30 sec - exiting 08:17:28 (9952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:17:30 (9952): No heartbeat from core client for 30 sec - exiting 17:50:18 (8588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:50:19 (8588): No heartbeat from core client for 30 sec - exiting 17:50:59 (3676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:41:04 (8568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:16:25 (9180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:16:41 (4132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:14:47 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:15 (4796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:57:56 (9376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:28:39 (10160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:42:30 (1672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:04:29 (7528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:31:37 (5320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:11:22 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:12:09 (5136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:50:48 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:49 (5524): No heartbeat from core client for 30 sec - exiting 19:50:50 (5524): No heartbeat from core client for 30 sec - exiting 20:00:56 (3972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:00:59 (3972): No heartbeat from core client for 30 sec - exiting 20:21:04 (10192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:21:06 (10192): No heartbeat from core client for 30 sec - exiting 20:22:04 (8408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 09:21:01 (4228): No heartbeat from core client for 30 sec - exiting 09:21:02 (4228): No heartbeat from core client for 30 sec - exiting 09:21:03 (4228): No heartbeat from core client for 30 sec - exiting 09:21:05 (4228): No heartbeat from core client for 30 sec - exiting 09:21:06 (4228): No heartbeat from core client for 30 sec - exiting 09:21:07 (4228): No heartbeat from core client for 30 sec - exiting 09:21:08 (4228): No heartbeat from core client for 30 sec - exiting 09:21:09 (4228): No heartbeat from core client for 30 sec - exiting 09:21:10 (4228): No heartbeat from core client for 30 sec - exiting 09:21:11 (4228): No heartbeat from core client for 30 sec - exiting 09:21:12 (4228): No heartbeat from core client for 30 sec - exiting 09:21:13 (4228): No heartbeat from core client for 30 sec - exiting 09:21:14 (4228): No heartbeat from core client for 30 sec - exiting 09:21:15 (4228): No heartbeat from core client for 30 sec - exiting 09:21:16 (4228): No heartbeat from core client for 30 sec - exiting 09:21:17 (4228): No heartbeat from core client for 30 sec - exiting 09:21:18 (4228): No heartbeat from core client for 30 sec - exiting 09:21:19 (4228): No heartbeat from core client for 30 sec - exiting 09:21:20 (4228): No heartbeat from core client for 30 sec - exiting 09:21:21 (4228): No heartbeat from core client for 30 sec - exiting 09:21:22 (4228): No heartbeat from core client for 30 sec - exiting 09:21:23 (4228): No heartbeat from core client for 30 sec - exiting 09:21:24 (4228): No heartbeat from core client for 30 sec - exiting 09:21:25 (4228): No heartbeat from core client for 30 sec - exiting 09:21:26 (4228): No heartbeat from core client for 30 sec - exiting 09:21:27 (4228): No heartbeat from core client for 30 sec - exiting 09:21:28 (4228): No heartbeat from core client for 30 sec - exiting 09:21:29 (4228): No heartbeat from core client for 30 sec - exiting 09:21:30 (4228): No heartbeat from core client for 30 sec - exiting 09:21:31 (4228): No heartbeat from core client for 30 sec - exiting 09:21:32 (4228): No heartbeat from core client for 30 sec - exiting 09:21:33 (4228): No heartbeat from core client for 30 sec - exiting 09:21:34 (4228): No heartbeat from core client for 30 sec - exiting 09:21:35 (4228): No heartbeat from core client for 30 sec - exiting 09:21:36 (4228): No heartbeat from core client for 30 sec - exiting 09:21:37 (4228): No heartbeat from core client for 30 sec - exiting 09:21:38 (4228): No heartbeat from core client for 30 sec - exiting 09:21:39 (4228): No heartbeat from core client for 30 sec - exiting 09:21:40 (4228): No heartbeat from core client for 30 sec - exiting 09:21:41 (4228): No heartbeat from core client for 30 sec - exiting 09:21:42 (4228): No heartbeat from core client for 30 sec - exiting 09:21:43 (4228): No heartbeat from core client for 30 sec - exiting 09:21:44 (4228): No heartbeat from core client for 30 sec - exiting 09:21:45 (4228): No heartbeat from core client for 30 sec - exiting 09:21:46 (4228): No heartbeat from core client for 30 sec - exiting 09:21:47 (4228): No heartbeat from core client for 30 sec - exiting 09:21:48 (4228): No heartbeat from core client for 30 sec - exiting 09:21:49 (4228): No heartbeat from core client for 30 sec - exiting 09:21:50 (4228): No heartbeat from core client for 30 sec - exiting 09:21:51 (4228): No heartbeat from core client for 30 sec - exiting 09:21:52 (4228): No heartbeat from core client for 30 sec - exiting 09:21:53 (4228): No heartbeat from core client for 30 sec - exiting 09:21:54 (4228): No heartbeat from core client for 30 sec - exiting 09:21:55 (4228): No heartbeat from core client for 30 sec - exiting 09:21:56 (4228): No heartbeat from core client for 30 sec - exiting 09:21:57 (4228): No heartbeat from core client for 30 sec - exiting 09:21:58 (4228): No heartbeat from core client for 30 sec - exiting 09:21:59 (4228): No heartbeat from core client for 30 sec - exiting 09:22:00 (4228): No heartbeat from core client for 30 sec - exiting 09:22:01 (4228): No heartbeat from core client for 30 sec - exiting 09:22:02 (4228): No heartbeat from core client for 30 sec - exiting 09:22:03 (4228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Oct 2013 06:24:41	980519	15995990	hadcm3n_834i_1980_40_008461989_0	1,036,800	1,899,260	1.8318
17 Oct 2013 16:37:41	980519	15995990	hadcm3n_834i_1980_40_008461989_0	1,010,880	1,902,244	1.8818
17 Oct 2013 02:35:24	980519	15995990	hadcm3n_834i_1980_40_008461989_0	984,960	1,871,902	1.9005
16 Oct 2013 12:31:20	980519	15995990	hadcm3n_834i_1980_40_008461989_0	959,040	1,840,595	1.9192
15 Oct 2013 22:46:32	980519	15995990	hadcm3n_834i_1980_40_008461989_0	933,120	1,795,142	1.9238
15 Oct 2013 07:29:51	980519	15995990	hadcm3n_834i_1980_40_008461989_0	907,200	1,748,677	1.9276
14 Oct 2013 17:26:43	980519	15995990	hadcm3n_834i_1980_40_008461989_0	881,280	1,701,713	1.9310
14 Oct 2013 03:04:26	980519	15995990	hadcm3n_834i_1980_40_008461989_0	855,360	1,654,076	1.9338
13 Oct 2013 11:46:04	980519	15995990	hadcm3n_834i_1980_40_008461989_0	829,440	1,607,465	1.9380
12 Oct 2013 21:44:33	980519	15995990	hadcm3n_834i_1980_40_008461989_0	803,520	1,560,703	1.9423
12 Oct 2013 07:24:16	980519	15995990	hadcm3n_834i_1980_40_008461989_0	777,600	1,512,998	1.9457
11 Oct 2013 17:00:50	980519	15995990	hadcm3n_834i_1980_40_008461989_0	751,680	1,464,947	1.9489
07 Oct 2013 22:27:35	980519	15995990	hadcm3n_834i_1980_40_008461989_0	725,760	1,416,581	1.9519
07 Oct 2013 07:44:31	980519	15995990	hadcm3n_834i_1980_40_008461989_0	699,840	1,368,432	1.9553
06 Oct 2013 18:04:41	980519	15995990	hadcm3n_834i_1980_40_008461989_0	673,920	1,321,455	1.9608
05 Oct 2013 23:09:28	980519	15995990	hadcm3n_834i_1980_40_008461989_0	648,000	1,273,398	1.9651
05 Oct 2013 06:18:04	980519	15995990	hadcm3n_834i_1980_40_008461989_0	622,080	1,224,808	1.9689
03 Oct 2013 16:15:52	980519	15995990	hadcm3n_834i_1980_40_008461989_0	596,160	1,176,668	1.9737
30 Sep 2013 00:46:33	980519	15995990	hadcm3n_834i_1980_40_008461989_0	570,240	1,128,909	1.9797
29 Sep 2013 10:24:26	980519	15995990	hadcm3n_834i_1980_40_008461989_0	544,320	1,081,027	1.9860