Task 16084149

Name	hadcm3n_102r_1980_40_008410649_1
Workunit	8561505
Created	25 Nov 2013, 22:21:26 UTC
Sent	26 Nov 2013, 0:04:54 UTC
Report deadline	25 Feb 2014, 7:32:05 UTC
Received	24 Jan 2014, 14:29:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1204099
Run time	24 days 12 hours 13 min 10 sec
CPU time	23 days 5 hours 8 min 47 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	2.25 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:40:14 (3876): No heartbeat from core client for 30 sec - exiting 16:40:15 (3876): No heartbeat from core client for 30 sec - exiting 16:40:16 (3876): No heartbeat from core client for 30 sec - exiting 16:40:17 (3876): No heartbeat from core client for 30 sec - exiting 16:40:18 (3876): No heartbeat from core client for 30 sec - exiting 16:40:19 (3876): No heartbeat from core client for 30 sec - exiting 16:40:20 (3876): No heartbeat from core client for 30 sec - exiting 16:40:21 (3876): No heartbeat from core client for 30 sec - exiting 16:40:22 (3876): No heartbeat from core client for 30 sec - exiting 16:40:23 (3876): No heartbeat from core client for 30 sec - exiting 16:40:25 (3876): No heartbeat from core client for 30 sec - exiting 16:40:26 (3876): No heartbeat from core client for 30 sec - exiting 16:40:27 (3876): No heartbeat from core client for 30 sec - exiting 16:40:28 (3876): No heartbeat from core client for 30 sec - exiting 16:40:29 (3876): No heartbeat from core client for 30 sec - exiting 16:40:30 (3876): No heartbeat from core client for 30 sec - exiting 16:40:31 (3876): No heartbeat from core client for 30 sec - exiting 16:40:32 (3876): No heartbeat from core client for 30 sec - exiting 16:40:33 (3876): No heartbeat from core client for 30 sec - exiting 16:40:34 (3876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1372, iMonCtr=1 Model crash detected, will try to restart... 14:41:14 (2356): No heartbeat from core client for 30 sec - exiting 14:41:15 (2356): No heartbeat from core client for 30 sec - exiting 14:41:16 (2356): No heartbeat from core client for 30 sec - exiting 14:41:17 (2356): No heartbeat from core client for 30 sec - exiting 14:41:18 (2356): No heartbeat from core client for 30 sec - exiting 14:41:19 (2356): No heartbeat from core client for 30 sec - exiting 14:41:20 (2356): No heartbeat from core client for 30 sec - exiting 14:41:21 (2356): No heartbeat from core client for 30 sec - exiting 14:41:23 (2356): No heartbeat from core client for 30 sec - exiting 14:41:24 (2356): No heartbeat from core client for 30 sec - exiting 14:41:25 (2356): No heartbeat from core client for 30 sec - exiting 14:41:26 (2356): No heartbeat from core client for 30 sec - exiting 14:41:27 (2356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:04:45 (10108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:37:53 (9068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=236, iMonCtr=1 Model crash detected, will try to restart... 08:48:53 (5908): No heartbeat from core client for 30 sec - exiting 08:48:54 (5908): No heartbeat from core client for 30 sec - exiting 08:48:55 (5908): No heartbeat from core client for 30 sec - exiting 08:48:56 (5908): No heartbeat from core client for 30 sec - exiting 08:48:57 (5908): No heartbeat from core client for 30 sec - exiting 08:48:58 (5908): No heartbeat from core client for 30 sec - exiting 08:48:59 (5908): No heartbeat from core client for 30 sec - exiting 08:49:00 (5908): No heartbeat from core client for 30 sec - exiting 08:49:01 (5908): No heartbeat from core client for 30 sec - exiting 08:49:02 (5908): No heartbeat from core client for 30 sec - exiting 08:49:04 (5908): No heartbeat from core client for 30 sec - exiting 08:49:05 (5908): No heartbeat from core client for 30 sec - exiting 08:49:06 (5908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=920, iMonCtr=1 Model crash detected, will try to restart... 10:19:42 (4024): No heartbeat from core client for 30 sec - exiting 10:19:43 (4024): No heartbeat from core client for 30 sec - exiting 10:19:44 (4024): No heartbeat from core client for 30 sec - exiting 10:19:45 (4024): No heartbeat from core client for 30 sec - exiting 10:19:46 (4024): No heartbeat from core client for 30 sec - exiting 10:19:47 (4024): No heartbeat from core client for 30 sec - exiting 10:19:48 (4024): No heartbeat from core client for 30 sec - exiting 10:19:50 (4024): No heartbeat from core client for 30 sec - exiting 10:19:51 (4024): No heartbeat from core client for 30 sec - exiting 10:19:52 (4024): No heartbeat from core client for 30 sec - exiting 10:19:53 (4024): No heartbeat from core client for 30 sec - exiting 10:19:54 (4024): No heartbeat from core client for 30 sec - exiting 10:19:55 (4024): No heartbeat from core client for 30 sec - exiting 10:19:56 (4024): No heartbeat from core client for 30 sec - exiting 10:19:57 (4024): No heartbeat from core client for 30 sec - exiting 10:19:58 (4024): No heartbeat from core client for 30 sec - exiting 10:19:59 (4024): No heartbeat from core client for 30 sec - exiting 10:20:00 (4024): No heartbeat from core client for 30 sec - exiting 10:20:02 (4024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4904, iMonCtr=1 Model crash detected, will try to restart... 08:38:50 (1524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:04:53 (5288): No heartbeat from core client for 30 sec - exiting 11:04:54 (5288): No heartbeat from core client for 30 sec - exiting 11:04:55 (5288): No heartbeat from core client for 30 sec - exiting 11:04:56 (5288): No heartbeat from core client for 30 sec - exiting 11:04:57 (5288): No heartbeat from core client for 30 sec - exiting 11:04:59 (5288): No heartbeat from core client for 30 sec - exiting 11:05:00 (5288): No heartbeat from core client for 30 sec - exiting 11:05:01 (5288): No heartbeat from core client for 30 sec - exiting 11:05:02 (5288): No heartbeat from core client for 30 sec - exiting 11:05:03 (5288): No heartbeat from core client for 30 sec - exiting 11:05:04 (5288): No heartbeat from core client for 30 sec - exiting 11:05:05 (5288): No heartbeat from core client for 30 sec - exiting 11:05:06 (5288): No heartbeat from core client for 30 sec - exiting 11:05:07 (5288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6112, iMonCtr=1 Model crash detected, will try to restart... 08:36:25 (5048): No heartbeat from core client for 30 sec - exiting 08:36:26 (5048): No heartbeat from core client for 30 sec - exiting 08:36:27 (5048): No heartbeat from core client for 30 sec - exiting 08:36:28 (5048): No heartbeat from core client for 30 sec - exiting 08:36:29 (5048): No heartbeat from core client for 30 sec - exiting 08:36:30 (5048): No heartbeat from core client for 30 sec - exiting 08:36:31 (5048): No heartbeat from core client for 30 sec - exiting 08:36:33 (5048): No heartbeat from core client for 30 sec - exiting 08:36:34 (5048): No heartbeat from core client for 30 sec - exiting 08:36:35 (5048): No heartbeat from core client for 30 sec - exiting 08:36:36 (5048): No heartbeat from core client for 30 sec - exiting 08:36:37 (5048): No heartbeat from core client for 30 sec - exiting 08:36:38 (5048): No heartbeat from core client for 30 sec - exiting 08:36:39 (5048): No heartbeat from core client for 30 sec - exiting 08:36:40 (5048): No heartbeat from core client for 30 sec - exiting 08:36:41 (5048): No heartbeat from core client for 30 sec - exiting 08:36:42 (5048): No heartbeat from core client for 30 sec - exiting 08:36:43 (5048): No heartbeat from core client for 30 sec - exiting 08:36:45 (5048): No heartbeat from core client for 30 sec - exiting 08:36:46 (5048): No heartbeat from core client for 30 sec - exiting 08:36:47 (5048): No heartbeat from core client for 30 sec - exiting 08:36:48 (5048): No heartbeat from core client for 30 sec - exiting 08:36:49 (5048): No heartbeat from core client for 30 sec - exiting 08:36:50 (5048): No heartbeat from core client for 30 sec - exiting 08:36:51 (5048): No heartbeat from core client for 30 sec - exiting 08:36:52 (5048): No heartbeat from core client for 30 sec - exiting 08:36:53 (5048): No heartbeat from core client for 30 sec - exiting 08:36:54 (5048): No heartbeat from core client for 30 sec - exiting 08:36:55 (5048): No heartbeat from core client for 30 sec - exiting 08:36:57 (5048): No heartbeat from core client for 30 sec - exiting 08:36:58 (5048): No heartbeat from core client for 30 sec - exiting 08:36:59 (5048): No heartbeat from core client for 30 sec - exiting 08:37:00 (5048): No heartbeat from core client for 30 sec - exiting 08:37:01 (5048): No heartbeat from core client for 30 sec - exiting 08:37:02 (5048): No heartbeat from core client for 30 sec - exiting 08:37:03 (5048): No heartbeat from core client for 30 sec - exiting 08:37:04 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:37:05 (5048): No heartbeat from core client for 30 sec - exiting 08:37:06 (5048): No heartbeat from core client for 30 sec - exiting 08:37:07 (5048): No heartbeat from core client for 30 sec - exiting 08:37:09 (5048): No heartbeat from core client for 30 sec - exiting 08:37:10 (5048): No heartbeat from core client for 30 sec - exiting 08:37:11 (5048): No heartbeat from core client for 30 sec - exiting 08:37:12 (5048): No heartbeat from core client for 30 sec - exiting 08:37:13 (5048): No heartbeat from core client for 30 sec - exiting 08:37:14 (5048): No heartbeat from core client for 30 sec - exiting 08:37:15 (5048): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:41:23 (5908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:52:19 (6872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1340, iMonCtr=1 Model crash detected, will try to restart... 11:05:24 (4768): No heartbeat from core client for 30 sec - exiting 11:05:25 (4768): No heartbeat from core client for 30 sec - exiting 11:05:27 (4768): No heartbeat from core client for 30 sec - exiting 11:05:28 (4768): No heartbeat from core client for 30 sec - exiting 11:05:29 (4768): No heartbeat from core client for 30 sec - exiting 11:05:30 (4768): No heartbeat from core client for 30 sec - exiting 11:05:31 (4768): No heartbeat from core client for 30 sec - exiting 11:05:32 (4768): No heartbeat from core client for 30 sec - exiting 11:05:33 (4768): No heartbeat from core client for 30 sec - exiting 11:05:34 (4768): No heartbeat from core client for 30 sec - exiting 11:05:35 (4768): No heartbeat from core client for 30 sec - exiting 11:05:36 (4768): No heartbeat from core client for 30 sec - exiting 11:05:37 (4768): No heartbeat from core client for 30 sec - exiting 11:05:39 (4768): No heartbeat from core client for 30 sec - exiting 11:05:40 (4768): No heartbeat from core client for 30 sec - exiting 11:05:41 (4768): No heartbeat from core client for 30 sec - exiting 11:05:42 (4768): No heartbeat from core client for 30 sec - exiting 11:05:43 (4768): No heartbeat from core client for 30 sec - exiting 11:05:44 (4768): No heartbeat from core client for 30 sec - exiting 11:05:45 (4768): No heartbeat from core client for 30 sec - exiting 11:05:46 (4768): No heartbeat from core client for 30 sec - exiting 11:05:47 (4768): No heartbeat from core client for 30 sec - exiting 11:05:48 (4768): No heartbeat from core client for 30 sec - exiting 11:05:49 (4768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:05:51 (4768): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:46:07 (3512): No heartbeat from core client for 30 sec - exiting 08:46:09 (3512): No heartbeat from core client for 30 sec - exiting 08:46:10 (3512): No heartbeat from core client for 30 sec - exiting 08:46:11 (3512): No heartbeat from core client for 30 sec - exiting 08:46:12 (3512): No heartbeat from core client for 30 sec - exiting 08:46:13 (3512): No heartbeat from core client for 30 sec - exiting 08:46:14 (3512): No heartbeat from core client for 30 sec - exiting 08:46:15 (3512): No heartbeat from core client for 30 sec - exiting 08:46:16 (3512): No heartbeat from core client for 30 sec - exiting 08:46:17 (3512): No heartbeat from core client for 30 sec - exiting 08:46:19 (3512): No heartbeat from core client for 30 sec - exiting 08:46:20 (3512): No heartbeat from core client for 30 sec - exiting 08:46:21 (3512): No heartbeat from core client for 30 sec - exiting 08:46:22 (3512): No heartbeat from core client for 30 sec - exiting 08:46:23 (3512): No heartbeat from core client for 30 sec - exiting 08:46:24 (3512): No heartbeat from core client for 30 sec - exiting 08:46:25 (3512): No heartbeat from core client for 30 sec - exiting 08:46:26 (3512): No heartbeat from core client for 30 sec - exiting 08:46:27 (3512): No heartbeat from core client for 30 sec - exiting 08:46:28 (3512): No heartbeat from core client for 30 sec - exiting 08:46:29 (3512): No heartbeat from core client for 30 sec - exiting 08:46:31 (3512): No heartbeat from core client for 30 sec - exiting 08:46:32 (3512): No heartbeat from core client for 30 sec - exiting 08:46:33 (3512): No heartbeat from core client for 30 sec - exiting 08:46:34 (3512): No heartbeat from core client for 30 sec - exiting 08:46:35 (3512): No heartbeat from core client for 30 sec - exiting 08:46:36 (3512): No heartbeat from core client for 30 sec - exiting 08:46:37 (3512): No heartbeat from core client for 30 sec - exiting 08:46:38 (3512): No heartbeat from core client for 30 sec - exiting 08:46:39 (3512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:06:07 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:29:53 (5060): No heartbeat from core client for 30 sec - exiting 13:29:54 (5060): No heartbeat from core client for 30 sec - exiting 13:29:55 (5060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:57:48 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Jan 2014 14:33:06	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	1,036,800	2,005,720	1.9345
24 Jan 2014 14:33:06	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	1,010,880	1,952,898	1.9319
13 Jan 2014 13:00:17	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	984,960	1,898,192	1.9272
09 Jan 2014 13:58:34	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	959,040	1,846,748	1.9256
08 Jan 2014 23:29:29	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	933,120	1,795,828	1.9245
08 Jan 2014 08:50:49	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	907,200	1,744,160	1.9226
06 Jan 2014 10:24:27	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	881,280	1,689,162	1.9167
19 Dec 2013 12:59:36	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	855,360	1,635,636	1.9122
18 Dec 2013 18:35:56	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	829,440	1,584,424	1.9102
18 Dec 2013 02:25:00	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	803,520	1,534,808	1.9101
17 Dec 2013 11:52:26	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	777,600	1,484,578	1.9092
16 Dec 2013 20:47:21	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	751,680	1,432,640	1.9059
13 Dec 2013 14:19:51	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	725,760	1,380,274	1.9018
12 Dec 2013 22:20:05	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	699,840	1,328,988	1.8990
12 Dec 2013 06:09:46	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	673,920	1,277,267	1.8953
11 Dec 2013 14:10:17	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	648,000	1,226,219	1.8923
10 Dec 2013 23:09:57	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	622,080	1,174,761	1.8884
10 Dec 2013 08:56:31	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	596,160	1,124,983	1.8870
09 Dec 2013 03:33:01	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	570,240	1,076,669	1.8881
08 Dec 2013 14:06:04	1204099	16084149	hadcm3n_102r_1980_40_008410649_1	544,320	1,028,845	1.8901