Task 16081650

Name	hadcm3n_o1da_1940_40_008383314_2
Workunit	8534173
Created	17 Nov 2013, 22:17:19 UTC
Sent	17 Nov 2013, 22:17:24 UTC
Report deadline	17 Feb 2014, 5:44:35 UTC
Received	2 Dec 2013, 15:05:44 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1288885
Run time	3 days 13 hours 19 min 37 sec
CPU time	2 days 11 hours 40 min 9 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 03:56:09 (7012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3352, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3352, iMonCtr=1 Model crash detected, will try to restart... 07:19:11 (3352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:09:08 (8132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:42:09 (4448): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9608, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9608, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 01:53:00 (9608): No heartbeat from core client for 30 sec - exiting 01:53:01 (9608): No heartbeat from core client for 30 sec - exiting 01:53:02 (9608): No heartbeat from core client for 30 sec - exiting 01:53:03 (9608): No heartbeat from core client for 30 sec - exiting 01:53:04 (9608): No heartbeat from core client for 30 sec - exiting 01:53:05 (9608): No heartbeat from core client for 30 sec - exiting 01:53:06 (9608): No heartbeat from core client for 30 sec - exiting 01:53:07 (9608): No heartbeat from core client for 30 sec - exiting 01:53:08 (9608): No heartbeat from core client for 30 sec - exiting 01:53:09 (9608): No heartbeat from core client for 30 sec - exiting 01:53:10 (9608): No heartbeat from core client for 30 sec - exiting 01:53:11 (9608): No heartbeat from core client for 30 sec - exiting 01:53:12 (9608): No heartbeat from core client for 30 sec - exiting 01:53:13 (9608): No heartbeat from core client for 30 sec - exiting 01:53:14 (9608): No heartbeat from core client for 30 sec - exiting 01:53:15 (9608): No heartbeat from core client for 30 sec - exiting 01:53:16 (9608): No heartbeat from core client for 30 sec - exiting 01:53:17 (9608): No heartbeat from core client for 30 sec - exiting 01:53:18 (9608): No heartbeat from core client for 30 sec - exiting 01:53:19 (9608): No heartbeat from core client for 30 sec - exiting 01:53:20 (9608): No heartbeat from core client for 30 sec - exiting 01:53:21 (9608): No heartbeat from core client for 30 sec - exiting 01:53:22 (9608): No heartbeat from core client for 30 sec - exiting 01:53:23 (9608): No heartbeat from core client for 30 sec - exiting 01:53:24 (9608): No heartbeat from core client for 30 sec - exiting 01:53:25 (9608): No heartbeat from core client for 30 sec - exiting 01:53:26 (9608): No heartbeat from core client for 30 sec - exiting 01:53:27 (9608): No heartbeat from core client for 30 sec - exiting 01:53:28 (9608): No heartbeat from core client for 30 sec - exiting 01:53:29 (9608): No heartbeat from core client for 30 sec - exiting 01:53:30 (9608): No heartbeat from core client for 30 sec - exiting 01:53:31 (9608): No heartbeat from core client for 30 sec - exiting 01:53:32 (9608): No heartbeat from core client for 30 sec - exiting 01:53:33 (9608): No heartbeat from core client for 30 sec - exiting 01:53:34 (9608): No heartbeat from core client for 30 sec - exiting 01:53:35 (9608): No heartbeat from core client for 30 sec - exiting 01:53:36 (9608): No heartbeat from core client for 30 sec - exiting 01:53:37 (9608): No heartbeat from core client for 30 sec - exiting 01:53:38 (9608): No heartbeat from core client for 30 sec - exiting 01:53:39 (9608): No heartbeat from core client for 30 sec - exiting 01:53:40 (9608): No heartbeat from core client for 30 sec - exiting 01:53:41 (9608): No heartbeat from core client for 30 sec - exiting 01:53:42 (9608): No heartbeat from core client for 30 sec - exiting 01:53:43 (9608): No heartbeat from core client for 30 sec - exiting 01:53:44 (9608): No heartbeat from core client for 30 sec - exiting 01:53:45 (9608): No heartbeat from core client for 30 sec - exiting 01:53:46 (9608): No heartbeat from core client for 30 sec - exiting 01:53:47 (9608): No heartbeat from core client for 30 sec - exiting 01:53:48 (9608): No heartbeat from core client for 30 sec - exiting 01:53:49 (9608): No heartbeat from core client for 30 sec - exiting 01:53:50 (9608): No heartbeat from core client for 30 sec - exiting 01:53:51 (9608): No heartbeat from core client for 30 sec - exiting 01:53:52 (9608): No heartbeat from core client for 30 sec - exiting 01:53:53 (9608): No heartbeat from core client for 30 sec - exiting 01:53:54 (9608): No heartbeat from core client for 30 sec - exiting 01:53:55 (9608): No heartbeat from core client for 30 sec - exiting 01:53:56 (9608): No heartbeat from core client for 30 sec - exiting 01:53:57 (9608): No heartbeat from core client for 30 sec - exiting 01:53:58 (9608): No heartbeat from core client for 30 sec - exiting 01:53:59 (9608): No heartbeat from core client for 30 sec - exiting 01:54:00 (9608): No heartbeat from core client for 30 sec - exiting 01:54:01 (9608): No heartbeat from core client for 30 sec - exiting 01:54:02 (9608): No heartbeat from core client for 30 sec - exiting 01:54:03 (9608): No heartbeat from core client for 30 sec - exiting 01:54:04 (9608): No heartbeat from core client for 30 sec - exiting 01:54:05 (9608): No heartbeat from core client for 30 sec - exiting 01:54:06 (9608): No heartbeat from core client for 30 sec - exiting 01:54:07 (9608): No heartbeat from core client for 30 sec - exiting 01:54:08 (9608): No heartbeat from core client for 30 sec - exiting 01:54:09 (9608): No heartbeat from core client for 30 sec - exiting 01:54:10 (9608): No heartbeat from core client for 30 sec - exiting 01:54:11 (9608): No heartbeat from core client for 30 sec - exiting 01:54:12 (9608): No heartbeat from core client for 30 sec - exiting 01:54:13 (9608): No heartbeat from core client for 30 sec - exiting 01:54:14 (9608): No heartbeat from core client for 30 sec - exiting 01:54:15 (9608): No heartbeat from core client for 30 sec - exiting 01:54:16 (9608): No heartbeat from core client for 30 sec - exiting 01:54:17 (9608): No heartbeat from core client for 30 sec - exiting 01:54:18 (9608): No heartbeat from core client for 30 sec - exiting 01:54:19 (9608): No heartbeat from core client for 30 sec - exiting 01:54:20 (9608): No heartbeat from core client for 30 sec - exiting 01:54:21 (9608): No heartbeat from core client for 30 sec - exiting 01:54:22 (9608): No heartbeat from core client for 30 sec - exiting 01:54:23 (9608): No heartbeat from core client for 30 sec - exiting 01:54:24 (9608): No heartbeat from core client for 30 sec - exiting 01:54:25 (9608): No heartbeat from core client for 30 sec - exiting 01:54:26 (9608): No heartbeat from core client for 30 sec - exiting 01:54:27 (9608): No heartbeat from core client for 30 sec - exiting 01:54:28 (9608): No heartbeat from core client for 30 sec - exiting 01:54:29 (9608): No heartbeat from core client for 30 sec - exiting 01:54:30 (9608): No heartbeat from core client for 30 sec - exiting 01:54:31 (9608): No heartbeat from core client for 30 sec - exiting 01:54:32 (9608): No heartbeat from core client for 30 sec - exiting 01:54:33 (9608): No heartbeat from core client for 30 sec - exiting 01:54:34 (9608): No heartbeat from core client for 30 sec - exiting 01:54:35 (9608): No heartbeat from core client for 30 sec - exiting 01:54:36 (9608): No heartbeat from core client for 30 sec - exiting 01:54:37 (9608): No heartbeat from core client for 30 sec - exiting 01:54:38 (9608): No heartbeat from core client for 30 sec - exiting 01:54:39 (9608): No heartbeat from core client for 30 sec - exiting 01:54:40 (9608): No heartbeat from core client for 30 sec - exiting 01:54:41 (9608): No heartbeat from core client for 30 sec - exiting 01:54:42 (9608): No heartbeat from core client for 30 sec - exiting 01:54:43 (9608): No heartbeat from core client for 30 sec - exiting 01:54:44 (9608): No heartbeat from core client for 30 sec - exiting 01:54:45 (9608): No heartbeat from core client for 30 sec - exiting 01:54:46 (9608): No heartbeat from core client for 30 sec - exiting 01:54:47 (9608): No heartbeat from core client for 30 sec - exiting 01:54:48 (9608): No heartbeat from core client for 30 sec - exiting 01:54:49 (9608): No heartbeat from core client for 30 sec - exiting 01:54:50 (9608): No heartbeat from core client for 30 sec - exiting 01:54:51 (9608): No heartbeat from core client for 30 sec - exiting 01:54:52 (9608): No heartbeat from core client for 30 sec - exiting 01:54:53 (9608): No heartbeat from core client for 30 sec - exiting 01:54:54 (9608): No heartbeat from core client for 30 sec - exiting 01:54:55 (9608): No heartbeat from core client for 30 sec - exiting 01:54:56 (9608): No heartbeat from core client for 30 sec - exiting 01:54:57 (9608): No heartbeat from core client for 30 sec - exiting 01:54:58 (9608): No heartbeat from core client for 30 sec - exiting 01:54:59 (9608): No heartbeat from core client for 30 sec - exiting 01:55:00 (9608): No heartbeat from core client for 30 sec - exiting 01:55:01 (9608): No heartbeat from core client for 30 sec - exiting 01:55:02 (9608): No heartbeat from core client for 30 sec - exiting 01:55:03 (9608): No heartbeat from core client for 30 sec - exiting 01:55:04 (9608): No heartbeat from core client for 30 sec - exiting 01:55:05 (9608): No heartbeat from core client for 30 sec - exiting 01:55:06 (9608): No heartbeat from core client for 30 sec - exiting 01:55:07 (9608): No heartbeat from core client for 30 sec - exiting 01:55:08 (9608): No heartbeat from core client for 30 sec - exiting 01:55:09 (9608): No heartbeat from core client for 30 sec - exiting 01:55:10 (9608): No heartbeat from core client for 30 sec - exiting 01:55:11 (9608): No heartbeat from core client for 30 sec - exiting 01:55:12 (9608): No heartbeat from core client for 30 sec - exiting 01:55:13 (9608): No heartbeat from core client for 30 sec - exiting 01:55:14 (9608): No heartbeat from core client for 30 sec - exiting 01:55:15 (9608): No heartbeat from core client for 30 sec - exiting 01:55:16 (9608): No heartbeat from core client for 30 sec - exiting 01:55:17 (9608): No heartbeat from core client for 30 sec - exiting 01:55:18 (9608): No heartbeat from core client for 30 sec - exiting 01:55:19 (9608): No heartbeat from core client for 30 sec - exiting 01:55:20 (9608): No heartbeat from core client for 30 sec - exiting 01:55:21 (9608): No heartbeat from core client for 30 sec - exiting 01:55:22 (9608): No heartbeat from core client for 30 sec - exiting 01:55:23 (9608): No heartbeat from core client for 30 sec - exiting 01:55:24 (9608): No heartbeat from core client for 30 sec - exiting 01:55:25 (9608): No heartbeat from core client for 30 sec - exiting 01:55:26 (9608): No heartbeat from core client for 30 sec - exiting 01:55:27 (9608): No heartbeat from core client for 30 sec - exiting 01:55:28 (9608): No heartbeat from core client for 30 sec - exiting 01:55:29 (9608): No heartbeat from core client for 30 sec - exiting 01:55:30 (9608): No heartbeat from core client for 30 sec - exiting 01:55:31 (9608): No heartbeat from core client for 30 sec - exiting 01:55:32 (9608): No heartbeat from core client for 30 sec - exiting 01:55:33 (9608): No heartbeat from core client for 30 sec - exiting 01:55:34 (9608): No heartbeat from core client for 30 sec - exiting 01:55:35 (9608): No heartbeat from core client for 30 sec - exiting 01:55:36 (9608): No heartbeat from core client for 30 sec - exiting 01:55:37 (9608): No heartbeat from core client for 30 sec - exiting 01:55:38 (9608): No heartbeat from core client for 30 sec - exiting 01:55:39 (9608): No heartbeat from core client for 30 sec - exiting 01:55:40 (9608): No heartbeat from core client for 30 sec - exiting 01:55:41 (9608): No heartbeat from core client for 30 sec - exiting 01:55:42 (9608): No heartbeat from core client for 30 sec - exiting 01:55:43 (9608): No heartbeat from core client for 30 sec - exiting 01:55:44 (9608): No heartbeat from core client for 30 sec - exiting 01:55:45 (9608): No heartbeat from core client for 30 sec - exiting 01:55:46 (9608): No heartbeat from core client for 30 sec - exiting 01:55:47 (9608): No heartbeat from core client for 30 sec - exiting 01:55:48 (9608): No heartbeat from core client for 30 sec - exiting 01:55:49 (9608): No heartbeat from core client for 30 sec - exiting 01:55:50 (9608): No heartbeat from core client for 30 sec - exiting 01:55:51 (9608): No heartbeat from core client for 30 sec - exiting 01:55:52 (9608): No heartbeat from core client for 30 sec - exiting 01:55:53 (9608): No heartbeat from core client for 30 sec - exiting 01:55:54 (9608): No heartbeat from core client for 30 sec - exiting 01:55:55 (9608): No heartbeat from core client for 30 sec - exiting 01:55:56 (9608): No heartbeat from core client for 30 sec - exiting 01:55:57 (9608): No heartbeat from core client for 30 sec - exiting 01:55:58 (9608): No heartbeat from core client for 30 sec - exiting 01:55:59 (9608): No heartbeat from core client for 30 sec - exiting 01:56:00 (9608): No heartbeat from core client for 30 sec - exiting 01:56:01 (9608): No heartbeat from core client for 30 sec - exiting 01:56:02 (9608): No heartbeat from core client for 30 sec - exiting 01:56:03 (9608): No heartbeat from core client for 30 sec - exiting 01:56:04 (9608): No heartbeat from core client for 30 sec - exiting 01:56:05 (9608): No heartbeat from core client for 30 sec - exiting 01:56:06 (9608): No heartbeat from core client for 30 sec - exiting 01:56:07 (9608): No heartbeat from core client for 30 sec - exiting 01:56:08 (9608): No heartbeat from core client for 30 sec - exiting 01:56:09 (9608): No heartbeat from core client for 30 sec - exiting 01:56:10 (9608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:56:11 (9608): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12948, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12948, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Nov 2013 01:51:44	1288885	16081650	hadcm3n_o1da_1940_40_008383314_2	155,520	210,179	1.3515
20 Nov 2013 03:45:18	1288885	16081650	hadcm3n_o1da_1940_40_008383314_2	129,600	175,100	1.3511
19 Nov 2013 16:50:59	1288885	16081650	hadcm3n_o1da_1940_40_008383314_2	103,680	139,029	1.3409
19 Nov 2013 05:58:19	1288885	16081650	hadcm3n_o1da_1940_40_008383314_2	77,760	103,135	1.3263
19 Nov 2013 02:37:46	1288885	16081650	hadcm3n_o1da_1940_40_008383314_2	51,840	70,162	1.3534
19 Nov 2013 02:37:46	1288885	16081650	hadcm3n_o1da_1940_40_008383314_2	25,920	39,030	1.5058