Task 15793076

Name	hadcm3n_3822_1940_40_008264145_4
Workunit	8419269
Created	22 May 2013, 8:00:49 UTC
Sent	22 May 2013, 8:00:56 UTC
Report deadline	21 Aug 2013, 15:28:07 UTC
Received	18 Jul 2013, 12:26:49 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	781392
Run time	20 days 16 hours 22 min 56 sec
CPU time	14 days 2 hours 10 min 45 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	1.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=66612, iMonCtr=1 Model crash detected, will try to restart... 09:57:21 (4204): No heartbeat from core client for 30 sec - exiting 09:57:22 (4204): No heartbeat from core client for 30 sec - exiting 09:57:23 (4204): No heartbeat from core client for 30 sec - exiting 09:57:24 (4204): No heartbeat from core client for 30 sec - exiting 09:57:25 (4204): No heartbeat from core client for 30 sec - exiting 09:57:26 (4204): No heartbeat from core client for 30 sec - exiting 09:57:27 (4204): No heartbeat from core client for 30 sec - exiting 09:57:28 (4204): No heartbeat from core client for 30 sec - exiting 09:57:30 (4204): No heartbeat from core client for 30 sec - exiting 09:57:31 (4204): No heartbeat from core client for 30 sec - exiting 09:57:32 (4204): No heartbeat from core client for 30 sec - exiting 09:57:33 (4204): No heartbeat from core client for 30 sec - exiting 09:57:34 (4204): No heartbeat from core client for 30 sec - exiting 09:57:35 (4204): No heartbeat from core client for 30 sec - exiting 09:57:36 (4204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2720, iMonCtr=1 Model crash detected, will try to restart... 12:54:39 (4220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:22:30 (5104): No heartbeat from core client for 30 sec - exiting 11:22:31 (5104): No heartbeat from core client for 30 sec - exiting 11:22:32 (5104): No heartbeat from core client for 30 sec - exiting 11:22:34 (5104): No heartbeat from core client for 30 sec - exiting 11:22:35 (5104): No heartbeat from core client for 30 sec - exiting 11:22:36 (5104): No heartbeat from core client for 30 sec - exiting 11:22:37 (5104): No heartbeat from core client for 30 sec - exiting 11:22:38 (5104): No heartbeat from core client for 30 sec - exiting 11:22:39 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2520, iMonCtr=1 Model crash detected, will try to restart... 11:18:47 (4812): No heartbeat from core client for 30 sec - exiting 11:18:49 (4812): No heartbeat from core client for 30 sec - exiting 11:18:50 (4812): No heartbeat from core client for 30 sec - exiting 11:18:51 (4812): No heartbeat from core client for 30 sec - exiting 11:18:52 (4812): No heartbeat from core client for 30 sec - exiting 11:18:53 (4812): No heartbeat from core client for 30 sec - exiting 11:18:54 (4812): No heartbeat from core client for 30 sec - exiting 11:18:55 (4812): No heartbeat from core client for 30 sec - exiting 11:18:56 (4812): No heartbeat from core client for 30 sec - exiting 11:18:57 (4812): No heartbeat from core client for 30 sec - exiting 11:18:58 (4812): No heartbeat from core client for 30 sec - exiting 11:19:00 (4812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:19:01 (4812): No heartbeat from core client for 30 sec - exiting 12:47:00 (5804): No heartbeat from core client for 30 sec - exiting 12:47:01 (5804): No heartbeat from core client for 30 sec - exiting 12:47:02 (5804): No heartbeat from core client for 30 sec - exiting 12:47:03 (5804): No heartbeat from core client for 30 sec - exiting 12:47:04 (5804): No heartbeat from core client for 30 sec - exiting 12:47:05 (5804): No heartbeat from core client for 30 sec - exiting 12:47:06 (5804): No heartbeat from core client for 30 sec - exiting 12:47:07 (5804): No heartbeat from core client for 30 sec - exiting 12:47:08 (5804): No heartbeat from core client for 30 sec - exiting 12:47:09 (5804): No heartbeat from core client for 30 sec - exiting 12:47:10 (5804): No heartbeat from core client for 30 sec - exiting 12:47:11 (5804): No heartbeat from core client for 30 sec - exiting 12:47:12 (5804): No heartbeat from core client for 30 sec - exiting 12:47:13 (5804): No heartbeat from core client for 30 sec - exiting 12:47:14 (5804): No heartbeat from core client for 30 sec - exiting 12:47:16 (5804): No heartbeat from core client for 30 sec - exiting 12:47:17 (5804): No heartbeat from core client for 30 sec - exiting 12:47:18 (5804): No heartbeat from core client for 30 sec - exiting 12:47:19 (5804): No heartbeat from core client for 30 sec - exiting 12:47:20 (5804): No heartbeat from core client for 30 sec - exiting 12:47:21 (5804): No heartbeat from core client for 30 sec - exiting 12:47:22 (5804): No heartbeat from core client for 30 sec - exiting 12:47:23 (5804): No heartbeat from core client for 30 sec - exiting 12:47:24 (5804): No heartbeat from core client for 30 sec - exiting 12:47:25 (5804): No heartbeat from core client for 30 sec - exiting 12:47:26 (5804): No heartbeat from core client for 30 sec - exiting 12:47:27 (5804): No heartbeat from core client for 30 sec - exiting 12:47:28 (5804): No heartbeat from core client for 30 sec - exiting 12:47:29 (5804): No heartbeat from core client for 30 sec - exiting 12:47:30 (5804): No heartbeat from core client for 30 sec - exiting 12:47:31 (5804): No heartbeat from core client for 30 sec - exiting 12:47:32 (5804): No heartbeat from core client for 30 sec - exiting 12:47:33 (5804): No heartbeat from core client for 30 sec - exiting 12:47:34 (5804): No heartbeat from core client for 30 sec - exiting 12:47:35 (5804): No heartbeat from core client for 30 sec - exiting 12:47:36 (5804): No heartbeat from core client for 30 sec - exiting 12:47:37 (5804): No heartbeat from core client for 30 sec - exiting 12:47:38 (5804): No heartbeat from core client for 30 sec - exiting 12:47:39 (5804): No heartbeat from core client for 30 sec - exiting 12:47:40 (5804): No heartbeat from core client for 30 sec - exiting 12:47:41 (5804): No heartbeat from core client for 30 sec - exiting 12:47:42 (5804): No heartbeat from core client for 30 sec - exiting 12:47:43 (5804): No heartbeat from core client for 30 sec - exiting 12:47:44 (5804): No heartbeat from core client for 30 sec - exiting 12:47:45 (5804): No heartbeat from core client for 30 sec - exiting 12:47:46 (5804): No heartbeat from core client for 30 sec - exiting 12:47:47 (5804): No heartbeat from core client for 30 sec - exiting 12:47:48 (5804): No heartbeat from core client for 30 sec - exiting 12:47:49 (5804): No heartbeat from core client for 30 sec - exiting 12:47:50 (5804): No heartbeat from core client for 30 sec - exiting 12:47:51 (5804): No heartbeat from core client for 30 sec - exiting 12:47:52 (5804): No heartbeat from core client for 30 sec - exiting 12:47:53 (5804): No heartbeat from core client for 30 sec - exiting 12:47:54 (5804): No heartbeat from core client for 30 sec - exiting 12:47:55 (5804): No heartbeat from core client for 30 sec - exiting 12:47:56 (5804): No heartbeat from core client for 30 sec - exiting 12:47:57 (5804): No heartbeat from core client for 30 sec - exiting 12:47:58 (5804): No heartbeat from core client for 30 sec - exiting 12:47:59 (5804): No heartbeat from core client for 30 sec - exiting 12:48:00 (5804): No heartbeat from core client for 30 sec - exiting 12:48:01 (5804): No heartbeat from core client for 30 sec - exiting 12:48:02 (5804): No heartbeat from core client for 30 sec - exiting 12:48:03 (5804): No heartbeat from core client for 30 sec - exiting 12:48:04 (5804): No heartbeat from core client for 30 sec - exiting 12:48:05 (5804): No heartbeat from core client for 30 sec - exiting 12:48:06 (5804): No heartbeat from core client for 30 sec - exiting 12:48:07 (5804): No heartbeat from core client for 30 sec - exiting 12:48:08 (5804): No heartbeat from core client for 30 sec - exiting 12:48:09 (5804): No heartbeat from core client for 30 sec - exiting 12:48:10 (5804): No heartbeat from core client for 30 sec - exiting 12:48:11 (5804): No heartbeat from core client for 30 sec - exiting 12:48:12 (5804): No heartbeat from core client for 30 sec - exiting 12:48:13 (5804): No heartbeat from core client for 30 sec - exiting 12:48:14 (5804): No heartbeat from core client for 30 sec - exiting 12:48:15 (5804): No heartbeat from core client for 30 sec - exiting 12:48:16 (5804): No heartbeat from core client for 30 sec - exiting 12:48:17 (5804): No heartbeat from core client for 30 sec - exiting 12:48:18 (5804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4560, iMonCtr=1 Model crash detected, will try to restart... 11:18:45 (4652): No heartbeat from core client for 30 sec - exiting 11:18:46 (4652): No heartbeat from core client for 30 sec - exiting 11:18:47 (4652): No heartbeat from core client for 30 sec - exiting 11:18:48 (4652): No heartbeat from core client for 30 sec - exiting 11:18:49 (4652): No heartbeat from core client for 30 sec - exiting 11:18:50 (4652): No heartbeat from core client for 30 sec - exiting 11:18:51 (4652): No heartbeat from core client for 30 sec - exiting 11:18:52 (4652): No heartbeat from core client for 30 sec - exiting 11:18:54 (4652): No heartbeat from core client for 30 sec - exiting 11:18:55 (4652): No heartbeat from core client for 30 sec - exiting 11:18:56 (4652): No heartbeat from core client for 30 sec - exiting 11:18:57 (4652): No heartbeat from core client for 30 sec - exiting 11:18:58 (4652): No heartbeat from core client for 30 sec - exiting 11:18:59 (4652): No heartbeat from core client for 30 sec - exiting 11:19:00 (4652): No heartbeat from core client for 30 sec - exiting 11:19:01 (4652): No heartbeat from core client for 30 sec - exiting 11:19:02 (4652): No heartbeat from core client for 30 sec - exiting 11:19:03 (4652): No heartbeat from core client for 30 sec - exiting 11:19:04 (4652): No heartbeat from core client for 30 sec - exiting 11:19:05 (4652): No heartbeat from core client for 30 sec - exiting 11:19:06 (4652): No heartbeat from core client for 30 sec - exiting 11:19:07 (4652): No heartbeat from core client for 30 sec - exiting 11:19:08 (4652): No heartbeat from core client for 30 sec - exiting 11:19:09 (4652): No heartbeat from core client for 30 sec - exiting 11:19:11 (4652): No heartbeat from core client for 30 sec - exiting 11:19:12 (4652): No heartbeat from core client for 30 sec - exiting 11:19:13 (4652): No heartbeat from core client for 30 sec - exiting 11:19:14 (4652): No heartbeat from core client for 30 sec - exiting 11:19:15 (4652): No heartbeat from core client for 30 sec - exiting 11:19:16 (4652): No heartbeat from core client for 30 sec - exiting 11:19:17 (4652): No heartbeat from core client for 30 sec - exiting 11:19:18 (4652): No heartbeat from core client for 30 sec - exiting 11:19:19 (4652): No heartbeat from core client for 30 sec - exiting 11:19:20 (4652): No heartbeat from core client for 30 sec - exiting 11:19:21 (4652): No heartbeat from core client for 30 sec - exiting 11:19:22 (4652): No heartbeat from core client for 30 sec - exiting 11:19:23 (4652): No heartbeat from core client for 30 sec - exiting 11:19:24 (4652): No heartbeat from core client for 30 sec - exiting 11:19:25 (4652): No heartbeat from core client for 30 sec - exiting 11:19:26 (4652): No heartbeat from core client for 30 sec - exiting 11:19:27 (4652): No heartbeat from core client for 30 sec - exiting 11:19:28 (4652): No heartbeat from core client for 30 sec - exiting 11:19:29 (4652): No heartbeat from core client for 30 sec - exiting 11:19:30 (4652): No heartbeat from core client for 30 sec - exiting 11:19:31 (4652): No heartbeat from core client for 30 sec - exiting 11:19:32 (4652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:09:00 (5696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:09:02 (5696): No heartbeat from core client for 30 sec - exiting 15:09:03 (5696): No heartbeat from core client for 30 sec - exiting 15:09:04 (5696): No heartbeat from core client for 30 sec - exiting 10:42:19 (7232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... 12:26:15 (4400): No heartbeat from core client for 30 sec - exiting 12:26:17 (4400): No heartbeat from core client for 30 sec - exiting 12:26:18 (4400): No heartbeat from core client for 30 sec - exiting 12:26:19 (4400): No heartbeat from core client for 30 sec - exiting 12:26:20 (4400): No heartbeat from core client for 30 sec - exiting 12:26:21 (4400): No heartbeat from core client for 30 sec - exiting 12:26:22 (4400): No heartbeat from core client for 30 sec - exiting 12:26:23 (4400): No heartbeat from core client for 30 sec - exiting 12:26:24 (4400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:13:00 (764): No heartbeat from core client for 30 sec - exiting 11:13:01 (764): No heartbeat from core client for 30 sec - exiting 11:13:02 (764): No heartbeat from core client for 30 sec - exiting 11:13:03 (764): No heartbeat from core client for 30 sec - exiting 11:13:04 (764): No heartbeat from core client for 30 sec - exiting 11:13:05 (764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:36:56 (4352): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:36:57 (4352): No heartbeat from core client for 30 sec - exiting 11:39:07 (5152): No heartbeat from core client for 30 sec - exiting 11:39:08 (5152): No heartbeat from core client for 30 sec - exiting 11:39:09 (5152): No heartbeat from core client for 30 sec - exiting 11:39:10 (5152): No heartbeat from core client for 30 sec - exiting 11:39:11 (5152): No heartbeat from core client for 30 sec - exiting 11:39:12 (5152): No heartbeat from core client for 30 sec - exiting 11:39:13 (5152): No heartbeat from core client for 30 sec - exiting 11:39:14 (5152): No heartbeat from core client for 30 sec - exiting 11:39:15 (5152): No heartbeat from core client for 30 sec - exiting 11:39:16 (5152): No heartbeat from core client for 30 sec - exiting 11:39:17 (5152): No heartbeat from core client for 30 sec - exiting 11:39:18 (5152): No heartbeat from core client for 30 sec - exiting 11:39:19 (5152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:05:47 (5192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:05:49 (5192): No heartbeat from core client for 30 sec - exiting 15:05:50 (5192): No heartbeat from core client for 30 sec - exiting 15:05:51 (5192): No heartbeat from core client for 30 sec - exiting 15:07:10 (6472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4752, iMonCtr=1 Model crash detected, will try to restart... 09:58:06 (4660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5424, iMonCtr=1 Model crash detected, will try to restart... 13:23:44 (5084): No heartbeat from core client for 30 sec - exiting 13:23:45 (5084): No heartbeat from core client for 30 sec - exiting 13:23:46 (5084): No heartbeat from core client for 30 sec - exiting 13:23:47 (5084): No heartbeat from core client for 30 sec - exiting 13:23:48 (5084): No heartbeat from core client for 30 sec - exiting 13:23:50 (5084): No heartbeat from core client for 30 sec - exiting 13:23:50 (5084): No heartbeat from core client for 30 sec - exiting 13:23:51 (5084): No heartbeat from core client for 30 sec - exiting 13:23:53 (5084): No heartbeat from core client for 30 sec - exiting 13:23:54 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... C09:04:47 (5896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:55 (1548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:24:57 (1548): No heartbeat from core client for 30 sec - exiting 10:24:58 (1548): No heartbeat from core client for 30 sec - exiting 10:27:46 (10404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:38 (11676): No heartbeat from core client for 30 sec - exiting 10:28:39 (11676): No heartbeat from core client for 30 sec - exiting 10:28:40 (11676): No heartbeat from core client for 30 sec - exiting 10:28:41 (11676): No heartbeat from core client for 30 sec - exiting 10:28:42 (11676): No heartbeat from core client for 30 sec - exiting 10:28:43 (11676): No heartbeat from core client for 30 sec - exiting 10:28:44 (11676): No heartbeat from core client for 30 sec - exiting 10:28:45 (11676): No heartbeat from core client for 30 sec - exiting 10:28:46 (11676): No heartbeat from core client for 30 sec - exiting 10:28:47 (11676): No heartbeat from core client for 30 sec - exiting 10:28:48 (11676): No heartbeat from core client for 30 sec - exiting 10:28:49 (11676): No heartbeat from core client for 30 sec - exiting 10:28:50 (11676): No heartbeat from core client for 30 sec - exiting 10:28:51 (11676): No heartbeat from core client for 30 sec - exiting 10:28:52 (11676): No heartbeat from core client for 30 sec - exiting 10:28:53 (11676): No heartbeat from core client for 30 sec - exiting 10:28:54 (11676): No heartbeat from core client for 30 sec - exiting 10:28:55 (11676): No heartbeat from core client for 30 sec - exiting 10:28:56 (11676): No heartbeat from core client for 30 sec - exiting 10:28:57 (11676): No heartbeat from core client for 30 sec - exiting 10:28:58 (11676): No heartbeat from core client for 30 sec - exiting 10:28:59 (11676): No heartbeat from core client for 30 sec - exiting 10:29:00 (11676): No heartbeat from core client for 30 sec - exiting 10:29:01 (11676): No heartbeat from core client for 30 sec - exiting 10:29:02 (11676): No heartbeat from core client for 30 sec - exiting 10:29:03 (11676): No heartbeat from core client for 30 sec - exiting 10:29:04 (11676): No heartbeat from core client for 30 sec - exiting 10:29:05 (11676): No heartbeat from core client for 30 sec - exiting 10:29:06 (11676): No heartbeat from core client for 30 sec - exiting 10:29:07 (11676): No heartbeat from core client for 30 sec - exiting 10:29:08 (11676): No heartbeat from core client for 30 sec - exiting 10:29:09 (11676): No heartbeat from core client for 30 sec - exiting 10:29:10 (11676): No heartbeat from core client for 30 sec - exiting 10:29:11 (11676): No heartbeat from core client for 30 sec - exiting 10:29:12 (11676): No heartbeat from core client for 30 sec - exiting 10:29:13 (11676): No heartbeat from core client for 30 sec - exiting 10:29:14 (11676): No heartbeat from core client for 30 sec - exiting 10:29:15 (11676): No heartbeat from core client for 30 sec - exiting 10:29:16 (11676): No heartbeat from core client for 30 sec - exiting 10:29:17 (11676): No heartbeat from core client for 30 sec - exiting 10:29:18 (11676): No heartbeat from core client for 30 sec - exiting 10:29:19 (11676): No heartbeat from core client for 30 sec - exiting 10:29:20 (11676): No heartbeat from core client for 30 sec - exiting 10:29:21 (11676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:59:44 (3780): No heartbeat from core client for 30 sec - exiting 11:59:46 (3780): No heartbeat from core client for 30 sec - exiting 11:59:47 (3780): No heartbeat from core client for 30 sec - exiting 11:59:48 (3780): No heartbeat from core client for 30 sec - exiting 11:59:49 (3780): No heartbeat from core client for 30 sec - exiting 11:59:50 (3780): No heartbeat from core client for 30 sec - exiting 11:59:51 (3780): No heartbeat from core client for 30 sec - exiting 11:59:52 (3780): No heartbeat from core client for 30 sec - exiting 11:59:53 (3780): No heartbeat from core client for 30 sec - exiting 11:59:54 (3780): No heartbeat from core client for 30 sec - exiting 11:59:55 (3780): No heartbeat from core client for 30 sec - exiting 11:59:56 (3780): No heartbeat from core client for 30 sec - exiting 11:59:57 (3780): No heartbeat from core client for 30 sec - exiting 11:59:58 (3780): No heartbeat from core client for 30 sec - exiting 11:59:59 (3780): No heartbeat from core client for 30 sec - exiting 12:00:00 (3780): No heartbeat from core client for 30 sec - exiting 12:00:01 (3780): No heartbeat from core client for 30 sec - exiting 12:00:03 (3780): No heartbeat from core client for 30 sec - exiting 12:00:04 (3780): No heartbeat from core client for 30 sec - exiting 12:00:05 (3780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:00:06 (3780): No heartbeat from core client for 30 sec - exiting 15:05:17 (8000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:05:18 (8000): No heartbeat from core client for 30 sec - exiting 15:05:58 (10600): No heartbeat from core client for 30 sec - exiting 15:05:59 (10600): No heartbeat from core client for 30 sec - exiting 15:06:00 (10600): No heartbeat from core client for 30 sec - exiting 15:06:01 (10600): No heartbeat from core client for 30 sec - exiting 15:06:02 (10600): No heartbeat from core client for 30 sec - exiting 15:06:03 (10600): No heartbeat from core client for 30 sec - exiting 15:06:04 (10600): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:23 (12240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:24 (12240): No heartbeat from core client for 30 sec - exiting 12:15:25 (12240): No heartbeat from core client for 30 sec - exiting 12:15:26 (12240): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11500, iMonCtr=1 Model crash detected, will try to restart... 13:55:33 (66868): No heartbeat from core client for 30 sec - exiting 13:55:34 (66868): No heartbeat from core client for 30 sec - exiting 13:55:35 (66868): No heartbeat from core client for 30 sec - exiting 13:55:36 (66868): No heartbeat from core client for 30 sec - exiting 13:55:37 (66868): No heartbeat from core client for 30 sec - exiting 13:55:38 (66868): No heartbeat from core client for 30 sec - exiting 13:55:39 (66868): No heartbeat from core client for 30 sec - exiting 13:55:40 (66868): No heartbeat from core client for 30 sec - exiting 13:55:41 (66868): No heartbeat from core client for 30 sec - exiting 13:55:42 (66868): No heartbeat from core client for 30 sec - exiting 13:55:44 (66868): No heartbeat from core client for 30 sec - exiting 13:55:45 (66868): No heartbeat from core client for 30 sec - exiting 13:55:46 (66868): No heartbeat from core client for 30 sec - exiting 13:55:47 (66868): No heartbeat from core client for 30 sec - exiting 13:55:48 (66868): No heartbeat from core client for 30 sec - exiting 13:55:49 (66868): No heartbeat from core client for 30 sec - exiting 13:55:50 (66868): No heartbeat from core client for 30 sec - exiting 13:55:51 (66868): No heartbeat from core client for 30 sec - exiting 13:55:52 (66868): No heartbeat from core client for 30 sec - exiting 13:55:53 (66868): No heartbeat from core client for 30 sec - exiting 13:55:54 (66868): No heartbeat from core client for 30 sec - exiting 13:55:55 (66868): No heartbeat from core client for 30 sec - exiting 13:55:56 (66868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:48:14 (3852): No heartbeat from core client for 30 sec - exiting 10:48:15 (3852): No heartbeat from core client for 30 sec - exiting 10:48:16 (3852): No heartbeat from core client for 30 sec - exiting 10:48:17 (3852): No heartbeat from core client for 30 sec - exiting 10:48:18 (3852): No heartbeat from core client for 30 sec - exiting 10:48:19 (3852): No heartbeat from core client for 30 sec - exiting 10:48:20 (3852): No heartbeat from core client for 30 sec - exiting 10:48:21 (3852): No heartbeat from core client for 30 sec - exiting 10:48:22 (3852): No heartbeat from core client for 30 sec - exiting 10:48:23 (3852): No heartbeat from core client for 30 sec - exiting 10:48:24 (3852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:16 (5988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:33:18 (5988): No heartbeat from core client for 30 sec - exiting 11:33:19 (5988): No heartbeat from core client for 30 sec - exiting 11:33:20 (5988): No heartbeat from core client for 30 sec - exiting 11:34:05 (4608): No heartbeat from core client for 30 sec - exiting 11:34:06 (4608): No heartbeat from core client for 30 sec - exiting 11:34:07 (4608): No heartbeat from core client for 30 sec - exiting 11:34:08 (4608): No heartbeat from core client for 30 sec - exiting 11:34:09 (4608): No heartbeat from core client for 30 sec - exiting 11:34:10 (4608): No heartbeat from core client for 30 sec - exiting 11:34:11 (4608): No heartbeat from core client for 30 sec - exiting 11:34:12 (4608): No heartbeat from core client for 30 sec - exiting 11:34:13 (4608): No heartbeat from core client for 30 sec - exiting 11:34:14 (4608): No heartbeat from core client for 30 sec - exiting 11:34:15 (4608): No heartbeat from core client for 30 sec - exiting 11:34:16 (4608): No heartbeat from core client for 30 sec - exiting 11:34:17 (4608): No heartbeat from core client for 30 sec - exiting 11:34:18 (4608): No heartbeat from core client for 30 sec - exiting 11:34:19 (4608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:36:41 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:37:23 (2312): No heartbeat from core client for 30 sec - exiting 11:37:24 (2312): No heartbeat from core client for 30 sec - exiting 11:37:25 (2312): No heartbeat from core client for 30 sec - exiting 11:37:26 (2312): No heartbeat from core client for 30 sec - exiting 11:37:27 (2312): No heartbeat from core client for 30 sec - exiting 11:37:28 (2312): No heartbeat from core client for 30 sec - exiting 11:37:29 (2312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5016, iMonCtr=1 Model crash detected, will try to restart... 11:51:30 (5096): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:29 (4312): No heartbeat from core client for 30 sec - exiting 11:34:30 (4312): No heartbeat from core client for 30 sec - exiting 11:34:31 (4312): No heartbeat from core client for 30 sec - exiting 11:34:32 (4312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:12:52 (5980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:12:54 (5980): No heartbeat from core client for 30 sec - exiting 03:22:09 (253024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:22:10 (253024): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=253332, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5176, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4212, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4436, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3912, iMonCtr=1 Model crash detected, will try to restart... 20:32:51 (4172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:32:52 (4172): No heartbeat from core client for 30 sec - exiting 22:18:33 (2260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11552, iMonCtr=1 Model crash detected, will try to restart... 16:38:00 (4776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:38:02 (4776): No heartbeat from core client for 30 sec - exiting 16:38:03 (4776): No heartbeat from core client for 30 sec - exiting 18:43:21 (95832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=95816, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5172, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5504, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4400, iMonCtr=1 Model crash detected, will try to restart... 15:48:27 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=276, iMonCtr=1 Model crash detected, will try to restart... 11:34:20 (4864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:34:22 (4864): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4136, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4552, iMonCtr=1 Model crash detected, will try to restart... 15:54:51 (2548): No heartbeat from core client for 30 sec - exiting 15:54:52 (2548): No heartbeat from core client for 30 sec - exiting 15:54:53 (2548): No heartbeat from core client for 30 sec - exiting 15:54:54 (2548): No heartbeat from core client for 30 sec - exiting 15:54:55 (2548): No heartbeat from core client for 30 sec - exiting 15:54:56 (2548): No heartbeat from core client for 30 sec - exiting 15:54:57 (2548): No heartbeat from core client for 30 sec - exiting 15:54:58 (2548): No heartbeat from core client for 30 sec - exiting 15:55:00 (2548): No heartbeat from core client for 30 sec - exiting 15:55:01 (2548): No heartbeat from core client for 30 sec - exiting 15:55:02 (2548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5940, iMonCtr=1 Model crash detected, will try to restart... 19:31:51 (5400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:10:22 (5632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:12:17 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:12:18 (5532): No heartbeat from core client for 30 sec - exiting 16:12:19 (5532): No heartbeat from core client for 30 sec - exiting 16:13:37 (1104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:23 (4396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:51:25 (4396): No heartbeat from core client for 30 sec - exiting 17:51:26 (4396): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6208, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4892, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Jul 2013 15:52:50	781392	15793076	hadcm3n_3822_1940_40_008264145_4	518,400	1,217,440	2.3485
23 Jul 2013 15:52:50	781392	15793076	hadcm3n_3822_1940_40_008264145_4	492,480	1,154,010	2.3433
23 Jul 2013 15:52:50	781392	15793076	hadcm3n_3822_1940_40_008264145_4	466,560	1,090,535	2.3374
10 Jul 2013 14:52:10	781392	15793076	hadcm3n_3822_1940_40_008264145_4	440,640	1,026,789	2.3302
09 Jul 2013 08:26:55	781392	15793076	hadcm3n_3822_1940_40_008264145_4	414,720	963,836	2.3241
08 Jul 2013 04:41:24	781392	15793076	hadcm3n_3822_1940_40_008264145_4	388,800	902,515	2.3213
07 Jul 2013 02:16:50	781392	15793076	hadcm3n_3822_1940_40_008264145_4	362,880	840,047	2.3149
21 Jun 2013 13:36:12	781392	15793076	hadcm3n_3822_1940_40_008264145_4	336,960	778,980	2.3118
20 Jun 2013 15:31:06	781392	15793076	hadcm3n_3822_1940_40_008264145_4	311,040	717,554	2.3070
18 Jun 2013 04:16:46	781392	15793076	hadcm3n_3822_1940_40_008264145_4	285,120	653,824	2.2932
14 Jun 2013 20:06:56	781392	15793076	hadcm3n_3822_1940_40_008264145_4	259,200	592,692	2.2866
12 Jun 2013 05:04:27	781392	15793076	hadcm3n_3822_1940_40_008264145_4	233,280	531,080	2.2766
11 Jun 2013 08:28:04	781392	15793076	hadcm3n_3822_1940_40_008264145_4	207,360	473,519	2.2836
10 Jun 2013 14:04:57	781392	15793076	hadcm3n_3822_1940_40_008264145_4	181,440	415,855	2.2920
08 Jun 2013 23:20:56	781392	15793076	hadcm3n_3822_1940_40_008264145_4	155,520	358,359	2.3043
05 Jun 2013 10:56:16	781392	15793076	hadcm3n_3822_1940_40_008264145_4	129,600	298,918	2.3065
03 Jun 2013 17:52:43	781392	15793076	hadcm3n_3822_1940_40_008264145_4	103,680	238,720	2.3025
02 Jun 2013 20:52:57	781392	15793076	hadcm3n_3822_1940_40_008264145_4	77,760	178,631	2.2972
30 May 2013 16:44:04	781392	15793076	hadcm3n_3822_1940_40_008264145_4	51,840	120,712	2.3285
23 May 2013 21:02:57	781392	15793076	hadcm3n_3822_1940_40_008264145_4	25,920	60,263	2.3250