Task 15833351

Name	hadcm3n_4ay6_1980_40_008324992_2
Workunit	8476127
Created	6 Jun 2013, 21:27:07 UTC
Sent	6 Jun 2013, 21:36:04 UTC
Report deadline	6 Sep 2013, 5:03:15 UTC
Received	14 Aug 2013, 16:09:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1271137
Run time	21 days 13 hours 56 min 2 sec
CPU time	17 days 3 hours 46 min 37 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	1.99 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2956, iMonCtr=1 Model crash detected, will try to restart... 20:53:35 (3532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:21:12 (4804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:57:25 (3836): No heartbeat from core client for 30 sec - exiting 17:57:26 (3836): No heartbeat from core client for 30 sec - exiting 17:57:27 (3836): No heartbeat from core client for 30 sec - exiting 17:57:28 (3836): No heartbeat from core client for 30 sec - exiting 17:57:29 (3836): No heartbeat from core client for 30 sec - exiting 17:57:30 (3836): No heartbeat from core client for 30 sec - exiting 17:57:31 (3836): No heartbeat from core client for 30 sec - exiting 17:57:33 (3836): No heartbeat from core client for 30 sec - exiting 17:57:34 (3836): No heartbeat from core client for 30 sec - exiting 17:57:35 (3836): No heartbeat from core client for 30 sec - exiting 17:57:36 (3836): No heartbeat from core client for 30 sec - exiting 17:57:37 (3836): No heartbeat from core client for 30 sec - exiting 17:57:38 (3836): No heartbeat from core client for 30 sec - exiting 17:57:39 (3836): No heartbeat from core client for 30 sec - exiting 17:57:40 (3836): No heartbeat from core client for 30 sec - exiting 17:57:41 (3836): No heartbeat from core client for 30 sec - exiting 17:57:42 (3836): No heartbeat from core client for 30 sec - exiting 17:57:44 (3836): No heartbeat from core client for 30 sec - exiting 17:57:45 (3836): No heartbeat from core client for 30 sec - exiting 17:57:46 (3836): No heartbeat from core client for 30 sec - exiting 17:57:47 (3836): No heartbeat from core client for 30 sec - exiting 17:57:48 (3836): No heartbeat from core client for 30 sec - exiting 17:57:49 (3836): No heartbeat from core client for 30 sec - exiting 17:57:50 (3836): No heartbeat from core client for 30 sec - exiting 17:57:51 (3836): No heartbeat from core client for 30 sec - exiting 17:57:52 (3836): No heartbeat from core client for 30 sec - exiting 17:57:53 (3836): No heartbeat from core client for 30 sec - exiting 17:57:55 (3836): No heartbeat from core client for 30 sec - exiting 17:57:56 (3836): No heartbeat from core client for 30 sec - exiting 17:57:57 (3836): No heartbeat from core client for 30 sec - exiting 17:57:58 (3836): No heartbeat from core client for 30 sec - exiting 17:57:59 (3836): No heartbeat from core client for 30 sec - exiting 17:58:00 (3836): No heartbeat from core client for 30 sec - exiting 17:58:01 (3836): No heartbeat from core client for 30 sec - exiting 17:58:02 (3836): No heartbeat from core client for 30 sec - exiting 17:58:03 (3836): No heartbeat from core client for 30 sec - exiting 17:58:04 (3836): No heartbeat from core client for 30 sec - exiting 17:58:05 (3836): No heartbeat from core client for 30 sec - exiting 17:58:07 (3836): No heartbeat from core client for 30 sec - exiting 17:58:08 (3836): No heartbeat from core client for 30 sec - exiting 17:58:09 (3836): No heartbeat from core client for 30 sec - exiting 17:58:10 (3836): No heartbeat from core client for 30 sec - exiting 17:58:11 (3836): No heartbeat from core client for 30 sec - exiting 17:58:12 (3836): No heartbeat from core client for 30 sec - exiting 17:58:13 (3836): No heartbeat from core client for 30 sec - exiting 17:58:15 (3836): No heartbeat from core client for 30 sec - exiting 17:58:16 (3836): No heartbeat from core client for 30 sec - exiting 17:58:17 (3836): No heartbeat from core client for 30 sec - exiting 17:58:18 (3836): No heartbeat from core client for 30 sec - exiting 17:58:19 (3836): No heartbeat from core client for 30 sec - exiting 17:58:20 (3836): No heartbeat from core client for 30 sec - exiting 17:58:21 (3836): No heartbeat from core client for 30 sec - exiting 17:58:22 (3836): No heartbeat from core client for 30 sec - exiting 17:58:23 (3836): No heartbeat from core client for 30 sec - exiting 17:58:24 (3836): No heartbeat from core client for 30 sec - exiting 17:58:26 (3836): No heartbeat from core client for 30 sec - exiting 17:58:27 (3836): No heartbeat from core client for 30 sec - exiting 17:58:28 (3836): No heartbeat from core client for 30 sec - exiting 17:58:29 (3836): No heartbeat from core client for 30 sec - exiting 17:58:30 (3836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3472, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 17:59:14 (3432): No heartbeat from core client for 30 sec - exiting 17:59:15 (3432): No heartbeat from core client for 30 sec - exiting 17:59:16 (3432): No heartbeat from core client for 30 sec - exiting 17:59:17 (3432): No heartbeat from core client for 30 sec - exiting 17:59:18 (3432): No heartbeat from core client for 30 sec - exiting 17:59:19 (3432): No heartbeat from core client for 30 sec - exiting 17:59:20 (3432): No heartbeat from core client for 30 sec - exiting 17:59:21 (3432): No heartbeat from core client for 30 sec - exiting 17:59:23 (3432): No heartbeat from core client for 30 sec - exiting 17:59:24 (3432): No heartbeat from core client for 30 sec - exiting 17:59:25 (3432): No heartbeat from core client for 30 sec - exiting 17:59:26 (3432): No heartbeat from core client for 30 sec - exiting 17:59:27 (3432): No heartbeat from core client for 30 sec - exiting 17:59:28 (3432): No heartbeat from core client for 30 sec - exiting 17:59:29 (3432): No heartbeat from core client for 30 sec - exiting 17:59:30 (3432): No heartbeat from core client for 30 sec - exiting 17:59:32 (3432): No heartbeat from core client for 30 sec - exiting 17:59:33 (3432): No heartbeat from core client for 30 sec - exiting 17:59:34 (3432): No heartbeat from core client for 30 sec - exiting 17:59:35 (3432): No heartbeat from core client for 30 sec - exiting 17:59:37 (3432): No heartbeat from core client for 30 sec - exiting 17:59:38 (3432): No heartbeat from core client for 30 sec - exiting 17:59:39 (3432): No heartbeat from core client for 30 sec - exiting 17:59:40 (3432): No heartbeat from core client for 30 sec - exiting 17:59:41 (3432): No heartbeat from core client for 30 sec - exiting 17:59:43 (3432): No heartbeat from core client for 30 sec - exiting 17:59:44 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:00:27 (3112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:29:00 (3364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:43:47 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:47:48 (3040): No heartbeat from core client for 30 sec - exiting 18:47:49 (3040): No heartbeat from core client for 30 sec - exiting 18:47:50 (3040): No heartbeat from core client for 30 sec - exiting 18:47:51 (3040): No heartbeat from core client for 30 sec - exiting 18:47:52 (3040): No heartbeat from core client for 30 sec - exiting 18:47:54 (3040): No heartbeat from core client for 30 sec - exiting 18:47:55 (3040): No heartbeat from core client for 30 sec - exiting 18:47:56 (3040): No heartbeat from core client for 30 sec - exiting 18:47:57 (3040): No heartbeat from core client for 30 sec - exiting 18:47:58 (3040): No heartbeat from core client for 30 sec - exiting 18:47:59 (3040): No heartbeat from core client for 30 sec - exiting 18:48:00 (3040): No heartbeat from core client for 30 sec - exiting 18:48:01 (3040): No heartbeat from core client for 30 sec - exiting 18:48:03 (3040): No heartbeat from core client for 30 sec - exiting 18:48:04 (3040): No heartbeat from core client for 30 sec - exiting 18:48:05 (3040): No heartbeat from core client for 30 sec - exiting 18:48:06 (3040): No heartbeat from core client for 30 sec - exiting 18:48:07 (3040): No heartbeat from core client for 30 sec - exiting 18:48:08 (3040): No heartbeat from core client for 30 sec - exiting 18:48:09 (3040): No heartbeat from core client for 30 sec - exiting 18:48:10 (3040): No heartbeat from core client for 30 sec - exiting 18:48:11 (3040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:30:46 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4756, iMonCtr=1 Model crash detected, will try to restart... 07:44:00 (3968): No heartbeat from core client for 30 sec - exiting 07:44:01 (3968): No heartbeat from core client for 30 sec - exiting 07:44:03 (3968): No heartbeat from core client for 30 sec - exiting 07:44:04 (3968): No heartbeat from core client for 30 sec - exiting 07:44:05 (3968): No heartbeat from core client for 30 sec - exiting 07:44:06 (3968): No heartbeat from core client for 30 sec - exiting 07:44:07 (3968): No heartbeat from core client for 30 sec - exiting 07:44:09 (3968): No heartbeat from core client for 30 sec - exiting 07:44:10 (3968): No heartbeat from core client for 30 sec - exiting 07:44:11 (3968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:00:53 (3260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:25 (3396): No heartbeat from core client for 30 sec - exiting 09:13:26 (3396): No heartbeat from core client for 30 sec - exiting 09:13:27 (3396): No heartbeat from core client for 30 sec - exiting 09:13:28 (3396): No heartbeat from core client for 30 sec - exiting 09:13:29 (3396): No heartbeat from core client for 30 sec - exiting 09:13:30 (3396): No heartbeat from core client for 30 sec - exiting 09:13:31 (3396): No heartbeat from core client for 30 sec - exiting 09:13:32 (3396): No heartbeat from core client for 30 sec - exiting 09:13:33 (3396): No heartbeat from core client for 30 sec - exiting 09:13:34 (3396): No heartbeat from core client for 30 sec - exiting 09:13:36 (3396): No heartbeat from core client for 30 sec - exiting 09:13:37 (3396): No heartbeat from core client for 30 sec - exiting 09:13:38 (3396): No heartbeat from core client for 30 sec - exiting 09:13:39 (3396): No heartbeat from core client for 30 sec - exiting 09:13:40 (3396): No heartbeat from core client for 30 sec - exiting 09:13:41 (3396): No heartbeat from core client for 30 sec - exiting 09:13:42 (3396): No heartbeat from core client for 30 sec - exiting 09:13:43 (3396): No heartbeat from core client for 30 sec - exiting 09:13:44 (3396): No heartbeat from core client for 30 sec - exiting 09:13:45 (3396): No heartbeat from core client for 30 sec - exiting 09:13:47 (3396): No heartbeat from core client for 30 sec - exiting 09:13:48 (3396): No heartbeat from core client for 30 sec - exiting 09:13:49 (3396): No heartbeat from core client for 30 sec - exiting 09:13:50 (3396): No heartbeat from core client for 30 sec - exiting 09:13:51 (3396): No heartbeat from core client for 30 sec - exiting 09:13:52 (3396): No heartbeat from core client for 30 sec - exiting 09:13:53 (3396): No heartbeat from core client for 30 sec - exiting 09:13:54 (3396): No heartbeat from core client for 30 sec - exiting 09:13:55 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:34:12 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:13:59 (4192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:56:09 (2292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:16:04 (3024): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:18:56 (3012): No heartbeat from core client for 30 sec - exiting 06:18:57 (3012): No heartbeat from core client for 30 sec - exiting 06:18:58 (3012): No heartbeat from core client for 30 sec - exiting 06:18:59 (3012): No heartbeat from core client for 30 sec - exiting 06:19:00 (3012): No heartbeat from core client for 30 sec - exiting 06:19:01 (3012): No heartbeat from core client for 30 sec - exiting 06:19:02 (3012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:03:50 (2952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:09:25 (2652): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:59:07 (2604): No heartbeat from core client for 30 sec - exiting 05:59:08 (2604): No heartbeat from core client for 30 sec - exiting 05:59:09 (2604): No heartbeat from core client for 30 sec - exiting 05:59:10 (2604): No heartbeat from core client for 30 sec - exiting 05:59:12 (2604): No heartbeat from core client for 30 sec - exiting 05:59:13 (2604): No heartbeat from core client for 30 sec - exiting 05:59:14 (2604): No heartbeat from core client for 30 sec - exiting 05:59:15 (2604): No heartbeat from core client for 30 sec - exiting 05:59:16 (2604): No heartbeat from core client for 30 sec - exiting 05:59:17 (2604): No heartbeat from core client for 30 sec - exiting 05:59:18 (2604): No heartbeat from core client for 30 sec - exiting 05:59:19 (2604): No heartbeat from core client for 30 sec - exiting 05:59:20 (2604): No heartbeat from core client for 30 sec - exiting 05:59:21 (2604): No heartbeat from core client for 30 sec - exiting 05:59:22 (2604): No heartbeat from core client for 30 sec - exiting 05:59:24 (2604): No heartbeat from core client for 30 sec - exiting 05:59:25 (2604): No heartbeat from core client for 30 sec - exiting 05:59:26 (2604): No heartbeat from core client for 30 sec - exiting 05:59:27 (2604): No heartbeat from core client for 30 sec - exiting 05:59:28 (2604): No heartbeat from core client for 30 sec - exiting 05:59:29 (2604): No heartbeat from core client for 30 sec - exiting 05:59:30 (2604): No heartbeat from core client for 30 sec - exiting 05:59:31 (2604): No heartbeat from core client for 30 sec - exiting 05:59:32 (2604): No heartbeat from core client for 30 sec - exiting 05:59:33 (2604): No heartbeat from core client for 30 sec - exiting 05:59:35 (2604): No heartbeat from core client for 30 sec - exiting 05:59:36 (2604): No heartbeat from core client for 30 sec - exiting 05:59:37 (2604): No heartbeat from core client for 30 sec - exiting 05:59:38 (2604): No heartbeat from core client for 30 sec - exiting 05:59:39 (2604): No heartbeat from core client for 30 sec - exiting 05:59:40 (2604): No heartbeat from core client for 30 sec - exiting 05:59:41 (2604): No heartbeat from core client for 30 sec - exiting 05:59:42 (2604): No heartbeat from core client for 30 sec - exiting 05:59:43 (2604): No heartbeat from core client for 30 sec - exiting 05:59:45 (2604): No heartbeat from core client for 30 sec - exiting 05:59:46 (2604): No heartbeat from core client for 30 sec - exiting 05:59:47 (2604): No heartbeat from core client for 30 sec - exiting 05:59:48 (2604): No heartbeat from core client for 30 sec - exiting 05:59:49 (2604): No heartbeat from core client for 30 sec - exiting 05:59:51 (2604): No heartbeat from core client for 30 sec - exiting 05:59:53 (2604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:03:43 (3260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2456, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3768, iMonCtr=1 Model crash detected, will try to restart... 10:52:42 (3396): No heartbeat from core client for 30 sec - exiting 10:52:43 (3396): No heartbeat from core client for 30 sec - exiting 10:52:44 (3396): No heartbeat from core client for 30 sec - exiting 10:52:45 (3396): No heartbeat from core client for 30 sec - exiting 10:52:46 (3396): No heartbeat from core client for 30 sec - exiting 10:52:47 (3396): No heartbeat from core client for 30 sec - exiting 10:52:48 (3396): No heartbeat from core client for 30 sec - exiting 10:52:49 (3396): No heartbeat from core client for 30 sec - exiting 10:52:50 (3396): No heartbeat from core client for 30 sec - exiting 10:52:52 (3396): No heartbeat from core client for 30 sec - exiting 10:52:53 (3396): No heartbeat from core client for 30 sec - exiting 10:52:54 (3396): No heartbeat from core client for 30 sec - exiting 10:52:55 (3396): No heartbeat from core client for 30 sec - exiting 10:52:56 (3396): No heartbeat from core client for 30 sec - exiting 10:52:57 (3396): No heartbeat from core client for 30 sec - exiting 10:52:58 (3396): No heartbeat from core client for 30 sec - exiting 10:52:59 (3396): No heartbeat from core client for 30 sec - exiting 10:53:00 (3396): No heartbeat from core client for 30 sec - exiting 10:53:01 (3396): No heartbeat from core client for 30 sec - exiting 10:53:03 (3396): No heartbeat from core client for 30 sec - exiting 10:53:04 (3396): No heartbeat from core client for 30 sec - exiting 10:53:05 (3396): No heartbeat from core client for 30 sec - exiting 10:53:06 (3396): No heartbeat from core client for 30 sec - exiting 10:53:07 (3396): No heartbeat from core client for 30 sec - exiting 10:53:08 (3396): No heartbeat from core client for 30 sec - exiting 10:53:09 (3396): No heartbeat from core client for 30 sec - exiting 10:53:10 (3396): No heartbeat from core client for 30 sec - exiting 10:53:11 (3396): No heartbeat from core client for 30 sec - exiting 10:53:12 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4664, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3512, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 06:18:35 (2284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:46:14 (2116): No heartbeat from core client for 30 sec - exiting 06:46:15 (2116): No heartbeat from core client for 30 sec - exiting 06:46:16 (2116): No heartbeat from core client for 30 sec - exiting 06:46:17 (2116): No heartbeat from core client for 30 sec - exiting 06:46:18 (2116): No heartbeat from core client for 30 sec - exiting 06:46:20 (2116): No heartbeat from core client for 30 sec - exiting 06:46:21 (2116): No heartbeat from core client for 30 sec - exiting 06:46:22 (2116): No heartbeat from core client for 30 sec - exiting 06:46:23 (2116): No heartbeat from core client for 30 sec - exiting 06:46:24 (2116): No heartbeat from core client for 30 sec - exiting 06:46:25 (2116): No heartbeat from core client for 30 sec - exiting 06:46:26 (2116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:51:18 (3540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:16:04 (3508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3260, iMonCtr=1 Model crash detected, will try to restart... 15:41:14 (3836): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4124, iMonCtr=1 Model crash detected, will try to restart... 06:17:55 (3588): No heartbeat from core client for 30 sec - exiting 06:17:56 (3588): No heartbeat from core client for 30 sec - exiting 06:17:58 (3588): No heartbeat from core client for 30 sec - exiting 06:17:59 (3588): No heartbeat from core client for 30 sec - exiting 06:18:00 (3588): No heartbeat from core client for 30 sec - exiting 06:18:01 (3588): No heartbeat from core client for 30 sec - exiting 06:18:02 (3588): No heartbeat from core client for 30 sec - exiting 06:18:03 (3588): No heartbeat from core client for 30 sec - exiting 06:18:04 (3588): No heartbeat from core client for 30 sec - exiting 06:18:05 (3588): No heartbeat from core client for 30 sec - exiting 06:18:06 (3588): No heartbeat from core client for 30 sec - exiting 06:18:07 (3588): No heartbeat from core client for 30 sec - exiting 06:18:08 (3588): No heartbeat from core client for 30 sec - exiting 06:18:10 (3588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:49:45 (3740): No heartbeat from core client for 30 sec - exiting 17:49:46 (3740): No heartbeat from core client for 30 sec - exiting 17:49:47 (3740): No heartbeat from core client for 30 sec - exiting 17:49:48 (3740): No heartbeat from core client for 30 sec - exiting 17:49:49 (3740): No heartbeat from core client for 30 sec - exiting 17:49:50 (3740): No heartbeat from core client for 30 sec - exiting 17:49:52 (3740): No heartbeat from core client for 30 sec - exiting 17:49:53 (3740): No heartbeat from core client for 30 sec - exiting 17:49:54 (3740): No heartbeat from core client for 30 sec - exiting 17:49:55 (3740): No heartbeat from core client for 30 sec - exiting 17:49:56 (3740): No heartbeat from core client for 30 sec - exiting 17:49:57 (3740): No heartbeat from core client for 30 sec - exiting 17:49:58 (3740): No heartbeat from core client for 30 sec - exiting 17:49:59 (3740): No heartbeat from core client for 30 sec - exiting 17:50:00 (3740): No heartbeat from core client for 30 sec - exiting 17:50:01 (3740): No heartbeat from core client for 30 sec - exiting 17:50:02 (3740): No heartbeat from core client for 30 sec - exiting 17:50:03 (3740): No heartbeat from core client for 30 sec - exiting 17:50:04 (3740): No heartbeat from core client for 30 sec - exiting 17:50:05 (3740): No heartbeat from core client for 30 sec - exiting 17:50:06 (3740): No heartbeat from core client for 30 sec - exiting 17:50:07 (3740): No heartbeat from core client for 30 sec - exiting 17:50:08 (3740): No heartbeat from core client for 30 sec - exiting 17:50:09 (3740): No heartbeat from core client for 30 sec - exiting 17:50:10 (3740): No heartbeat from core client for 30 sec - exiting 17:50:11 (3740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:52:29 (3672): No heartbeat from core client for 30 sec - exiting 17:52:30 (3672): No heartbeat from core client for 30 sec - exiting 17:52:31 (3672): No heartbeat from core client for 30 sec - exiting 17:52:33 (3672): No heartbeat from core client for 30 sec - exiting 17:52:34 (3672): No heartbeat from core client for 30 sec - exiting 17:52:35 (3672): No heartbeat from core client for 30 sec - exiting 17:52:36 (3672): No heartbeat from core client for 30 sec - exiting 17:52:37 (3672): No heartbeat from core client for 30 sec - exiting 17:52:38 (3672): No heartbeat from core client for 30 sec - exiting 17:52:39 (3672): No heartbeat from core client for 30 sec - exiting 17:52:40 (3672): No heartbeat from core client for 30 sec - exiting 17:52:41 (3672): No heartbeat from core client for 30 sec - exiting 17:52:42 (3672): No heartbeat from core client for 30 sec - exiting 17:52:43 (3672): No heartbeat from core client for 30 sec - exiting 17:52:44 (3672): No heartbeat from core client for 30 sec - exiting 17:52:45 (3672): No heartbeat from core client for 30 sec - exiting 17:52:46 (3672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 05:43:24 (3320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:43:55 (2260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:43:15 (3232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x771C3760 read attempt to address 0x3DEEA1D5 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x771C71F3 read attempt to address 0x3DEEA1D5 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Aug 2013 16:18:16	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	518,400	1,482,524	2.8598
14 Aug 2013 16:18:16	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	492,480	1,408,580	2.8602
14 Aug 2013 16:18:16	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	466,560	1,334,170	2.8596
14 Aug 2013 16:18:16	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	440,640	1,258,623	2.8564
14 Aug 2013 16:18:16	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	414,720	1,182,610	2.8516
14 Aug 2013 16:18:16	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	388,800	1,107,930	2.8496
14 Aug 2013 16:18:16	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	362,880	1,034,250	2.8501
23 Jul 2013 21:15:40	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	336,960	959,291	2.8469
23 Jul 2013 20:10:59	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	311,040	886,504	2.8501
23 Jul 2013 17:45:11	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	285,120	812,108	2.8483
07 Jul 2013 19:32:42	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	259,200	738,928	2.8508
06 Jul 2013 05:42:27	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	233,280	663,367	2.8437
02 Jul 2013 10:52:42	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	207,360	589,798	2.8443
02 Jul 2013 09:58:43	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	181,440	517,556	2.8525
27 Jun 2013 17:28:32	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	155,520	443,274	2.8503
23 Jun 2013 03:04:18	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	129,600	368,582	2.8440
20 Jun 2013 02:55:29	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	103,680	294,068	2.8363
16 Jun 2013 14:23:14	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	77,760	220,042	2.8298
15 Jun 2013 08:04:24	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	51,840	147,381	2.8430
11 Jun 2013 22:25:32	1271137	15833351	hadcm3n_4ay6_1980_40_008324992_2	25,920	74,455	2.8725