Task 16590140

Name	hadcm3n_8c8o_1980_40_008725283_0
Workunit	8871261
Created	23 Apr 2014, 13:26:01 UTC
Sent	1 May 2014, 7:42:04 UTC
Report deadline	31 Jul 2014, 15:09:15 UTC
Received	10 Jun 2014, 20:05:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1290168
Run time	7 days 2 hours 48 min 32 sec
CPU time	6 days 3 hours 13 min 6 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:12:38 (3896): No heartbeat from core client for 30 sec - exiting 16:12:39 (3896): No heartbeat from core client for 30 sec - exiting 16:12:40 (3896): No heartbeat from core client for 30 sec - exiting 16:12:41 (3896): No heartbeat from core client for 30 sec - exiting 16:12:42 (3896): No heartbeat from core client for 30 sec - exiting 16:12:43 (3896): No heartbeat from core client for 30 sec - exiting 16:12:44 (3896): No heartbeat from core client for 30 sec - exiting 16:12:45 (3896): No heartbeat from core client for 30 sec - exiting 16:12:47 (3896): No heartbeat from core client for 30 sec - exiting 16:12:48 (3896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:22:44 (4008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:23:35 (2380): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:19:04 (1052): No heartbeat from core client for 30 sec - exiting 19:19:05 (1052): No heartbeat from core client for 30 sec - exiting 19:19:06 (1052): No heartbeat from core client for 30 sec - exiting 19:19:07 (1052): No heartbeat from core client for 30 sec - exiting 19:19:09 (1052): No heartbeat from core client for 30 sec - exiting 19:19:10 (1052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:07:13 (3084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:07:55 (4404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:39:11 (3620): No heartbeat from core client for 30 sec - exiting 18:39:12 (3620): No heartbeat from core client for 30 sec - exiting 18:39:13 (3620): No heartbeat from core client for 30 sec - exiting 18:39:15 (3620): No heartbeat from core client for 30 sec - exiting 18:39:16 (3620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:42:41 (5104): No heartbeat from core client for 30 sec - exiting 09:42:43 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3828, iMonCtr=1 Model crash detected, will try to restart... 19:18:30 (3160): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:35:48 (3152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:36:29 (4204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:41:22 (3108): No heartbeat from core client for 30 sec - exiting 15:41:23 (3108): No heartbeat from core client for 30 sec - exiting 15:41:24 (3108): No heartbeat from core client for 30 sec - exiting 15:41:26 (3108): No heartbeat from core client for 30 sec - exiting 15:41:27 (3108): No heartbeat from core client for 30 sec - exiting 15:41:28 (3108): No heartbeat from core client for 30 sec - exiting 15:41:29 (3108): No heartbeat from core client for 30 sec - exiting 15:41:30 (3108): No heartbeat from core client for 30 sec - exiting 15:41:31 (3108): No heartbeat from core client for 30 sec - exiting 15:41:32 (3108): No heartbeat from core client for 30 sec - exiting 15:41:33 (3108): No heartbeat from core client for 30 sec - exiting 15:41:34 (3108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:23:18 (5092): No heartbeat from core client for 30 sec - exiting 20:23:20 (5092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:49:06 (3768): No heartbeat from core client for 30 sec - exiting 06:49:07 (3768): No heartbeat from core client for 30 sec - exiting 06:49:08 (3768): No heartbeat from core client for 30 sec - exiting 06:49:09 (3768): No heartbeat from core client for 30 sec - exiting 06:49:10 (3768): No heartbeat from core client for 30 sec - exiting 06:49:11 (3768): No heartbeat from core client for 30 sec - exiting 06:49:12 (3768): No heartbeat from core client for 30 sec - exiting 06:49:13 (3768): No heartbeat from core client for 30 sec - exiting 06:49:14 (3768): No heartbeat from core client for 30 sec - exiting 06:49:15 (3768): No heartbeat from core client for 30 sec - exiting 06:49:17 (3768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:49:18 (3768): No heartbeat from core client for 30 sec - exiting 06:49:19 (3768): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:14:16 (3536): No heartbeat from core client for 30 sec - exiting 17:14:17 (3536): No heartbeat from core client for 30 sec - exiting 17:14:18 (3536): No heartbeat from core client for 30 sec - exiting 17:14:19 (3536): No heartbeat from core client for 30 sec - exiting 17:14:20 (3536): No heartbeat from core client for 30 sec - exiting 17:14:22 (3536): No heartbeat from core client for 30 sec - exiting 17:14:23 (3536): No heartbeat from core client for 30 sec - exiting 17:14:24 (3536): No heartbeat from core client for 30 sec - exiting 17:14:25 (3536): No heartbeat from core client for 30 sec - exiting 17:14:26 (3536): No heartbeat from core client for 30 sec - exiting 17:14:27 (3536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:59:12 (4044): No heartbeat from core client for 30 sec - exiting 15:59:13 (4044): No heartbeat from core client for 30 sec - exiting 15:59:14 (4044): No heartbeat from core client for 30 sec - exiting 15:59:16 (4044): No heartbeat from core client for 30 sec - exiting 15:59:17 (4044): No heartbeat from core client for 30 sec - exiting 15:59:18 (4044): No heartbeat from core client for 30 sec - exiting 15:59:19 (4044): No heartbeat from core client for 30 sec - exiting 15:59:20 (4044): No heartbeat from core client for 30 sec - exiting 15:59:21 (4044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:03:33 (4080): No heartbeat from core client for 30 sec - exiting 21:03:35 (4080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:55:38 (2172): No heartbeat from core client for 30 sec - exiting 06:55:39 (2172): No heartbeat from core client for 30 sec - exiting 06:55:41 (2172): No heartbeat from core client for 30 sec - exiting 06:55:42 (2172): No heartbeat from core client for 30 sec - exiting 06:55:43 (2172): No heartbeat from core client for 30 sec - exiting 06:55:44 (2172): No heartbeat from core client for 30 sec - exiting 06:55:45 (2172): No heartbeat from core client for 30 sec - exiting 06:55:46 (2172): No heartbeat from core client for 30 sec - exiting 06:55:47 (2172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:55:48 (2172): No heartbeat from core client for 30 sec - exiting 06:56:27 (4888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:26:55 (3824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4244, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:35:06 (3432): No heartbeat from core client for 30 sec - exiting 18:35:07 (3432): No heartbeat from core client for 30 sec - exiting 18:35:08 (3432): No heartbeat from core client for 30 sec - exiting 18:35:09 (3432): No heartbeat from core client for 30 sec - exiting 18:35:10 (3432): No heartbeat from core client for 30 sec - exiting 18:35:11 (3432): No heartbeat from core client for 30 sec - exiting 18:35:12 (3432): No heartbeat from core client for 30 sec - exiting 18:35:13 (3432): No heartbeat from core client for 30 sec - exiting 18:35:14 (3432): No heartbeat from core client for 30 sec - exiting 18:35:15 (3432): No heartbeat from core client for 30 sec - exiting 18:35:49 (3432): No heartbeat from core client for 30 sec - exiting 18:35:50 (3432): No heartbeat from core client for 30 sec - exiting 18:35:51 (3432): No heartbeat from core client for 30 sec - exiting 18:35:53 (3432): No heartbeat from core client for 30 sec - exiting 18:35:54 (3432): No heartbeat from core client for 30 sec - exiting 18:35:55 (3432): No heartbeat from core client for 30 sec - exiting 18:35:56 (3432): No heartbeat from core client for 30 sec - exiting 18:35:57 (3432): No heartbeat from core client for 30 sec - exiting 18:35:58 (3432): No heartbeat from core client for 30 sec - exiting 18:35:59 (3432): No heartbeat from core client for 30 sec - exiting 18:36:00 (3432): No heartbeat from core client for 30 sec - exiting 18:36:01 (3432): No heartbeat from core client for 30 sec - exiting 18:36:02 (3432): No heartbeat from core client for 30 sec - exiting 18:36:03 (3432): No heartbeat from core client for 30 sec - exiting 18:36:05 (3432): No heartbeat from core client for 30 sec - exiting 18:36:06 (3432): No heartbeat from core client for 30 sec - exiting 18:36:07 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:36:53 (2432): No heartbeat from core client for 30 sec - exiting 09:36:54 (2432): No heartbeat from core client for 30 sec - exiting 09:36:55 (2432): No heartbeat from core client for 30 sec - exiting 09:36:56 (2432): No heartbeat from core client for 30 sec - exiting 09:36:58 (2432): No heartbeat from core client for 30 sec - exiting 09:36:59 (2432): No heartbeat from core client for 30 sec - exiting 09:37:00 (2432): No heartbeat from core client for 30 sec - exiting 09:37:01 (2432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:53:13 (3316): No heartbeat from core client for 30 sec - exiting 12:53:14 (3316): No heartbeat from core client for 30 sec - exiting 12:53:15 (3316): No heartbeat from core client for 30 sec - exiting 12:53:16 (3316): No heartbeat from core client for 30 sec - exiting 12:53:17 (3316): No heartbeat from core client for 30 sec - exiting 12:53:18 (3316): No heartbeat from core client for 30 sec - exiting 12:53:19 (3316): No heartbeat from core client for 30 sec - exiting 12:53:21 (3316): No heartbeat from core client for 30 sec - exiting 12:53:22 (3316): No heartbeat from core client for 30 sec - exiting 12:53:23 (3316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:25:14 (1260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3184, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... 06:41:33 (3464): No heartbeat from core client for 30 sec - exiting 06:41:34 (3464): No heartbeat from core client for 30 sec - exiting 06:41:35 (3464): No heartbeat from core client for 30 sec - exiting 06:41:36 (3464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 21:46:33 (4320): No heartbeat from core client for 30 sec - exiting 21:46:34 (4320): No heartbeat from core client for 30 sec - exiting 21:46:35 (4320): No heartbeat from core client for 30 sec - exiting 21:46:36 (4320): No heartbeat from core client for 30 sec - exiting 21:46:37 (4320): No heartbeat from core client for 30 sec - exiting 21:46:38 (4320): No heartbeat from core client for 30 sec - exiting 21:46:39 (4320): No heartbeat from core client for 30 sec - exiting 21:46:40 (4320): No heartbeat from core client for 30 sec - exiting 21:46:41 (4320): No heartbeat from core client for 30 sec - exiting 21:46:42 (4320): No heartbeat from core client for 30 sec - exiting 21:46:43 (4320): No heartbeat from core client for 30 sec - exiting 21:46:44 (4320): No heartbeat from core client for 30 sec - exiting 21:46:45 (4320): No heartbeat from core client for 30 sec - exiting 21:46:46 (4320): No heartbeat from core client for 30 sec - exiting 21:46:47 (4320): No heartbeat from core client for 30 sec - exiting 21:46:48 (4320): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4416, iMonCtr=1 Model crash detected, will try to restart... 19:26:32 (3340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 08:34:12 (3264): No heartbeat from core client for 30 sec - exiting 08:34:13 (3264): No heartbeat from core client for 30 sec - exiting 08:34:14 (3264): No heartbeat from core client for 30 sec - exiting 08:34:15 (3264): No heartbeat from core client for 30 sec - exiting 08:34:17 (3264): No heartbeat from core client for 30 sec - exiting 08:34:18 (3264): No heartbeat from core client for 30 sec - exiting 08:34:19 (3264): No heartbeat from core client for 30 sec - exiting 08:34:20 (3264): No heartbeat from core client for 30 sec - exiting 08:34:21 (3264): No heartbeat from core client for 30 sec - exiting 08:34:22 (3264): No heartbeat from core client for 30 sec - exiting 08:34:23 (3264): No heartbeat from core client for 30 sec - exiting 08:34:24 (3264): No heartbeat from core client for 30 sec - exiting 08:34:25 (3264): No heartbeat from core client for 30 sec - exiting 08:34:26 (3264): No heartbeat from core client for 30 sec - exiting 08:34:27 (3264): No heartbeat from core client for 30 sec - exiting 08:34:29 (3264): No heartbeat from core client for 30 sec - exiting 08:34:30 (3264): No heartbeat from core client for 30 sec - exiting 08:34:31 (3264): No heartbeat from core client for 30 sec - exiting 08:34:32 (3264): No heartbeat from core client for 30 sec - exiting 08:34:33 (3264): No heartbeat from core client for 30 sec - exiting 08:34:34 (3264): No heartbeat from core client for 30 sec - exiting 08:34:35 (3264): No heartbeat from core client for 30 sec - exiting 08:34:36 (3264): No heartbeat from core client for 30 sec - exiting 08:34:37 (3264): No heartbeat from core client for 30 sec - exiting 08:34:38 (3264): No heartbeat from core client for 30 sec - exiting 08:34:39 (3264): No heartbeat from core client for 30 sec - exiting 08:34:41 (3264): No heartbeat from core client for 30 sec - exiting 08:34:42 (3264): No heartbeat from core client for 30 sec - exiting 08:34:43 (3264): No heartbeat from core client for 30 sec - exiting 08:34:44 (3264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 18:24:25 (3856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:45:26 (3308): No heartbeat from core client for 30 sec - exiting 09:45:27 (3308): No heartbeat from core client for 30 sec - exiting 09:45:28 (3308): No heartbeat from core client for 30 sec - exiting 09:45:29 (3308): No heartbeat from core client for 30 sec - exiting 09:45:30 (3308): No heartbeat from core client for 30 sec - exiting 09:45:31 (3308): No heartbeat from core client for 30 sec - exiting 09:45:32 (3308): No heartbeat from core client for 30 sec - exiting 09:45:34 (3308): No heartbeat from core client for 30 sec - exiting 09:45:35 (3308): No heartbeat from core client for 30 sec - exiting 09:45:36 (3308): No heartbeat from core client for 30 sec - exiting 09:45:37 (3308): No heartbeat from core client for 30 sec - exiting 09:45:38 (3308): No heartbeat from core client for 30 sec - exiting 09:45:39 (3308): No heartbeat from core client for 30 sec - exiting 09:45:40 (3308): No heartbeat from core client for 30 sec - exiting 09:45:41 (3308): No heartbeat from core client for 30 sec - exiting 09:45:42 (3308): No heartbeat from core client for 30 sec - exiting 09:45:43 (3308): No heartbeat from core client for 30 sec - exiting 09:45:44 (3308): No heartbeat from core client for 30 sec - exiting 09:45:46 (3308): No heartbeat from core client for 30 sec - exiting 09:45:47 (3308): No heartbeat from core client for 30 sec - exiting 09:45:48 (3308): No heartbeat from core client for 30 sec - exiting 09:45:49 (3308): No heartbeat from core client for 30 sec - exiting 09:45:50 (3308): No heartbeat from core client for 30 sec - exiting 09:45:51 (3308): No heartbeat from core client for 30 sec - exiting 09:45:52 (3308): No heartbeat from core client for 30 sec - exiting 09:45:53 (3308): No heartbeat from core client for 30 sec - exiting 09:45:54 (3308): No heartbeat from core client for 30 sec - exiting 09:45:55 (3308): No heartbeat from core client for 30 sec - exiting 09:45:56 (3308): No heartbeat from core client for 30 sec - exiting 09:45:58 (3308): No heartbeat from core client for 30 sec - exiting 09:45:59 (3308): No heartbeat from core client for 30 sec - exiting 09:46:00 (3308): No heartbeat from core client for 30 sec - exiting 09:46:01 (3308): No heartbeat from core client for 30 sec - exiting 09:46:02 (3308): No heartbeat from core client for 30 sec - exiting 09:46:03 (3308): No heartbeat from core client for 30 sec - exiting 09:46:04 (3308): No heartbeat from core client for 30 sec - exiting 09:46:05 (3308): No heartbeat from core client for 30 sec - exiting 09:46:06 (3308): No heartbeat from core client for 30 sec - exiting 09:46:07 (3308): No heartbeat from core client for 30 sec - exiting 09:46:08 (3308): No heartbeat from core client for 30 sec - exiting 09:46:10 (3308): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:02:49 (892): No heartbeat from core client for 30 sec - exiting 21:02:50 (892): No heartbeat from core client for 30 sec - exiting 21:02:52 (892): No heartbeat from core client for 30 sec - exiting 21:02:53 (892): No heartbeat from core client for 30 sec - exiting 21:02:54 (892): No heartbeat from core client for 30 sec - exiting 21:02:55 (892): No heartbeat from core client for 30 sec - exiting 21:02:56 (892): No heartbeat from core client for 30 sec - exiting 21:02:57 (892): No heartbeat from core client for 30 sec - exiting 21:02:58 (892): No heartbeat from core client for 30 sec - exiting 21:02:59 (892): No heartbeat from core client for 30 sec - exiting 21:03:00 (892): No heartbeat from core client for 30 sec - exiting 21:03:01 (892): No heartbeat from core client for 30 sec - exiting 21:03:03 (892): No heartbeat from core client for 30 sec - exiting 21:03:04 (892): No heartbeat from core client for 30 sec - exiting 21:03:05 (892): No heartbeat from core client for 30 sec - exiting 21:03:06 (892): No heartbeat from core client for 30 sec - exiting 21:03:07 (892): No heartbeat from core client for 30 sec - exiting 21:03:08 (892): No heartbeat from core client for 30 sec - exiting 21:03:09 (892): No heartbeat from core client for 30 sec - exiting 21:03:10 (892): No heartbeat from core client for 30 sec - exiting 21:03:11 (892): No heartbeat from core client for 30 sec - exiting 21:03:12 (892): No heartbeat from core client for 30 sec - exiting 21:03:13 (892): No heartbeat from core client for 30 sec - exiting 21:03:15 (892): No heartbeat from core client for 30 sec - exiting 21:03:16 (892): No heartbeat from core client for 30 sec - exiting 21:03:17 (892): No heartbeat from core client for 30 sec - exiting 21:03:18 (892): No heartbeat from core client for 30 sec - exiting 21:03:19 (892): No heartbeat from core client for 30 sec - exiting 21:03:20 (892): No heartbeat from core client for 30 sec - exiting 21:03:21 (892): No heartbeat from core client for 30 sec - exiting 21:03:22 (892): No heartbeat from core client for 30 sec - exiting 21:03:23 (892): No heartbeat from core client for 30 sec - exiting 21:03:24 (892): No heartbeat from core client for 30 sec - exiting 21:03:25 (892): No heartbeat from core client for 30 sec - exiting 21:03:26 (892): No heartbeat from core client for 30 sec - exiting 21:03:27 (892): No heartbeat from core client for 30 sec - exiting 21:03:28 (892): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:03:29 (892): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:47:37 (4448): No heartbeat from core client for 30 sec - exiting 19:47:38 (4448): No heartbeat from core client for 30 sec - exiting 19:47:39 (4448): No heartbeat from core client for 30 sec - exiting 19:47:40 (4448): No heartbeat from core client for 30 sec - exiting 19:47:41 (4448): No heartbeat from core client for 30 sec - exiting 19:47:42 (4448): No heartbeat from core client for 30 sec - exiting 19:47:43 (4448): No heartbeat from core client for 30 sec - exiting 19:47:44 (4448): No heartbeat from core client for 30 sec - exiting 19:47:45 (4448): No heartbeat from core client for 30 sec - exiting 19:47:47 (4448): No heartbeat from core client for 30 sec - exiting 19:47:48 (4448): No heartbeat from core client for 30 sec - exiting 19:47:49 (4448): No heartbeat from core client for 30 sec - exiting 19:47:50 (4448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:25 (4820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:57:40 (3764): No heartbeat from core client for 30 sec - exiting 11:57:41 (3764): No heartbeat from core client for 30 sec - exiting 11:57:43 (3764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:34:04 (3608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:34:55 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 20:55:03 (3828): No heartbeat from core client for 30 sec - exiting 20:55:04 (3828): No heartbeat from core client for 30 sec - exiting 20:55:05 (3828): No heartbeat from core client for 30 sec - exiting 20:55:07 (3828): No heartbeat from core client for 30 sec - exiting 20:55:08 (3828): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:33:42 (2928): No heartbeat from core client for 30 sec - exiting 21:33:43 (2928): No heartbeat from core client for 30 sec - exiting 21:33:44 (2928): No heartbeat from core client for 30 sec - exiting 21:33:46 (2928): No heartbeat from core client for 30 sec - exiting 21:33:47 (2928): No heartbeat from core client for 30 sec - exiting 21:33:48 (2928): No heartbeat from core client for 30 sec - exiting 21:33:49 (2928): No heartbeat from core client for 30 sec - exiting 21:33:50 (2928): No heartbeat from core client for 30 sec - exiting 21:33:51 (2928): No heartbeat from core client for 30 sec - exiting 21:33:52 (2928): No heartbeat from core client for 30 sec - exiting 21:33:53 (2928): No heartbeat from core client for 30 sec - exiting 21:33:54 (2928): No heartbeat from core client for 30 sec - exiting 21:33:55 (2928): No heartbeat from core client for 30 sec - exiting 21:33:56 (2928): No heartbeat from core client for 30 sec - exiting 21:33:58 (2928): No heartbeat from core client for 30 sec - exiting 21:33:59 (2928): No heartbeat from core client for 30 sec - exiting 21:34:00 (2928): No heartbeat from core client for 30 sec - exiting 21:34:01 (2928): No heartbeat from core client for 30 sec - exiting 21:34:02 (2928): No heartbeat from core client for 30 sec - exiting 21:34:03 (2928): No heartbeat from core client for 30 sec - exiting 21:34:04 (2928): No heartbeat from core client for 30 sec - exiting 21:34:05 (2928): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:34:06 (2928): No heartbeat from core client for 30 sec - exiting 21:34:07 (2928): No heartbeat from core client for 30 sec - exiting 21:34:09 (2928): No heartbeat from core client for 30 sec - exiting 21:34:10 (2928): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:36:25 (1604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:36:26 (1604): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... 20:40:58 (3584): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:58:34 (3612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 16:29:10 (3236): No heartbeat from core client for 30 sec - exiting 16:29:11 (3236): No heartbeat from core client for 30 sec - exiting 16:29:12 (3236): No heartbeat from core client for 30 sec - exiting 16:29:13 (3236): No heartbeat from core client for 30 sec - exiting 16:29:14 (3236): No heartbeat from core client for 30 sec - exiting 16:29:15 (3236): No heartbeat from core client for 30 sec - exiting 16:29:16 (3236): No heartbeat from core client for 30 sec - exiting 16:29:17 (3236): No heartbeat from core client for 30 sec - exiting 16:29:19 (3236): No heartbeat from core client for 30 sec - exiting 16:29:20 (3236): No heartbeat from core client for 30 sec - exiting 16:29:21 (3236): No heartbeat from core client for 30 sec - exiting 16:29:22 (3236): No heartbeat from core client for 30 sec - exiting 16:29:23 (3236): No heartbeat from core client for 30 sec - exiting 16:29:24 (3236): No heartbeat from core client for 30 sec - exiting 16:29:25 (3236): No heartbeat from core client for 30 sec - exiting 16:29:26 (3236): No heartbeat from core client for 30 sec - exiting 16:29:27 (3236): No heartbeat from core client for 30 sec - exiting 16:29:28 (3236): No heartbeat from core client for 30 sec - exiting 16:29:30 (3236): No heartbeat from core client for 30 sec - exiting 16:29:31 (3236): No heartbeat from core client for 30 sec - exiting 16:29:32 (3236): No heartbeat from core client for 30 sec - exiting 16:29:33 (3236): No heartbeat from core client for 30 sec - exiting 16:29:34 (3236): No heartbeat from core client for 30 sec - exiting 16:29:35 (3236): No heartbeat from core client for 30 sec - exiting 16:29:36 (3236): No heartbeat from core client for 30 sec - exiting 16:29:37 (3236): No heartbeat from core client for 30 sec - exiting 16:29:38 (3236): No heartbeat from core client for 30 sec - exiting 16:29:39 (3236): No heartbeat from core client for 30 sec - exiting 16:29:40 (3236): No heartbeat from core client for 30 sec - exiting 16:29:42 (3236): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3652, iMonCtr=1 Model crash detected, will try to restart... 08:33:01 (3756): No heartbeat from core client for 30 sec - exiting 08:33:02 (3756): No heartbeat from core client for 30 sec - exiting 08:33:03 (3756): No heartbeat from core client for 30 sec - exiting 08:33:04 (3756): No heartbeat from core client for 30 sec - exiting 08:33:05 (3756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 14:27:19 (1360): No heartbeat from core client for 30 sec - exiting 14:27:20 (1360): No heartbeat from core client for 30 sec - exiting 14:27:21 (1360): No heartbeat from core client for 30 sec - exiting 14:27:22 (1360): No heartbeat from core client for 30 sec - exiting 14:27:23 (1360): No heartbeat from core client for 30 sec - exiting 14:27:25 (1360): No heartbeat from core client for 30 sec - exiting 14:27:26 (1360): No heartbeat from core client for 30 sec - exiting 14:27:27 (1360): No heartbeat from core client for 30 sec - exiting 14:27:28 (1360): No heartbeat from core client for 30 sec - exiting 14:27:29 (1360): No heartbeat from core client for 30 sec - exiting 14:27:30 (1360): No heartbeat from core client for 30 sec - exiting 14:27:31 (1360): No heartbeat from core client for 30 sec - exiting 14:27:32 (1360): No heartbeat from core client for 30 sec - exiting 14:27:33 (1360): No heartbeat from core client for 30 sec - exiting 14:27:34 (1360): No heartbeat from core client for 30 sec - exiting 14:27:35 (1360): No heartbeat from core client for 30 sec - exiting 14:27:37 (1360): No heartbeat from core client for 30 sec - exiting 14:27:38 (1360): No heartbeat from core client for 30 sec - exiting 14:27:39 (1360): No heartbeat from core client for 30 sec - exiting 14:27:40 (1360): No heartbeat from core client for 30 sec - exiting 14:27:41 (1360): No heartbeat from core client for 30 sec - exiting 14:27:42 (1360): No heartbeat from core client for 30 sec - exiting 14:27:43 (1360): No heartbeat from core client for 30 sec - exiting 14:27:44 (1360): No heartbeat from core client for 30 sec - exiting 14:27:45 (1360): No heartbeat from core client for 30 sec - exiting 14:27:46 (1360): No heartbeat from core client for 30 sec - exiting 14:27:47 (1360): No heartbeat from core client for 30 sec - exiting 14:27:49 (1360): No heartbeat from core client for 30 sec - exiting 14:27:50 (1360): No heartbeat from core client for 30 sec - exiting 14:27:51 (1360): No heartbeat from core client for 30 sec - exiting 14:27:52 (1360): No heartbeat from core client for 30 sec - exiting 14:27:53 (1360): No heartbeat from core client for 30 sec - exiting 14:27:54 (1360): No heartbeat from core client for 30 sec - exiting 14:27:55 (1360): No heartbeat from core client for 30 sec - exiting 14:27:56 (1360): No heartbeat from core client for 30 sec - exiting 14:27:57 (1360): No heartbeat from core client for 30 sec - exiting 14:27:58 (1360): No heartbeat from core client for 30 sec - exiting 14:27:59 (1360): No heartbeat from core client for 30 sec - exiting 14:28:01 (1360): No heartbeat from core client for 30 sec - exiting 14:28:02 (1360): No heartbeat from core client for 30 sec - exiting 14:28:03 (1360): No heartbeat from core client for 30 sec - exiting 14:28:04 (1360): No heartbeat from core client for 30 sec - exiting 14:28:05 (1360): No heartbeat from core client for 30 sec - exiting 14:28:06 (1360): No heartbeat from core client for 30 sec - exiting 14:28:07 (1360): No heartbeat from core client for 30 sec - exiting 14:28:08 (1360): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 21:23:33 (5084): No heartbeat from core client for 30 sec - exiting 21:23:34 (5084): No heartbeat from core client for 30 sec - exiting 21:23:35 (5084): No heartbeat from core client for 30 sec - exiting 21:23:36 (5084): No heartbeat from core client for 30 sec - exiting 21:23:38 (5084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:06:33 (1232): No heartbeat from core client for 30 sec - exiting 09:06:35 (1232): No heartbeat from core client for 30 sec - exiting 09:06:36 (1232): No heartbeat from core client for 30 sec - exiting 09:06:37 (1232): No heartbeat from core client for 30 sec - exiting 09:06:38 (1232): No heartbeat from core client for 30 sec - exiting 09:06:39 (1232): No heartbeat from core client for 30 sec - exiting 09:06:40 (1232): No heartbeat from core client for 30 sec - exiting 09:06:41 (1232): No heartbeat from core client for 30 sec - exiting 09:06:42 (1232): No heartbeat from core client for 30 sec - exiting 09:06:43 (1232): No heartbeat from core client for 30 sec - exiting 09:06:44 (1232): No heartbeat from core client for 30 sec - exiting 09:06:45 (1232): No heartbeat from core client for 30 sec - exiting 09:06:47 (1232): No heartbeat from core client for 30 sec - exiting 09:06:48 (1232): No heartbeat from core client for 30 sec - exiting 09:06:49 (1232): No heartbeat from core client for 30 sec - exiting 09:06:50 (1232): No heartbeat from core client for 30 sec - exiting 09:06:51 (1232): No heartbeat from core client for 30 sec - exiting 09:06:52 (1232): No heartbeat from core client for 30 sec - exiting 09:06:53 (1232): No heartbeat from core client for 30 sec - exiting 09:06:54 (1232): No heartbeat from core client for 30 sec - exiting 09:06:55 (1232): No heartbeat from core client for 30 sec - exiting 09:06:56 (1232): No heartbeat from core client for 30 sec - exiting 09:06:57 (1232): No heartbeat from core client for 30 sec - exiting 09:06:59 (1232): No heartbeat from core client for 30 sec - exiting 09:07:00 (1232): No heartbeat from core client for 30 sec - exiting 09:07:01 (1232): No heartbeat from core client for 30 sec - exiting 09:07:02 (1232): No heartbeat from core client for 30 sec - exiting 09:07:03 (1232): No heartbeat from core client for 30 sec - exiting 09:07:04 (1232): No heartbeat from core client for 30 sec - exiting 09:07:05 (1232): No heartbeat from core client for 30 sec - exiting 09:07:06 (1232): No heartbeat from core client for 30 sec - exiting 09:07:07 (1232): No heartbeat from core client for 30 sec - exiting 09:07:08 (1232): No heartbeat from core client for 30 sec - exiting 09:07:09 (1232): No heartbeat from core client for 30 sec - exiting 09:07:11 (1232): No heartbeat from core client for 30 sec - exiting 09:07:12 (1232): No heartbeat from core client for 30 sec - exiting 09:07:13 (1232): No heartbeat from core client for 30 sec - exiting 09:07:14 (1232): No heartbeat from core client for 30 sec - exiting 09:07:15 (1232): No heartbeat from core client for 30 sec - exiting 09:07:16 (1232): No heartbeat from core client for 30 sec - exiting 09:07:17 (1232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773F3770 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x773F3AC3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
10 Jun 2014 09:03:35	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	259,200	530,075	2.0450
10 Jun 2014 09:00:41	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	233,280	475,400	2.0379
03 Jun 2014 18:33:26	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	207,360	420,496	2.0279
31 May 2014 17:44:36	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	181,440	365,518	2.0145
27 May 2014 19:58:52	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	155,520	311,694	2.0042
20 May 2014 10:02:10	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	129,600	261,376	2.0168
15 May 2014 15:08:38	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	103,680	210,211	2.0275
08 May 2014 22:08:21	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	77,760	158,224	2.0348
04 May 2014 08:30:42	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	51,840	104,752	2.0207
02 May 2014 14:31:57	1290168	16590140	hadcm3n_8c8o_1980_40_008725283_0	25,920	52,128	2.0111