Task 13092506

Name	hadcm3n_y8ue_1900_40_007344320_1
Workunit	7541750
Created	6 Jul 2011, 13:23:48 UTC
Sent	22 Jul 2011, 16:11:15 UTC
Report deadline	21 Oct 2011, 23:38:26 UTC
Received	20 Sep 2011, 23:37:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1064648
Run time	32 days 8 hours 8 min 3 sec
CPU time	26 days 4 hours 36 min 27 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.95 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 12:17:59 (4484): No heartbeat from core client for 30 sec - exiting 12:18:01 (4484): No heartbeat from core client for 30 sec - exiting 12:18:02 (4484): No heartbeat from core client for 30 sec - exiting 12:18:03 (4484): No heartbeat from core client for 30 sec - exiting 12:18:04 (4484): No heartbeat from core client for 30 sec - exiting 12:18:05 (4484): No heartbeat from core client for 30 sec - exiting 12:18:06 (4484): No heartbeat from core client for 30 sec - exiting 12:18:07 (4484): No heartbeat from core client for 30 sec - exiting 12:18:08 (4484): No heartbeat from core client for 30 sec - exiting 12:18:09 (4484): No heartbeat from core client for 30 sec - exiting 12:18:10 (4484): No heartbeat from core client for 30 sec - exiting 12:18:11 (4484): No heartbeat from core client for 30 sec - exiting 12:18:13 (4484): No heartbeat from core client for 30 sec - exiting 12:18:14 (4484): No heartbeat from core client for 30 sec - exiting 12:18:15 (4484): No heartbeat from core client for 30 sec - exiting 12:18:16 (4484): No heartbeat from core client for 30 sec - exiting 12:18:17 (4484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:18:18 (4484): No heartbeat from core client for 30 sec - exiting 12:18:19 (4484): No heartbeat from core client for 30 sec - exiting C11:40:28 (4460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3212, iMonCtr=1 Model crash detected, will try to restart... 10:51:03 (4092): No heartbeat from core client for 30 sec - exiting 10:51:05 (4092): No heartbeat from core client for 30 sec - exiting 10:51:06 (4092): No heartbeat from core client for 30 sec - exiting 10:51:07 (4092): No heartbeat from core client for 30 sec - exiting 10:51:08 (4092): No heartbeat from core client for 30 sec - exiting 10:51:09 (4092): No heartbeat from core client for 30 sec - exiting 10:51:10 (4092): No heartbeat from core client for 30 sec - exiting 10:51:11 (4092): No heartbeat from core client for 30 sec - exiting 10:51:13 (4092): No heartbeat from core client for 30 sec - exiting 10:51:14 (4092): No heartbeat from core client for 30 sec - exiting 10:51:15 (4092): No heartbeat from core client for 30 sec - exiting 10:51:16 (4092): No heartbeat from core client for 30 sec - exiting 10:51:17 (4092): No heartbeat from core client for 30 sec - exiting 10:51:18 (4092): No heartbeat from core client for 30 sec - exiting 10:51:20 (4092): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:44:56 (49148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:51:34 (4940): No heartbeat from core client for 30 sec - exiting 10:51:35 (4940): No heartbeat from core client for 30 sec - exiting 10:51:36 (4940): No heartbeat from core client for 30 sec - exiting 10:51:38 (4940): No heartbeat from core client for 30 sec - exiting 10:51:39 (4940): No heartbeat from core client for 30 sec - exiting 10:51:40 (4940): No heartbeat from core client for 30 sec - exiting 10:51:41 (4940): No heartbeat from core client for 30 sec - exiting 10:51:42 (4940): No heartbeat from core client for 30 sec - exiting 10:51:43 (4940): No heartbeat from core client for 30 sec - exiting 10:51:44 (4940): No heartbeat from core client for 30 sec - exiting 10:51:45 (4940): No heartbeat from core client for 30 sec - exiting 10:51:46 (4940): No heartbeat from core client for 30 sec - exiting 10:51:47 (4940): No heartbeat from core client for 30 sec - exiting 10:51:49 (4940): No heartbeat from core client for 30 sec - exiting 10:51:50 (4940): No heartbeat from core client for 30 sec - exiting 10:51:51 (4940): No heartbeat from core client for 30 sec - exiting 10:51:52 (4940): No heartbeat from core client for 30 sec - exiting 10:51:53 (4940): No heartbeat from core client for 30 sec - exiting 10:51:54 (4940): No heartbeat from core client for 30 sec - exiting 10:51:55 (4940): No heartbeat from core client for 30 sec - exiting 10:51:56 (4940): No heartbeat from core client for 30 sec - exiting 10:51:57 (4940): No heartbeat from core client for 30 sec - exiting 10:51:58 (4940): No heartbeat from core client for 30 sec - exiting 10:52:00 (4940): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:02:28 (5048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:46:40 (4272): No heartbeat from core client for 30 sec - exiting 00:46:41 (4272): No heartbeat from core client for 30 sec - exiting 00:46:42 (4272): No heartbeat from core client for 30 sec - exiting 00:46:43 (4272): No heartbeat from core client for 30 sec - exiting 00:46:44 (4272): No heartbeat from core client for 30 sec - exiting 00:46:46 (4272): No heartbeat from core client for 30 sec - exiting 00:46:47 (4272): No heartbeat from core client for 30 sec - exiting 00:46:48 (4272): No heartbeat from core client for 30 sec - exiting 00:46:49 (4272): No heartbeat from core client for 30 sec - exiting 00:46:50 (4272): No heartbeat from core client for 30 sec - exiting 00:46:51 (4272): No heartbeat from core client for 30 sec - exiting 00:46:52 (4272): No heartbeat from core client for 30 sec - exiting 00:46:53 (4272): No heartbeat from core client for 30 sec - exiting 00:46:54 (4272): No heartbeat from core client for 30 sec - exiting 00:46:55 (4272): No heartbeat from core client for 30 sec - exiting 00:46:56 (4272): No heartbeat from core client for 30 sec - exiting 00:46:58 (4272): No heartbeat from core client for 30 sec - exiting 00:46:59 (4272): No heartbeat from core client for 30 sec - exiting 00:47:00 (4272): No heartbeat from core client for 30 sec - exiting 00:47:01 (4272): No heartbeat from core client for 30 sec - exiting 00:47:02 (4272): No heartbeat from core client for 30 sec - exiting 00:47:03 (4272): No heartbeat from core client for 30 sec - exiting 00:47:04 (4272): No heartbeat from core client for 30 sec - exiting 00:47:05 (4272): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:47:07 (4272): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5032, iMonCtr=1 Model crash detected, will try to restart... 10:45:50 (4736): No heartbeat from core client for 30 sec - exiting 10:45:51 (4736): No heartbeat from core client for 30 sec - exiting 10:45:52 (4736): No heartbeat from core client for 30 sec - exiting 10:45:53 (4736): No heartbeat from core client for 30 sec - exiting 10:45:55 (4736): No heartbeat from core client for 30 sec - exiting 10:45:56 (4736): No heartbeat from core client for 30 sec - exiting 10:45:57 (4736): No heartbeat from core client for 30 sec - exiting 10:45:58 (4736): No heartbeat from core client for 30 sec - exiting 10:45:59 (4736): No heartbeat from core client for 30 sec - exiting 10:46:00 (4736): No heartbeat from core client for 30 sec - exiting 10:46:01 (4736): No heartbeat from core client for 30 sec - exiting 10:46:02 (4736): No heartbeat from core client for 30 sec - exiting 10:46:03 (4736): No heartbeat from core client for 30 sec - exiting 10:46:04 (4736): No heartbeat from core client for 30 sec - exiting 10:46:06 (4736): No heartbeat from core client for 30 sec - exiting 10:46:07 (4736): No heartbeat from core client for 30 sec - exiting 10:46:08 (4736): No heartbeat from core client for 30 sec - exiting 10:46:09 (4736): No heartbeat from core client for 30 sec - exiting 10:46:10 (4736): No heartbeat from core client for 30 sec - exiting 10:46:11 (4736): No heartbeat from core client for 30 sec - exiting 10:46:12 (4736): No heartbeat from core client for 30 sec - exiting 10:46:13 (4736): No heartbeat from core client for 30 sec - exiting 10:46:14 (4736): No heartbeat from core client for 30 sec - exiting 10:46:15 (4736): No heartbeat from core client for 30 sec - exiting 10:46:16 (4736): No heartbeat from core client for 30 sec - exiting 10:46:18 (4736): No heartbeat from core client for 30 sec - exiting 10:46:19 (4736): No heartbeat from core client for 30 sec - exiting 10:46:20 (4736): No heartbeat from core client for 30 sec - exiting 10:46:21 (4736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:54:19 (4328): No heartbeat from core client for 30 sec - exiting 09:54:21 (4328): No heartbeat from core client for 30 sec - exiting 09:54:22 (4328): No heartbeat from core client for 30 sec - exiting 09:54:23 (4328): No heartbeat from core client for 30 sec - exiting 09:54:24 (4328): No heartbeat from core client for 30 sec - exiting 09:54:25 (4328): No heartbeat from core client for 30 sec - exiting 09:54:26 (4328): No heartbeat from core client for 30 sec - exiting 09:54:27 (4328): No heartbeat from core client for 30 sec - exiting 09:54:28 (4328): No heartbeat from core client for 30 sec - exiting 09:54:30 (4328): No heartbeat from core client for 30 sec - exiting 09:54:31 (4328): No heartbeat from core client for 30 sec - exiting 09:54:32 (4328): No heartbeat from core client for 30 sec - exiting 09:54:33 (4328): No heartbeat from core client for 30 sec - exiting 09:54:34 (4328): No heartbeat from core client for 30 sec - exiting 09:54:35 (4328): No heartbeat from core client for 30 sec - exiting 09:54:36 (4328): No heartbeat from core client for 30 sec - exiting 09:54:37 (4328): No heartbeat from core client for 30 sec - exiting 09:54:38 (4328): No heartbeat from core client for 30 sec - exiting 09:54:40 (4328): No heartbeat from core client for 30 sec - exiting 09:54:41 (4328): No heartbeat from core client for 30 sec - exiting 09:54:42 (4328): No heartbeat from core client for 30 sec - exiting 09:54:43 (4328): No heartbeat from core client for 30 sec - exiting 09:54:44 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:56:23 (2440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:19:50 (3368): No heartbeat from core client for 30 sec - exiting 10:19:52 (3368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:35:47 (3168): No heartbeat from core client for 30 sec - exiting 23:35:48 (3168): No heartbeat from core client for 30 sec - exiting 23:35:49 (3168): No heartbeat from core client for 30 sec - exiting 23:35:50 (3168): No heartbeat from core client for 30 sec - exiting 23:35:51 (3168): No heartbeat from core client for 30 sec - exiting 23:35:52 (3168): No heartbeat from core client for 30 sec - exiting 23:35:53 (3168): No heartbeat from core client for 30 sec - exiting 23:35:54 (3168): No heartbeat from core client for 30 sec - exiting 23:35:55 (3168): No heartbeat from core client for 30 sec - exiting 23:35:56 (3168): No heartbeat from core client for 30 sec - exiting 23:35:58 (3168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2324, iMonCtr=1 Model crash detected, will try to restart... 10:53:48 (3900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:53:49 (3900): No heartbeat from core client for 30 sec - exiting 10:53:51 (3900): No heartbeat from core client for 30 sec - exiting 10:53:52 (3900): No heartbeat from core client for 30 sec - exiting 10:53:53 (3900): No heartbeat from core client for 30 sec - exiting 10:53:54 (3900): No heartbeat from core client for 30 sec - exiting 10:03:21 (4072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:43:09 (67704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:07:02 (67636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:15:45 (73112): No heartbeat from core client for 30 sec - exiting 22:15:46 (73112): No heartbeat from core client for 30 sec - exiting 22:15:47 (73112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=91648, iMonC10:26:43 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:26:44 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:39:28 (3980): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:47:55 (3760): No heartbeat from core client for 30 sec - exiting 13:47:56 (3760): No heartbeat from core client for 30 sec - exiting 13:47:57 (3760): No heartbeat from core client for 30 sec - exiting 13:47:58 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:47:59 (3760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:14:36 (3396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:46:13 (3180): No heartbeat from core client for 30 sec - exiting 19:46:15 (3180): No heartbeat from core client for 30 sec - exiting 19:46:16 (3180): No heartbeat from core client for 30 sec - exiting 19:46:17 (3180): No heartbeat from core client for 30 sec - exiting 19:46:18 (3180): No heartbeat from core client for 30 sec - exiting 19:46:19 (3180): No heartbeat from core client for 30 sec - exiting 19:46:20 (3180): No heartbeat from core client for 30 sec - exiting 19:46:22 (3180): No heartbeat from core client for 30 sec - exiting 19:46:23 (3180): No heartbeat from core client for 30 sec - exiting 19:46:24 (3180): No heartbeat from core client for 30 sec - exiting 19:46:25 (3180): No heartbeat from core client for 30 sec - exiting 19:46:26 (3180): No heartbeat from core client for 30 sec - exiting 19:46:27 (3180): No heartbeat from core client for 30 sec - exiting 19:46:28 (3180): No heartbeat from core client for 30 sec - exiting 19:46:29 (3180): No heartbeat from core client for 30 sec - exiting 19:46:31 (3180): No heartbeat from core client for 30 sec - exiting 19:46:32 (3180): No heartbeat from core client for 30 sec - exiting 19:46:33 (3180): No heartbeat from core client for 30 sec - exiting 19:46:35 (3180): No heartbeat from core client for 30 sec - exiting 19:46:36 (3180): No heartbeat from core client for 30 sec - exiting 19:46:37 (3180): No heartbeat from core client for 30 sec - exiting 19:46:38 (3180): No heartbeat from core client for 30 sec - exiting 19:46:39 (3180): No heartbeat from core client for 30 sec - exiting 19:46:42 (3180): No heartbeat from core client for 30 sec - exiting 19:46:43 (3180): No heartbeat from core client for 30 sec - exiting 19:46:44 (3180): No heartbeat from core client for 30 sec - exiting 19:46:45 (3180): No heartbeat from core client for 30 sec - exiting 19:46:46 (3180): No heartbeat from core client for 30 sec - exiting 19:46:47 (3180): No heartbeat from core client for 30 sec - exiting 19:46:48 (3180): No heartbeat from core client for 30 sec - exiting 19:46:49 (3180): No heartbeat from core client for 30 sec - exiting 19:46:50 (3180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:48:34 (4404): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6340, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6452, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:26:05 (3784): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11060, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4548, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x774E2AB1 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=8252, selfPID=8252, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x774D361F read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Sep 2011 22:25:22	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	1,036,800	2,262,450	2.1821
18 Sep 2011 01:35:49	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	1,010,880	2,206,441	2.1827
15 Sep 2011 23:51:28	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	984,960	2,148,413	2.1812
13 Sep 2011 16:12:37	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	959,040	2,088,040	2.1772
10 Sep 2011 20:57:24	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	933,120	2,028,739	2.1741
10 Sep 2011 00:39:18	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	907,200	1,973,256	2.1751
08 Sep 2011 22:30:42	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	881,280	1,911,671	2.1692
08 Sep 2011 04:32:14	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	855,360	1,855,627	2.1694
06 Sep 2011 05:35:02	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	829,440	1,797,431	2.1670
05 Sep 2011 02:37:32	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	803,520	1,740,363	2.1659
03 Sep 2011 17:59:44	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	777,600	1,683,302	2.1647
01 Sep 2011 07:54:19	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	751,680	1,623,370	2.1597
30 Aug 2011 20:28:14	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	725,760	1,558,173	2.1470
29 Aug 2011 15:08:03	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	699,840	1,503,961	2.1490
28 Aug 2011 09:27:39	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	673,920	1,447,538	2.1479
27 Aug 2011 00:37:55	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	648,000	1,389,111	2.1437
26 Aug 2011 03:56:35	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	622,080	1,331,890	2.1410
25 Aug 2011 07:34:18	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	596,160	1,277,287	2.1425
24 Aug 2011 09:17:21	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	570,240	1,224,596	2.1475
23 Aug 2011 11:24:39	1064648	13092506	hadcm3n_y8ue_1900_40_007344320_1	544,320	1,172,568	2.1542