Task 15547358

Name	hadcm3n_n3mf_1880_40_008285519_0
Workunit	8436654
Created	16 Jan 2013, 20:01:49 UTC
Sent	16 Jan 2013, 20:06:28 UTC
Report deadline	18 Apr 2013, 3:33:39 UTC
Received	7 May 2013, 17:58:05 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1107940
Run time	19 days 2 hours 29 min 43 sec
CPU time	13 days 1 hours 0 min 23 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.72 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:14:55 (5132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:34:02 (4752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:42:38 (7108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:39 (7108): No heartbeat from core client for 30 sec - exiting 21:43:33 (8000): No heartbeat from core client for 30 sec - exiting 21:43:34 (8000): No heartbeat from core client for 30 sec - exiting 21:43:35 (8000): No heartbeat from core client for 30 sec - exiting 21:43:36 (8000): No heartbeat from core client for 30 sec - exiting 21:43:37 (8000): No heartbeat from core client for 30 sec - exiting 21:43:38 (8000): No heartbeat from core client for 30 sec - exiting 21:43:39 (8000): No heartbeat from core client for 30 sec - exiting 21:43:40 (8000): No heartbeat from core client for 30 sec - exiting 21:43:41 (8000): No heartbeat from core client for 30 sec - exiting 21:43:42 (8000): No heartbeat from core client for 30 sec - exiting 21:43:43 (8000): No heartbeat from core client for 30 sec - exiting 21:43:44 (8000): No heartbeat from core client for 30 sec - exiting 21:43:45 (8000): No heartbeat from core client for 30 sec - exiting 21:43:46 (8000): No heartbeat from core client for 30 sec - exiting 21:43:47 (8000): No heartbeat from core client for 30 sec - exiting 21:43:48 (8000): No heartbeat from core client for 30 sec - exiting 21:43:49 (8000): No heartbeat from core client for 30 sec - exiting 21:43:50 (8000): No heartbeat from core client for 30 sec - exiting 21:43:51 (8000): No heartbeat from core client for 30 sec - exiting 21:43:52 (8000): No heartbeat from core client for 30 sec - exiting 21:43:53 (8000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:43:54 (8000): No heartbeat from core client for 30 sec - exiting 21:43:55 (8000): No heartbeat from core client for 30 sec - exiting 21:43:56 (8000): No heartbeat from core client for 30 sec - exiting 21:43:57 (8000): No heartbeat from core client for 30 sec - exiting 21:43:58 (8000): No heartbeat from core client for 30 sec - exiting 21:43:59 (8000): No heartbeat from core client for 30 sec - exiting 21:44:00 (8000): No heartbeat from core client for 30 sec - exiting 21:44:01 (8000): No heartbeat from core client for 30 sec - exiting 21:44:02 (8000): No heartbeat from core client for 30 sec - exiting 21:44:03 (8000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:17:04 (4640): No heartbeat from core client for 30 sec - exiting 21:17:07 (4640): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:09:19 (4548): No heartbeat from core client for 30 sec - exiting 13:09:20 (4548): No heartbeat from core client for 30 sec - exiting 13:09:21 (4548): No heartbeat from core client for 30 sec - exiting 13:09:22 (4548): No heartbeat from core client for 30 sec - exiting 13:09:24 (4548): No heartbeat from core client for 30 sec - exiting 13:09:25 (4548): No heartbeat from core client for 30 sec - exiting 13:09:26 (4548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:30:23 (5520): No heartbeat from core client for 30 sec - exiting 16:30:24 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:30:25 (5520): No heartbeat from core client for 30 sec - exiting 16:30:26 (5520): No heartbeat from core client for 30 sec - exiting 16:30:27 (5520): No heartbeat from core client for 30 sec - exiting 16:30:28 (5520): No heartbeat from core client for 30 sec - exiting 16:30:29 (5520): No heartbeat from core client for 30 sec - exiting 16:30:30 (5520): No heartbeat from core client for 30 sec - exiting 16:30:31 (5520): No heartbeat from core client for 30 sec - exiting 16:30:32 (5520): No heartbeat from core client for 30 sec - exiting 16:30:33 (5520): No heartbeat from core client for 30 sec - exiting 16:30:34 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: TEMPHIST: Failed in OPEN of history file tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:34:12 (5444): No heartbeat from core client for 30 sec - exiting 16:34:13 (5444): No heartbeat from core client for 30 sec - exiting 16:34:14 (5444): No heartbeat from core client for 30 sec - exiting 16:34:15 (5444): No heartbeat from core client for 30 sec - exiting 16:34:16 (5444): No heartbeat from core client for 30 sec - exiting 16:34:17 (5444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:34:18 (5444): No heartbeat from core client for 30 sec - exiting 16:34:20 (5444): No heartbeat from core client for 30 sec - exiting 16:34:21 (5444): No heartbeat from core client for 30 sec - exiting 16:34:22 (5444): No heartbeat from core client for 30 sec - exiting 16:34:23 (5444): No heartbeat from core client for 30 sec - exiting 16:34:24 (5444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:53:52 (5532): No heartbeat from core client for 30 sec - exiting 14:53:53 (5532): No heartbeat from core client for 30 sec - exiting 14:53:54 (5532): No heartbeat from core client for 30 sec - exiting 14:53:55 (5532): No heartbeat from core client for 30 sec - exiting 14:53:57 (5532): No heartbeat from core client for 30 sec - exiting 14:53:58 (5532): No heartbeat from core client for 30 sec - exiting 14:54:00 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:54:02 (5532): No heartbeat from core client for 30 sec - exiting 14:54:03 (5532): No heartbeat from core client for 30 sec - exiting 14:54:05 (5532): No heartbeat from core client for 30 sec - exiting 14:54:07 (5532): No heartbeat from core client for 30 sec - exiting 14:54:08 (5532): No heartbeat from core client for 30 sec - exiting 14:54:09 (5532): No heartbeat from core client for 30 sec - exiting 14:54:10 (5532): No heartbeat from core client for 30 sec - exiting 14:54:11 (5532): No heartbeat from core client for 30 sec - exiting 14:54:12 (5532): No heartbeat from core client for 30 sec - exiting 14:54:13 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 19:36:33 (5140): No heartbeat from core client for 30 sec - exiting 19:36:35 (5140): No heartbeat from core client for 30 sec - exiting 19:36:36 (5140): No heartbeat from core client for 30 sec - exiting 19:36:37 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:36:39 (5140): No heartbeat from core client for 30 sec - exiting 19:36:40 (5140): No heartbeat from core client for 30 sec - exiting 19:36:41 (5140): No heartbeat from core client for 30 sec - exiting 19:36:42 (5140): No heartbeat from core client for 30 sec - exiting 19:36:43 (5140): No heartbeat from core client for 30 sec - exiting 19:36:44 (5140): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 08:22:16 (5068): No heartbeat from core client for 30 sec - exiting 08:22:17 (5068): No heartbeat from core client for 30 sec - exiting 08:22:18 (5068): No heartbeat from core client for 30 sec - exiting 08:22:19 (5068): No heartbeat from core client for 30 sec - exiting 08:22:20 (5068): No heartbeat from core client for 30 sec - exiting 08:22:22 (5068): No heartbeat from core client for 30 sec - exiting 08:22:23 (5068): No heartbeat from core client for 30 sec - exiting 08:22:24 (5068): No heartbeat from core client for 30 sec - exiting 08:22:25 (5068): No heartbeat from core client for 30 sec - exiting 08:22:26 (5068): No heartbeat from core client for 30 sec - exiting 08:22:27 (5068): No heartbeat from core client for 30 sec - exiting 08:22:28 (5068): No heartbeat from core client for 30 sec - exiting 08:22:29 (5068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:22:30 (5068): No heartbeat from core client for 30 sec - exiting 08:22:31 (5068): No heartbeat from core client for 30 sec - exiting 08:22:32 (5068): No heartbeat from core client for 30 sec - exiting 08:22:33 (5068): No heartbeat from core client for 30 sec - exiting 08:22:34 (5068): No heartbeat from core client for 30 sec - exiting 08:22:35 (5068): No heartbeat from core client for 30 sec - exiting 08:22:36 (5068): No heartbeat from core client for 30 sec - exiting 08:22:37 (5068): No heartbeat from core client for 30 sec - exiting 08:22:39 (5068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:41:38 (5856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 20:40:35 (8692): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:36:36 (4988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:36:37 (4988): No heartbeat from core client for 30 sec - exiting 16:36:38 (4988): No heartbeat from core client for 30 sec - exiting 16:36:39 (4988): No heartbeat from core client for 30 sec - exiting 16:36:40 (4988): No heartbeat from core client for 30 sec - exiting 16:36:41 (4988): No heartbeat from core client for 30 sec - exiting 16:36:42 (4988): No heartbeat from core client for 30 sec - exiting 16:36:43 (4988): No heartbeat from core client for 30 sec - exiting 16:36:44 (4988): No heartbeat from core client for 30 sec - exiting 16:36:45 (4988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:47:15 (5300): No heartbeat from core client for 30 sec - exiting 21:47:16 (5300): No heartbeat from core client for 30 sec - exiting 21:47:17 (5300): No heartbeat from core client for 30 sec - exiting 21:47:18 (5300): No heartbeat from core client for 30 sec - exiting 21:47:19 (5300): No heartbeat from core client for 30 sec - exiting 21:47:20 (5300): No heartbeat from core client for 30 sec - exiting 21:47:21 (5300): No heartbeat from core client for 30 sec - exiting 21:47:22 (5300): No heartbeat from core client for 30 sec - exiting 21:47:24 (5300): No heartbeat from core client for 30 sec - exiting 21:47:25 (5300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1736, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1736, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish CPDN Monitor - Quit request from BOINC... 16:20:59 (5108): No heartbeat from core client for 30 sec - exiting 16:21:00 (5108): No heartbeat from core client for 30 sec - exiting 16:21:01 (5108): No heartbeat from core client for 30 sec - exiting 16:21:02 (5108): No heartbeat from core client for 30 sec - exiting 16:21:03 (5108): No heartbeat from core client for 30 sec - exiting 16:21:04 (5108): No heartbeat from core client for 30 sec - exiting 16:21:05 (5108): No heartbeat from core client for 30 sec - exiting 16:21:06 (5108): No heartbeat from core client for 30 sec - exiting 16:21:07 (5108): No heartbeat from core client for 30 sec - exiting 16:21:08 (5108): No heartbeat from core client for 30 sec - exiting 16:21:09 (5108): No heartbeat from core client for 30 sec - exiting 16:21:10 (5108): No heartbeat from core client for 30 sec - exiting 16:21:11 (5108): No heartbeat from core client for 30 sec - exiting 16:21:12 (5108): No heartbeat from core client for 30 sec - exiting 16:21:13 (5108): No heartbeat from core client for 30 sec - exiting 16:21:14 (5108): No heartbeat from core client for 30 sec - exiting 16:21:15 (5108): No heartbeat from core client for 30 sec - exiting 16:21:16 (5108): No heartbeat from core client for 30 sec - exiting 16:21:17 (5108): No heartbeat from core client for 30 sec - exiting 16:21:19 (5108): No heartbeat from core client for 30 sec - exiting 16:21:20 (5108): No heartbeat from core client for 30 sec - exiting 16:21:23 (5108): No heartbeat from core client for 30 sec - exiting 16:21:25 (5108): No heartbeat from core client for 30 sec - exiting 16:21:27 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:21:28 (5108): No heartbeat from core client for 30 sec - exiting 16:21:29 (5108): No heartbeat from core client for 30 sec - exiting 16:21:30 (5108): No heartbeat from core client for 30 sec - exiting 16:21:31 (5108): No heartbeat from core client for 30 sec - exiting 16:21:32 (5108): No heartbeat from core client for 30 sec - exiting 16:21:33 (5108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request froCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:38:31 (4872): No heartbeat from core client for 30 sec - exiting 13:38:32 (4872): No heartbeat from core client for 30 sec - exiting 13:38:33 (4872): No heartbeat from core client for 30 sec - exiting 13:38:35 (4872): No heartbeat from core client for 30 sec - exiting 13:38:36 (4872): No heartbeat from core client for 30 sec - exiting 13:38:37 (4872): No heartbeat from core client for 30 sec - exiting 13:38:38 (4872): No heartbeat from core client for 30 sec - exiting 13:38:40 (4872): No heartbeat from core client for 30 sec - exiting 13:38:41 (4872): No heartbeat from core client for 30 sec - exiting 13:38:42 (4872): No heartbeat from core client for 30 sec - exiting 13:38:43 (4872): No heartbeat from core client for 30 sec - exiting 13:38:44 (4872): No heartbeat from core client for 30 sec - exiting 13:38:45 (4872): No heartbeat from core client for 30 sec - exiting 13:38:46 (4872): No heartbeat from core client for 30 sec - exiting 13:38:47 (4872): No heartbeat from core client for 30 sec - exiting 13:38:48 (4872): No heartbeat from core client for 30 sec - exiting 13:38:49 (4872): No heartbeat from core client for 30 sec - exiting 13:38:50 (4872): No heartbeat from core client for 30 sec - exiting 13:38:51 (4872): No heartbeat from core client for 30 sec - exiting 13:38:52 (4872): No heartbeat from core client for 30 sec - exiting 13:38:53 (4872): No heartbeat from core client for 30 sec - exiting 13:38:54 (4872): No heartbeat from core client for 30 sec - exiting 13:38:55 (4872): No heartbeat from core client for 30 sec - exiting 13:38:56 (4872): No heartbeat from core client for 30 sec - exiting 13:38:57 (4872): No heartbeat from core client for 30 sec - exiting 13:38:59 (4872): No heartbeat from core client for 30 sec - exiting 13:39:00 (4872): No heartbeat from core client for 30 sec - exiting 13:39:01 (4872): No heartbeat from core client for 30 sec - exiting 13:39:02 (4872): No heartbeat from core client for 30 sec - exiting 13:39:03 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:07:21 (5344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:06:10 (7808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN MonitorBUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/n3mfko.pj84c10 Error converting file to netcdf: dataout/n3mfko.pi84c10 Error converting file to netcdf: dataout/n3mfko.pf84c10 Error converting file to netcdf: dataout/n3mfka.ph84c10 Error converting file to netcdf: dataout/n3mfka.pg84c10 Error converting file to netcdf: dataout/n3mfka.pe84c10 Error converting file to netcdf: dataout/n3mfka.pd84c10 CPDN Monitor - Quit reCPDN Monitor - Quit request from BOINC... 13:33:27 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:33:58 (4568): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:33:59 (4568): No heartbeat from core client for 30 sec - exiting 15:34:00 (4568): No heartbeat from core client for 30 sec - exiting 15:34:01 (4568): No heartbeat from core client for 30 sec - exiting 15:34:02 (4568): No heartbeat from core client for 30 sec - exiting 15:34:03 (4568): No heartbeat from core client for 30 sec - exiting 15:34:04 (4568): No heartbeat from core client for 30 sec - exiting 15:38:48 (5732): No heartbeat from core client for 30 sec - exiting 15:38:49 (5732): No heartbeat from core client for 30 sec - exiting 15:38:50 (5732): No heartbeat from core client for 30 sec - exiting 15:38:51 (5732): No heartbeat from core client for 30 sec - exiting 15:38:52 (5732): No heartbeat from core client for 30 sec - exiting 15:38:53 (5732): No heartbeat from core client for 30 sec - exiting 15:38:54 (5732): No heartbeat from core client for 30 sec - exiting 15:38:55 (5732): No heartbeat from core client for 30 sec - exiting 15:38:56 (5732): No heartbeat from core client for 30 sec - exiting 15:38:58 (5732): No heartbeat from core client for 30 sec - exiting 15:38:59 (5732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:39:00 (5732): No heartbeat from core client for 30 sec - exiting 15:39:01 (5732): No heartbeat from core client for 30 sec - exiting 15:39:02 (5732): No heartbeat from core client for 30 sec - exiting 15:39:03 (5732): No heartbeat from core client for 30 sec - exiting 15:39:04 (5732): No heartbeat from core client for 30 sec - exiting 15:39:05 (5732): No heartbeat from core client for 30 sec - exiting 15:39:06 (5732): No heartbeat from core client for 30 sec - exiting 15:39:08 (5732): No heartbeat from core client for 30 sec - exiting 15:39:09 (5732): No heartbeat from core client for 30 sec - exiting 15:39:10 (5732): No heartbeat from core client for 30 sec - exiting 15:42:26 (724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:42:27 (724): No heartbeat from core client for 30 sec - exiting 15:42:28 (724): No heartbeat from core client for 30 sec - exiting 15:42:29 (724): No heartbeat from core client for 30 sec - exiting 15:42:30 (724): No heartbeat from core client for 30 sec - exiting 15:42:31 (724): No heartbeat from core client for 30 sec - exiting 15:42:32 (724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - QuitCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:24:04 (4224): No heartbeat from core client for 30 sec - exiting 15:24:06 (4224): No heartbeat from core client for 30 sec - exiting 15:24:07 (4224): No heartbeat from core client for 30 sec - exiting 15:24:09 (4224): No heartbeat from core client for 30 sec - exiting 15:24:10 (4224): No heartbeat from core client for 30 sec - exiting 15:24:11 (4224): No heartbeat from core client for 30 sec - exiting 15:24:12 (4224): No heartbeat from core client for 30 sec - exiting 15:24:14 (4224): No heartbeat from core client for 30 sec - exiting 15:24:15 (4224): No heartbeat from core client for 30 sec - exiting 15:24:16 (4224): No heartbeat from core client for 30 sec - exiting 15:24:17 (4224): No heartbeat from core client for 30 sec - exiting 15:24:18 (4224): No heartbeat from core client for 30 sec - exiting 15:24:19 (4224): No heartbeat from core client for 30 sec - exiting 15:24:20 (4224): No heartbeat from core client for 30 sec - exiting 15:24:21 (4224): No heartbeat from core client for 30 sec - exiting 15:24:22 (4224): No heartbeat from core client for 30 sec - exiting 15:24:23 (4224): No heartbeat from core client for 30 sec - exiting 15:24:24 (4224): No heartbeat from core client for 30 sec - exiting 15:24:26 (4224): No heartbeat from core client for 30 sec - exiting 15:24:27 (4224): No heartbeat from core client for 30 sec - exiting 15:24:28 (4224): No heartbeat from core client for 30 sec - exiting 15:24:29 (4224): No heartbeat from core client for 30 sec - exiting 15:24:30 (4224): No heartbeat from core client for 30 sec - exiting 15:24:31 (4224): No heartbeat from core client for 30 sec - exiting 15:24:32 (4224): No heartbeat from core client for 30 sec - exiting 15:24:33 (4224): No heartbeat from core client for 30 sec - exiting 15:24:34 (4224): No heartbeat from core client for 30 sec - exiting 15:24:35 (4224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:24:36 (4224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:27:45 (4956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:01:39 (6012): No heartbeat from core client for 30 sec - exiting 20:01:40 (6012): No heartbeat from core client for 30 sec - exiting 20:01:41 (6012): No heartbeat from core client for 30 sec - exiting 20:01:42 (6012): No heartbeat from core client for 30 sec - exiting 20:01:43 (6012): No heartbeat from core client for 30 sec - exiting 20:01:44 (6012): No heartbeat from core client for 30 sec - exiting 20:01:45 (6012): No heartbeat from core client for 30 sec - exiting 20:01:46 (6012): No heartbeat from core client for 30 sec - exiting 20:01:47 (6012): No heartbeat from core client for 30 sec - exiting 20:01:48 (6012): No heartbeat from core client for 30 sec - exiting 20:01:49 (6012): No heartbeat from core client for 30 sec - exiting 20:01:50 (6012): No heartbeat from core client for 30 sec - exiting 20:01:51 (6012): No heartbeat from core client for 30 sec - exiting 20:01:52 (6012): No heartbeat from core client for 30 sec - exiting 20:01:53 (6012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:01:54 (6012): No heartbeat from core client for 30 sec - exiting 20:01:55 (6012): No heartbeat from core client for 30 sec - exiting 20:01:56 (6012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:45:13 (5480): No heartbeat from core client for 30 sec - exiting 07:45:14 (5480): No heartbeat from core client for 30 sec - exiting 07:45:15 (5480): No heartbeat from core client for 30 sec - exiting 07:45:16 (5480): No heartbeat from core client for 30 sec - exiting 07:45:17 (5480): No heartbeat from core client for 30 sec - exiting 07:45:18 (5480): No heartbeat from core client for 30 sec - exiting 07:45:19 (5480): No heartbeat from core client for 30 sec - exiting 07:45:20 (5480): No heartbeat from core client for 30 sec - exiting 07:45:21 (5480): No heartbeat from core client for 30 sec - exiting 07:45:22 (5480): No heartbeat from core client for 30 sec - exiting 07:45:23 (5480): No heartbeat from core client for 30 sec - exiting 07:45:24 (5480): No heartbeat from core client for 30 sec - exiting 07:45:25 (5480): No heartbeat from core client for 30 sec - exiting 07:45:26 (5480): No heartbeat from core client for 30 sec - exiting 07:45:27 (5480): No heartbeat from core client for 30 sec - exiting 07:45:28 (5480): No heartbeat from core client for 30 sec - exiting 07:45:29 (5480): No heartbeat from core client for 30 sec - exiting 07:45:30 (5480): No heartbeat from core client for 30 sec - exiting 07:45:31 (5480): No heartbeat from core client for 30 sec - exiting 07:45:32 (5480): No heartbeat from core client for 30 sec - exiting 07:45:33 (5480): No heartbeat from core client for 30 sec - exiting 07:45:36 (5480): No heartbeat from core client for 30 sec - exiting 07:45:37 (5480): No heartbeat from core client for 30 sec - exiting 07:45:38 (5480): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:06:06 (5700): No heartbeat from core client for 30 sec - exiting 17:06:07 (5700): No heartbeat from core client for 30 sec - exiting 17:06:09 (5700): No heartbeat from core client for 30 sec - exiting 17:06:11 (5700): No heartbeat from core client for 30 sec - exiting 17:06:12 (5700): No heartbeat from core client for 30 sec - exiting 17:06:13 (5700): No heartbeat from core client for 30 sec - exiting 17:06:15 (5700): No heartbeat from core client for 30 sec - exiting 17:06:17 (5700): No heartbeat from core client for 30 sec - exiting 17:06:18 (5700): No heartbeat from core client for 30 sec - exiting 17:06:19 (5700): No heartbeat from core client for 30 sec - exiting 17:06:20 (5700): No heartbeat from core client for 30 sec - exiting 17:06:22 (5700): No heartbeat from core client for 30 sec - exiting 17:06:23 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:45:40 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:41 (5312): No heartbeat from core client for 30 sec - exiting 17:45:42 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 16:15:19 (4904): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:33:56 (7540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:33:57 (7540): No heartbeat from core client for 30 sec - exiting 19:40:25 (9580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:40:26 (9580): No heartbeat from core client for 30 sec - exiting 19:40:27 (9580): No heartbeat from core client for 30 sec - exiting 19:45:41 (7884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:45:42 (7884): No heartbeat from core client for 30 sec - exiting 19:45:43 (7884): No heartbeat from core client for 30 sec - exiting 19:50:19 (9240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:50:20 (9240): No heartbeat from core client for 30 sec - exiting 19:50:21 (9240): No heartbeat from core client for 30 sec - exiting 19:50:22 (9240): No heartbeat from core client for 30 sec - exiting 19:50:23 (9240): No heartbeat from core client for 30 sec - exiting 19:50:24 (9240): No heartbeat from core client for 30 sec - exiting 19:50:26 (9240): No heartbeat from core client for 30 sec - exiting 19:50:27 (9240): No heartbeat from core client for 30 sec - exiting 19:50:28 (9240): No heartbeat from core client for 30 sec - exiting 19:50:29 (9240): No heartbeat from core client for 30 sec - exiting 19:50:30 (9240): No heartbeat from core client for 30 sec - exiting 19:55:09 (10072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:55:10 (10072): No heartbeat from core client for 30 sec - exiting 20:01:34 (7868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:01:35 (7868): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit requCPDN Monitor - Quit req13:21:01 (5644): No heartbeat from core client for 30 sec - exiting 13:21:02 (5644): No heartbeat from core client for 30 sec - exiting 13:21:03 (5644): No heartbeat from core client for 30 sec - exiting 13:21:04 (5644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:21:05 (5644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:04:16 (4244): No heartbeat from core client for 30 sec - exiting 13:04:17 (4244): No heartbeat from core client for 30 sec - exiting 13:04:18 (4244): No heartbeat from core client for 30 sec - exiting 13:04:19 (4244): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:04:20 (4244): No heartbeat from core client for 30 sec - exiting 13:04:21 (4244): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x7C921780 read attempt to address 0x3FE505CE Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 May 2013 17:01:32	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	259,200	1,126,801	4.3472
06 May 2013 11:59:24	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	233,280	1,067,463	4.5759
16 Mar 2013 09:13:19	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	207,360	478,194	2.3061
11 Mar 2013 18:39:16	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	181,440	418,064	2.3041
06 Mar 2013 19:58:24	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	155,520	356,446	2.2920
25 Feb 2013 17:52:43	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	129,600	295,478	2.2799
15 Feb 2013 20:50:14	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	103,680	244,844	2.3615
12 Feb 2013 17:50:41	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	77,760	183,317	2.3575
27 Jan 2013 20:26:06	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	51,840	122,973	2.3722
25 Jan 2013 16:10:44	1107940	15547358	hadcm3n_n3mf_1880_40_008285519_0	25,920	60,557	2.3363