Task 13020177

Name	hadcm3n_t106_1940_40_007313932_0
Workunit	7511362
Created	28 Jun 2011, 13:01:52 UTC
Sent	28 Jun 2011, 13:04:01 UTC
Report deadline	27 Sep 2011, 20:31:12 UTC
Received	7 Aug 2011, 21:57:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1114842
Run time	22 days 0 hours 25 min 46 sec
CPU time	19 days 8 hours 16 min 34 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.09 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7292, iMonCtr=1 Model crash detected, will try to restart... 08:51:00 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:51:01 (5968): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:13:26 (8844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=37212, selfPID=37212, iMonCtr=1 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t106ko.pjf2c10 Error converting file to netcdf: dataout/t106ko.pif2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:25:32 (5368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:14:32 (1224): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:38:27 (6508): No heartbeat from core client for 30 sec - exiting 18:38:28 (6508): No heartbeat from core client for 30 sec - exiting 18:38:29 (6508): No heartbeat from core client for 30 sec - exiting 18:38:30 (6508): No heartbeat from core client for 30 sec - exiting 18:38:31 (6508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:38:32 (6508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:53:52 (6588): No heartbeat from core client for 30 sec - exiting 19:53:53 (6588): No heartbeat from core client for 30 sec - exiting 19:53:54 (6588): No heartbeat from core client for 30 sec - exiting 19:53:55 (6588): No heartbeat from core client for 30 sec - exiting 19:53:56 (6588): No heartbeat from core client for 30 sec - exiting 19:53:57 (6588): No heartbeat from core client for 30 sec - exiting 19:53:58 (6588): No heartbeat from core client for 30 sec - exiting 19:53:59 (6588): No heartbeat from core client for 30 sec - exiting 19:54:00 (6588): No heartbeat from core client for 30 sec - exiting 19:54:01 (6588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:54:02 (6588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/t106ko.pjg9c10 Error converting file to netcdf: dataout/t106ko.pig9c10 Error converting file to netcdf: dataout/t106ko.pfg9c10 Error converting file to netcdf: dataout/t106ka.phg9c10 Error converting file to netcdf: dataout/t106ka.pgg9c10 Error converting file to netcdf: dataout/t106ka.peg9c10 Error converting file to netcdf: dataout/t106ka.pdg9c10 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:32:01 (6184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x775471D3 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=7660, selfPID=7660, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x775471D3 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
06 Aug 2011 19:42:14	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	777,600	1,635,082	2.1027
05 Aug 2011 18:32:24	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	751,680	1,583,753	2.1070
04 Aug 2011 23:34:40	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	725,760	1,532,513	2.1116
03 Aug 2011 23:28:30	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	699,840	1,476,420	2.1097
02 Aug 2011 22:17:37	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	673,920	1,420,234	2.1074
02 Aug 2011 03:04:26	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	648,000	1,371,170	2.1160
01 Aug 2011 09:08:21	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	622,080	1,323,488	2.1275
31 Jul 2011 08:48:22	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	596,160	1,268,049	2.1270
30 Jul 2011 08:19:04	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	570,240	1,211,066	2.1238
29 Jul 2011 18:50:45	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	544,320	1,164,298	2.1390
29 Jul 2011 02:13:43	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	518,400	1,114,923	2.1507
28 Jul 2011 10:51:39	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	492,480	1,062,999	2.1585
27 Jul 2011 18:57:10	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	466,560	1,012,624	2.1704
26 Jul 2011 23:01:19	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	440,640	958,888	2.1761
26 Jul 2011 06:25:43	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	414,720	906,211	2.1851
25 Jul 2011 22:52:04	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	388,800	857,109	2.2045
25 Jul 2011 21:50:47	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	362,880	810,187	2.2327
25 Jul 2011 19:35:45	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	336,960	759,942	2.2553
25 Jul 2011 19:03:59	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	311,040	709,151	2.2799
25 Jul 2011 18:52:59	1114842	13020177	hadcm3n_t106_1940_40_007313932_0	285,120	655,582	2.2993