Task 12927653

Name	hadcm3n_o6nx_1940_40_007267073_0
Workunit	7465313
Created	3 Jun 2011, 1:15:55 UTC
Sent	3 Jun 2011, 1:16:05 UTC
Report deadline	2 Sep 2011, 8:43:16 UTC
Received	9 Jul 2011, 7:53:22 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1144464
Run time	8 days 2 hours 24 min 9 sec
CPU time	7 days 16 hours 50 min 18 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.45 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.60</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3348, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1076, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o6nxko.pje5c10 Error converting file to netcdf: dataout/o6nxko.pie5c10 Error converting file to netcdf: dataout/o6nxko.pfe5c10 Error converting file to netcdf: dataout/o6nxka.phe5c10 Error converting file to netcdf: dataout/o6nxka.pge5c10 Error converting file to netcdf: dataout/o6nxka.pee5c10 Error converting file to netcdf: dataout/o6nxka.pde5c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 13:07:28 (2616): No heartbeat from core client for 30 sec - exiting 13:07:29 (2616): No heartbeat from core client for 30 sec - exiting 13:07:30 (2616): No heartbeat from core client for 30 sec - exiting 13:07:31 (2616): No heartbeat from core client for 30 sec - exiting 13:07:32 (2616): No heartbeat from core client for 30 sec - exiting 13:07:33 (2616): No heartbeat from core client for 30 sec - exiting 13:07:34 (2616): No heartbeat from core client for 30 sec - exiting 13:07:35 (2616): No heartbeat from core client for 30 sec - exiting 13:07:36 (2616): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3828, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77887353 read attempt to address 0x40AC186B Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Jul 2011 08:28:48	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	518,400	664,104	1.2811
02 Jul 2011 04:07:55	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	492,480	630,838	1.2809
30 Jun 2011 03:02:17	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	466,560	597,742	1.2812
28 Jun 2011 04:56:46	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	440,640	563,720	1.2793
27 Jun 2011 01:06:42	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	414,720	529,624	1.2771
26 Jun 2011 06:38:48	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	388,800	496,242	1.2763
26 Jun 2011 03:09:34	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	362,880	462,243	1.2738
24 Jun 2011 06:28:46	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	336,960	427,977	1.2701
23 Jun 2011 06:06:19	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	311,040	394,355	1.2679
22 Jun 2011 03:33:32	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	285,120	360,784	1.2654
21 Jun 2011 02:16:35	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	259,200	327,195	1.2623
19 Jun 2011 22:05:26	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	233,280	294,603	1.2629
16 Jun 2011 02:13:41	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	207,360	262,142	1.2642
14 Jun 2011 01:49:17	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	181,440	229,306	1.2638
12 Jun 2011 07:02:35	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	155,520	196,886	1.2660
11 Jun 2011 21:28:18	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	129,600	164,310	1.2678
10 Jun 2011 09:44:19	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	103,680	130,622	1.2599
10 Jun 2011 01:03:41	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	77,760	98,701	1.2693
09 Jun 2011 01:52:44	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	51,840	65,879	1.2708
08 Jun 2011 01:29:33	1144464	12927653	hadcm3n_o6nx_1940_40_007267073_0	25,920	33,143	1.2787