Task 12750659

Name	hadcm3n_o78o_1900_40_007204715_1
Workunit	7402995
Created	28 Mar 2011, 14:19:42 UTC
Sent	28 Mar 2011, 18:01:47 UTC
Report deadline	28 Jun 2011, 1:28:58 UTC
Received	27 Apr 2011, 16:52:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1073186
Run time	6 days 7 hours 4 min 34 sec
CPU time	5 days 4 hours 13 min 47 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.56 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Courier-Bold.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Bookman-Light.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Helvetica-Oblique.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Helvetica-Bold.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Helvetica-BoldOblique.txf error: cannot create C:/utils/BOINC/data/projects/climateprediction.net/txf/Palatino-Roman.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Courier.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Palatino-Italic.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/AvantGarde-Book.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Times-Bold.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/ZapfChancery-MediumItalic.txf error: cannot delete old C:/utils/BOINC/data/projects/climateprediction.net/txf/Palatino-BoldItalic.txf CPDN Monitor - Quit request from BOINC... 23:01:45 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:01:46 (5604): No heartbeat from core client for 30 sec - exiting 23:01:47 (5604): No heartbeat from core client for 30 sec - exiting 23:01:48 (5604): No heartbeat from core client for 30 sec - exiting 23:01:49 (5604): No heartbeat from core client for 30 sec - exiting 23:01:50 (5604): No heartbeat from core client for 30 sec - exiting 23:01:51 (5604): No heartbeat from core client for 30 sec - exiting 23:01:52 (5604): No heartbeat from core client for 30 sec - exiting 23:01:53 (5604): No heartbeat from core client for 30 sec - exiting 23:01:54 (5604): No heartbeat from core client for 30 sec - exiting 23:01:55 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3940, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/o78oko.pja2c10 Error converting file to netcdf: dataout/o78oko.pia2c10 Error converting file to netcdf: dataout/o78oko.pfa2c10 Error converting file to netcdf: dataout/o78oka.pha2c10 Error converting file to netcdf: dataout/o78oka.pga2c10 Error converting file to netcdf: dataout/o78oka.pea2c10 Error converting file to netcdf: dataout/o78oka.pda2c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F72AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=24660, selfPID=24660, iMonCtr=1 Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F72AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F72AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... 07:56:31 (10520): No heartbeat from core client for 30 sec - exiting 07:56:32 (10520): No heartbeat from core client for 30 sec - exiting 07:56:33 (10520): No heartbeat from core client for 30 sec - exiting 07:56:34 (10520): No heartbeat from core client for 30 sec - exiting 07:56:35 (10520): No heartbeat from core client for 30 sec - exiting Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F72AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F72AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F72AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x76F72AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77322AB1 read attempt to address 0x4302F19E Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Apr 2011 20:24:27	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	259,200	444,076	1.7133
23 Apr 2011 12:10:36	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	233,280	399,396	1.7121
22 Apr 2011 21:37:16	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	207,360	354,250	1.7084
21 Apr 2011 20:18:19	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	181,440	308,755	1.7017
21 Apr 2011 08:42:06	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	155,520	264,277	1.6993
21 Apr 2011 08:42:06	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	129,600	220,777	1.7035
12 Apr 2011 17:45:09	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	103,680	177,212	1.7092
09 Apr 2011 11:05:33	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	77,760	132,769	1.7074
05 Apr 2011 20:38:16	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	51,840	88,541	1.7080
30 Mar 2011 20:26:49	1073186	12750659	hadcm3n_o78o_1900_40_007204715_1	25,920	45,106	1.7402