Task 14387328

Name	hadcm3n_yn1c_1980_40_007865167_3
Workunit	8020279
Created	10 Apr 2012, 10:23:25 UTC
Sent	10 Apr 2012, 10:23:49 UTC
Report deadline	10 Jul 2012, 17:51:00 UTC
Received	26 Apr 2012, 6:37:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1102272
Run time	12 days 4 hours 24 min 54 sec
CPU time	11 days 14 hours 32 min 3 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.52 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.25</core_client_version> <![CDATA[ <message> - exit code -1073741819 (0xc0000005) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:02:25 (4288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:05:22 (6128): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:14:21 (2768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:31 (5688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:15:32 (5688): No heartbeat from core client for 30 sec - exiting 11:15:33 (5688): No heartbeat from core client for 30 sec - exiting 11:15:34 (5688): No heartbeat from core client for 30 sec - exiting 11:15:35 (5688): No heartbeat from core client for 30 sec - exiting 11:15:36 (5688): No heartbeat from core client for 30 sec - exiting 11:15:37 (5688): No heartbeat from core client for 30 sec - exiting 11:15:38 (5688): No heartbeat from core client for 30 sec - exiting 11:15:39 (5688): No heartbeat from core client for 30 sec - exiting 11:15:40 (5688): No heartbeat from core client for 30 sec - exiting 11:15:41 (5688): No heartbeat from core client for 30 sec - exiting 11:16:28 (2276): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:17:54 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:17:55 (4872): No heartbeat from core client for 30 sec - exiting 11:17:56 (4872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 00:04:26 (808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:05:16 (2312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:05:17 (2312): No heartbeat from core client for 30 sec - exiting 14:36:29 (3936): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3984, iMonCtr=1 Model crash detected, will try to restart... Atmos Hold Restart file rename failed on atmos_restart.hold CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:54:28 (4372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:54:29 (4372): No heartbeat from core client for 30 sec - exiting 14:54:31 (4372): No heartbeat from core client for 30 sec - exiting 14:54:32 (4372): No heartbeat from core client for 30 sec - exiting 14:54:33 (4372): No heartbeat from core client for 30 sec - exiting 14:54:34 (4372): No heartbeat from core client for 30 sec - exiting 14:54:35 (4372): No heartbeat from core client for 30 sec - exiting 14:56:15 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:56:20 (5472): No heartbeat from core client for 30 sec - exiting 14:56:21 (5472): No heartbeat from core client for 30 sec - exiting 14:56:22 (5472): No heartbeat from core client for 30 sec - exiting 14:56:23 (5472): No heartbeat from core client for 30 sec - exiting 14:56:24 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:55:18 (3432): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:56:31 (2300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:48:14 (5460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:50:49 (4012): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5848, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:03:44 (5176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77723AB3 read attempt to address 0x40D751B4 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
26 Apr 2012 05:17:07	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	518,400	1,002,711	1.9342
25 Apr 2012 10:04:25	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	492,480	952,061	1.9332
24 Apr 2012 17:14:01	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	466,560	893,651	1.9154
24 Apr 2012 00:25:08	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	440,640	835,827	1.8968
23 Apr 2012 07:20:32	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	414,720	777,770	1.8754
22 Apr 2012 13:57:51	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	388,800	719,518	1.8506
21 Apr 2012 11:09:02	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	362,880	661,082	1.8218
20 Apr 2012 12:18:14	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	336,960	608,098	1.8047
19 Apr 2012 16:13:41	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	311,040	559,815	1.7998
18 Apr 2012 19:05:57	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	285,120	514,317	1.8039
18 Apr 2012 04:47:49	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	259,200	466,018	1.7979
17 Apr 2012 02:26:30	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	233,280	418,186	1.7926
16 Apr 2012 06:46:21	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	207,360	371,970	1.7938
15 Apr 2012 13:44:28	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	181,440	326,473	1.7993
15 Apr 2012 13:44:28	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	155,520	279,283	1.7958
14 Apr 2012 10:17:57	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	129,600	232,622	1.7949
14 Apr 2012 10:17:57	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	103,680	184,096	1.7756
12 Apr 2012 21:04:16	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	77,760	137,794	1.7720
12 Apr 2012 02:17:19	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	51,840	93,065	1.7952
11 Apr 2012 05:18:31	1102272	14387328	hadcm3n_yn1c_1980_40_007865167_3	25,920	46,696	1.8015