Task 15773146

Name	hadcm3n_3efe_1980_40_008363862_0
Workunit	8514721
Created	10 May 2013, 18:20:59 UTC
Sent	10 May 2013, 18:24:44 UTC
Report deadline	10 Aug 2013, 1:51:55 UTC
Received	13 Jun 2013, 15:22:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	-1073741819 (0xC0000005) STATUS_ACCESS_VIOLATION
Computer ID	1057729
Run time	7 days 2 hours 32 min 11 sec
CPU time	5 days 12 hours 19 min 24 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.53 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1073741819 (0xc0000005) </message> <stderr_txt> 09:18:19 (9856): No heartbeat from core client for 30 sec - exiting 09:18:20 (9856): No heartbeat from core client for 30 sec - exiting 09:18:22 (9856): No heartbeat from core client for 30 sec - exiting 09:18:23 (9856): No heartbeat from core client for 30 sec - exiting 09:18:24 (9856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7064, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1140, iMonCtr=1 Model crash detected, will try to restart... 18:08:18 (9408): No heartbeat from core client for 30 sec - exiting 18:08:19 (9408): No heartbeat from core client for 30 sec - exiting 18:08:20 (9408): No heartbeat from core client for 30 sec - exiting 18:08:21 (9408): No heartbeat from core client for 30 sec - exiting 18:08:22 (9408): No heartbeat from core client for 30 sec - exiting 18:08:23 (9408): No heartbeat from core client for 30 sec - exiting 18:08:24 (9408): No heartbeat from core client for 30 sec - exiting 18:08:25 (9408): No heartbeat from core client for 30 sec - exiting 18:08:26 (9408): No heartbeat from core client for 30 sec - exiting 18:08:27 (9408): No heartbeat from core client for 30 sec - exiting 18:08:29 (9408): No heartbeat from core client for 30 sec - exiting 18:08:30 (9408): No heartbeat from core client for 30 sec - exiting 18:08:31 (9408): No heartbeat from core client for 30 sec - exiting 18:08:32 (9408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10444, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11232, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6432, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6004, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:46:04 (10008): No heartbeat from core client for 30 sec - exiting 08:46:05 (10008): No heartbeat from core client for 30 sec - exiting 08:46:06 (10008): No heartbeat from core client for 30 sec - exiting 08:46:07 (10008): No heartbeat from core client for 30 sec - exiting 08:46:08 (10008): No heartbeat from core client for 30 sec - exiting 08:46:10 (10008): No heartbeat from core client for 30 sec - exiting 08:46:11 (10008): No heartbeat from core client for 30 sec - exiting 08:46:12 (10008): No heartbeat from core client for 30 sec - exiting 08:46:13 (10008): No heartbeat from core client for 30 sec - exiting 08:46:14 (10008): No heartbeat from core client for 30 sec - exiting 08:46:15 (10008): No heartbeat from core client for 30 sec - exiting 08:46:16 (10008): No heartbeat from core client for 30 sec - exiting 08:46:17 (10008): No heartbeat from core client for 30 sec - exiting 08:46:18 (10008): No heartbeat from core client for 30 sec - exiting 08:46:19 (10008): No heartbeat from core client for 30 sec - exiting 08:46:20 (10008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:41:18 (5052): No heartbeat from core client for 30 sec - exiting 17:41:19 (5052): No heartbeat from core client for 30 sec - exiting 17:41:20 (5052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9308, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8680, iMonCtr=1 Model crash detected, will try to restart... 17:00:16 (4184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:00:18 (4184): No heartbeat from core client for 30 sec - exiting 17:00:19 (4184): No heartbeat from core client for 30 sec - exiting 17:00:20 (4184): No heartbeat from core client for 30 sec - exiting 17:00:21 (4184): No heartbeat from core client for 30 sec - exiting 17:00:22 (4184): No heartbeat from core client for 30 sec - exiting 17:00:23 (4184): No heartbeat from core client for 30 sec - exiting 17:00:24 (4184): No heartbeat from core client for 30 sec - exiting 17:00:25 (4184): No heartbeat from core client for 30 sec - exiting 17:00:27 (4184): No heartbeat from core client for 30 sec - exiting 17:00:28 (4184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 14:28:14 (6920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2912, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10068, iMonCtr=1 Model crash detected, will try to restart... 17:02:55 (2420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5256, iMonCtr=1 Model crash detected, will try to restart... 20:12:09 (2552): No heartbeat from core client for 30 sec - exiting 20:12:10 (2552): No heartbeat from core client for 30 sec - exiting 20:12:11 (2552): No heartbeat from core client for 30 sec - exiting 20:12:12 (2552): No heartbeat from core client for 30 sec - exiting 20:12:13 (2552): No heartbeat from core client for 30 sec - exiting 20:12:14 (2552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:41:48 (5172): No heartbeat from core client for 30 sec - exiting 20:41:49 (5172): No heartbeat from core client for 30 sec - exiting 20:41:50 (5172): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5908, iMonCtr=1 Model crash detected, will try to restart... 20:35:34 (5676): No heartbeat from core client for 30 sec - exiting 20:35:35 (5676): No heartbeat from core client for 30 sec - exiting 20:35:36 (5676): No heartbeat from core client for 30 sec - exiting 20:35:37 (5676): No heartbeat from core client for 30 sec - exiting 20:35:38 (5676): No heartbeat from core client for 30 sec - exiting 20:35:39 (5676): No heartbeat from core client for 30 sec - exiting 20:35:40 (5676): No heartbeat from core client for 30 sec - exiting 20:35:41 (5676): No heartbeat from core client for 30 sec - exiting 20:35:42 (5676): No heartbeat from core client for 30 sec - exiting 20:35:43 (5676): No heartbeat from core client for 30 sec - exiting 20:35:45 (5676): No heartbeat from core client for 30 sec - exiting 20:35:46 (5676): No heartbeat from core client for 30 sec - exiting 20:35:47 (5676): No heartbeat from core client for 30 sec - exiting 20:35:48 (5676): No heartbeat from core client for 30 sec - exiting 20:35:49 (5676): No heartbeat from core client for 30 sec - exiting 20:35:50 (5676): No heartbeat from core client for 30 sec - exiting 20:35:51 (5676): No heartbeat from core client for 30 sec - exiting 20:35:52 (5676): No heartbeat from core client for 30 sec - exiting 20:35:53 (5676): No heartbeat from core client for 30 sec - exiting 20:35:54 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:15:47 (10048): No heartbeat from core client for 30 sec - exiting 14:15:48 (10048): No heartbeat from core client for 30 sec - exiting 14:15:49 (10048): No heartbeat from core client for 30 sec - exiting 14:15:50 (10048): No heartbeat from core client for 30 sec - exiting 14:15:51 (10048): No heartbeat from core client for 30 sec - exiting 14:15:52 (10048): No heartbeat from core client for 30 sec - exiting 14:15:53 (10048): No heartbeat from core client for 30 sec - exiting 14:15:54 (10048): No heartbeat from core client for 30 sec - exiting 14:15:55 (10048): No heartbeat from core client for 30 sec - exiting 14:15:56 (10048): No heartbeat from core client for 30 sec - exiting 14:15:57 (10048): No heartbeat from core client for 30 sec - exiting 14:15:59 (10048): No heartbeat from core client for 30 sec - exiting 14:16:00 (10048): No heartbeat from core client for 30 sec - exiting 14:16:01 (10048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:17:04 (10132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77E17373 read attempt to address 0x40E02A5A Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77443AB3 read attempt to address 0x40E02A62 Engaging BOINC Windows Runtime Debugger... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
12 Jun 2013 20:57:40	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	259,200	470,892	1.8167
10 Jun 2013 18:48:03	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	233,280	427,669	1.8333
08 Jun 2013 12:25:06	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	207,360	379,107	1.8283
05 Jun 2013 17:29:41	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	181,440	331,778	1.8286
03 Jun 2013 12:31:40	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	155,520	284,702	1.8306
30 May 2013 06:33:16	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	129,600	237,626	1.8335
26 May 2013 17:43:03	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	103,680	190,491	1.8373
18 May 2013 10:14:12	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	77,760	143,356	1.8436
15 May 2013 19:20:00	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	51,840	96,350	1.8586
12 May 2013 20:37:08	1057729	15773146	hadcm3n_3efe_1980_40_008363862_0	25,920	48,239	1.8611