Task 12927961

Name	hadcm3n_o4x8_1940_40_007267217_1
Workunit	7465457
Created	3 Jun 2011, 2:04:16 UTC
Sent	3 Jun 2011, 2:04:30 UTC
Report deadline	2 Sep 2011, 9:31:41 UTC
Received	19 Jun 2011, 10:12:04 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1064102
Run time	10 days 5 hours 15 min 56 sec
CPU time	9 days 15 hours 14 min 42 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.79 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:00:32 (4676): No heartbeat from core client for 30 sec - exiting 08:00:34 (4676): No heartbeat from core client for 30 sec - exiting 08:00:35 (4676): No heartbeat from core client for 30 sec - exiting 08:00:36 (4676): No heartbeat from core client for 30 sec - exiting 08:00:37 (4676): No heartbeat from core client for 30 sec - exiting 08:00:38 (4676): No heartbeat from core client for 30 sec - exiting 08:00:39 (4676): No heartbeat from core client for 30 sec - exiting 08:00:40 (4676): No heartbeat from core client for 30 sec - exiting 08:00:41 (4676): No heartbeat from core client for 30 sec - exiting 08:00:42 (4676): No heartbeat from core client for 30 sec - exiting 08:00:43 (4676): No heartbeat from core client for 30 sec - exiting 08:00:44 (4676): No heartbeat from core client for 30 sec - exiting 08:00:45 (4676): No heartbeat from core client for 30 sec - exiting 08:00:46 (4676): No heartbeat from core client for 30 sec - exiting 08:00:47 (4676): No heartbeat from core client for 30 sec - exiting 08:00:48 (4676): No heartbeat from core client for 30 sec - exiting 08:00:49 (4676): No heartbeat from core client for 30 sec - exiting 08:00:50 (4676): No heartbeat from core client for 30 sec - exiting 08:00:51 (4676): No heartbeat from core client for 30 sec - exiting 08:00:52 (4676): No heartbeat from core client for 30 sec - exiting 08:00:53 (4676): No heartbeat from core client for 30 sec - exiting 08:00:54 (4676): No heartbeat from core client for 30 sec - exiting 08:00:55 (4676): No heartbeat from core client for 30 sec - exiting 08:00:56 (4676): No heartbeat from core client for 30 sec - exiting 08:00:57 (4676): No heartbeat from core client for 30 sec - exiting 08:00:58 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:35:09 (5292): No heartbeat from core client for 30 sec - exiting 07:35:10 (5292): No heartbeat from core client for 30 sec - exiting 07:35:11 (5292): No heartbeat from core client for 30 sec - exiting 07:35:12 (5292): No heartbeat from core client for 30 sec - exiting 07:35:13 (5292): No heartbeat from core client for 30 sec - exiting 07:35:14 (5292): No heartbeat from core client for 30 sec - exiting 07:35:15 (5292): No heartbeat from core client for 30 sec - exiting 07:35:16 (5292): No heartbeat from core client for 30 sec - exiting 07:35:17 (5292): No heartbeat from core client for 30 sec - exiting 07:35:18 (5292): No heartbeat from core client for 30 sec - exiting 07:35:19 (5292): No heartbeat from core client for 30 sec - exiting 07:35:20 (5292): No heartbeat from core client for 30 sec - exiting 07:35:21 (5292): No heartbeat from core client for 30 sec - exiting 07:35:22 (5292): No heartbeat from core client for 30 sec - exiting 07:35:23 (5292): No heartbeat from core client for 30 sec - exiting 07:35:24 (5292): No heartbeat from core client for 30 sec - exiting 07:35:25 (5292): No heartbeat from core client for 30 sec - exiting 07:35:26 (5292): No heartbeat from core client for 30 sec - exiting 07:35:27 (5292): No heartbeat from core client for 30 sec - exiting 07:35:28 (5292): No heartbeat from core client for 30 sec - exiting 07:35:29 (5292): No heartbeat from core client for 30 sec - exiting 07:35:30 (5292): No heartbeat from core client for 30 sec - exiting 07:35:31 (5292): No heartbeat from core client for 30 sec - exiting 07:35:32 (5292): No heartbeat from core client for 30 sec - exiting 07:35:33 (5292): No heartbeat from core client for 30 sec - exiting 07:35:35 (5292): No heartbeat from core client for 30 sec - exiting 07:35:36 (5292): No heartbeat from core client for 30 sec - exiting 07:35:37 (5292): No heartbeat from core client for 30 sec - exiting 07:35:38 (5292): No heartbeat from core client for 30 sec - exiting 07:35:39 (5292): No heartbeat from core client for 30 sec - exiting 07:35:40 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:35:41 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:33:04 (3468): No heartbeat from core client for 30 sec - exiting 15:33:05 (3468): No heartbeat from core client for 30 sec - exiting 15:33:06 (3468): No heartbeat from core client for 30 sec - exiting 15:33:07 (3468): No heartbeat from core client for 30 sec - exiting 15:33:08 (3468): No heartbeat from core client for 30 sec - exiting 15:33:09 (3468): No heartbeat from core client for 30 sec - exiting 15:33:10 (3468): No heartbeat from core client for 30 sec - exiting 15:33:11 (3468): No heartbeat from core client for 30 sec - exiting 15:33:12 (3468): No heartbeat from core client for 30 sec - exiting 15:33:13 (3468): No heartbeat from core client for 30 sec - exiting 15:33:14 (3468): No heartbeat from core client for 30 sec - exiting 15:33:15 (3468): No heartbeat from core client for 30 sec - exiting 15:33:16 (3468): No heartbeat from core client for 30 sec - exiting 15:33:17 (3468): No heartbeat from core client for 30 sec - exiting 15:33:18 (3468): No heartbeat from core client for 30 sec - exiting 15:33:19 (3468): No heartbeat from core client for 30 sec - exiting 15:33:20 (3468): No heartbeat from core client for 30 sec - exiting 15:33:21 (3468): No heartbeat from core client for 30 sec - exiting 15:33:22 (3468): No heartbeat from core client for 30 sec - exiting 15:33:23 (3468): No heartbeat from core client for 30 sec - exiting 15:33:24 (3468): No heartbeat from core client for 30 sec - exiting 15:33:25 (3468): No heartbeat from core client for 30 sec - exiting 15:33:26 (3468): No heartbeat from core client for 30 sec - exiting 15:33:27 (3468): No heartbeat from core client for 30 sec - exiting 15:33:28 (3468): No heartbeat from core client for 30 sec - exiting 15:33:29 (3468): No heartbeat from core client for 30 sec - exiting 15:33:30 (3468): No heartbeat from core client for 30 sec - exiting 15:33:31 (3468): No heartbeat from core client for 30 sec - exiting 15:33:32 (3468): No heartbeat from core client for 30 sec - exiting 15:33:33 (3468): No heartbeat from core client for 30 sec - exiting 15:33:34 (3468): No heartbeat from core client for 30 sec - exiting 15:33:36 (3468): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:06:04 (6900): No heartbeat from core client for 30 sec - exiting 19:06:05 (6900): No heartbeat from core client for 30 sec - exiting 19:06:06 (6900): No heartbeat from core client for 30 sec - exiting 19:06:07 (6900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:06:08 (6900): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:42:59 (4684): No heartbeat from core client for 30 sec - exiting 00:43:01 (4684): No heartbeat from core client for 30 sec - exiting 00:43:02 (4684): No heartbeat from core client for 30 sec - exiting 00:43:03 (4684): No heartbeat from core client for 30 sec - exiting 00:43:04 (4684): No heartbeat from core client for 30 sec - exiting 00:43:05 (4684): No heartbeat from core client for 30 sec - exiting 00:43:06 (4684): No heartbeat from core client for 30 sec - exiting 00:43:07 (4684): No heartbeat from core client for 30 sec - exiting 00:43:08 (4684): No heartbeat from core client for 30 sec - exiting 00:43:09 (4684): No heartbeat from core client for 30 sec - exiting 00:43:10 (4684): No heartbeat from core client for 30 sec - exiting 00:43:11 (4684): No heartbeat from core client for 30 sec - exiting 00:43:12 (4684): No heartbeat from core client for 30 sec - exiting 00:43:13 (4684): No heartbeat from core client for 30 sec - exiting 00:43:14 (4684): No heartbeat from core client for 30 sec - exiting 00:43:15 (4684): No heartbeat from core client for 30 sec - exiting 00:43:16 (4684): No heartbeat from core client for 30 sec - exiting 00:43:17 (4684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:22:30 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:35:48 (5712): No heartbeat from core client for 30 sec - exiting 07:35:49 (5712): No heartbeat from core client for 30 sec - exiting 07:35:50 (5712): No heartbeat from core client for 30 sec - exiting 07:35:51 (5712): No heartbeat from core client for 30 sec - exiting 07:35:52 (5712): No heartbeat from core client for 30 sec - exiting 07:35:53 (5712): No heartbeat from core client for 30 sec - exiting 07:35:54 (5712): No heartbeat from core client for 30 sec - exiting 07:35:55 (5712): No heartbeat from core client for 30 sec - exiting 07:35:56 (5712): No heartbeat from core client for 30 sec - exiting 07:35:57 (5712): No heartbeat from core client for 30 sec - exiting 07:35:58 (5712): No heartbeat from core client for 30 sec - exiting 07:35:59 (5712): No heartbeat from core client for 30 sec - exiting 07:36:00 (5712): No heartbeat from core client for 30 sec - exiting 07:36:01 (5712): No heartbeat from core client for 30 sec - exiting 07:36:02 (5712): No heartbeat from core client for 30 sec - exiting 07:36:03 (5712): No heartbeat from core client for 30 sec - exiting 07:36:04 (5712): No heartbeat from core client for 30 sec - exiting 07:36:05 (5712): No heartbeat from core client for 30 sec - exiting 07:36:06 (5712): No heartbeat from core client for 30 sec - exiting 07:36:07 (5712): No heartbeat from core client for 30 sec - exiting 07:36:08 (5712): No heartbeat from core client for 30 sec - exiting 07:36:09 (5712): No heartbeat from core client for 30 sec - exiting 07:36:10 (5712): No heartbeat from core client for 30 sec - exiting 07:36:11 (5712): No heartbeat from core client for 30 sec - exiting 07:36:12 (5712): No heartbeat from core client for 30 sec - exiting 07:36:13 (5712): No heartbeat from core client for 30 sec - exiting 07:36:14 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77D07353 read attempt to address 0x40443AE7 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_o4x8_1940_40_007267217/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
19 Jun 2011 22:05:31	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	259,200	832,465	3.2117
17 Jun 2011 12:32:04	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	233,280	763,372	3.2723
16 Jun 2011 13:01:52	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	207,360	686,153	3.3090
15 Jun 2011 09:41:43	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	181,440	602,484	3.3206
14 Jun 2011 10:37:07	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	155,520	529,288	3.4033
13 Jun 2011 01:31:47	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	129,600	444,574	3.4304
12 Jun 2011 01:10:05	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	103,680	360,456	3.4766
10 Jun 2011 15:27:15	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	77,760	277,699	3.5712
09 Jun 2011 10:56:19	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	51,840	184,714	3.5632
07 Jun 2011 22:47:26	1064102	12927961	hadcm3n_o4x8_1940_40_007267217_1	25,920	90,818	3.5038