Task 13773443

Name	hadcm3n_ygrr_1940_40_007615251_2
Workunit	7793381
Created	13 Dec 2011, 9:04:02 UTC
Sent	13 Dec 2011, 9:07:55 UTC
Report deadline	13 Mar 2012, 16:35:06 UTC
Received	21 Dec 2011, 8:04:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	959754
Run time	4 days 23 hours 16 min 9 sec
CPU time	4 days 23 hours 16 min 9 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.64 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.4.7</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 14:16:31 (11220): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:19:07 (11464): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:21:01 (11988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:23:46 (11428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:26:21 (11592): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:29:32 (11500): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:32:37 (9672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:36:20 (10864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:41:15 (7228): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:29:10 (11448): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:29:12 (11448): No heartbeat from core client for 30 sec - exiting 22:29:13 (11448): No heartbeat from core client for 30 sec - exiting 22:29:14 (11448): No heartbeat from core client for 30 sec - exiting 22:31:20 (10496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:11:39 (10444): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:11:40 (10444): No heartbeat from core client for 30 sec - exiting 01:24:34 (9440): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:34:33 (10752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:59:46 (10668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:03:29 (9104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:04:56 (11016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:24:36 (4368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:24:37 (4368): No heartbeat from core client for 30 sec - exiting 02:29:14 (10972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:29:15 (10972): No heartbeat from core client for 30 sec - exiting 02:29:16 (10972): No heartbeat from core client for 30 sec - exiting 02:29:17 (10972): No heartbeat from core client for 30 sec - exiting 02:29:18 (10972): No heartbeat from core client for 30 sec - exiting 02:29:19 (10972): No heartbeat from core client for 30 sec - exiting 02:29:20 (10972): No heartbeat from core client for 30 sec - exiting 02:43:30 (7524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:45:38 (11100): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:45 (12084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:47 (12084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 04:07:09 (13240): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:29:17 (13080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:10:12 (12396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:11:13 (13148): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:39:34 (13588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:54:57 (13312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:08:52 (13700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:12:57 (13232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:16:20 (14176): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:16:21 (14176): No heartbeat from core client for 30 sec - exiting 06:16:22 (14176): No heartbeat from core client for 30 sec - exiting 06:16:23 (14176): No heartbeat from core client for 30 sec - exiting 06:16:24 (14176): No heartbeat from core client for 30 sec - exiting 06:16:25 (14176): No heartbeat from core client for 30 sec - exiting 06:16:26 (14176): No heartbeat from core client for 30 sec - exiting 06:29:08 (8620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:32:50 (13872): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:40:22 (12932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:44:47 (14084): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:43:11 (11516): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:07:53 (3580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:49:40 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:49:41 (3164): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:46:24 (9988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:49:36 (1348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:48:02 (6752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:18:53 (7896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:55 (7896): No heartbeat from core client for 30 sec - exiting 02:18:56 (7896): No heartbeat from core client for 30 sec - exiting 02:18:57 (7896): No heartbeat from core client for 30 sec - exiting 02:18:58 (7896): No heartbeat from core client for 30 sec - exiting 02:18:59 (7896): No heartbeat from core client for 30 sec - exiting 02:19:00 (7896): No heartbeat from core client for 30 sec - exiting 02:19:01 (7896): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:02:07 (9700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:51:12 (8248): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:51:13 (8248): No heartbeat from core client for 30 sec - exiting 17:38:54 (1132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:38:55 (1132): No heartbeat from core client for 30 sec - exiting 17:41:32 (8412): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:41:46 (8412): No heartbeat from core client for 30 sec - exiting 17:41:47 (8412): No heartbeat from core client for 30 sec - exiting 17:41:48 (8412): No heartbeat from core client for 30 sec - exiting 17:41:49 (8412): No heartbeat from core client for 30 sec - exiting 17:41:50 (8412): No heartbeat from core client for 30 sec - exiting 17:41:51 (8412): No heartbeat from core client for 30 sec - exiting 17:41:52 (8412): No heartbeat from core client for 30 sec - exiting 17:41:53 (8412): No heartbeat from core client for 30 sec - exiting 17:41:54 (8412): No heartbeat from core client for 30 sec - exiting 17:53:56 (8204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:53:57 (8204): No heartbeat from core client for 30 sec - exiting 17:53:59 (8204): No heartbeat from core client for 30 sec - exiting 17:54:00 (8204): No heartbeat from core client for 30 sec - exiting 17:54:01 (8204): No heartbeat from core client for 30 sec - exiting 17:54:02 (8204): No heartbeat from core client for 30 sec - exiting 17:54:03 (8204): No heartbeat from core client for 30 sec - exiting 17:54:04 (8204): No heartbeat from core client for 30 sec - exiting 17:54:05 (8204): No heartbeat from core client for 30 sec - exiting 17:54:06 (8204): No heartbeat from core client for 30 sec - exiting 17:54:07 (8204): No heartbeat from core client for 30 sec - exiting 17:54:08 (8204): No heartbeat from core client for 30 sec - exiting 17:54:09 (8204): No heartbeat from core client for 30 sec - exiting 17:54:10 (8204): No heartbeat from core client for 30 sec - exiting 17:54:11 (8204): No heartbeat from core client for 30 sec - exiting 17:54:12 (8204): No heartbeat from core client for 30 sec - exiting 17:54:13 (8204): No heartbeat from core client for 30 sec - exiting 17:54:14 (8204): No heartbeat from core client for 30 sec - exiting 17:57:36 (5796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:19:36 (12200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:19:37 (12200): No heartbeat from core client for 30 sec - exiting 18:19:38 (12200): No heartbeat from core client for 30 sec - exiting 18:22:10 (11580): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:28:00 (7312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:49:36 (7972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:31:56 (11420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:34:07 (8880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:34:08 (8880): No heartbeat from core client for 30 sec - exiting 21:34:09 (8880): No heartbeat from core client for 30 sec - exiting 21:34:10 (8880): No heartbeat from core client for 30 sec - exiting 21:36:52 (9392): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:39:22 (11820): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:40:31 (12264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:18 (10976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:42:19 (10976): No heartbeat from core client for 30 sec - exiting 21:42:20 (10976): No heartbeat from core client for 30 sec - exiting 21:42:21 (10976): No heartbeat from core client for 30 sec - exiting 21:42:22 (10976): No heartbeat from core client for 30 sec - exiting 21:42:23 (10976): No heartbeat from core client for 30 sec - exiting 21:42:24 (10976): No heartbeat from core client for 30 sec - exiting 21:43:30 (9760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:45:48 (9880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:51:17 (11008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:00 (9496): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:48 (11144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:57:49 (11144): No heartbeat from core client for 30 sec - exiting 21:57:50 (11144): No heartbeat from core client for 30 sec - exiting 21:57:51 (11144): No heartbeat from core client for 30 sec - exiting 21:57:52 (11144): No heartbeat from core client for 30 sec - exiting 21:57:54 (11144): No heartbeat from core client for 30 sec - exiting 21:57:55 (11144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 22:50:46 (10348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:16:07 (10520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:16:08 (10520): No heartbeat from core client for 30 sec - exiting 01:16:09 (10520): No heartbeat from core client for 30 sec - exiting 01:16:10 (10520): No heartbeat from core client for 30 sec - exiting 01:16:11 (10520): No heartbeat from core client for 30 sec - exiting 01:22:15 (6452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:21:56 (6324): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:52:50 (13548): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:55:20 (8156): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:37:31 (12864): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:11:05 (13540): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 01:46:00 (14720): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:53:26 (16336): No heartbeat from core client for 30 sec - exiting 13:53:27 (16336): No heartbeat from core client for 30 sec - exiting 13:53:28 (16336): No heartbeat from core client for 30 sec - exiting 13:53:29 (16336): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:31:49 (21884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:53:18 (18808): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:00:49 (20400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:20:14 (16356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776B7217 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x776B6E0F read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_ygrr_1940_40_007615251/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
21 Dec 2011 00:45:49	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	259,200	429,445	1.6568
20 Dec 2011 06:22:18	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	233,280	385,756	1.6536
19 Dec 2011 13:23:45	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	207,360	342,252	1.6505
18 Dec 2011 21:55:01	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	181,440	299,050	1.6482
17 Dec 2011 05:13:04	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	155,520	256,646	1.6502
16 Dec 2011 14:38:23	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	129,600	213,525	1.6476
16 Dec 2011 00:49:27	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	103,680	170,057	1.6402
15 Dec 2011 08:44:30	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	77,760	127,737	1.6427
14 Dec 2011 15:04:46	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	51,840	84,846	1.6367
14 Dec 2011 00:50:34	959754	13773443	hadcm3n_ygrr_1940_40_007615251_2	25,920	43,220	1.6674