Task 15521476

Name	hadcm3n_zcqz_1880_40_008252147_3
Workunit	8407271
Created	2 Jan 2013, 12:39:13 UTC
Sent	2 Jan 2013, 12:39:15 UTC
Report deadline	3 Apr 2013, 20:06:26 UTC
Received	7 Jan 2013, 1:39:35 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1228333
Run time	4 days 7 hours 58 min 46 sec
CPU time	3 days 20 hours 16 min 36 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> 01:23:30 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:56 (11948): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:31:59 (11948): No heartbeat from core client for 30 sec - exiting 02:45:15 (12408): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:45:16 (12408): No heartbeat from core client for 30 sec - exiting 02:45:17 (12408): No heartbeat from core client for 30 sec - exiting 02:45:18 (12408): No heartbeat from core client for 30 sec - exiting 02:45:19 (12408): No heartbeat from core client for 30 sec - exiting 02:45:20 (12408): No heartbeat from core client for 30 sec - exiting 02:45:21 (12408): No heartbeat from core client for 30 sec - exiting 02:45:22 (12408): No heartbeat from core client for 30 sec - exiting 06:16:50 (15316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:17:04 (15316): No heartbeat from core client for 30 sec - exiting 06:17:05 (15316): No heartbeat from core client for 30 sec - exiting 06:17:06 (15316): No heartbeat from core client for 30 sec - exiting 06:17:07 (15316): No heartbeat from core client for 30 sec - exiting 06:17:08 (15316): No heartbeat from core client for 30 sec - exiting 06:17:09 (15316): No heartbeat from core client for 30 sec - exiting 06:17:10 (15316): No heartbeat from core client for 30 sec - exiting 06:17:11 (15316): No heartbeat from core client for 30 sec - exiting 06:17:12 (15316): No heartbeat from core client for 30 sec - exiting 06:17:13 (15316): No heartbeat from core client for 30 sec - exiting 06:18:02 (11132): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:20:05 (9860): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 06:21:59 (11912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 07:13:32 (2180): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:39:36 (7960): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:35:14 (13704): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:48:58 (15752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:07:51 (9044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:07:52 (9044): No heartbeat from core client for 30 sec - exiting 02:07:53 (9044): No heartbeat from core client for 30 sec - exiting 02:07:54 (9044): No heartbeat from core client for 30 sec - exiting 02:07:55 (9044): No heartbeat from core client for 30 sec - exiting 02:07:56 (9044): No heartbeat from core client for 30 sec - exiting 02:07:57 (9044): No heartbeat from core client for 30 sec - exiting 02:07:58 (9044): No heartbeat from core client for 30 sec - exiting 02:08:01 (9044): No heartbeat from core client for 30 sec - exiting 02:31:48 (16376): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:31:50 (16376): No heartbeat from core client for 30 sec - exiting 02:31:51 (16376): No heartbeat from core client for 30 sec - exiting 02:31:52 (16376): No heartbeat from core client for 30 sec - exiting 02:31:53 (16376): No heartbeat from core client for 30 sec - exiting 02:31:54 (16376): No heartbeat from core client for 30 sec - exiting 02:31:55 (16376): No heartbeat from core client for 30 sec - exiting 02:31:56 (16376): No heartbeat from core client for 30 sec - exiting 02:31:57 (16376): No heartbeat from core client for 30 sec - exiting 02:31:58 (16376): No heartbeat from core client for 30 sec - exiting 02:34:01 (2716): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:34:05 (2716): No heartbeat from core client for 30 sec - exiting 02:45:20 (10364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:03:38 (8620): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:05:12 (11952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:06:57 (5524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:07:54 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:07:59 (5700): No heartbeat from core client for 30 sec - exiting 04:09:48 (6916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:09:57 (6916): No heartbeat from core client for 30 sec - exiting 04:09:58 (6916): No heartbeat from core client for 30 sec - exiting 05:46:09 (13724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:07:19 (16876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:07:22 (16876): No heartbeat from core client for 30 sec - exiting 06:07:24 (16876): No heartbeat from core client for 30 sec - exiting 06:08:00 (7340): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:09:06 (11688): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:14 (5852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:11:15 (5852): No heartbeat from core client for 30 sec - exiting 06:11:16 (5852): No heartbeat from core client for 30 sec - exiting 06:11:17 (5852): No heartbeat from core client for 30 sec - exiting 06:11:18 (5852): No heartbeat from core client for 30 sec - exiting 06:11:19 (5852): No heartbeat from core client for 30 sec - exiting 06:11:20 (5852): No heartbeat from core client for 30 sec - exiting 06:11:21 (5852): No heartbeat from core client for 30 sec - exiting 06:11:22 (5852): No heartbeat from core client for 30 sec - exiting 06:11:23 (5852): No heartbeat from core client for 30 sec - exiting 06:11:24 (5852): No heartbeat from core client for 30 sec - exiting 06:19:32 (11684): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:34:14 (13300): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 06:05:31 (12260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:05:36 (12260): No heartbeat from core client for 30 sec - exiting 06:05:37 (12260): No heartbeat from core client for 30 sec - exiting 06:05:38 (12260): No heartbeat from core client for 30 sec - exiting 06:05:39 (12260): No heartbeat from core client for 30 sec - exiting 06:05:40 (12260): No heartbeat from core client for 30 sec - exiting 07:45:25 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:45:26 (8824): No heartbeat from core client for 30 sec - exiting 07:45:27 (8824): No heartbeat from core client for 30 sec - exiting 07:45:28 (8824): No heartbeat from core client for 30 sec - exiting 07:45:29 (8824): No heartbeat from core client for 30 sec - exiting 07:45:30 (8824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 07:30:11 (17696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:35:20 (15796): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:46:50 (12768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:47:21 (8832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:47:22 (8832): No heartbeat from core client for 30 sec - exiting 19:47:23 (8832): No heartbeat from core client for 30 sec - exiting 19:47:24 (8832): No heartbeat from core client for 30 sec - exiting 19:47:25 (8832): No heartbeat from core client for 30 sec - exiting 19:47:26 (8832): No heartbeat from core client for 30 sec - exiting 19:47:27 (8832): No heartbeat from core client for 30 sec - exiting 19:47:28 (8832): No heartbeat from core client for 30 sec - exiting 00:49:15 (1760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:49:16 (1760): No heartbeat from core client for 30 sec - exiting 00:49:17 (1760): No heartbeat from core client for 30 sec - exiting 00:49:18 (1760): No heartbeat from core client for 30 sec - exiting 00:49:19 (1760): No heartbeat from core client for 30 sec - exiting 00:49:20 (1760): No heartbeat from core client for 30 sec - exiting 00:49:21 (1760): No heartbeat from core client for 30 sec - exiting 00:49:22 (1760): No heartbeat from core client for 30 sec - exiting 00:49:23 (1760): No heartbeat from core client for 30 sec - exiting 00:49:24 (1760): No heartbeat from core client for 30 sec - exiting 00:50:11 (19304): No heartbeat from core client for 30 sec - exiting 00:50:12 (19304): No heartbeat from core client for 30 sec - exiting 00:50:13 (19304): No heartbeat from core client for 30 sec - exiting 00:50:14 (19304):CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting 01:44:19 (11884): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:08:36 (8216): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:13:01 (17048): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:13:02 (17048): No heartbeat from core client for 30 sec - exiting 03:22:21 (3576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:22:29 (3576): No heartbeat from core client for 30 sec - exiting 03:22:35 (3576): No heartbeat from core client for 30 sec - exiting 03:22:37 (3576): No heartbeat from core client for 30 sec - exiting 03:22:38 (3576): No heartbeat from core client for 30 sec - exiting 03:33:13 (16712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:33:14 (16712): No heartbeat from core client for 30 sec - exiting 03:33:15 (16712): No heartbeat from core client for 30 sec - exiting 03:33:16 (16712): No heartbeat from core client for 30 sec - exiting 03:44:12 (19400): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:44:13 (19400): No heartbeat from core client for 30 sec - exiting 03:44:14 (19400): No heartbeat from core client for 30 sec - exiting 03:44:15 (19400): No heartbeat from core client for 30 sec - exiting 03:44:16 (19400): No heartbeat from core client for 30 sec - exiting 03:44:17 (19400): No heartbeat from core client for 30 sec - exiting 03:44:18 (19400): No heartbeat from core client for 30 sec - exiting 03:44:19 (19400): No heartbeat from core client for 30 sec - exiting 03:44:20 (19400): No heartbeat from core client for 30 sec - exiting 03:44:21 (19400): No heartbeat from core client for 30 sec - exiting 03:55:01 (11076): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:55:10 (11076): No heartbeat from core client for 30 sec - exiting 03:55:11 (11076): No heartbeat from core client for 30 sec - exiting 04:45:01 (6428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:27:26 (4916): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:27:35 (4916): No heartbeat from core client for 30 sec - exiting 07:27:36 (4916): No heartbeat from core client for 30 sec - exiting 07:27:40 (4916): No heartbeat from core client for 30 sec - exiting 07:27:41 (4916): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C73AB3 read attempt to address 0x4052E846 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zcqz_1880_40_008252147/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Jan 2013 01:44:30	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	259,200	332,194	1.2816
06 Jan 2013 13:22:03	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	233,280	300,944	1.2901
06 Jan 2013 02:39:47	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	207,360	269,919	1.3017
05 Jan 2013 15:45:53	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	181,440	238,474	1.3143
05 Jan 2013 06:35:20	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	155,520	207,396	1.3336
04 Jan 2013 22:10:11	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	129,600	176,865	1.3647
04 Jan 2013 13:31:39	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	103,680	145,906	1.4073
04 Jan 2013 02:53:52	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	77,760	114,514	1.4727
03 Jan 2013 14:45:45	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	51,840	80,679	1.5563
03 Jan 2013 03:04:45	1228333	15521476	hadcm3n_zcqz_1880_40_008252147_3	25,920	39,646	1.5296