Task 13357924

Name	hadcm3n_t0pr_1940_40_007446863_1
Workunit	7644366
Created	9 Sep 2011, 16:37:48 UTC
Sent	16 Sep 2011, 20:37:17 UTC
Report deadline	17 Dec 2011, 4:04:28 UTC
Received	15 Oct 2011, 20:01:39 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1063295
Run time	4 days 15 hours 48 min 53 sec
CPU time	4 days 7 hours 17 min 16 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	3.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:03:52 (3532): No heartbeat from core client for 30 sec - exiting 19:03:53 (3532): No heartbeat from core client for 30 sec - exiting 19:03:54 (3532): No heartbeat from core client for 30 sec - exiting 19:03:55 (3532): No heartbeat from core client for 30 sec - exiting 19:03:56 (3532): No heartbeat from core client for 30 sec - exiting 19:03:57 (3532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:03:58 (3532): No heartbeat from core client for 30 sec - exiting 19:03:59 (3532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:23:08 (4772): No heartbeat from core client for 30 sec - exiting 11:23:09 (4772): No heartbeat from core client for 30 sec - exiting 11:23:10 (4772): No heartbeat from core client for 30 sec - exiting 11:23:11 (4772): No heartbeat from core client for 30 sec - exiting 11:23:12 (4772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:27:39 (4792): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:08:27 (4452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 16:26:42 (4112): No heartbeat from core client for 30 sec - exiting 16:26:43 (4112): No heartbeat from core client for 30 sec - exiting 16:26:44 (4112): No heartbeat from core client for 30 sec - exiting 16:26:46 (4112): No heartbeat from core client for 30 sec - exiting 16:26:47 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:26:48 (4112): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:49:22 (4524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:46:48 (4344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:51:01 (4976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:46:34 (3856): No heartbeat from core client for 30 sec - exiting 11:46:35 (3856): No heartbeat from core client for 30 sec - exiting 11:46:36 (3856): No heartbeat from core client for 30 sec - exiting 11:46:37 (3856): No heartbeat from core client for 30 sec - exiting 11:46:38 (3856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:25:10 (5268): No heartbeat from core client for 30 sec - exiting 22:25:11 (5268): No heartbeat from core client for 30 sec - exiting 22:25:12 (5268): No heartbeat from core client for 30 sec - exiting 22:25:13 (5268): No heartbeat from core client for 30 sec - exiting 22:25:14 (5268): No heartbeat from core client for 30 sec - exiting 22:25:15 (5268): No heartbeat from core client for 30 sec - exiting 22:25:16 (5268): No heartbeat from core client for 30 sec - exiting 22:25:17 (5268): No heartbeat from core client for 30 sec - exiting 22:25:18 (5268): No heartbeat from core client for 30 sec - exiting 22:25:19 (5268): No heartbeat from core client for 30 sec - exiting 22:25:20 (5268): No heartbeat from core client for 30 sec - exiting 22:25:21 (5268): No heartbeat from core client for 30 sec - exiting 22:25:22 (5268): No heartbeat from core client for 30 sec - exiting 22:25:23 (5268): No heartbeat from core client for 30 sec - exiting 22:25:24 (5268): No heartbeat from core client for 30 sec - exiting 22:25:25 (5268): No heartbeat from core client for 30 sec - exiting 22:25:26 (5268): No heartbeat from core client for 30 sec - exiting 22:25:27 (5268): No heartbeat from core client for 30 sec - exiting 22:25:28 (5268): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 18:23:52 (3552): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5612, iMonCtr=1 Model crash detected, will try to restart... 11:26:23 (3776): No heartbeat from core client for 30 sec - exiting 11:26:24 (3776): No heartbeat from core client for 30 sec - exiting 11:26:25 (3776): No heartbeat from core client for 30 sec - exiting 11:26:26 (3776): No heartbeat from core client for 30 sec - exiting 11:26:27 (3776): No heartbeat from core client for 30 sec - exiting 11:26:28 (3776): No heartbeat from core client for 30 sec - exiting 11:26:29 (3776): No heartbeat from core client for 30 sec - exiting 11:26:30 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:26:31 (3776): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:23:41 (5040): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:43:01 (4740): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4736, iMonCtr=1 Model crash detected, will try to restart... 12:40:41 (4824): No heartbeat from core client for 30 sec - exiting 12:40:42 (4824): No heartbeat from core client for 30 sec - exiting 12:40:43 (4824): No heartbeat from core client for 30 sec - exiting 12:40:44 (4824): No heartbeat from core client for 30 sec - exiting 12:40:45 (4824): No heartbeat from core client for 30 sec - exiting 12:40:46 (4824): No heartbeat from core client for 30 sec - exiting 12:40:48 (4824): No heartbeat from core client for 30 sec - exiting 12:40:49 (4824): No heartbeat from core client for 30 sec - exiting 12:40:50 (4824): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:39:47 (5644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:54:50 (4604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:49:57 (4192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77B33A93 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_t0pr_1940_40_007446863/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Oct 2011 19:03:58	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	259,200	371,832	1.4345
11 Oct 2011 19:52:23	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	233,280	334,885	1.4355
10 Oct 2011 20:19:34	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	207,360	297,570	1.4350
08 Oct 2011 20:40:23	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	181,440	260,475	1.4356
06 Oct 2011 21:38:05	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	155,520	223,430	1.4367
05 Oct 2011 02:28:40	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	129,600	186,241	1.4370
02 Oct 2011 21:11:48	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	103,680	148,750	1.4347
30 Sep 2011 23:44:44	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	77,760	112,215	1.4431
28 Sep 2011 17:48:52	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	51,840	74,759	1.4421
22 Sep 2011 23:33:24	1063295	13357924	hadcm3n_t0pr_1940_40_007446863_1	25,920	37,459	1.4452