Task 14036810

Name	hadcm3n_yfgv_1940_40_007743196_3
Workunit	7898304
Created	31 Jan 2012, 6:19:30 UTC
Sent	31 Jan 2012, 6:39:11 UTC
Report deadline	1 May 2012, 14:06:22 UTC
Received	23 Mar 2012, 19:01:53 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1113742
Run time	26 days 12 hours 48 min 42 sec
CPU time	25 days 10 hours 10 min 13 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	1.87 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4432, iMonCtr=1 Model crash detected, will try to restart... 12:48:27 (1724): No heartbeat from core client for 30 sec - exiting 12:48:28 (1724): No heartbeat from core client for 30 sec - exiting 12:48:29 (1724): No heartbeat from core client for 30 sec - exiting 12:48:30 (1724): No heartbeat from core client for 30 sec - exiting 12:48:32 (1724): No heartbeat from core client for 30 sec - exiting 12:48:33 (1724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:58:11 (4328): No heartbeat from core client for 30 sec - exiting 09:58:12 (4328): No heartbeat from core client for 30 sec - exiting 09:58:13 (4328): No heartbeat from core client for 30 sec - exiting 09:58:14 (4328): No heartbeat from core client for 30 sec - exiting 09:58:15 (4328): No heartbeat from core client for 30 sec - exiting 09:58:16 (4328): No heartbeat from core client for 30 sec - exiting 09:58:17 (4328): No heartbeat from core client for 30 sec - exiting 09:58:19 (4328): No heartbeat from core client for 30 sec - exiting 09:58:20 (4328): No heartbeat from core client for 30 sec - exiting 09:58:21 (4328): No heartbeat from core client for 30 sec - exiting 09:58:22 (4328): No heartbeat from core client for 30 sec - exiting 09:58:24 (4328): No heartbeat from core client for 30 sec - exiting 09:58:25 (4328): No heartbeat from core client for 30 sec - exiting 09:58:26 (4328): No heartbeat from core client for 30 sec - exiting 09:58:27 (4328): No heartbeat from core client for 30 sec - exiting 09:58:28 (4328): No heartbeat from core client for 30 sec - exiting 09:58:29 (4328): No heartbeat from core client for 30 sec - exiting 09:58:31 (4328): No heartbeat from core client for 30 sec - exiting 09:58:32 (4328): No heartbeat from core client for 30 sec - exiting 09:58:33 (4328): No heartbeat from core client for 30 sec - exiting 09:58:34 (4328): No heartbeat from core client for 30 sec - exiting 09:58:35 (4328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:55:35 (5316): No heartbeat from core client for 30 sec - exiting 09:55:36 (5316): No heartbeat from core client for 30 sec - exiting 09:55:38 (5316): No heartbeat from core client for 30 sec - exiting 09:55:39 (5316): No heartbeat from core client for 30 sec - exiting 09:55:40 (5316): No heartbeat from core client for 30 sec - exiting 09:55:41 (5316): No heartbeat from core client for 30 sec - exiting 09:55:42 (5316): No heartbeat from core client for 30 sec - exiting 09:55:43 (5316): No heartbeat from core client for 30 sec - exiting 09:55:44 (5316): No heartbeat from core client for 30 sec - exiting 09:55:45 (5316): No heartbeat from core client for 30 sec - exiting 09:55:47 (5316): No heartbeat from core client for 30 sec - exiting 09:55:48 (5316): No heartbeat from core client for 30 sec - exiting 09:55:49 (5316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:36:39 (5364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:57:43 (6604): No heartbeat from core client for 30 sec - exiting 17:57:44 (6604): No heartbeat from core client for 30 sec - exiting 17:57:45 (6604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:24:33 (5152): No heartbeat from core client for 30 sec - exiting 09:24:34 (5152): No heartbeat from core client for 30 sec - exiting 09:24:35 (5152): No heartbeat from core client for 30 sec - exiting 09:24:36 (5152): No heartbeat from core client for 30 sec - exiting 09:24:37 (5152): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:58:39 (4136): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:58:40 (4136): No heartbeat from core client for 30 sec - exiting 10:45:44 (4956): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5924, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5928, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2228, iMonCtr=1 Model crash detected, will try to restart... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77C23AB3 read attempt to address 0x00000000 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_yfgv_1940_40_007743196/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Mar 2012 18:01:57	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	1,036,800	2,196,604	2.1186
20 Mar 2012 11:11:12	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	1,010,880	2,137,362	2.1144
19 Mar 2012 17:06:42	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	984,960	2,079,644	2.1114
19 Mar 2012 01:07:04	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	959,040	2,021,288	2.1076
18 Mar 2012 07:50:23	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	933,120	1,963,623	2.1044
17 Mar 2012 14:56:00	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	907,200	1,903,901	2.0987
16 Mar 2012 21:47:01	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	881,280	1,843,266	2.0916
16 Mar 2012 04:14:59	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	855,360	1,782,620	2.0841
15 Mar 2012 11:23:59	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	829,440	1,723,294	2.0777
14 Mar 2012 19:19:14	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	803,520	1,664,961	2.0721
14 Mar 2012 00:40:38	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	777,600	1,601,561	2.0596
13 Mar 2012 05:55:37	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	751,680	1,536,197	2.0437
12 Mar 2012 13:02:28	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	725,760	1,474,554	2.0317
05 Mar 2012 10:45:27	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	699,840	1,418,410	2.0268
04 Mar 2012 16:37:02	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	673,920	1,362,017	2.0210
04 Mar 2012 00:39:41	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	648,000	1,305,300	2.0144
03 Mar 2012 08:48:58	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	622,080	1,249,456	2.0085
02 Mar 2012 16:15:12	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	596,160	1,191,145	1.9980
02 Mar 2012 00:17:22	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	570,240	1,135,310	1.9909
01 Mar 2012 08:20:35	1113742	14036810	hadcm3n_yfgv_1940_40_007743196_3	544,320	1,078,990	1.9823