Task 15284382

Name	hadcm3n_zgb5_1880_40_008200721_2
Workunit	8355845
Created	14 Sep 2012, 9:17:09 UTC
Sent	14 Sep 2012, 9:26:40 UTC
Report deadline	14 Dec 2012, 16:53:51 UTC
Received	24 Oct 2012, 20:40:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1217816
Run time	8 days 23 hours 38 min 46 sec
CPU time	8 days 21 hours 56 min 37 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.37 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 03:05:20 (8528): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:07:22 (12020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:42:12 (5676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1616, iMonCtr=1 Model crash detected, will try to restart... 13:19:58 (3608): No heartbeat from core client for 30 sec - exiting 13:19:59 (3608): No heartbeat from core client for 30 sec - exiting 13:20:00 (3608): No heartbeat from core client for 30 sec - exiting 13:20:01 (3608): No heartbeat from core client for 30 sec - exiting 13:20:03 (3608): No heartbeat from core client for 30 sec - exiting 13:20:04 (3608): No heartbeat from core client for 30 sec - exiting 13:20:05 (3608): No heartbeat from core client for 30 sec - exiting 13:20:06 (3608): No heartbeat from core client for 30 sec - exiting 13:20:07 (3608): No heartbeat from core client for 30 sec - exiting 13:20:08 (3608): No heartbeat from core client for 30 sec - exiting 13:20:09 (3608): No heartbeat from core client for 30 sec - exiting 13:20:10 (3608): No heartbeat from core client for 30 sec - exiting 13:20:11 (3608): No heartbeat from core client for 30 sec - exiting 13:20:12 (3608): No heartbeat from core client for 30 sec - exiting 13:20:13 (3608): No heartbeat from core client for 30 sec - exiting 13:20:15 (3608): No heartbeat from core client for 30 sec - exiting 13:20:16 (3608): No heartbeat from core client for 30 sec - exiting 13:20:17 (3608): No heartbeat from core client for 30 sec - exiting 13:20:18 (3608): No heartbeat from core client for 30 sec - exiting 13:20:19 (3608): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5068, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2656, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5740, iMonCtr=1 Model crashCPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:45:58 (3908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6120, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5520, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3284, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5948, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5456, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5928, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unhandled Exception Detected... - Unhandled Exception Record - Reason: Access Violation (0xc0000005) at address 0x77743F99 read attempt to address 0xFFFFFFF8 Engaging BOINC Windows Runtime Debugger... Cannot serialize file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_zgb5_1880_40_008200721/dataout/shmem_restart.day Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Oct 2012 20:43:26	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	777,600	770,192	0.9905
24 Oct 2012 00:10:06	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	751,680	745,395	0.9916
23 Oct 2012 01:36:28	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	725,760	720,455	0.9927
21 Oct 2012 21:50:50	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	699,840	695,478	0.9938
20 Oct 2012 06:26:15	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	673,920	670,707	0.9952
19 Oct 2012 06:21:10	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	648,000	645,910	0.9968
16 Oct 2012 21:02:28	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	622,080	620,441	0.9974
15 Oct 2012 08:10:35	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	596,160	595,295	0.9985
13 Oct 2012 01:46:20	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	570,240	569,589	0.9989
02 Oct 2012 12:02:47	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	544,320	543,703	0.9989
01 Oct 2012 10:50:47	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	518,400	517,920	0.9991
01 Oct 2012 03:14:17	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	492,480	491,602	0.9982
29 Sep 2012 19:26:46	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	466,560	466,108	0.9990
28 Sep 2012 10:06:37	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	440,640	440,524	0.9997
27 Sep 2012 04:58:19	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	414,720	414,810	1.0002
26 Sep 2012 21:54:49	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	388,800	389,118	1.0008
26 Sep 2012 02:11:10	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	362,880	363,390	1.0014
25 Sep 2012 08:02:48	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	336,960	337,652	1.0021
25 Sep 2012 00:51:12	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	311,040	311,780	1.0024
24 Sep 2012 10:30:23	1217816	15284382	hadcm3n_zgb5_1880_40_008200721_2	285,120	286,069	1.0033