Task 13421855

Name	hadcm3n_u1sr_1980_40_007458216_3
Workunit	7655719
Created	26 Sep 2011, 4:03:23 UTC
Sent	26 Sep 2011, 4:03:25 UTC
Report deadline	26 Dec 2011, 11:30:36 UTC
Received	18 Oct 2011, 12:19:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1160470
Run time	14 days 6 hours 8 min 8 sec
CPU time	12 days 17 hours 17 min 4 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	1.85 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:15:21 (2544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4324, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=10484, iMonCtr=1 Model crash detected, will try to restart... 11:38:33 (7004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:16:52 (17184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:36:11 (15452): No heartbeat from core client for 30 sec - exiting 09:36:12 (15452): No heartbeat from core client for 30 sec - exiting 09:36:13 (15452): No heartbeat from core client for 30 sec - exiting 09:36:14 (15452): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:48:33 (1104): No heartbeat from core client for 30 sec - exiting 11:48:34 (1104): No heartbeat from core client for 30 sec - exiting 11:48:35 (1104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:48:36 (1104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:27:12 (12768): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:28 (8080): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:30:14 (7284): No heartbeat from core client for 30 sec - exiting 22:30:15 (7284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12152, iMonCtr=1 Model crash detected, will try to restart... 17:40:16 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:40:18 (3800): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6252, iMonCtr=1 Model crash detected, will try to restart... 21:04:12 (5036): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21016, iMonCtr=1 Model crash detected, will try to restart... 13:25:10 (288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7556, iMonCtr=1 Model crash detected, will try to restart... 23:55:07 (8104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:33:33 (7008): No heartbeat from core client for 30 sec - exiting 10:33:34 (7008): No heartbeat from core client for 30 sec - exiting 10:33:35 (7008): No heartbeat from core client for 30 sec - exiting 10:33:36 (7008): No heartbeat from core client for 30 sec - exiting 10:33:37 (7008): No heartbeat from core client for 30 sec - exiting 10:33:38 (7008): No heartbeat from core client for 30 sec - exiting 10:33:39 (7008): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:44:03 (6524): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:26:28 (2832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:54:41 (8484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:04:50 (6748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:05:22 (8016): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2376, iMonCtr=1 Model crash detected, will try to restart... 22:32:27 (7760): No heartbeat from core client for 30 sec - exiting 22:32:28 (7760): No heartbeat from core client for 30 sec - exiting 22:32:29 (7760): No heartbeat from core client for 30 sec - exiting 22:32:30 (7760): No heartbeat from core client for 30 sec - exiting 22:32:31 (7760): No heartbeat from core client for 30 sec - exiting 22:32:32 (7760): No heartbeat from core client for 30 sec - exiting 22:32:33 (7760): No heartbeat from core client for 30 sec - exiting 22:32:34 (7760): No heartbeat from core client for 30 sec - exiting 22:32:35 (7760): No heartbeat from core client for 30 sec - exiting 22:32:36 (7760): No heartbeat from core client for 30 sec - exiting 22:32:37 (7760): No heartbeat from core client for 30 sec - exiting 22:32:38 (7760): No heartbeat from core client for 30 sec - exiting 22:32:39 (7760): No heartbeat from core client for 30 sec - exiting 22:32:40 (7760): No heartbeat from core client for 30 sec - exiting 22:32:41 (7760): No heartbeat from core client for 30 sec - exiting 22:32:42 (7760): No heartbeat from core client for 30 sec - exiting 22:32:43 (7760): No heartbeat from core client for 30 sec - exiting 22:32:44 (7760): No heartbeat from core client for 30 sec - exiting 22:32:45 (7760): No heartbeat from core client for 30 sec - exiting 22:32:46 (7760): No heartbeat from core client for 30 sec - exiting 22:32:47 (7760): No heartbeat from core client for 30 sec - exiting 22:32:48 (7760): No heartbeat from core client for 30 sec - exiting 22:32:49 (7760): No heartbeat from core client for 30 sec - exiting 22:32:50 (7760): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:34:01 (1232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:39:06 (5532): No heartbeat from core client for 30 sec - exiting 10:39:07 (5532): No heartbeat from core client for 30 sec - exiting 10:39:08 (5532): No heartbeat from core client for 30 sec - exiting 10:39:09 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:39:10 (5532): No heartbeat from core client for 30 sec - exiting 10:39:11 (5532): No heartbeat from core client for 30 sec - exiting 10:39:12 (5532): No heartbeat from core client for 30 sec - exiting 10:39:13 (5532): No heartbeat from core client for 30 sec - exiting 10:39:14 (5532): No heartbeat from core client for 30 sec - exiting 10:39:15 (5532): No heartbeat from core client for 30 sec - exiting 10:39:16 (5532): No heartbeat from core client for 30 sec - exiting 10:39:17 (5532): No heartbeat from core client for 30 sec - exiting 10:39:18 (5532): No heartbeat from core client for 30 sec - exiting 14:16:44 (12564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:18:05 (12564): No heartbeat from core client for 30 sec - exiting 14:18:06 (12564): No heartbeat from core client for 30 sec - exiting 14:18:07 (12564): No heartbeat from core client for 30 sec - exiting 14:18:08 (12564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 15:24:24 (7184): No heartbeat from core client for 30 sec - exiting 15:24:25 (7184): No heartbeat from core client for 30 sec - exiting 15:24:26 (7184): No heartbeat from core client for 30 sec - exiting 15:24:27 (7184): No heartbeat from core client for 30 sec - exiting 15:24:28 (7184): No heartbeat from core client for 30 sec - exiting 15:24:29 (7184): No heartbeat from core client for 30 sec - exiting 15:24:30 (7184): No heartbeat from core client for 30 sec - exiting 15:24:31 (7184): No heartbeat from core client for 30 sec - exiting 15:24:32 (7184): No heartbeat from core client for 30 sec - exiting 15:24:33 (7184): No heartbeat from core client for 30 sec - exiting 15:24:34 (7184): No heartbeat from core client for 30 sec - exiting 15:24:35 (7184): No heartbeat from core client for 30 sec - exiting 15:24:36 (7184): No heartbeat from core client for 30 sec - exiting 15:24:37 (7184): No heartbeat from core client for 30 sec - exiting 15:24:38 (7184): No heartbeat from core client for 30 sec - exiting 15:24:39 (7184): No heartbeat from core client for 30 sec - exiting 15:24:40 (7184): No heartbeat from core client for 30 sec - exiting 15:24:41 (7184): No heartbeat from core client for 30 sec - exiting 15:24:42 (7184): No heartbeat from core client for 30 sec - exiting 15:24:43 (7184): No heartbeat from core client for 30 sec - exiting 15:24:44 (7184): No heartbeat from core client for 30 sec - exiting 15:24:45 (7184): No heartbeat from core client for 30 sec - exiting 15:24:46 (7184): No heartbeat from core client for 30 sec - exiting 15:24:47 (7184): No heartbeat from core client for 30 sec - exiting 15:24:48 (7184): No heartbeat from core client for 30 sec - exiting 15:24:49 (7184): No heartbeat from core client for 30 sec - exiting 15:24:50 (7184): No heartbeat from core client for 30 sec - exiting 15:24:51 (7184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Atmos Hold Restart file rename failed on atmos_restart.hold 15:43:10 (11108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:43:11 (11108): No heartbeat from core client for 30 sec - exiting 15:43:12 (11108): No heartbeat from core client for 30 sec - exiting 15:43:13 (11108): No heartbeat from core client for 30 sec - exiting 15:43:14 (11108): No heartbeat from core client for 30 sec - exiting 15:43:15 (11108): No heartbeat from core client for 30 sec - exiting 15:43:16 (11108): No heartbeat from core client for 30 sec - exiting 15:43:17 (11108): No heartbeat from core client for 30 sec - exiting 15:43:18 (11108): No heartbeat from core client for 30 sec - exiting 15:43:19 (11108): No heartbeat from core client for 30 sec - exiting 15:43:20 (11108): No heartbeat from core client for 30 sec - exiting 15:43:21 (11108): No heartbeat from core client for 30 sec - exiting 15:43:22 (11108): No heartbeat from core client for 30 sec - exiting 15:43:23 (11108): No heartbeat from core client for 30 sec - exiting Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=16052, iMonCtr=1 Model crash detected, will try to restart... 21:48:08 (7920): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:24:04 (8368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:24:05 (8368): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
18 Oct 2011 11:20:14	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	518,400	1,099,020	2.1200
17 Oct 2011 12:24:57	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	492,480	1,044,014	2.1199
16 Oct 2011 05:19:22	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	466,560	987,775	2.1171
15 Oct 2011 03:43:01	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	440,640	932,913	2.1172
14 Oct 2011 02:28:39	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	414,720	878,139	2.1174
13 Oct 2011 05:05:14	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	388,800	824,025	2.1194
11 Oct 2011 17:43:26	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	362,880	768,841	2.1187
11 Oct 2011 02:06:17	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	336,960	713,555	2.1176
09 Oct 2011 17:58:44	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	311,040	655,906	2.1088
08 Oct 2011 23:17:58	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	285,120	599,810	2.1037
08 Oct 2011 06:50:46	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	259,200	547,108	2.1108
07 Oct 2011 06:43:32	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	233,280	493,132	2.1139
07 Oct 2011 03:19:29	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	207,360	451,999	2.1798
06 Oct 2011 03:33:06	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	181,440	405,534	2.2351
04 Oct 2011 03:36:46	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	155,520	335,758	2.1589
02 Oct 2011 05:17:59	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	129,600	279,741	2.1585
30 Sep 2011 09:26:53	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	103,680	223,077	2.1516
30 Sep 2011 02:25:03	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	77,760	166,438	2.1404
28 Sep 2011 11:57:05	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	51,840	113,376	2.1870
27 Sep 2011 10:24:09	1160470	13421855	hadcm3n_u1sr_1980_40_007458216_3	25,920	59,871	2.3098