Task 12737224

Name	hadcm3n_o22o_1900_40_007198019_0
Workunit	7396299
Created	28 Mar 2011, 14:02:19 UTC
Sent	1 Apr 2011, 7:52:01 UTC
Report deadline	1 Jul 2011, 15:19:12 UTC
Received	16 Jun 2011, 17:39:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1041883
Run time	61 days 12 hours 41 min 26 sec
CPU time	36 days 18 hours 20 min 56 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	1.93 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.26</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6072, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 07:00:02 (2988): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:00:54 (5292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:06:28 (512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 07:21:22 (3772): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:44:20 (3812): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:18:42 (4544): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:15 (4192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:16 (4192): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 20:33:09 (3052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:33:10 (3052): No heartbeat from core client for 30 sec - exiting 20:33:11 (3052): No heartbeat from core client for 30 sec - exiting 20:33:12 (3052): No heartbeat from core client for 30 sec - exiting 20:33:13 (3052): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 20:52:39 (5588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:13 (3876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:13:23 (5328): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 22:46:57 (3364): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:02:17 (3596): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:17:34 (3456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:17:35 (3456): No heartbeat from core client for 30 sec - exiting 23:17:36 (3456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 14:29:08 (4908): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 15:38:24 (5700): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:30:39 (5316): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:49:55 (4284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:49:56 (4284): No heartbeat from core client for 30 sec - exiting 21:49:57 (4284): No heartbeat from core client for 30 sec - exiting 22:26:07 (4644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 11:17:35 (5472): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:22 (5880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:36:23 (5880): No heartbeat from core client for 30 sec - exiting 23:20:42 (5520): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:58:40 (6632): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:58:41 (6632): No heartbeat from core client for 30 sec - exiting 00:58:42 (6632): No heartbeat from core client for 30 sec - exiting 09:31:51 (5284): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:46:22 (3436): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:50:11 (7108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:28:18 (1844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 02:45:50 (5712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 06:26:43 (752): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 00:51:12 (5456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:41:33 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:41:34 (4304): No heartbeat from core client for 30 sec - exiting 08:41:35 (4304): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:11:48 (2108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:17:52 (2460): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:51:38 (1456): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:54:35 (5212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:06:44 (5532): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:51:39 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:51:40 (4428): No heartbeat from core client for 30 sec - exiting 11:51:41 (4428): No heartbeat from core client for 30 sec - exiting 11:51:42 (4428): No heartbeat from core client for 30 sec - exiting 11:51:43 (4428): No heartbeat from core client for 30 sec - exiting 11:51:44 (4428): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:25:54 (5104): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:51:42 (560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:06:58 (1484): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:02 (6072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:02:03 (6072): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:36:27 (2424): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:36:28 (2424): No heartbeat from core client for 30 sec - exiting 09:36:29 (2424): No heartbeat from core client for 30 sec - exiting 09:36:30 (2424): No heartbeat from core client for 30 sec - exiting 09:36:31 (2424): No heartbeat from core client for 30 sec - exiting 09:36:32 (2424): No heartbeat from core client for 30 sec - exiting 12:15:06 (2168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:15:07 (2168): No heartbeat from core client for 30 sec - exiting 12:15:08 (2168): No heartbeat from core client for 30 sec - exiting 12:15:09 (2168): No heartbeat from core client for 30 sec - exiting 12:15:10 (2168): No heartbeat from core client for 30 sec - exiting 12:15:11 (2168): No heartbeat from core client for 30 sec - exiting 08:40:24 (1144): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:25 (1144): No heartbeat from core client for 30 sec - exiting 08:40:26 (1144): No heartbeat from core client for 30 sec - exiting 08:01:26 (4232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:40:59 (4260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:22:21 (3736): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:27:03 (3348): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:45:13 (3972): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:19 (4564): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:48:20 (4564): No heartbeat from core client for 30 sec - exiting 03:39:53 (3996): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:39:54 (3996): No heartbeat from core client for 30 sec - exiting 03:39:55 (3996): No heartbeat from core client for 30 sec - exiting 03:39:56 (3996): No heartbeat from core client for 30 sec - exiting 03:39:57 (3996): No heartbeat from core client for 30 sec - exiting 03:43:03 (5628): No heartbeat from core client for 30 sec - exiting 03:43:04 (5628): No heartbeat from core client for 30 sec - exiting 03:43:05 (5628): No heartbeat from core client for 30 sec - exiting 03:43:06 (5628): No heartbeat from core client for 30 sec - exiting 03:43:07 (5628): No heartbeat from core client for 30 sec - exiting 03:43:08 (5628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:52:05 (4296): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:52:06 (4296): No heartbeat from core client for 30 sec - exiting 03:52:07 (4296): No heartbeat from core client for 30 sec - exiting 03:52:08 (4296): No heartbeat from core client for 30 sec - exiting 03:52:09 (4296): No heartbeat from core client for 30 sec - exiting 03:52:10 (4296): No heartbeat from core client for 30 sec - exiting 04:07:16 (5732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 04:07:17 (5732): No heartbeat from core client for 30 sec - exiting 04:07:18 (5732): No heartbeat from core client for 30 sec - exiting 04:07:19 (5732): No heartbeat from core client for 30 sec - exiting 04:07:20 (5732): No heartbeat from core client for 30 sec - exiting 04:07:21 (5732): No heartbeat from core client for 30 sec - exiting 08:56:04 (2060): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:56:19 (5348): Can't acquire lockfile (32) - waiting 35s Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6100, iMonCtr=1 Model crash detected, will try to restart... 17:45:44 (7288): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 17:45:45 (7288): No heartbeat from core client for 30 sec - exiting </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Jun 2011 17:35:40	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	777,600	3,176,454	4.0849
13 Jun 2011 22:17:57	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	751,680	3,031,291	4.0327
11 Jun 2011 13:37:09	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	725,760	2,911,287	4.0114
09 Jun 2011 17:35:36	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	699,840	2,800,881	4.0022
07 Jun 2011 18:53:41	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	673,920	2,687,576	3.9880
04 Jun 2011 06:52:11	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	648,000	2,570,286	3.9665
21 May 2011 12:39:44	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	622,080	2,452,308	3.9421
18 May 2011 19:36:27	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	596,160	2,331,552	3.9110
16 May 2011 09:12:32	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	570,240	2,202,420	3.8623
14 May 2011 02:09:58	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	544,320	2,079,165	3.8197
11 May 2011 15:55:01	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	518,400	1,957,847	3.7767
08 May 2011 20:26:30	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	492,480	1,808,863	3.6730
05 May 2011 18:16:19	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	466,560	1,654,151	3.5454
03 May 2011 13:12:30	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	440,640	1,548,949	3.5152
01 May 2011 16:10:26	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	414,720	1,449,645	3.4955
29 Apr 2011 13:46:29	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	388,800	1,346,250	3.4626
27 Apr 2011 05:25:10	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	362,880	1,250,902	3.4472
24 Apr 2011 23:02:26	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	336,960	1,153,118	3.4221
22 Apr 2011 22:43:09	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	311,040	1,059,810	3.4073
21 Apr 2011 07:41:06	1041883	12737224	hadcm3n_o22o_1900_40_007198019_0	285,120	970,412	3.4035