Task 12827669

Name	hadcm3n_p4un_1900_40_007223871_1
Workunit	7422111
Created	26 Apr 2011, 15:30:39 UTC
Sent	29 Apr 2011, 0:01:43 UTC
Report deadline	29 Jul 2011, 7:28:54 UTC
Received	27 May 2011, 1:41:19 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1131576
Run time	11 days 7 hours 56 min 56 sec
CPU time	11 days 4 hours 43 min 35 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.83 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1016, selfPID=1016, iMonCtr=1 01:00:36 (3028): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:00:37 (3028): No heartbeat from core client for 30 sec - exiting 01:00:38 (3028): No heartbeat from core client for 30 sec - exiting 01:00:39 (3028): No heartbeat from core client for 30 sec - exiting 01:00:40 (3028): No heartbeat from core client for 30 sec - exiting 01:00:41 (3028): No heartbeat from core client for 30 sec - exiting 01:00:42 (3028): No heartbeat from core client for 30 sec - exiting 01:00:43 (3028): No heartbeat from core client for 30 sec - exiting 01:00:44 (3028): No heartbeat from core client for 30 sec - exiting 01:00:45 (3028): No heartbeat from core client for 30 sec - exiting 01:00:47 (3028): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3440, selfPID=3440, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4064, selfPID=4064, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3800, selfPID=3800, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6244, selfPID=6244, iMonCtr=1 09:37:34 (168): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:37:35 (168): No heartbeat from core client for 30 sec - exiting 09:37:36 (168): No heartbeat from core client for 30 sec - exiting 09:37:37 (168): No heartbeat from core client for 30 sec - exiting 09:37:38 (168): No heartbeat from core client for 30 sec - exiting 09:37:39 (168): No heartbeat from core client for 30 sec - exiting 09:37:40 (168): No heartbeat from core client for 30 sec - exiting 09:37:41 (168): No heartbeat from core client for 30 sec - exiting forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4356, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. 09:38:57 (4356): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:38:58 (4356): No heartbeat from core client for 30 sec - exiting 09:38:59 (4356): No heartbeat from core client for 30 sec - exiting 09:39:00 (4356): No heartbeat from core client for 30 sec - exiting 09:39:01 (4356): No heartbeat from core client for 30 sec - exiting 09:39:02 (4356): No heartbeat from core client for 30 sec - exiting 09:39:04 (4356): No heartbeat from core client for 30 sec - exiting 09:39:05 (4356): No heartbeat from core client for 30 sec - exiting 09:39:06 (4356): No heartbeat from core client for 30 sec - exiting 09:39:07 (4356): No heartbeat from core client for 30 sec - exiting 09:39:08 (4356): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3696, selfPID=3696, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4768, selfPID=4768, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5480, selfPID=5480, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1000, selfPID=1000, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5432, selfPID=5432, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4192, selfPID=4192, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5208, selfPID=5208, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2136, selfPID=2136, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2300, selfPID=2300, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4348, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=568, selfPID=568, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5760, selfPID=5760, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4060, selfPID=4060, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4720, selfPID=4720, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=6012, selfPID=6012, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1408, selfPID=1408, iMonCtr=1 Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=908, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4988, selfPID=4988, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4816, selfPID=4816, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1304, selfPID=1304, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4164, selfPID=4164, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3232, selfPID=3232, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2236, selfPID=2236, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4656, selfPID=4656, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5728, selfPID=5728, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=1824, selfPID=1824, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3324, selfPID=3324, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5508, selfPID=5508, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4780, selfPID=4780, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5072, selfPID=5072, iMonCtr=1 10:11:35 (4756): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:11:36 (4756): No heartbeat from core client for 30 sec - exiting 10:11:37 (4756): No heartbeat from core client for 30 sec - exiting 10:11:38 (4756): No heartbeat from core client for 30 sec - exiting 10:11:39 (4756): No heartbeat from core client for 30 sec - exiting 10:11:40 (4756): No heartbeat from core client for 30 sec - exiting 10:11:41 (4756): No heartbeat from core client for 30 sec - exiting 10:11:42 (4756): No heartbeat from core client for 30 sec - exiting 10:11:43 (4756): No heartbeat from core client for 30 sec - exiting 10:11:44 (4756): No heartbeat from core client for 30 sec - exiting 10:11:46 (4756): No heartbeat from core client for 30 sec - exiting Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=4416, selfPID=4416, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2468, selfPID=2468, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5108, selfPID=5108, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3780, selfPID=3780, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=2792, selfPID=2792, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5604, selfPID=5604, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=5432, selfPID=5432, iMonCtr=1 CPDN Monitor - Quit request from BOINC... Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3740, selfPID=3740, iMonCtr=1 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... No Process Handle Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=3576, selfPID=3576, iMonCtr=1 Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 May 2011 01:42:16	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	518,400	967,397	1.8661
26 May 2011 00:20:25	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	492,480	919,899	1.8679
24 May 2011 10:44:27	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	466,560	871,891	1.8688
23 May 2011 08:37:10	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	440,640	822,685	1.8670
22 May 2011 07:06:23	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	414,720	774,429	1.8674
21 May 2011 05:43:56	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	388,800	726,603	1.8688
20 May 2011 04:15:45	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	362,880	678,650	1.8702
19 May 2011 02:32:39	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	336,960	630,743	1.8719
18 May 2011 01:03:28	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	311,040	582,780	1.8736
16 May 2011 11:29:12	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	285,120	534,333	1.8741
15 May 2011 09:58:04	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	259,200	486,465	1.8768
14 May 2011 02:55:32	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	233,280	437,880	1.8771
12 May 2011 02:02:12	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	207,360	388,079	1.8715
10 May 2011 11:48:30	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	181,440	337,367	1.8594
09 May 2011 06:49:49	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	155,520	289,435	1.8611
06 May 2011 00:52:57	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	129,600	239,739	1.8498
03 May 2011 11:16:05	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	103,680	191,250	1.8446
02 May 2011 09:40:19	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	77,760	143,357	1.8436
01 May 2011 08:05:09	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	51,840	95,529	1.8428
30 Apr 2011 06:42:38	1131576	12827669	hadcm3n_p4un_1900_40_007223871_1	25,920	47,756	1.8424