Task 12928416

Name	hadcm3n_o7hh_1940_40_007267422_0
Workunit	7465662
Created	3 Jun 2011, 3:15:44 UTC
Sent	3 Jun 2011, 3:15:53 UTC
Report deadline	2 Sep 2011, 10:43:04 UTC
Received	3 Aug 2011, 1:19:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	890420
Run time	13 days 23 hours 36 min 16 sec
CPU time	12 days 10 hours 43 min 43 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	2.28 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> - exit code 193 (0xc1) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3832, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4844, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4740, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4760, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5188, iMonCtr=1 Model crash detected, will try to restart... 09:26:23 (4712): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:28:55 (4560): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5728, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4472, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5852, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4948, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4728, iMonCtr=1 Model crash detected, will try to restart... 09:49:50 (4932): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5760, iMonCtr=1 Model crash detected, will try to restart... 09:16:42 (4676): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5860, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4748, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4644, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4824, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4528, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4656, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5168, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4568, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4620, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4620, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4620, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5540, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4612, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5292, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4660, iMonCtr=1 Model crash detected, will try to restart... 09:06:09 (4788): No heartbeat from core client for 30 sec - exiting 09:06:14 (4788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4880, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4680, iMonCtr=1 Model crash detected, will try to restart... 10:05:08 (4924): No heartbeat from core client for 30 sec - exiting 10:05:14 (4924): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:07:51 (4120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5896, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5896, iMonCtr=1 Model crash detected, will try to restart... 09:23:03 (4832): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4308, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5556, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5556, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Aug 2011 10:29:34	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	518,400	1,075,398	2.0745
01 Aug 2011 03:24:53	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	492,480	1,021,930	2.0751
28 Jul 2011 05:53:57	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	466,560	968,577	2.0760
26 Jul 2011 08:22:17	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	440,640	915,256	2.0771
25 Jul 2011 22:28:41	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	414,720	861,925	2.0783
25 Jul 2011 22:28:41	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	388,800	808,454	2.0794
25 Jul 2011 17:46:54	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	362,880	754,977	2.0805
25 Jul 2011 16:46:55	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	336,960	700,260	2.0782
25 Jul 2011 16:46:55	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	311,040	646,881	2.0797
25 Jul 2011 16:46:55	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	285,120	593,611	2.0820
08 Jul 2011 05:17:24	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	259,200	540,041	2.0835
07 Jul 2011 15:39:13	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	233,280	486,619	2.0860
05 Jul 2011 02:10:57	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	207,360	433,216	2.0892
30 Jun 2011 03:57:46	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	181,440	379,878	2.0937
28 Jun 2011 06:27:42	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	155,520	326,541	2.0997
17 Jun 2011 08:30:29	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	129,600	273,247	2.1084
15 Jun 2011 12:23:08	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	103,680	219,195	2.1141
14 Jun 2011 05:10:22	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	77,760	164,533	2.1159
10 Jun 2011 07:18:13	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	51,840	109,615	2.1145
08 Jun 2011 09:51:30	890420	12928416	hadcm3n_o7hh_1940_40_007267422_0	25,920	55,172	2.1285