Task 15488750

Name	hadcm3n_3mw6_1940_40_008260712_0
Workunit	8415836
Created	20 Dec 2012, 19:18:36 UTC
Sent	20 Dec 2012, 19:22:34 UTC
Report deadline	22 Mar 2013, 2:49:45 UTC
Received	22 Feb 2013, 22:04:22 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1185493
Run time	12 days 21 hours 57 min 48 sec
CPU time	12 days 13 hours 16 min 22 sec
Validate state	Invalid
Credit	5,287.68
Device peak FLOPS	1.59 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3364, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2624, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2624, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 08:12:48 (3372): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:12:50 (3372): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:39:21 (840): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2208, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 00:27:35 (1232): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:27:38 (1232): No heartbeat from core client for 30 sec - exiting 00:27:41 (1232): No heartbeat from core client for 30 sec - exiting Atmos Hold Restart file rename failed on atmos_restart.hold 10:19:14 (2508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:20:04 (2508): No heartbeat from core client for 30 sec - exiting 10:20:05 (2508): No heartbeat from core client for 30 sec - exiting 10:20:06 (2508): No heartbeat from core client for 30 sec - exiting 10:20:07 (2508): No heartbeat from core client for 30 sec - exiting 10:20:11 (2508): No heartbeat from core client for 30 sec - exiting 10:20:12 (2508): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2816, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 22:34:08 (3672): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=964, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3728, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Feb 2013 05:22:22	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	440,640	1,028,082	2.3332
19 Feb 2013 15:25:53	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	414,720	963,787	2.3239
19 Feb 2013 15:25:53	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	388,800	901,427	2.3185
19 Feb 2013 15:25:53	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	362,880	839,548	2.3136
19 Feb 2013 15:25:53	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	336,960	777,137	2.3063
13 Feb 2013 21:12:12	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	311,040	713,938	2.2953
12 Feb 2013 16:16:31	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	285,120	650,749	2.2824
11 Feb 2013 20:19:38	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	259,200	589,308	2.2736
10 Jan 2013 10:23:54	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	233,280	526,672	2.2577
09 Jan 2013 16:55:53	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	207,360	464,885	2.2419
08 Jan 2013 23:17:59	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	181,440	402,504	2.2184
04 Jan 2013 19:52:01	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	155,520	339,676	2.1841
04 Jan 2013 02:28:46	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	129,600	278,100	2.1458
03 Jan 2013 09:01:00	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	103,680	216,529	2.0884
03 Jan 2013 07:45:43	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	77,760	161,653	2.0789
28 Dec 2012 01:07:38	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	51,840	101,071	1.9497
21 Dec 2012 10:22:04	1185493	15488750	hadcm3n_3mw6_1940_40_008260712_0	25,920	52,850	2.0390