Task 13371784

Name	hadcm3n_t5x1_1940_40_007452562_1
Workunit	7650065
Created	10 Sep 2011, 13:39:19 UTC
Sent	11 Sep 2011, 3:37:27 UTC
Report deadline	11 Dec 2011, 11:04:38 UTC
Received	15 Sep 2011, 13:02:54 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1148900
Run time	4 days 1 hours 26 min 17 sec
CPU time	3 days 14 hours 9 min 36 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	2.48 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... 10:31:58 (344): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:59 (344): No heartbeat from core client for 30 sec - exiting 10:32:00 (344): No heartbeat from core client for 30 sec - exiting 10:32:01 (344): No heartbeat from core client for 30 sec - exiting 10:32:02 (344): No heartbeat from core client for 30 sec - exiting 10:32:03 (344): No heartbeat from core client for 30 sec - exiting 10:32:04 (344): No heartbeat from core client for 30 sec - exiting 10:32:05 (344): No heartbeat from core client for 30 sec - exiting 10:32:06 (344): No heartbeat from core client for 30 sec - exiting 10:32:08 (344): No heartbeat from core client for 30 sec - exiting 10:32:09 (344): No heartbeat from core client for 30 sec - exiting 11:16:02 (3116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:16:03 (3116): No heartbeat from core client for 30 sec - exiting 11:16:04 (3116): No heartbeat from core client for 30 sec - exiting 11:16:05 (3116): No heartbeat from core client for 30 sec - exiting 11:16:06 (3116): No heartbeat from core client for 30 sec - exiting 11:16:07 (3116): No heartbeat from core client for 30 sec - exiting 11:16:09 (3116): No heartbeat from core client for 30 sec - exiting 11:16:10 (3116): No heartbeat from core client for 30 sec - exiting 11:16:11 (3116): No heartbeat from core client for 30 sec - exiting 11:16:12 (3116): No heartbeat from core client for 30 sec - exiting 11:16:13 (3116): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:52:46 (7204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2612, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
15 Sep 2011 07:58:55	1148900	13371784	hadcm3n_t5x1_1940_40_007452562_1	155,520	311,668	2.0040
14 Sep 2011 18:26:21	1148900	13371784	hadcm3n_t5x1_1940_40_007452562_1	129,600	266,245	2.0544
14 Sep 2011 03:49:08	1148900	13371784	hadcm3n_t5x1_1940_40_007452562_1	103,680	222,376	2.1448
13 Sep 2011 07:32:26	1148900	13371784	hadcm3n_t5x1_1940_40_007452562_1	77,760	167,015	2.1478
12 Sep 2011 15:08:59	1148900	13371784	hadcm3n_t5x1_1940_40_007452562_1	51,840	111,293	2.1469
11 Sep 2011 19:57:11	1148900	13371784	hadcm3n_t5x1_1940_40_007452562_1	25,920	55,819	2.1535