Task 15854270

Name	hadcm3n_4f44_2020_40_008392828_0
Workunit	8543687
Created	21 Jun 2013, 20:01:50 UTC
Sent	24 Jun 2013, 9:58:37 UTC
Report deadline	23 Sep 2013, 17:25:48 UTC
Received	16 Aug 2013, 9:15:10 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1279203
Run time	2 days 6 hours 11 min 16 sec
CPU time	2 days 1 hours 35 min 10 sec
Validate state	Invalid
Credit	1,244.16
Device peak FLOPS	2.72 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.64</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... 11:31:25 (1444): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 19:38:42 (3644): No heartbeat from core client for 30 sec - exiting 19:38:43 (3644): No heartbeat from core client for 30 sec - exiting 19:38:44 (3644): No heartbeat from core client for 30 sec - exiting 19:38:45 (3644): No heartbeat from core client for 30 sec - exiting 19:38:46 (3644): No heartbeat from core client for 30 sec - exiting 19:38:47 (3644): No heartbeat from core client for 30 sec - exiting 19:38:48 (3644): No heartbeat from core client for 30 sec - exiting 19:38:49 (3644): No heartbeat from core client for 30 sec - exiting 19:38:50 (3644): No heartbeat from core client for 30 sec - exiting 19:38:51 (3644): No heartbeat from core client for 30 sec - exiting 19:38:52 (3644): No heartbeat from core client for 30 sec - exiting 19:38:53 (3644): No heartbeat from core client for 30 sec - exiting 19:38:54 (3644): No heartbeat from core client for 30 sec - exiting 19:38:55 (3644): No heartbeat from core client for 30 sec - exiting 19:38:56 (3644): No heartbeat from core client for 30 sec - exiting 19:38:57 (3644): No heartbeat from core client for 30 sec - exiting 19:38:58 (3644): No heartbeat from core client for 30 sec - exiting 19:38:59 (3644): No heartbeat from core client for 30 sec - exiting 19:39:00 (3644): No heartbeat from core client for 30 sec - exiting 19:39:01 (3644): No heartbeat from core client for 30 sec - exiting 19:39:02 (3644): No heartbeat from core client for 30 sec - exiting 19:39:03 (3644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:28:14 (5624): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 12:52:58 (1800): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5628, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 15:54:25 (3212): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:22:38 (6976): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... 09:40:48 (124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7056, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:11:50 (3388): No heartbeat from core client for 30 sec - exiting 09:11:51 (3388): No heartbeat from core client for 30 sec - exiting 09:11:52 (3388): No heartbeat from core client for 30 sec - exiting 09:11:53 (3388): No heartbeat from core client for 30 sec - exiting 09:11:54 (3388): No heartbeat from core client for 30 sec - exiting 09:11:55 (3388): No heartbeat from core client for 30 sec - exiting 09:11:56 (3388): No heartbeat from core client for 30 sec - exiting 09:11:57 (3388): No heartbeat from core client for 30 sec - exiting 09:11:58 (3388): No heartbeat from core client for 30 sec - exiting 09:11:59 (3388): No heartbeat from core client for 30 sec - exiting 09:12:00 (3388): No heartbeat from core client for 30 sec - exiting 09:12:01 (3388): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 11:02:37 (5724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=568, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7160, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:52:36 (4952): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=376, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=376, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6304, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6304, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6304, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6304, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6304, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Aug 2013 09:19:44	1279203	15854270	hadcm3n_4f44_2020_40_008392828_0	103,680	167,098	1.6117
30 Jul 2013 09:50:14	1279203	15854270	hadcm3n_4f44_2020_40_008392828_0	77,760	126,324	1.6245
25 Jul 2013 09:30:15	1279203	15854270	hadcm3n_4f44_2020_40_008392828_0	51,840	84,169	1.6236
28 Jun 2013 10:19:10	1279203	15854270	hadcm3n_4f44_2020_40_008392828_0	25,920	39,350	1.5181