Task 11904174

Name	hadsm3dhet2_u0is_006726375_8
Workunit	6929718
Created	17 Sep 2010, 8:09:25 UTC
Sent	19 Sep 2010, 5:18:41 UTC
Report deadline	1 Sep 2011, 10:38:41 UTC
Received	22 Oct 2010, 12:29:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1095267
Run time	3 days 5 hours 40 min 39 sec
CPU time	3 days 5 hours 24 min 44 sec
Validate state	Invalid
Credit	2,282.60
Device peak FLOPS	2.66 GFLOPS
Application version	UK Met Office HadSM3 Slab Model v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=9196, selfPID=9196, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6664, selfPID=6664, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=3800, selfPID=3800, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7440, selfPID=7440, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7960, selfPID=7960, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6980, selfPID=6980, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7388, selfPID=7388, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6300, selfPID=6300, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1268, selfPID=1268, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=3744, selfPID=3744, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7104, selfPID=7104, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4704, selfPID=4704, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=6140, selfPID=6140, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4320, selfPID=4320, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6172, selfPID=6172, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7904, selfPID=7904, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2496, selfPID=2496, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=8688, selfPID=8688, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4508, selfPID=4508, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=9084, selfPID=9084, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5472, selfPID=5472, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5716, selfPID=5716, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=8380, selfPID=8380, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=6732, selfPID=6732, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6000, selfPID=6000, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5932, selfPID=5932, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5912, selfPID=5912, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2340, selfPID=2340, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7676, selfPID=7676, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5524, selfPID=5524, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1864, selfPID=1864, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4396, selfPID=4396, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2848, selfPID=2848, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7120, selfPID=7120, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=6224, selfPID=6224, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7440, selfPID=7440, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4092, selfPID=4092, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6464, selfPID=6464, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7464, selfPID=7464, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7424, selfPID=7424, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7640, selfPID=7640, iMonCtr=1 No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7308, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7308, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=7144, selfPID=7144, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=908, selfPID=908, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=6788, selfPID=6788, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7672, selfPID=7672, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=3048, selfPID=3048, iMonCtr=1 Suspended CPDN Monitor - Quit request from BOINC... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4204, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=3272, selfPID=3272, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7860, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7860, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7860, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7860, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7860, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7860, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( called boinc_finish CPDN process is not running, exiting, bRetVal = 1, checkPID=8080, selfPID=8080, iMonCtr=1 </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
22 Oct 2010 09:56:59	1095267	11904174	hadsm3dhet2_u0is_006726375_8	248,446	278,324	1.1203
20 Oct 2010 17:44:31	1095267	11904174	hadsm3dhet2_u0is_006726375_8	237,644	265,966	1.1192
18 Oct 2010 11:28:07	1095267	11904174	hadsm3dhet2_u0is_006726375_8	226,842	253,513	1.1176
16 Oct 2010 08:29:45	1095267	11904174	hadsm3dhet2_u0is_006726375_8	216,040	241,118	1.1161
15 Oct 2010 06:30:42	1095267	11904174	hadsm3dhet2_u0is_006726375_8	205,238	228,724	1.1144
14 Oct 2010 06:16:21	1095267	11904174	hadsm3dhet2_u0is_006726375_8	194,436	216,336	1.1126
11 Oct 2010 23:51:30	1095267	11904174	hadsm3dhet2_u0is_006726375_8	183,634	203,858	1.1101
11 Oct 2010 05:28:42	1095267	11904174	hadsm3dhet2_u0is_006726375_8	172,832	191,509	1.1081
09 Oct 2010 22:47:47	1095267	11904174	hadsm3dhet2_u0is_006726375_8	162,030	179,142	1.1056
09 Oct 2010 22:47:47	1095267	11904174	hadsm3dhet2_u0is_006726375_8	151,228	166,756	1.1027
09 Oct 2010 22:47:47	1095267	11904174	hadsm3dhet2_u0is_006726375_8	140,426	154,375	1.0993
09 Oct 2010 22:47:47	1095267	11904174	hadsm3dhet2_u0is_006726375_8	129,624	142,101	1.0963
03 Oct 2010 05:24:40	1095267	11904174	hadsm3dhet2_u0is_006726375_8	118,822	129,709	1.0916
02 Oct 2010 07:03:55	1095267	11904174	hadsm3dhet2_u0is_006726375_8	108,020	118,092	1.0932
01 Oct 2010 08:05:34	1095267	11904174	hadsm3dhet2_u0is_006726375_8	97,218	106,383	1.0943
30 Sep 2010 09:16:38	1095267	11904174	hadsm3dhet2_u0is_006726375_8	86,416	94,724	1.0961
29 Sep 2010 03:37:13	1095267	11904174	hadsm3dhet2_u0is_006726375_8	75,614	83,175	1.1000
26 Sep 2010 22:41:47	1095267	11904174	hadsm3dhet2_u0is_006726375_8	64,812	71,528	1.1036
26 Sep 2010 03:14:42	1095267	11904174	hadsm3dhet2_u0is_006726375_8	54,010	59,861	1.1083
24 Sep 2010 23:44:50	1095267	11904174	hadsm3dhet2_u0is_006726375_8	43,208	48,137	1.1141