Task 11014802

Name	hadsm3dhet2_joxl_006595307_4
Workunit	6798680
Created	15 Mar 2010, 12:01:05 UTC
Sent	4 Oct 2010, 12:30:49 UTC
Report deadline	16 Sep 2011, 17:50:49 UTC
Received	24 Oct 2010, 20:12:45 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1042170
Run time	2 days 7 hours 28 min 50 sec
CPU time	2 days 3 hours 29 min 58 sec
Validate state	Invalid
Credit	1,091.68
Device peak FLOPS	2.54 GFLOPS
Application version	UK Met Office HadSM3 Slab Model v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6148, selfPID=6148, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=7028, selfPID=7028, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2592, selfPID=2592, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=588, selfPID=588, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4536, selfPID=4536, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1812, selfPID=1812, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=4312, selfPID=4312, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4312, selfPID=4312, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4816, selfPID=4816, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6008, selfPID=6008, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5580, selfPID=5580, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5576, selfPID=5576, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4240, selfPID=4240, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1236, selfPID=1236, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1512, selfPID=1512, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2116, selfPID=2116, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5416, selfPID=5416, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5232, selfPID=5232, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5972, selfPID=5972, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=960, selfPID=960, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5360, selfPID=5360, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=572, selfPID=572, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=368, selfPID=368, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=308, selfPID=308, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4840, selfPID=4840, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5280, selfPID=5280, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2912, selfPID=2912, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5728, selfPID=5728, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=3716, selfPID=3716, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4840, selfPID=4840, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=6496, selfPID=6496, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4272, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4272, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4272, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4272, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4680, selfPID=4680, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6288, selfPID=6288, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1752, selfPID=1752, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6664, selfPID=6664, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=5352, selfPID=5352, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4640, selfPID=4640, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1608, selfPID=1608, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5864, selfPID=5864, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6000, selfPID=6000, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5500, selfPID=5500, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5752, selfPID=5752, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=912, selfPID=912, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6172, selfPID=6172, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6668, selfPID=6668, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5696, selfPID=5696, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5356, selfPID=5356, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=3916, selfPID=3916, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6428, selfPID=6428, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6332, selfPID=6332, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5928, selfPID=5928, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5032, selfPID=5032, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2068, selfPID=2068, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4772, selfPID=4772, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6000, selfPID=6000, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6328, selfPID=6328, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1528, selfPID=1528, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=2292, selfPID=2292, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5516, selfPID=5516, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6100, selfPID=6100, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5424, selfPID=5424, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5116, selfPID=5116, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6996, selfPID=6996, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=5476, selfPID=5476, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=3696, selfPID=3696, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2680, selfPID=2680, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5484, selfPID=5484, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=5944, selfPID=5944, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4664, selfPID=4664, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5064, selfPID=5064, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=3996, selfPID=3996, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5192, selfPID=5192, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4260, selfPID=4260, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6988, selfPID=6988, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5396, selfPID=5396, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=132, selfPID=132, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=3536, selfPID=3536, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=6296, selfPID=6296, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4784, selfPID=4784, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4828, selfPID=4828, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5668, selfPID=5668, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=2584, selfPID=2584, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=4904, selfPID=4904, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5636, selfPID=5636, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=6784, selfPID=6784, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=1644, selfPID=1644, iMonCtr=1 CPDN Monitor - Quit request from BOINC... No Process Handle CPDN process is not running, exiting, bRetVal = 1, checkPID=5820, selfPID=5820, iMonCtr=1 CPDN Monitor - Quit request from BOINC... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6972, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6972, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6972, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6972, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6972, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6972, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 Model crashed: 7R called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
23 Oct 2010 15:53:49	1042170	11014802	hadsm3dhet2_joxl_006595307_4	118,822	174,059	1.4649
22 Oct 2010 15:48:40	1042170	11014802	hadsm3dhet2_joxl_006595307_4	108,020	158,371	1.4661
22 Oct 2010 11:08:05	1042170	11014802	hadsm3dhet2_joxl_006595307_4	97,218	142,640	1.4672
21 Oct 2010 18:11:07	1042170	11014802	hadsm3dhet2_joxl_006595307_4	86,416	126,999	1.4696
21 Oct 2010 10:31:56	1042170	11014802	hadsm3dhet2_joxl_006595307_4	75,614	111,484	1.4744
18 Oct 2010 14:06:38	1042170	11014802	hadsm3dhet2_joxl_006595307_4	64,812	95,676	1.4762
16 Oct 2010 21:44:16	1042170	11014802	hadsm3dhet2_joxl_006595307_4	54,010	79,757	1.4767
16 Oct 2010 15:05:19	1042170	11014802	hadsm3dhet2_joxl_006595307_4	43,208	63,700	1.4743
15 Oct 2010 22:38:06	1042170	11014802	hadsm3dhet2_joxl_006595307_4	32,406	47,795	1.4749
14 Oct 2010 22:03:15	1042170	11014802	hadsm3dhet2_joxl_006595307_4	21,604	31,863	1.4749
13 Oct 2010 09:23:18	1042170	11014802	hadsm3dhet2_joxl_006595307_4	10,802	15,978	1.4792