Task 10993799

Name	hadsm3dhet2_jnb9_006593207_2
Workunit	6796580
Created	15 Mar 2010, 11:58:25 UTC
Sent	13 Oct 2010, 21:02:50 UTC
Report deadline	26 Sep 2011, 2:22:50 UTC
Received	31 Oct 2010, 21:08:41 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1095901
Run time
CPU time	21 hours 21 min
Validate state	Invalid
Credit	694.70
Device peak FLOPS	3.10 GFLOPS
Application version	UK Met Office HadSM3 Slab Model v6.07 windows_intelx86
Stderr	<core_client_version>6.2.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5660, iMonCtr=1 Model crash detected, will try to restart... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=4628, selfPID=4628, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=4848, selfPID=4848, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=3432, selfPID=3432, iMonCtr=1 CNo heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=3624, selfPID=3624, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=1048, selfPID=1048, iMonCtr=1 Suspended CPDN Monitor - Quit request from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=2576, selfPID=2576, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=3704, selfPID=3704, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=4276, selfPID=4276, iMonCtr=1 CPDN Monitor - Quit request from BOINC... CPDN process is not running, exiting, bRetVal = 1, checkPID=7056, selfPID=7056, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=6952, selfPID=6952, iMonCtr=1 No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting CPDN process is not running, exiting, bRetVal = 1, checkPID=4588, selfPID=4588, iMonCtr=1 No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - No 'heartbeat' from BOINC... No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting No heartbeat from core client for 30 sec - exiting forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... forrtl: Access is denied. CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5008, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Oct 2010 19:14:04	1095901	10993799	hadsm3dhet2_jnb9_006593207_2	75,614	70,392	0.9309
24 Oct 2010 20:51:36	1095901	10993799	hadsm3dhet2_jnb9_006593207_2	64,812	60,253	0.9297
22 Oct 2010 22:36:41	1095901	10993799	hadsm3dhet2_jnb9_006593207_2	54,010	50,141	0.9284
21 Oct 2010 07:11:26	1095901	10993799	hadsm3dhet2_jnb9_006593207_2	43,208	40,340	0.9336
19 Oct 2010 07:56:07	1095901	10993799	hadsm3dhet2_jnb9_006593207_2	32,406	30,607	0.9445
18 Oct 2010 11:28:00	1095901	10993799	hadsm3dhet2_jnb9_006593207_2	21,604	20,057	0.9284
15 Oct 2010 21:51:25	1095901	10993799	hadsm3dhet2_jnb9_006593207_2	10,802	9,666	0.8948