Task 15748304

Name	hadcm3n_49n5_1980_40_008333403_4
Workunit	8484264
Created	24 Apr 2013, 4:29:33 UTC
Sent	24 Apr 2013, 4:29:49 UTC
Report deadline	24 Jul 2013, 11:57:00 UTC
Received	9 May 2013, 5:44:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1272265
Run time	5 days 19 hours 6 min
CPU time	5 days 9 hours 1 min 16 sec
Validate state	Invalid
Credit	2,488.32
Device peak FLOPS	2.00 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:06:01 (5576): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:18:36 (7992): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:35:25 (9020): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/49n5ko.pji4c10 Error converting file to netcdf: dataout/49n5ko.pii4c10 Error converting file to netcdf: dataout/49n5ko.pfi4c10 Error converting file to netcdf: dataout/49n5ka.phi4c10 Error converting file to netcdf: dataout/49n5ka.pgi4c10 Error converting file to netcdf: dataout/49n5ka.pei4c10 Error converting file to netcdf: dataout/49n5ka.pdi4c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:23:51 (5264): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9112, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8660, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8660, iMonCtr=1 Model crash detected, will try to restart... 01:11:19 (3748): No heartbeat from core client for 30 sec - exiting 01:11:20 (3748): No heartbeat from core client for 30 sec - exiting 01:11:21 (3748): No heartbeat from core client for 30 sec - exiting 01:11:22 (3748): No heartbeat from core client for 30 sec - exiting 01:11:23 (3748): No heartbeat from core client for 30 sec - exiting 01:11:24 (3748): No heartbeat from core client for 30 sec - exiting 01:11:25 (3748): No heartbeat from core client for 30 sec - exiting 01:11:26 (3748): No heartbeat from core client for 30 sec - exiting 01:11:27 (3748): No heartbeat from core client for 30 sec - exiting 01:11:28 (3748): No heartbeat from core client for 30 sec - exiting 01:11:29 (3748): No heartbeat from core client for 30 sec - exiting 01:11:30 (3748): No heartbeat from core client for 30 sec - exiting 01:11:31 (3748): No heartbeat from core client for 30 sec - exiting 01:11:32 (3748): No heartbeat from core client for 30 sec - exiting 01:11:33 (3748): No heartbeat from core client for 30 sec - exiting 01:11:34 (3748): No heartbeat from core client for 30 sec - exiting 01:11:35 (3748): No heartbeat from core client for 30 sec - exiting 01:11:36 (3748): No heartbeat from core client for 30 sec - exiting 01:11:37 (3748): No heartbeat from core client for 30 sec - exiting 01:11:38 (3748): No heartbeat from core client for 30 sec - exiting 01:11:39 (3748): No heartbeat from core client for 30 sec - exiting 01:11:40 (3748): No heartbeat from core client for 30 sec - exiting 01:11:41 (3748): No heartbeat from core client for 30 sec - exiting 01:11:42 (3748): No heartbeat from core client for 30 sec - exiting 01:11:43 (3748): No heartbeat from core client for 30 sec - exiting 01:11:44 (3748): No heartbeat from core client for 30 sec - exiting 01:11:45 (3748): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=9020, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 May 2013 16:15:55	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	207,360	435,466	2.1000
02 May 2013 03:59:30	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	181,440	382,174	2.1063
01 May 2013 05:48:01	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	155,520	331,473	2.1314
29 Apr 2013 19:14:11	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	129,600	279,797	2.1589
28 Apr 2013 19:42:24	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	103,680	227,263	2.1920
27 Apr 2013 08:58:01	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	77,760	168,925	2.1724
26 Apr 2013 14:46:15	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	51,840	114,023	2.1995
25 Apr 2013 21:08:27	1272265	15748304	hadcm3n_49n5_1980_40_008333403_4	25,920	58,580	2.2600