Task 15598972

Name	hadcm3n_4cjt_1940_40_008311370_0
Workunit	8462505
Created	8 Feb 2013, 4:15:08 UTC
Sent	8 Feb 2013, 20:57:25 UTC
Report deadline	11 May 2013, 4:24:36 UTC
Received	17 Mar 2013, 8:08:46 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1212841
Run time	8 days 19 hours 33 min 3 sec
CPU time	8 days 17 hours 43 min 23 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1196, iMonCtr=1 Model crash detected, will try to restart... 12:24:44 (4724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:24:45 (4724): No heartbeat from core client for 30 sec - exiting 12:24:46 (4724): No heartbeat from core client for 30 sec - exiting 12:24:47 (4724): No heartbeat from core client for 30 sec - exiting 12:24:48 (4724): No heartbeat from core client for 30 sec - exiting 12:24:49 (4724): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4516, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4408, iMonCtr=1 Model crash detected, will try to restart... CSuspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5276, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4764, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4768, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3300, iMonCtr=1 Model crash detected, will try to restart... Atmos Hold Restart file rename failed on atmos_restart.hold BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Error converting file to netcdf: dataout/4cjtko.pjg3c10 Error converting file to netcdf: dataout/4cjtko.pig3c10 Error converting file to netcdf: dataout/4cjtko.pfg3c10 Error converting file to netcdf: dataout/4cjtka.phg3c10 Error converting file to netcdf: dataout/4cjtka.pgg3c10 Error converting file to netcdf: dataout/4cjtka.peg3c10 Error converting file to netcdf: dataout/4cjtka.pdg3c10 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4140, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=732, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_4cjt_1940_40_008311370/dataout/atmos_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_4cjt_1940_40_008311370/dataout/atmos_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_4cjt_1940_40_008311370/dataout/atmos_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_4cjt_1940_40_008311370/dataout/atmos_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 cpdnmonitor: cannot open input file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_4cjt_1940_40_008311370/dataout/atmos_restart.day after 11 attempts Model crashed: READ_FLH: I/O error tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Mar 2013 21:15:32	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	777,600	754,984	0.9709
16 Mar 2013 13:55:23	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	751,680	730,379	0.9717
15 Mar 2013 19:24:48	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	725,760	705,415	0.9720
12 Mar 2013 20:24:25	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	699,840	680,019	0.9717
10 Mar 2013 16:30:43	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	673,920	654,316	0.9709
10 Mar 2013 09:36:38	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	648,000	628,412	0.9698
09 Mar 2013 13:40:06	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	622,080	602,652	0.9688
09 Mar 2013 06:36:50	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	596,160	576,995	0.9679
06 Mar 2013 20:39:28	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	570,240	550,951	0.9662
03 Mar 2013 20:11:54	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	544,320	524,944	0.9644
03 Mar 2013 13:05:06	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	518,400	499,013	0.9626
01 Mar 2013 21:43:07	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	492,480	472,822	0.9601
27 Feb 2013 21:59:28	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	466,560	446,807	0.9577
25 Feb 2013 20:25:02	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	440,640	420,978	0.9554
24 Feb 2013 14:50:56	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	414,720	396,177	0.9553
23 Feb 2013 19:42:37	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	388,800	370,978	0.9542
23 Feb 2013 12:55:08	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	362,880	345,250	0.9514
22 Feb 2013 20:22:59	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	336,960	319,496	0.9482
19 Feb 2013 18:49:47	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	311,040	294,210	0.9459
17 Feb 2013 16:20:46	1212841	15598972	hadcm3n_4cjt_1940_40_008311370_0	285,120	269,587	0.9455