Task 13016686

Name	hadcm3n_t47o_1940_40_007312601_0
Workunit	7510031
Created	28 Jun 2011, 4:03:56 UTC
Sent	28 Jun 2011, 4:05:22 UTC
Report deadline	27 Sep 2011, 11:32:33 UTC
Received	29 Aug 2011, 6:50:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1089771
Run time	15 days 7 hours 4 min 1 sec
CPU time	14 days 20 hours 21 min 55 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	2.42 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.56</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... BUFFIN: C I/O Error feof - Unit 63 - Return code = 16 BUFFIN: C I/O Error feof - Unit 64 - Return code = 16 BUFFIN: C I/O Error feof - Unit 65 - Return code = 16 BUFFIN: C I/O Error feof - Unit 66 - Return code = 16 BUFFIN: C I/O Error feof - Unit 67 - Return code = 16 BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/atmos_restart.day after 11 attempts cpdnmonitor: cannot open input file C:\Users\zingerli\AppData\Local\Climate175/projects/climateprediction.net/hadcm3n_t47o_1940_40_007312601/dataout/ocean_restart.day after 11 attempts Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4944, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
25 Jul 2011 22:48:46	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	881,280	1,282,197	1.4549
25 Jul 2011 22:17:18	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	855,360	1,243,431	1.4537
25 Jul 2011 21:55:04	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	829,440	1,204,669	1.4524
25 Jul 2011 21:05:48	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	803,520	1,165,867	1.4509
25 Jul 2011 20:38:04	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	777,600	1,126,576	1.4488
25 Jul 2011 20:24:52	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	751,680	1,086,992	1.4461
25 Jul 2011 19:11:28	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	725,760	1,042,674	1.4367
25 Jul 2011 17:39:33	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	699,840	997,157	1.4248
25 Jul 2011 16:25:21	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	673,920	954,466	1.4163
25 Jul 2011 16:00:29	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	648,000	911,581	1.4068
25 Jul 2011 15:42:56	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	622,080	868,329	1.3958
25 Jul 2011 15:02:07	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	596,160	824,793	1.3835
25 Jul 2011 13:03:28	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	570,240	782,014	1.3714
25 Jul 2011 13:03:28	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	544,320	742,202	1.3635
25 Jul 2011 13:03:28	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	518,400	701,822	1.3538
25 Jul 2011 13:03:28	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	492,480	661,525	1.3433
10 Jul 2011 18:11:02	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	466,560	622,339	1.3339
10 Jul 2011 06:51:03	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	440,640	583,415	1.3240
09 Jul 2011 19:06:00	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	414,720	544,236	1.3123
09 Jul 2011 08:25:13	1089771	13016686	hadcm3n_t47o_1940_40_007312601_0	388,800	504,761	1.2983