Task 16044571

Name	hadcm3n_oegn_1900_40_008473866_0
Workunit	8624705
Created	27 Sep 2013, 10:24:40 UTC
Sent	28 Sep 2013, 14:48:12 UTC
Report deadline	28 Dec 2013, 22:15:23 UTC
Received	7 Oct 2013, 9:56:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1271167
Run time	4 days 8 hours 15 min 17 sec
CPU time	4 days 5 hours 47 min 5 sec
Validate state	Invalid
Credit	2,177.28
Device peak FLOPS	2.25 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oegn_1900_40_008473866/dataout/ocean_restart.day 05:49:17 (28068): No heartbeat from core client for 30 sec - exiting 05:49:23 (28068): No heartbeat from core client for 30 sec - exiting 05:49:25 (28068): No heartbeat from core client for 30 sec - exiting 05:49:26 (28068): No heartbeat from core client for 30 sec - exiting 05:49:27 (28068): No heartbeat from core client for 30 sec - exiting 05:49:28 (28068): No heartbeat from core client for 30 sec - exiting 05:49:29 (28068): No heartbeat from core client for 30 sec - exiting 05:49:30 (28068): No heartbeat from core client for 30 sec - exiting 05:49:31 (28068): No heartbeat from core client for 30 sec - exiting 05:49:32 (28068): No heartbeat from core client for 30 sec - exiting 05:49:33 (28068): No heartbeat from core client for 30 sec - exiting 05:49:34 (28068): No heartbeat from core client for 30 sec - exiting 05:49:35 (28068): No heartbeat from core client for 30 sec - exiting 05:49:37 (28068): No heartbeat from core client for 30 sec - exiting 05:49:38 (28068): No heartbeat from core client for 30 sec - exiting 05:49:39 (28068): No heartbeat from core client for 30 sec - exiting 05:49:40 (28068): No heartbeat from core client for 30 sec - exiting 05:49:41 (28068): No heartbeat from core client for 30 sec - exiting 05:49:42 (28068): No heartbeat from core client for 30 sec - exiting 05:49:43 (28068): No heartbeat from core client for 30 sec - exiting 05:49:44 (28068): No heartbeat from core client for 30 sec - exiting 05:49:45 (28068): No heartbeat from core client for 30 sec - exiting 05:49:46 (28068): No heartbeat from core client for 30 sec - exiting 05:49:47 (28068): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oegn_1900_40_008473866/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oegn_1900_40_008473866/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oegn_1900_40_008473866/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oegn_1900_40_008473866/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oegn_1900_40_008473866/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 cpdnmonitor: error reading file C:\ProgramData\BOINC/projects/climateprediction.net/hadcm3n_oegn_1900_40_008473866/dataout/ocean_restart.day BUFFIN: C I/O Error ferror - Unit 41 - Return code = 32 Model crashed: READDUMP: BAD BUFFIN OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Oct 2013 19:57:53	1271167	16044571	hadcm3n_oegn_1900_40_008473866_0	181,440	328,075	1.8082
03 Oct 2013 14:29:40	1271167	16044571	hadcm3n_oegn_1900_40_008473866_0	155,520	278,945	1.7936
02 Oct 2013 09:50:02	1271167	16044571	hadcm3n_oegn_1900_40_008473866_0	129,600	233,660	1.8029
01 Oct 2013 15:54:03	1271167	16044571	hadcm3n_oegn_1900_40_008473866_0	103,680	185,023	1.7846
30 Sep 2013 18:28:39	1271167	16044571	hadcm3n_oegn_1900_40_008473866_0	77,760	136,621	1.7570
29 Sep 2013 18:48:51	1271167	16044571	hadcm3n_oegn_1900_40_008473866_0	51,840	90,899	1.7535
29 Sep 2013 03:41:58	1271167	16044571	hadcm3n_oegn_1900_40_008473866_0	25,920	45,504	1.7556