Task 15426219

Name	hadcm3n_zk14_1880_40_008199059_4
Workunit	8354183
Created	3 Nov 2012, 17:34:01 UTC
Sent	3 Nov 2012, 17:34:09 UTC
Report deadline	3 Feb 2013, 1:01:20 UTC
Received	8 Feb 2013, 17:33:34 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1240590
Run time	7 days 6 hours 38 min
CPU time	4 days 1 hours 58 min 51 sec
Validate state	Invalid
Credit	7,776.00
Device peak FLOPS	3.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3484, iMonCtr=1 Model crash detected, will try to restart... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=41468, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:40:59 (7204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITHEAD: I/O error tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Feb 2013 15:35:47	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	648,000	402,822	0.6216
08 Feb 2013 08:42:01	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	622,080	378,497	0.6084
08 Feb 2013 01:24:08	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	596,160	350,549	0.5880
30 Dec 2012 07:29:09	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	570,240	319,662	0.5606
22 Dec 2012 05:28:50	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	544,320	287,284	0.5278
22 Dec 2012 05:28:50	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	518,400	255,435	0.4927
01 Dec 2012 12:52:12	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	492,480	228,383	0.4637
01 Dec 2012 07:15:57	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	466,560	208,485	0.4469
01 Dec 2012 01:54:41	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	440,640	189,105	0.4292
30 Nov 2012 20:32:38	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	414,720	169,959	0.4098
30 Nov 2012 15:04:06	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	388,800	150,457	0.3870
30 Nov 2012 09:21:56	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	362,880	220,156	0.6067
30 Nov 2012 03:35:42	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	336,960	198,545	0.5892
29 Nov 2012 21:54:19	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	311,040	176,968	0.5690
29 Nov 2012 14:58:27	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	285,120	151,189	0.5303
23 Nov 2012 16:20:08	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	259,200	128,632	0.4963
23 Nov 2012 10:41:16	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	233,280	206,148	0.8837
23 Nov 2012 02:04:14	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	207,360	177,729	0.8571
22 Nov 2012 16:53:37	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	181,440	148,609	0.8191
06 Nov 2012 09:05:43	1240590	15426219	hadcm3n_zk14_1880_40_008199059_4	155,520	117,977	0.7586