Task 12920727

Name	hadcm3n_o16x_1940_40_007264845_2
Workunit	7463085
Created	2 Jun 2011, 3:44:05 UTC
Sent	2 Jun 2011, 3:44:08 UTC
Report deadline	1 Sep 2011, 11:11:19 UTC
Received	27 Jul 2011, 16:10:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	902832
Run time	23 days 22 hours 3 min 57 sec
CPU time	22 days 6 hours 1 min 29 sec
Validate state	Invalid
Credit	11,508.48
Device peak FLOPS	2.04 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.18</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:16:58 (3200): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:09:14 (856): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=512, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... 08:28:28 (3420): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: There is not enough space on the disk. Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=716, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
27 Jul 2011 16:14:41	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	959,040	1,891,248	1.9720
27 Jul 2011 16:14:41	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	933,120	1,840,327	1.9722
27 Jul 2011 16:14:41	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	907,200	1,788,823	1.9718
27 Jul 2011 16:14:41	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	881,280	1,738,416	1.9726
25 Jul 2011 18:58:51	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	855,360	1,687,787	1.9732
25 Jul 2011 18:18:57	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	829,440	1,638,282	1.9752
25 Jul 2011 17:54:11	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	803,520	1,587,477	1.9757
25 Jul 2011 17:26:56	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	777,600	1,532,100	1.9703
07 Jul 2011 19:02:44	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	751,680	1,486,068	1.9770
06 Jul 2011 00:22:54	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	725,760	1,435,278	1.9776
05 Jul 2011 04:32:17	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	699,840	1,384,741	1.9787
03 Jul 2011 22:26:26	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	673,920	1,333,459	1.9787
03 Jul 2011 08:29:31	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	648,000	1,284,263	1.9819
02 Jul 2011 18:36:06	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	622,080	1,235,172	1.9856
02 Jul 2011 02:57:18	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	596,160	1,180,701	1.9805
01 Jul 2011 10:49:33	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	570,240	1,125,379	1.9735
30 Jun 2011 20:22:57	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	544,320	1,075,390	1.9757
30 Jun 2011 04:58:08	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	518,400	1,025,884	1.9789
29 Jun 2011 12:50:49	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	492,480	975,903	1.9816
28 Jun 2011 10:29:39	902832	12920727	hadcm3n_o16x_1940_40_007264845_2	466,560	919,772	1.9714