Task 14098521

Name	hadcm3n_yg71_1980_40_007752139_1
Workunit	7907248
Created	15 Feb 2012, 18:22:57 UTC
Sent	15 Feb 2012, 18:23:08 UTC
Report deadline	17 May 2012, 1:50:19 UTC
Received	14 Apr 2012, 17:26:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1099430
Run time	15 days 8 hours 10 min 48 sec
CPU time	10 days 14 hours 38 min 18 sec
Validate state	Invalid
Credit	10,575.36
Device peak FLOPS	3.00 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Ocean Restart file copy failed on yg71ko.dai1cm0 Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 17:58:32 (804): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:31:05 (5312): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:31:06 (5312): No heartbeat from core client for 30 sec - exiting 10:31:07 (5312): No heartbeat from core client for 30 sec - exiting 10:31:08 (5312): No heartbeat from core client for 30 sec - exiting Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=17972, iMonCtr=1 Model crash detected, will try to restart... 10:39:42 (9644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFOUT: C I/O Error - Return code = 32 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3472, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3472, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3472, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3472, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3472, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
14 Apr 2012 08:10:41	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	881,280	1,089,466	1.2362
13 Apr 2012 22:50:18	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	855,360	1,057,330	1.2361
13 Apr 2012 13:37:07	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	829,440	1,024,957	1.2357
13 Apr 2012 04:01:56	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	803,520	992,670	1.2354
12 Apr 2012 19:21:59	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	777,600	960,387	1.2351
12 Apr 2012 09:14:55	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	751,680	928,107	1.2347
11 Apr 2012 15:31:27	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	725,760	895,986	1.2345
11 Apr 2012 02:56:00	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	699,840	864,277	1.2350
10 Apr 2012 13:23:41	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	673,920	832,782	1.2357
10 Apr 2012 01:46:19	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	648,000	801,210	1.2364
09 Apr 2012 12:38:45	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	622,080	769,573	1.2371
09 Apr 2012 01:21:24	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	596,160	738,387	1.2386
08 Apr 2012 09:55:04	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	570,240	706,645	1.2392
07 Apr 2012 22:58:01	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	544,320	674,754	1.2396
07 Apr 2012 11:04:25	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	518,400	642,945	1.2402
07 Apr 2012 00:27:58	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	492,480	611,337	1.2413
06 Apr 2012 10:48:16	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	466,560	579,522	1.2421
06 Apr 2012 00:06:47	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	440,640	547,994	1.2436
05 Apr 2012 04:36:25	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	414,720	516,196	1.2447
04 Apr 2012 15:03:57	1099430	14098521	hadcm3n_yg71_1980_40_007752139_1	388,800	484,102	1.2451