Task 15596083

Name	hadcm3n_4m8d_1940_40_008309099_0
Workunit	8460234
Created	7 Feb 2013, 19:51:13 UTC
Sent	7 Feb 2013, 19:59:08 UTC
Report deadline	10 May 2013, 3:26:19 UTC
Received	22 Feb 2013, 19:14:48 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1202811
Run time	12 days 21 hours 7 min 40 sec
CPU time	12 days 3 hours 28 min 42 sec
Validate state	Invalid
Credit	10,264.32
Device peak FLOPS	3.32 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>7.0.28</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> 15:44:41 (5184): No heartbeat from core client for 30 sec - exiting 15:44:42 (5184): No heartbeat from core client for 30 sec - exiting 15:44:43 (5184): No heartbeat from core client for 30 sec - exiting 15:44:44 (5184): No heartbeat from core client for 30 sec - exiting 15:44:45 (5184): No heartbeat from core client for 30 sec - exiting 15:44:46 (5184): No heartbeat from core client for 30 sec - exiting 15:44:47 (5184): No heartbeat from core client for 30 sec - exiting 15:44:48 (5184): No heartbeat from core client for 30 sec - exiting 15:44:49 (5184): No heartbeat from core client for 30 sec - exiting 15:44:51 (5184): No heartbeat from core client for 30 sec - exiting 15:44:52 (5184): No heartbeat from core client for 30 sec - exiting 15:44:53 (5184): No heartbeat from core client for 30 sec - exiting 15:44:54 (5184): No heartbeat from core client for 30 sec - exiting 15:44:55 (5184): No heartbeat from core client for 30 sec - exiting 15:44:56 (5184): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3744, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:19:41 (902588): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:19:43 (902588): No heartbeat from core client for 30 sec - exiting 21:19:44 (902588): No heartbeat from core client for 30 sec - exiting 21:19:45 (902588): No heartbeat from core client for 30 sec - exiting 21:19:46 (902588): No heartbeat from core client for 30 sec - exiting 21:19:47 (902588): No heartbeat from core client for 30 sec - exiting 21:19:48 (902588): No heartbeat from core client for 30 sec - exiting 21:19:49 (902588): No heartbeat from core client for 30 sec - exiting 21:19:50 (902588): No heartbeat from core client for 30 sec - exiting 21:19:51 (902588): No heartbeat from core client for 30 sec - exiting 21:19:52 (902588): No heartbeat from core client for 30 sec - exiting 21:20:43 (1790636): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:24:41 (1789292): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... cpdnmonitor: error reading file dataout/ocean_restart.day Model crashed: TEMPHIST: Write ERROR on history file for namelistNLIHISTO tmp/pipe_dummy 2048 Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5840, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Feb 2013 19:39:18	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	855,360	1,027,506	1.2013
20 Feb 2013 09:28:14	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	829,440	994,803	1.1994
19 Feb 2013 23:21:05	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	803,520	962,109	1.1974
19 Feb 2013 13:30:30	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	777,600	929,491	1.1953
19 Feb 2013 03:42:08	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	751,680	897,222	1.1936
18 Feb 2013 17:41:21	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	725,760	864,402	1.1910
18 Feb 2013 08:08:10	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	699,840	832,246	1.1892
17 Feb 2013 22:45:56	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	673,920	800,479	1.1878
17 Feb 2013 13:25:12	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	648,000	768,927	1.1866
17 Feb 2013 01:00:26	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	622,080	738,331	1.1869
16 Feb 2013 16:20:51	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	596,160	706,912	1.1858
15 Feb 2013 22:56:46	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	570,240	673,578	1.1812
15 Feb 2013 13:40:15	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	544,320	642,438	1.1803
15 Feb 2013 04:31:18	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	518,400	610,868	1.1784
14 Feb 2013 19:37:01	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	492,480	578,969	1.1756
14 Feb 2013 10:16:01	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	466,560	547,503	1.1735
14 Feb 2013 01:24:04	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	440,640	516,547	1.1723
13 Feb 2013 16:35:51	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	414,720	485,915	1.1717
13 Feb 2013 07:46:10	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	388,800	455,465	1.1715
12 Feb 2013 22:49:10	1202811	15596083	hadcm3n_4m8d_1940_40_008309099_0	362,880	425,311	1.1720