Task 13783834

Name	hadcm3n_t5pi_1940_40_007448301_3
Workunit	7645804
Created	15 Dec 2011, 19:15:06 UTC
Sent	15 Dec 2011, 19:16:43 UTC
Report deadline	16 Mar 2012, 2:43:54 UTC
Received	11 Jan 2012, 21:01:16 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1090056
Run time	18 days 17 hours 30 min 36 sec
CPU time	14 days 20 hours 6 min 3 sec
Validate state	Invalid
Credit	7,464.96
Device peak FLOPS	2.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 windows_intelx86
Stderr	<core_client_version>6.12.34</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4072, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1108, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=5692, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4516, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3740, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2028, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2172, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2052, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3980, iMonCtr=1 Model crash detected, will try to restart... Signal 22 received, exiting... Called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3980, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 Jan 2012 05:43:14	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	622,080	1,265,077	2.0336
10 Jan 2012 06:42:52	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	596,160	1,201,705	2.0157
09 Jan 2012 11:08:57	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	570,240	1,134,116	1.9888
08 Jan 2012 15:21:23	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	544,320	1,066,422	1.9592
07 Jan 2012 19:46:34	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	518,400	999,052	1.9272
07 Jan 2012 01:30:43	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	492,480	951,327	1.9317
06 Jan 2012 03:36:59	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	466,560	904,843	1.9394
04 Jan 2012 17:59:30	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	440,640	857,822	1.9468
03 Jan 2012 15:12:47	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	414,720	810,937	1.9554
01 Jan 2012 16:44:07	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	388,800	763,131	1.9628
30 Dec 2011 10:06:27	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	362,880	713,967	1.9675
29 Dec 2011 16:36:18	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	336,960	662,819	1.9671
28 Dec 2011 21:35:24	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	311,040	610,819	1.9638
27 Dec 2011 23:12:27	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	285,120	556,395	1.9514
27 Dec 2011 02:15:56	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	259,200	505,727	1.9511
25 Dec 2011 14:30:03	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	233,280	457,083	1.9594
24 Dec 2011 10:05:10	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	207,360	404,545	1.9509
23 Dec 2011 07:46:03	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	181,440	354,053	1.9514
22 Dec 2011 09:23:49	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	155,520	297,707	1.9143
20 Dec 2011 21:44:52	1090056	13783834	hadcm3n_t5pi_1940_40_007448301_3	129,600	245,518	1.8944