Task 12391177

Name	famous_wcib_599_200_007061282_0
Workunit	7264582
Created	17 Dec 2010, 13:27:51 UTC
Sent	17 Dec 2010, 15:25:11 UTC
Report deadline	18 Mar 2011, 22:52:22 UTC
Received	2 Jan 2011, 23:14:24 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1264721
Run time	4 days 10 hours 57 min 39 sec
CPU time	4 days 9 hours 57 min 47 sec
Validate state	Invalid
Credit	2,810.31
Device peak FLOPS	3.19 GFLOPS
Application version	UK Met Office FAMOUS v6.11 windows_intelx86
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> The device does not recognize the command. (0x16) - exit code 22 (0x16) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 01:46:12 (3604): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3508, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 03:01:08 (2896): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3508, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 03:09:54 (3984): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3508, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 06:33:24 (3648): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3508, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 CPDN Monitor - Quit request from BOINC... 07:16:46 (2788): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Signal 11 received, exiting... 07:46:46 (2560): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3304, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 11:09:40 (3960): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 11:44:30 (516): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 12:38:32 (2924): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 14:23:03 (1736): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 14:43:18 (3812): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 15:05:02 (3556): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2372, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 11 received, exiting... 03:36:48 (2404): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 04:38:47 (1840): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 05:54:31 (648): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 09:42:52 (1200): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 10:33:36 (544): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 10:35:57 (2572): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 11:03:30 (1804): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 12:08:40 (180): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 12:45:39 (1676): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 14:27:29 (2376): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 14:54:40 (2448): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 15:01:03 (2004): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 15:20:23 (2768): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 16:08:23 (2756): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 16:30:30 (1772): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 16:39:00 (1876): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 16:58:32 (2844): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 17:11:38 (2712): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 17:26:03 (1132): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 16 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 16 Signal 11 received, exiting... 18:24:40 (2756): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 18:29:37 (2504): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Signal 11 received, exiting... 19:34:42 (2316): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2212, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:44:46 (2004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:37:28 (3044): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 19:36:20 (5204): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:35:11 (5604): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:34:09 (3844): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( 06:39:16 (1824): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Jan 2011 22:00:33	1008312	12391177	famous_wcib_599_200_007061282_0	851,786	378,942	0.4449
02 Jan 2011 20:58:49	1008312	12391177	famous_wcib_599_200_007061282_0	842,426	374,724	0.4448
02 Jan 2011 20:51:11	1008312	12391177	famous_wcib_599_200_007061282_0	833,066	370,505	0.4447
02 Jan 2011 19:47:49	1008312	12391177	famous_wcib_599_200_007061282_0	823,706	366,272	0.4447
02 Jan 2011 18:29:13	1008312	12391177	famous_wcib_599_200_007061282_0	814,346	362,036	0.4446
02 Jan 2011 16:08:57	1008312	12391177	famous_wcib_599_200_007061282_0	804,986	357,854	0.4445
02 Jan 2011 14:55:21	1008312	12391177	famous_wcib_599_200_007061282_0	795,626	353,610	0.4444
02 Jan 2011 13:46:11	1008312	12391177	famous_wcib_599_200_007061282_0	786,266	349,212	0.4441
02 Jan 2011 12:32:10	1008312	12391177	famous_wcib_599_200_007061282_0	776,906	344,819	0.4438
02 Jan 2011 11:17:54	1008312	12391177	famous_wcib_599_200_007061282_0	767,546	340,428	0.4435
02 Jan 2011 10:03:32	1008312	12391177	famous_wcib_599_200_007061282_0	758,186	336,008	0.4432
02 Jan 2011 08:49:52	1008312	12391177	famous_wcib_599_200_007061282_0	748,826	331,617	0.4428
02 Jan 2011 07:35:08	1008312	12391177	famous_wcib_599_200_007061282_0	739,466	327,256	0.4426
02 Jan 2011 06:29:18	1008312	12391177	famous_wcib_599_200_007061282_0	730,106	322,968	0.4424
02 Jan 2011 03:34:05	1008312	12391177	famous_wcib_599_200_007061282_0	720,746	318,734	0.4422
01 Jan 2011 14:05:37	1008312	12391177	famous_wcib_599_200_007061282_0	711,386	314,513	0.4421
01 Jan 2011 11:58:35	1008312	12391177	famous_wcib_599_200_007061282_0	702,026	310,124	0.4418
01 Jan 2011 09:22:01	1008312	12391177	famous_wcib_599_200_007061282_0	692,666	305,958	0.4417
01 Jan 2011 08:02:50	1008312	12391177	famous_wcib_599_200_007061282_0	683,306	301,590	0.4414
01 Jan 2011 06:54:33	1008312	12391177	famous_wcib_599_200_007061282_0	673,946	297,411	0.4413