Task 12426015

Name	famous_xp2a_799_200_007094313_0
Workunit	7297613
Created	18 Dec 2010, 17:01:10 UTC
Sent	22 Dec 2010, 15:56:50 UTC
Report deadline	23 Mar 2011, 23:24:01 UTC
Received	6 Jan 2011, 11:13:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1113596
Run time	3 days 11 hours 17 min 16 sec
CPU time	3 days 6 hours 18 min 32 sec
Validate state	Invalid
Credit	1,389.76
Device peak FLOPS	1.42 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... (1699): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 SIGSEGV: segmentation violation Stack trace (15 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xaa3400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82e705a] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81e4153] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x827157e] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81299c3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/libc.so.6(__libc_start_main+0xe7)[0x14cce7] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2187, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: Invalid argument BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (15 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xd96400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82e705a] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81e4153] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x827157e] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81299c3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/libc.so.6(__libc_start_main+0xe7)[0x9bbce7] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=19825, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (1743): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
05 Jan 2011 22:07:49	1113596	12426015	famous_xp2a_799_200_007094313_0	421,226	279,455	0.6634
05 Jan 2011 14:48:27	1113596	12426015	famous_xp2a_799_200_007094313_0	411,866	273,301	0.6636
05 Jan 2011 12:49:11	1113596	12426015	famous_xp2a_799_200_007094313_0	402,506	266,888	0.6631
05 Jan 2011 09:16:44	1113596	12426015	famous_xp2a_799_200_007094313_0	393,146	260,669	0.6630
05 Jan 2011 07:28:15	1113596	12426015	famous_xp2a_799_200_007094313_0	383,786	254,489	0.6631
04 Jan 2011 14:28:45	1113596	12426015	famous_xp2a_799_200_007094313_0	374,426	248,298	0.6631
04 Jan 2011 12:43:37	1113596	12426015	famous_xp2a_799_200_007094313_0	365,066	242,109	0.6632
04 Jan 2011 10:49:23	1113596	12426015	famous_xp2a_799_200_007094313_0	355,706	235,901	0.6632
04 Jan 2011 09:00:44	1113596	12426015	famous_xp2a_799_200_007094313_0	346,346	229,688	0.6632
03 Jan 2011 18:35:45	1113596	12426015	famous_xp2a_799_200_007094313_0	336,986	223,480	0.6632
03 Jan 2011 16:25:14	1113596	12426015	famous_xp2a_799_200_007094313_0	327,626	217,296	0.6632
03 Jan 2011 14:33:41	1113596	12426015	famous_xp2a_799_200_007094313_0	318,266	211,099	0.6633
03 Jan 2011 12:38:25	1113596	12426015	famous_xp2a_799_200_007094313_0	308,906	204,889	0.6633
03 Jan 2011 10:51:18	1113596	12426015	famous_xp2a_799_200_007094313_0	299,546	198,687	0.6633
02 Jan 2011 18:32:20	1113596	12426015	famous_xp2a_799_200_007094313_0	290,186	192,473	0.6633
01 Jan 2011 19:55:49	1113596	12426015	famous_xp2a_799_200_007094313_0	280,826	186,257	0.6632
31 Dec 2010 17:07:15	1113596	12426015	famous_xp2a_799_200_007094313_0	271,466	180,068	0.6633
31 Dec 2010 12:39:32	1113596	12426015	famous_xp2a_799_200_007094313_0	262,106	173,867	0.6633
31 Dec 2010 10:50:59	1113596	12426015	famous_xp2a_799_200_007094313_0	252,746	167,671	0.6634
31 Dec 2010 08:52:07	1113596	12426015	famous_xp2a_799_200_007094313_0	243,386	161,428	0.6633