Task 11511407

Name	famous_uevb_1799_200_006652114_1
Workunit	6855486
Created	10 Jun 2010, 13:48:36 UTC
Sent	18 Aug 2010, 18:11:25 UTC
Report deadline	18 Nov 2010, 1:38:36 UTC
Received	9 Sep 2010, 19:34:01 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	997948
Run time	4 days 8 hours 31 min 46 sec
CPU time	4 days 3 hours 33 min 44 sec
Validate state	Invalid
Credit	2,130.92
Device peak FLOPS	2.37 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... (5134): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (6671): called boinc_finish CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (21310): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xf7740400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f906] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf747abd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (9700): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xf77ae400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f906] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf74e8bd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... (8286): called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x809e59c] [0xf7796400] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x804f906] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x805085a] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu[0x8050ad6] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf74d0bd6] ../../projects/climateprediction.net/famous_6.11_i686-pc-linux-gnu(__gxx_personality_v0+0xe1)[0x804c449] Exiting... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 3 received, exiting... (1851): called boinc_finish Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1846, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Signal 15 received, exiting... (1875): called boinc_finish Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (1705): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
09 Sep 2010 18:36:28	997948	11511407	famous_uevb_1799_200_006652114_1	645,866	354,870	0.5494
09 Sep 2010 17:04:27	997948	11511407	famous_uevb_1799_200_006652114_1	636,506	349,739	0.5495
09 Sep 2010 05:48:10	997948	11511407	famous_uevb_1799_200_006652114_1	627,146	344,521	0.5493
08 Sep 2010 19:08:06	997948	11511407	famous_uevb_1799_200_006652114_1	617,786	339,281	0.5492
08 Sep 2010 17:37:55	997948	11511407	famous_uevb_1799_200_006652114_1	608,426	333,989	0.5489
08 Sep 2010 05:39:23	997948	11511407	famous_uevb_1799_200_006652114_1	599,066	328,757	0.5488
07 Sep 2010 20:09:05	997948	11511407	famous_uevb_1799_200_006652114_1	589,706	323,587	0.5487
07 Sep 2010 18:42:39	997948	11511407	famous_uevb_1799_200_006652114_1	580,346	318,396	0.5486
07 Sep 2010 17:10:03	997948	11511407	famous_uevb_1799_200_006652114_1	570,986	313,146	0.5484
07 Sep 2010 05:55:16	997948	11511407	famous_uevb_1799_200_006652114_1	561,626	307,999	0.5484
06 Sep 2010 20:24:58	997948	11511407	famous_uevb_1799_200_006652114_1	552,266	302,838	0.5484
06 Sep 2010 18:53:51	997948	11511407	famous_uevb_1799_200_006652114_1	542,906	297,567	0.5481
05 Sep 2010 20:34:48	997948	11511407	famous_uevb_1799_200_006652114_1	533,546	292,406	0.5480
05 Sep 2010 18:58:25	997948	11511407	famous_uevb_1799_200_006652114_1	524,186	287,237	0.5480
05 Sep 2010 17:27:17	997948	11511407	famous_uevb_1799_200_006652114_1	514,826	282,139	0.5480
05 Sep 2010 12:42:01	997948	11511407	famous_uevb_1799_200_006652114_1	505,466	277,034	0.5481
05 Sep 2010 11:10:53	997948	11511407	famous_uevb_1799_200_006652114_1	496,106	271,912	0.5481
05 Sep 2010 09:44:49	997948	11511407	famous_uevb_1799_200_006652114_1	486,746	266,788	0.5481
04 Sep 2010 06:47:01	997948	11511407	famous_uevb_1799_200_006652114_1	477,386	261,611	0.5480
04 Sep 2010 06:08:01	997948	11511407	famous_uevb_1799_200_006652114_1	468,026	256,514	0.5481