Task 11766495

Name	famous_vc0r_1599_200_006701345_1
Workunit	6904598
Created	26 Aug 2010, 16:42:36 UTC
Sent	2 Dec 2010, 9:33:38 UTC
Report deadline	3 Mar 2011, 17:00:49 UTC
Received	4 Dec 2010, 8:20:38 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1039575
Run time	1 days 17 hours 23 min 52 sec
CPU time	1 days 15 hours 6 min 55 sec
Validate state	Invalid
Credit	679.48
Device peak FLOPS	1.78 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /home/BOINC/projects/climateprediction.net/famous_vc0r_1599_200_006701345/dataout/stdout_um.txt Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083E03A1 Unknown Unknown Unknown famous_um_6.11_i6 08267D05 Unknown Unknown Unknown famous_um_6.11_i6 082EFC2C Unknown Unknown Unknown famous_um_6.11_i6 08328714 Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 F761EC0E Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1337, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1337, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /home/BOINC/projects/climateprediction.net/famous_vc0r_1599_200_006701345/dataout/stdout_um.txt Image PC Routine Line Source famous_um_6.11_i6 0840BA88 Unknown Unknown Unknown famous_um_6.11_i6 0840A221 Unknown Unknown Unknown famous_um_6.11_i6 083ED787 Unknown Unknown Unknown famous_um_6.11_i6 083BA000 Unknown Unknown Unknown famous_um_6.11_i6 083DEAA4 Unknown Unknown Unknown famous_um_6.11_i6 080B47B7 Unknown Unknown Unknown famous_um_6.11_i6 0807C91A Unknown Unknown Unknown famous_um_6.11_i6 08314811 Unknown Unknown Unknown famous_um_6.11_i6 0832DC4D Unknown Unknown Unknown famous_um_6.11_i6 08330BDF Unknown Unknown Unknown famous_um_6.11_i6 083415D2 Unknown Unknown Unknown libc.so.6 F75DFC0E Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1337, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (11087): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
04 Dec 2010 09:01:28	1039575	11766495	famous_vc0r_1599_200_006701345_1	205,946	136,383	0.6622
04 Dec 2010 09:01:28	1039575	11766495	famous_vc0r_1599_200_006701345_1	196,586	130,238	0.6625
04 Dec 2010 09:01:28	1039575	11766495	famous_vc0r_1599_200_006701345_1	187,226	124,085	0.6628
03 Dec 2010 21:45:35	1039575	11766495	famous_vc0r_1599_200_006701345_1	177,866	117,928	0.6630
03 Dec 2010 19:21:23	1039575	11766495	famous_vc0r_1599_200_006701345_1	168,506	111,774	0.6633
03 Dec 2010 17:32:13	1039575	11766495	famous_vc0r_1599_200_006701345_1	159,146	105,622	0.6637
03 Dec 2010 15:34:34	1039575	11766495	famous_vc0r_1599_200_006701345_1	149,786	99,465	0.6640
03 Dec 2010 13:38:56	1039575	11766495	famous_vc0r_1599_200_006701345_1	140,426	93,300	0.6644
03 Dec 2010 11:45:32	1039575	11766495	famous_vc0r_1599_200_006701345_1	131,066	87,144	0.6649
03 Dec 2010 10:47:28	1039575	11766495	famous_vc0r_1599_200_006701345_1	121,706	80,981	0.6654
03 Dec 2010 09:06:41	1039575	11766495	famous_vc0r_1599_200_006701345_1	112,346	74,826	0.6660
03 Dec 2010 09:06:41	1039575	11766495	famous_vc0r_1599_200_006701345_1	102,986	68,659	0.6667
03 Dec 2010 09:06:41	1039575	11766495	famous_vc0r_1599_200_006701345_1	93,626	61,657	0.6585
03 Dec 2010 09:06:41	1039575	11766495	famous_vc0r_1599_200_006701345_1	84,266	55,489	0.6585
03 Dec 2010 00:08:19	1039575	11766495	famous_vc0r_1599_200_006701345_1	74,906	49,327	0.6585
02 Dec 2010 22:16:43	1039575	11766495	famous_vc0r_1599_200_006701345_1	65,546	43,163	0.6585
02 Dec 2010 20:26:42	1039575	11766495	famous_vc0r_1599_200_006701345_1	56,186	37,007	0.6587
02 Dec 2010 18:40:55	1039575	11766495	famous_vc0r_1599_200_006701345_1	46,826	30,853	0.6589
02 Dec 2010 16:49:56	1039575	11766495	famous_vc0r_1599_200_006701345_1	37,466	24,691	0.6590
02 Dec 2010 14:56:56	1039575	11766495	famous_vc0r_1599_200_006701345_1	28,106	18,523	0.6590