Task 11496193

Name	famous_ucit_1599_200_006649072_1
Workunit	6852444
Created	10 Jun 2010, 13:22:08 UTC
Sent	13 Aug 2010, 15:29:34 UTC
Report deadline	12 Nov 2010, 22:56:45 UTC
Received	24 Aug 2010, 21:48:57 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	786733
Run time	6 days 12 hours 27 min
CPU time	6 days 6 hours 48 min 42 sec
Validate state	Invalid
Credit	3,613.24
Device peak FLOPS	2.92 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.36</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77c1400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x827215d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81299c3] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7615bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7761400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8137f1a] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8128d21] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75b5bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (3 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77c3400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8270ec0] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (11 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77cb400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82f1594] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054c65] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf761fbb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (12 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf77c3400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82a5714] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8323874] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x829e450] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81ef859] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x831bcec] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832da0f] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7617bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf773f400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8137ed8] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8128d21] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7593bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (9 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7754400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82dbe6a] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82b0e6a] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832b7ba] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75a8bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4048, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (11 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7711400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82f1594] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054c65] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7565bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2536, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy (2536): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGSEGV: segmentation violation Stack trace (12 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7755400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82a5714] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83210a8] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x829e450] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81ef859] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x831bcec] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832da0f] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75a9bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4029, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (14 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7794400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81fdbc3] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81381e2] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8128d21] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82b668d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x823a5a7] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75e8bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2806, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf770d400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8137f1a] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8128d21] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81b6e3e] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82376ab] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7561bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2806, iMonCtr=1 Model crash detected, will try to restart... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (12 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf7759400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82a5714] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8323874] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x829e450] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x81ef859] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x831bcec] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832da0f] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf75adbb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3287, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (13 frames): /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xf76f5400] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8137ed8] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8128d21] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x82b668d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x823a5a7] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8077bf6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8314811] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf7549bb6] /var/lib/boinc/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=8130, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( (10975): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
24 Aug 2010 12:32:23	786733	11496193	famous_ucit_1599_200_006649072_1	1,095,146	539,284	0.4924
23 Aug 2010 18:49:15	786733	11496193	famous_ucit_1599_200_006649072_1	1,085,786	534,263	0.4921
23 Aug 2010 17:31:41	786733	11496193	famous_ucit_1599_200_006649072_1	1,076,426	529,745	0.4921
23 Aug 2010 16:10:31	786733	11496193	famous_ucit_1599_200_006649072_1	1,067,066	525,231	0.4922
23 Aug 2010 14:49:34	786733	11496193	famous_ucit_1599_200_006649072_1	1,057,706	520,449	0.4921
23 Aug 2010 13:16:04	786733	11496193	famous_ucit_1599_200_006649072_1	1,048,346	515,537	0.4918
23 Aug 2010 12:01:50	786733	11496193	famous_ucit_1599_200_006649072_1	1,038,986	511,025	0.4918
23 Aug 2010 10:46:27	786733	11496193	famous_ucit_1599_200_006649072_1	1,029,626	506,515	0.4919
23 Aug 2010 09:27:10	786733	11496193	famous_ucit_1599_200_006649072_1	1,020,266	502,003	0.4920
23 Aug 2010 08:11:53	786733	11496193	famous_ucit_1599_200_006649072_1	1,010,906	497,477	0.4921
23 Aug 2010 06:24:50	786733	11496193	famous_ucit_1599_200_006649072_1	1,001,546	492,961	0.4922
23 Aug 2010 05:11:19	786733	11496193	famous_ucit_1599_200_006649072_1	992,186	488,418	0.4923
23 Aug 2010 03:52:39	786733	11496193	famous_ucit_1599_200_006649072_1	982,826	483,886	0.4923
23 Aug 2010 02:38:32	786733	11496193	famous_ucit_1599_200_006649072_1	973,466	479,408	0.4925
23 Aug 2010 01:30:21	786733	11496193	famous_ucit_1599_200_006649072_1	964,106	474,929	0.4926
23 Aug 2010 01:26:34	786733	11496193	famous_ucit_1599_200_006649072_1	954,746	470,444	0.4927
22 Aug 2010 22:49:06	786733	11496193	famous_ucit_1599_200_006649072_1	945,386	465,960	0.4929
22 Aug 2010 21:30:15	786733	11496193	famous_ucit_1599_200_006649072_1	936,026	461,488	0.4930
22 Aug 2010 20:14:58	786733	11496193	famous_ucit_1599_200_006649072_1	926,666	457,009	0.4932
22 Aug 2010 19:02:35	786733	11496193	famous_ucit_1599_200_006649072_1	917,306	452,531	0.4933