Task 11712450

Name	famous_v3oi_1399_200_006690536_1
Workunit	6893789
Created	26 Aug 2010, 15:54:44 UTC
Sent	6 Sep 2010, 6:23:51 UTC
Report deadline	6 Dec 2010, 13:51:02 UTC
Received	9 Sep 2010, 10:34:03 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1098907
Run time	1 days 12 hours 34 min 47 sec
CPU time	1 days 10 hours 23 min 56 sec
Validate state	Invalid
Credit	494.19
Device peak FLOPS	1.33 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb78ce400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7758c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1964, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb785e400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb76e8c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6809, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb772c400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb75b6c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6809, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb77b2400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb763cc76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=20438, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb787a400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7704c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=27689, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7813400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb769dc76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=31338, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7888400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7712c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=31338, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7779400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7603c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=3766, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb776c400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb75f6c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=12352, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb787f400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7709c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21608, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7703400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb758dc76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26318, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb78a9400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7733c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26318, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb77d7400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7661c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=31659, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7773400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb75fdc76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=11216, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb78db400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7765c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13777, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7825400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb76afc76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=13777, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb76e5400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb756fc76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=23530, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb783e400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb76c8c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=23530, iMonCtr=1 Model crash detected, will try to restart... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb774a400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb75d4c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=23530, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb77cc400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb7656c76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=880, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb77d1400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb765bc76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=880, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SETPOS: Seek Failed: Invalid argument SETPOS: Unit 61 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 61 to Word Address -198 Failed with Error Code -1 Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy Model crashed: P_TH_ADJ : NEGATIVE PRESSURE VALUE CREATED. tmp/pipe_dummy SETPOS: Seek Failed: Invalid argument SETPOS: Unit 61 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 61 to Word Address -198 Failed with Error Code -1 SETPOS: Seek Failed: Invalid argument SETPOS: Unit 61 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 61 to Word Address -198 Failed with Error Code -1 SIGSEGV: segmentation violation Stack trace (10 frames): /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(boinc_catch_signal+0x58)[0x83aa400] [0xb7824400] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8054d6b] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8067f6c] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x830cfc3] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x832dc4d] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x8330bdf] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu[0x83415d2] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6)[0xb76aec76] /var/lib/boinc-client/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu(pthread_mutex_unlock+0x149)[0x804c041] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1986, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( (1986): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
08 Sep 2010 04:46:57	1098907	11712450	famous_v3oi_1399_200_006690536_1	149,786	115,605	0.7718
08 Sep 2010 02:30:08	1098907	11712450	famous_v3oi_1399_200_006690536_1	140,426	108,437	0.7722
08 Sep 2010 00:11:35	1098907	11712450	famous_v3oi_1399_200_006690536_1	131,066	101,253	0.7725
07 Sep 2010 22:00:16	1098907	11712450	famous_v3oi_1399_200_006690536_1	121,706	94,395	0.7756
07 Sep 2010 19:46:47	1098907	11712450	famous_v3oi_1399_200_006690536_1	112,346	87,068	0.7750
07 Sep 2010 17:42:57	1098907	11712450	famous_v3oi_1399_200_006690536_1	102,986	80,400	0.7807
07 Sep 2010 15:27:23	1098907	11712450	famous_v3oi_1399_200_006690536_1	93,626	73,068	0.7804
07 Sep 2010 12:06:20	1098907	11712450	famous_v3oi_1399_200_006690536_1	84,266	65,457	0.7768
07 Sep 2010 09:40:38	1098907	11712450	famous_v3oi_1399_200_006690536_1	74,906	57,931	0.7734
07 Sep 2010 03:13:43	1098907	11712450	famous_v3oi_1399_200_006690536_1	65,546	51,102	0.7796
07 Sep 2010 00:27:23	1098907	11712450	famous_v3oi_1399_200_006690536_1	56,186	43,394	0.7723
06 Sep 2010 22:05:54	1098907	11712450	famous_v3oi_1399_200_006690536_1	46,826	35,961	0.7680
06 Sep 2010 20:03:18	1098907	11712450	famous_v3oi_1399_200_006690536_1	37,466	29,017	0.7745
06 Sep 2010 17:33:16	1098907	11712450	famous_v3oi_1399_200_006690536_1	28,106	21,136	0.7520
06 Sep 2010 15:36:16	1098907	11712450	famous_v3oi_1399_200_006690536_1	18,746	14,475	0.7722
06 Sep 2010 12:21:53	1098907	11712450	famous_v3oi_1399_200_006690536_1	9,386	7,492	0.7982