Task 12346960

Name	famous_vf5a_1899_200_006705396_6
Workunit	6908649
Created	27 Nov 2010, 22:23:57 UTC
Sent	27 Nov 2010, 22:41:41 UTC
Report deadline	27 Feb 2011, 6:08:52 UTC
Received	29 Nov 2010, 10:25:50 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	922557
Run time
CPU time	5 hours 49 min 36 sec
Validate state	Invalid
Credit	61.85
Device peak FLOPS	1.61 GFLOPS
Application version	UK Met Office FAMOUS v6.11 i686-pc-linux-gnu
Stderr	<core_client_version>5.10.45</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Starting work on result famous_vf5a_1899_200_006705396_6... Starting model in /home/jsc/BOINC/projects/climateprediction.net... Unzipping global model data file... Unzipping workunit data file... Created shared memory region key = 135765 of size 743572 bytes (version 608) Copying files for startup... In pre_initialise_phase (part 1 of 3) In initialise_phase (part 2 of 3) In startup_phase (part 3 of 3) Copying model control files... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 17596 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0000001 A - 01/12/1899 01:00 - H:M:S=0000:00:00 AVG= 0.00 DLT= 0.00 famous_vf5a_1899_200_006705396 - PH 1 TS 0000391 A - 16/12/1899 01:00 - H:M:S=0000:04:39 AVG= 0.71 DLT= 1.79 famous_vf5a_1899_200_006705396 - PH 1 TS 0000781 A - 01/01/1900 01:00 - H:M:S=0000:09:17 AVG= 0.71 DLT= 1.53 famous_vf5a_1899_200_006705396 - PH 1 TS 0001171 A - 16/01/1900 01:00 - H:M:S=0000:13:55 AVG= 0.71 DLT= 1.18 famous_vf5a_1899_200_006705396 - PH 1 TS 0001561 A - 01/02/1900 01:00 - H:M:S=0000:18:29 AVG= 0.71 DLT= 1.84 famous_vf5a_1899_200_006705396 - PH 1 TS 0001951 A - 16/02/1900 01:00 - H:M:S=0000:23:01 AVG= 0.71 DLT= 1.40 famous_vf5a_1899_200_006705396 - PH 1 TS 0002341 A - 01/03/1900 01:00 - H:M:S=0000:27:33 AVG= 0.71 DLT= 1.20 famous_vf5a_1899_200_006705396 - PH 1 TS 0002731 A - 16/03/1900 01:00 - H:M:S=0000:32:24 AVG= 0.71 DLT= 1.32 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Program launched with process id # 20059 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 famous_vf5a_1899_200_006705396 - PH 1 TS 0002731 A - 16/03/1900 01:00 - H:M:S=0000:32:40 AVG= 0.72 DLT= 0.00 famous_vf5a_1899_200_006705396 - PH 1 TS 0003121 A - 01/04/1900 01:00 - H:M:S=0000:37:16 AVG= 0.72 DLT= 1.58 famous_vf5a_1899_200_006705396 - PH 1 TS 0003511 A - 16/04/1900 01:00 - H:M:S=0000:41:47 AVG= 0.71 DLT= 1.79 famous_vf5a_1899_200_006705396 - PH 1 TS 0003901 A - 01/05/1900 01:00 - H:M:S=0000:46:18 AVG= 0.71 DLT= 1.23 famous_vf5a_1899_200_006705396 - PH 1 TS 0004291 A - 16/05/1900 01:00 - H:M:S=0000:50:52 AVG= 0.71 DLT= 1.33 famous_vf5a_1899_200_006705396 - PH 1 TS 0004681 A - 01/06/1900 01:00 - H:M:S=0000:55:25 AVG= 0.71 DLT= 1.76 famous_vf5a_1899_200_006705396 - PH 1 TS 0005071 A - 16/06/1900 01:00 - H:M:S=0000:59:54 AVG= 0.71 DLT= 1.59 famous_vf5a_1899_200_006705396 - PH 1 TS 0005461 A - 01/07/1900 01:00 - H:M:S=0001:04:27 AVG= 0.71 DLT= 1.74 famous_vf5a_1899_200_006705396 - PH 1 TS 0005851 A - 16/07/1900 01:00 - H:M:S=0001:09:01 AVG= 0.71 DLT= 1.65 famous_vf5a_1899_200_006705396 - PH 1 TS 0006241 A - 01/08/1900 01:00 - H:M:S=0001:13:34 AVG= 0.71 DLT= 1.49 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 22467 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0006631 A - 16/08/1900 01:00 - H:M:S=0001:19:54 AVG= 0.72 DLT= 1.68 famous_vf5a_1899_200_006705396 - PH 1 TS 0007021 A - 01/09/1900 01:00 - H:M:S=0001:24:25 AVG= 0.72 DLT= 1.40 famous_vf5a_1899_200_006705396 - PH 1 TS 0007411 A - 16/09/1900 01:00 - H:M:S=0001:29:02 AVG= 0.72 DLT= 1.80 famous_vf5a_1899_200_006705396 - PH 1 TS 0007801 A - 01/10/1900 01:00 - H:M:S=0001:33:41 AVG= 0.72 DLT= 1.11 famous_vf5a_1899_200_006705396 - PH 1 TS 0008191 A - 16/10/1900 01:00 - H:M:S=0001:38:19 AVG= 0.72 DLT= 1.42 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 23853 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 2... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 23905 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0008581 A - 01/11/1900 01:00 - H:M:S=0001:48:55 AVG= 0.76 DLT= 1.86 famous_vf5a_1899_200_006705396 - PH 1 TS 0008971 A - 16/11/1900 01:00 - H:M:S=0001:53:28 AVG= 0.76 DLT= 1.78 Trickling yearly means for 1900 Building file list for 1900 in dataout -- 4 entries Creating trickle file trickle_famous_vf5a_1899_200_006705396_6_1900.zip for upload... famous_vf5a_1899_200_006705396 - PH 1 TS 0009361 A - 01/12/1900 01:00 - H:M:S=0001:57:58 AVG= 0.76 DLT= 1.49 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 24745 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 60 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 61 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 famous_vf5a_1899_200_006705396 - PH 1 TS 0009751 A - 16/12/1900 01:00 - H:M:S=0002:05:22 AVG= 0.77 DLT= 1.58 famous_vf5a_1899_200_006705396 - PH 1 TS 0010141 A - 01/01/1901 01:00 - H:M:S=0002:09:50 AVG= 0.77 DLT= 1.28 famous_vf5a_1899_200_006705396 - PH 1 TS 0010531 A - 16/01/1901 01:00 - H:M:S=0002:14:17 AVG= 0.77 DLT= 1.13 famous_vf5a_1899_200_006705396 - PH 1 TS 0010921 A - 01/02/1901 01:00 - H:M:S=0002:18:44 AVG= 0.76 DLT= 1.77 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Program launched with process id # 25811 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 famous_vf5a_1899_200_006705396 - PH 1 TS 0011311 A - 16/02/1901 01:00 - H:M:S=0002:23:46 AVG= 0.76 DLT= 1.78 famous_vf5a_1899_200_006705396 - PH 1 TS 0011701 A - 01/03/1901 01:00 - H:M:S=0002:28:19 AVG= 0.76 DLT= 1.81 famous_vf5a_1899_200_006705396 - PH 1 TS 0012091 A - 16/03/1901 01:00 - H:M:S=0002:32:53 AVG= 0.76 DLT= 1.55 famous_vf5a_1899_200_006705396 - PH 1 TS 0012481 A - 01/04/1901 01:00 - H:M:S=0002:37:28 AVG= 0.76 DLT= 1.46 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 27067 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 2... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 27084 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0012871 A - 16/04/1901 01:00 - H:M:S=0002:46:39 AVG= 0.78 DLT= 1.66 famous_vf5a_1899_200_006705396 - PH 1 TS 0013261 A - 01/05/1901 01:00 - H:M:S=0002:51:13 AVG= 0.77 DLT= 1.22 famous_vf5a_1899_200_006705396 - PH 1 TS 0013651 A - 16/05/1901 01:00 - H:M:S=0002:55:42 AVG= 0.77 DLT= 1.63 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 27904 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0014041 A - 01/06/1901 01:00 - H:M:S=0003:01:04 AVG= 0.77 DLT= 1.26 famous_vf5a_1899_200_006705396 - PH 1 TS 0014431 A - 16/06/1901 01:00 - H:M:S=0003:05:40 AVG= 0.77 DLT= 1.11 famous_vf5a_1899_200_006705396 - PH 1 TS 0014821 A - 01/07/1901 01:00 - H:M:S=0003:10:17 AVG= 0.77 DLT= 1.32 famous_vf5a_1899_200_006705396 - PH 1 TS 0015211 A - 16/07/1901 01:00 - H:M:S=0003:14:53 AVG= 0.77 DLT= 1.87 famous_vf5a_1899_200_006705396 - PH 1 TS 0015601 A - 01/08/1901 01:00 - H:M:S=0003:19:29 AVG= 0.77 DLT= 1.57 famous_vf5a_1899_200_006705396 - PH 1 TS 0015991 A - 16/08/1901 01:00 - H:M:S=0003:24:07 AVG= 0.77 DLT= 1.32 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 29290 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0015991 A - 16/08/1901 01:00 - H:M:S=0003:24:16 AVG= 0.77 DLT= 0.00 famous_vf5a_1899_200_006705396 - PH 1 TS 0016381 A - 01/09/1901 01:00 - H:M:S=0003:28:50 AVG= 0.76 DLT= 1.53 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 29543 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0016771 A - 16/09/1901 01:00 - H:M:S=0003:37:47 AVG= 0.78 DLT= 1.58 famous_vf5a_1899_200_006705396 - PH 1 TS 0017161 A - 01/10/1901 01:00 - H:M:S=0003:42:20 AVG= 0.78 DLT= 1.71 famous_vf5a_1899_200_006705396 - PH 1 TS 0017551 A - 16/10/1901 01:00 - H:M:S=0003:46:55 AVG= 0.78 DLT= 1.48 famous_vf5a_1899_200_006705396 - PH 1 TS 0017941 A - 01/11/1901 01:00 - H:M:S=0003:51:26 AVG= 0.77 DLT= 1.44 famous_vf5a_1899_200_006705396 - PH 1 TS 0018331 A - 16/11/1901 01:00 - H:M:S=0003:55:58 AVG= 0.77 DLT= 1.67 Trickling yearly means for 1901 Building file list for 1901 in dataout -- 4 entries Creating trickle file trickle_famous_vf5a_1899_200_006705396_6_1901.zip for upload... famous_vf5a_1899_200_006705396 - PH 1 TS 0018721 A - 01/12/1901 01:00 - H:M:S=0004:00:26 AVG= 0.77 DLT= 2.30 famous_vf5a_1899_200_006705396 - PH 1 TS 0019111 A - 16/12/1901 01:00 - H:M:S=0004:04:56 AVG= 0.77 DLT= 1.29 famous_vf5a_1899_200_006705396 - PH 1 TS 0019501 A - 01/01/1902 01:00 - H:M:S=0004:09:25 AVG= 0.77 DLT= 1.79 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 31388 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0019891 A - 16/01/1902 01:00 - H:M:S=0004:17:04 AVG= 0.78 DLT= 1.37 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 31801 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0020281 A - 01/02/1902 01:00 - H:M:S=0004:24:37 AVG= 0.78 DLT= 1.23 famous_vf5a_1899_200_006705396 - PH 1 TS 0020671 A - 16/02/1902 01:00 - H:M:S=0004:29:07 AVG= 0.78 DLT= 1.53 famous_vf5a_1899_200_006705396 - PH 1 TS 0021061 A - 01/03/1902 01:00 - H:M:S=0004:33:40 AVG= 0.78 DLT= 1.15 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 32705 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0021451 A - 16/03/1902 01:00 - H:M:S=0004:40:29 AVG= 0.78 DLT= 1.37 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 718 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 2... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 828 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0021841 A - 01/04/1902 01:00 - H:M:S=0004:50:58 AVG= 0.80 DLT= 1.35 famous_vf5a_1899_200_006705396 - PH 1 TS 0022231 A - 16/04/1902 01:00 - H:M:S=0004:55:30 AVG= 0.80 DLT= 1.51 famous_vf5a_1899_200_006705396 - PH 1 TS 0022621 A - 01/05/1902 01:00 - H:M:S=0004:59:59 AVG= 0.80 DLT= 1.29 famous_vf5a_1899_200_006705396 - PH 1 TS 0023011 A - 16/05/1902 01:00 - H:M:S=0005:04:32 AVG= 0.79 DLT= 1.34 famous_vf5a_1899_200_006705396 - PH 1 TS 0023401 A - 01/06/1902 01:00 - H:M:S=0005:09:09 AVG= 0.79 DLT= 1.53 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 2281 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0023791 A - 16/06/1902 01:00 - H:M:S=0005:16:54 AVG= 0.80 DLT= 1.48 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 2695 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 2... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 2836 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0024181 A - 01/07/1902 01:00 - H:M:S=0005:29:14 AVG= 0.82 DLT= 1.25 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 3252 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. famous_vf5a_1899_200_006705396 - PH 1 TS 0024571 A - 16/07/1902 01:00 - H:M:S=0005:36:22 AVG= 0.82 DLT= 1.91 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 1... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Program launched with process id # 4554 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 2... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Program launched with process id # 4834 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 3... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 5288 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 4... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 5370 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Model restart required... Preparing for restart attempt # 5... Starting model ID famous_vf5a_1899_200_006705396 Phase 1 Getting pthread attributes - retval=0 Setting pthread size (62914560 bytes) - retval=0 Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu Executing program /home/jsc/BOINC/projects/climateprediction.net/famous_um_6.11_i686-pc-linux-gnu 135765 /home/jsc/BOINC/slots/0 Program launched with process id # 6532 Climate model starting - use graphics to monitor progress. Or visit the website to see the graphs for this run. Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy Sorry, too many model crashes! :-( Model restart required... Sorry, too many model crashes! :-( Cleaning up from the run... (17593): called boinc_finish Detaching shared memory... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
28 Nov 2010 07:28:11	922557	12346960	famous_vf5a_1899_200_006705396_6	18,746	14,426	0.7696
28 Nov 2010 07:28:11	922557	12346960	famous_vf5a_1899_200_006705396_6	9,386	7,078	0.7541