Task 15544776

Name	hadcm3n_o545_2140_40_008269420_2
Workunit	8424544
Created	15 Jan 2013, 14:42:00 UTC
Sent	15 Jan 2013, 14:42:13 UTC
Report deadline	16 Apr 2013, 22:09:24 UTC
Received	1 Feb 2013, 8:27:28 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1214462
Run time	8 days 19 hours 53 min 25 sec
CPU time	7 days 20 hours 25 min 23 sec
Validate state	Invalid
Credit	9,331.20
Device peak FLOPS	3.31 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 22:27:51 (44573): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:29:29 (1606): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:30:34 (18628): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:30:18 (21153): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... hadcm3n_6.07_i686-apple-darwin(24277,0xad1c42c0) malloc: * error for object 0x106aa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24277,0xad1c42c0) malloc: * error for object 0x106aa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24277,0xad1c42c0) malloc: * error for object 0x106aa04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24277,0xad1c42c0) malloc: * error for object 0x106aa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(24277,0xad1c42c0) malloc: * error for object 0x106aa00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e5a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e6804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e6800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e6804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e6800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e6804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e6800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(831,0xad1c42c0) malloc: * error for object 0x8e6800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(1823,0xad1c42c0) malloc: * error for object 0x297c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 15:18:17 (17964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:18:18 (17964): No heartbeat from core client for 30 sec - exiting 15:18:19 (17964): No heartbeat from core client for 30 sec - exiting 15:18:20 (17964): No heartbeat from core client for 30 sec - exiting 15:18:21 (17964): No heartbeat from core client for 30 sec - exiting 15:18:22 (17964): No heartbeat from core client for 30 sec - exiting 15:18:23 (17964): No heartbeat from core client for 30 sec - exiting 15:18:24 (17964): No heartbeat from core client for 30 sec - exiting 15:18:25 (17964): No heartbeat from core client for 30 sec - exiting 15:18:26 (17964): No heartbeat from core client for 30 sec - exiting 15:18:27 (17964): No heartbeat from core client for 30 sec - exiting 15:18:28 (17964): No heartbeat from core client for 30 sec - exiting 15:18:29 (17964): No heartbeat from core client for 30 sec - exiting 15:18:30 (17964): No heartbeat from core client for 30 sec - exiting 15:18:31 (17964): No heartbeat from core client for 30 sec - exiting 15:18:32 (17964): No heartbeat from core client for 30 sec - exiting 15:18:33 (17964): No heartbeat from core client for 30 sec - exiting 15:18:34 (17964): No heartbeat from core client for 30 sec - exiting 15:18:35 (17964): No heartbeat from core client for 30 sec - exiting 15:18:36 (17964): No heartbeat from core client for 30 sec - exiting 15:18:37 (17964): No heartbeat from core client for 30 sec - exiting 15:18:38 (17964): No heartbeat from core client for 30 sec - exiting 15:18:39 (17964): No heartbeat from core client for 30 sec - exiting 15:18:40 (17964): No heartbeat from core client for 30 sec - exiting 15:18:41 (17964): No heartbeat from core client for 30 sec - exiting 15:18:42 (17964): No heartbeat from core client for 30 sec - exiting 15:18:43 (17964): No heartbeat from core client for 30 sec - exiting 15:18:44 (17964): No heartbeat from core client for 30 sec - exiting 15:18:45 (17964): No heartbeat from core client for 30 sec - exiting 15:18:46 (17964): No heartbeat from core client for 30 sec - exiting 15:18:48 (17964): No heartbeat from core client for 30 sec - exiting 15:18:49 (17964): No heartbeat from core client for 30 sec - exiting 15:18:50 (17964): No heartbeat from core client for 30 sec - exiting 15:18:51 (17964): No heartbeat from core client for 30 sec - exiting 15:18:52 (17964): No heartbeat from core client for 30 sec - exiting 15:18:53 (17964): No heartbeat from core client for 30 sec - exiting 15:18:54 (17964): No heartbeat from core client for 30 sec - exiting 15:18:55 (17964): No heartbeat from core client for 30 sec - exiting 15:18:56 (17964): No heartbeat from core client for 30 sec - exiting 15:18:57 (17964): No heartbeat from core client for 30 sec - exiting 15:18:58 (17964): No heartbeat from core client for 30 sec - exiting 15:18:59 (17964): No heartbeat from core client for 30 sec - exiting 15:19:00 (17964): No heartbeat from core client for 30 sec - exiting 15:19:01 (17964): No heartbeat from core client for 30 sec - exiting 15:19:02 (17964): No heartbeat from core client for 30 sec - exiting 15:19:03 (17964): No heartbeat from core client for 30 sec - exiting 15:19:04 (17964): No heartbeat from core client for 30 sec - exiting 15:19:05 (17964): No heartbeat from core client for 30 sec - exiting 15:19:06 (17964): No heartbeat from core client for 30 sec - exiting 15:19:07 (17964): No heartbeat from core client for 30 sec - exiting 15:19:08 (17964): No heartbeat from core client for 30 sec - exiting 15:19:09 (17964): No heartbeat from core client for 30 sec - exiting 15:19:10 (17964): No heartbeat from core client for 30 sec - exiting 15:19:11 (17964): No heartbeat from core client for 30 sec - exiting 15:19:12 (17964): No heartbeat from core client for 30 sec - exiting 15:19:13 (17964): No heartbeat from core client for 30 sec - exiting 15:19:14 (17964): No heartbeat from core client for 30 sec - exiting 15:19:15 (17964): No heartbeat from core client for 30 sec - exiting 15:19:16 (17964): No heartbeat from core client for 30 sec - exiting 15:19:17 (17964): No heartbeat from core client for 30 sec - exiting 15:19:18 (17964): No heartbeat from core client for 30 sec - exiting 15:19:19 (17964): No heartbeat from core client for 30 sec - exiting 15:19:20 (17964): No heartbeat from core client for 30 sec - exiting 15:19:21 (17964): No heartbeat from core client for 30 sec - exiting 15:19:22 (17964): No heartbeat from core client for 30 sec - exiting 15:19:23 (17964): No heartbeat from core client for 30 sec - exiting 15:19:24 (17964): No heartbeat from core client for 30 sec - exiting 15:19:25 (17964): No heartbeat from core client for 30 sec - exiting 15:19:26 (17964): No heartbeat from core client for 30 sec - exiting 15:19:27 (17964): No heartbeat from core client for 30 sec - exiting 15:19:28 (17964): No heartbeat from core client for 30 sec - exiting 15:19:29 (17964): No heartbeat from core client for 30 sec - exiting 15:19:30 (17964): No heartbeat from core client for 30 sec - exiting 15:19:31 (17964): No heartbeat from core client for 30 sec - exiting 15:19:32 (17964): No heartbeat from core client for 30 sec - exiting 15:19:33 (17964): No heartbeat from core client for 30 sec - exiting 15:19:34 (17964): No heartbeat from core client for 30 sec - exiting 15:19:35 (17964): No heartbeat from core client for 30 sec - exiting 15:19:36 (17964): No heartbeat from core client for 30 sec - exiting 15:19:37 (17964): No heartbeat from core client for 30 sec - exiting 15:19:38 (17964): No heartbeat from core client for 30 sec - exiting 15:19:39 (17964): No heartbeat from core client for 30 sec - exiting 15:19:40 (17964): No heartbeat from core client for 30 sec - exiting 15:19:41 (17964): No heartbeat from core client for 30 sec - exiting 15:19:42 (17964): No heartbeat from core client for 30 sec - exiting 15:19:43 (17964): No heartbeat from core client for 30 sec - exiting 15:19:44 (17964): No heartbeat from core client for 30 sec - exiting 15:19:45 (17964): No heartbeat from core client for 30 sec - exiting 15:19:46 (17964): No heartbeat from core client for 30 sec - exiting 15:19:47 (17964): No heartbeat from core client for 30 sec - exiting 15:19:48 (17964): No heartbeat from core client for 30 sec - exiting 15:19:49 (17964): No heartbeat from core client for 30 sec - exiting 15:19:50 (17964): No heartbeat from core client for 30 sec - exiting 15:19:51 (17964): No heartbeat from core client for 30 sec - exiting 15:19:52 (17964): No heartbeat from core client for 30 sec - exiting 15:19:53 (17964): No heartbeat from core client for 30 sec - exiting 15:19:54 (17964): No heartbeat from core client for 30 sec - exiting 15:19:55 (17964): No heartbeat from core client for 30 sec - exiting 15:19:56 (17964): No heartbeat from core client for 30 sec - exiting 15:19:57 (17964): No heartbeat from core client for 30 sec - exiting 15:19:58 (17964): No heartbeat from core client for 30 sec - exiting 15:19:59 (17964): No heartbeat from core client for 30 sec - exiting 15:20:00 (17964): No heartbeat from core client for 30 sec - exiting 15:20:01 (17964): No heartbeat from core client for 30 sec - exiting 15:20:02 (17964): No heartbeat from core client for 30 sec - exiting 15:20:03 (17964): No heartbeat from core client for 30 sec - exiting 15:20:04 (17964): No heartbeat from core client for 30 sec - exiting 15:20:05 (17964): No heartbeat from core client for 30 sec - exiting 15:20:06 (17964): No heartbeat from core client for 30 sec - exiting 15:20:07 (17964): No heartbeat from core client for 30 sec - exiting 15:20:08 (17964): No heartbeat from core client for 30 sec - exiting 15:20:09 (17964): No heartbeat from core client for 30 sec - exiting 15:20:10 (17964): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:41:45 (46951): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 11:07:47 (1850): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 11:07:48 (1850): No heartbeat from core client for 30 sec - exiting 11:07:49 (1850): No heartbeat from core client for 30 sec - exiting 11:07:50 (1850): No heartbeat from core client for 30 sec - exiting 11:07:51 (1850): No heartbeat from core client for 30 sec - exiting 11:07:52 (1850): No heartbeat from core client for 30 sec - exiting 11:07:53 (1850): No heartbeat from core client for 30 sec - exiting 11:07:54 (1850): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 13:46:40 (21479): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:41:31 (24705): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:45:35 (44462): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 21:43:31 (48479): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 22:43:07 (51899): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:43:41 (55000): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 02:56:21 PM No files match the supplied pattern. MainError: 02:56:21 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 10:46:21 AM No files match the supplied pattern. MainError: 10:46:21 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:14:41 (25731): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 09:37:09 AM No files match the supplied pattern. MainError: 09:37:09 AM No files match the supplied pattern. MainError: 04:29:32 PM No files match the supplied pattern. MainError: 04:29:32 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 09:49:39 (8378): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... MainError: 09:17:38 AM No files match the supplied pattern. MainError: 09:17:38 AM No files match the supplied pattern. MainError: 04:23:40 PM No files match the supplied pattern. MainError: 04:23:40 PM No files match the supplied pattern. 20:48:22 (9503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:48:23 (9503): No heartbeat from core client for 30 sec - exiting 20:48:24 (9503): No heartbeat from core client for 30 sec - exiting 20:48:25 (9503): No heartbeat from core client for 30 sec - exiting 20:48:26 (9503): No heartbeat from core client for 30 sec - exiting 20:48:27 (9503): No heartbeat from core client for 30 sec - exiting 20:48:28 (9503): No heartbeat from core client for 30 sec - exiting 20:48:29 (9503): No heartbeat from core client for 30 sec - exiting 20:48:30 (9503): No heartbeat from core client for 30 sec - exiting 20:48:31 (9503): No heartbeat from core client for 30 sec - exiting 20:48:32 (9503): No heartbeat from core client for 30 sec - exiting 20:48:33 (9503): No heartbeat from core client for 30 sec - exiting 20:48:35 (9503): No heartbeat from core client for 30 sec - exiting 20:48:36 (9503): No heartbeat from core client for 30 sec - exiting 20:48:37 (9503): No heartbeat from core client for 30 sec - exiting 20:48:38 (9503): No heartbeat from core client for 30 sec - exiting 20:48:39 (9503): No heartbeat from core client for 30 sec - exiting 20:48:40 (9503): No heartbeat from core client for 30 sec - exiting 20:48:41 (9503): No heartbeat from core client for 30 sec - exiting 20:48:42 (9503): No heartbeat from core client for 30 sec - exiting 20:48:43 (9503): No heartbeat from core client for 30 sec - exiting 20:48:44 (9503): No heartbeat from core client for 30 sec - exiting 20:48:45 (9503): No heartbeat from core client for 30 sec - exiting 20:48:46 (9503): No heartbeat from core client for 30 sec - exiting 20:48:47 (9503): No heartbeat from core client for 30 sec - exiting 20:48:48 (9503): No heartbeat from core client for 30 sec - exiting 20:48:49 (9503): No heartbeat from core client for 30 sec - exiting 20:48:50 (9503): No heartbeat from core client for 30 sec - exiting 20:48:51 (9503): No heartbeat from core client for 30 sec - exiting 20:48:52 (9503): No heartbeat from core client for 30 sec - exiting 20:48:53 (9503): No heartbeat from core client for 30 sec - exiting 20:48:54 (9503): No heartbeat from core client for 30 sec - exiting 20:48:55 (9503): No heartbeat from core client for 30 sec - exiting 20:48:56 (9503): No heartbeat from core client for 30 sec - exiting 20:48:57 (9503): No heartbeat from core client for 30 sec - exiting 20:48:58 (9503): No heartbeat from core client for 30 sec - exiting 20:48:59 (9503): No heartbeat from core client for 30 sec - exiting 20:49:00 (9503): No heartbeat from core client for 30 sec - exiting 20:49:01 (9503): No heartbeat from core client for 30 sec - exiting 20:49:02 (9503): No heartbeat from core client for 30 sec - exiting 20:49:03 (9503): No heartbeat from core client for 30 sec - exiting 20:49:04 (9503): No heartbeat from core client for 30 sec - exiting 20:49:05 (9503): No heartbeat from core client for 30 sec - exiting 20:49:06 (9503): No heartbeat from core client for 30 sec - exiting 20:49:07 (9503): No heartbeat from core client for 30 sec - exiting 20:49:08 (9503): No heartbeat from core client for 30 sec - exiting 20:49:09 (9503): No heartbeat from core client for 30 sec - exiting 20:49:10 (9503): No heartbeat from core client for 30 sec - exiting 20:49:11 (9503): No heartbeat from core client for 30 sec - exiting 20:49:12 (9503): No heartbeat from core client for 30 sec - exiting 20:49:13 (9503): No heartbeat from core client for 30 sec - exiting 20:49:14 (9503): No heartbeat from core client for 30 sec - exiting 20:49:15 (9503): No heartbeat from core client for 30 sec - exiting 20:49:16 (9503): No heartbeat from core client for 30 sec - exiting 20:49:17 (9503): No heartbeat from core client for 30 sec - exiting CPDN Monitor - Quit request from BOINC... 23:47:32 (48598): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... MainError: 07:20:30 AM No files match the supplied pattern. MainError: 07:20:30 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 02:23:41 PM No files match the supplied pattern. MainError: 02:23:41 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 09:27:05 PM No files match the supplied pattern. MainError: 09:27:05 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... MainError: 11:38:36 AM No files match the supplied pattern. MainError: 11:38:36 AM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Error: Input file: dataout/o545ko.pjz2c10 is not a valid UM file. Error converting file to netcdf: dataout/o545ko.pjz2c10 Error: Input file: dataout/o545ko.piz2c10 is not a valid UM file. Error converting file to netcdf: dataout/o545ko.piz2c10 Error: Input file: dataout/o545ko.pfz2c10 is not a valid UM file. Error converting file to netcdf: dataout/o545ko.pfz2c10 Error: Input file: dataout/o545ka.phz2c10 is not a valid UM file. Error converting file to netcdf: dataout/o545ka.phz2c10 Error converting file to netcdf: dataout/o545ka.ph11c10 Error: Input file: dataout/o545ka.pgz2c10 is not a valid UM file. Error converting file to netcdf: dataout/o545ka.pgz2c10 Error converting file to netcdf: dataout/o545ka.pg11c10 Error: Input file: dataout/o545ka.pez2c10 is not a valid UM file. Error converting file to netcdf: dataout/o545ka.pez2c10 Error converting file to netcdf: dataout/o545ka.pe11c10 MainError: 08:34:07 PM No files match the supplied pattern. MainError: 08:34:07 PM No files match the supplied pattern. CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: Inappropriate ioctl for device BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: Result too large BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 Model crashed: STWORK : I/O error - PP fixed length header tmp/pipe_dummy 2048 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
31 Jan 2013 20:37:51	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	777,600	678,011	0.8719
31 Jan 2013 11:56:12	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	751,680	655,703	0.8723
30 Jan 2013 21:28:16	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	725,760	633,317	0.8726
30 Jan 2013 14:25:23	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	699,840	610,996	0.8731
30 Jan 2013 07:22:58	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	673,920	588,485	0.8732
29 Jan 2013 16:25:44	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	648,000	565,250	0.8723
29 Jan 2013 09:19:36	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	622,080	542,211	0.8716
28 Jan 2013 16:30:45	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	596,160	519,282	0.8710
28 Jan 2013 09:39:39	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	570,240	496,934	0.8714
27 Jan 2013 10:46:26	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	544,320	473,786	0.8704
26 Jan 2013 15:00:32	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	518,400	451,526	0.8710
26 Jan 2013 08:14:49	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	492,480	429,496	0.8721
25 Jan 2013 16:50:51	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	466,560	407,014	0.8724
25 Jan 2013 09:45:29	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	440,640	383,804	0.8710
24 Jan 2013 17:31:14	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	414,720	360,726	0.8698
24 Jan 2013 10:16:22	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	388,800	338,294	0.8701
23 Jan 2013 19:22:34	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	362,880	315,906	0.8706
23 Jan 2013 11:43:44	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	336,960	293,749	0.8718
22 Jan 2013 21:10:19	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	311,040	270,820	0.8707
22 Jan 2013 13:30:20	1214462	15544776	hadcm3n_o545_2140_40_008269420_2	285,120	248,318	0.8709