climateprediction.net home page
Task 13357877

Task 13357877

Name hadcm3n_t0mq_1940_40_007446841_0
Workunit 7644344
Created 9 Sep 2011, 16:34:28 UTC
Sent 9 Sep 2011, 16:37:03 UTC
Report deadline 10 Dec 2011, 0:04:14 UTC
Received 28 Sep 2011, 18:26:37 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 896411
Run time 5 days 7 hours 40 min 30 sec
CPU time 4 days 15 hours 52 min 56 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 3.28 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/t0mqko.pje5c10 is not a valid UM file.
Error converting file to netcdf: dataout/t0mqko.pje5c10
Error: Input file: dataout/t0mqko.pie5c10 is not a valid UM file.
Error converting file to netcdf: dataout/t0mqko.pie5c10
Error: Input file: dataout/t0mqko.pfe5c10 is not a valid UM file.
Error converting file to netcdf: dataout/t0mqko.pfe5c10
Error: Input file: dataout/t0mqka.phe5c10 is not a valid UM file.
Error converting file to netcdf: dataout/t0mqka.phe5c10
Error: Input file: dataout/t0mqka.pge5c10 is not a valid UM file.
Error converting file to netcdf: dataout/t0mqka.pge5c10
Error: Input file: dataout/t0mqka.pee5c10 is not a valid UM file.
Error converting file to netcdf: dataout/t0mqka.pee5c10
Error: Input file: dataout/t0mqka.pde5c10 is not a valid UM file.
Error converting file to netcdf: dataout/t0mqka.pde5c10
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
14:10:11 (32462): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x381f604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x381f600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x381f604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x381f600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x201f604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x201f600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x985a204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x985a200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x985a204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x985a200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x981ba04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x981ba00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x830a04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x830a00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x3032604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x3032600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x812200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(78735,0xa09a5540) malloc: *** error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K549
Sun Sep 25 13:01:11 2011

Thread 0 Crashed:
atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386.
0   libSystem.B.dylib                   0x9812db03 small_free_list_remove_ptr + 234
1   libSystem.B.dylib                   0x9812a5cc szone_free_definite_size + 3457
2   libSystem.B.dylib                   0x981295e8 free + 244
3   hadcm3n_6.07_i686-apple-darwin      0x00022341 ncio_px_free + 97
4   hadcm3n_6.07_i686-apple-darwin      0x0002290c ncio_free + 28
5   hadcm3n_6.07_i686-apple-darwin      0x00022f09 ncio_close + 57
6   hadcm3n_6.07_i686-apple-darwin      0x0001cdd7 nc_close + 103
7   hadcm3n_6.07_i686-apple-darwin      0x0000bab9 annual_cycle(std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const*, char const*, int, int) + 3579
8   hadcm3n_6.07_i686-apple-darwin      0x0000d36b decadalMeans(int, char const*) + 957
9   hadcm3n_6.07_i686-apple-darwin      0x000067ff doCM3Proc() + 185
10  hadcm3n_6.07_i686-apple-darwin      0x0000791c mainLoop() + 410
11  hadcm3n_6.07_i686-apple-darwin      0x000087c7 worker() + 2989
12  hadcm3n_6.07_i686-apple-darwin      0x00008aa9 main + 491
13  hadcm3n_6.07_i686-apple-darwin      0x00002676 start + 54

Thread 1:
Worker:: CPDN process is not running, exiting, bRetVal = 1, checkPID=78743, selfPID=78743, iMonCtr=1
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x105ac04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x81c604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(81271,0xa09a5540) malloc: *** error for object 0x3036204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x1054604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x81c604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x81d404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x81d400: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(90884,0xa09a5540) malloc: *** error for object 0x381ea00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Signal 3 received, exiting...
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
25 Sep 2011 19:04:35 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 259,200 359,073 1.3853
25 Sep 2011 05:38:45 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 233,280 324,276 1.3901
24 Sep 2011 15:01:02 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 207,360 289,155 1.3945
23 Sep 2011 16:01:24 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 181,440 254,338 1.4018
23 Sep 2011 03:01:28 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 155,520 218,365 1.4041
21 Sep 2011 17:33:06 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 129,600 182,210 1.4059
20 Sep 2011 17:57:24 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 103,680 145,604 1.4044
20 Sep 2011 01:27:15 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 77,760 110,203 1.4172
19 Sep 2011 10:54:46 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 51,840 73,170 1.4115
18 Sep 2011 20:12:31 896411 13357877 hadcm3n_t0mq_1940_40_007446841_0 25,920 35,749 1.3792


©2024 climateprediction.net