climateprediction.net home page
Task 18436118

Task 18436118

Name hadcm3n_xdd1_1940_40_009534183_3
Workunit 9615710
Created 10 May 2015, 13:22:30 UTC
Sent 10 May 2015, 13:28:01 UTC
Report deadline 9 Aug 2015, 20:55:12 UTC
Received 31 Jul 2015, 20:33:33 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1361780
Run time 12 days 4 hours 32 min 36 sec
CPU time 10 days 14 hours 8 min 11 sec
Validate state Invalid
Credit 9,953.28
Device peak FLOPS 3.60 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr
<core_client_version>7.4.42</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
Wed Jul  8 07:04:08 2015

Thread 0 Crashed:

Thread 1:

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbeb85d08 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x903b5000 - 0x90421fff /usr/lib/system/libcorecrypto.dylib
0x90682000 - 0x906b5fff /usr/lib/system/libsystem_m.dylib
0x90de7000 - 0x90de9fff /usr/lib/system/libsystem_secinit.dylib
0x91a35000 - 0x91a37fff /usr/lib/system/libsystem_sandbox.dylib
0x9256a000 - 0x9256ffff /usr/lib/system/libmacho.dylib
0x928b9000 - 0x928f6fff /usr/lib/system/libsystem_network.dylib
0x92a1e000 - 0x92a1ffff /usr/lib/system/libsystem_blocks.dylib
0x94503000 - 0x94506fff /usr/lib/system/libdyld.dylib
0x94b08000 - 0x94b0efff /usr/lib/system/libsystem_networkextension.dylib
0x94c57000 - 0x94c59fff /usr/lib/system/libquarantine.dylib
0x95279000 - 0x95285fff /usr/lib/system/libcommonCrypto.dylib
0x95342000 - 0x95349fff /usr/lib/system/libunwind.dylib
0x958a6000 - 0x958aafff /usr/lib/system/libcache.dylib
0x958ab000 - 0x958b4fff /usr/lib/system/libcopyfile.dylib
0x958c7000 - 0x958edfff /usr/lib/system/libxpc.dylib
0x958ee000 - 0x95983fff /usr/lib/system/libsystem_c.dylib
0x95b60000 - 0x95b60fff /usr/lib/system/liblaunch.dylib
0x95cb6000 - 0x95cc4fff /usr/lib/libz.1.dylib
0x96404000 - 0x9640afff /usr/lib/system/libsystem_trace.dylib
0x9699d000 - 0x9699dfff /usr/lib/system/libkeymgr.dylib
0x98147000 - 0x9814ffff /usr/lib/system/libsystem_pthread.dylib
0x98194000 - 0x9819afff /usr/lib/system/libsystem_platform.dylib
0x9819b000 - 0x981a0fff /usr/lib/system/libcompiler_rt.dylib
0x98391000 - 0x98392fff /usr/lib/system/libremovefile.dylib
0x98393000 - 0x98394fff /usr/lib/libSystem.B.dylib
0x99141000 - 0x99143fff /usr/lib/system/libsystem_configuration.dylib
0x9975a000 - 0x99781fff /usr/lib/system/libdispatch.dylib
0x998e3000 - 0x998e3fff /usr/lib/system/libunc.dylib
0x99f76000 - 0x99f8ffff /usr/lib/system/libsystem_malloc.dylib
0x9a00e000 - 0x9a010fff /usr/lib/system/libsystem_coreservices.dylib
0x9a016000 - 0x9a027fff /usr/lib/system/libsystem_coretls.dylib
0x9a54b000 - 0x9a553fff /usr/lib/system/libsystem_dnssd.dylib
0x9a591000 - 0x9a5a8fff /usr/lib/system/libsystem_asl.dylib
0x9a5e4000 - 0x9a60dfff /usr/lib/system/libsystem_info.dylib
0x9a857000 - 0x9a876fff /usr/lib/system/libsystem_kernel.dylib
0x9acfd000 - 0x9ad06fff /usr/lib/system/libsystem_notify.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=31081, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x1048c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x1048c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x2832204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x83e604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x83e600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x83e604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x83e600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x1048c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x1048c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x823004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3472,0xa10421d4) malloc: *** error for object 0x81c804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(411,0xa10421d4) malloc: *** error for object 0x202f204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(2159,0xa10421d4) malloc: *** error for object 0x2048204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2159,0xa10421d4) malloc: *** error for object 0x2809e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(5587,0xa10421d4) malloc: *** error for object 0x2825c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5587,0xa10421d4) malloc: *** error for object 0x2825c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5587,0xa10421d4) malloc: *** error for object 0x1009e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(5644,0xa10421d4) malloc: *** error for object 0x283b604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5644,0xa10421d4) malloc: *** error for object 0x101de04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5644,0xa10421d4) malloc: *** error for object 0x101de00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5644,0xa10421d4) malloc: *** error for object 0x101de04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(8094,0xa10421d4) malloc: *** error for object 0x2838204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8094,0xa10421d4) malloc: *** error for object 0x1830604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8094,0xa10421d4) malloc: *** error for object 0x102f204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8094,0xa10421d4) malloc: *** error for object 0x102f200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8094,0xa10421d4) malloc: *** error for object 0x102f204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(8216,0xa10421d4) malloc: *** error for object 0x821004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(8938,0xa10421d4) malloc: *** error for object 0x2039c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8938,0xa10421d4) malloc: *** error for object 0x2039c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8938,0xa10421d4) malloc: *** error for object 0x802604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
Fri Jul 10 17:14:56 2015

hadcm3n_6.07_i686-apple-darwin(9202,0xa10421d4) malloc: *** error for object 0x2029c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(9202,0xa10421d4) malloc: *** error for object 0x2029c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(9202,0xa10421d4) malloc: *** error for object 0x1011e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(11350,0xa10421d4) malloc: *** error for object 0x1040204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(11350,0xa10421d4) malloc: *** error for object 0x829004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(11350,0xa10421d4) malloc: *** error for object 0x1040204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(11350,0xa10421d4) malloc: *** error for object 0x2028604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(16023,0xa10421d4) malloc: *** error for object 0x2048604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(16023,0xa10421d4) malloc: *** error for object 0x1020604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(16023,0xa10421d4) malloc: *** error for object 0x2048604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(16023,0xa10421d4) malloc: *** error for object 0x280ae04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(16023,0xa10421d4) malloc: *** error for object 0x280ae00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(16023,0xa10421d4) malloc: *** error for object 0x200a004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(17243,0xa10421d4) malloc: *** error for object 0x1830604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17243,0xa10421d4) malloc: *** error for object 0x1830600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x2823204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x202c604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x202c604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x2011e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x2011e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(17714,0xa10421d4) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x2830604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x182c804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x2830604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x182c804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x103c604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x2830604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x182ec04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x182ec00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20706,0xa10421d4) malloc: *** error for object 0x182ec04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(963,0xa096e1d4) malloc: *** error for object 0x829004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(963,0xa096e1d4) malloc: *** error for object 0x1040c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(963,0xa096e1d4) malloc: *** error for object 0x102e404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(1382,0xa096e1d4) malloc: *** error for object 0x82e804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(1382,0xa096e1d4) malloc: *** error for object 0x82e800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(1382,0xa096e1d4) malloc: *** error for object 0x82e804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(1444,0xa096e1d4) malloc: *** error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x801c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2001e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x2010404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2281,0xa096e1d4) malloc: *** error for object 0x201ea04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(7619,0xa096e1d4) malloc: *** error for object 0x835004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(7619,0xa096e1d4) malloc: *** error for object 0x1039e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(7619,0xa096e1d4) malloc: *** error for object 0x1039e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(7619,0xa096e1d4) malloc: *** error for object 0x2009e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x1838604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x103cc04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x103cc04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x1814e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x101a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x101a800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x101a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x101a800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x1001204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x1001200: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x280e204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(431,0xa096e1d4) malloc: *** error for object 0x281c804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
hadcm3n_6.07_i686-apple-darwin(523,0xa096e1d4) malloc: *** error for object 0x102c604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
Tue Jul 14 22:32:12 2015

hadcm3n_6.07_i686-apple-darwin(5861,0xa096e1d4) malloc: *** error for object 0x828804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(6411,0xa096e1d4) malloc: *** error for object 0x82cc04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(6628,0xa096e1d4) malloc: *** error for object 0x824c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(6628,0xa096e1d4) malloc: *** error for object 0x2837604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(6628,0xa096e1d4) malloc: *** error for object 0x2019e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(6628,0xa096e1d4) malloc: *** error for object 0x2800e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(6628,0xa096e1d4) malloc: *** error for object 0x2800e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
Wed Jul 15 22:16:28 2015

Thread 0 Crashed:
atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386.
hadcm3n_6.07_i686-apple-darwin(15706,0xa096e1d4) malloc: *** error for object 0x2829e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15706,0xa096e1d4) malloc: *** error for object 0x2829e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15706,0xa096e1d4) malloc: *** error for object 0x81b204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
Fri Jul 17 01:19:46 2015

Thread 0 Crashed:
atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386.

Thread 1:
hadcm3n_6.07_i686-apple-darwin(19160,0xa096e1d4) malloc: *** error for object 0x102b004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(19160,0xa096e1d4) malloc: *** error for object 0x102b000: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(19160,0xa096e1d4) malloc: *** error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(20201,0xa096e1d4) malloc: *** error for object 0x1820c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20201,0xa096e1d4) malloc: *** error for object 0x828c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20201,0xa096e1d4) malloc: *** error for object 0x2031e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20201,0xa096e1d4) malloc: *** error for object 0x2031e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(20201,0xa096e1d4) malloc: *** error for object 0x2031e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(22306,0xa096e1d4) malloc: *** error for object 0x2829c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(22306,0xa096e1d4) malloc: *** error for object 0x2829c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(22306,0xa096e1d4) malloc: *** error for object 0x101a404: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(30353,0xa096e1d4) malloc: *** error for object 0x845004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(30353,0xa096e1d4) malloc: *** error for object 0x2020604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(30353,0xa096e1d4) malloc: *** error for object 0x82e804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(30353,0xa096e1d4) malloc: *** error for object 0x82e800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(30353,0xa096e1d4) malloc: *** error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x2030604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x2030604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x840804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x2823604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x2823600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x202dc04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x202dc00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(33436,0xa096e1d4) malloc: *** error for object 0x202dc04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(38667,0xa096e1d4) malloc: *** error for object 0x103e804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(38667,0xa096e1d4) malloc: *** error for object 0x103e800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(38667,0xa096e1d4) malloc: *** error for object 0x103e804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(390,0xa096e1d4) malloc: *** error for object 0x1030804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(390,0xa096e1d4) malloc: *** error for object 0x82d004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(390,0xa096e1d4) malloc: *** error for object 0x2828604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(390,0xa096e1d4) malloc: *** error for object 0x82d004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(2581,0xa096e1d4) malloc: *** error for object 0x1016804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2581,0xa096e1d4) malloc: *** error for object 0x1016800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(2581,0xa096e1d4) malloc: *** error for object 0x2829204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(3534,0xa096e1d4) malloc: *** error for object 0x103a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3534,0xa096e1d4) malloc: *** error for object 0x103a800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3534,0xa096e1d4) malloc: *** error for object 0x2801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
Mon Jul 20 21:42:29 2015

hadcm3n_6.07_i686-apple-darwin(8864,0xa096e1d4) malloc: *** error for object 0x831004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8864,0xa096e1d4) malloc: *** error for object 0x831004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8864,0xa096e1d4) malloc: *** error for object 0x82a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8864,0xa096e1d4) malloc: *** error for object 0x82a800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(8864,0xa096e1d4) malloc: *** error for object 0x2006204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
Tue Jul 21 12:30:57 2015

hadcm3n_6.07_i686-apple-darwin(518,0xa096e1d4) malloc: *** error for object 0x829004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(518,0xa096e1d4) malloc: *** error for object 0x2840604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(3958,0xa096e1d4) malloc: *** error for object 0x281f604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3958,0xa096e1d4) malloc: *** error for object 0x821004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3958,0xa096e1d4) malloc: *** error for object 0x2047604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3958,0xa096e1d4) malloc: *** error for object 0x1032804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3958,0xa096e1d4) malloc: *** error for object 0x1032800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(3958,0xa096e1d4) malloc: *** error for object 0x802804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x2843604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x2815c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x2815c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x80e804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x80e800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x2814e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x800e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(4006,0xa096e1d4) malloc: *** error for object 0x800e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

Crashed executable name: hadcm3n_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.10.4 build 14E46
hadcm3n_6.07_i686-apple-darwin(13983,0xa096e1d4) malloc: *** error for object 0x835004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(13983,0xa096e1d4) malloc: *** error for object 0x202b204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(13983,0xa096e1d4) malloc: *** error for object 0x282de04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(13983,0xa096e1d4) malloc: *** error for object 0x282de00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(13983,0xa096e1d4) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(461,0xa096e1d4) malloc: *** error for object 0x2820604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(764,0xa096e1d4) malloc: *** error for object 0x80a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(472,0xa096e1d4) malloc: *** error for object 0x824c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(5529,0xa096e1d4) malloc: *** error for object 0x820c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5529,0xa096e1d4) malloc: *** error for object 0x101a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5529,0xa096e1d4) malloc: *** error for object 0x101a800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(5529,0xa096e1d4) malloc: *** error for object 0x801c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(10732,0xa096e1d4) malloc: *** error for object 0x202e604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(10732,0xa096e1d4) malloc: *** error for object 0x202e600: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(10732,0xa096e1d4) malloc: *** error for object 0x802804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(15626,0xa096e1d4) malloc: *** error for object 0x2030604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15626,0xa096e1d4) malloc: *** error for object 0x821004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15626,0xa096e1d4) malloc: *** error for object 0x2829c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15626,0xa096e1d4) malloc: *** error for object 0x2829c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(15626,0xa096e1d4) malloc: *** error for object 0x2829c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(21699,0xa096e1d4) malloc: *** error for object 0x2020604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21699,0xa096e1d4) malloc: *** error for object 0x1038604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21699,0xa096e1d4) malloc: *** error for object 0x101a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21699,0xa096e1d4) malloc: *** error for object 0x101a800: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21699,0xa096e1d4) malloc: *** error for object 0x101a804: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(21751,0xa096e1d4) malloc: *** error for object 0x1048204: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21751,0xa096e1d4) malloc: *** error for object 0x201f604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21751,0xa096e1d4) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21751,0xa096e1d4) malloc: *** error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(21751,0xa096e1d4) malloc: *** error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(36190,0xa096e1d4) malloc: *** error for object 0x855004: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(36190,0xa096e1d4) malloc: *** error for object 0x2029c04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(36190,0xa096e1d4) malloc: *** error for object 0x2029c00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(36190,0xa096e1d4) malloc: *** error for object 0x2809e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(43489,0xa096e1d4) malloc: *** error for object 0x2820604: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
SIGSEGV: segmentation violation
hadcm3n_6.07_i686-apple-darwin(45136,0xa096e1d4) malloc: *** error for object 0x1000e04: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
hadcm3n_6.07_i686-apple-darwin(45136,0xa096e1d4) malloc: *** error for object 0x1000e00: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
31 Jul 2015 19:33:44 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 829,440 921,180 1.1106
30 Jul 2015 07:38:05 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 803,520 888,960 1.1063
09 Jul 2015 16:46:31 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 777,600 857,217 1.1024
08 Jul 2015 11:13:39 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 751,680 829,915 1.1041
07 Jul 2015 22:40:00 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 725,760 800,790 1.1034
07 Jul 2015 08:20:43 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 699,840 770,905 1.1015
05 Jul 2015 05:55:41 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 673,920 739,884 1.0979
03 Jul 2015 08:34:29 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 648,000 710,690 1.0967
14 Jun 2015 17:21:09 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 622,080 681,504 1.0955
14 Jun 2015 08:48:29 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 596,160 654,969 1.0986
13 Jun 2015 22:49:58 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 570,240 626,822 1.0992
13 Jun 2015 09:49:02 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 544,320 600,711 1.1036
12 Jun 2015 20:24:05 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 518,400 573,992 1.1072
11 Jun 2015 20:31:57 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 492,480 547,500 1.1117
23 May 2015 16:48:38 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 466,560 521,736 1.1183
23 May 2015 06:10:19 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 440,640 491,681 1.1158
22 May 2015 08:43:19 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 414,720 460,427 1.1102
22 May 2015 08:25:48 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 388,800 429,143 1.1038
20 May 2015 08:03:14 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 362,880 398,030 1.0969
19 May 2015 17:25:58 1361780 18436118 hadcm3n_xdd1_1940_40_009534183_3 336,960 371,441 1.1023


©2024 climateprediction.net