Task 19135403

Name	hadcm3n_sc5k_198012_480_010223092_1
Workunit	10223092
Created	5 Dec 2015, 3:58:40 UTC
Sent	5 Dec 2015, 4:00:09 UTC
Report deadline	16 Nov 2016, 9:20:09 UTC
Received	29 Dec 2015, 4:00:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1280444
Run time	15 days 17 hours 46 min 2 sec
CPU time	13 days 11 hours 12 min 12 sec
Validate state	Invalid
Credit	11,819.52
Device peak FLOPS	3.85 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.12</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x101e804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x101e800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(41920,0xa08881d4) malloc: * error for object 0x2001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 Error: Input file: dataout/sc5kko.pjj9c10 is not a valid UM file. Error converting file to netcdf: dataout/sc5kko.pjj9c10 Error: Input file: dataout/sc5kko.pij9c10 is not a valid UM file. Error converting file to netcdf: dataout/sc5kko.pij9c10 Error: Input file: dataout/sc5kko.pfj9c10 is not a valid UM file. Error converting file to netcdf: dataout/sc5kko.pfj9c10 Error: Input file: dataout/sc5kka.phj9c10 is not a valid UM file. Error converting file to netcdf: dataout/sc5kka.phj9c10 Error: Input file: dataout/sc5kka.pgj9c10 is not a valid UM file. Error converting file to netcdf: dataout/sc5kka.pgj9c10 Error: Input file: dataout/sc5kka.pej9c10 is not a valid UM file. Error converting file to netcdf: dataout/sc5kka.pej9c10 Error: Input file: dataout/sc5kka.pdj9c10 is not a valid UM file. Error converting file to netcdf: dataout/sc5kka.pdj9c10 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 13:39:28 (35764): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 08:33:48 (1842): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.5 build 14F1509 Mon Dec 28 08:56:30 2015 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4638 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90317000 - 0x90317fff /usr/lib/system/libunc.dylib 0x9039e000 - 0x903a6fff /usr/lib/system/libsystem_pthread.dylib 0x90a1e000 - 0x90a37fff /usr/lib/system/libsystem_malloc.dylib 0x91687000 - 0x9168dfff /usr/lib/system/libsystem_networkextension.dylib 0x92055000 - 0x92059fff /usr/lib/system/libcache.dylib 0x92afd000 - 0x92b02fff /usr/lib/system/libmacho.dylib 0x931b9000 - 0x931ecfff /usr/lib/system/libsystem_m.dylib 0x935de000 - 0x935e7fff /usr/lib/system/libsystem_notify.dylib 0x93ccc000 - 0x93cf2fff /usr/lib/system/libxpc.dylib 0x94d3f000 - 0x94d41fff /usr/lib/system/libsystem_coreservices.dylib 0x95062000 - 0x95081fff /usr/lib/system/libsystem_kernel.dylib 0x9508f000 - 0x95091fff /usr/lib/system/libsystem_sandbox.dylib 0x95092000 - 0x950cffff /usr/lib/system/libsystem_network.dylib 0x951d2000 - 0x951f9fff /usr/lib/system/libdispatch.dylib 0x95274000 - 0x95280fff /usr/lib/system/libcommonCrypto.dylib 0x96054000 - 0x96065fff /usr/lib/system/libsystem_coretls.dylib 0x96cb5000 - 0x96cb5fff /usr/lib/system/liblaunch.dylib 0x96cb6000 - 0x96cdffff /usr/lib/system/libsystem_info.dylib 0x96f8f000 - 0x96f95fff /usr/lib/system/libsystem_trace.dylib 0x9701c000 - 0x9701ffff /usr/lib/system/libdyld.dylib 0x98778000 - 0x9877ffff /usr/lib/system/libunwind.dylib 0x98780000 - 0x98781fff /usr/lib/libSystem.B.dylib 0x989f6000 - 0x98a8bfff /usr/lib/system/libsystem_c.dylib 0x98caa000 - 0x98d16fff /usr/lib/system/libcorecrypto.dylib 0x98f2d000 - 0x98f2ffff /usr/lib/system/libsystem_configuration.dylib 0x98f30000 - 0x98f39fff /usr/lib/system/libcopyfile.dylib 0x98f3a000 - 0x98f3afff /usr/lib/system/libkeymgr.dylib 0x98f48000 - 0x98f5ffff /usr/lib/system/libsystem_asl.dylib 0x98f60000 - 0x98f65fff /usr/lib/system/libcompiler_rt.dylib 0x9946e000 - 0x9946ffff /usr/lib/system/libremovefile.dylib 0x9b637000 - 0x9b639fff /usr/lib/system/libsystem_secinit.dylib 0x9b823000 - 0x9b82bfff /usr/lib/system/libsystem_dnssd.dylib 0x9b8f0000 - 0x9b8f2fff /usr/lib/system/libquarantine.dylib 0x9b970000 - 0x9b97efff /usr/lib/libz.1.dylib 0x9c1d6000 - 0x9c1dcfff /usr/lib/system/libsystem_platform.dylib 0x9c47f000 - 0x9c480fff /usr/lib/system/libsystem_blocks.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1989, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.5 build 14F1509 Mon Dec 28 08:56:41 2015 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4638 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90317000 - 0x90317fff /usr/lib/system/libunc.dylib 0x9039e000 - 0x903a6fff /usr/lib/system/libsystem_pthread.dylib 0x90a1e000 - 0x90a37fff /usr/lib/system/libsystem_malloc.dylib 0x91687000 - 0x9168dfff /usr/lib/system/libsystem_networkextension.dylib 0x92055000 - 0x92059fff /usr/lib/system/libcache.dylib 0x92afd000 - 0x92b02fff /usr/lib/system/libmacho.dylib 0x931b9000 - 0x931ecfff /usr/lib/system/libsystem_m.dylib 0x935de000 - 0x935e7fff /usr/lib/system/libsystem_notify.dylib 0x93ccc000 - 0x93cf2fff /usr/lib/system/libxpc.dylib 0x94d3f000 - 0x94d41fff /usr/lib/system/libsystem_coreservices.dylib 0x95062000 - 0x95081fff /usr/lib/system/libsystem_kernel.dylib 0x9508f000 - 0x95091fff /usr/lib/system/libsystem_sandbox.dylib 0x95092000 - 0x950cffff /usr/lib/system/libsystem_network.dylib 0x951d2000 - 0x951f9fff /usr/lib/system/libdispatch.dylib 0x95274000 - 0x95280fff /usr/lib/system/libcommonCrypto.dylib 0x96054000 - 0x96065fff /usr/lib/system/libsystem_coretls.dylib 0x96cb5000 - 0x96cb5fff /usr/lib/system/liblaunch.dylib 0x96cb6000 - 0x96cdffff /usr/lib/system/libsystem_info.dylib 0x96f8f000 - 0x96f95fff /usr/lib/system/libsystem_trace.dylib 0x9701c000 - 0x9701ffff /usr/lib/system/libdyld.dylib 0x98778000 - 0x9877ffff /usr/lib/system/libunwind.dylib 0x98780000 - 0x98781fff /usr/lib/libSystem.B.dylib 0x989f6000 - 0x98a8bfff /usr/lib/system/libsystem_c.dylib 0x98caa000 - 0x98d16fff /usr/lib/system/libcorecrypto.dylib 0x98f2d000 - 0x98f2ffff /usr/lib/system/libsystem_configuration.dylib 0x98f30000 - 0x98f39fff /usr/lib/system/libcopyfile.dylib 0x98f3a000 - 0x98f3afff /usr/lib/system/libkeymgr.dylib 0x98f48000 - 0x98f5ffff /usr/lib/system/libsystem_asl.dylib 0x98f60000 - 0x98f65fff /usr/lib/system/libcompiler_rt.dylib 0x9946e000 - 0x9946ffff /usr/lib/system/libremovefile.dylib 0x9b637000 - 0x9b639fff /usr/lib/system/libsystem_secinit.dylib 0x9b823000 - 0x9b82bfff /usr/lib/system/libsystem_dnssd.dylib 0x9b8f0000 - 0x9b8f2fff /usr/lib/system/libquarantine.dylib 0x9b970000 - 0x9b97efff /usr/lib/libz.1.dylib 0x9c1d6000 - 0x9c1dcfff /usr/lib/system/libsystem_platform.dylib 0x9c47f000 - 0x9c480fff /usr/lib/system/libsystem_blocks.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1989, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.5 build 14F1509 Mon Dec 28 08:56:49 2015 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4638 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90317000 - 0x90317fff /usr/lib/system/libunc.dylib 0x9039e000 - 0x903a6fff /usr/lib/system/libsystem_pthread.dylib 0x90a1e000 - 0x90a37fff /usr/lib/system/libsystem_malloc.dylib 0x91687000 - 0x9168dfff /usr/lib/system/libsystem_networkextension.dylib 0x92055000 - 0x92059fff /usr/lib/system/libcache.dylib 0x92afd000 - 0x92b02fff /usr/lib/system/libmacho.dylib 0x931b9000 - 0x931ecfff /usr/lib/system/libsystem_m.dylib 0x935de000 - 0x935e7fff /usr/lib/system/libsystem_notify.dylib 0x93ccc000 - 0x93cf2fff /usr/lib/system/libxpc.dylib 0x94d3f000 - 0x94d41fff /usr/lib/system/libsystem_coreservices.dylib 0x95062000 - 0x95081fff /usr/lib/system/libsystem_kernel.dylib 0x9508f000 - 0x95091fff /usr/lib/system/libsystem_sandbox.dylib 0x95092000 - 0x950cffff /usr/lib/system/libsystem_network.dylib 0x951d2000 - 0x951f9fff /usr/lib/system/libdispatch.dylib 0x95274000 - 0x95280fff /usr/lib/system/libcommonCrypto.dylib 0x96054000 - 0x96065fff /usr/lib/system/libsystem_coretls.dylib 0x96cb5000 - 0x96cb5fff /usr/lib/system/liblaunch.dylib 0x96cb6000 - 0x96cdffff /usr/lib/system/libsystem_info.dylib 0x96f8f000 - 0x96f95fff /usr/lib/system/libsystem_trace.dylib 0x9701c000 - 0x9701ffff /usr/lib/system/libdyld.dylib 0x98778000 - 0x9877ffff /usr/lib/system/libunwind.dylib 0x98780000 - 0x98781fff /usr/lib/libSystem.B.dylib 0x989f6000 - 0x98a8bfff /usr/lib/system/libsystem_c.dylib 0x98caa000 - 0x98d16fff /usr/lib/system/libcorecrypto.dylib 0x98f2d000 - 0x98f2ffff /usr/lib/system/libsystem_configuration.dylib 0x98f30000 - 0x98f39fff /usr/lib/system/libcopyfile.dylib 0x98f3a000 - 0x98f3afff /usr/lib/system/libkeymgr.dylib 0x98f48000 - 0x98f5ffff /usr/lib/system/libsystem_asl.dylib 0x98f60000 - 0x98f65fff /usr/lib/system/libcompiler_rt.dylib 0x9946e000 - 0x9946ffff /usr/lib/system/libremovefile.dylib 0x9b637000 - 0x9b639fff /usr/lib/system/libsystem_secinit.dylib 0x9b823000 - 0x9b82bfff /usr/lib/system/libsystem_dnssd.dylib 0x9b8f0000 - 0x9b8f2fff /usr/lib/system/libquarantine.dylib 0x9b970000 - 0x9b97efff /usr/lib/libz.1.dylib 0x9c1d6000 - 0x9c1dcfff /usr/lib/system/libsystem_platform.dylib 0x9c47f000 - 0x9c480fff /usr/lib/system/libsystem_blocks.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1989, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.5 build 14F1509 Mon Dec 28 08:56:56 2015 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4638 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90317000 - 0x90317fff /usr/lib/system/libunc.dylib 0x9039e000 - 0x903a6fff /usr/lib/system/libsystem_pthread.dylib 0x90a1e000 - 0x90a37fff /usr/lib/system/libsystem_malloc.dylib 0x91687000 - 0x9168dfff /usr/lib/system/libsystem_networkextension.dylib 0x92055000 - 0x92059fff /usr/lib/system/libcache.dylib 0x92afd000 - 0x92b02fff /usr/lib/system/libmacho.dylib 0x931b9000 - 0x931ecfff /usr/lib/system/libsystem_m.dylib 0x935de000 - 0x935e7fff /usr/lib/system/libsystem_notify.dylib 0x93ccc000 - 0x93cf2fff /usr/lib/system/libxpc.dylib 0x94d3f000 - 0x94d41fff /usr/lib/system/libsystem_coreservices.dylib 0x95062000 - 0x95081fff /usr/lib/system/libsystem_kernel.dylib 0x9508f000 - 0x95091fff /usr/lib/system/libsystem_sandbox.dylib 0x95092000 - 0x950cffff /usr/lib/system/libsystem_network.dylib 0x951d2000 - 0x951f9fff /usr/lib/system/libdispatch.dylib 0x95274000 - 0x95280fff /usr/lib/system/libcommonCrypto.dylib 0x96054000 - 0x96065fff /usr/lib/system/libsystem_coretls.dylib 0x96cb5000 - 0x96cb5fff /usr/lib/system/liblaunch.dylib 0x96cb6000 - 0x96cdffff /usr/lib/system/libsystem_info.dylib 0x96f8f000 - 0x96f95fff /usr/lib/system/libsystem_trace.dylib 0x9701c000 - 0x9701ffff /usr/lib/system/libdyld.dylib 0x98778000 - 0x9877ffff /usr/lib/system/libunwind.dylib 0x98780000 - 0x98781fff /usr/lib/libSystem.B.dylib 0x989f6000 - 0x98a8bfff /usr/lib/system/libsystem_c.dylib 0x98caa000 - 0x98d16fff /usr/lib/system/libcorecrypto.dylib 0x98f2d000 - 0x98f2ffff /usr/lib/system/libsystem_configuration.dylib 0x98f30000 - 0x98f39fff /usr/lib/system/libcopyfile.dylib 0x98f3a000 - 0x98f3afff /usr/lib/system/libkeymgr.dylib 0x98f48000 - 0x98f5ffff /usr/lib/system/libsystem_asl.dylib 0x98f60000 - 0x98f65fff /usr/lib/system/libcompiler_rt.dylib 0x9946e000 - 0x9946ffff /usr/lib/system/libremovefile.dylib 0x9b637000 - 0x9b639fff /usr/lib/system/libsystem_secinit.dylib 0x9b823000 - 0x9b82bfff /usr/lib/system/libsystem_dnssd.dylib 0x9b8f0000 - 0x9b8f2fff /usr/lib/system/libquarantine.dylib 0x9b970000 - 0x9b97efff /usr/lib/libz.1.dylib 0x9c1d6000 - 0x9c1dcfff /usr/lib/system/libsystem_platform.dylib 0x9c47f000 - 0x9c480fff /usr/lib/system/libsystem_blocks.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1989, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.5 build 14F1509 Mon Dec 28 08:57:04 2015 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4638 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90317000 - 0x90317fff /usr/lib/system/libunc.dylib 0x9039e000 - 0x903a6fff /usr/lib/system/libsystem_pthread.dylib 0x90a1e000 - 0x90a37fff /usr/lib/system/libsystem_malloc.dylib 0x91687000 - 0x9168dfff /usr/lib/system/libsystem_networkextension.dylib 0x92055000 - 0x92059fff /usr/lib/system/libcache.dylib 0x92afd000 - 0x92b02fff /usr/lib/system/libmacho.dylib 0x931b9000 - 0x931ecfff /usr/lib/system/libsystem_m.dylib 0x935de000 - 0x935e7fff /usr/lib/system/libsystem_notify.dylib 0x93ccc000 - 0x93cf2fff /usr/lib/system/libxpc.dylib 0x94d3f000 - 0x94d41fff /usr/lib/system/libsystem_coreservices.dylib 0x95062000 - 0x95081fff /usr/lib/system/libsystem_kernel.dylib 0x9508f000 - 0x95091fff /usr/lib/system/libsystem_sandbox.dylib 0x95092000 - 0x950cffff /usr/lib/system/libsystem_network.dylib 0x951d2000 - 0x951f9fff /usr/lib/system/libdispatch.dylib 0x95274000 - 0x95280fff /usr/lib/system/libcommonCrypto.dylib 0x96054000 - 0x96065fff /usr/lib/system/libsystem_coretls.dylib 0x96cb5000 - 0x96cb5fff /usr/lib/system/liblaunch.dylib 0x96cb6000 - 0x96cdffff /usr/lib/system/libsystem_info.dylib 0x96f8f000 - 0x96f95fff /usr/lib/system/libsystem_trace.dylib 0x9701c000 - 0x9701ffff /usr/lib/system/libdyld.dylib 0x98778000 - 0x9877ffff /usr/lib/system/libunwind.dylib 0x98780000 - 0x98781fff /usr/lib/libSystem.B.dylib 0x989f6000 - 0x98a8bfff /usr/lib/system/libsystem_c.dylib 0x98caa000 - 0x98d16fff /usr/lib/system/libcorecrypto.dylib 0x98f2d000 - 0x98f2ffff /usr/lib/system/libsystem_configuration.dylib 0x98f30000 - 0x98f39fff /usr/lib/system/libcopyfile.dylib 0x98f3a000 - 0x98f3afff /usr/lib/system/libkeymgr.dylib 0x98f48000 - 0x98f5ffff /usr/lib/system/libsystem_asl.dylib 0x98f60000 - 0x98f65fff /usr/lib/system/libcompiler_rt.dylib 0x9946e000 - 0x9946ffff /usr/lib/system/libremovefile.dylib 0x9b637000 - 0x9b639fff /usr/lib/system/libsystem_secinit.dylib 0x9b823000 - 0x9b82bfff /usr/lib/system/libsystem_dnssd.dylib 0x9b8f0000 - 0x9b8f2fff /usr/lib/system/libquarantine.dylib 0x9b970000 - 0x9b97efff /usr/lib/libz.1.dylib 0x9c1d6000 - 0x9c1dcfff /usr/lib/system/libsystem_platform.dylib 0x9c47f000 - 0x9c480fff /usr/lib/system/libsystem_blocks.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1989, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.5 build 14F1509 Mon Dec 28 08:57:11 2015 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbebb4638 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90317000 - 0x90317fff /usr/lib/system/libunc.dylib 0x9039e000 - 0x903a6fff /usr/lib/system/libsystem_pthread.dylib 0x90a1e000 - 0x90a37fff /usr/lib/system/libsystem_malloc.dylib 0x91687000 - 0x9168dfff /usr/lib/system/libsystem_networkextension.dylib 0x92055000 - 0x92059fff /usr/lib/system/libcache.dylib 0x92afd000 - 0x92b02fff /usr/lib/system/libmacho.dylib 0x931b9000 - 0x931ecfff /usr/lib/system/libsystem_m.dylib 0x935de000 - 0x935e7fff /usr/lib/system/libsystem_notify.dylib 0x93ccc000 - 0x93cf2fff /usr/lib/system/libxpc.dylib 0x94d3f000 - 0x94d41fff /usr/lib/system/libsystem_coreservices.dylib 0x95062000 - 0x95081fff /usr/lib/system/libsystem_kernel.dylib 0x9508f000 - 0x95091fff /usr/lib/system/libsystem_sandbox.dylib 0x95092000 - 0x950cffff /usr/lib/system/libsystem_network.dylib 0x951d2000 - 0x951f9fff /usr/lib/system/libdispatch.dylib 0x95274000 - 0x95280fff /usr/lib/system/libcommonCrypto.dylib 0x96054000 - 0x96065fff /usr/lib/system/libsystem_coretls.dylib 0x96cb5000 - 0x96cb5fff /usr/lib/system/liblaunch.dylib 0x96cb6000 - 0x96cdffff /usr/lib/system/libsystem_info.dylib 0x96f8f000 - 0x96f95fff /usr/lib/system/libsystem_trace.dylib 0x9701c000 - 0x9701ffff /usr/lib/system/libdyld.dylib 0x98778000 - 0x9877ffff /usr/lib/system/libunwind.dylib 0x98780000 - 0x98781fff /usr/lib/libSystem.B.dylib 0x989f6000 - 0x98a8bfff /usr/lib/system/libsystem_c.dylib 0x98caa000 - 0x98d16fff /usr/lib/system/libcorecrypto.dylib 0x98f2d000 - 0x98f2ffff /usr/lib/system/libsystem_configuration.dylib 0x98f30000 - 0x98f39fff /usr/lib/system/libcopyfile.dylib 0x98f3a000 - 0x98f3afff /usr/lib/system/libkeymgr.dylib 0x98f48000 - 0x98f5ffff /usr/lib/system/libsystem_asl.dylib 0x98f60000 - 0x98f65fff /usr/lib/system/libcompiler_rt.dylib 0x9946e000 - 0x9946ffff /usr/lib/system/libremovefile.dylib 0x9b637000 - 0x9b639fff /usr/lib/system/libsystem_secinit.dylib 0x9b823000 - 0x9b82bfff /usr/lib/system/libsystem_dnssd.dylib 0x9b8f0000 - 0x9b8f2fff /usr/lib/system/libquarantine.dylib 0x9b970000 - 0x9b97efff /usr/lib/libz.1.dylib 0x9c1d6000 - 0x9c1dcfff /usr/lib/system/libsystem_platform.dylib 0x9c47f000 - 0x9c480fff /usr/lib/system/libsystem_blocks.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1989, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Dec 2015 04:02:36	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	984,960	1,140,816	1.1582
23 Dec 2015 05:03:40	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	959,040	1,109,928	1.1573
23 Dec 2015 04:02:46	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	933,120	1,080,619	1.1581
22 Dec 2015 07:52:48	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	907,200	1,050,450	1.1579
22 Dec 2015 04:00:35	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	881,280	1,020,891	1.1584
21 Dec 2015 08:52:28	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	855,360	992,098	1.1599
21 Dec 2015 04:04:59	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	829,440	962,281	1.1602
21 Dec 2015 04:04:59	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	803,520	932,652	1.1607
20 Dec 2015 05:01:01	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	777,600	903,673	1.1621
20 Dec 2015 04:04:56	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	751,680	874,326	1.1632
19 Dec 2015 08:41:36	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	725,760	844,352	1.1634
19 Dec 2015 04:03:41	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	699,840	813,656	1.1626
18 Dec 2015 08:32:38	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	673,920	783,594	1.1627
18 Dec 2015 04:05:20	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	648,000	754,281	1.1640
17 Dec 2015 06:44:06	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	622,080	723,782	1.1635
17 Dec 2015 04:01:58	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	596,160	693,946	1.1640
16 Dec 2015 05:02:18	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	570,240	664,102	1.1646
16 Dec 2015 04:00:59	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	544,320	634,630	1.1659
15 Dec 2015 05:11:25	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	518,400	605,698	1.1684
15 Dec 2015 04:00:32	1280444	19135403	hadcm3n_sc5k_198012_480_010223092_1	492,480	575,576	1.1687