Task 15892810

Name	hadcm3n_o4po_1980_40_008399012_1
Workunit	8549868
Created	12 Jul 2013, 0:47:00 UTC
Sent	12 Jul 2013, 0:56:56 UTC
Report deadline	11 Oct 2013, 8:24:07 UTC
Received	21 Aug 2013, 0:05:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1230357
Run time	7 days 14 hours 51 min 21 sec
CPU time	7 days 12 hours 31 min 24 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	2.81 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 09:26:51 (4670): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:49:35 (55199): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:52:07 (55208): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... 10:07:08 (55218): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G56 Tue Aug 20 11:40:59 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c4d000 - 0x90c4efff /usr/lib/system/libquarantine.dylib 0x9149b000 - 0x9149ffff /usr/lib/system/libcache.dylib 0x92ce9000 - 0x92cecfff /usr/lib/system/libmathCommon.A.dylib 0x9351f000 - 0x93520fff /usr/lib/system/libsystem_sandbox.dylib 0x93ae9000 - 0x93b2cfff /usr/lib/system/libcommonCrypto.dylib 0x93b6e000 - 0x93b76fff /usr/lib/system/libunwind.dylib 0x93bf0000 - 0x93bf8fff /usr/lib/system/libcopyfile.dylib 0x93c03000 - 0x93c06fff /usr/lib/system/libcompiler_rt.dylib 0x94924000 - 0x94953fff /usr/lib/system/libsystem_info.dylib 0x949cb000 - 0x949ccfff /usr/lib/system/libremovefile.dylib 0x94fa2000 - 0x9506dfff /usr/lib/system/libsystem_c.dylib 0x9507a000 - 0x9507bfff /usr/lib/system/libsystem_blocks.dylib 0x9547d000 - 0x9549bfff /usr/lib/system/libsystem_kernel.dylib 0x9549c000 - 0x954aafff /usr/lib/libz.1.dylib 0x96232000 - 0x96233fff /usr/lib/system/libunc.dylib 0x963cd000 - 0x963d4fff /usr/lib/system/libsystem_dnssd.dylib 0x9657f000 - 0x96595fff /usr/lib/system/libxpc.dylib 0x98d1d000 - 0x98d24fff /usr/lib/system/libsystem_notify.dylib 0x99cbf000 - 0x99cc3fff /usr/lib/system/libsystem_network.dylib 0x9a7b2000 - 0x9a7b2fff /usr/lib/system/libkeymgr.dylib 0x9b47f000 - 0x9b47ffff /usr/lib/system/libdnsinfo.dylib 0x9b5c0000 - 0x9b5c5fff /usr/lib/system/libmacho.dylib 0x9bd10000 - 0x9bd12fff /usr/lib/system/libdyld.dylib 0x9bd1f000 - 0x9bd2dfff /usr/lib/system/libdispatch.dylib 0x9cc91000 - 0x9cc99fff /usr/lib/system/liblaunch.dylib 0x9cd00000 - 0x9cd2efff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G56 Tue Aug 20 11:41:02 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c4d000 - 0x90c4efff /usr/lib/system/libquarantine.dylib 0x9149b000 - 0x9149ffff /usr/lib/system/libcache.dylib 0x92ce9000 - 0x92cecfff /usr/lib/system/libmathCommon.A.dylib 0x9351f000 - 0x93520fff /usr/lib/system/libsystem_sandbox.dylib 0x93ae9000 - 0x93b2cfff /usr/lib/system/libcommonCrypto.dylib 0x93b6e000 - 0x93b76fff /usr/lib/system/libunwind.dylib 0x93bf0000 - 0x93bf8fff /usr/lib/system/libcopyfile.dylib 0x93c03000 - 0x93c06fff /usr/lib/system/libcompiler_rt.dylib 0x94924000 - 0x94953fff /usr/lib/system/libsystem_info.dylib 0x949cb000 - 0x949ccfff /usr/lib/system/libremovefile.dylib 0x94fa2000 - 0x9506dfff /usr/lib/system/libsystem_c.dylib 0x9507a000 - 0x9507bfff /usr/lib/system/libsystem_blocks.dylib 0x9547d000 - 0x9549bfff /usr/lib/system/libsystem_kernel.dylib 0x9549c000 - 0x954aafff /usr/lib/libz.1.dylib 0x96232000 - 0x96233fff /usr/lib/system/libunc.dylib 0x963cd000 - 0x963d4fff /usr/lib/system/libsystem_dnssd.dylib 0x9657f000 - 0x96595fff /usr/lib/system/libxpc.dylib 0x98d1d000 - 0x98d24fff /usr/lib/system/libsystem_notify.dylib 0x99cbf000 - 0x99cc3fff /usr/lib/system/libsystem_network.dylib 0x9a7b2000 - 0x9a7b2fff /usr/lib/system/libkeymgr.dylib 0x9b47f000 - 0x9b47ffff /usr/lib/system/libdnsinfo.dylib 0x9b5c0000 - 0x9b5c5fff /usr/lib/system/libmacho.dylib 0x9bd10000 - 0x9bd12fff /usr/lib/system/libdyld.dylib 0x9bd1f000 - 0x9bd2dfff /usr/lib/system/libdispatch.dylib 0x9cc91000 - 0x9cc99fff /usr/lib/system/liblaunch.dylib 0x9cd00000 - 0x9cd2efff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G56 Tue Aug 20 11:41:05 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c4d000 - 0x90c4efff /usr/lib/system/libquarantine.dylib 0x9149b000 - 0x9149ffff /usr/lib/system/libcache.dylib 0x92ce9000 - 0x92cecfff /usr/lib/system/libmathCommon.A.dylib 0x9351f000 - 0x93520fff /usr/lib/system/libsystem_sandbox.dylib 0x93ae9000 - 0x93b2cfff /usr/lib/system/libcommonCrypto.dylib 0x93b6e000 - 0x93b76fff /usr/lib/system/libunwind.dylib 0x93bf0000 - 0x93bf8fff /usr/lib/system/libcopyfile.dylib 0x93c03000 - 0x93c06fff /usr/lib/system/libcompiler_rt.dylib 0x94924000 - 0x94953fff /usr/lib/system/libsystem_info.dylib 0x949cb000 - 0x949ccfff /usr/lib/system/libremovefile.dylib 0x94fa2000 - 0x9506dfff /usr/lib/system/libsystem_c.dylib 0x9507a000 - 0x9507bfff /usr/lib/system/libsystem_blocks.dylib 0x9547d000 - 0x9549bfff /usr/lib/system/libsystem_kernel.dylib 0x9549c000 - 0x954aafff /usr/lib/libz.1.dylib 0x96232000 - 0x96233fff /usr/lib/system/libunc.dylib 0x963cd000 - 0x963d4fff /usr/lib/system/libsystem_dnssd.dylib 0x9657f000 - 0x96595fff /usr/lib/system/libxpc.dylib 0x98d1d000 - 0x98d24fff /usr/lib/system/libsystem_notify.dylib 0x99cbf000 - 0x99cc3fff /usr/lib/system/libsystem_network.dylib 0x9a7b2000 - 0x9a7b2fff /usr/lib/system/libkeymgr.dylib 0x9b47f000 - 0x9b47ffff /usr/lib/system/libdnsinfo.dylib 0x9b5c0000 - 0x9b5c5fff /usr/lib/system/libmacho.dylib 0x9bd10000 - 0x9bd12fff /usr/lib/system/libdyld.dylib 0x9bd1f000 - 0x9bd2dfff /usr/lib/system/libdispatch.dylib 0x9cc91000 - 0x9cc99fff /usr/lib/system/liblaunch.dylib 0x9cd00000 - 0x9cd2efff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... Suspended CPDN Monitor - Suspend request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G56 Tue Aug 20 11:44:56 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c4d000 - 0x90c4efff /usr/lib/system/libquarantine.dylib 0x9149b000 - 0x9149ffff /usr/lib/system/libcache.dylib 0x92ce9000 - 0x92cecfff /usr/lib/system/libmathCommon.A.dylib 0x9351f000 - 0x93520fff /usr/lib/system/libsystem_sandbox.dylib 0x93ae9000 - 0x93b2cfff /usr/lib/system/libcommonCrypto.dylib 0x93b6e000 - 0x93b76fff /usr/lib/system/libunwind.dylib 0x93bf0000 - 0x93bf8fff /usr/lib/system/libcopyfile.dylib 0x93c03000 - 0x93c06fff /usr/lib/system/libcompiler_rt.dylib 0x94924000 - 0x94953fff /usr/lib/system/libsystem_info.dylib 0x949cb000 - 0x949ccfff /usr/lib/system/libremovefile.dylib 0x94fa2000 - 0x9506dfff /usr/lib/system/libsystem_c.dylib 0x9507a000 - 0x9507bfff /usr/lib/system/libsystem_blocks.dylib 0x9547d000 - 0x9549bfff /usr/lib/system/libsystem_kernel.dylib 0x9549c000 - 0x954aafff /usr/lib/libz.1.dylib 0x96232000 - 0x96233fff /usr/lib/system/libunc.dylib 0x963cd000 - 0x963d4fff /usr/lib/system/libsystem_dnssd.dylib 0x9657f000 - 0x96595fff /usr/lib/system/libxpc.dylib 0x98d1d000 - 0x98d24fff /usr/lib/system/libsystem_notify.dylib 0x99cbf000 - 0x99cc3fff /usr/lib/system/libsystem_network.dylib 0x9a7b2000 - 0x9a7b2fff /usr/lib/system/libkeymgr.dylib 0x9b47f000 - 0x9b47ffff /usr/lib/system/libdnsinfo.dylib 0x9b5c0000 - 0x9b5c5fff /usr/lib/system/libmacho.dylib 0x9bd10000 - 0x9bd12fff /usr/lib/system/libdyld.dylib 0x9bd1f000 - 0x9bd2dfff /usr/lib/system/libdispatch.dylib 0x9cc91000 - 0x9cc99fff /usr/lib/system/liblaunch.dylib 0x9cd00000 - 0x9cd2efff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G56 Tue Aug 20 11:45:00 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c4d000 - 0x90c4efff /usr/lib/system/libquarantine.dylib 0x9149b000 - 0x9149ffff /usr/lib/system/libcache.dylib 0x92ce9000 - 0x92cecfff /usr/lib/system/libmathCommon.A.dylib 0x9351f000 - 0x93520fff /usr/lib/system/libsystem_sandbox.dylib 0x93ae9000 - 0x93b2cfff /usr/lib/system/libcommonCrypto.dylib 0x93b6e000 - 0x93b76fff /usr/lib/system/libunwind.dylib 0x93bf0000 - 0x93bf8fff /usr/lib/system/libcopyfile.dylib 0x93c03000 - 0x93c06fff /usr/lib/system/libcompiler_rt.dylib 0x94924000 - 0x94953fff /usr/lib/system/libsystem_info.dylib 0x949cb000 - 0x949ccfff /usr/lib/system/libremovefile.dylib 0x94fa2000 - 0x9506dfff /usr/lib/system/libsystem_c.dylib 0x9507a000 - 0x9507bfff /usr/lib/system/libsystem_blocks.dylib 0x9547d000 - 0x9549bfff /usr/lib/system/libsystem_kernel.dylib 0x9549c000 - 0x954aafff /usr/lib/libz.1.dylib 0x96232000 - 0x96233fff /usr/lib/system/libunc.dylib 0x963cd000 - 0x963d4fff /usr/lib/system/libsystem_dnssd.dylib 0x9657f000 - 0x96595fff /usr/lib/system/libxpc.dylib 0x98d1d000 - 0x98d24fff /usr/lib/system/libsystem_notify.dylib 0x99cbf000 - 0x99cc3fff /usr/lib/system/libsystem_network.dylib 0x9a7b2000 - 0x9a7b2fff /usr/lib/system/libkeymgr.dylib 0x9b47f000 - 0x9b47ffff /usr/lib/system/libdnsinfo.dylib 0x9b5c0000 - 0x9b5c5fff /usr/lib/system/libmacho.dylib 0x9bd10000 - 0x9bd12fff /usr/lib/system/libdyld.dylib 0x9bd1f000 - 0x9bd2dfff /usr/lib/system/libdispatch.dylib 0x9cc91000 - 0x9cc99fff /usr/lib/system/liblaunch.dylib 0x9cd00000 - 0x9cd2efff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.5 build 11G56 Tue Aug 20 11:45:02 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90c4d000 - 0x90c4efff /usr/lib/system/libquarantine.dylib 0x9149b000 - 0x9149ffff /usr/lib/system/libcache.dylib 0x92ce9000 - 0x92cecfff /usr/lib/system/libmathCommon.A.dylib 0x9351f000 - 0x93520fff /usr/lib/system/libsystem_sandbox.dylib 0x93ae9000 - 0x93b2cfff /usr/lib/system/libcommonCrypto.dylib 0x93b6e000 - 0x93b76fff /usr/lib/system/libunwind.dylib 0x93bf0000 - 0x93bf8fff /usr/lib/system/libcopyfile.dylib 0x93c03000 - 0x93c06fff /usr/lib/system/libcompiler_rt.dylib 0x94924000 - 0x94953fff /usr/lib/system/libsystem_info.dylib 0x949cb000 - 0x949ccfff /usr/lib/system/libremovefile.dylib 0x94fa2000 - 0x9506dfff /usr/lib/system/libsystem_c.dylib 0x9507a000 - 0x9507bfff /usr/lib/system/libsystem_blocks.dylib 0x9547d000 - 0x9549bfff /usr/lib/system/libsystem_kernel.dylib 0x9549c000 - 0x954aafff /usr/lib/libz.1.dylib 0x96232000 - 0x96233fff /usr/lib/system/libunc.dylib 0x963cd000 - 0x963d4fff /usr/lib/system/libsystem_dnssd.dylib 0x9657f000 - 0x96595fff /usr/lib/system/libxpc.dylib 0x98d1d000 - 0x98d24fff /usr/lib/system/libsystem_notify.dylib 0x99cbf000 - 0x99cc3fff /usr/lib/system/libsystem_network.dylib 0x9a7b2000 - 0x9a7b2fff /usr/lib/system/libkeymgr.dylib 0x9b47f000 - 0x9b47ffff /usr/lib/system/libdnsinfo.dylib 0x9b5c0000 - 0x9b5c5fff /usr/lib/system/libmacho.dylib 0x9bd10000 - 0x9bd12fff /usr/lib/system/libdyld.dylib 0x9bd1f000 - 0x9bd2dfff /usr/lib/system/libdispatch.dylib 0x9cc91000 - 0x9cc99fff /usr/lib/system/liblaunch.dylib 0x9cd00000 - 0x9cd2efff /usr/lib/libSystem.B.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=268, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
20 Aug 2013 16:55:36	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	725,760	646,713	0.8911
20 Aug 2013 15:29:54	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	699,840	626,486	0.8952
20 Aug 2013 15:29:54	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	673,920	606,529	0.9000
20 Aug 2013 15:29:54	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	648,000	586,119	0.9045
20 Aug 2013 15:29:54	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	622,080	565,429	0.9089
20 Aug 2013 15:29:54	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	596,160	545,312	0.9147
20 Aug 2013 15:29:54	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	570,240	525,483	0.9215
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	544,320	505,661	0.9290
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	518,400	485,815	0.9371
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	492,480	465,801	0.9458
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	466,560	445,945	0.9558
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	440,640	426,082	0.9670
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	414,720	405,440	0.9776
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	388,800	385,129	0.9906
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	362,880	365,107	1.0061
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	336,960	344,696	1.0230
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	311,040	324,800	1.0442
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	285,120	304,219	1.0670
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	259,200	283,669	1.0944
20 Aug 2013 15:29:53	1230357	15892810	hadcm3n_o4po_1980_40_008399012_1	233,280	263,231	1.1284