Task 15675098

Name	hadcm3n_3f6e_1940_40_008258520_1
Workunit	8413644
Created	21 Mar 2013, 18:40:19 UTC
Sent	21 Mar 2013, 18:40:24 UTC
Report deadline	21 Jun 2013, 2:07:35 UTC
Received	30 Mar 2013, 0:46:14 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1289096
Run time	3 days 12 hours 34 min 35 sec
CPU time	3 days 1 hours 35 min 49 sec
Validate state	Invalid
Credit	1,866.24
Device peak FLOPS	3.35 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.0.31</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 21:58:17 (7662): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... 23:47:09 (18925): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Fri Mar 29 13:32:59 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34598, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Fri Mar 29 13:33:01 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34598, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Fri Mar 29 13:33:04 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34598, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Fri Mar 29 13:33:07 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34598, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Fri Mar 29 13:33:09 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34598, iMonCtr=1 Model crash detected, will try to restart... SIGABRT: abort called Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.7.4 build 11E53 Fri Mar 29 13:33:12 2013 Thread 0 Crashed: Thread 1: Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffccc8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2 cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin 0x90207000 - 0x90209fff /usr/lib/system/libdyld.dylib 0x902e3000 - 0x90326fff /usr/lib/system/libcommonCrypto.dylib 0x90fad000 - 0x90faefff /usr/lib/system/libunc.dylib 0x910e7000 - 0x910e8fff /usr/lib/system/libremovefile.dylib 0x913d3000 - 0x913d7fff /usr/lib/system/libsystem_network.dylib 0x9165e000 - 0x9167cfff /usr/lib/system/libsystem_kernel.dylib 0x9375e000 - 0x9375efff /usr/lib/system/libdnsinfo.dylib 0x93bcc000 - 0x93bcdfff /usr/lib/system/libsystem_blocks.dylib 0x93bcf000 - 0x93bfefff /usr/lib/system/libsystem_info.dylib 0x96760000 - 0x96765fff /usr/lib/system/libmacho.dylib 0x97181000 - 0x9724cfff /usr/lib/system/libsystem_c.dylib 0x97e86000 - 0x97e87fff /usr/lib/system/libquarantine.dylib 0x989b8000 - 0x989bffff /usr/lib/system/libsystem_notify.dylib 0x98e7e000 - 0x98e7efff /usr/lib/system/libkeymgr.dylib 0x98fac000 - 0x98fb4fff /usr/lib/system/libunwind.dylib 0x9a01b000 - 0x9a023fff /usr/lib/system/liblaunch.dylib 0x9a85e000 - 0x9a86cfff /usr/lib/libz.1.dylib 0x9a87a000 - 0x9a882fff /usr/lib/system/libcopyfile.dylib 0x9a883000 - 0x9a88afff /usr/lib/system/libsystem_dnssd.dylib 0x9b6d8000 - 0x9b6dbfff /usr/lib/system/libmathCommon.A.dylib 0x9b7aa000 - 0x9b7aefff /usr/lib/system/libcache.dylib 0x9c0e6000 - 0x9c0f4fff /usr/lib/system/libdispatch.dylib 0x9c101000 - 0x9c117fff /usr/lib/system/libxpc.dylib 0x9c9f1000 - 0x9ca1ffff /usr/lib/libSystem.B.dylib 0x9ca20000 - 0x9ca23fff /usr/lib/system/libcompiler_rt.dylib 0x9ca5e000 - 0x9ca5ffff /usr/lib/system/libsystem_sandbox.dylib Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=34598, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
29 Mar 2013 09:56:20	1216073	15675098	hadcm3n_3f6e_1940_40_008258520_1	155,520	250,912	1.6134
28 Mar 2013 16:16:23	1216073	15675098	hadcm3n_3f6e_1940_40_008258520_1	129,600	211,478	1.6318
28 Mar 2013 01:46:12	1216073	15675098	hadcm3n_3f6e_1940_40_008258520_1	103,680	169,554	1.6354
27 Mar 2013 02:24:58	1216073	15675098	hadcm3n_3f6e_1940_40_008258520_1	77,760	127,387	1.6382
26 Mar 2013 02:48:49	1216073	15675098	hadcm3n_3f6e_1940_40_008258520_1	51,840	85,319	1.6458
22 Mar 2013 14:15:15	1216073	15675098	hadcm3n_3f6e_1940_40_008258520_1	25,920	41,574	1.6039