Task 17796091

Name	hadcm3n_xafh_1980_40_009465910_0
Workunit	9548144
Created	15 Jan 2015, 16:38:42 UTC
Sent	18 Jan 2015, 13:51:17 UTC
Report deadline	19 Apr 2015, 21:18:28 UTC
Received	2 Feb 2015, 1:28:25 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1451375
Run time	7 days 14 hours 47 min 45 sec
CPU time	7 days 2 hours 14 min 13 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	3.77 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.4.36</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 10:18:56 (66179): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 19:33:01 (22396): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 03:04:38 (35612): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 03:49:37 (39674): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:52:40 (49004): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:56:09 (52512): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 20:24:15 (58056): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 21:29:12 (64661): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:54:21 (68527): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:23:05 (79725): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:45:13 (83843): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:13:04 (13501): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 23:04:24 (20814): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 09:41:14 (32696): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:49:10 (41188): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:07:19 (45198): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:17:34 (59971): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 23:58:49 (63556): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:00:52 (73476): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:56:34 (269): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 04:09:38 (5825): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:54:01 (28499): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:03:10 (31906): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 12:16:57 (36654): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 12:41:40 (40124): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:14:53 (43681): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 17:16:56 (48705): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 20:27:28 (60337): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:29:00 (73265): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:50:09 (89880): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 13:08:06 (94631): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:17:28 (98275): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 13:25:38 (1912): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 15:08:43 (5567): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:06:05 (14617): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:07:24 (18190): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 18:33:25 (21593): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:07:52 (8849): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:11:03 (12780): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:46:16 (43120): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 16:10:57 (46955): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 03:42:15 (76269): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:30:55 (79852): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 08:23:10 (85485): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 17:20:27 (96660): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 19:03:49 (1225): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:23:15 (4761): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 19:31:48 (8303): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 01:51:01 (14338): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:22:50 (18079): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:46:57 (21799): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 08:42:12 (76810): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:37:16 (87309): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:11:22 (10245): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:22:54 (20079): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 14:23:29 (23664): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 23:56:49 (53732): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:20:42 (57149): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 00:37:54 (60771): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 01:22:03 (64347): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 04:06:20 (76668): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 07:37:31 (85247): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 08:24:46 (89303): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:23:17 (93169): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 09:47:02 (97154): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 10:48:02 (959): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 14:06:19 (6679): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... 18:53:43 (19335): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 06:11:14 (40207): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x831004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x84c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x84c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x301b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x301b600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x301b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x301b600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x84c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x84c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x301b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x301b600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x831004: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x84c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x84c600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x2851204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x4823a04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(60748,0xa06c51d4) malloc: * error for object 0x4823a00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel 80486 (32-bit executable) System version: Macintosh OS 10.10.2 build 14C109 Sun Feb 1 18:26:20 2015 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fb68 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/7/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x9068d000 - 0x9068dfff /usr/lib/system/liblaunch.dylib 0x9068e000 - 0x90696fff /usr/lib/system/libsystem_dnssd.dylib 0x90697000 - 0x90699fff /usr/lib/system/libquarantine.dylib 0x90c0d000 - 0x90c11fff /usr/lib/system/libcache.dylib 0x90e42000 - 0x90e61fff /usr/lib/system/libsystem_kernel.dylib 0x911cc000 - 0x911cdfff /usr/lib/system/libsystem_blocks.dylib 0x911ce000 - 0x911f5fff /usr/lib/system/libdispatch.dylib 0x9124d000 - 0x9124efff /usr/lib/system/libremovefile.dylib 0x918b3000 - 0x91906fff /usr/lib/libstdc++.6.dylib 0x92012000 - 0x9207bfff /usr/lib/system/libcorecrypto.dylib 0x9207c000 - 0x9207ffff /usr/lib/system/libdyld.dylib 0x92147000 - 0x92150fff /usr/lib/system/libsystem_notify.dylib 0x92439000 - 0x92462fff /usr/lib/system/libsystem_info.dylib 0x9246b000 - 0x92491fff /usr/lib/libc++abi.dylib 0x92c90000 - 0x92c92fff /usr/lib/system/libsystem_secinit.dylib 0x938a6000 - 0x938a8fff /usr/lib/system/libsystem_coreservices.dylib 0x93c05000 - 0x93c0dfff /usr/lib/system/libsystem_pthread.dylib 0x93e28000 - 0x93e2ffff /usr/lib/system/libunwind.dylib 0x93e30000 - 0x93ec5fff /usr/lib/system/libsystem_c.dylib 0x94473000 - 0x94475fff /usr/lib/system/libsystem_configuration.dylib 0x95051000 - 0x95056fff /usr/lib/system/libmacho.dylib 0x952ba000 - 0x952bafff /usr/lib/system/libkeymgr.dylib 0x96953000 - 0x96954fff /usr/lib/libSystem.B.dylib 0x96955000 - 0x9695afff /usr/lib/system/libcompiler_rt.dylib 0x96c70000 - 0x96c79fff /usr/lib/system/libcopyfile.dylib 0x96c9b000 - 0x96ccefff /usr/lib/system/libsystem_m.dylib 0x977fb000 - 0x977fbfff /usr/lib/system/libunc.dylib 0x978e7000 - 0x97900fff /usr/lib/system/libsystem_malloc.dylib 0x98252000 - 0x98263fff /usr/lib/system/libsystem_coretls.dylib 0x98a1e000 - 0x98a24fff /usr/lib/system/libsystem_trace.dylib 0x98bd4000 - 0x98bdafff /usr/lib/system/libsystem_platform.dylib 0x98f95000 - 0x98fbbfff /usr/lib/system/libxpc.dylib 0x993f5000 - 0x99432fff /usr/lib/system/libsystem_network.dylib 0x99433000 - 0x99439fff /usr/lib/system/libsystem_networkextension.dylib 0x99f14000 - 0x99f2bfff /usr/lib/system/libsystem_asl.dylib 0x9ab6a000 - 0x9ab78fff /usr/lib/libz.1.dylib 0x9ae18000 - 0x9ae24fff /usr/lib/system/libcommonCrypto.dylib 0x9ae25000 - 0x9ae27fff /usr/lib/system/libsystem_sandbox.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Feb 2015 01:31:02	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	518,400	612,853	1.1822
01 Feb 2015 10:18:24	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	492,480	584,246	1.1863
01 Feb 2015 01:24:44	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	466,560	555,289	1.1902
31 Jan 2015 11:12:22	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	440,640	526,499	1.1949
30 Jan 2015 12:45:00	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	414,720	497,299	1.1991
29 Jan 2015 20:59:21	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	388,800	468,777	1.2057
29 Jan 2015 07:37:38	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	362,880	440,274	1.2133
28 Jan 2015 08:52:02	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	336,960	411,512	1.2212
27 Jan 2015 18:59:41	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	311,040	382,129	1.2286
27 Jan 2015 02:56:03	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	285,120	351,723	1.2336
26 Jan 2015 09:57:52	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	259,200	321,661	1.2410
25 Jan 2015 03:16:16	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	233,280	290,692	1.2461
24 Jan 2015 09:44:02	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	207,360	258,984	1.2490
23 Jan 2015 20:50:43	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	181,440	227,296	1.2527
22 Jan 2015 22:59:01	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	155,520	195,466	1.2569
22 Jan 2015 04:57:35	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	129,600	162,806	1.2562
21 Jan 2015 18:47:16	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	103,680	129,964	1.2535
21 Jan 2015 03:08:27	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	77,760	97,832	1.2581
19 Jan 2015 17:50:53	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	51,840	66,090	1.2749
19 Jan 2015 03:01:56	1313549	17796091	hadcm3n_xafh_1980_40_009465910_0	25,920	33,093	1.2767