Task 19135363

Name	hadcm3n_scrf_198012_480_010222290_2
Workunit	10222290
Created	5 Dec 2015, 3:39:07 UTC
Sent	5 Dec 2015, 3:45:22 UTC
Report deadline	16 Nov 2016, 9:05:22 UTC
Received	3 Feb 2016, 3:29:32 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1381327
Run time	13 days 11 hours 36 min 32 sec
CPU time	12 days 11 hours 47 min 53 sec
Validate state	Invalid
Credit	12,441.60
Device peak FLOPS	4.60 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.12</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x3002600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x1800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(32741,0xa05211d4) malloc: * error for object 0x1800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x183be04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x183be00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x183be04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x183be00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x183be04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x183be00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x483c604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.10.5 build 14F1605 hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x1820804: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(7855,0xa0b1b1d4) malloc: * error for object 0x1820800: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Tue Feb 2 20:28:48 2016 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fa58 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/12/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x9023e000 - 0x90267fff /usr/lib/system/libsystem_info.dylib 0x902c9000 - 0x902d2fff /usr/lib/system/libcopyfile.dylib 0x9081e000 - 0x90820fff /usr/lib/system/libsystem_configuration.dylib 0x90821000 - 0x9085efff /usr/lib/system/libsystem_network.dylib 0x9085f000 - 0x908b2fff /usr/lib/libstdc++.6.dylib 0x90931000 - 0x90939fff /usr/lib/system/libsystem_dnssd.dylib 0x90f2a000 - 0x90f30fff /usr/lib/system/libsystem_networkextension.dylib 0x91516000 - 0x9151cfff /usr/lib/system/libsystem_trace.dylib 0x91cb6000 - 0x91ccdfff /usr/lib/system/libsystem_asl.dylib 0x922ef000 - 0x922effff /usr/lib/system/liblaunch.dylib 0x9231f000 - 0x92345fff /usr/lib/system/libxpc.dylib 0x927bd000 - 0x927befff /usr/lib/system/libremovefile.dylib 0x927bf000 - 0x927c4fff /usr/lib/system/libcompiler_rt.dylib 0x93120000 - 0x93120fff /usr/lib/system/libkeymgr.dylib 0x93121000 - 0x931b6fff /usr/lib/system/libsystem_c.dylib 0x93c48000 - 0x93c6ffff /usr/lib/system/libdispatch.dylib 0x93cb1000 - 0x93cb5fff /usr/lib/system/libcache.dylib 0x9438a000 - 0x94393fff /usr/lib/system/libsystem_notify.dylib 0x94394000 - 0x94394fff /usr/lib/system/libunc.dylib 0x95465000 - 0x9547efff /usr/lib/system/libsystem_malloc.dylib 0x9547f000 - 0x95480fff /usr/lib/system/libsystem_blocks.dylib 0x957d1000 - 0x957d2fff /usr/lib/libSystem.B.dylib 0x96851000 - 0x96854fff /usr/lib/system/libdyld.dylib 0x96d01000 - 0x96d09fff /usr/lib/system/libsystem_pthread.dylib 0x97d70000 - 0x97d8ffff /usr/lib/system/libsystem_kernel.dylib 0x98e08000 - 0x98e0efff /usr/lib/system/libsystem_platform.dylib 0x9923d000 - 0x9923ffff /usr/lib/system/libsystem_sandbox.dylib 0x99555000 - 0x99563fff /usr/lib/libz.1.dylib 0x99564000 - 0x99575fff /usr/lib/system/libsystem_coretls.dylib 0x998d8000 - 0x998ddfff /usr/lib/system/libmacho.dylib 0x9b20d000 - 0x9b240fff /usr/lib/system/libsystem_m.dylib 0x9b3b1000 - 0x9b3b8fff /usr/lib/system/libunwind.dylib 0x9b6e4000 - 0x9b750fff /usr/lib/system/libcorecrypto.dylib 0x9b7f5000 - 0x9b7f7fff /usr/lib/system/libquarantine.dylib 0x9b7f8000 - 0x9b7fafff /usr/lib/system/libsystem_secinit.dylib 0x9b9d2000 - 0x9b9defff /usr/lib/system/libcommonCrypto.dylib 0x9b9e0000 - 0x9ba06fff /usr/lib/libc++abi.dylib 0x9c497000 - 0x9c499fff /usr/lib/system/libsystem_coreservices.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
03 Feb 2016 02:30:49	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	1,036,800	1,079,272	1.0410
02 Feb 2016 19:48:44	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	1,010,880	1,053,700	1.0424
02 Feb 2016 02:15:31	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	984,960	1,027,947	1.0436
01 Feb 2016 12:02:36	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	959,040	1,001,344	1.0441
01 Feb 2016 00:11:22	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	933,120	974,750	1.0446
30 Jan 2016 20:29:00	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	907,200	948,049	1.0450
29 Jan 2016 04:20:33	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	881,280	921,288	1.0454
28 Jan 2016 20:07:07	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	855,360	893,783	1.0449
28 Jan 2016 06:17:10	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	829,440	866,936	1.0452
27 Jan 2016 22:33:42	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	803,520	841,019	1.0467
26 Jan 2016 16:02:31	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	777,600	814,416	1.0473
25 Jan 2016 03:16:22	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	751,680	788,279	1.0487
24 Jan 2016 07:12:31	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	725,760	760,195	1.0474
22 Jan 2016 15:04:50	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	699,840	731,750	1.0456
20 Dec 2015 04:15:12	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	673,920	705,917	1.0475
19 Dec 2015 20:54:20	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	648,000	680,247	1.0498
19 Dec 2015 12:02:44	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	622,080	653,471	1.0505
18 Dec 2015 20:35:10	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	596,160	628,091	1.0536
18 Dec 2015 13:03:50	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	570,240	602,536	1.0566
18 Dec 2015 01:28:37	1381327	19135363	hadcm3n_scrf_198012_480_010222290_2	544,320	577,319	1.0606