Task 19449874

Name	hadcm3n_ldfz_198012_480_373_010412384_1
Workunit	10412384
Created	23 Mar 2016, 10:21:12 UTC
Sent	23 Mar 2016, 10:50:42 UTC
Report deadline	5 Mar 2017, 16:10:42 UTC
Received	1 Apr 2016, 23:12:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1381208
Run time	5 days 16 hours 49 min 31 sec
CPU time	5 days 6 hours 26 min 4 sec
Validate state	Invalid
Credit	6,220.80
Device peak FLOPS	5.25 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-apple-darwin
Stderr	<core_client_version>7.6.12</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... forrtl: severe (17): syntax error in NAMELIST input, unit 5, file /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_ldfz_198012_480_373_010412384/jobs/climate.cpdc, line 393, position 19 Image PC Routine Line Source hadcm3n_um_6.07_i 0042E622 Unknown Unknown Unknown hadcm3n_um_6.07_i 0042D0EB Unknown Unknown Unknown hadcm3n_um_6.07_i 003EF4DD Unknown Unknown Unknown hadcm3n_um_6.07_i 003A66F5 Unknown Unknown Unknown hadcm3n_um_6.07_i 003A5E57 Unknown Unknown Unknown hadcm3n_um_6.07_i 003CACB0 Unknown Unknown Unknown hadcm3n_um_6.07_i 00261D1E Unknown Unknown Unknown hadcm3n_um_6.07_i 0016269A Unknown Unknown Unknown hadcm3n_um_6.07_i 00164453 Unknown Unknown Unknown hadcm3n_um_6.07_i 0038CE7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0038A1ED Unknown Unknown Unknown hadcm3n_um_6.07_i 000048E1 Unknown Unknown Unknown hadcm3n_um_6.07_i 000027FB Unknown Unknown Unknown hadcm3n_um_6.07_i 00002729 Unknown Unknown Unknown Unknown 00000003 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=32521, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... hadcm3n_6.07_i686-apple-darwin(39254,0xa39eb000) malloc: * error for object 0x4007604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(42775,0xa39eb000) malloc: * error for object 0x80b604: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(42775,0xa39eb000) malloc: * error for object 0x80b600: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(44369,0xa39eb000) malloc: * error for object 0x812c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(44369,0xa39eb000) malloc: * error for object 0x812c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(46095,0xa39eb000) malloc: * error for object 0x100ee04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(46095,0xa39eb000) malloc: * error for object 0x100ee00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation SIGSEGV: segmentation violation SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(47427,0xa39eb000) malloc: * error for object 0x1800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(47427,0xa39eb000) malloc: * error for object 0x1800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Fri Apr 1 11:32:41 2016 hadcm3n_6.07_i686-apple-darwin(47497,0xa39eb000) malloc: * error for object 0x2049404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(47497,0xa39eb000) malloc: * error for object 0x2049400: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(47896,0xa39eb000) malloc: * error for object 0x1015e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(47896,0xa39eb000) malloc: * error for object 0x1015e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation SIGSEGV: segmentation violation hadcm3n_6.07_i686-apple-darwin(52215,0xa39eb000) malloc: * error for object 0x3000e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Fri Apr 1 13:44:43 2016 hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x102d204: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x102d200: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x3001c04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x3001c00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x1800e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x1800e00: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x102c404: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug hadcm3n_6.07_i686-apple-darwin(56386,0xa39eb000) malloc: * error for object 0x4004e04: incorrect checksum for freed object - object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug SIGSEGV: segmentation violation Crashed executable name: hadcm3n_6.07_i686-apple-darwin built using BOINC library version 6.13.0 Machine type Intel x86-64h Haswell (32-bit executable) System version: Macintosh OS 10.11.4 build 15E65 Fri Apr 1 17:11:27 2016 Thread 0 Crashed: Thread 1: atos cannot load symbols for the file hadcm3n_6.07_i686-apple-darwin for architecture i386. Thread 0 crashed with X86 Thread State (32-bit): eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000 edi: 0x00000000 esi: 0x00000000 ebp: 0xbff8fce8 esp: 0x00000000 ss: 0x00000000 efl: 0x00000000 eip: 0x0007697e cs: 0x00000000 ds: 0x00000000 es: 0x00000000 fs: 0x00000000 gs: 0x00000000 Binary Images Description: 0x1000 - 0x93fff /Library/Application Support/BOINC Data/slots/9/../../projects/climateprediction.net/hadcm3n_6.07_i686-apple-darwin 0x127000 - 0x17afff /usr/lib/libstdc++.6.dylib 0x90011000 - 0x9001cfff /usr/lib/system/libcommonCrypto.dylib 0x906cc000 - 0x9073afff /usr/lib/system/libsystem_network.dylib 0x92a83000 - 0x92a8cfff /usr/lib/system/libsystem_networkextension.dylib 0x92b7c000 - 0x92b7dfff /usr/lib/system/libremovefile.dylib 0x92bd8000 - 0x92bfbfff /usr/lib/libc++abi.dylib 0x92bfd000 - 0x92bfffff /usr/lib/system/libsystem_configuration.dylib 0x93a7d000 - 0x93a9efff /usr/lib/system/libsystem_kernel.dylib 0x948d4000 - 0x948d5fff /usr/lib/libSystem.B.dylib 0x94d79000 - 0x94d7bfff /usr/lib/system/libsystem_coreservices.dylib 0x95767000 - 0x95769fff /usr/lib/system/libquarantine.dylib 0x95827000 - 0x95827fff /usr/lib/system/libunc.dylib 0x9584b000 - 0x95854fff /usr/lib/system/libcopyfile.dylib 0x95857000 - 0x9585ffff /usr/lib/system/libsystem_pthread.dylib 0x95e95000 - 0x95e95fff /usr/lib/system/liblaunch.dylib 0x9649f000 - 0x964b6fff /usr/lib/system/libsystem_coretls.dylib 0x96774000 - 0x96777fff /usr/lib/system/libdyld.dylib 0x96dc6000 - 0x96dcbfff /usr/lib/system/libcompiler_rt.dylib 0x96e66000 - 0x96e6efff /usr/lib/system/libsystem_dnssd.dylib 0x97c9c000 - 0x97d03fff /usr/lib/system/libcorecrypto.dylib 0x97de8000 - 0x97de8fff /usr/lib/system/libkeymgr.dylib 0x996a4000 - 0x996aafff /usr/lib/system/libunwind.dylib 0x996bf000 - 0x996c1fff /usr/lib/system/libsystem_secinit.dylib 0x996c2000 - 0x996c7fff /usr/lib/system/libmacho.dylib 0x997cd000 - 0x997f7fff /usr/lib/system/libdispatch.dylib 0x99848000 - 0x99870fff /usr/lib/system/libxpc.dylib 0x999d7000 - 0x999e5fff /usr/lib/libz.1.dylib 0x9a1b0000 - 0x9a1e3fff /usr/lib/system/libsystem_m.dylib 0x9a4a5000 - 0x9a4cffff /usr/lib/system/libsystem_info.dylib 0x9a83b000 - 0x9a84cfff /usr/lib/system/libsystem_trace.dylib 0x9a84d000 - 0x9a84dfff /usr/lib/system/libsystem_blocks.dylib 0x9c1db000 - 0x9c1f4fff /usr/lib/system/libsystem_malloc.dylib 0x9dc7f000 - 0x9dc89fff /usr/lib/system/libsystem_notify.dylib 0x9dd6d000 - 0x9dd70fff /usr/lib/system/libsystem_sandbox.dylib 0x9de17000 - 0x9de30fff /usr/lib/system/libsystem_asl.dylib 0x9ec19000 - 0x9ec1ffff /usr/lib/system/libsystem_platform.dylib 0x9ecc1000 - 0x9ecc5fff /usr/lib/system/libcache.dylib 0x9eced000 - 0x9ed82fff /usr/lib/system/libsystem_c.dylib Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Apr 2016 08:00:13	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	518,400	444,666	0.8578
31 Mar 2016 22:27:12	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	492,480	421,622	0.8561
31 Mar 2016 12:16:24	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	466,560	399,242	0.8557
30 Mar 2016 21:47:15	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	440,640	376,522	0.8545
30 Mar 2016 10:24:46	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	414,720	354,252	0.8542
29 Mar 2016 20:37:17	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	388,800	331,431	0.8524
29 Mar 2016 11:24:55	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	362,880	308,958	0.8514
29 Mar 2016 01:46:53	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	336,960	286,347	0.8498
28 Mar 2016 17:43:12	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	311,040	263,852	0.8483
28 Mar 2016 06:09:53	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	285,120	241,618	0.8474
27 Mar 2016 23:32:05	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	259,200	218,674	0.8436
27 Mar 2016 11:33:33	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	233,280	197,163	0.8452
27 Mar 2016 04:51:25	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	207,360	175,890	0.8482
26 Mar 2016 20:05:21	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	181,440	152,990	0.8432
26 Mar 2016 07:46:18	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	155,520	132,287	0.8506
25 Mar 2016 21:26:19	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	129,600	110,754	0.8546
25 Mar 2016 10:03:52	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	103,680	88,883	0.8573
24 Mar 2016 21:49:41	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	77,760	66,431	0.8543
24 Mar 2016 11:39:32	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	51,840	44,138	0.8514
23 Mar 2016 22:14:58	1381208	19449874	hadcm3n_ldfz_198012_480_373_010412384_1	25,920	21,836	0.8424