Task 15482080

Name	hadcm3n_zi1j_1960_40_008257215_0
Workunit	8412339
Created	17 Dec 2012, 0:05:42 UTC
Sent	17 Dec 2012, 0:05:47 UTC
Report deadline	18 Mar 2013, 7:32:58 UTC
Received	1 Feb 2013, 5:46:02 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1142687
Run time	15 days 4 hours 31 min 51 sec
CPU time	13 days 23 hours 40 min 48 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	0.96 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.17</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 02:14:59 (18330): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:15:42 (20739): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:16:43 (20761): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:17:32 (20779): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:18:30 (20799): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:19:16 (20815): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:20:39 (20843): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:21:39 (20862): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 02:22:46 (20888): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 05:24:47 (1667): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... 05:25:39 (7108): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08c4dd08 *** ======= Backtrace: ========= /lib32/libc.so.6(+0x6efe1)[0xf74a5fe1] /lib32/libc.so.6(+0x70868)[0xf74a7868] /lib32/libc.so.6(cfree+0x6d)[0xf74aa94d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7697a91] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf7697aed] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf744dbd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 08048000-080e3000 r-xp 00000000 08:01 9706373 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 08:01 9706373 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08bf8000-08c5e000 rw-p 00000000 00:00 0 [heap] f6e00000-f6e21000 rw-p 00000000 00:00 0 f6e21000-f6f00000 ---p 00000000 00:00 0 f6fba000-f7435000 rw-s 00000000 08:01 9723116 /var/lib/boinc-client/slots/3/136345 f7435000-f7437000 rw-p 00000000 00:00 0 f7437000-f7590000 r-xp 00000000 08:01 1463219 /lib32/libc-2.11.1.so f7590000-f7591000 ---p 00159000 08:01 1463219 /lib32/libc-2.11.1.so f7591000-f7593000 r--p 00159000 08:01 1463219 /lib32/libc-2.11.1.so f7593000-f7594000 rw-p 0015b000 08:01 1463219 /lib32/libc-2.11.1.so f7594000-f7597000 rw-p 00000000 00:00 0 f7597000-f75b4000 r-xp 00000000 08:01 16783532 /usr/lib32/libgcc_s.so.1 f75b4000-f75b5000 r--p 0001c000 08:01 16783532 /usr/lib32/libgcc_s.so.1 f75b5000-f75b6000 rw-p 0001d000 08:01 16783532 /usr/lib32/libgcc_s.so.1 f75b6000-f75da000 r-xp 00000000 08:01 1463202 /lib32/libm-2.11.1.so f75da000-f75db000 r--p 00023000 08:01 1463202 /lib32/libm-2.11.1.so f75db000-f75dc000 rw-p 00024000 08:01 1463202 /lib32/libm-2.11.1.so f75dc000-f76c5000 r-xp 00000000 08:01 16780942 /usr/lib32/libstdc++.so.6.0.13 f76c5000-f76c9000 r--p 000e9000 08:01 16780942 /usr/lib32/libstdc++.so.6.0.13 f76c9000-f76ca000 rw-p 000ed000 08:01 16780942 /usr/lib32/libstdc++.so.6.0.13 f76ca000-f76d1000 rw-p 00000000 00:00 0 f76d1000-f76d3000 r-xp 00000000 08:01 1463217 /lib32/libdl-2.11.1.so f76d3000-f76d4000 r--p 00001000 08:01 1463217 /lib32/libdl-2.11.1.so f76d4000-f76d5000 rw-p 00002000 08:01 1463217 /lib32/libdl-2.11.1.so f76d5000-f76d6000 rw-p 00000000 00:00 0 f76d6000-f76eb000 r-xp 00000000 08:01 1463213 /lib32/libpthread-2.11.1.so f76eb000-f76ec000 r--p 00014000 08:01 1463213 /lib32/libpthread-2.11.1.so f76ec000-f76ed000 rw-p 00015000 08:01 1463213 /lib32/libpthread-2.11.1.so f76ed000-f76ef000 rw-p 00000000 00:00 0 f770a000-f770b000 rw-p 00000000 00:00 0 f770b000-f770c000 ---p 00000000 00:00 0 f770c000-f770f000 rw-p 00000000 00:00 0 f770f000-f7711000 rw-s 00000000 08:01 9708601 /var/lib/boinc-client/slots/3/boinc_mmap_file f7711000-f7713000 rw-p 00000000 00:00 0 f7713000-f7714000 r-xp 00000000 00:00 0 [vdso] f7714000-f7730000 r-xp 00000000 08:01 1463210 /lib32/ld-2.11.1.so f7730000-f7731000 r--p 0001b000 08:01 1463210 /lib32/ld-2.11.1.so f7731000-f7732000 rw-p 0001c000 08:01 1463210 /lib32/ld-2.11.1.so ffec9000-fff39000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (19 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xf7713400] [0xf7713430] /lib32/libc.so.6(gsignal+0x51)[0xf7461951] /lib32/libc.so.6(abort+0x182)[0xf7464d82] /lib32/libc.so.6(+0x64eed)[0xf749beed] /lib32/libc.so.6(+0x6efe1)[0xf74a5fe1] /lib32/libc.so.6(+0x70868)[0xf74a7868] /lib32/libc.so.6(cfree+0x6d)[0xf74aa94d] /usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7697a91] /usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf7697aed] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib32/libc.so.6(__libc_start_main+0xe6)[0xf744dbd6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
01 Feb 2013 05:46:14	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	259,200	1,208,441	4.6622
28 Jan 2013 02:23:29	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	233,280	1,088,547	4.6663
23 Jan 2013 06:42:51	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	207,360	969,735	4.6766
19 Jan 2013 03:34:03	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	181,440	850,331	4.6866
14 Jan 2013 23:46:31	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	155,520	730,685	4.6983
10 Jan 2013 05:08:02	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	129,600	613,606	4.7346
06 Jan 2013 00:24:12	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	103,680	492,167	4.7470
01 Jan 2013 03:05:09	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	77,760	369,196	4.7479
27 Dec 2012 02:58:10	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	51,840	246,214	4.7495
21 Dec 2012 05:54:37	1142687	15482080	hadcm3n_zi1j_1960_40_008257215_0	25,920	123,098	4.7492