Task 15896988

Name	hadcm3n_zkr6_1960_40_008393702_3
Workunit	8544561
Created	19 Jul 2013, 2:24:34 UTC
Sent	19 Jul 2013, 2:25:42 UTC
Report deadline	18 Oct 2013, 9:52:53 UTC
Received	17 Aug 2013, 11:22:08 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1239443
Run time	5 days 19 hours 47 min 47 sec
CPU time	4 days 13 hours 19 min 55 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	1.92 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.45</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> 20:38:30 (21876): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08b5f008 *** ======= Backtrace: ========= /lib/libc.so.6(+0x70e31)[0x327e31] /lib/libc.so.6(+0x73571)[0x32a571] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x279552] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x2795ae] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x2cdce6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 001ca000-002ab000 r-xp 00000000 fd:00 32021 /usr/lib/libstdc++.so.6.0.13.#prelink#.3C5FcR (deleted) 002ab000-002af000 r--p 000e0000 fd:00 32021 /usr/lib/libstdc++.so.6.0.13.#prelink#.3C5FcR (deleted) 002af000-002b1000 rw-p 000e4000 fd:00 32021 /usr/lib/libstdc++.so.6.0.13.#prelink#.3C5FcR (deleted) 002b1000-002b7000 rw-p 00000000 00:00 0 002b7000-00447000 r-xp 00000000 fd:00 262212 /lib/libc-2.12.so (deleted) 00447000-00448000 ---p 00190000 fd:00 262212 /lib/libc-2.12.so (deleted) 00448000-0044a000 r--p 00190000 fd:00 262212 /lib/libc-2.12.so (deleted) 0044a000-0044b000 rw-p 00192000 fd:00 262212 /lib/libc-2.12.so (deleted) 0044b000-0044e000 rw-p 00000000 00:00 0 00466000-00484000 r-xp 00000000 fd:00 262205 /lib/ld-2.12.so (deleted) 00484000-00485000 r--p 0001d000 fd:00 262205 /lib/ld-2.12.so (deleted) 00485000-00486000 rw-p 0001e000 fd:00 262205 /lib/ld-2.12.so (deleted) 0053a000-00557000 r-xp 00000000 fd:00 264947 /lib/libgcc_s-4.4.7-20120601.so.1.#prelink#.E6fMIo (deleted) 00557000-00558000 rw-p 0001d000 fd:00 264947 /lib/libgcc_s-4.4.7-20120601.so.1.#prelink#.E6fMIo (deleted) 006be000-006e6000 r-xp 00000000 fd:00 262220 /lib/libm-2.12.so (deleted) 006e6000-006e7000 r--p 00027000 fd:00 262220 /lib/libm-2.12.so (deleted) 006e7000-006e8000 rw-p 00028000 fd:00 262220 /lib/libm-2.12.so (deleted) 007e6000-007e9000 r-xp 00000000 fd:00 262218 /lib/libdl-2.12.so (deleted) 007e9000-007ea000 r--p 00002000 fd:00 262218 /lib/libdl-2.12.so (deleted) 007ea000-007eb000 rw-p 00003000 fd:00 262218 /lib/libdl-2.12.so (deleted) 00998000-009af000 r-xp 00000000 fd:00 262236 /lib/libpthread-2.12.so.#prelink#.W9h3E5 (deleted) 009af000-009b0000 r--p 00016000 fd:00 262236 /lib/libpthread-2.12.so.#prelink#.W9h3E5 (deleted) 009b0000-009b1000 rw-p 00017000 fd:00 262236 /lib/libpthread-2.12.so.#prelink#.W9h3E5 (deleted) 009b1000-009b3000 rw-p 00000000 00:00 0 009ce000-009cf000 r-xp 00000000 00:00 0 [vdso] 08048000-080e3000 r-xp 00000000 fd:02 11011483 /home/boinc/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 fd:02 11011483 /home/boinc/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08b09000-08b90000 rw-p 00000000 00:00 0 [heap] f7200000-f7221000 rw-p 00000000 00:00 0 f7221000-f7300000 ---p 00000000 00:00 0 f7325000-f77a0000 rw-s 00000000 fd:02 11010272 /home/boinc/boinc/slots/1/137025 f77a0000-f77a3000 rw-p 00000000 00:00 0 f77b7000-f77b8000 rw-p 00000000 00:00 0 f77b8000-f77b9000 ---p 00000000 00:00 0 f77b9000-f77bc000 rw-p 00000000 00:00 0 f77bc000-f77be000 rw-s 00000000 fd:02 11010341 /home/boinc/boinc/slots/1/boinc_mmap_file f77be000-f77c0000 rw-p 00000000 00:00 0 ffb9d000-ffc0e000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (18 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0x9ce400] [0x9ce430] /lib/libc.so.6(gsignal+0x51)[0x2e1b11] /lib/libc.so.6(abort+0x17a)[0x2e33ea] /lib/libc.so.6(+0x6a9d5)[0x3219d5] /lib/libc.so.6(+0x70e31)[0x327e31] /lib/libc.so.6(+0x73571)[0x32a571] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x279552] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x2795ae] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x2cdce6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
17 Aug 2013 11:25:06	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	259,200	393,593	1.5185
17 Aug 2013 11:25:07	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	233,280	354,741	1.5207
17 Aug 2013 11:25:06	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	207,360	315,527	1.5216
17 Aug 2013 11:25:06	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	181,440	275,894	1.5206
17 Aug 2013 11:25:07	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	155,520	236,507	1.5207
17 Aug 2013 11:25:06	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	129,600	196,959	1.5197
17 Aug 2013 11:25:07	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	103,680	157,346	1.5176
17 Aug 2013 11:25:07	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	77,760	117,734	1.5141
17 Aug 2013 11:25:07	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	51,840	78,528	1.5148
17 Aug 2013 11:25:07	1239443	15896988	hadcm3n_zkr6_1960_40_008393702_3	25,920	39,386	1.5195