Task 16072706

Name	hadcm3n_o23j_2020_40_008403728_2
Workunit	8554584
Created	23 Oct 2013, 3:35:57 UTC
Sent	23 Oct 2013, 3:36:00 UTC
Report deadline	22 Jan 2014, 11:03:11 UTC
Received	2 Dec 2013, 11:37:27 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	193 (0x000000C1) EXIT_SIGNAL
Computer ID	1239443
Run time	5 days 20 hours 27 min 51 sec
CPU time	4 days 14 hours 17 min 23 sec
Validate state	Invalid
Credit	3,110.40
Device peak FLOPS	2.67 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.45</core_client_version> <![CDATA[ <message> process exited with code 193 (0xc1, -63) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 15:06:57 (16527): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 63 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 64 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 65 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 66 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 67 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 68 - Return code = 1 BUFFIN: Read Failed: No such file or directory BUFFIN: C I/O Error feof - Unit 69 - Return code = 1 CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 10:35:58 (25606): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... 12:16:14 (28693): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... * glibc detected * ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: free(): invalid next size (normal): 0x083243b8 *** ======= Backtrace: ========= /lib/libc.so.6[0x464df1] /lib/libc.so.6[0x467531] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x2b1552] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x2b15ae] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x40ace6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] ======= Memory map: ======== 001e2000-001ff000 r-xp 00000000 fd:00 260205 /lib/libgcc_s-4.4.7-20120601.so.1 001ff000-00200000 rw-p 0001d000 fd:00 260205 /lib/libgcc_s-4.4.7-20120601.so.1 00202000-002e3000 r-xp 00000000 fd:00 20113 /usr/lib/libstdc++.so.6.0.13 002e3000-002e7000 r--p 000e0000 fd:00 20113 /usr/lib/libstdc++.so.6.0.13 002e7000-002e9000 rw-p 000e4000 fd:00 20113 /usr/lib/libstdc++.so.6.0.13 002e9000-002ef000 rw-p 00000000 00:00 0 003d2000-003f0000 r-xp 00000000 fd:00 261963 /lib/ld-2.12.so 003f0000-003f1000 r--p 0001d000 fd:00 261963 /lib/ld-2.12.so 003f1000-003f2000 rw-p 0001e000 fd:00 261963 /lib/ld-2.12.so 003f4000-00584000 r-xp 00000000 fd:00 262632 /lib/libc-2.12.so 00584000-00585000 ---p 00190000 fd:00 262632 /lib/libc-2.12.so 00585000-00587000 r--p 00190000 fd:00 262632 /lib/libc-2.12.so 00587000-00588000 rw-p 00192000 fd:00 262632 /lib/libc-2.12.so 00588000-0058b000 rw-p 00000000 00:00 0 0058d000-00590000 r-xp 00000000 fd:00 262731 /lib/libdl-2.12.so 00590000-00591000 r--p 00002000 fd:00 262731 /lib/libdl-2.12.so 00591000-00592000 rw-p 00003000 fd:00 262731 /lib/libdl-2.12.so 00594000-005bc000 r-xp 00000000 fd:00 262739 /lib/libm-2.12.so 005bc000-005bd000 r--p 00027000 fd:00 262739 /lib/libm-2.12.so 005bd000-005be000 rw-p 00028000 fd:00 262739 /lib/libm-2.12.so 005c0000-005d7000 r-xp 00000000 fd:00 261546 /lib/libpthread-2.12.so 005d7000-005d8000 r--p 00016000 fd:00 261546 /lib/libpthread-2.12.so 005d8000-005d9000 rw-p 00017000 fd:00 261546 /lib/libpthread-2.12.so 005d9000-005db000 rw-p 00000000 00:00 0 00cfb000-00cfc000 r-xp 00000000 00:00 0 [vdso] 08048000-080e3000 r-xp 00000000 fd:02 11011483 /home/boinc/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e3000-080e4000 rw-p 0009b000 fd:02 11011483 /home/boinc/boinc/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu 080e4000-0813b000 rw-p 00000000 00:00 0 08260000-08360000 rw-p 00000000 00:00 0 [heap] f7200000-f7221000 rw-p 00000000 00:00 0 f7221000-f7300000 ---p 00000000 00:00 0 f731d000-f7798000 rw-s 00000000 fd:02 11011481 /home/boinc/boinc/slots/2/133010 f7798000-f779b000 rw-p 00000000 00:00 0 f77af000-f77b0000 rw-p 00000000 00:00 0 f77b0000-f77b1000 ---p 00000000 00:00 0 f77b1000-f77b4000 rw-p 00000000 00:00 0 f77b4000-f77b6000 rw-s 00000000 fd:02 11011145 /home/boinc/boinc/slots/2/boinc_mmap_file f77b6000-f77b8000 rw-p 00000000 00:00 0 ff8f7000-ff968000 rw-p 00000000 00:00 0 [stack] SIGABRT: abort called Stack trace (18 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xcfb400] [0xcfb430] /lib/libc.so.6(gsignal+0x51)[0x41ead1] /lib/libc.so.6(abort+0x17a)[0x4203aa] /lib/libc.so.6[0x45e995] /lib/libc.so.6[0x464df1] /lib/libc.so.6[0x467531] /usr/lib/libstdc++.so.6(_ZdlPv+0x22)[0x2b1552] /usr/lib/libstdc++.so.6(_ZdaPv+0x1e)[0x2b15ae] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xe6)[0x40ace6] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
02 Dec 2013 11:39:41	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	259,200	397,039	1.5318
01 Dec 2013 12:54:05	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	233,280	357,257	1.5315
30 Nov 2013 16:18:59	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	207,360	317,376	1.5306
29 Nov 2013 16:39:50	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	181,440	277,379	1.5288
28 Nov 2013 18:45:26	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	155,520	237,433	1.5267
27 Nov 2013 13:49:48	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	129,600	197,559	1.5244
26 Nov 2013 15:46:02	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	103,680	157,768	1.5217
25 Nov 2013 22:27:49	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	77,760	118,443	1.5232
22 Nov 2013 09:06:18	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	51,840	78,716	1.5184
21 Nov 2013 10:53:13	1239443	16072706	hadcm3n_o23j_2020_40_008403728_2	25,920	39,110	1.5089