climateprediction.net home page
Task 14792949

Task 14792949

Name hadcm3n_o4tc_2100_40_008000634_0
Workunit 8155748
Created 15 Jun 2012, 21:03:58 UTC
Sent 15 Jun 2012, 21:04:10 UTC
Report deadline 15 Sep 2012, 4:31:21 UTC
Received 22 Jul 2012, 2:51:16 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 972238
Run time 15 days 15 hours 45 min 36 sec
CPU time 13 days 15 hours 44 min 25 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.23 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.27</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
12:44:13 (1791): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
00:22:32 (1661): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:37:59 (1771): No heartbeat from core client for 30 sec - exiting
21:38:01 (1771): No heartbeat from core client for 30 sec - exiting
21:38:02 (1771): No heartbeat from core client for 30 sec - exiting
21:38:03 (1771): No heartbeat from core client for 30 sec - exiting
21:38:04 (1771): No heartbeat from core client for 30 sec - exiting
21:38:05 (1771): No heartbeat from core client for 30 sec - exiting
21:38:06 (1771): No heartbeat from core client for 30 sec - exiting
21:38:07 (1771): No heartbeat from core client for 30 sec - exiting
21:38:08 (1771): No heartbeat from core client for 30 sec - exiting
21:38:09 (1771): No heartbeat from core client for 30 sec - exiting
21:38:10 (1771): No heartbeat from core client for 30 sec - exiting
21:38:11 (1771): No heartbeat from core client for 30 sec - exiting
21:38:12 (1771): No heartbeat from core client for 30 sec - exiting
21:38:43 (1771): No heartbeat from core client for 30 sec - exiting
21:38:46 (1771): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
11:38:18 (1743): No heartbeat from core client for 30 sec - exiting
11:38:20 (1743): No heartbeat from core client for 30 sec - exiting
11:38:21 (1743): No heartbeat from core client for 30 sec - exiting
11:38:22 (1743): No heartbeat from core client for 30 sec - exiting
11:38:23 (1743): No heartbeat from core client for 30 sec - exiting
11:38:24 (1743): No heartbeat from core client for 30 sec - exiting
11:38:25 (1743): No heartbeat from core client for 30 sec - exiting
11:38:26 (1743): No heartbeat from core client for 30 sec - exiting
11:38:27 (1743): No heartbeat from core client for 30 sec - exiting
11:38:28 (1743): No heartbeat from core client for 30 sec - exiting
11:38:29 (1743): No heartbeat from core client for 30 sec - exiting
11:38:30 (1743): No heartbeat from core client for 30 sec - exiting
11:38:31 (1743): No heartbeat from core client for 30 sec - exiting
11:38:32 (1743): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
06:27:26 (1738): No heartbeat from core client for 30 sec - exiting
06:27:27 (1738): No heartbeat from core client for 30 sec - exiting
06:27:28 (1738): No heartbeat from core client for 30 sec - exiting
06:27:29 (1738): No heartbeat from core client for 30 sec - exiting
06:27:30 (1738): No heartbeat from core client for 30 sec - exiting
06:27:31 (1738): No heartbeat from core client for 30 sec - exiting
06:27:32 (1738): No heartbeat from core client for 30 sec - exiting
06:27:33 (1738): No heartbeat from core client for 30 sec - exiting
06:27:34 (1738): No heartbeat from core client for 30 sec - exiting
06:27:35 (1738): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
22:29:11 (1945): No heartbeat from core client for 30 sec - exiting
22:29:12 (1945): No heartbeat from core client for 30 sec - exiting
22:29:13 (1945): No heartbeat from core client for 30 sec - exiting
22:29:14 (1945): No heartbeat from core client for 30 sec - exiting
22:29:15 (1945): No heartbeat from core client for 30 sec - exiting
22:29:16 (1945): No heartbeat from core client for 30 sec - exiting
22:29:17 (1945): No heartbeat from core client for 30 sec - exiting
22:29:18 (1945): No heartbeat from core client for 30 sec - exiting
22:29:19 (1945): No heartbeat from core client for 30 sec - exiting
22:29:20 (1945): No heartbeat from core client for 30 sec - exiting
22:29:21 (1945): No heartbeat from core client for 30 sec - exiting
22:29:22 (1945): No heartbeat from core client for 30 sec - exiting
22:29:23 (1945): No heartbeat from core client for 30 sec - exiting
22:29:24 (1945): No heartbeat from core client for 30 sec - exiting
22:29:25 (1945): No heartbeat from core client for 30 sec - exiting
22:29:26 (1945): No heartbeat from core client for 30 sec - exiting
22:29:27 (1945): No heartbeat from core client for 30 sec - exiting
22:29:28 (1945): No heartbeat from core client for 30 sec - exiting
22:29:29 (1945): No heartbeat from core client for 30 sec - exiting
22:29:30 (1945): No heartbeat from core client for 30 sec - exiting
22:29:31 (1945): No heartbeat from core client for 30 sec - exiting
22:29:32 (1945): No heartbeat from core client for 30 sec - exiting
22:29:33 (1945): No heartbeat from core client for 30 sec - exiting
22:29:34 (1945): No heartbeat from core client for 30 sec - exiting
22:29:35 (1945): No heartbeat from core client for 30 sec - exiting
22:29:36 (1945): No heartbeat from core client for 30 sec - exiting
22:29:37 (1945): No heartbeat from core client for 30 sec - exiting
22:29:38 (1945): No heartbeat from core client for 30 sec - exiting
22:29:39 (1945): No heartbeat from core client for 30 sec - exiting
22:29:40 (1945): No heartbeat from core client for 30 sec - exiting
22:29:41 (1945): No heartbeat from core client for 30 sec - exiting
22:29:42 (1945): No heartbeat from core client for 30 sec - exiting
22:29:43 (1945): No heartbeat from core client for 30 sec - exiting
22:29:44 (1945): No heartbeat from core client for 30 sec - exiting
22:29:45 (1945): No heartbeat from core client for 30 sec - exiting
22:29:46 (1945): No heartbeat from core client for 30 sec - exiting
22:29:47 (1945): No heartbeat from core client for 30 sec - exiting
22:29:48 (1945): No heartbeat from core client for 30 sec - exiting
22:29:49 (1945): No heartbeat from core client for 30 sec - exiting
22:29:50 (1945): No heartbeat from core client for 30 sec - exiting
22:29:51 (1945): No heartbeat from core client for 30 sec - exiting
22:30:22 (1945): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08210a58 ***
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x6ff22)[0xb74ebf22]
/lib/i386-linux-gnu/libc.so.6(+0x70bc2)[0xb74ecbc2]
/lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0xb74efcad]
/usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xb76ef80f]
/usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xb76ef86b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb7495113]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:05 8642597    /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:05 8642597    /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
081bb000-08220000 rw-p 00000000 00:00 0          [heap]
b6e00000-b6e21000 rw-p 00000000 00:00 0 
b6e21000-b6f00000 ---p 00000000 00:00 0 
b6ffe000-b7479000 rw-s 00000000 08:05 8652690    /var/lib/boinc-client/slots/3/134440
b7479000-b747c000 rw-p 00000000 00:00 0 
b747c000-b75f4000 r-xp 00000000 08:05 6250666    /lib/i386-linux-gnu/libc-2.13.so
b75f4000-b75f6000 r--p 00178000 08:05 6250666    /lib/i386-linux-gnu/libc-2.13.so
b75f6000-b75f7000 rw-p 0017a000 08:05 6250666    /lib/i386-linux-gnu/libc-2.13.so
b75f7000-b75fa000 rw-p 00000000 00:00 0 
b75fa000-b7616000 r-xp 00000000 08:05 6250516    /lib/i386-linux-gnu/libgcc_s.so.1
b7616000-b7617000 r--p 0001b000 08:05 6250516    /lib/i386-linux-gnu/libgcc_s.so.1
b7617000-b7618000 rw-p 0001c000 08:05 6250516    /lib/i386-linux-gnu/libgcc_s.so.1
b7618000-b7640000 r-xp 00000000 08:05 6250675    /lib/i386-linux-gnu/libm-2.13.so
b7640000-b7641000 r--p 00028000 08:05 6250675    /lib/i386-linux-gnu/libm-2.13.so
b7641000-b7642000 rw-p 00029000 08:05 6250675    /lib/i386-linux-gnu/libm-2.13.so
b7642000-b7720000 r-xp 00000000 08:05 934246     /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16
b7720000-b7721000 ---p 000de000 08:05 934246     /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16
b7721000-b7725000 r--p 000de000 08:05 934246     /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16
b7725000-b7726000 rw-p 000e2000 08:05 934246     /usr/lib/i386-linux-gnu/libstdc++.so.6.0.16
b7726000-b772e000 rw-p 00000000 00:00 0 
b772e000-b7731000 r-xp 00000000 08:05 6250674    /lib/i386-linux-gnu/libdl-2.13.so
b7731000-b7732000 r--p 00002000 08:05 6250674    /lib/i386-linux-gnu/libdl-2.13.so
b7732000-b7733000 rw-p 00003000 08:05 6250674    /lib/i386-linux-gnu/libdl-2.13.so
b7733000-b774a000 r-xp 00000000 08:05 6250701    /lib/i386-linux-gnu/libpthread-2.13.so
b774a000-b774b000 r--p 00016000 08:05 6250701    /lib/i386-linux-gnu/libpthread-2.13.so
b774b000-b774c000 rw-p 00017000 08:05 6250701    /lib/i386-linux-gnu/libpthread-2.13.so
b774c000-b774e000 rw-p 00000000 00:00 0 
b7763000-b7764000 rw-p 00000000 00:00 0 
b7764000-b7765000 ---p 00000000 00:00 0 
b7765000-b7768000 rw-p 00000000 00:00 0 
b7768000-b776a000 rw-s 00000000 08:05 8652688    /var/lib/boinc-client/slots/3/boinc_mmap_file
b776a000-b776c000 rw-p 00000000 00:00 0 
b776c000-b776d000 r-xp 00000000 00:00 0          [vdso]
b776d000-b778b000 r-xp 00000000 08:05 6250613    /lib/i386-linux-gnu/ld-2.13.so
b778b000-b778c000 r--p 0001d000 08:05 6250613    /lib/i386-linux-gnu/ld-2.13.so
b778c000-b778d000 rw-p 0001e000 08:05 6250613    /lib/i386-linux-gnu/ld-2.13.so
bfc15000-bfc85000 rw-p 00000000 00:00 0          [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xb776c400]
[0xb776c424]
/lib/i386-linux-gnu/libc.so.6(gsignal+0x4f)[0xb74a9c8f]
/lib/i386-linux-gnu/libc.so.6(abort+0x175)[0xb74ad2b5]
/lib/i386-linux-gnu/libc.so.6(+0x6515c)[0xb74e115c]
/lib/i386-linux-gnu/libc.so.6(+0x6ff22)[0xb74ebf22]
/lib/i386-linux-gnu/libc.so.6(+0x70bc2)[0xb74ecbc2]
/lib/i386-linux-gnu/libc.so.6(cfree+0x6d)[0xb74efcad]
/usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdlPv+0x1f)[0xb76ef80f]
/usr/lib/i386-linux-gnu/libstdc++.so.6(_ZdaPv+0x1b)[0xb76ef86b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0xb7495113]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
21 Jul 2012 09:11:23 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 518,400 1,179,861 2.2760
20 Jul 2012 13:42:58 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 492,480 1,119,034 2.2722
19 Jul 2012 17:33:55 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 466,560 1,058,317 2.2683
11 Jul 2012 19:22:06 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 440,640 997,638 2.2641
11 Jul 2012 00:16:41 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 414,720 936,790 2.2588
10 Jul 2012 05:09:41 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 388,800 875,969 2.2530
08 Jul 2012 23:33:08 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 362,880 816,867 2.2511
08 Jul 2012 05:14:20 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 336,960 757,623 2.2484
07 Jul 2012 05:09:35 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 311,040 698,922 2.2470
06 Jul 2012 00:02:37 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 285,120 640,505 2.2464
04 Jul 2012 12:22:34 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 259,200 583,570 2.2514
03 Jul 2012 18:11:10 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 233,280 527,708 2.2621
02 Jul 2012 22:16:05 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 207,360 467,565 2.2548
02 Jul 2012 10:55:56 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 181,440 409,279 2.2557
02 Jul 2012 10:55:56 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 155,520 350,335 2.2527
29 Jun 2012 16:15:08 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 129,600 291,429 2.2487
25 Jun 2012 00:09:12 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 103,680 232,715 2.2446
24 Jun 2012 03:24:10 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 77,760 174,462 2.2436
21 Jun 2012 10:28:03 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 51,840 116,049 2.2386
18 Jun 2012 07:58:54 972238 14792949 hadcm3n_o4tc_2100_40_008000634_0 25,920 57,477 2.2175


©2024 climateprediction.net