climateprediction.net home page
Task 12995358

Task 12995358

Name hadcm3n_o379_1940_40_007301663_0
Workunit 7499087
Created 22 Jun 2011, 14:24:54 UTC
Sent 22 Jun 2011, 14:25:03 UTC
Report deadline 21 Sep 2011, 21:52:14 UTC
Received 3 Jul 2011, 1:44:37 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1151942
Run time 7 days 22 hours 13 min 15 sec
CPU time 7 days 2 hours 38 min 1 sec
Validate state Invalid
Credit 3,110.40
Device peak FLOPS 2.24 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
10:20:11 (11032): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:20:12 (11032): No heartbeat from core client for 30 sec - exiting
10:20:13 (11032): No heartbeat from core client for 30 sec - exiting
10:20:14 (11032):10:31:48 (22431): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:32:55 (23619): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:32:56 (23619): No heartbeat from core client for 30 sec - exiting
04:14:05 (1338): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:50:42 (12874): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:50:43 (12874): No heartbeat from core client for 30 sec - exiting
04:50:44 (12874): No heartbeat from core client for 30 sec - exiting
04:50:45 (12874): No heartbeat from core client for 30 sec - exiting
04:50:46 (12874): No heartbeat from core client for 30 sec - exiting
04:50:47 (12874): No heartbeat from core client for 30 sec - exiting
04:50:48 (12874): No heartbeat from core client for 30 sec - exiting
04:50:49 (12874): No heartbeat from core client for 30 sec - exiting
04:50:50 (12874): No heartbeat from core client for 30 sec - exiting
04:50:51 (12874): No heartbeat from core client for 30 sec - exiting
04:50:52 (12874): No heartbeat from core client for 30 sec - exiting
05:32:53 (17123): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:52:44 (21060): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:55:13 (23351): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:56:52 (23975): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:38:40 (24057): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:39:57 (9879): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:39:58 (9879): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
04:02:20 (22631): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
06:50:43 (26428): No heartbeat from core client for 30 sec - exiting
06:50:45 (26428): No heartbeat from core client for 30 sec - exiting
06:50:46 (26428): No heartbeat from core client for 30 sec - exiting
06:50:47 (26428): No heartbeat from core client for 30 sec - exiting
06:50:48 (26428): No heartbeat from core client for 30 sec - exiting
06:50:49 (26428): No heartbeat from core client for 30 sec - exiting
06:50:50 (26428): No heartbeat from core client for 30 sec - exiting
06:50:51 (26428): No heartbeat from core client for 30 sec - exiting
06:50:52 (26428): No heartbeat from core client for 30 sec - exiting
06:50:53 (26428): No heartbeat from core client for 30 sec - exiting
06:50:54 (26428): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
06:14:08 (4365): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:40:56 (21462): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:40:57 (21462): No heartbeat from core client for 30 sec - exiting
07:40:58 (21462): No heartbeat from core client for 30 sec - exiting
07:40:59 (21462): No heartbeat from core client for 30 sec - exiting
07:41:00 (21462): No heartbeat from core client for 30 sec - exiting
07:41:01 (21462): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
02:00:44 (17458): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:46:13 (26073): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:29:46 (7099): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:59:20 (27564): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:31:00 (24826): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:46:56 (28752): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:46:57 (28752): No heartbeat from core client for 30 sec - exiting
04:46:58 (28752): No heartbeat from core client for 30 sec - exiting
04:46:59 (28752): No heartbeat from core client for 30 sec - exiting
04:47:00 (28752): No heartbeat from core client for 30 sec - exiting
04:47:01 (28752): No heartbeat from core client for 30 sec - exiting
04:47:02 (28752): No heartbeat from core client for 30 sec - exiting
04:47:03 (28752): No heartbeat from core client for 30 sec - exiting
04:47:04 (28752): No heartbeat from core client for 30 sec - exiting
04:47:05 (28752): No heartbeat from core client for 30 sec - exiting
04:47:06 (28752): No heartbeat from core client for 30 sec - exiting
04:47:07 (28752): No heartbeat from core client for 30 sec - exiting
04:47:08 (28752): No heartbeat from core client for 30 sec - exiting
04:47:09 (28752): No heartbeat from core client for 30 sec - exiting
04:47:10 (28752): No heartbeat from core client for 30 sec - exiting
04:47:11 (28752): No heartbeat from core client for 30 sec - exiting
04:47:12 (28752): No heartbeat from core client for 30 sec - exiting
04:47:13 (28752): No heartbeat from core client for 30 sec - exiting
04:47:14 (28752): No heartbeat from core client for 30 sec - exiting
04:47:15 (28752): No heartbeat from core client for 30 sec - exiting
04:47:16 (28752): No heartbeat from core client for 30 sec - exiting
04:47:17 (28752): No heartbeat from core client for 30 sec - exiting
04:47:18 (28752): No heartbeat from core client for 30 sec - exiting
04:47:19 (28752): No heartbeat from core client for 30 sec - exiting
04:47:20 (28752): No heartbeat from core client for 30 sec - exiting
04:47:21 (28752): No heartbeat from core client for 30 sec - exiting
04:47:22 (28752): No heartbeat from core client for 30 sec - exiting
04:47:23 (28752): No heartbeat from core client for 30 sec - exiting
04:47:24 (28752): No heartbeat from core client for 30 sec - exiting
04:47:25 (28752): No heartbeat from core client for 30 sec - exiting
04:47:26 (28752): No heartbeat from core client for 30 sec - exiting
04:47:27 (28752): No heartbeat from core client for 30 sec - exiting
04:47:28 (28752): No heartbeat from core client for 30 sec - exiting
04:47:29 (28752): No heartbeat from core client for 30 sec - exiting
04:47:30 (28752): No heartbeat from core client for 30 sec - exiting
04:48:06 (11985): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:27:53 (12040): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
11:32:19 (16532): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:46:52 (24774): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:29:18 (11123): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:30:36 (22473): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:30:37 (22473): No heartbeat from core client for 30 sec - exiting
06:30:38 (22473): No heartbeat from core client for 30 sec - exiting
06:30:39 (22473): No heartbeat from core client for 30 sec - exiting
06:30:40 (22473): No heartbeat from core client for 30 sec - exiting
06:30:41 (22473): No heartbeat from core client for 30 sec - exiting
06:30:42 (22473): No heartbeat from core client for 30 sec - exiting
06:30:43 (22473): No heartbeat from core client for 30 sec - exiting
06:30:44 (22473): No heartbeat from core client for 30 sec - exiting
06:30:45 (22473): No heartbeat from core client for 30 sec - exiting
06:30:46 (22473): No heartbeat from core client for 30 sec - exiting
06:30:47 (22473): No heartbeat from core client for 30 sec - exiting
06:30:48 (22473): No heartbeat from core client for 30 sec - exiting
06:30:49 (22473): No heartbeat from core client for 30 sec - exiting
03:35:05 (22830): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:43:44 (3068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x0829d7c8 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6c231)[0xf74d4231]
/lib32/libc.so.6(+0x6dab8)[0xf74d5ab8]
/lib32/libc.so.6(cfree+0x6d)[0xf74d8b9d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76c2a91]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76c2aed]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf747ebd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 fb:00 44331351                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 fb:00 44331351                           /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
08248000-082ad000 rw-p 00000000 00:00 0                                  [heap]
f6e00000-f6e21000 rw-p 00000000 00:00 0 
f6e21000-f6f00000 ---p 00000000 00:00 0 
f6feb000-f7466000 rw-s 00000000 fb:00 44389575                           /var/lib/boinc-client/slots/14/130480
f7466000-f7468000 rw-p 00000000 00:00 0 
f7468000-f75bb000 r-xp 00000000 fb:00 213647365                          /lib32/libc-2.11.1.so
f75bb000-f75bc000 ---p 00153000 fb:00 213647365                          /lib32/libc-2.11.1.so
f75bc000-f75be000 r--p 00153000 fb:00 213647365                          /lib32/libc-2.11.1.so
f75be000-f75bf000 rw-p 00155000 fb:00 213647365                          /lib32/libc-2.11.1.so
f75bf000-f75c2000 rw-p 00000000 00:00 0 
f75c2000-f75df000 r-xp 00000000 fb:00 109451375                          /usr/lib32/libgcc_s.so.1
f75df000-f75e0000 r--p 0001c000 fb:00 109451375                          /usr/lib32/libgcc_s.so.1
f75e0000-f75e1000 rw-p 0001d000 fb:00 109451375                          /usr/lib32/libgcc_s.so.1
f75e1000-f7605000 r-xp 00000000 fb:00 213647369                          /lib32/libm-2.11.1.so
f7605000-f7606000 r--p 00023000 fb:00 213647369                          /lib32/libm-2.11.1.so
f7606000-f7607000 rw-p 00024000 fb:00 213647369                          /lib32/libm-2.11.1.so
f7607000-f76f0000 r-xp 00000000 fb:00 109451378                          /usr/lib32/libstdc++.so.6.0.13
f76f0000-f76f1000 ---p 000e9000 fb:00 109451378                          /usr/lib32/libstdc++.so.6.0.13
f76f1000-f76f5000 r--p 000e9000 fb:00 109451378                          /usr/lib32/libstdc++.so.6.0.13
f76f5000-f76f6000 rw-p 000ed000 fb:00 109451378                          /usr/lib32/libstdc++.so.6.0.13
f76f6000-f76fd000 rw-p 00000000 00:00 0 
f76fd000-f76ff000 r-xp 00000000 fb:00 213647368                          /lib32/libdl-2.11.1.so
f76ff000-f7700000 r--p 00001000 fb:00 213647368                          /lib32/libdl-2.11.1.so
f7700000-f7701000 rw-p 00002000 fb:00 213647368                          /lib32/libdl-2.11.1.so
f7701000-f7702000 rw-p 00000000 00:00 0 
f7702000-f7717000 r-xp 00000000 fb:00 213647379                          /lib32/libpthread-2.11.1.so
f7717000-f7718000 r--p 00014000 fb:00 213647379                          /lib32/libpthread-2.11.1.so
f7718000-f7719000 rw-p 00015000 fb:00 213647379                          /lib32/libpthread-2.11.1.so
f7719000-f771b000 rw-p 00000000 00:00 0 
f7725000-f7726000 rw-p 00000000 00:00 0 
f7726000-f7727000 ---p 00000000 00:00 0 
f7727000-f772a000 rw-p 00000000 00:00 0 
f772a000-f772c000 rw-s 00000000 fb:00 44389393                           /var/lib/boinc-client/slots/14/boinc_mmap_file
f772c000-f772e000 rw-p 00000000 00:00 0 
f772e000-f772f000 r-xp 00000000 00:00 0                                  [vdso]
f772f000-f774b000 r-xp 00000000 fb:00 213647362                          /lib32/ld-2.11.1.so
f774b000-f774c000 r--p 0001b000 fb:00 213647362                          /lib32/ld-2.11.1.so
f774c000-f774d000 rw-p 0001c000 fb:00 213647362                          /lib32/ld-2.11.1.so
ffb4b000-ffbbc000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf772e400]
[0xf772e430]
/lib32/libc.so.6(gsignal+0x51)[0xf7492921]
/lib32/libc.so.6(abort+0x182)[0xf7495d52]
/lib32/libc.so.6(+0x6213d)[0xf74ca13d]
/lib32/libc.so.6(+0x6c231)[0xf74d4231]
/lib32/libc.so.6(+0x6dab8)[0xf74d5ab8]
/lib32/libc.so.6(cfree+0x6d)[0xf74d8b9d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76c2a91]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76c2aed]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf747ebd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
03 Jul 2011 01:47:23 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 259,200 614,267 2.3699
02 Jul 2011 08:05:41 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 233,280 554,411 2.3766
01 Jul 2011 11:44:52 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 207,360 493,886 2.3818
30 Jun 2011 17:36:54 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 181,440 433,701 2.3903
29 Jun 2011 21:45:19 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 155,520 372,800 2.3971
28 Jun 2011 04:05:24 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 129,600 310,519 2.3960
26 Jun 2011 17:05:16 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 103,680 248,123 2.3932
25 Jun 2011 00:59:16 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 77,760 186,191 2.3944
24 Jun 2011 05:44:52 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 51,840 122,833 2.3695
23 Jun 2011 10:30:24 1151942 12995358 hadcm3n_o379_1940_40_007301663_0 25,920 60,920 2.3503


©2024 climateprediction.net