climateprediction.net home page
Task 12824668

Task 12824668

Name hadcm3n_p3p0_1900_40_007222372_1
Workunit 7420612
Created 26 Apr 2011, 15:25:45 UTC
Sent 30 Apr 2011, 18:21:38 UTC
Report deadline 31 Jul 2011, 1:48:49 UTC
Received 5 May 2011, 20:22:36 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1026335
Run time 5 days 1 hours 18 min 21 sec
CPU time 4 days 23 hours 13 min 44 sec
Validate state Invalid
Credit 2,799.36
Device peak FLOPS 3.28 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
03:53:05 (8276): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:41:22 (8284): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:01:17 (8298): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:07:34 (8306): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:13:38 (8321): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:19:38 (8344): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:25:47 (8534): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
10:31:41 (8551): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:34:26 (8563): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
16:35:15 (8603): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
22:59:10 (8617): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:41:18 (8662): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
11:48:13 (8700): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:53:20 (8902): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
23:24:30 (8934): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
00:24:17 (8989): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:29:17 (9003): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
06:03:06 (9035): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
12:44:13 (9297): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:48:56 (9312): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
00:58:50 (9344): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:57:15 (9400): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
13:10:08 (9432): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:04:08 (9644): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
01:13:49 (9675): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:09:53 (9732): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
05:50:25 (9764): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:05:10 (9777): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:27:03 (9794): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:22:00 (12013): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
01:28:24 (12307): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09c88540 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6c231)[0xf7516231]
/lib32/libc.so.6(+0x6dab8)[0xf7517ab8]
/lib32/libc.so.6(cfree+0x6d)[0xf751ab9d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7704a91]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf7704aed]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf74c0bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:01 8136955                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:01 8136955                            /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09c33000-09c98000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f702d000-f74a8000 rw-s 00000000 08:01 8137335                            /var/lib/boinc-client/slots/0/132425
f74a8000-f74aa000 rw-p 00000000 00:00 0 
f74aa000-f75fd000 r-xp 00000000 08:01 5636172                            /lib32/libc-2.11.1.so
f75fd000-f75fe000 ---p 00153000 08:01 5636172                            /lib32/libc-2.11.1.so
f75fe000-f7600000 r--p 00153000 08:01 5636172                            /lib32/libc-2.11.1.so
f7600000-f7601000 rw-p 00155000 08:01 5636172                            /lib32/libc-2.11.1.so
f7601000-f7604000 rw-p 00000000 00:00 0 
f7604000-f7621000 r-xp 00000000 08:01 655963                             /usr/lib32/libgcc_s.so.1
f7621000-f7622000 r--p 0001c000 08:01 655963                             /usr/lib32/libgcc_s.so.1
f7622000-f7623000 rw-p 0001d000 08:01 655963                             /usr/lib32/libgcc_s.so.1
f7623000-f7647000 r-xp 00000000 08:01 5636178                            /lib32/libm-2.11.1.so
f7647000-f7648000 r--p 00023000 08:01 5636178                            /lib32/libm-2.11.1.so
f7648000-f7649000 rw-p 00024000 08:01 5636178                            /lib32/libm-2.11.1.so
f7649000-f7732000 r-xp 00000000 08:01 655966                             /usr/lib32/libstdc++.so.6.0.13
f7732000-f7733000 ---p 000e9000 08:01 655966                             /usr/lib32/libstdc++.so.6.0.13
f7733000-f7737000 r--p 000e9000 08:01 655966                             /usr/lib32/libstdc++.so.6.0.13
f7737000-f7738000 rw-p 000ed000 08:01 655966                             /usr/lib32/libstdc++.so.6.0.13
f7738000-f773f000 rw-p 00000000 00:00 0 
f773f000-f7741000 r-xp 00000000 08:01 5636176                            /lib32/libdl-2.11.1.so
f7741000-f7742000 r--p 00001000 08:01 5636176                            /lib32/libdl-2.11.1.so
f7742000-f7743000 rw-p 00002000 08:01 5636176                            /lib32/libdl-2.11.1.so
f7743000-f7744000 rw-p 00000000 00:00 0 
f7744000-f7759000 r-xp 00000000 08:01 5636192                            /lib32/libpthread-2.11.1.so
f7759000-f775a000 r--p 00014000 08:01 5636192                            /lib32/libpthread-2.11.1.so
f775a000-f775b000 rw-p 00015000 08:01 5636192                            /lib32/libpthread-2.11.1.so
f775b000-f775d000 rw-p 00000000 00:00 0 
f7772000-f7773000 rw-p 00000000 00:00 0 
f7773000-f7774000 ---p 00000000 00:00 0 
f7774000-f7777000 rw-p 00000000 00:00 0 
f7777000-f7779000 rw-s 00000000 08:01 8135061                            /var/lib/boinc-client/slots/0/boinc_mmap_file
f7779000-f777b000 rw-p 00000000 00:00 0 
f777b000-f777c000 r-xp 00000000 00:00 0                                  [vdso]
f777c000-f7798000 r-xp 00000000 08:01 5636137                            /lib32/ld-2.11.1.so
f7798000-f7799000 r--p 0001b000 08:01 5636137                            /lib32/ld-2.11.1.so
f7799000-f779a000 rw-p 0001c000 08:01 5636137                            /lib32/ld-2.11.1.so
ffc34000-ffca4000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf777b400]
[0xf777b430]
/lib32/libc.so.6(gsignal+0x51)[0xf74d4921]
/lib32/libc.so.6(abort+0x182)[0xf74d7d52]
/lib32/libc.so.6(+0x6213d)[0xf750c13d]
/lib32/libc.so.6(+0x6c231)[0xf7516231]
/lib32/libc.so.6(+0x6dab8)[0xf7517ab8]
/lib32/libc.so.6(cfree+0x6d)[0xf751ab9d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf7704a91]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf7704aed]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf74c0bd6]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
05 May 2011 08:17:40 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 233,280 386,436 1.6565
04 May 2011 20:05:34 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 207,360 343,849 1.6582
04 May 2011 08:04:58 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 181,440 301,043 1.6592
03 May 2011 19:55:16 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 155,520 258,142 1.6599
03 May 2011 07:48:30 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 129,600 215,174 1.6603
02 May 2011 19:25:43 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 103,680 171,864 1.6576
02 May 2011 06:53:07 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 77,760 127,425 1.6387
01 May 2011 18:40:23 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 51,840 84,218 1.6246
01 May 2011 07:34:39 1026335 12824668 hadcm3n_p3p0_1900_40_007222372_1 25,920 41,542 1.6027


©2024 climateprediction.net