climateprediction.net home page
Task 14368677

Task 14368677

Name hadcm3n_o0wn_2020_40_007857088_4
Workunit 8012200
Created 6 Apr 2012, 2:09:58 UTC
Sent 6 Apr 2012, 2:10:19 UTC
Report deadline 6 Jul 2012, 9:37:30 UTC
Received 26 Apr 2012, 1:36:40 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1186606
Run time 15 days 11 hours 2 min 49 sec
CPU time 12 days 9 hours 16 min 55 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.70 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
14:46:45 (30911): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
00:22:43 (28952): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:33:01 (12317): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:38:10 (19676): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:04:09 (26757): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
21:49:26 (18805): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Error: Input file: dataout/o0wnko.pjm8c10 is not a valid UM file.
Error converting file to netcdf: dataout/o0wnko.pjm8c10
Error: Input file: dataout/o0wnko.pim8c10 is not a valid UM file.
Error converting file to netcdf: dataout/o0wnko.pim8c10
Error: Input file: dataout/o0wnko.pfm8c10 is not a valid UM file.
Error converting file to netcdf: dataout/o0wnko.pfm8c10
Error: Input file: dataout/o0wnka.phm8c10 is not a valid UM file.
Error converting file to netcdf: dataout/o0wnka.phm8c10
Error: Input file: dataout/o0wnka.pgm8c10 is not a valid UM file.
Error converting file to netcdf: dataout/o0wnka.pgm8c10
Error: Input file: dataout/o0wnka.pem8c10 is not a valid UM file.
Error converting file to netcdf: dataout/o0wnka.pem8c10
Error: Input file: dataout/o0wnka.pdm8c10 is not a valid UM file.
Error converting file to netcdf: dataout/o0wnka.pdm8c10
13:34:41 (3003): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:04:57 (30492): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
17:09:10 (14277): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
14:41:19 (11273): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
17:33:24 (4659): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:47:05 (10124): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:47:19 (2262): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:04:19 (4442): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
17:40:32 (20495): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
22:57:21 (27025): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:37:16 (29016): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:02:58 (30720): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:33:42 (4412): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
03:08:47 (5630): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
03:38:40 (7061): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:09:06 (8215): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
04:44:01 (9393): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:14:18 (10751): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:44:00 (11866): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
05:49:03 (13007): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
06:29:58 (13295): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
08:40:18 (17406): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:01:42 (4741): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:17:49 (24749): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:12:10 (1936): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
08:31:08 (22155): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
08:31:09 (22155): No heartbeat from core client for 30 sec - exiting
08:31:10 (22155): No heartbeat from core client for 30 sec - exiting
08:31:11 (22155): No heartbeat from core client for 30 sec - exiting
08:31:12 (22155): No heartbeat from core client for 30 sec - exiting
08:31:13 (22155): No heartbeat from core client for 30 sec - exiting
08:31:14 (22155): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:26:06 (10776): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:28:14 (18314): No heartbeat from core client for 30 sec - exiting
16:28:15 (18314): No heartbeat from core client for 30 sec - exiting
16:28:16 (18314): No heartbeat from core client for 30 sec - exiting
16:28:17 (18314): No heartbeat from core client for 30 sec - exiting
16:28:18 (18314): No heartbeat from core client for 30 sec - exiting
16:28:19 (18314): No heartbeat from core client for 30 sec - exiting
16:28:20 (18314): No heartbeat from core client for 30 sec - exiting
16:28:21 (18314): No heartbeat from core client for 30 sec - exiting
16:28:22 (18314): No heartbeat from core client for 30 sec - exiting
16:28:23 (18314): No heartbeat from core client for 30 sec - exiting
16:28:24 (18314): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:25:25 (18927): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:11:17 (22888): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x09fc5ca8 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6ff02)[0xf7535f02]
/lib32/libc.so.6(+0x70ba2)[0xf7536ba2]
/lib32/libc.so.6(cfree+0x6d)[0xf7539c5d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf7735baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf7735c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf74df0f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 00:11 681896                             /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 00:11 681896                             /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0 
09f70000-09fd6000 rw-p 00000000 00:00 0                                  [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0 
f6f21000-f7000000 ---p 00000000 00:00 0 
f7048000-f74c3000 rw-s 00000000 00:11 847629                             /var/lib/boinc-client/slots/3/137945
f74c3000-f74c6000 rw-p 00000000 00:00 0 
f74c6000-f763a000 r-xp 00000000 00:11 752103                             /lib32/libc-2.13.so
f763a000-f763c000 r--p 00174000 00:11 752103                             /lib32/libc-2.13.so
f763c000-f763d000 rw-p 00176000 00:11 752103                             /lib32/libc-2.13.so
f763d000-f7640000 rw-p 00000000 00:00 0 
f7640000-f765c000 r-xp 00000000 00:11 393851                             /usr/lib32/libgcc_s.so.1
f765c000-f765d000 r--p 0001b000 00:11 393851                             /usr/lib32/libgcc_s.so.1
f765d000-f765e000 rw-p 0001c000 00:11 393851                             /usr/lib32/libgcc_s.so.1
f765e000-f7686000 r-xp 00000000 00:11 752107                             /lib32/libm-2.13.so
f7686000-f7687000 r--p 00028000 00:11 752107                             /lib32/libm-2.13.so
f7687000-f7688000 rw-p 00029000 00:11 752107                             /lib32/libm-2.13.so
f7688000-f7766000 r-xp 00000000 00:11 393881                             /usr/lib32/libstdc++.so.6.0.16
f7766000-f7767000 ---p 000de000 00:11 393881                             /usr/lib32/libstdc++.so.6.0.16
f7767000-f776b000 r--p 000de000 00:11 393881                             /usr/lib32/libstdc++.so.6.0.16
f776b000-f776c000 rw-p 000e2000 00:11 393881                             /usr/lib32/libstdc++.so.6.0.16
f776c000-f7774000 rw-p 00000000 00:00 0 
f7774000-f7777000 r-xp 00000000 00:11 752106                             /lib32/libdl-2.13.so
f7777000-f7778000 r--p 00002000 00:11 752106                             /lib32/libdl-2.13.so
f7778000-f7779000 rw-p 00003000 00:11 752106                             /lib32/libdl-2.13.so
f7779000-f7790000 r-xp 00000000 00:11 752117                             /lib32/libpthread-2.13.so
f7790000-f7791000 r--p 00016000 00:11 752117                             /lib32/libpthread-2.13.so
f7791000-f7792000 rw-p 00017000 00:11 752117                             /lib32/libpthread-2.13.so
f7792000-f7794000 rw-p 00000000 00:00 0 
f77ae000-f77af000 rw-p 00000000 00:00 0 
f77af000-f77b0000 ---p 00000000 00:00 0 
f77b0000-f77b3000 rw-p 00000000 00:00 0 
f77b3000-f77b5000 rw-s 00000000 00:11 847534                             /var/lib/boinc-client/slots/3/boinc_mmap_file
f77b5000-f77b7000 rw-p 00000000 00:00 0 
f77b7000-f77b8000 r-xp 00000000 00:00 0                                  [vdso]
f77b8000-f77d6000 r-xp 00000000 00:11 752948                             /lib/i386-linux-gnu/ld-2.13.so
f77d6000-f77d7000 r--p 0001d000 00:11 752948                             /lib/i386-linux-gnu/ld-2.13.so
f77d7000-f77d8000 rw-p 0001e000 00:11 752948                             /lib/i386-linux-gnu/ld-2.13.so
ffb34000-ffba4000 rw-p 00000000 00:00 0                                  [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf77b7400]
[0xf77b7425]
/lib32/libc.so.6(gsignal+0x4f)[0xf74f3c4f]
/lib32/libc.so.6(abort+0x175)[0xf74f7175]
/lib32/libc.so.6(+0x6521c)[0xf752b21c]
/lib32/libc.so.6(+0x6ff02)[0xf7535f02]
/lib32/libc.so.6(+0x70ba2)[0xf7536ba2]
/lib32/libc.so.6(cfree+0x6d)[0xf7539c5d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x1f)[0xf7735baf]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1b)[0xf7735c0b]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf74df0f3]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
26 Apr 2012 00:40:08 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 518,400 1,070,210 2.0644
24 Apr 2012 16:12:31 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 492,480 1,016,068 2.0632
23 Apr 2012 11:26:25 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 466,560 962,472 2.0629
22 Apr 2012 14:03:01 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 440,640 909,362 2.0637
21 Apr 2012 03:55:49 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 414,720 855,707 2.0633
20 Apr 2012 09:13:52 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 388,800 802,141 2.0631
19 Apr 2012 12:54:40 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 362,880 748,900 2.0638
18 Apr 2012 10:30:08 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 336,960 695,048 2.0627
17 Apr 2012 10:21:37 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 311,040 640,963 2.0607
16 Apr 2012 15:22:38 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 285,120 585,634 2.0540
15 Apr 2012 21:20:35 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 259,200 531,199 2.0494
14 Apr 2012 21:56:46 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 233,280 478,113 2.0495
13 Apr 2012 19:55:27 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 207,360 424,614 2.0477
13 Apr 2012 00:58:36 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 181,440 371,140 2.0455
12 Apr 2012 03:38:40 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 155,520 317,728 2.0430
11 Apr 2012 07:36:46 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 129,600 264,462 2.0406
10 Apr 2012 13:59:22 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 103,680 211,560 2.0405
09 Apr 2012 20:31:58 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 77,760 159,027 2.0451
09 Apr 2012 02:58:06 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 51,840 106,511 2.0546
07 Apr 2012 08:46:57 1186606 14368677 hadcm3n_o0wn_2020_40_007857088_4 25,920 53,064 2.0472


©2024 climateprediction.net