<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_p1mp_1900_40_007219697/dataout/stdout_um.txt
Image PC Routine Line Source
hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown
hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown
hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown
hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown
hadcm3n_um_6.07_i 08450E2C Unknown Unknown Unknown
hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0822D68F Unknown Unknown Unknown
hadcm3n_um_6.07_i 0818A767 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0818D749 Unknown Unknown Unknown
hadcm3n_um_6.07_i 08391957 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0838F8B7 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown
libc.so.6 F755CCE7 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21369, iMonCtr=1
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_p1mp_1900_40_007219697/dataout/stdout_um.txt
Image PC Routine Line Source
hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown
hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown
hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown
hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown
hadcm3n_um_6.07_i 08450E2C Unknown Unknown Unknown
hadcm3n_um_6.07_i 0844E937 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0822D68F Unknown Unknown Unknown
hadcm3n_um_6.07_i 0818A767 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0818D749 Unknown Unknown Unknown
hadcm3n_um_6.07_i 08391957 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0838F8B7 Unknown Unknown Unknown
hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown
libc.so.6 F761FCE7 Unknown Unknown Unknown
hadcm3Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=21369, iMonCtr=1
Model crash detected, will try to restart...
19:31:11 (21369): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:53:20 (29911): Can't acquire lockfile (-154) - waiting 35s
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
*** glibc detected *** ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu: double free or corruption (out): 0x08c07c20 ***
======= Backtrace: =========
/lib32/libc.so.6(+0x6c421)[0xf7514421]
/lib32/libc.so.6(+0x6dcb0)[0xf7515cb0]
/lib32/libc.so.6(cfree+0x6d)[0xf7518d5d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76ef7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76ef80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74bece7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
======= Memory map: ========
08048000-080e3000 r-xp 00000000 08:01 4546641 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e3000-080e4000 rw-p 0009b000 08:01 4546641 /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu
080e4000-0813b000 rw-p 00000000 00:00 0
08bb2000-08c18000 rw-p 00000000 00:00 0 [heap]
f6f00000-f6f21000 rw-p 00000000 00:00 0
f6f21000-f7000000 ---p 00000000 00:00 0
f702a000-f74a5000 rw-s 00000000 08:01 4194337 /var/lib/boinc-client/slots/0/137900
f74a5000-f74a8000 rw-p 00000000 00:00 0
f74a8000-f75fc000 r-xp 00000000 08:01 11927585 /lib32/libc-2.12.1.so
f75fc000-f75fd000 ---p 00154000 08:01 11927585 /lib32/libc-2.12.1.so
f75fd000-f75ff000 r--p 00154000 08:01 11927585 /lib32/libc-2.12.1.so
f75ff000-f7600000 rw-p 00156000 08:01 11927585 /lib32/libc-2.12.1.so
f7600000-f7603000 rw-p 00000000 00:00 0
f7603000-f761d000 r-xp 00000000 08:01 3072013 /usr/lib32/libgcc_s.so.1
f761d000-f761e000 r--p 00019000 08:01 3072013 /usr/lib32/libgcc_s.so.1
f761e000-f761f000 rw-p 0001a000 08:01 3072013 /usr/lib32/libgcc_s.so.1
f761f000-f7643000 r-xp 00000000 08:01 11927598 /lib32/libm-2.12.1.so
f7643000-f7644000 r--p 00023000 08:01 11927598 /lib32/libm-2.12.1.so
f7644000-f7645000 rw-p 00024000 08:01 11927598 /lib32/libm-2.12.1.so
f7645000-f7724000 r-xp 00000000 08:01 3072022 /usr/lib32/libstdc++.so.6.0.14
f7724000-f7728000 r--p 000de000 08:01 3072022 /usr/lib32/libstdc++.so.6.0.14
f7728000-f7729000 rw-p 000e2000 08:01 3072022 /usr/lib32/libstdc++.so.6.0.14
f7729000-f7730000 rw-p 00000000 00:00 0
f7730000-f7732000 r-xp 00000000 08:01 11927615 /lib32/libdl-2.12.1.so
f7732000-f7733000 r--p 00001000 08:01 11927615 /lib32/libdl-2.12.1.so
f7733000-f7734000 rw-p 00002000 08:01 11927615 /lib32/libdl-2.12.1.so
f7734000-f7735000 rw-p 00000000 00:00 0
f7735000-f774a000 r-xp 00000000 08:01 11927589 /lib32/libpthread-2.12.1.so
f774a000-f774b000 r--p 00014000 08:01 11927589 /lib32/libpthread-2.12.1.so
f774b000-f774c000 rw-p 00015000 08:01 11927589 /lib32/libpthread-2.12.1.so
f774c000-f774e000 rw-p 00000000 00:00 0
f7768000-f7769000 rw-p 00000000 00:00 0
f7769000-f776a000 ---p 00000000 00:00 0
f776a000-f776d000 rw-p 00000000 00:00 0
f776d000-f776f000 rw-s 00000000 08:01 4194334 /var/lib/boinc-client/slots/0/boinc_mmap_file
f776f000-f7771000 rw-p 00000000 00:00 0
f7771000-f7772000 r-xp 00000000 00:00 0 [vdso]
f7772000-f778e000 r-xp 00000000 08:01 11927599 /lib32/ld-2.12.1.so
f778e000-f778f000 r--p 0001b000 08:01 11927599 /lib32/ld-2.12.1.so
f778f000-f7790000 rw-p 0001c000 08:01 11927599 /lib32/ld-2.12.1.so
ff9a6000-ffa16000 rw-p 00000000 00:00 0 [stack]
SIGABRT: abort called
Stack trace (19 frames):
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df]
[0xf7771400]
[0xf7771430]
/lib32/libc.so.6(gsignal+0x51)[0xf74d2a01]
/lib32/libc.so.6(abort+0x182)[0xf74d5e42]
/lib32/libc.so.6(+0x62345)[0xf750a345]
/lib32/libc.so.6(+0x6c421)[0xf7514421]
/lib32/libc.so.6(+0x6dcb0)[0xf7515cb0]
/lib32/libc.so.6(cfree+0x6d)[0xf7518d5d]
/usr/lib32/libstdc++.so.6(_ZdlPv+0x21)[0xf76ef7b1]
/usr/lib32/libstdc++.so.6(_ZdaPv+0x1d)[0xf76ef80d]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8053e8e]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8057bc4]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x804f232]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x8050491]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a]
/lib32/libc.so.6(__libc_start_main+0xe7)[0xf74bece7]
../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51]
Exiting...
</stderr_txt>
]]>