climateprediction.net home page
Task 21927690

Task 21927690

Name hadam4h_a0hw_209311_4_868_012015693_1
Workunit 12015693
Created 6 Apr 2020, 7:07:53 UTC
Sent 20 May 2020, 2:50:16 UTC
Report deadline 2 May 2021, 8:10:16 UTC
Received 31 May 2020, 9:14:52 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 193 (0x000000C1) EXIT_SIGNAL
Computer ID 1494978
Run time 4 days 20 hours 24 min 59 sec
CPU time 4 days 19 hours 59 min 51 sec
Validate state Invalid
Credit 20,375.94
Device peak FLOPS 7.40 GFLOPS
Application version UK Met Office HadAM4 at N216 resolution v8.52
i686-pc-linux-gnu
Peak working set size 1,373.04 MB
Peak swap size 1,394.22 MB
Peak disk usage 107.68 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 193 (0xc1, -63)</message>
<stderr_txt>
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
18:56:54 (3936): called boinc_finish(193)
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
18:56:55 (3936): called boinc_finish(193)
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fa4ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7aa0f62]
/usr/lib32/libc.so.6(+0xbb6cd)[0xf7b256cd]
/usr/lib32/libc.so.6(+0xbba7e)[0xf7b25a7e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7b239ab]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fa4ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fa4ed9]
/usr/lib32/libc.so.6(+0xc5c2d)[0xf7b2fc2d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b2fd02]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b36424]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b691b5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a88efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f5def0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a59f62]
/usr/lib32/libc.so.6(+0xbb6cd)[0xf7ade6cd]
/usr/lib32/libc.so.6(+0xbba7e)[0xf7adea7e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7adc9ab]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f5def0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f5ded9]
/usr/lib32/libc.so.6(+0xc5c2d)[0xf7ae8c2d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7ae8d02]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7aef424]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b221b5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a41efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f6bef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a67f62]
/usr/lib32/libc.so.6(+0xbb6cd)[0xf7aec6cd]
/usr/lib32/libc.so.6(+0xbba7e)[0xf7aeca7e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7aea9ab]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f6bef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f6bed9]
/usr/lib32/libc.so.6(+0xc5c2d)[0xf7af6c2d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7af6d02]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7afd424]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b301b5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a4fefe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f29ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a25f62]
/usr/lib32/libc.so.6(+0xbb6cd)[0xf7aaa6cd]
/usr/lib32/libc.so.6(+0xbba7e)[0xf7aaaa7e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7aa89ab]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f29ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f29ed9]
/usr/lib32/libc.so.6(+0xc5c2d)[0xf7ab4c2d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7ab4d02]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7abb424]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7aee1b5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a0defe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fbdef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7ab9f62]
/usr/lib32/libc.so.6(+0xbb6cd)[0xf7b3e6cd]
/usr/lib32/libc.so.6(+0xbba7e)[0xf7b3ea7e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7b3c9ab]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fbdef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fbded9]
/usr/lib32/libc.so.6(+0xc5c2d)[0xf7b48c2d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b48d02]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b4f424]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b821b5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7aa1efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (22 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f12ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a0ef62]
/usr/lib32/libc.so.6(+0xbb6cd)[0xf7a936cd]
/usr/lib32/libc.so.6(+0xbba7e)[0xf7a93a7e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7a919ab]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f12ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f12ed9]
/usr/lib32/libc.so.6(+0xc5c2d)[0xf7a9dc2d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7a9dd02]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7aa4424]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7ad71b5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf79f6efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
15:41:55 (13848): called boinc_finish(193)
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (22 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f10ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a0cf62]
/usr/lib32/libc.so.6(+0xbb6cd)[0xf7a916cd]
/usr/lib32/libc.so.6(+0xbba7e)[0xf7a91a7e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7a8f9ab]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f10ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f10ed9]
/usr/lib32/libc.so.6(+0xc5c2d)[0xf7a9bc2d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7a9bd02]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7aa2424]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7ad51b5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf79f4efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (22 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7eeeef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf79eaf72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7a6f6dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7a6fa8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7a6d9bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7eeeef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7eeeed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7a79c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7a79d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7a80434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7ab31c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf79d2efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (22 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f59ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a55f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7ada6dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7adaa8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7ad89bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f59ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f59ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7ae4c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7ae4d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7aeb434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b1e1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a3defe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Suspended CPDN Monitor - Suspend request from BOINC...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (22 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f89ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a85f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7b0a6dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7b0aa8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7b089bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f89ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f89ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7b14c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b14d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b1b434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b4e1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804fca4]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503d7]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a6defe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7edeef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf79daf72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7a5f6dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7a5fa8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7a5d9bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7edeef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7edeed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7a69c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7a69d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7a70434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7aa31c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf79c2efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f46ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a42f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7ac76dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7ac7a8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7ac59bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f46ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f46ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7ad1c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7ad1d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7ad8434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b0b1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a2aefe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
16:37:53 (2821): called boinc_finish(193)
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f75ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a71f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7af66dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7af6a8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7af49bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f75ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f75ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7b00c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b00d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b07434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b3a1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a59efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f57ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a53f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7ad86dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7ad8a8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7ad69bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f57ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f57ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7ae2c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7ae2d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7ae9434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b1c1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a3befe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f57ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a53f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7ad86dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7ad8a8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7ad69bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f57ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f57ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7ae2c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7ae2d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7ae9434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b1c1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a3befe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f2cef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a28f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7aad6dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7aada8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7aab9bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f2cef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f2ced9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7ab7c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7ab7d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7abe434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7af11c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a10efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
17:48:52 (3354): called boinc_finish(193)
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fc3ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7abff72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7b446dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7b44a8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7b429bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fc3ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fc3ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7b4ec3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b4ed12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b55434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b881c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7aa7efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f76ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a72f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7af76dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7af7a8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7af59bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f76ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f76ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7b01c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b01d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b08434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b3b1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a5aefe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fa9ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7aa5f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7b2a6dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7b2aa8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7b289bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fa9ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7fa9ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7b34c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b34d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b3b434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b6e1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a8defe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 15 received: Software termination signal from kill 
Signal 15 received: Abnormal termination triggered by abort call
Signal 15 received, exiting...
SIGSEGV: segmentation violation
Stack trace (21 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f85ef0]
/usr/lib32/libc.so.6(getenv+0xa2)[0xf7a81f72]
/usr/lib32/libc.so.6(+0xbb6dd)[0xf7b066dd]
/usr/lib32/libc.so.6(+0xbba8e)[0xf7b06a8e]
/usr/lib32/libc.so.6(localtime_r+0x1b)[0xf7b049bb]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d01b2]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d0900]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80d09f1]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7f85ef0]
linux-gate.so.1(__kernel_vsyscall+0x9)[0xf7f85ed9]
/usr/lib32/libc.so.6(+0xc5c3d)[0xf7b10c3d]
/usr/lib32/libc.so.6(clock_nanosleep+0x52)[0xf7b10d12]
/usr/lib32/libc.so.6(nanosleep+0x24)[0xf7b17434]
/usr/lib32/libc.so.6(usleep+0x45)[0xf7b4a1c5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80e78a5]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x80503e8]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051b13]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a69efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...
Signal 3 received, exiting...
12:13:01 (2792): called boinc_finish(193)
SIGSEGV: segmentation violation
Stack trace (7 frames):
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu(boinc_catch_signal+0x67)[0x80d4cf7]
linux-gate.so.1(__kernel_sigreturn+0x0)[0xf7fa4ef0]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804d0fe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8050d88]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x8051d8b]
/usr/lib32/libc.so.6(__libc_start_main+0x10e)[0xf7a88efe]
../../projects/climateprediction.net/hadam4_8.52_i686-pc-linux-gnu[0x804cd21]

Exiting...

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
28 May 2020 14:21:29 1494978 21927690 hadam4h_a0hw_209311_4_868_012015693_1 26,123 321,444 12.3050
26 May 2020 09:48:22 1494978 21927690 hadam4h_a0hw_209311_4_868_012015693_1 17,483 216,969 12.4103
23 May 2020 16:07:04 1494978 21927690 hadam4h_a0hw_209311_4_868_012015693_1 8,843 111,305 12.5868


©2024 climateprediction.net