climateprediction.net home page
Task 16603732

Task 16603732

Name hadcm3n_8d4k_1980_40_008726431_1
Workunit 8872409
Created 29 Apr 2014, 10:19:52 UTC
Sent 29 Apr 2014, 10:21:05 UTC
Report deadline 29 Jul 2014, 17:48:16 UTC
Received 5 May 2014, 4:21:08 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1325999
Run time 5 days 15 hours 42 min 30 sec
CPU time 10 hours 23 min 42 sec
Validate state Invalid
Credit 4,043.52
Device peak FLOPS 3.02 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>7.2.33</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>

Model crashed: ATM_DYN : INVALID THETA DETECTED.                                                                                                                                                                                                                               tmp/pipe_dummy                                                                  2048    
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2135, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2135, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2135, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2135, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2135, iMonCtr=1
Model crash detected, will try to restart...
SIGSEGV: segmentation violation
Stack trace (13 frames):
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7]
/data1/pvh/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2135, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
04 May 2014 22:21:38 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 336,960 370,000 1.0981
04 May 2014 12:31:36 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 311,040 336,261 1.0811
04 May 2014 02:24:26 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 285,120 302,460 1.0608
03 May 2014 16:55:16 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 259,200 268,642 1.0364
03 May 2014 07:52:13 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 233,280 234,231 1.0041
02 May 2014 21:45:23 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 207,360 199,965 0.9643
02 May 2014 11:49:42 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 181,440 165,568 0.9125
02 May 2014 01:42:18 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 155,520 131,317 0.8444
01 May 2014 16:02:55 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 129,600 96,600 0.7454
01 May 2014 05:26:49 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 103,680 61,427 0.5925
30 Apr 2014 18:42:38 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 77,760 26,363 0.3390
30 Apr 2014 07:52:07 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 51,840 71,687 1.3829
29 Apr 2014 22:15:18 1325999 16603732 hadcm3n_8d4k_1980_40_008726431_1 25,920 35,987 1.3884


©2024 climateprediction.net