climateprediction.net home page
Task 13637458

Task 13637458

Name hadcm3n_u3bz_1980_40_007544066_2
Workunit 7741298
Created 16 Nov 2011, 1:59:51 UTC
Sent 16 Nov 2011, 2:20:42 UTC
Report deadline 15 Feb 2012, 9:47:53 UTC
Received 11 Dec 2011, 20:26:16 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1140527
Run time 9 days 21 hours 23 min 13 sec
CPU time 8 days 23 hours 47 min 56 sec
Validate state Invalid
Credit 4,976.64
Device peak FLOPS 2.82 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.58</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
16:55:13 (3937): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:55:16 (3937): No heartbeat from core client for 30 sec - exiting
16:55:17 (3937): No heartbeat from core client for 30 sec - exiting
16:55:18 (3937): No heartbeat from core client for 30 sec - exiting
16:55:19 (3937): No heartbeat from core client for 30 sec - exiting
16:55:20 (3937): No heartbeat from core client for 30 sec - exiting
16:55:21 (3937): No heartbeat from core client for 30 sec - exiting
16:55:22 (3937): No heartbeat from core client for 30 sec - exiting
16:55:23 (3937): No heartbeat from core client for 30 sec - exiting
16:55:24 (3937): No heartbeat from core client for 30 sec - exiting
16:55:25 (3937): No heartbeat from core client for 30 sec - exiting
16:55:26 (3937): No heartbeat from core client for 30 sec - exiting
16:55:27 (3937): No heartbeat from core client for 30 sec - exiting
16:55:28 (3937): No heartbeat from core client for 30 sec - exiting
16:55:29 (3937): No heartbeat from core client for 30 sec - exiting
16:55:30 (3937): No heartbeat from core client for 30 sec - exiting
16:55:31 (3937): No heartbeat from core client for 30 sec - exiting
16:55:32 (3937): No heartbeat from core client for 30 sec - exiting
16:55:33 (3937): No heartbeat from core client for 30 sec - exiting
16:55:34 (3937): No heartbeat from core client for 30 sec - exiting
16:55:35 (3937): No heartbeat from core client for 30 sec - exiting
16:55:36 (3937): No heartbeat from core client for 30 sec - exiting
16:55:37 (3937): No heartbeat from core client for 30 sec - exiting
16:55:38 (3937): No heartbeat from core client for 30 sec - exiting
16:55:39 (3937): No heartbeat from core client for 30 sec - exiting
16:55:40 (3937): No heartbeat from core client for 30 sec - exiting
16:55:41 (3937): No heartbeat from core client for 30 sec - exiting
16:55:42 (3937): No heartbeat from core client for 30 sec - exiting
16:55:43 (3937): No heartbeat from core client for 30 sec - exiting
16:55:44 (3937): No heartbeat from core client for 30 sec - exiting
16:55:45 (3937): No heartbeat from core client for 30 sec - exiting
16:55:46 (3937): No heartbeat from core client for 30 sec - exiting
16:55:47 (3937): No heartbeat from core client for 30 sec - exiting
16:55:48 (3937): No heartbeat from core client for 30 sec - exiting
16:55:49 (3937): No heartbeat from core client for 30 sec - exiting
16:55:50 (3937): No heartbeat from core client for 30 sec - exiting
16:55:51 (3937): No heartbeat from core client for 30 sec - exiting
16:55:52 (3937): No heartbeat from core client for 30 sec - exiting
16:55:53 (3937): No heartbeat from core client for 30 sec - exiting
16:55:54 (3937): No heartbeat from core client for 30 sec - exiting
16:55:55 (3937): No heartbeat from core client for 30 sec - exiting
16:55:56 (3937): No heartbeat from core client for 30 sec - exiting
16:55:57 (3937): No heartbeat from core client for 30 sec - exiting
16:55:58 (3937): No heartbeat from core client for 30 sec - exiting
16:55:59 (3937): No heartbeat from core client for 30 sec - exiting
16:56:00 (3937): No heartbeat from core client for 30 sec - exiting
16:56:01 (3937): No heartbeat from core client for 30 sec - exiting
16:56:02 (3937): No heartbeat from core client for 30 sec - exiting
16:56:03 (3937): No heartbeat from core client for 30 sec - exiting
16:56:04 (3937): No heartbeat from core client for 30 sec - exiting
16:56:05 (3937): No heartbeat from core client for 30 sec - exiting
16:56:06 (3937): No heartbeat from core client for 30 sec - exiting
16:56:07 (3937): No heartbeat from core client for 30 sec - exiting
16:56:08 (3937): No heartbeat from core client for 30 sec - exiting
16:56:09 (3937): No heartbeat from core client for 30 sec - exiting
16:56:10 (3937): No heartbeat from core client for 30 sec - exiting
16:56:11 (3937): No heartbeat from core client for 30 sec - exiting
16:56:12 (3937): No heartbeat from core client for 30 sec - exiting
16:56:13 (3937): No heartbeat from core client for 30 sec - exiting
16:56:14 (3937): No heartbeat from core client for 30 sec - exiting
16:56:15 (3937): No heartbeat from core client for 30 sec - exiting
16:56:16 (3937): No heartbeat from core client for 30 sec - exiting
16:56:17 (3937): No heartbeat from core client for 30 sec - exiting
16:56:18 (3937): No heartbeat from core client for 30 sec - exiting
16:56:19 (3937): No heartbeat from core client for 30 sec - exiting
16:56:20 (3937): No heartbeat from core client for 30 sec - exiting
16:56:21 (3937): No heartbeat from core client for 30 sec - exiting
16:56:22 (3937): No heartbeat from core client for 30 sec - exiting
16:56:23 (3937): No heartbeat from core client for 30 sec - exiting
16:56:24 (3937): No heartbeat from core client for 30 sec - exiting
16:56:25 (3937): No heartbeat from core client for 30 sec - exiting
16:56:26 (3937): No heartbeat from core client for 30 sec - exiting
16:56:27 (3937): No heartbeat from core client for 30 sec - exiting
16:56:28 (3937): No heartbeat from core client for 30 sec - exiting
16:56:29 (3937): No heartbeat from core client for 30 sec - exiting
16:56:30 (3937): No heartbeat from core client for 30 sec - exiting
16:56:31 (3937): No heartbeat from core client for 30 sec - exiting
16:56:32 (3937): No heartbeat from core client for 30 sec - exiting
16:56:33 (3937): No heartbeat from core client for 30 sec - exiting
16:56:34 (3937): No heartbeat from core client for 30 sec - exiting
16:56:35 (3937): No heartbeat from core client for 30 sec - exiting
16:56:36 (3937): No heartbeat from core client for 30 sec - exiting
16:56:37 (3937): No heartbeat from core client for 30 sec - exiting
16:56:38 (3937): No heartbeat from core client for 30 sec - exiting
16:56:39 (3937): No heartbeat from core client for 30 sec - exiting
16:56:40 (3937): No heartbeat from core client for 30 sec - exiting
16:56:41 (3937): No heartbeat from core client for 30 sec - exiting
16:56:42 (3937): No heartbeat from core client for 30 sec - exiting
16:56:43 (3937): No heartbeat from core client for 30 sec - exiting
16:56:44 (3937): No heartbeat from core client for 30 sec - exiting
16:56:45 (3937): No heartbeat from core client for 30 sec - exiting
16:56:46 (3937): No heartbeat from core client for 30 sec - exiting
16:56:47 (3937): No heartbeat from core client for 30 sec - exiting
16:56:48 (3937): No heartbeat from core client for 30 sec - exiting
16:56:49 (3937): No heartbeat from core client for 30 sec - exiting
16:56:50 (3937): No heartbeat from core client for 30 sec - exiting
16:56:51 (3937): No heartbeat from core client for 30 sec - exiting
16:56:52 (3937): No heartbeat from core client for 30 sec - exiting
16:56:53 (3937): No heartbeat from core client for 30 sec - exiting
16:56:54 (3937): No heartbeat from core client for 30 sec - exiting
16:56:55 (3937): No heartbeat from core client for 30 sec - exiting
16:56:56 (3937): No heartbeat from core client for 30 sec - exiting
16:56:57 (3937): No heartbeat from core client for 30 sec - exiting
16:56:58 (3937): No heartbeat from core client for 30 sec - exiting
16:56:59 (3937): No heartbeat from core client for 30 sec - exiting
16:57:00 (3937): No heartbeat from core client for 30 sec - exiting
16:57:01 (3937): No heartbeat from core client for 30 sec - exiting
16:57:02 (3937): No heartbeat from core client for 30 sec - exiting
16:57:03 (3937): No heartbeat from core client for 30 sec - exiting
16:57:04 (3937): No heartbeat from core client for 30 sec - exiting
16:57:05 (3937): No heartbeat from core client for 30 sec - exiting
16:57:06 (3937): No heartbeat from core client for 30 sec - exiting
16:57:07 (3937): No heartbeat from core client for 30 sec - exiting
16:57:08 (3937): No heartbeat from core client for 30 sec - exiting
16:57:09 (3937): No heartbeat from core client for 30 sec - exiting
16:57:10 (3937): No heartbeat from core client for 30 sec - exiting
16:58:24 (7791): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
16:58:25 (7791): No heartbeat from core client for 30 sec - exiting
16:58:26 (7791): No heartbeat from core client for 30 sec - exiting
16:58:27 (7791): No heartbeat from core client for 30 sec - exiting
17:01:31 (7925): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
17:01:32 (7925): No heartbeat from core client for 30 sec - exiting
17:01:33 (7925): No heartbeat from core client for 30 sec - exiting
17:01:34 (7925): No heartbeat from core client for 30 sec - exiting
20:25:43 (4351): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:25:45 (4351): No heartbeat from core client for 30 sec - exiting
20:25:46 (4351): No heartbeat from core client for 30 sec - exiting
20:25:47 (4351): No heartbeat from core client for 30 sec - exiting
20:25:48 (4351): No heartbeat from core client for 30 sec - exiting
20:25:49 (4351): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
20:27:08 (6664): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:27:09 (6664): No heartbeat from core client for 30 sec - exiting
20:27:10 (6664): No heartbeat from core client for 30 sec - exiting
20:27:11 (6664): No heartbeat from core client for 30 sec - exiting
20:27:12 (6664): No heartbeat from core client for 30 sec - exiting
20:27:13 (6664): No heartbeat from core client for 30 sec - exiting
20:27:14 (6664): No heartbeat from core client for 30 sec - exiting
20:27:15 (6664): No heartbeat from core client for 30 sec - exiting
20:27:16 (6664): No heartbeat from core client for 30 sec - exiting
20:27:17 (6664): No heartbeat from core client for 30 sec - exiting
20:27:18 (6664): No heartbeat from core client for 30 sec - exiting
20:27:19 (6664): No heartbeat from core client for 30 sec - exiting
20:27:20 (6664): No heartbeat from core client for 30 sec - exiting
20:27:21 (6664): No heartbeat from core client for 30 sec - exiting
20:27:22 (6664): No heartbeat from core client for 30 sec - exiting
20:27:23 (6664): No heartbeat from core client for 30 sec - exiting
20:27:24 (6664): No heartbeat from core client for 30 sec - exiting
20:27:25 (6664): No heartbeat from core client for 30 sec - exiting
20:27:26 (6664): No heartbeat from core client for 30 sec - exiting
20:27:27 (6664): No heartbeat from core client for 30 sec - exiting
20:27:28 (6664): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called
Stack trace (10 frames):
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf773b400]
[0xf773b430]
/lib32/libc.so.6(gsignal+0x51)[0xf75aa921]
/lib32/libc.so.6(abort+0x182)[0xf75add52]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7596bd6]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6861, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7795400]
[0xf7795430]
/lib32/libc.so.6(gsignal+0x51)[0xf7604921]
/lib32/libc.so.6(abort+0x182)[0xf7607d52]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75f0bd6]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6861, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77a1400]
[0xf77a1430]
/lib32/libc.so.6(gsignal+0x51)[0xf7610921]
/lib32/libc.so.6(abort+0x182)[0xf7613d52]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75fcbd6]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6861, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77a8400]
[0xf77a8430]
/lib32/libc.so.6(gsignal+0x51)[0xf7617921]
/lib32/libc.so.6(abort+0x182)[0xf761ad52]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7603bd6]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6861, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf777b400]
[0xf777b430]
/lib32/libc.so.6(gsignal+0x51)[0xf75ea921]
/lib32/libc.so.6(abort+0x182)[0xf75edd52]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75d6bd6]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6861, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf76e4400]
[0xf76e4430]
/lib32/libc.so.6(gsignal+0x51)[0xf7553921]
/lib32/libc.so.6(abort+0x182)[0xf7556d52]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf753fbd6]
/igel2/Vol1/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=6861, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
11 Dec 2011 09:26:56 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 414,720 740,953 1.7866
10 Dec 2011 17:13:03 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 388,800 682,768 1.7561
28 Nov 2011 07:25:10 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 362,880 630,041 1.7362
27 Nov 2011 14:07:07 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 336,960 582,112 1.7275
26 Nov 2011 22:52:43 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 311,040 533,007 1.7136
26 Nov 2011 04:23:31 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 285,120 482,761 1.6932
24 Nov 2011 13:32:56 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 259,200 434,022 1.6745
24 Nov 2011 02:55:05 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 233,280 397,852 1.7055
21 Nov 2011 15:55:23 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 207,360 359,253 1.7325
21 Nov 2011 03:47:15 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 181,440 317,777 1.7514
20 Nov 2011 16:04:28 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 155,520 273,424 1.7581
20 Nov 2011 03:41:43 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 129,600 230,768 1.7806
19 Nov 2011 15:40:34 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 103,680 186,702 1.8008
19 Nov 2011 02:57:10 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 77,760 142,161 1.8282
18 Nov 2011 05:18:15 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 51,840 95,910 1.8501
16 Nov 2011 18:31:40 1140527 13637458 hadcm3n_u3bz_1980_40_007544066_2 25,920 47,054 1.8154


©2024 climateprediction.net