climateprediction.net home page
Task 13416572

Task 13416572

Name hadcm3n_u3j8_1980_40_007460413_2
Workunit 7657916
Created 24 Sep 2011, 3:25:06 UTC
Sent 24 Sep 2011, 3:28:21 UTC
Report deadline 24 Dec 2011, 10:55:32 UTC
Received 15 Nov 2011, 17:33:47 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1151942
Run time 5 days 2 hours 50 min 12 sec
CPU time 4 days 12 hours 0 min 17 sec
Validate state Invalid
Credit 1,866.24
Device peak FLOPS 2.23 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr 
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
03:11:16 (24079): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:11:21 (24079): No heartbeat from core client for 30 sec - exiting
03:16:22 (5833): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:16:37 (5833): No heartbeat from core client for 30 sec - exiting
03:16:38 (5833): No heartbeat from core client for 30 sec - exiting
03:16:39 (5833): No heartbeat from core client for 30 sec - exiting
03:16:40 (5833): No heartbeat from core client for 30 sec - exiting
03:16:41 (5833): No heartbeat from core client for 30 sec - exiting
03:21:15 (5964): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:27:31 (6061): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:27:32 (6061): No heartbeat from core client for 30 sec - exiting

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
19:58:39 (832): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
00:34:59 (31208): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:59:55 (13488): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:16:18 (23978): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:17:18 (25353): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:47:19 (25433): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
02:47:20 (25433): No heartbeat from core client for 30 sec - exiting
02:47:21 (25433): No heartbeat from core client for 30 sec - exiting
02:47:22 (25433): No heartbeat from core client for 30 sec - exiting
02:47:23 (25433): No heartbeat from core client for 30 sec - exiting
02:47:24 (25433): No heartbeat from core client for 30 sec - exiting
02:47:25 (25433): No heartbeat from core client for 30 sec - exiting
03:02:34 (29114): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:45:32 (31134): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:45:33 (31134): No heartbeat from core client for 30 sec - exiting
03:45:34 (31134): No heartbeat from core client for 30 sec - exiting
03:45:35 (31134): No heartbeat from core client for 30 sec - exiting
03:45:36 (31134): No heartbeat from core client for 30 sec - exiting
03:45:37 (31134): No heartbeat from core client for 30 sec - exiting
03:45:38 (31134): No heartbeat from core client for 30 sec - exiting
03:45:39 (31134): No heartbeat from core client for 30 sec - exiting
03:45:40 (31134): No heartbeat from core client for 30 sec - exiting
03:45:41 (31134): No heartbeat from core client for 30 sec - exiting
03:45:42 (31134): No heartbeat from core client for 30 sec - exiting
03:46:33 (4013): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:46:34 (4013): No heartbeat from core client for 30 sec - exiting
06:05:10 (4622): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOU08:51:05 (21857): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
03:21:11 (20142): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:38:37 (30435): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:38:38 (30435): No heartbeat from core client for 30 sec - exiting
03:12:36 (20904): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:13:40 (8239): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:30:37 (8450): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:07:19 (11856): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:11:06 (7730): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:14:53 (8578): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:18:40 (9639): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:20:20 (9816): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:37:16 (10551): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:37:17 (10551): No heartbeat from core client for 30 sec - exiting
13:37:18 (10551): No heartbeat from core client for 30 sec - exiting
13:41:25 (14785): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:54:48 (15112): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
13:57:56 (18134): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:01:13 (18788): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:18:02 (19936): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:19:12 (22682): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:58:35 (23576): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:04:02 (32205): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:42:11 (32343): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
15:42:12 (32343): No heartbeat from core client for 30 sec - exiting
15:42:13 (32343): No heartbeat from core client for 30 sec - exiting
15:42:14 (32343): No heartbeat from core client for 30 sec - exiting
15:42:15 (32343): No heartbeat from core client for 30 sec - exiting
15:42:16 (32343): No heartbeat from core client for 30 sec - exiting
15:42:17 (32343): No heartbeat from core client for 30 sec - exiting
15:42:18 (32343): No heartbeat from core client for 30 sec - exiting
15:42:19 (32343): No heartbeat from core client for 30 sec - exiting
15:42:20 (32343): No heartbeat from core client for 30 sec - exiting
15:42:21 (32343): No heartbeat from core client for 30 sec - exiting
15:42:22 (32343): No heartbeat from core client for 30 sec - exiting
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf77bd400]
[0xf77bd430]
/lib32/libc.so.6(gsignal+0x51)[0xf7634921]
/lib32/libc.so.6(abort+0x182)[0xf7637d52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7620bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7526, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf772d400]
[0xf772d430]
/lib32/libc.so.6(gsignal+0x51)[0xf75a4921]
/lib32/libc.so.6(abort+0x182)[0xf75a7d52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf7590bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7526, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7746400]
[0xf7746430]
/lib32/libc.so.6(gsignal+0x51)[0xf75bd921]
/lib32/libc.so.6(abort+0x182)[0xf75c0d52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75a9bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7526, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7771400]
[0xf7771430]
/lib32/libc.so.6(gsignal+0x51)[0xf75e8921]
/lib32/libc.so.6(abort+0x182)[0xf75ebd52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75d4bd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7526, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7799400]
[0xf7799430]
/lib32/libc.so.6(gsignal+0x51)[0xf7610921]
/lib32/libc.so.6(abort+0x182)[0xf7613d52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf75fcbd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7526, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (10 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7739400]
[0xf7739430]
/lib32/libc.so.6(gsignal+0x51)[0xf75b0921]
/lib32/libc.so.6(abort+0x182)[0xf75b3d52]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xe6)[0xf759cbd6]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x804cb11]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=7526, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Nov 2011 18:07:48 1151942 13416572 hadcm3n_u3j8_1980_40_007460413_2 155,520 381,204 2.4512
15 Nov 2011 18:08:02 1151942 13416572 hadcm3n_u3j8_1980_40_007460413_2 129,600 317,222 2.4477
09 Nov 2011 15:23:17 1151942 13416572 hadcm3n_u3j8_1980_40_007460413_2 103,680 252,748 2.4378
26 Sep 2011 14:28:34 1151942 13416572 hadcm3n_u3j8_1980_40_007460413_2 77,760 197,940 2.5455
25 Sep 2011 18:59:31 1151942 13416572 hadcm3n_u3j8_1980_40_007460413_2 51,840 132,400 2.5540
25 Sep 2011 00:09:03 1151942 13416572 hadcm3n_u3j8_1980_40_007460413_2 25,920 66,168 2.5528


©2024 climateprediction.net