climateprediction.net home page
Task 13537751

Task 13537751

Name hadcm3n_yho0_1900_40_007516040_1
Workunit 7713515
Created 28 Oct 2011, 12:45:05 UTC
Sent 23 Nov 2011, 6:14:13 UTC
Report deadline 22 Feb 2012, 13:41:24 UTC
Received 11 Dec 2011, 10:31:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1181044
Run time 8 days 8 hours 49 min 52 sec
CPU time 7 days 5 hours 0 min 43 sec
Validate state Invalid
Credit 2,799.36
Device peak FLOPS 2.13 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
18:53:00 (1658): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:55:57 (25610): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:59:14 (25689): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:01:46 (25829): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:04:34 (25941): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:07:46 (26068): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:10:07 (26203): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:13:20 (26320): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:16:42 (26454): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:19:09 (26599): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:21:56 (26709): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:24:48 (26861): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:27:45 (27037): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:30:37 (27213): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:33:09 (27388): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:36:22 (27541): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:39:39 (27727): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:42:16 (27934): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:45:07 (28098): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:48:13 (28273): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:50:39 (28462): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:53:35 (28600): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:56:06 (28779): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
19:59:28 (28929): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:02:09 (29127): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:04:59 (29292): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:08:16 (29454): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:11:18 (29645): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:14:31 (29834): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:17:02 (30014): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:19:28 (30165): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:22:08 (30311): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:24:35 (30465): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:27:06 (30609): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:30:12 (30756): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:33:23 (30936): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:36:00 (31118): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:38:56 (31280): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:42:08 (31458): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:44:44 (31643): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:48:04 (31803): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:50:47 (31990): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:53:57 (32146): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:56:49 (32338): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:59:35 (32499): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:02:36 (32663): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:05:28 (380): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:08:28 (547): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
21:11:00 (731): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
09:49:35 (2121): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
23:02:16 (2087): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
07:32:00 (2255): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
07:32:31 (2255): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
14:24:27 (16939): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:25:52 (18461): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:27:44 (18528): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
14:30:11 (18610): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
20:31:20 (1652): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:31:21 (1652): No heartbeat from core client for 30 sec - exiting
20:31:22 (1652): No heartbeat from core client for 30 sec - exiting
20:31:23 (1652): No heartbeat from core client for 30 sec - exiting
20:31:24 (1652): No heartbeat from core client for 30 sec - exiting
20:31:25 (1652): No heartbeat from core client for 30 sec - exiting
20:31:26 (1652): No heartbeat from core client for 30 sec - exiting
20:31:27 (1652): No heartbeat from core client for 30 sec - exiting
20:31:28 (1652): No heartbeat from core client for 30 sec - exiting
20:31:29 (1652): No heartbeat from core client for 30 sec - exiting
20:31:30 (1652): No heartbeat from core client for 30 sec - exiting
20:31:31 (1652): No heartbeat from core client for 30 sec - exiting
20:31:32 (1652): No heartbeat from core client for 30 sec - exiting
20:31:33 (1652): No heartbeat from core client for 30 sec - exiting
20:31:34 (1652): No heartbeat from core client for 30 sec - exiting
20:31:35 (1652): No heartbeat from core client for 30 sec - exiting
20:31:36 (1652): No heartbeat from core client for 30 sec - exiting
20:31:37 (1652): No heartbeat from core client for 30 sec - exiting
20:31:38 (1652): No heartbeat from core client for 30 sec - exiting
20:31:39 (1652): No heartbeat from core client for 30 sec - exiting
20:31:40 (1652): No heartbeat from core client for 30 sec - exiting
20:31:41 (1652): No heartbeat from core client for 30 sec - exiting
20:31:42 (1652): No heartbeat from core client for 30 sec - exiting
20:31:43 (1652): No heartbeat from core client for 30 sec - exiting
20:31:44 (1652): No heartbeat from core client for 30 sec - exiting
20:31:45 (1652): No heartbeat from core client for 30 sec - exiting
20:31:46 (1652): No heartbeat from core client for 30 sec - exiting
23:16:20 (31145): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:16:22 (31145): No heartbeat from core client for 30 sec - exiting
23:16:23 (31145): No heartbeat from core client for 30 sec - exiting
23:16:24 (31145): No heartbeat from core client for 30 sec - exiting
23:16:25 (31145): No heartbeat from core client for 30 sec - exiting
23:16:26 (31145): No heartbeat from core client for 30 sec - exiting
23:16:27 (31145): No heartbeat from core client for 30 sec - exiting
23:16:28 (31145): No heartbeat from core client for 30 sec - exiting
23:16:29 (31145): No heartbeat from core client for 30 sec - exiting
23:16:30 (31145): No heartbeat from core client for 30 sec - exiting
23:16:31 (31145): No heartbeat from core client for 30 sec - exiting
23:16:32 (31145): No heartbeat from core client for 30 sec - exiting
23:16:33 (31145): No heartbeat from core client for 30 sec - exiting
23:16:34 (31145): No heartbeat from core client for 30 sec - exiting
23:16:35 (31145): No heartbeat from core client for 30 sec - exiting
23:16:36 (31145): No heartbeat from core client for 30 sec - exiting
23:16:37 (31145): No heartbeat from core client for 30 sec - exiting
23:16:38 (31145): No heartbeat from core client for 30 sec - exiting
23:16:39 (31145): No heartbeat from core client for 30 sec - exiting
23:16:40 (31145): No heartbeat from core client for 30 sec - exiting
23:16:41 (31145): No heartbeat from core client for 30 sec - exiting
23:16:42 (31145): No heartbeat from core client for 30 sec - exiting
23:16:43 (31145): No heartbeat from core client for 30 sec - exiting
23:16:44 (31145): No heartbeat from core client for 30 sec - exiting
23:16:45 (31145): No heartbeat from core client for 30 sec - exiting
23:16:46 (31145): No heartbeat from core client for 30 sec - exiting
23:16:47 (31145): No heartbeat from core client for 30 sec - exiting
23:16:48 (31145): No heartbeat from core client for 30 sec - exiting
23:16:49 (31145): No heartbeat from core client for 30 sec - exiting
23:16:50 (31145): No heartbeat from core client for 30 sec - exiting
23:16:51 (31145): No heartbeat from core client for 30 sec - exiting
23:16:52 (31145): No heartbeat from core client for 30 sec - exiting
23:16:53 (31145): No heartbeat from core client for 30 sec - exiting
23:16:54 (31145): No heartbeat from core client for 30 sec - exiting
23:16:55 (31145): No heartbeat from core client for 30 sec - exiting
23:16:56 (31145): No heartbeat from core client for 30 sec - exiting
23:16:57 (31145): No heartbeat from core client for 30 sec - exiting
23:16:58 (31145): No heartbeat from core client for 30 sec - exiting
23:16:59 (31145): No heartbeat from core client for 30 sec - exiting
23:17:00 (31145): No heartbeat from core client for 30 sec - exiting
23:17:01 (31145): No heartbeat from core client for 30 sec - exiting
23:17:02 (31145): No heartbeat from core client for 30 sec - exiting
23:17:03 (31145): No heartbeat from core client for 30 sec - exiting
23:17:04 (31145): No heartbeat from core client for 30 sec - exiting
23:17:05 (31145): No heartbeat from core client for 30 sec - exiting
23:17:06 (31145): No heartbeat from core client for 30 sec - exiting
23:17:07 (31145): No heartbeat from core client for 30 sec - exiting
23:17:08 (31145): No heartbeat from core client for 30 sec - exiting
23:17:09 (31145): No heartbeat from core client for 30 sec - exiting
23:17:10 (31145): No heartbeat from core client for 30 sec - exiting
23:17:11 (31145): No heartbeat from core client for 30 sec - exiting
23:17:12 (31145): No heartbeat from core client for 30 sec - exiting
23:17:13 (31145): No heartbeat from core client for 30 sec - exiting
23:17:14 (31145): No heartbeat from core client for 30 sec - exiting
23:17:15 (31145): No heartbeat from core client for 30 sec - exiting
23:17:16 (31145): No heartbeat from core client for 30 sec - exiting
23:17:17 (31145): No heartbeat from core client for 30 sec - exiting
23:33:51 (6345): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Atmos Hold Restart file rename failed on atmos_restart.hold
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
23:39:30 (2170): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:40:19 (2170): No heartbeat from core client for 30 sec - exiting
23:40:20 (2170): No heartbeat from core client for 30 sec - exiting
23:40:21 (2170): No heartbeat from core client for 30 sec - exiting
23:40:22 (2170): No heartbeat from core client for 30 sec - exiting
23:40:23 (2170): No heartbeat from core client for 30 sec - exiting
23:40:24 (2170): No heartbeat from core client for 30 sec - exiting
23:40:25 (2170): No heartbeat from core client for 30 sec - exiting
23:40:26 (2170): No heartbeat from core client for 30 sec - exiting
23:40:27 (2170): No heartbeat from core client for 30 sec - exiting
23:40:28 (2170): No heartbeat from core client for 30 sec - exiting
23:40:29 (2170): No heartbeat from core client for 30 sec - exiting
23:40:30 (2170): No heartbeat from core client for 30 sec - exiting
23:40:31 (2170): No heartbeat from core client for 30 sec - exiting
23:40:32 (2170): No heartbeat from core client for 30 sec - exiting
23:40:33 (2170): No heartbeat from core client for 30 sec - exiting
23:40:34 (2170): No heartbeat from core client for 30 sec - exiting
23:40:35 (2170): No heartbeat from core client for 30 sec - exiting
23:40:36 (2170): No heartbeat from core client for 30 sec - exiting
23:40:37 (2170): No heartbeat from core client for 30 sec - exiting
23:40:38 (2170): No heartbeat from core client for 30 sec - exiting
23:40:39 (2170): No heartbeat from core client for 30 sec - exiting
23:40:40 (2170): No heartbeat from core client for 30 sec - exiting
23:40:41 (2170): No heartbeat from core client for 30 sec - exiting
23:40:42 (2170): No heartbeat from core client for 30 sec - exiting
23:40:43 (2170): No heartbeat from core client for 30 sec - exiting
23:40:44 (2170): No heartbeat from core client for 30 sec - exiting
23:40:45 (2170): No heartbeat from core client for 30 sec - exiting
23:40:46 (2170): No heartbeat from core client for 30 sec - exiting
23:40:47 (2170): No heartbeat from core client for 30 sec - exiting
23:40:48 (2170): No heartbeat from core client for 30 sec - exiting
23:40:49 (2170): No heartbeat from core client for 30 sec - exiting
23:40:50 (2170): No heartbeat from core client for 30 sec - exiting
23:40:51 (2170): No heartbeat from core client for 30 sec - exiting
23:40:52 (2170): No heartbeat from core client for 30 sec - exiting
23:40:53 (2170): No heartbeat from core client for 30 sec - exiting
23:40:54 (2170): No heartbeat from core client for 30 sec - exiting
23:40:55 (2170): No heartbeat from core client for 30 sec - exiting
23:40:56 (2170): No heartbeat from core client for 30 sec - exiting
23:40:57 (2170): No heartbeat from core client for 30 sec - exiting
23:40:58 (2170): No heartbeat from core client for 30 sec - exiting
23:40:59 (2170): No heartbeat from core client for 30 sec - exiting
23:41:00 (2170): No heartbeat from core client for 30 sec - exiting
23:41:01 (2170): No heartbeat from core client for 30 sec - exiting
23:41:02 (2170): No heartbeat from core client for 30 sec - exiting
23:41:09 (2170): No heartbeat from core client for 30 sec - exiting
23:41:10 (2170): No heartbeat from core client for 30 sec - exiting
23:41:11 (2170): No heartbeat from core client for 30 sec - exiting
23:41:12 (2170): No heartbeat from core client for 30 sec - exiting
23:41:13 (2170): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
23:43:37 (2273): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
23:43:53 (2273): No heartbeat from core client for 30 sec - exiting
23:43:54 (2273): No heartbeat from core client for 30 sec - exiting
23:43:55 (2273): No heartbeat from core client for 30 sec - exiting
23:43:56 (2273): No heartbeat from core client for 30 sec - exiting
23:43:57 (2273): No heartbeat from core client for 30 sec - exiting
23:43:58 (2273): No heartbeat from core client for 30 sec - exiting
23:43:59 (2273): No heartbeat from core client for 30 sec - exiting
23:44:00 (2273): No heartbeat from core client for 30 sec - exiting
23:44:01 (2273): No heartbeat from core client for 30 sec - exiting
23:44:02 (2273): No heartbeat from core client for 30 sec - exiting
23:44:03 (2273): No heartbeat from core client for 30 sec - exiting
23:44:04 (2273): No heartbeat from core client for 30 sec - exiting
23:44:05 (2273): No heartbeat from core client for 30 sec - exiting
23:44:06 (2273): No heartbeat from core client for 30 sec - exiting
23:44:07 (2273): No heartbeat from core client for 30 sec - exiting
23:44:08 (2273): No heartbeat from core client for 30 sec - exiting
23:44:09 (2273): No heartbeat from core client for 30 sec - exiting
23:44:14 (2273): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
18:12:13 (1724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
18:12:15 (1724): No heartbeat from core client for 30 sec - exiting
18:12:16 (1724): No heartbeat from core client for 30 sec - exiting
18:12:17 (1724): No heartbeat from core client for 30 sec - exiting
18:12:18 (1724): No heartbeat from core client for 30 sec - exiting
18:12:19 (1724): No heartbeat from core client for 30 sec - exiting
18:12:20 (1724): No heartbeat from core client for 30 sec - exiting
18:12:21 (1724): No heartbeat from core client for 30 sec - exiting
18:12:22 (1724): No heartbeat from core client for 30 sec - exiting
18:12:23 (1724): No heartbeat from core client for 30 sec - exiting
18:12:24 (1724): No heartbeat from core client for 30 sec - exiting
18:12:25 (1724): No heartbeat from core client for 30 sec - exiting
18:12:26 (1724): No heartbeat from core client for 30 sec - exiting
18:12:27 (1724): No heartbeat from core client for 30 sec - exiting
18:12:28 (1724): No heartbeat from core client for 30 sec - exiting
18:12:29 (1724): No heartbeat from core client for 30 sec - exiting
18:12:30 (1724): No heartbeat from core client for 30 sec - exiting
18:12:31 (1724): No heartbeat from core client for 30 sec - exiting
18:12:32 (1724): No heartbeat from core client for 30 sec - exiting
18:12:33 (1724): No heartbeat from core client for 30 sec - exiting
18:12:34 (1724): No heartbeat from core client for 30 sec - exiting
18:12:35 (1724): No heartbeat from core client for 30 sec - exiting
18:12:36 (1724): No heartbeat from core client for 30 sec - exiting
18:12:37 (1724): No heartbeat from core client for 30 sec - exiting
18:12:38 (1724): No heartbeat from core client for 30 sec - exiting
18:12:39 (1724): No heartbeat from core client for 30 sec - exiting
18:12:40 (1724): No heartbeat from core client for 30 sec - exiting
18:12:41 (1724): No heartbeat from core client for 30 sec - exiting
18:12:42 (1724): No heartbeat from core client for 30 sec - exiting
18:12:43 (1724): No heartbeat from core client for 30 sec - exiting
18:12:44 (1724): No heartbeat from core client for 30 sec - exiting
18:12:45 (1724): No heartbeat from core client for 30 sec - exiting
18:12:46 (1724): No heartbeat from core client for 30 sec - exiting
18:12:47 (1724): No heartbeat from core client for 30 sec - exiting
18:12:48 (1724): No heartbeat from core client for 30 sec - exiting
18:12:49 (1724): No heartbeat from core client for 30 sec - exiting
18:12:50 (1724): No heartbeat from core client for 30 sec - exiting
18:12:51 (1724): No heartbeat from core client for 30 sec - exiting
18:12:52 (1724): No heartbeat from core client for 30 sec - exiting
18:12:53 (1724): No heartbeat from core client for 30 sec - exiting
18:12:54 (1724): No heartbeat from core client for 30 sec - exiting
18:12:55 (1724): No heartbeat from core client for 30 sec - exiting
18:12:56 (1724): No heartbeat from core client for 30 sec - exiting
18:12:57 (1724): No heartbeat from core client for 30 sec - exiting
18:12:58 (1724): No heartbeat from core client for 30 sec - exiting
18:12:59 (1724): No heartbeat from core client for 30 sec - exiting
18:13:00 (1724): No heartbeat from core client for 30 sec - exiting
18:13:01 (1724): No heartbeat from core client for 30 sec - exiting
18:13:02 (1724): No heartbeat from core client for 30 sec - exiting
18:13:03 (1724): No heartbeat from core client for 30 sec - exiting
18:13:04 (1724): No heartbeat from core client for 30 sec - exiting
18:13:05 (1724): No heartbeat from core client for 30 sec - exiting
18:13:06 (1724): No heartbeat from core client for 30 sec - exiting
18:13:07 (1724): No heartbeat from core client for 30 sec - exiting
18:13:08 (1724): No heartbeat from core client for 30 sec - exiting
18:13:09 (1724): No heartbeat from core client for 30 sec - exiting
18:13:10 (1724): No heartbeat from core client for 30 sec - exiting
18:13:12 (1724): No heartbeat from core client for 30 sec - exiting
18:13:13 (1724): No heartbeat from core client for 30 sec - exiting
18:13:14 (1724): No heartbeat from core client for 30 sec - exiting
18:13:15 (1724): No heartbeat from core client for 30 sec - exiting

BUFFOUT: Write Failed: No space left on device
BUFFOUT: C I/O Error - Return code = 1

Model crashed: WRITDUMP: BAD BUFFOUT OF DATA                                                                                                                                                                                                                                   tmp/pipe_dummy                                                                  2048    
18:23:26 (28360): No heartbeat from core client for 30 sec - exiting
18:23:27 (28360): No heartbeat from core client for 30 sec - exiting
18:23:28 (28360): No heartbeat from core client for 30 sec - exiting
18:23:29 (28360): No heartbeat from core client for 30 sec - exiting
18:23:30 (28360): No heartbeat from core client for 30 sec - exiting
18:23:31 (28360): No heartbeat from core client for 30 sec - exiting
18:23:32 (28360): No heartbeat from core client for 30 sec - exiting
18:23:33 (28360): No heartbeat from core client for 30 sec - exiting
18:23:34 (28360): No heartbeat from core client for 30 sec - exiting
18:23:35 (28360): No heartbeat from core client for 30 sec - exiting
18:23:36 (28360): No heartbeat from core client for 30 sec - exiting
18:23:37 (28360): No heartbeat from core client for 30 sec - exiting
18:23:38 (28360): No heartbeat from core client for 30 sec - exiting
18:23:39 (28360): No heartbeat from core client for 30 sec - exiting
18:23:40 (28360): No heartbeat from core client for 30 sec - exiting
18:23:41 (28360): No heartbeat from core client for 30 sec - exiting
18:23:42 (28360): No heartbeat from core client for 30 sec - exiting
18:23:43 (28360): No heartbeat from core client for 30 sec - exiting
18:23:44 (28360): No heartbeat from core client for 30 sec - exiting
18:23:45 (28360): No heartbeat from core client for 30 sec - exiting
18:23:46 (28360): No heartbeat from core client for 30 sec - exiting
18:23:47 (28360): No heartbeat from core client for 30 sec - exiting
18:23:48 (28360): No heartbeat from core client for 30 sec - exiting
18:23:49 (28360): No heartbeat from core client for 30 sec - exiting
18:23:50 (28360): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...

SETPOS: Seek Failed: No space left on device
SETPOS: Unit 42 to Word Address 14159872 Failed with Error Code -1

Model crashed: SETPOS: Unit 42 to Word Address 14159872 Failed with Error Code -1
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yho0_1900_40_007516040/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08451069  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0822DB4A  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818A767  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818D749  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F754A0F3  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28834, iMonCtr=1
Model crash detected, will try to restart...
forrtl: No space left on device
forrtl: severe (38): error during write, unit 6, file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_yho0_1900_40_007516040/dataout/stdout_um.txt
Image              PC        Routine            Line        Source             
hadcm3n_um_6.07_i  0848EB7D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0848D975  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0845F3CF  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F90D  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0841F257  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08451069  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0822DB4A  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818A767  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0818D749  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  08391957  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0838F8B7  Unknown               Unknown  Unknown
hadcm3n_um_6.07_i  0839BDF8  Unknown               Unknown  Unknown
libc.so.6          F75EA0F3  Unknown               Unknown  Unknown
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=28834, iMonCtr=1
Model crash detected, will try to restart...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
20:40:23 (2533): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
20:40:26 (2533): No heartbeat from core client for 30 sec - exiting
20:40:27 (2533): No heartbeat from core client for 30 sec - exiting
20:40:28 (2533): No heartbeat from core client for 30 sec - exiting
20:40:29 (2533): No heartbeat from core client for 30 sec - exiting
20:40:30 (2533): No heartbeat from core client for 30 sec - exiting
20:40:31 (2533): No heartbeat from core client for 30 sec - exiting
20:40:32 (2533): No heartbeat from core client for 30 sec - exiting
20:40:33 (2533): No heartbeat from core client for 30 sec - exiting
20:40:34 (2533): No heartbeat from core client for 30 sec - exiting
20:40:35 (2533): No heartbeat from core client for 30 sec - exiting
20:40:38 (2533): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf778e400]
[0xf778e430]
/lib32/libc.so.6(gsignal+0x4f)[0xf75ccc4f]
/lib32/libc.so.6(abort+0x175)[0xf75d0175]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf75b80f3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1286, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf774a400]
[0xf774a430]
/lib32/libc.so.6(gsignal+0x4f)[0xf7588c4f]
/lib32/libc.so.6(abort+0x175)[0xf758c175]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf75740f3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1286, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7710400]
[0xf7710430]
/lib32/libc.so.6(gsignal+0x4f)[0xf754ec4f]
/lib32/libc.so.6(abort+0x175)[0xf7552175]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf753a0f3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1286, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf779f400]
[0xf779f430]
/lib32/libc.so.6(gsignal+0x4f)[0xf75ddc4f]
/lib32/libc.so.6(abort+0x175)[0xf75e1175]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf75c90f3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1286, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf7730400]
[0xf7730430]
/lib32/libc.so.6(gsignal+0x4f)[0xf756ec4f]
/lib32/libc.so.6(abort+0x175)[0xf7572175]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf755a0f3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1286, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called
Stack trace (9 frames):
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f]
[0xf773b400]
[0xf773b430]
/lib32/libc.so.6(gsignal+0x4f)[0xf7579c4f]
/lib32/libc.so.6(abort+0x175)[0xf757d175]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83400c3]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f395]
/var/lib/boinc-client/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8]
/lib32/libc.so.6(__libc_start_main+0xf3)[0xf75650f3]

Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1286, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
10 Dec 2011 13:17:04 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 233,280 601,192 2.5771
09 Dec 2011 09:57:56 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 207,360 538,075 2.5949
07 Dec 2011 11:37:40 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 181,440 474,349 2.6144
05 Dec 2011 21:15:36 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 155,520 406,014 2.6107
03 Dec 2011 22:02:02 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 129,600 338,588 2.6126
01 Dec 2011 20:48:35 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 103,680 271,112 2.6149
27 Nov 2011 11:16:21 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 77,760 204,211 2.6262
25 Nov 2011 20:15:47 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 51,840 137,489 2.6522
24 Nov 2011 11:22:18 1181044 13537751 hadcm3n_yho0_1900_40_007516040_1 25,920 68,730 2.6516


©2024 climateprediction.net