climateprediction.net home page
Task 13610563

Task 13610563

Name hadcm3n_t0mk_1940_40_007539980_1
Workunit 7737212
Created 6 Nov 2011, 4:44:47 UTC
Sent 9 Nov 2011, 2:10:35 UTC
Report deadline 8 Feb 2012, 9:37:46 UTC
Received 9 Dec 2011, 14:12:14 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1172856
Run time 15 days 5 hours 33 min 41 sec
CPU time 13 days 13 hours 46 min 21 sec
Validate state Invalid
Credit 6,220.80
Device peak FLOPS 2.40 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>6.12.33</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 63 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 65 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 68 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 69 - Return code = 1
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
01:13:38 (1832): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
01:13:39 (1832): No heartbeat from core client for 30 sec - exiting
01:13:40 (1832): No heartbeat from core client for 30 sec - exiting
01:13:41 (1832): No heartbeat from core client for 30 sec - exiting
01:13:42 (1832): No heartbeat from core client for 30 sec - exiting
01:13:43 (1832): No heartbeat from core client for 30 sec - exiting
01:13:44 (1832): No heartbeat from core client for 30 sec - exiting
01:13:45 (1832): No heartbeat from core client for 30 sec - exiting
01:13:46 (1832): No heartbeat from core client for 30 sec - exiting
01:13:47 (1832): No heartbeat from core client for 30 sec - exiting
01:13:48 (1832): No heartbeat from core client for 30 sec - exiting
01:13:49 (1832): No heartbeat from core client for 30 sec - exiting
01:13:50 (1832): No heartbeat from core client for 30 sec - exiting
01:13:51 (1832): No heartbeat from core client for 30 sec - exiting
01:13:52 (1832): No heartbeat from core client for 30 sec - exiting
01:13:53 (1832): No heartbeat from core client for 30 sec - exiting
01:13:54 (1832): No heartbeat from core client for 30 sec - exiting
01:13:55 (1832): No heartbeat from core client for 30 sec - exiting
01:13:56 (1832): No heartbeat from core client for 30 sec - exiting
01:13:57 (1832): No heartbeat from core client for 30 sec - exiting
01:13:58 (1832): No heartbeat from core client for 30 sec - exiting
01:13:59 (1832): No heartbeat from core client for 30 sec - exiting
01:14:00 (1832): No heartbeat from core client for 30 sec - exiting
01:14:01 (1832): No heartbeat from core client for 30 sec - exiting
01:14:02 (1832): No heartbeat from core client for 30 sec - exiting
01:14:03 (1832): No heartbeat from core client for 30 sec - exiting
01:14:04 (1832): No heartbeat from core client for 30 sec - exiting
01:14:05 (1832): No heartbeat from core client for 30 sec - exiting
01:14:06 (1832): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
CPDN Monitor - Quit request from BOINC...
09:04:20 (1858): No heartbeat from core client for 30 sec - exiting
09:04:24 (1858): No heartbeat from core client for 30 sec - exiting
09:04:25 (1858): No heartbeat from core client for 30 sec - exiting
09:04:26 (1858): No heartbeat from core client for 30 sec - exiting
09:04:27 (1858): No heartbeat from core client for 30 sec - exiting
09:04:28 (1858): No heartbeat from core client for 30 sec - exiting
09:04:29 (1858): No heartbeat from core client for 30 sec - exiting
09:04:30 (1858): No heartbeat from core client for 30 sec - exiting
09:04:31 (1858): No heartbeat from core client for 30 sec - exiting
09:04:32 (1858): No heartbeat from core client for 30 sec - exiting
09:04:33 (1858): No heartbeat from core client for 30 sec - exiting
09:04:34 (1858): No heartbeat from core client for 30 sec - exiting
09:04:35 (1858): No heartbeat from core client for 30 sec - exiting
09:04:36 (1858): No heartbeat from core client for 30 sec - exiting
09:04:37 (1858): No heartbeat from core client for 30 sec - exiting
09:04:38 (1858): No heartbeat from core client for 30 sec - exiting
09:04:39 (1858): No heartbeat from core client for 30 sec - exiting
09:04:40 (1858): No heartbeat from core client for 30 sec - exiting
09:04:41 (1858): No heartbeat from core client for 30 sec - exiting
09:04:42 (1858): No heartbeat from core client for 30 sec - exiting
09:04:43 (1858): No heartbeat from core client for 30 sec - exiting
09:04:44 (1858): No heartbeat from core client for 30 sec - exiting
09:04:45 (1858): No heartbeat from core client for 30 sec - exiting
09:04:46 (1858): No heartbeat from core client for 30 sec - exiting
09:04:47 (1858): No heartbeat from core client for 30 sec - exiting
09:04:48 (1858): No heartbeat from core client for 30 sec - exiting
09:04:49 (1858): No heartbeat from core client for 30 sec - exiting
09:04:50 (1858): No heartbeat from core client for 30 sec - exiting
09:04:51 (1858): No heartbeat from core client for 30 sec - exiting
09:04:52 (1858): No heartbeat from core client for 30 sec - exiting
09:04:53 (1858): No heartbeat from core client for 30 sec - exiting
09:04:54 (1858): No heartbeat from core client for 30 sec - exiting
09:04:55 (1858): No heartbeat from core client for 30 sec - exiting
09:04:56 (1858): No heartbeat from core client for 30 sec - exiting
09:04:57 (1858): No heartbeat from core client for 30 sec - exiting
09:04:58 (1858): No heartbeat from core client for 30 sec - exiting
09:04:59 (1858): No heartbeat from core client for 30 sec - exiting
09:05:00 (1858): No heartbeat from core client for 30 sec - exiting
09:05:01 (1858): No heartbeat from core client for 30 sec - exiting
09:05:02 (1858): No heartbeat from core client for 30 sec - exiting
09:05:03 (1858): No heartbeat from core client for 30 sec - exiting
09:05:04 (1858): No heartbeat from core client for 30 sec - exiting
09:05:05 (1858): No heartbeat from core client for 30 sec - exiting
09:05:06 (1858): No heartbeat from core client for 30 sec - exiting
09:05:07 (1858): No heartbeat from core client for 30 sec - exiting
09:05:08 (1858): No heartbeat from core client for 30 sec - exiting
09:05:09 (1858): No heartbeat from core client for 30 sec - exiting
09:05:10 (1858): No heartbeat from core client for 30 sec - exiting
09:05:11 (1858): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
01:36:53 (1876): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
01:26:40 (8256): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:19:51 (6450): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
14:20:03 (6450): No heartbeat from core client for 30 sec - exiting
14:20:04 (6450): No heartbeat from core client for 30 sec - exiting
14:20:05 (6450): No heartbeat from core client for 30 sec - exiting
14:20:06 (6450): No heartbeat from core client for 30 sec - exiting
14:20:07 (6450): No heartbeat from core client for 30 sec - exiting
14:20:08 (6450): No heartbeat from core client for 30 sec - exiting
14:20:09 (6450): No heartbeat from core client for 30 sec - exiting
14:20:10 (6450): No heartbeat from core client for 30 sec - exiting
14:20:11 (6450): No heartbeat from core client for 30 sec - exiting
14:20:12 (6450): No heartbeat from core client for 30 sec - exiting
14:20:13 (6450): No heartbeat from core client for 30 sec - exiting
14:20:14 (6450): No heartbeat from core client for 30 sec - exiting
14:20:15 (6450): No heartbeat from core client for 30 sec - exiting
14:20:57 (7626): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/atmos_restart.day after 11 attempts
cpdnmonitor: cannot open input file /var/lib/boinc-client/projects/climateprediction.net/hadcm3n_t0mk_1940_40_007539980/dataout/ocean_restart.day after 11 attempts

Model crashed: READ_FLH: I/O error                                                                                                                                                                                                                                             tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish
14:59:17 (1978): No heartbeat from core client for 30 sec - exiting
14:59:18 (1978): No heartbeat from core client for 30 sec - exiting

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 Dec 2011 14:13:58 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 518,400 1,172,813 2.2624
08 Dec 2011 23:05:14 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 492,480 1,115,040 2.2641
08 Dec 2011 14:24:36 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 466,560 1,056,547 2.2645
07 Dec 2011 07:30:41 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 440,640 997,916 2.2647
06 Dec 2011 12:46:01 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 414,720 939,360 2.2650
05 Dec 2011 18:14:28 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 388,800 880,765 2.2653
05 Dec 2011 00:03:49 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 362,880 822,596 2.2669
04 Dec 2011 05:49:12 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 336,960 764,419 2.2686
03 Dec 2011 12:01:59 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 311,040 706,348 2.2709
02 Dec 2011 17:39:33 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 285,120 647,357 2.2705
01 Dec 2011 17:43:46 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 259,200 588,570 2.2707
01 Dec 2011 00:02:39 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 233,280 528,908 2.2673
30 Nov 2011 03:39:57 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 207,360 471,054 2.2717
29 Nov 2011 07:20:49 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 181,440 412,582 2.2739
28 Nov 2011 02:13:37 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 155,520 352,591 2.2672
25 Nov 2011 04:44:53 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 129,600 294,403 2.2716
24 Nov 2011 10:16:58 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 103,680 236,455 2.2806
15 Nov 2011 20:07:38 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 77,760 178,152 2.2910
15 Nov 2011 20:07:38 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 51,840 118,837 2.2924
15 Nov 2011 20:07:38 1172856 13610563 hadcm3n_t0mk_1940_40_007539980_1 25,920 59,707 2.3035


©2024 climateprediction.net