climateprediction.net home page
Task 15700776

Task 15700776

Name hadcm3n_o4ah_2140_40_008279902_3
Workunit 8431037
Created 3 Apr 2013, 9:19:21 UTC
Sent 3 Apr 2013, 9:19:24 UTC
Report deadline 3 Jul 2013, 16:46:35 UTC
Received 16 Apr 2013, 22:29:18 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 1271464
Run time 13 days 9 hours 48 min 13 sec
CPU time 12 days 22 hours 59 min 33 sec
Validate state Invalid
Credit 9,331.20
Device peak FLOPS 3.57 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-pc-linux-gnu
Stderr
<core_client_version>7.0.29</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
03:14:07 (28210): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:16:43 (11985): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:20:27 (12006): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:20:28 (12006): No heartbeat from core client for 30 sec - exiting
03:20:29 (12006): No heartbeat from core client for 30 sec - exiting
03:20:30 (12006): No heartbeat from core client for 30 sec - exiting
03:20:31 (12006): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
03:13:50 (12469): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:13:51 (12469): No heartbeat from core client for 30 sec - exiting
03:13:52 (12469): No heartbeat from core client for 30 sec - exiting
03:13:53 (12469): No heartbeat from core client for 30 sec - exiting
03:13:54 (12469): No heartbeat from core client for 30 sec - exiting
03:13:55 (12469): No heartbeat from core client for 30 sec - exiting
03:16:58 (21102): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:16:59 (21102): No heartbeat from core client for 30 sec - exiting
03:17:00 (21102): No heartbeat from core client for 30 sec - exiting
03:17:01 (21102): No heartbeat from core client for 30 sec - exiting
03:17:02 (21102): No heartbeat from core client for 30 sec - exiting
Atmos Hold Restart file rename failed on atmos_restart.hold
03:16:19 (21559): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:18:16 (16451): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:18:17 (16451): No heartbeat from core client for 30 sec - exiting
Suspended CPDN Monitor - Suspend request from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
03:14:55 (16910): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:16:01 (27580): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:16:02 (27580): No heartbeat from core client for 30 sec - exiting
03:16:03 (27580): No heartbeat from core client for 30 sec - exiting
03:16:04 (27580): No heartbeat from core client for 30 sec - exiting
MainError:	04:20:07 AM	No files match the supplied pattern.
MainError:	04:20:07 AM	No files match the supplied pattern.
09:55:56 (28030): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
09:57:58 (21229): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
MainError:	03:04:07 PM	No files match the supplied pattern.
MainError:	03:04:07 PM	No files match the supplied pattern.
MainError:	01:43:40 AM	No files match the supplied pattern.
MainError:	01:43:40 AM	No files match the supplied pattern.
MainError:	12:23:24 AM	No files match the supplied pattern.
MainError:	12:23:24 AM	No files match the supplied pattern.
MainError:	11:28:31 PM	No files match the supplied pattern.
MainError:	11:28:31 PM	No files match the supplied pattern.
03:15:02 (21682): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:15:03 (21682): No heartbeat from core client for 30 sec - exiting
03:15:04 (21682): No heartbeat from core client for 30 sec - exiting
03:15:05 (21682): No heartbeat from core client for 30 sec - exiting
03:16:29 (25700): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
Suspended CPDN Monitor - Suspend request from BOINC...
MainError:	11:04:41 AM	No files match the supplied pattern.
MainError:	11:04:41 AM	No files match the supplied pattern.
MainError:	10:58:38 PM	No files match the supplied pattern.
MainError:	10:58:38 PM	No files match the supplied pattern.
03:14:33 (26150): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:14:39 (26150): No heartbeat from core client for 30 sec - exiting
03:14:40 (26150): No heartbeat from core client for 30 sec - exiting
03:14:41 (26150): No heartbeat from core client for 30 sec - exiting
03:15:15 (17341): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:15:19 (17341): No heartbeat from core client for 30 sec - exiting
03:20:33 (17787): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:20:34 (17787): No heartbeat from core client for 30 sec - exiting
MainError:	10:54:28 AM	No files match the supplied pattern.
MainError:	10:54:28 AM	No files match the supplied pattern.
MainError:	09:37:54 PM	No files match the supplied pattern.
MainError:	09:37:54 PM	No files match the supplied pattern.
03:13:54 (18240): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:13:55 (18240): No heartbeat from core client for 30 sec - exiting
03:13:56 (18240): No heartbeat from core client for 30 sec - exiting
03:13:57 (18240): No heartbeat from core client for 30 sec - exiting
03:13:58 (18240): No heartbeat from core client for 30 sec - exiting
03:13:59 (18240): No heartbeat from core client for 30 sec - exiting
03:14:00 (18240): No heartbeat from core client for 30 sec - exiting
03:16:04 (10378): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:21:00 (10843): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
03:21:01 (10843): No heartbeat from core client for 30 sec - exiting
03:21:02 (10843): No heartbeat from core client for 30 sec - exiting
03:21:03 (10843): No heartbeat from core client for 30 sec - exiting
03:21:04 (10843): No heartbeat from core client for 30 sec - exiting
03:21:05 (10843): No heartbeat from core client for 30 sec - exiting
MainError:	08:34:36 AM	No files match the supplied pattern.
MainError:	08:34:36 AM	No files match the supplied pattern.
Error converting file to netcdf: dataout/o4ahka.ph11c10
Error converting file to netcdf: dataout/o4ahka.pg11c10
Error converting file to netcdf: dataout/o4ahka.pe11c10
MainError:	07:28:44 PM	No files match the supplied pattern.
MainError:	07:28:44 PM	No files match the supplied pattern.

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: No such file or directory
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    

BUFFIN: Read Failed: Inappropriate ioctl for device
BUFFIN: C I/O Error feof - Unit 64 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 66 - Return code = 1

BUFFIN: Read Failed: Invalid argument
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

BUFFIN: Read Failed: Numerical result out of range
BUFFIN: C I/O Error feof - Unit 67 - Return code = 1

Model crashed: STWORK  : I/O error - PP fixed length header                                                                                                                                                                                                                    tmp/pipe_dummy                                                                  2048    
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
16 Apr 2013 19:30:32 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 777,600 1,148,163 1.4765
16 Apr 2013 08:39:39 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 751,680 1,109,070 1.4755
15 Apr 2013 22:42:38 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 725,760 1,070,422 1.4749
15 Apr 2013 14:01:48 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 699,840 1,031,956 1.4746
14 Apr 2013 23:17:21 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 673,920 989,713 1.4686
14 Apr 2013 11:37:30 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 648,000 947,037 1.4615
14 Apr 2013 00:03:23 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 622,080 905,936 1.4563
13 Apr 2013 13:14:54 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 596,160 866,177 1.4529
13 Apr 2013 01:46:10 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 570,240 827,930 1.4519
12 Apr 2013 15:05:43 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 544,320 789,705 1.4508
12 Apr 2013 04:22:51 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 518,400 751,803 1.4502
11 Apr 2013 17:47:27 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 492,480 713,979 1.4498
11 Apr 2013 07:13:32 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 466,560 676,003 1.4489
10 Apr 2013 19:53:14 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 440,640 635,904 1.4431
10 Apr 2013 08:33:06 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 414,720 595,208 1.4352
09 Apr 2013 21:18:27 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 388,800 554,938 1.4273
09 Apr 2013 10:35:38 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 362,880 516,564 1.4235
08 Apr 2013 23:36:34 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 336,960 477,513 1.4171
08 Apr 2013 12:33:28 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 311,040 437,950 1.4080
08 Apr 2013 01:09:13 1271464 15700776 hadcm3n_o4ah_2140_40_008279902_3 285,120 399,000 1.3994


©2024 climateprediction.net