climateprediction.net home page
Task 22251021

Task 22251021

Name oifs_43r3_ps_0166_1981050100_123_950_12166810_0
Workunit 12166810
Created 20 Dec 2022, 18:48:39 UTC
Sent 20 Dec 2022, 18:51:17 UTC
Report deadline 19 Jan 2023, 18:51:17 UTC
Received 21 Dec 2022, 15:10:51 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 9 (0x00000009) Unknown error code
Computer ID 1534452
Run time 16 hours 58 min 52 sec
CPU time 16 hours 23 min 24 sec
Validate state Invalid
Credit 7,862.40
Device peak FLOPS 4.58 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.15 MB
Peak swap size 4,978.07 MB
Peak disk usage 4,447.32 MB
Stderr
<core_client_version>7.9.3</core_client_version>
<![CDATA[
<message>
process exited with code 9 (0x9, -247)</message>
<stderr_txt>
121 H=1121:00 +CPU= 28.883
  08:45:02 STEP 1122 H=1122:00 +CPU= 44.177
  08:45:32 STEP 1123 H=1123:00 +CPU= 28.632
  08:46:02 STEP 1124 H=1124:00 +CPU= 29.677
  08:46:45 STEP 1125 H=1125:00 +CPU= 43.282
  08:47:15 STEP 1126 H=1126:00 +CPU= 29.239
  08:47:44 STEP 1127 H=1127:00 +CPU= 28.329
  08:48:55 STEP 1128 H=1128:00 +CPU= 50.707
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001128
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001116
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_46.zip
Uploading the intermediate file: upload_file_46.zip
  08:49:24 STEP 1129 H=1129:00 +CPU= 28.798
  08:49:56 STEP 1130 H=1130:00 +CPU= 29.783
  08:50:41 STEP 1131 H=1131:00 +CPU= 43.611
  08:51:11 STEP 1132 H=1132:00 +CPU= 29.361
  08:51:40 STEP 1133 H=1133:00 +CPU= 28.899
  08:52:27 STEP 1134 H=1134:00 +CPU= 44.669
  08:52:58 STEP 1135 H=1135:00 +CPU= 29.002
  08:53:29 STEP 1136 H=1136:00 +CPU= 29.081
  08:54:12 STEP 1137 H=1137:00 +CPU= 42.901
  08:54:42 STEP 1138 H=1138:00 +CPU= 29.638
  08:55:11 STEP 1139 H=1139:00 +CPU= 29.107
  08:56:01 STEP 1140 H=1140:00 +CPU= 48.113
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001140
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001140
  08:56:30 STEP 1141 H=1141:00 +CPU= 28.959
  08:57:00 STEP 1142 H=1142:00 +CPU= 29.603
  08:57:45 STEP 1143 H=1143:00 +CPU= 43.882
  08:58:15 STEP 1144 H=1144:00 +CPU= 29.677
  08:58:44 STEP 1145 H=1145:00 +CPU= 28.943
  08:59:29 STEP 1146 H=1146:00 +CPU= 44.064
  08:59:59 STEP 1147 H=1147:00 +CPU= 29.017
  09:00:31 STEP 1148 H=1148:00 +CPU= 30.024
  09:01:15 STEP 1149 H=1149:00 +CPU= 43.235
  09:01:47 STEP 1150 H=1150:00 +CPU= 29.585
  09:02:18 STEP 1151 H=1151:00 +CPU= 29.763
  09:03:11 STEP 1152 H=1152:00 +CPU= 50.347
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001152
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_47.zip
Uploading the intermediate file: upload_file_47.zip
  09:03:41 STEP 1153 H=1153:00 +CPU= 29.081
  09:04:13 STEP 1154 H=1154:00 +CPU= 29.531
  09:04:56 STEP 1155 H=1155:00 +CPU= 43.138
  09:05:26 STEP 1156 H=1156:00 +CPU= 29.629
  09:05:55 STEP 1157 H=1157:00 +CPU= 28.603
  09:06:41 STEP 1158 H=1158:00 +CPU= 44.619
  09:07:11 STEP 1159 H=1159:00 +CPU= 28.915
  09:07:43 STEP 1160 H=1160:00 +CPU= 30.013
  09:08:31 STEP 1161 H=1161:00 +CPU= 44.088
  09:09:03 STEP 1162 H=1162:00 +CPU= 29.677
  09:09:33 STEP 1163 H=1163:00 +CPU= 29.627
  09:10:25 STEP 1164 H=1164:00 +CPU= 48.768
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001164
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001164
  09:10:56 STEP 1165 H=1165:00 +CPU= 28.734
  09:11:26 STEP 1166 H=1166:00 +CPU= 29.783
  09:12:11 STEP 1167 H=1167:00 +CPU= 43.828
  09:12:41 STEP 1168 H=1168:00 +CPU= 30.149
  09:13:11 STEP 1169 H=1169:00 +CPU= 28.203
  09:13:57 STEP 1170 H=1170:00 +CPU= 44.070
  09:14:29 STEP 1171 H=1171:00 +CPU= 29.366
  09:15:00 STEP 1172 H=1172:00 +CPU= 29.939
  09:15:43 STEP 1173 H=1173:00 +CPU= 43.000
  09:16:13 STEP 1174 H=1174:00 +CPU= 29.178
  09:16:42 STEP 1175 H=1175:00 +CPU= 28.519
  09:17:35 STEP 1176 H=1176:00 +CPU= 50.589
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001176
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001164
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_48.zip
Uploading the intermediate file: upload_file_48.zip
  09:18:04 STEP 1177 H=1177:00 +CPU= 28.622
  09:18:35 STEP 1178 H=1178:00 +CPU= 29.805
  09:19:19 STEP 1179 H=1179:00 +CPU= 43.721
  09:19:48 STEP 1180 H=1180:00 +CPU= 29.506
  09:20:17 STEP 1181 H=1181:00 +CPU= 28.874
  09:21:03 STEP 1182 H=1182:00 +CPU= 44.673
  09:21:32 STEP 1183 H=1183:00 +CPU= 28.537
  09:22:03 STEP 1184 H=1184:00 +CPU= 29.638
  09:22:47 STEP 1185 H=1185:00 +CPU= 43.814
  09:23:17 STEP 1186 H=1186:00 +CPU= 29.595
  09:23:45 STEP 1187 H=1187:00 +CPU= 28.041
  09:24:37 STEP 1188 H=1188:00 +CPU= 48.659
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001188
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001188
  09:25:06 STEP 1189 H=1189:00 +CPU= 28.625
  09:25:37 STEP 1190 H=1190:00 +CPU= 29.835
  09:26:21 STEP 1191 H=1191:00 +CPU= 43.837
  09:26:53 STEP 1192 H=1192:00 +CPU= 29.964
  09:27:22 STEP 1193 H=1193:00 +CPU= 28.556
  09:28:07 STEP 1194 H=1194:00 +CPU= 45.448
  09:28:37 STEP 1195 H=1195:00 +CPU= 28.856
  09:29:08 STEP 1196 H=1196:00 +CPU= 29.899
  09:29:53 STEP 1197 H=1197:00 +CPU= 44.173
  09:30:24 STEP 1198 H=1198:00 +CPU= 29.932
  09:30:53 STEP 1199 H=1199:00 +CPU= 28.105
  09:31:53 STEP 1200 H=1200:00 +CPU= 51.068
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001200
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001188
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_49.zip
Uploading the intermediate file: upload_file_49.zip
  09:32:22 STEP 1201 H=1201:00 +CPU= 29.007
  09:32:53 STEP 1202 H=1202:00 +CPU= 29.440
  09:33:37 STEP 1203 H=1203:00 +CPU= 44.025
  09:34:08 STEP 1204 H=1204:00 +CPU= 29.639
  09:34:37 STEP 1205 H=1205:00 +CPU= 28.837
  09:35:23 STEP 1206 H=1206:00 +CPU= 44.214
  09:35:54 STEP 1207 H=1207:00 +CPU= 29.129
  09:36:25 STEP 1208 H=1208:00 +CPU= 30.171
  09:37:13 STEP 1209 H=1209:00 +CPU= 43.952
  09:37:45 STEP 1210 H=1210:00 +CPU= 29.837
  09:38:14 STEP 1211 H=1211:00 +CPU= 29.352
  09:39:04 STEP 1212 H=1212:00 +CPU= 48.334
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001212
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001212
  09:39:34 STEP 1213 H=1213:00 +CPU= 29.448
  09:40:06 STEP 1214 H=1214:00 +CPU= 29.712
  09:40:52 STEP 1215 H=1215:00 +CPU= 43.742
  09:41:22 STEP 1216 H=1216:00 +CPU= 30.073
  09:41:52 STEP 1217 H=1217:00 +CPU= 29.093
  09:42:38 STEP 1218 H=1218:00 +CPU= 44.903
  09:43:09 STEP 1219 H=1219:00 +CPU= 29.330
  09:43:40 STEP 1220 H=1220:00 +CPU= 29.807
  09:44:26 STEP 1221 H=1221:00 +CPU= 44.020
  09:45:00 STEP 1222 H=1222:00 +CPU= 30.466
  09:45:27 STEP 1223 H=1223:00 +CPU= 26.274
  09:46:21 STEP 1224 H=1224:00 +CPU= 51.103
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001224
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001212
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_50.zip
Uploading the intermediate file: upload_file_50.zip
  09:46:50 STEP 1225 H=1225:00 +CPU= 28.893
  09:47:21 STEP 1226 H=1226:00 +CPU= 29.549
  09:48:05 STEP 1227 H=1227:00 +CPU= 43.766
  09:48:35 STEP 1228 H=1228:00 +CPU= 29.892
  09:49:05 STEP 1229 H=1229:00 +CPU= 29.024
  09:49:53 STEP 1230 H=1230:00 +CPU= 45.094
  09:50:22 STEP 1231 H=1231:00 +CPU= 29.167
  09:50:54 STEP 1232 H=1232:00 +CPU= 30.017
  09:51:40 STEP 1233 H=1233:00 +CPU= 44.076
  09:52:10 STEP 1234 H=1234:00 +CPU= 30.331
  09:52:40 STEP 1235 H=1235:00 +CPU= 28.847
  09:53:32 STEP 1236 H=1236:00 +CPU= 49.105
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001236
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001236
  09:54:02 STEP 1237 H=1237:00 +CPU= 29.018
  09:54:32 STEP 1238 H=1238:00 +CPU= 29.719
  09:55:16 STEP 1239 H=1239:00 +CPU= 43.517
  09:55:46 STEP 1240 H=1240:00 +CPU= 29.844
  09:56:18 STEP 1241 H=1241:00 +CPU= 28.914
  09:57:03 STEP 1242 H=1242:00 +CPU= 44.277
  09:57:33 STEP 1243 H=1243:00 +CPU= 28.697
  09:58:03 STEP 1244 H=1244:00 +CPU= 29.699
  09:58:47 STEP 1245 H=1245:00 +CPU= 43.902
  09:59:18 STEP 1246 H=1246:00 +CPU= 30.520
  09:59:45 STEP 1247 H=1247:00 +CPU= 27.340
  10:00:36 STEP 1248 H=1248:00 +CPU= 48.335
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001248
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_51.zip
Uploading the intermediate file: upload_file_51.zip
  10:01:05 STEP 1249 H=1249:00 +CPU= 28.836
  10:01:37 STEP 1250 H=1250:00 +CPU= 29.822
  10:02:22 STEP 1251 H=1251:00 +CPU= 44.084
  10:02:52 STEP 1252 H=1252:00 +CPU= 29.958
  10:03:23 STEP 1253 H=1253:00 +CPU= 29.143
  10:04:10 STEP 1254 H=1254:00 +CPU= 44.684
  10:04:40 STEP 1255 H=1255:00 +CPU= 29.302
  10:05:10 STEP 1256 H=1256:00 +CPU= 29.777
  10:05:56 STEP 1257 H=1257:00 +CPU= 44.123
  10:06:25 STEP 1258 H=1258:00 +CPU= 29.674
  10:06:55 STEP 1259 H=1259:00 +CPU= 28.808
  10:07:45 STEP 1260 H=1260:00 +CPU= 48.196
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001260
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001260
  10:08:13 STEP 1261 H=1261:00 +CPU= 28.842
  10:08:44 STEP 1262 H=1262:00 +CPU= 29.883
  10:09:28 STEP 1263 H=1263:00 +CPU= 43.970
  10:09:58 STEP 1264 H=1264:00 +CPU= 29.471
  10:10:27 STEP 1265 H=1265:00 +CPU= 28.632
  10:11:14 STEP 1266 H=1266:00 +CPU= 44.793
  10:11:44 STEP 1267 H=1267:00 +CPU= 28.959
  10:12:14 STEP 1268 H=1268:00 +CPU= 30.258
  10:12:58 STEP 1269 H=1269:00 +CPU= 44.044
  10:13:30 STEP 1270 H=1270:00 +CPU= 29.874
  10:14:01 STEP 1271 H=1271:00 +CPU= 29.407
  10:15:03 STEP 1272 H=1272:00 +CPU= 50.913
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001272
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001260
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_52.zip
Uploading the intermediate file: upload_file_52.zip
  10:15:34 STEP 1273 H=1273:00 +CPU= 29.363
  10:16:04 STEP 1274 H=1274:00 +CPU= 29.707
  10:16:49 STEP 1275 H=1275:00 +CPU= 44.072
  10:17:19 STEP 1276 H=1276:00 +CPU= 29.843
  10:17:48 STEP 1277 H=1277:00 +CPU= 28.795
  10:18:34 STEP 1278 H=1278:00 +CPU= 45.411
  10:19:05 STEP 1279 H=1279:00 +CPU= 29.498
  10:19:36 STEP 1280 H=1280:00 +CPU= 30.278
  10:20:20 STEP 1281 H=1281:00 +CPU= 43.473
  10:20:51 STEP 1282 H=1282:00 +CPU= 30.042
  10:21:20 STEP 1283 H=1283:00 +CPU= 28.947
  10:22:09 STEP 1284 H=1284:00 +CPU= 48.500
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001284
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001284
  10:22:37 STEP 1285 H=1285:00 +CPU= 28.464
  10:23:08 STEP 1286 H=1286:00 +CPU= 29.821
  10:23:56 STEP 1287 H=1287:00 +CPU= 43.903
  10:24:28 STEP 1288 H=1288:00 +CPU= 30.076
  10:25:00 STEP 1289 H=1289:00 +CPU= 29.419
  10:25:46 STEP 1290 H=1290:00 +CPU= 44.656
  10:26:16 STEP 1291 H=1291:00 +CPU= 29.280
  10:26:46 STEP 1292 H=1292:00 +CPU= 29.006
  10:27:33 STEP 1293 H=1293:00 +CPU= 43.687
  10:28:04 STEP 1294 H=1294:00 +CPU= 30.327
  10:28:35 STEP 1295 H=1295:00 +CPU= 29.213
  10:29:28 STEP 1296 H=1296:00 +CPU= 49.897
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001296
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001284
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_53.zip
Uploading the intermediate file: upload_file_53.zip
  10:29:57 STEP 1297 H=1297:00 +CPU= 28.651
  10:30:28 STEP 1298 H=1298:00 +CPU= 29.556
  10:31:12 STEP 1299 H=1299:00 +CPU= 43.462
  10:31:42 STEP 1300 H=1300:00 +CPU= 29.588
  10:32:11 STEP 1301 H=1301:00 +CPU= 28.797
  10:32:58 STEP 1302 H=1302:00 +CPU= 44.860
  10:33:28 STEP 1303 H=1303:00 +CPU= 29.306
  10:34:00 STEP 1304 H=1304:00 +CPU= 29.719
  10:34:44 STEP 1305 H=1305:00 +CPU= 43.709
  10:35:13 STEP 1306 H=1306:00 +CPU= 29.568
  10:35:42 STEP 1307 H=1307:00 +CPU= 29.000
  10:36:33 STEP 1308 H=1308:00 +CPU= 49.108
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001308
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001308
  10:37:02 STEP 1309 H=1309:00 +CPU= 28.636
  10:37:32 STEP 1310 H=1310:00 +CPU= 29.821
  10:38:16 STEP 1311 H=1311:00 +CPU= 44.301
  10:38:47 STEP 1312 H=1312:00 +CPU= 29.666
  10:39:18 STEP 1313 H=1313:00 +CPU= 29.290
  10:40:04 STEP 1314 H=1314:00 +CPU= 44.983
  10:40:32 STEP 1315 H=1315:00 +CPU= 28.583
  10:41:02 STEP 1316 H=1316:00 +CPU= 29.852
  10:41:47 STEP 1317 H=1317:00 +CPU= 44.516
  10:42:20 STEP 1318 H=1318:00 +CPU= 30.375
  10:42:51 STEP 1319 H=1319:00 +CPU= 30.163
  10:43:44 STEP 1320 H=1320:00 +CPU= 50.319
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001320
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001308
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_54.zip
Uploading the intermediate file: upload_file_54.zip
Uploading trickle at timestep: 4752000
  10:44:15 STEP 1321 H=1321:00 +CPU= 28.909
  10:44:45 STEP 1322 H=1322:00 +CPU= 29.479
  10:45:30 STEP 1323 H=1323:00 +CPU= 43.757
  10:46:01 STEP 1324 H=1324:00 +CPU= 30.005
  10:46:32 STEP 1325 H=1325:00 +CPU= 28.948
  10:47:17 STEP 1326 H=1326:00 +CPU= 45.000
  10:47:47 STEP 1327 H=1327:00 +CPU= 29.410
  10:48:17 STEP 1328 H=1328:00 +CPU= 29.717
  10:49:02 STEP 1329 H=1329:00 +CPU= 43.922
  10:49:33 STEP 1330 H=1330:00 +CPU= 29.584
  10:50:02 STEP 1331 H=1331:00 +CPU= 28.865
  10:50:51 STEP 1332 H=1332:00 +CPU= 48.342
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001332
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001332
  10:51:21 STEP 1333 H=1333:00 +CPU= 29.514
  10:51:53 STEP 1334 H=1334:00 +CPU= 29.950
  10:52:37 STEP 1335 H=1335:00 +CPU= 43.862
  10:53:07 STEP 1336 H=1336:00 +CPU= 29.683
  10:53:36 STEP 1337 H=1337:00 +CPU= 28.497
  10:54:22 STEP 1338 H=1338:00 +CPU= 45.230
  10:54:51 STEP 1339 H=1339:00 +CPU= 28.964
  10:55:21 STEP 1340 H=1340:00 +CPU= 30.120
  10:56:06 STEP 1341 H=1341:00 +CPU= 44.535
  10:56:36 STEP 1342 H=1342:00 +CPU= 29.650
  10:57:07 STEP 1343 H=1343:00 +CPU= 29.716
  10:58:11 STEP 1344 H=1344:00 +CPU= 50.806
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001344
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_55.zip
Uploading the intermediate file: upload_file_55.zip
  10:58:42 STEP 1345 H=1345:00 +CPU= 29.212
  10:59:12 STEP 1346 H=1346:00 +CPU= 30.000
  10:59:56 STEP 1347 H=1347:00 +CPU= 43.861
  11:00:26 STEP 1348 H=1348:00 +CPU= 29.316
  11:00:57 STEP 1349 H=1349:00 +CPU= 29.142
  11:01:41 STEP 1350 H=1350:00 +CPU= 44.430
  11:02:11 STEP 1351 H=1351:00 +CPU= 29.197
  11:02:42 STEP 1352 H=1352:00 +CPU= 29.426
  11:03:27 STEP 1353 H=1353:00 +CPU= 44.322
  11:03:58 STEP 1354 H=1354:00 +CPU= 30.011
  11:04:27 STEP 1355 H=1355:00 +CPU= 29.448
  11:05:16 STEP 1356 H=1356:00 +CPU= 48.505
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001356
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001356
  11:05:46 STEP 1357 H=1357:00 +CPU= 28.576
  11:06:17 STEP 1358 H=1358:00 +CPU= 30.415
  11:07:02 STEP 1359 H=1359:00 +CPU= 44.430
  11:07:32 STEP 1360 H=1360:00 +CPU= 29.916
  11:08:01 STEP 1361 H=1361:00 +CPU= 28.947
  11:08:47 STEP 1362 H=1362:00 +CPU= 45.065
  11:09:15 STEP 1363 H=1363:00 +CPU= 28.377
  11:09:47 STEP 1364 H=1364:00 +CPU= 29.908
  11:10:31 STEP 1365 H=1365:00 +CPU= 43.786
  11:11:02 STEP 1366 H=1366:00 +CPU= 29.537
  11:11:32 STEP 1367 H=1367:00 +CPU= 29.377
  11:12:24 STEP 1368 H=1368:00 +CPU= 48.994
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001368
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001356
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_56.zip
Uploading the intermediate file: upload_file_56.zip
  11:12:52 STEP 1369 H=1369:00 +CPU= 28.312
  11:13:23 STEP 1370 H=1370:00 +CPU= 29.790
  11:14:07 STEP 1371 H=1371:00 +CPU= 43.720
  11:14:39 STEP 1372 H=1372:00 +CPU= 29.619
  11:15:11 STEP 1373 H=1373:00 +CPU= 28.736
  11:15:57 STEP 1374 H=1374:00 +CPU= 44.790
  11:16:26 STEP 1375 H=1375:00 +CPU= 28.230
  11:16:56 STEP 1376 H=1376:00 +CPU= 29.676
  11:17:41 STEP 1377 H=1377:00 +CPU= 43.175
  11:18:10 STEP 1378 H=1378:00 +CPU= 28.994
  11:18:39 STEP 1379 H=1379:00 +CPU= 28.819
  11:19:30 STEP 1380 H=1380:00 +CPU= 48.759
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001380
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001380
  11:19:59 STEP 1381 H=1381:00 +CPU= 29.219
  11:20:34 STEP 1382 H=1382:00 +CPU= 30.291
  11:21:19 STEP 1383 H=1383:00 +CPU= 44.214
  11:21:50 STEP 1384 H=1384:00 +CPU= 30.094
  11:22:19 STEP 1385 H=1385:00 +CPU= 29.040
  11:23:06 STEP 1386 H=1386:00 +CPU= 44.751
  11:23:35 STEP 1387 H=1387:00 +CPU= 28.685
  11:24:07 STEP 1388 H=1388:00 +CPU= 29.860
  11:24:52 STEP 1389 H=1389:00 +CPU= 44.647
  11:25:24 STEP 1390 H=1390:00 +CPU= 29.764
  11:25:51 STEP 1391 H=1391:00 +CPU= 27.043
  11:26:46 STEP 1392 H=1392:00 +CPU= 50.874
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001392
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001380
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_57.zip
Uploading the intermediate file: upload_file_57.zip
  11:27:17 STEP 1393 H=1393:00 +CPU= 28.450
  11:27:47 STEP 1394 H=1394:00 +CPU= 29.542
  11:28:33 STEP 1395 H=1395:00 +CPU= 43.633
  11:29:02 STEP 1396 H=1396:00 +CPU= 28.896
  11:29:31 STEP 1397 H=1397:00 +CPU= 28.887
  11:30:16 STEP 1398 H=1398:00 +CPU= 44.890
  11:30:46 STEP 1399 H=1399:00 +CPU= 28.552
  11:31:16 STEP 1400 H=1400:00 +CPU= 30.036
  11:32:00 STEP 1401 H=1401:00 +CPU= 43.746
  11:32:30 STEP 1402 H=1402:00 +CPU= 30.046
  11:33:01 STEP 1403 H=1403:00 +CPU= 29.001
  11:33:53 STEP 1404 H=1404:00 +CPU= 49.060
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001404
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001404
  11:34:22 STEP 1405 H=1405:00 +CPU= 29.117
  11:34:53 STEP 1406 H=1406:00 +CPU= 30.374
  11:35:39 STEP 1407 H=1407:00 +CPU= 44.186
  11:36:09 STEP 1408 H=1408:00 +CPU= 29.465
  11:36:38 STEP 1409 H=1409:00 +CPU= 28.966
  11:37:26 STEP 1410 H=1410:00 +CPU= 45.006
  11:37:55 STEP 1411 H=1411:00 +CPU= 28.853
  11:38:25 STEP 1412 H=1412:00 +CPU= 30.016
  11:39:10 STEP 1413 H=1413:00 +CPU= 44.350
  11:39:39 STEP 1414 H=1414:00 +CPU= 29.144
  11:40:14 STEP 1415 H=1415:00 +CPU= 29.510
  11:41:11 STEP 1416 H=1416:00 +CPU= 50.875
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001416
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001404
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_58.zip
Uploading the intermediate file: upload_file_58.zip
  11:41:42 STEP 1417 H=1417:00 +CPU= 28.778
  11:42:12 STEP 1418 H=1418:00 +CPU= 29.915
  11:42:57 STEP 1419 H=1419:00 +CPU= 44.072
  11:43:28 STEP 1420 H=1420:00 +CPU= 29.779
  11:43:58 STEP 1421 H=1421:00 +CPU= 29.086
  11:44:43 STEP 1422 H=1422:00 +CPU= 44.697
  11:45:12 STEP 1423 H=1423:00 +CPU= 28.449
  11:45:44 STEP 1424 H=1424:00 +CPU= 30.088
  11:46:30 STEP 1425 H=1425:00 +CPU= 44.069
  11:47:00 STEP 1426 H=1426:00 +CPU= 29.430
  11:47:29 STEP 1427 H=1427:00 +CPU= 28.356
  11:48:18 STEP 1428 H=1428:00 +CPU= 49.048
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001428
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001428
  11:48:47 STEP 1429 H=1429:00 +CPU= 28.360
  11:49:19 STEP 1430 H=1430:00 +CPU= 29.894
  11:50:02 STEP 1431 H=1431:00 +CPU= 43.710
  11:50:32 STEP 1432 H=1432:00 +CPU= 29.347
  11:51:00 STEP 1433 H=1433:00 +CPU= 28.500
  11:51:45 STEP 1434 H=1434:00 +CPU= 44.692
  11:52:15 STEP 1435 H=1435:00 +CPU= 28.739
  11:52:45 STEP 1436 H=1436:00 +CPU= 29.775
  11:53:30 STEP 1437 H=1437:00 +CPU= 43.556
  11:54:01 STEP 1438 H=1438:00 +CPU= 29.221
  11:54:34 STEP 1439 H=1439:00 +CPU= 29.424
  11:55:41 STEP 1440 H=1440:00 +CPU= 48.538
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001440
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001428
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_59.zip
Uploading the intermediate file: upload_file_59.zip
  11:56:11 STEP 1441 H=1441:00 +CPU= 28.799
  11:56:41 STEP 1442 H=1442:00 +CPU= 29.687
  11:57:28 STEP 1443 H=1443:00 +CPU= 44.129
  11:57:57 STEP 1444 H=1444:00 +CPU= 29.577
  11:58:27 STEP 1445 H=1445:00 +CPU= 29.090
  11:59:12 STEP 1446 H=1446:00 +CPU= 45.094
  11:59:40 STEP 1447 H=1447:00 +CPU= 28.656
  12:00:10 STEP 1448 H=1448:00 +CPU= 29.933
  12:00:56 STEP 1449 H=1449:00 +CPU= 43.903
  12:01:26 STEP 1450 H=1450:00 +CPU= 29.587
  12:01:56 STEP 1451 H=1451:00 +CPU= 29.705
  12:02:47 STEP 1452 H=1452:00 +CPU= 49.415
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001452
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001452
  12:03:16 STEP 1453 H=1453:00 +CPU= 28.312
  12:03:46 STEP 1454 H=1454:00 +CPU= 29.407
  12:04:31 STEP 1455 H=1455:00 +CPU= 44.125
  12:05:00 STEP 1456 H=1456:00 +CPU= 28.864
  12:05:32 STEP 1457 H=1457:00 +CPU= 29.377
  12:06:22 STEP 1458 H=1458:00 +CPU= 46.270
  12:06:51 STEP 1459 H=1459:00 +CPU= 28.070
  12:07:21 STEP 1460 H=1460:00 +CPU= 28.826
  12:08:06 STEP 1461 H=1461:00 +CPU= 44.487
  12:08:37 STEP 1462 H=1462:00 +CPU= 28.278
  12:09:10 STEP 1463 H=1463:00 +CPU= 29.801
  12:10:11 STEP 1464 H=1464:00 +CPU= 51.151
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001464
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001452
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001452
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_60.zip
Uploading the intermediate file: upload_file_60.zip
  12:10:41 STEP 1465 H=1465:00 +CPU= 28.911
  12:11:12 STEP 1466 H=1466:00 +CPU= 29.854
  12:11:57 STEP 1467 H=1467:00 +CPU= 44.130
  12:12:27 STEP 1468 H=1468:00 +CPU= 29.259
  12:12:57 STEP 1469 H=1469:00 +CPU= 29.426
  12:13:42 STEP 1470 H=1470:00 +CPU= 45.310
  12:14:10 STEP 1471 H=1471:00 +CPU= 28.177
  12:14:40 STEP 1472 H=1472:00 +CPU= 29.907
  12:15:24 STEP 1473 H=1473:00 +CPU= 43.694
  12:15:54 STEP 1474 H=1474:00 +CPU= 29.327
  12:16:26 STEP 1475 H=1475:00 +CPU= 29.478
  12:17:17 STEP 1476 H=1476:00 +CPU= 49.607
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001476
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001476
  12:17:45 STEP 1477 H=1477:00 +CPU= 28.163
  12:18:15 STEP 1478 H=1478:00 +CPU= 29.690
  12:19:01 STEP 1479 H=1479:00 +CPU= 43.685
  12:19:31 STEP 1480 H=1480:00 +CPU= 29.728
  12:20:00 STEP 1481 H=1481:00 +CPU= 28.488
  12:20:49 STEP 1482 H=1482:00 +CPU= 45.488
  12:21:17 STEP 1483 H=1483:00 +CPU= 28.849
  12:21:48 STEP 1484 H=1484:00 +CPU= 30.186
  12:22:34 STEP 1485 H=1485:00 +CPU= 44.348
  12:23:03 STEP 1486 H=1486:00 +CPU= 28.844
  12:23:37 STEP 1487 H=1487:00 +CPU= 29.364
  12:24:43 STEP 1488 H=1488:00 +CPU= 50.733
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001488
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001464
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001476
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001476
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_61.zip
Uploading the intermediate file: upload_file_61.zip
  12:25:13 STEP 1489 H=1489:00 +CPU= 29.229
  12:25:43 STEP 1490 H=1490:00 +CPU= 29.669
  12:26:29 STEP 1491 H=1491:00 +CPU= 44.024
  12:27:00 STEP 1492 H=1492:00 +CPU= 30.564
  12:27:28 STEP 1493 H=1493:00 +CPU= 28.551
  12:28:13 STEP 1494 H=1494:00 +CPU= 44.497
  12:28:42 STEP 1495 H=1495:00 +CPU= 28.479
  12:29:11 STEP 1496 H=1496:00 +CPU= 29.089
  12:29:57 STEP 1497 H=1497:00 +CPU= 44.186
  12:30:28 STEP 1498 H=1498:00 +CPU= 29.547
  12:30:59 STEP 1499 H=1499:00 +CPU= 29.383
  12:31:52 STEP 1500 H=1500:00 +CPU= 48.619
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001500
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001500
  12:32:22 STEP 1501 H=1501:00 +CPU= 29.456
  12:32:52 STEP 1502 H=1502:00 +CPU= 29.643
  12:33:37 STEP 1503 H=1503:00 +CPU= 44.196
  12:34:09 STEP 1504 H=1504:00 +CPU= 30.389
  12:34:40 STEP 1505 H=1505:00 +CPU= 29.469
  12:35:29 STEP 1506 H=1506:00 +CPU= 45.194
  12:36:00 STEP 1507 H=1507:00 +CPU= 29.806
  12:36:31 STEP 1508 H=1508:00 +CPU= 30.656
  12:37:19 STEP 1509 H=1509:00 +CPU= 44.331
  12:37:48 STEP 1510 H=1510:00 +CPU= 29.119
  12:38:17 STEP 1511 H=1511:00 +CPU= 28.262
  12:39:15 STEP 1512 H=1512:00 +CPU= 50.902
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001512
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001488
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001500
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001500
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_62.zip
Uploading the intermediate file: upload_file_62.zip
  12:39:44 STEP 1513 H=1513:00 +CPU= 29.172
  12:40:14 STEP 1514 H=1514:00 +CPU= 29.453
  12:40:59 STEP 1515 H=1515:00 +CPU= 43.615
  12:41:30 STEP 1516 H=1516:00 +CPU= 30.192
  12:41:59 STEP 1517 H=1517:00 +CPU= 28.721
  12:42:43 STEP 1518 H=1518:00 +CPU= 44.291
  12:43:14 STEP 1519 H=1519:00 +CPU= 28.733
  12:43:44 STEP 1520 H=1520:00 +CPU= 29.682
  12:44:28 STEP 1521 H=1521:00 +CPU= 43.360
  12:44:58 STEP 1522 H=1522:00 +CPU= 29.595
  12:45:29 STEP 1523 H=1523:00 +CPU= 29.148
  12:46:18 STEP 1524 H=1524:00 +CPU= 48.621
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001524
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001524
  12:46:48 STEP 1525 H=1525:00 +CPU= 29.127
  12:47:20 STEP 1526 H=1526:00 +CPU= 29.557
  12:48:04 STEP 1527 H=1527:00 +CPU= 44.285
  12:48:35 STEP 1528 H=1528:00 +CPU= 30.386
  12:49:03 STEP 1529 H=1529:00 +CPU= 28.240
  12:49:50 STEP 1530 H=1530:00 +CPU= 45.093
  12:50:22 STEP 1531 H=1531:00 +CPU= 29.543
  12:50:54 STEP 1532 H=1532:00 +CPU= 29.388
  12:51:37 STEP 1533 H=1533:00 +CPU= 42.706
  12:52:08 STEP 1534 H=1534:00 +CPU= 30.090
  12:52:38 STEP 1535 H=1535:00 +CPU= 29.815
  12:53:33 STEP 1536 H=1536:00 +CPU= 48.497
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001536
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001512
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001524
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001524
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_63.zip
Uploading the intermediate file: upload_file_63.zip
  12:54:05 STEP 1537 H=1537:00 +CPU= 28.847
  12:54:37 STEP 1538 H=1538:00 +CPU= 30.543
  12:55:23 STEP 1539 H=1539:00 +CPU= 44.650
  12:56:01 STEP 1540 H=1540:00 +CPU= 37.335
  12:56:35 STEP 1541 H=1541:00 +CPU= 32.728
  12:57:30 STEP 1542 H=1542:00 +CPU= 51.702
  12:58:07 STEP 1543 H=1543:00 +CPU= 36.712
  12:58:43 STEP 1544 H=1544:00 +CPU= 34.894
  12:59:31 STEP 1545 H=1545:00 +CPU= 47.438
  13:00:03 STEP 1546 H=1546:00 +CPU= 31.114
  13:00:33 STEP 1547 H=1547:00 +CPU= 28.869
  13:01:23 STEP 1548 H=1548:00 +CPU= 48.988
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001548
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001548
  13:01:52 STEP 1549 H=1549:00 +CPU= 28.798
  13:02:24 STEP 1550 H=1550:00 +CPU= 30.100
  13:03:09 STEP 1551 H=1551:00 +CPU= 43.972
  13:03:39 STEP 1552 H=1552:00 +CPU= 29.308
  13:04:07 STEP 1553 H=1553:00 +CPU= 28.573
  13:04:56 STEP 1554 H=1554:00 +CPU= 48.216
  13:05:29 STEP 1555 H=1555:00 +CPU= 33.107
  13:06:05 STEP 1556 H=1556:00 +CPU= 34.740
  13:06:55 STEP 1557 H=1557:00 +CPU= 49.725
  13:07:34 STEP 1558 H=1558:00 +CPU= 37.286
  13:08:09 STEP 1559 H=1559:00 +CPU= 33.967
  13:09:19 STEP 1560 H=1560:00 +CPU= 51.300
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001560
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001536
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001548
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001548
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_64.zip
Uploading the intermediate file: upload_file_64.zip
Uploading trickle at timestep: 5616000
  13:09:49 STEP 1561 H=1561:00 +CPU= 29.100
  13:10:19 STEP 1562 H=1562:00 +CPU= 29.674
  13:11:05 STEP 1563 H=1563:00 +CPU= 43.455
  13:11:36 STEP 1564 H=1564:00 +CPU= 30.228
  13:12:05 STEP 1565 H=1565:00 +CPU= 29.201
  13:12:51 STEP 1566 H=1566:00 +CPU= 44.036
  13:13:20 STEP 1567 H=1567:00 +CPU= 29.065
  13:13:52 STEP 1568 H=1568:00 +CPU= 29.923
  13:14:41 STEP 1569 H=1569:00 +CPU= 49.101
  13:15:15 STEP 1570 H=1570:00 +CPU= 32.251
  13:15:47 STEP 1571 H=1571:00 +CPU= 31.076
  13:16:44 STEP 1572 H=1572:00 +CPU= 54.554
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001572
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001572
  13:17:22 STEP 1573 H=1573:00 +CPU= 33.509
  13:17:53 STEP 1574 H=1574:00 +CPU= 30.156
  13:18:36 STEP 1575 H=1575:00 +CPU= 43.473
  13:19:06 STEP 1576 H=1576:00 +CPU= 29.520
  13:19:35 STEP 1577 H=1577:00 +CPU= 28.539
  13:20:21 STEP 1578 H=1578:00 +CPU= 44.963
  13:20:50 STEP 1579 H=1579:00 +CPU= 28.789
  13:21:21 STEP 1580 H=1580:00 +CPU= 30.393
  13:22:08 STEP 1581 H=1581:00 +CPU= 44.200
  13:22:39 STEP 1582 H=1582:00 +CPU= 30.065
  13:23:10 STEP 1583 H=1583:00 +CPU= 30.849
  13:24:11 STEP 1584 H=1584:00 +CPU= 59.417
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001584
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001560
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001572
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001572
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_65.zip
Uploading the intermediate file: upload_file_65.zip
  13:24:49 STEP 1585 H=1585:00 +CPU= 36.680
  13:25:23 STEP 1586 H=1586:00 +CPU= 32.560
  13:26:10 STEP 1587 H=1587:00 +CPU= 46.878
  13:26:46 STEP 1588 H=1588:00 +CPU= 34.615
  13:27:17 STEP 1589 H=1589:00 +CPU= 28.646
  13:28:04 STEP 1590 H=1590:00 +CPU= 45.151
  13:28:35 STEP 1591 H=1591:00 +CPU= 29.099
  13:29:06 STEP 1592 H=1592:00 +CPU= 30.208
  13:29:52 STEP 1593 H=1593:00 +CPU= 44.242
  13:30:23 STEP 1594 H=1594:00 +CPU= 29.472
  13:30:55 STEP 1595 H=1595:00 +CPU= 29.455
  13:31:48 STEP 1596 H=1596:00 +CPU= 48.697
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001596
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001596
  13:32:17 STEP 1597 H=1597:00 +CPU= 28.711
  13:32:51 STEP 1598 H=1598:00 +CPU= 32.802
  13:33:40 STEP 1599 H=1599:00 +CPU= 46.221
  13:34:16 STEP 1600 H=1600:00 +CPU= 34.326
  13:34:48 STEP 1601 H=1601:00 +CPU= 31.034
  13:35:39 STEP 1602 H=1602:00 +CPU= 49.563
  13:36:10 STEP 1603 H=1603:00 +CPU= 30.957
  13:36:44 STEP 1604 H=1604:00 +CPU= 29.980
  13:37:31 STEP 1605 H=1605:00 +CPU= 43.424
  13:38:02 STEP 1606 H=1606:00 +CPU= 29.557
  13:38:33 STEP 1607 H=1607:00 +CPU= 29.525
  13:39:27 STEP 1608 H=1608:00 +CPU= 47.412
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001608
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001584
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001596
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001596
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_66.zip
Uploading the intermediate file: upload_file_66.zip
  13:39:55 STEP 1609 H=1609:00 +CPU= 28.592
  13:40:26 STEP 1610 H=1610:00 +CPU= 29.765
  13:41:12 STEP 1611 H=1611:00 +CPU= 44.811
  13:41:43 STEP 1612 H=1612:00 +CPU= 30.372
  13:42:15 STEP 1613 H=1613:00 +CPU= 31.141
  13:43:03 STEP 1614 H=1614:00 +CPU= 47.919
  13:43:37 STEP 1615 H=1615:00 +CPU= 33.130
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1981050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0166
(argv4) batchid: 950
(argv5) wuid: 12166810
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/10
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_1c37079b5d9f79fc08ceab8d718ad7d9 to: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0166_1981050100_123_950_12166810.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/10/oifs_43r3_ps_0166_1981050100_123_950_12166810.zip
ic_ancil_file: ic_ancil_12166810
ifsdata_file: ifsdata_12166810
climate_data_file: clim_data_12166810
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_d1febfbd10c772719a1d6013a23befd8 to: /var/lib/boinc-client/slots/10/ic_ancil_12166810.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/10/ic_ancil_12166810.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ce52956dec540c6f54fdad308c66eb52 to: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166810.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/10/ifsdata/ifsdata_12166810.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166810.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/10/159l_2/clim_data_12166810.zip
Checking for progress XML file: /var/lib/boinc-client/slots/10/progress_file_12166810.xml
Opened progress file ok : /var/lib/boinc-client/slots/10/progress_file_12166810.xml
-- Model is restarting --
Adjusting last_iter, 1615, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/10/progress_file_12166810.xml
last_cpu_time: 57238
upload_file_number: 67
last_iter: 1607
last_upload: 5788800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500
The child process has been launched with process id: 8760
Executing the command: /var/lib/boinc-client/slots/10/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:WRZTELEBIMWSKL04:1:1:8760:8760] [20221221:153024:1671633024:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  15:32:11 STEP 1608 H=1608:00 +CPU= 48.884
  15:32:40 STEP 1609 H=1609:00 +CPU= 27.588
  15:33:08 STEP 1610 H=1610:00 +CPU= 27.943
  15:33:52 STEP 1611 H=1611:00 +CPU= 42.346
  15:34:22 STEP 1612 H=1612:00 +CPU= 28.848
  15:34:51 STEP 1613 H=1613:00 +CPU= 28.132
  15:35:42 STEP 1614 H=1614:00 +CPU= 48.421
  15:36:16 STEP 1615 H=1615:00 +CPU= 32.579
  15:36:51 STEP 1616 H=1616:00 +CPU= 34.097
  15:37:41 STEP 1617 H=1617:00 +CPU= 48.408
  15:38:14 STEP 1618 H=1618:00 +CPU= 31.996
  15:38:43 STEP 1619 H=1619:00 +CPU= 28.062
  15:39:31 STEP 1620 H=1620:00 +CPU= 47.296
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001620
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001620
  15:39:59 STEP 1621 H=1621:00 +CPU= 27.836
  15:40:27 STEP 1622 H=1622:00 +CPU= 27.963
  15:41:10 STEP 1623 H=1623:00 +CPU= 42.210
  15:41:40 STEP 1624 H=1624:00 +CPU= 29.091
  15:42:08 STEP 1625 H=1625:00 +CPU= 27.698
  15:42:54 STEP 1626 H=1626:00 +CPU= 43.635
  15:43:21 STEP 1627 H=1627:00 +CPU= 27.539
  15:43:52 STEP 1628 H=1628:00 +CPU= 30.079
  15:44:43 STEP 1629 H=1629:00 +CPU= 50.720
  15:45:17 STEP 1630 H=1630:00 +CPU= 33.782
  15:45:51 STEP 1631 H=1631:00 +CPU= 33.428
  15:46:45 STEP 1632 H=1632:00 +CPU= 50.447
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001632
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001608
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001620
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001620
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_67.zip
Uploading the intermediate file: upload_file_67.zip
  15:47:15 STEP 1633 H=1633:00 +CPU= 27.839
  15:47:43 STEP 1634 H=1634:00 +CPU= 27.716
  15:48:25 STEP 1635 H=1635:00 +CPU= 41.838
  15:48:55 STEP 1636 H=1636:00 +CPU= 29.192
  15:49:25 STEP 1637 H=1637:00 +CPU= 27.905
  15:50:09 STEP 1638 H=1638:00 +CPU= 44.320
  15:50:39 STEP 1639 H=1639:00 +CPU= 28.324
  15:51:06 STEP 1640 H=1640:00 +CPU= 26.800
  15:51:46 STEP 1641 H=1641:00 +CPU= 40.132
  15:52:17 STEP 1642 H=1642:00 +CPU= 31.046
  15:52:50 STEP 1643 H=1643:00 +CPU= 32.254
  15:53:43 STEP 1644 H=1644:00 +CPU= 52.917
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001644
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001644
  15:54:17 STEP 1645 H=1645:00 +CPU= 33.321
  15:54:51 STEP 1646 H=1646:00 +CPU= 32.574
  15:55:36 STEP 1647 H=1647:00 +CPU= 42.060
  15:56:08 STEP 1648 H=1648:00 +CPU= 29.160
  15:56:35 STEP 1649 H=1649:00 +CPU= 27.202
  15:57:19 STEP 1650 H=1650:00 +CPU= 43.204
  15:57:47 STEP 1651 H=1651:00 +CPU= 27.042
  15:58:16 STEP 1652 H=1652:00 +CPU= 29.275
  15:58:59 STEP 1653 H=1653:00 +CPU= 42.380
  15:59:28 STEP 1654 H=1654:00 +CPU= 28.593
  15:59:57 STEP 1655 H=1655:00 +CPU= 28.385
  16:00:57 STEP 1656 H=1656:00 +CPU= 56.516
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMGGhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMSHhq0f+001656
Moving to projects directory: /var/lib/boinc-client/slots/10/ICMUAhq0f+001656
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001632
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMGGhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMSHhq0f+001644
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12166810/ICMUAhq0f+001644
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0166_1981050100_123_950_12166810_0_r200399500_68.zip
Uploading the intermediate file: upload_file_68.zip
  16:01:34 STEP 1657 H=1657:00 +CPU= 34.035
double free or corruption (out)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:45:56 1534452 22251021 oifs_43r3_ps_0166_1981050100_123_950_12166810_0 5,616,000 55,259 0.0098
09 May 2023 15:45:46 1534452 22251021 oifs_43r3_ps_0166_1981050100_123_950_12166810_0 4,752,000 46,854 0.0099
09 May 2023 15:45:17 1534452 22251021 oifs_43r3_ps_0166_1981050100_123_950_12166810_0 2,592,000 25,744 0.0099
09 May 2023 15:45:13 1534452 22251021 oifs_43r3_ps_0166_1981050100_123_950_12166810_0 1,728,000 16,692 0.0097
09 May 2023 15:45:07 1534452 22251021 oifs_43r3_ps_0166_1981050100_123_950_12166810_0 864,000 8,294 0.0096


©2024 climateprediction.net