climateprediction.net home page
Task 22443233

Task 22443233

Name oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0
Workunit 12289646
Created 12 Jun 2024, 21:52:33 UTC
Sent 13 Jun 2024, 10:00:54 UTC
Report deadline 12 Aug 2024, 10:00:54 UTC
Received 15 Jun 2024, 23:20:22 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1522755
Run time 14 hours 20 min 22 sec
CPU time 14 hours 13 min 38 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 7.59 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,010.03 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,383.86 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
8:58:26 STEP 1275 H= 318:45 +CPU= 16.751
  18:58:44 STEP 1276 H= 319:00 +CPU= 17.940
  18:59:01 STEP 1277 H= 319:15 +CPU= 17.108
  18:59:18 STEP 1278 H= 319:30 +CPU= 16.856
  18:59:34 STEP 1279 H= 319:45 +CPU= 16.192
  18:59:51 STEP 1280 H= 320:00 +CPU= 16.997
  19:00:08 STEP 1281 H= 320:15 +CPU= 16.357
  19:00:23 STEP 1282 H= 320:30 +CPU= 15.471
  19:00:39 STEP 1283 H= 320:45 +CPU= 16.088
  19:00:59 STEP 1284 H= 321:00 +CPU= 19.090
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001284
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001284
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001284
  19:01:13 STEP 1285 H= 321:15 +CPU= 14.798
  19:01:28 STEP 1286 H= 321:30 +CPU= 14.916
  19:01:44 STEP 1287 H= 321:45 +CPU= 15.288
  19:01:59 STEP 1288 H= 322:00 +CPU= 15.659
  19:02:14 STEP 1289 H= 322:15 +CPU= 15.077
  19:02:29 STEP 1290 H= 322:30 +CPU= 14.994
  19:02:44 STEP 1291 H= 322:45 +CPU= 14.354
  19:02:59 STEP 1292 H= 323:00 +CPU= 14.799
  19:03:12 STEP 1293 H= 323:15 +CPU= 13.831
  19:03:27 STEP 1294 H= 323:30 +CPU= 14.495
  19:03:42 STEP 1295 H= 323:45 +CPU= 14.648
  19:04:02 STEP 1296 H= 324:00 +CPU= 20.386
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001296
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001296
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001296
  19:04:17 STEP 1297 H= 324:15 +CPU= 14.692
  19:04:31 STEP 1298 H= 324:30 +CPU= 14.256
  19:04:45 STEP 1299 H= 324:45 +CPU= 13.674
  19:05:00 STEP 1300 H= 325:00 +CPU= 15.071
  19:05:14 STEP 1301 H= 325:15 +CPU= 14.060
  19:05:28 STEP 1302 H= 325:30 +CPU= 14.186
  19:05:42 STEP 1303 H= 325:45 +CPU= 13.934
  19:05:56 STEP 1304 H= 326:00 +CPU= 14.173
  19:06:10 STEP 1305 H= 326:15 +CPU= 13.792
  19:06:25 STEP 1306 H= 326:30 +CPU= 14.662
  19:06:39 STEP 1307 H= 326:45 +CPU= 13.982
  19:06:57 STEP 1308 H= 327:00 +CPU= 17.962
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001308
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001308
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001308
  19:07:11 STEP 1309 H= 327:15 +CPU= 14.007
  19:07:25 STEP 1310 H= 327:30 +CPU= 13.715
  19:07:39 STEP 1311 H= 327:45 +CPU= 13.973
  19:07:53 STEP 1312 H= 328:00 +CPU= 14.796
  19:08:08 STEP 1313 H= 328:15 +CPU= 14.193
  19:08:22 STEP 1314 H= 328:30 +CPU= 14.239
  19:08:36 STEP 1315 H= 328:45 +CPU= 14.490
  19:08:50 STEP 1316 H= 329:00 +CPU= 13.982
  19:09:05 STEP 1317 H= 329:15 +CPU= 14.329
  19:09:19 STEP 1318 H= 329:30 +CPU= 14.657
  19:09:33 STEP 1319 H= 329:45 +CPU= 13.539
  19:09:50 STEP 1320 H= 330:00 +CPU= 17.141
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001320
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001320
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001320
  19:10:03 STEP 1321 H= 330:15 +CPU= 13.358
  19:10:17 STEP 1322 H= 330:30 +CPU= 13.263
  19:10:30 STEP 1323 H= 330:45 +CPU= 13.420
  19:10:44 STEP 1324 H= 331:00 +CPU= 13.852
  19:10:58 STEP 1325 H= 331:15 +CPU= 13.679
  19:11:12 STEP 1326 H= 331:30 +CPU= 13.889
  19:11:26 STEP 1327 H= 331:45 +CPU= 13.908
  19:11:40 STEP 1328 H= 332:00 +CPU= 14.044
  19:11:54 STEP 1329 H= 332:15 +CPU= 14.290
  19:12:08 STEP 1330 H= 332:30 +CPU= 13.539
  19:12:21 STEP 1331 H= 332:45 +CPU= 13.557
  19:12:39 STEP 1332 H= 333:00 +CPU= 17.615
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001332
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001332
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001332
  19:12:53 STEP 1333 H= 333:15 +CPU= 13.789
  19:13:06 STEP 1334 H= 333:30 +CPU= 13.552
  19:13:20 STEP 1335 H= 333:45 +CPU= 13.493
  19:13:33 STEP 1336 H= 334:00 +CPU= 13.792
  19:13:47 STEP 1337 H= 334:15 +CPU= 13.619
  19:14:01 STEP 1338 H= 334:30 +CPU= 13.426
  19:14:14 STEP 1339 H= 334:45 +CPU= 13.321
  19:14:27 STEP 1340 H= 335:00 +CPU= 13.554
  19:14:41 STEP 1341 H= 335:15 +CPU= 13.207
  19:14:54 STEP 1342 H= 335:30 +CPU= 13.029
  19:15:07 STEP 1343 H= 335:45 +CPU= 13.030
  19:15:25 STEP 1344 H= 336:00 +CPU= 18.472
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001344
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001344
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001332
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001332
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001332
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0_r1787494828_13.zip
Uploading the intermediate file: upload_file_13.zip
  19:15:38 STEP 1345 H= 336:15 +CPU= 13.007
  19:15:52 STEP 1346 H= 336:30 +CPU= 13.184
  19:16:05 STEP 1347 H= 336:45 +CPU= 13.193
  19:16:18 STEP 1348 H= 337:00 +CPU= 13.711
  19:16:32 STEP 1349 H= 337:15 +CPU= 13.470
  19:16:45 STEP 1350 H= 337:30 +CPU= 13.042
  19:16:58 STEP 1351 H= 337:45 +CPU= 13.015
  19:17:12 STEP 1352 H= 338:00 +CPU= 13.449
  19:17:25 STEP 1353 H= 338:15 +CPU= 13.041
  19:17:38 STEP 1354 H= 338:30 +CPU= 13.020
  19:17:51 STEP 1355 H= 338:45 +CPU= 12.952
  19:18:07 STEP 1356 H= 339:00 +CPU= 16.648
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001356
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001356
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001356
  19:18:20 STEP 1357 H= 339:15 +CPU= 13.050
  19:18:34 STEP 1358 H= 339:30 +CPU= 13.293
  19:18:47 STEP 1359 H= 339:45 +CPU= 13.285
  19:19:01 STEP 1360 H= 340:00 +CPU= 13.498
  19:19:14 STEP 1361 H= 340:15 +CPU= 13.132
  19:19:27 STEP 1362 H= 340:30 +CPU= 13.063
  19:19:40 STEP 1363 H= 340:45 +CPU= 13.045
  19:19:53 STEP 1364 H= 341:00 +CPU= 13.441
  19:20:06 STEP 1365 H= 341:15 +CPU= 13.135
  19:20:20 STEP 1366 H= 341:30 +CPU= 13.124
  19:20:33 STEP 1367 H= 341:45 +CPU= 13.346
  19:20:50 STEP 1368 H= 342:00 +CPU= 16.698
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001368
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001368
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001368
  19:21:03 STEP 1369 H= 342:15 +CPU= 13.297
  19:21:16 STEP 1370 H= 342:30 +CPU= 13.237
  19:21:29 STEP 1371 H= 342:45 +CPU= 13.106
  19:21:43 STEP 1372 H= 343:00 +CPU= 13.621
  19:21:56 STEP 1373 H= 343:15 +CPU= 13.199
  19:22:09 STEP 1374 H= 343:30 +CPU= 13.157
  19:22:23 STEP 1375 H= 343:45 +CPU= 13.117
  19:22:36 STEP 1376 H= 344:00 +CPU= 13.518
  19:22:49 STEP 1377 H= 344:15 +CPU= 13.170
  19:23:02 STEP 1378 H= 344:30 +CPU= 13.058
  19:23:16 STEP 1379 H= 344:45 +CPU= 13.084
  19:23:32 STEP 1380 H= 345:00 +CPU= 16.528
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001380
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001380
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001380
  19:23:45 STEP 1381 H= 345:15 +CPU= 13.218
  19:23:59 STEP 1382 H= 345:30 +CPU= 13.405
  19:24:12 STEP 1383 H= 345:45 +CPU= 13.184
  19:24:25 STEP 1384 H= 346:00 +CPU= 13.357
  19:24:38 STEP 1385 H= 346:15 +CPU= 12.976
  19:24:51 STEP 1386 H= 346:30 +CPU= 13.051
  19:25:04 STEP 1387 H= 346:45 +CPU= 13.045
  19:25:18 STEP 1388 H= 347:00 +CPU= 13.748
  19:25:32 STEP 1389 H= 347:15 +CPU= 13.459
  19:25:45 STEP 1390 H= 347:30 +CPU= 13.360
  19:25:58 STEP 1391 H= 347:45 +CPU= 13.267
  19:26:17 STEP 1392 H= 348:00 +CPU= 18.945
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001392
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001392
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001392
  19:26:30 STEP 1393 H= 348:15 +CPU= 13.057
  19:26:44 STEP 1394 H= 348:30 +CPU= 13.280
  19:26:57 STEP 1395 H= 348:45 +CPU= 13.570
  19:27:12 STEP 1396 H= 349:00 +CPU= 14.705
  19:27:26 STEP 1397 H= 349:15 +CPU= 14.243
  19:27:42 STEP 1398 H= 349:30 +CPU= 15.964
  19:28:08 STEP 1399 H= 349:45 +CPU= 25.624
  19:28:34 STEP 1400 H= 350:00 +CPU= 25.822
  19:28:58 STEP 1401 H= 350:15 +CPU= 24.116
  19:29:21 STEP 1402 H= 350:30 +CPU= 22.640
  19:29:45 STEP 1403 H= 350:45 +CPU= 24.532
  19:30:16 STEP 1404 H= 351:00 +CPU= 30.205
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001404
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001404
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001404
  19:30:41 STEP 1405 H= 351:15 +CPU= 24.700
  19:31:05 STEP 1406 H= 351:30 +CPU= 23.983
  19:31:29 STEP 1407 H= 351:45 +CPU= 24.443
  19:31:54 STEP 1408 H= 352:00 +CPU= 24.436
  19:32:19 STEP 1409 H= 352:15 +CPU= 25.352
  19:32:43 STEP 1410 H= 352:30 +CPU= 24.347
  19:33:08 STEP 1411 H= 352:45 +CPU= 24.167
  19:33:31 STEP 1412 H= 353:00 +CPU= 23.371
  19:33:55 STEP 1413 H= 353:15 +CPU= 23.279
  19:34:20 STEP 1414 H= 353:30 +CPU= 25.132
  19:34:45 STEP 1415 H= 353:45 +CPU= 25.587
  19:35:18 STEP 1416 H= 354:00 +CPU= 32.139
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001416
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001416
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001416
  19:35:43 STEP 1417 H= 354:15 +CPU= 25.217
  19:36:10 STEP 1418 H= 354:30 +CPU= 26.687
  19:36:35 STEP 1419 H= 354:45 +CPU= 24.735
  19:37:00 STEP 1420 H= 355:00 +CPU= 25.690
  19:37:24 STEP 1421 H= 355:15 +CPU= 23.643
  19:37:49 STEP 1422 H= 355:30 +CPU= 24.789
  19:38:15 STEP 1423 H= 355:45 +CPU= 26.239
  19:38:40 STEP 1424 H= 356:00 +CPU= 24.811
  19:39:05 STEP 1425 H= 356:15 +CPU= 24.965
  19:39:30 STEP 1426 H= 356:30 +CPU= 24.415
  19:39:54 STEP 1427 H= 356:45 +CPU= 24.408
  19:40:24 STEP 1428 H= 357:00 +CPU= 29.994
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001428
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001428
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001428
  19:40:49 STEP 1429 H= 357:15 +CPU= 24.941
  19:41:14 STEP 1430 H= 357:30 +CPU= 24.760
  19:41:39 STEP 1431 H= 357:45 +CPU= 24.814
  19:42:05 STEP 1432 H= 358:00 +CPU= 25.720
  19:42:30 STEP 1433 H= 358:15 +CPU= 24.770
  19:42:55 STEP 1434 H= 358:30 +CPU= 25.194
  19:43:19 STEP 1435 H= 358:45 +CPU= 24.059
  19:43:45 STEP 1436 H= 359:00 +CPU= 25.916
  19:44:11 STEP 1437 H= 359:15 +CPU= 25.936
  19:44:37 STEP 1438 H= 359:30 +CPU= 25.739
  19:45:04 STEP 1439 H= 359:45 +CPU= 27.318
  19:45:41 STEP 1440 H= 360:00 +CPU= 36.026
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001440
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001440
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001428
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001428
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001428
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0_r1787494828_14.zip
Uploading the intermediate file: upload_file_14.zip
  19:46:06 STEP 1441 H= 360:15 +CPU= 25.764
  19:46:33 STEP 1442 H= 360:30 +CPU= 26.693
  19:47:00 STEP 1443 H= 360:45 +CPU= 26.537
  19:47:26 STEP 1444 H= 361:00 +CPU= 26.204
  19:47:52 STEP 1445 H= 361:15 +CPU= 25.531
  19:48:17 STEP 1446 H= 361:30 +CPU= 25.304
  19:48:40 STEP 1447 H= 361:45 +CPU= 23.309
  19:49:06 STEP 1448 H= 362:00 +CPU= 25.710
  19:49:32 STEP 1449 H= 362:15 +CPU= 25.644
  19:49:57 STEP 1450 H= 362:30 +CPU= 25.352
  19:50:22 STEP 1451 H= 362:45 +CPU= 24.689
  19:50:54 STEP 1452 H= 363:00 +CPU= 31.873
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001452
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001452
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001452
  19:51:20 STEP 1453 H= 363:15 +CPU= 25.575
  19:51:45 STEP 1454 H= 363:30 +CPU= 25.025
  19:52:10 STEP 1455 H= 363:45 +CPU= 25.043
  19:52:37 STEP 1456 H= 364:00 +CPU= 27.282
  19:53:03 STEP 1457 H= 364:15 +CPU= 25.619
  19:53:29 STEP 1458 H= 364:30 +CPU= 26.208
  19:53:53 STEP 1459 H= 364:45 +CPU= 23.531
  19:54:19 STEP 1460 H= 365:00 +CPU= 25.570
  19:54:43 STEP 1461 H= 365:15 +CPU= 24.451
  19:55:07 STEP 1462 H= 365:30 +CPU= 23.528
  19:55:32 STEP 1463 H= 365:45 +CPU= 25.264
  19:56:03 STEP 1464 H= 366:00 +CPU= 31.132
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001464
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001464
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001464
  19:56:27 STEP 1465 H= 366:15 +CPU= 23.358
  19:56:52 STEP 1466 H= 366:30 +CPU= 24.755
  19:57:18 STEP 1467 H= 366:45 +CPU= 26.075
  19:57:42 STEP 1468 H= 367:00 +CPU= 24.555
  19:58:09 STEP 1469 H= 367:15 +CPU= 26.277
  19:58:33 STEP 1470 H= 367:30 +CPU= 24.683
  19:58:59 STEP 1471 H= 367:45 +CPU= 25.355
  19:59:24 STEP 1472 H= 368:00 +CPU= 25.326
  19:59:49 STEP 1473 H= 368:15 +CPU= 25.025
  20:00:13 STEP 1474 H= 368:30 +CPU= 23.564
  20:00:39 STEP 1475 H= 368:45 +CPU= 25.746
  20:01:10 STEP 1476 H= 369:00 +CPU= 30.969
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001476
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001476
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001476
  20:01:35 STEP 1477 H= 369:15 +CPU= 24.958
  20:02:00 STEP 1478 H= 369:30 +CPU= 24.581
  20:02:24 STEP 1479 H= 369:45 +CPU= 24.686
  20:02:52 STEP 1480 H= 370:00 +CPU= 27.200
  20:03:18 STEP 1481 H= 370:15 +CPU= 26.320
  20:03:45 STEP 1482 H= 370:30 +CPU= 27.158
  20:04:10 STEP 1483 H= 370:45 +CPU= 24.525
  20:04:33 STEP 1484 H= 371:00 +CPU= 23.365
  20:04:58 STEP 1485 H= 371:15 +CPU= 24.694
  20:05:22 STEP 1486 H= 371:30 +CPU= 24.200
  20:05:48 STEP 1487 H= 371:45 +CPU= 25.467
  20:06:23 STEP 1488 H= 372:00 +CPU= 35.146
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001488
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001488
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001488
  20:06:49 STEP 1489 H= 372:15 +CPU= 26.130
  20:07:17 STEP 1490 H= 372:30 +CPU= 27.886
  20:07:43 STEP 1491 H= 372:45 +CPU= 26.022
  20:08:11 STEP 1492 H= 373:00 +CPU= 27.137
  20:08:37 STEP 1493 H= 373:15 +CPU= 26.561
  20:09:03 STEP 1494 H= 373:30 +CPU= 25.969
  20:09:27 STEP 1495 H= 373:45 +CPU= 23.436
  20:09:52 STEP 1496 H= 374:00 +CPU= 25.201
  20:10:16 STEP 1497 H= 374:15 +CPU= 24.125
  20:10:42 STEP 1498 H= 374:30 +CPU= 25.623
  20:11:07 STEP 1499 H= 374:45 +CPU= 24.818
  20:11:38 STEP 1500 H= 375:00 +CPU= 30.688
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001500
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001500
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001500
  20:12:02 STEP 1501 H= 375:15 +CPU= 24.601
  20:12:28 STEP 1502 H= 375:30 +CPU= 25.552
  20:12:53 STEP 1503 H= 375:45 +CPU= 25.032
  20:13:20 STEP 1504 H= 376:00 +CPU= 26.763
  20:13:46 STEP 1505 H= 376:15 +CPU= 26.045
  20:14:11 STEP 1506 H= 376:30 +CPU= 25.280
  20:14:37 STEP 1507 H= 376:45 +CPU= 25.260
  20:15:04 STEP 1508 H= 377:00 +CPU= 26.724
  20:15:30 STEP 1509 H= 377:15 +CPU= 26.794
  20:15:56 STEP 1510 H= 377:30 +CPU= 25.998
  20:16:23 STEP 1511 H= 377:45 +CPU= 26.017
  20:16:55 STEP 1512 H= 378:00 +CPU= 32.660
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001512
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001512
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001512
  20:17:21 STEP 1513 H= 378:15 +CPU= 26.154
  20:17:47 STEP 1514 H= 378:30 +CPU= 25.330
  20:18:13 STEP 1515 H= 378:45 +CPU= 25.871
  20:18:40 STEP 1516 H= 379:00 +CPU= 26.578
  20:19:04 STEP 1517 H= 379:15 +CPU= 24.287
  20:19:29 STEP 1518 H= 379:30 +CPU= 24.577
  20:19:51 STEP 1519 H= 379:45 +CPU= 22.519
  20:20:19 STEP 1520 H= 380:00 +CPU= 27.621
  20:20:46 STEP 1521 H= 380:15 +CPU= 26.688
  20:21:13 STEP 1522 H= 380:30 +CPU= 27.500
  20:21:39 STEP 1523 H= 380:45 +CPU= 26.249
  20:22:14 STEP 1524 H= 381:00 +CPU= 33.989
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001524
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001524
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001524
  20:22:40 STEP 1525 H= 381:15 +CPU= 26.043
  20:23:05 STEP 1526 H= 381:30 +CPU= 25.323
  20:23:31 STEP 1527 H= 381:45 +CPU= 25.682
  20:23:56 STEP 1528 H= 382:00 +CPU= 24.759
  20:24:23 STEP 1529 H= 382:15 +CPU= 27.116
  20:24:49 STEP 1530 H= 382:30 +CPU= 25.969
  20:25:14 STEP 1531 H= 382:45 +CPU= 25.134
  20:25:40 STEP 1532 H= 383:00 +CPU= 26.130
  20:26:05 STEP 1533 H= 383:15 +CPU= 24.251
  20:26:30 STEP 1534 H= 383:30 +CPU= 25.486
  20:26:56 STEP 1535 H= 383:45 +CPU= 25.850
  20:27:32 STEP 1536 H= 384:00 +CPU= 35.660
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001536
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001536
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001524
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001524
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001524
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0_r1787494828_15.zip
Uploading the intermediate file: upload_file_15.zip
  20:27:58 STEP 1537 H= 384:15 +CPU= 25.849
  20:28:25 STEP 1538 H= 384:30 +CPU= 27.375
  20:28:53 STEP 1539 H= 384:45 +CPU= 27.723
  20:29:19 STEP 1540 H= 385:00 +CPU= 25.927
  20:29:44 STEP 1541 H= 385:15 +CPU= 24.735
  20:30:09 STEP 1542 H= 385:30 +CPU= 24.975
  20:30:33 STEP 1543 H= 385:45 +CPU= 23.903
  20:30:58 STEP 1544 H= 386:00 +CPU= 24.921
  20:31:23 STEP 1545 H= 386:15 +CPU= 25.471
  20:31:49 STEP 1546 H= 386:30 +CPU= 25.263
  20:32:13 STEP 1547 H= 386:45 +CPU= 24.235
  20:32:45 STEP 1548 H= 387:00 +CPU= 31.603
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001548
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001548
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001548
  20:33:10 STEP 1549 H= 387:15 +CPU= 24.907
  20:33:36 STEP 1550 H= 387:30 +CPU= 26.205
  20:34:01 STEP 1551 H= 387:45 +CPU= 25.242
  20:34:27 STEP 1552 H= 388:00 +CPU= 25.970
  20:34:54 STEP 1553 H= 388:15 +CPU= 26.367
  20:35:20 STEP 1554 H= 388:30 +CPU= 25.598
  20:35:44 STEP 1555 H= 388:45 +CPU= 24.549
  20:36:09 STEP 1556 H= 389:00 +CPU= 25.024
  20:36:33 STEP 1557 H= 389:15 +CPU= 24.017
  20:36:58 STEP 1558 H= 389:30 +CPU= 24.973
  20:37:25 STEP 1559 H= 389:45 +CPU= 26.866
  20:37:57 STEP 1560 H= 390:00 +CPU= 31.920
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001560
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001560
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001560
  20:38:23 STEP 1561 H= 390:15 +CPU= 25.302
  20:38:48 STEP 1562 H= 390:30 +CPU= 25.381
  20:39:16 STEP 1563 H= 390:45 +CPU= 27.675
  20:39:42 STEP 1564 H= 391:00 +CPU= 26.186
  20:40:08 STEP 1565 H= 391:15 +CPU= 25.816
  20:40:33 STEP 1566 H= 391:30 +CPU= 24.767
  20:40:59 STEP 1567 H= 391:45 +CPU= 25.741
  20:41:27 STEP 1568 H= 392:00 +CPU= 28.001
  20:41:53 STEP 1569 H= 392:15 +CPU= 26.179
  20:42:20 STEP 1570 H= 392:30 +CPU= 26.761
  20:42:46 STEP 1571 H= 392:45 +CPU= 26.414
  20:43:19 STEP 1572 H= 393:00 +CPU= 32.076
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001572
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001572
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001572
  20:43:44 STEP 1573 H= 393:15 +CPU= 25.051
  20:44:09 STEP 1574 H= 393:30 +CPU= 24.821
  20:44:34 STEP 1575 H= 393:45 +CPU= 25.173
  20:45:01 STEP 1576 H= 394:00 +CPU= 27.032
  20:45:28 STEP 1577 H= 394:15 +CPU= 26.513
  20:45:53 STEP 1578 H= 394:30 +CPU= 25.099
  20:46:18 STEP 1579 H= 394:45 +CPU= 25.330
  20:46:44 STEP 1580 H= 395:00 +CPU= 26.116
  20:47:09 STEP 1581 H= 395:15 +CPU= 24.854
  20:47:33 STEP 1582 H= 395:30 +CPU= 24.118
  20:47:59 STEP 1583 H= 395:45 +CPU= 25.945
  20:48:33 STEP 1584 H= 396:00 +CPU= 33.681
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001584
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001584
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001584
  20:48:59 STEP 1585 H= 396:15 +CPU= 26.186
  20:49:25 STEP 1586 H= 396:30 +CPU= 25.473
  20:49:53 STEP 1587 H= 396:45 +CPU= 27.657
  20:50:20 STEP 1588 H= 397:00 +CPU= 27.338
  20:50:45 STEP 1589 H= 397:15 +CPU= 25.309
  20:51:11 STEP 1590 H= 397:30 +CPU= 25.932
  20:51:38 STEP 1591 H= 397:45 +CPU= 26.603
  20:52:04 STEP 1592 H= 398:00 +CPU= 25.765
  20:52:29 STEP 1593 H= 398:15 +CPU= 25.122
  20:52:55 STEP 1594 H= 398:30 +CPU= 25.698
  20:53:21 STEP 1595 H= 398:45 +CPU= 25.644
  20:53:51 STEP 1596 H= 399:00 +CPU= 30.654
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001596
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001596
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001596
  20:54:15 STEP 1597 H= 399:15 +CPU= 23.327
  20:54:37 STEP 1598 H= 399:30 +CPU= 22.301
  20:55:03 STEP 1599 H= 399:45 +CPU= 25.308
  20:55:28 STEP 1600 H= 400:00 +CPU= 25.231
  20:55:54 STEP 1601 H= 400:15 +CPU= 26.026
  20:56:19 STEP 1602 H= 400:30 +CPU= 25.423
  20:56:44 STEP 1603 H= 400:45 +CPU= 24.646
  20:57:11 STEP 1604 H= 401:00 +CPU= 26.687
  20:57:37 STEP 1605 H= 401:15 +CPU= 26.273
  20:58:04 STEP 1606 H= 401:30 +CPU= 26.265
  20:58:28 STEP 1607 H= 401:45 +CPU= 24.205
  20:59:00 STEP 1608 H= 402:00 +CPU= 31.708
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001608
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001608
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001608
  20:59:25 STEP 1609 H= 402:15 +CPU= 25.153
  20:59:49 STEP 1610 H= 402:30 +CPU= 23.801
  21:00:14 STEP 1611 H= 402:45 +CPU= 25.719
  21:00:42 STEP 1612 H= 403:00 +CPU= 27.312
  21:01:08 STEP 1613 H= 403:15 +CPU= 25.881
  21:01:33 STEP 1614 H= 403:30 +CPU= 24.642
  21:01:59 STEP 1615 H= 403:45 +CPU= 26.518
  21:02:25 STEP 1616 H= 404:00 +CPU= 25.877
  21:02:51 STEP 1617 H= 404:15 +CPU= 25.382
  21:03:16 STEP 1618 H= 404:30 +CPU= 25.782
  21:03:44 STEP 1619 H= 404:45 +CPU= 27.286
  21:04:16 STEP 1620 H= 405:00 +CPU= 31.708
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001620
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001620
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001620
  21:04:42 STEP 1621 H= 405:15 +CPU= 26.420
  21:05:06 STEP 1622 H= 405:30 +CPU= 24.178
  21:05:32 STEP 1623 H= 405:45 +CPU= 26.077
  21:05:59 STEP 1624 H= 406:00 +CPU= 26.721
  21:06:24 STEP 1625 H= 406:15 +CPU= 24.950
  21:06:50 STEP 1626 H= 406:30 +CPU= 25.541
  21:07:18 STEP 1627 H= 406:45 +CPU= 27.665
  21:07:46 STEP 1628 H= 407:00 +CPU= 28.195
  21:08:12 STEP 1629 H= 407:15 +CPU= 26.248
  21:08:40 STEP 1630 H= 407:30 +CPU= 27.178
  21:09:07 STEP 1631 H= 407:45 +CPU= 27.167
  21:09:41 STEP 1632 H= 408:00 +CPU= 34.432
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001632
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001632
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001620
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001620
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001620
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0_r1787494828_16.zip
Uploading the intermediate file: upload_file_16.zip
  21:10:09 STEP 1633 H= 408:15 +CPU= 27.244
  21:10:38 STEP 1634 H= 408:30 +CPU= 28.863
  21:11:05 STEP 1635 H= 408:45 +CPU= 27.222
  21:11:32 STEP 1636 H= 409:00 +CPU= 27.004
  21:11:57 STEP 1637 H= 409:15 +CPU= 24.691
  21:12:22 STEP 1638 H= 409:30 +CPU= 25.181
  21:12:48 STEP 1639 H= 409:45 +CPU= 25.646
  21:13:14 STEP 1640 H= 410:00 +CPU= 26.068
  21:13:39 STEP 1641 H= 410:15 +CPU= 25.411
  21:14:05 STEP 1642 H= 410:30 +CPU= 25.100
  21:14:29 STEP 1643 H= 410:45 +CPU= 24.363
  21:15:01 STEP 1644 H= 411:00 +CPU= 31.510
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001644
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001644
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001644
  21:15:26 STEP 1645 H= 411:15 +CPU= 25.781
  21:15:53 STEP 1646 H= 411:30 +CPU= 26.679
  21:16:18 STEP 1647 H= 411:45 +CPU= 25.061
  21:16:45 STEP 1648 H= 412:00 +CPU= 27.060
  21:17:12 STEP 1649 H= 412:15 +CPU= 26.507
  21:17:38 STEP 1650 H= 412:30 +CPU= 26.060
  21:18:04 STEP 1651 H= 412:45 +CPU= 26.205
  21:18:32 STEP 1652 H= 413:00 +CPU= 27.460
  21:18:59 STEP 1653 H= 413:15 +CPU= 27.278
  21:19:25 STEP 1654 H= 413:30 +CPU= 25.243
  21:19:51 STEP 1655 H= 413:45 +CPU= 25.806
  21:20:20 STEP 1656 H= 414:00 +CPU= 29.004
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001656
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001656
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001656
  21:20:45 STEP 1657 H= 414:15 +CPU= 24.975
  21:21:10 STEP 1658 H= 414:30 +CPU= 25.519
  21:21:36 STEP 1659 H= 414:45 +CPU= 25.691
  21:22:02 STEP 1660 H= 415:00 +CPU= 26.148
  21:22:28 STEP 1661 H= 415:15 +CPU= 26.035
  21:22:54 STEP 1662 H= 415:30 +CPU= 25.169
  21:23:19 STEP 1663 H= 415:45 +CPU= 24.998
  21:23:46 STEP 1664 H= 416:00 +CPU= 27.313
  21:24:13 STEP 1665 H= 416:15 +CPU= 26.795
  21:24:38 STEP 1666 H= 416:30 +CPU= 24.729
  21:25:03 STEP 1667 H= 416:45 +CPU= 25.412
  21:25:35 STEP 1668 H= 417:00 +CPU= 31.615
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001668
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001668
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001668
  21:26:03 STEP 1669 H= 417:15 +CPU= 27.897
  21:26:29 STEP 1670 H= 417:30 +CPU= 26.572
  21:26:55 STEP 1671 H= 417:45 +CPU= 25.869
  21:27:20 STEP 1672 H= 418:00 +CPU= 24.251
  21:27:44 STEP 1673 H= 418:15 +CPU= 24.138
  21:28:10 STEP 1674 H= 418:30 +CPU= 25.955
  21:28:36 STEP 1675 H= 418:45 +CPU= 25.913
  21:29:03 STEP 1676 H= 419:00 +CPU= 27.156
  21:29:29 STEP 1677 H= 419:15 +CPU= 26.142
  21:29:54 STEP 1678 H= 419:30 +CPU= 24.634
  21:30:21 STEP 1679 H= 419:45 +CPU= 26.851
  21:30:55 STEP 1680 H= 420:00 +CPU= 34.263
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001680
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001680
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001680
  21:31:21 STEP 1681 H= 420:15 +CPU= 25.482
  21:31:45 STEP 1682 H= 420:30 +CPU= 23.882
  21:32:11 STEP 1683 H= 420:45 +CPU= 25.907
  21:32:38 STEP 1684 H= 421:00 +CPU= 27.249
  21:33:04 STEP 1685 H= 421:15 +CPU= 25.492
  21:33:29 STEP 1686 H= 421:30 +CPU= 25.345
  21:33:55 STEP 1687 H= 421:45 +CPU= 25.566
  21:34:21 STEP 1688 H= 422:00 +CPU= 25.822
  21:34:46 STEP 1689 H= 422:15 +CPU= 24.893
  21:35:12 STEP 1690 H= 422:30 +CPU= 26.101
  21:35:37 STEP 1691 H= 422:45 +CPU= 24.601
  21:36:09 STEP 1692 H= 423:00 +CPU= 31.963
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001692
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001692
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001692
  21:36:35 STEP 1693 H= 423:15 +CPU= 26.056
  21:37:01 STEP 1694 H= 423:30 +CPU= 26.308
  21:37:28 STEP 1695 H= 423:45 +CPU= 26.949
  21:37:54 STEP 1696 H= 424:00 +CPU= 26.044
  21:38:20 STEP 1697 H= 424:15 +CPU= 26.092
  21:38:48 STEP 1698 H= 424:30 +CPU= 27.339
  21:39:14 STEP 1699 H= 424:45 +CPU= 26.278
  21:39:42 STEP 1700 H= 425:00 +CPU= 27.953
  21:40:07 STEP 1701 H= 425:15 +CPU= 24.908
  21:40:34 STEP 1702 H= 425:30 +CPU= 26.711
  21:41:01 STEP 1703 H= 425:45 +CPU= 27.261
  21:41:34 STEP 1704 H= 426:00 +CPU= 33.114
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001704
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001704
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001704
  21:42:02 STEP 1705 H= 426:15 +CPU= 27.310
  21:42:30 STEP 1706 H= 426:30 +CPU= 27.800
  21:42:56 STEP 1707 H= 426:45 +CPU= 25.768
  21:43:21 STEP 1708 H= 427:00 +CPU= 25.356
  21:43:45 STEP 1709 H= 427:15 +CPU= 24.279
  21:44:10 STEP 1710 H= 427:30 +CPU= 24.987
  21:44:35 STEP 1711 H= 427:45 +CPU= 24.883
  21:45:02 STEP 1712 H= 428:00 +CPU= 26.582
  21:45:28 STEP 1713 H= 428:15 +CPU= 26.164
  21:45:54 STEP 1714 H= 428:30 +CPU= 25.691
  21:46:21 STEP 1715 H= 428:45 +CPU= 26.721
  21:46:53 STEP 1716 H= 429:00 +CPU= 32.675
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001716
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001716
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001716
  21:47:21 STEP 1717 H= 429:15 +CPU= 27.571
  21:47:49 STEP 1718 H= 429:30 +CPU= 27.690
  21:48:15 STEP 1719 H= 429:45 +CPU= 26.227
  21:48:42 STEP 1720 H= 430:00 +CPU= 26.803
  21:49:09 STEP 1721 H= 430:15 +CPU= 26.828
  21:49:36 STEP 1722 H= 430:30 +CPU= 27.399
  21:50:03 STEP 1723 H= 430:45 +CPU= 26.742
  21:50:30 STEP 1724 H= 431:00 +CPU= 26.393
  21:50:57 STEP 1725 H= 431:15 +CPU= 26.780
  21:51:23 STEP 1726 H= 431:30 +CPU= 26.103
  21:51:51 STEP 1727 H= 431:45 +CPU= 27.719
  21:52:26 STEP 1728 H= 432:00 +CPU= 35.615
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001728
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001728
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001716
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001716
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001716
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0_r1787494828_17.zip
Uploading the intermediate file: upload_file_17.zip
  21:52:54 STEP 1729 H= 432:15 +CPU= 27.373
  21:53:23 STEP 1730 H= 432:30 +CPU= 28.793
  21:53:51 STEP 1731 H= 432:45 +CPU= 28.153
  21:54:18 STEP 1732 H= 433:00 +CPU= 26.985
  21:54:44 STEP 1733 H= 433:15 +CPU= 26.051
  21:55:10 STEP 1734 H= 433:30 +CPU= 25.531
  21:55:36 STEP 1735 H= 433:45 +CPU= 26.569
  21:56:02 STEP 1736 H= 434:00 +CPU= 25.912
  21:56:28 STEP 1737 H= 434:15 +CPU= 25.542
  21:56:55 STEP 1738 H= 434:30 +CPU= 26.987
  21:57:21 STEP 1739 H= 434:45 +CPU= 26.175
  21:57:55 STEP 1740 H= 435:00 +CPU= 33.444
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001740
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001740
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001740
  21:58:22 STEP 1741 H= 435:15 +CPU= 27.048
  21:58:49 STEP 1742 H= 435:30 +CPU= 27.412
  21:59:17 STEP 1743 H= 435:45 +CPU= 27.079
  21:59:44 STEP 1744 H= 436:00 +CPU= 27.009
  22:00:09 STEP 1745 H= 436:15 +CPU= 25.238
  22:00:36 STEP 1746 H= 436:30 +CPU= 27.012
  22:01:02 STEP 1747 H= 436:45 +CPU= 25.915
  22:01:29 STEP 1748 H= 437:00 +CPU= 27.023
  22:01:55 STEP 1749 H= 437:15 +CPU= 26.358
  22:02:23 STEP 1750 H= 437:30 +CPU= 27.365
  22:02:49 STEP 1751 H= 437:45 +CPU= 26.449
  22:03:22 STEP 1752 H= 438:00 +CPU= 32.877
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001752
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001752
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001752
  22:03:49 STEP 1753 H= 438:15 +CPU= 26.567
  22:04:16 STEP 1754 H= 438:30 +CPU= 26.482
  22:04:43 STEP 1755 H= 438:45 +CPU= 27.279
  22:05:09 STEP 1756 H= 439:00 +CPU= 26.328
  22:05:37 STEP 1757 H= 439:15 +CPU= 27.180
  22:06:04 STEP 1758 H= 439:30 +CPU= 27.145
  22:06:32 STEP 1759 H= 439:45 +CPU= 27.671
  22:06:59 STEP 1760 H= 440:00 +CPU= 27.593
  22:07:24 STEP 1761 H= 440:15 +CPU= 25.125
  22:07:50 STEP 1762 H= 440:30 +CPU= 25.912
  22:08:17 STEP 1763 H= 440:45 +CPU= 26.228
  22:08:51 STEP 1764 H= 441:00 +CPU= 34.028
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001764
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001764
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001764
  22:09:16 STEP 1765 H= 441:15 +CPU= 25.420
  22:09:42 STEP 1766 H= 441:30 +CPU= 25.312
  22:10:08 STEP 1767 H= 441:45 +CPU= 25.806
  22:10:35 STEP 1768 H= 442:00 +CPU= 26.981
  22:11:00 STEP 1769 H= 442:15 +CPU= 25.230
  22:11:25 STEP 1770 H= 442:30 +CPU= 25.191
  22:11:51 STEP 1771 H= 442:45 +CPU= 25.354
  22:12:16 STEP 1772 H= 443:00 +CPU= 24.859
  22:12:43 STEP 1773 H= 443:15 +CPU= 27.640
  22:13:10 STEP 1774 H= 443:30 +CPU= 26.694
  22:13:35 STEP 1775 H= 443:45 +CPU= 25.069
  22:14:11 STEP 1776 H= 444:00 +CPU= 36.073
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001776
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001776
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001776
  22:14:38 STEP 1777 H= 444:15 +CPU= 26.997
  22:15:04 STEP 1778 H= 444:30 +CPU= 25.654
  22:15:31 STEP 1779 H= 444:45 +CPU= 26.349
  22:15:57 STEP 1780 H= 445:00 +CPU= 26.689
  22:16:24 STEP 1781 H= 445:15 +CPU= 27.086
  22:16:51 STEP 1782 H= 445:30 +CPU= 26.772
  22:17:18 STEP 1783 H= 445:45 +CPU= 26.579
  22:17:45 STEP 1784 H= 446:00 +CPU= 26.986
  22:18:12 STEP 1785 H= 446:15 +CPU= 26.616
  22:18:38 STEP 1786 H= 446:30 +CPU= 26.263
  22:19:06 STEP 1787 H= 446:45 +CPU= 27.735
  22:19:38 STEP 1788 H= 447:00 +CPU= 32.301
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001788
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001788
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001788
  22:20:05 STEP 1789 H= 447:15 +CPU= 27.046
  22:20:30 STEP 1790 H= 447:30 +CPU= 24.714
  22:20:55 STEP 1791 H= 447:45 +CPU= 25.262
  22:21:22 STEP 1792 H= 448:00 +CPU= 26.754
  22:21:49 STEP 1793 H= 448:15 +CPU= 26.381
  22:22:15 STEP 1794 H= 448:30 +CPU= 26.339
  22:22:41 STEP 1795 H= 448:45 +CPU= 25.735
  22:23:07 STEP 1796 H= 449:00 +CPU= 25.693
  22:23:32 STEP 1797 H= 449:15 +CPU= 25.645
  22:23:59 STEP 1798 H= 449:30 +CPU= 26.194
  22:24:26 STEP 1799 H= 449:45 +CPU= 27.150
  22:24:57 STEP 1800 H= 450:00 +CPU= 30.684
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001800
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001800
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001800
  22:25:23 STEP 1801 H= 450:15 +CPU= 26.322
  22:25:49 STEP 1802 H= 450:30 +CPU= 25.674
  22:26:17 STEP 1803 H= 450:45 +CPU= 28.387
  22:26:45 STEP 1804 H= 451:00 +CPU= 27.256
  22:27:12 STEP 1805 H= 451:15 +CPU= 27.568
  22:27:41 STEP 1806 H= 451:30 +CPU= 28.335
  22:28:08 STEP 1807 H= 451:45 +CPU= 27.331
  22:28:36 STEP 1808 H= 452:00 +CPU= 27.406
  22:29:00 STEP 1809 H= 452:15 +CPU= 24.721
  22:29:25 STEP 1810 H= 452:30 +CPU= 24.820
  22:29:51 STEP 1811 H= 452:45 +CPU= 26.047
  22:30:24 STEP 1812 H= 453:00 +CPU= 32.596
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001812
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001812
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001812
  22:30:49 STEP 1813 H= 453:15 +CPU= 24.694
  22:31:14 STEP 1814 H= 453:30 +CPU= 24.694
  22:31:39 STEP 1815 H= 453:45 +CPU= 25.399
  22:32:04 STEP 1816 H= 454:00 +CPU= 25.335
  22:32:31 STEP 1817 H= 454:15 +CPU= 27.002
  22:32:58 STEP 1818 H= 454:30 +CPU= 26.931
  22:33:24 STEP 1819 H= 454:45 +CPU= 25.947
  22:33:51 STEP 1820 H= 455:00 +CPU= 26.583
  22:34:19 STEP 1821 H= 455:15 +CPU= 27.441
  22:34:46 STEP 1822 H= 455:30 +CPU= 27.615
  22:35:12 STEP 1823 H= 455:45 +CPU= 25.450
  22:35:48 STEP 1824 H= 456:00 +CPU= 36.454
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001824
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001824
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001812
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001812
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001812
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0_r1787494828_18.zip
Uploading the intermediate file: upload_file_18.zip
  22:36:16 STEP 1825 H= 456:15 +CPU= 27.793
  22:36:44 STEP 1826 H= 456:30 +CPU= 27.986
  22:37:13 STEP 1827 H= 456:45 +CPU= 28.703
  22:37:40 STEP 1828 H= 457:00 +CPU= 26.518
  22:38:06 STEP 1829 H= 457:15 +CPU= 26.127
  22:38:32 STEP 1830 H= 457:30 +CPU= 25.452
  22:38:56 STEP 1831 H= 457:45 +CPU= 24.873
  22:39:21 STEP 1832 H= 458:00 +CPU= 24.871
  22:39:47 STEP 1833 H= 458:15 +CPU= 25.330
  22:40:14 STEP 1834 H= 458:30 +CPU= 26.787
  22:40:39 STEP 1835 H= 458:45 +CPU= 25.075
  22:41:09 STEP 1836 H= 459:00 +CPU= 29.790
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001836
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001836
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001836
  22:41:30 STEP 1837 H= 459:15 +CPU= 20.903
  22:41:55 STEP 1838 H= 459:30 +CPU= 24.946
  22:42:22 STEP 1839 H= 459:45 +CPU= 26.828
  22:42:49 STEP 1840 H= 460:00 +CPU= 27.156
  22:43:14 STEP 1841 H= 460:15 +CPU= 25.149
  22:43:41 STEP 1842 H= 460:30 +CPU= 26.957
  22:44:08 STEP 1843 H= 460:45 +CPU= 26.727
  22:44:35 STEP 1844 H= 461:00 +CPU= 27.350
  22:45:00 STEP 1845 H= 461:15 +CPU= 25.130
  22:45:26 STEP 1846 H= 461:30 +CPU= 25.153
  22:45:49 STEP 1847 H= 461:45 +CPU= 23.375
  22:46:21 STEP 1848 H= 462:00 +CPU= 32.137
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001848
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001848
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001848
  22:46:48 STEP 1849 H= 462:15 +CPU= 26.860
  22:47:11 STEP 1850 H= 462:30 +CPU= 22.146
  22:47:34 STEP 1851 H= 462:45 +CPU= 23.904
  22:48:01 STEP 1852 H= 463:00 +CPU= 26.787
  22:48:26 STEP 1853 H= 463:15 +CPU= 24.309
  22:48:51 STEP 1854 H= 463:30 +CPU= 25.460
  22:49:16 STEP 1855 H= 463:45 +CPU= 24.919
  22:49:42 STEP 1856 H= 464:00 +CPU= 25.820
  22:50:07 STEP 1857 H= 464:15 +CPU= 24.679
  22:50:34 STEP 1858 H= 464:30 +CPU= 26.934
  22:50:59 STEP 1859 H= 464:45 +CPU= 24.973
  22:51:31 STEP 1860 H= 465:00 +CPU= 31.941
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001860
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001860
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001860
  22:51:56 STEP 1861 H= 465:15 +CPU= 25.248
  22:52:23 STEP 1862 H= 465:30 +CPU= 26.495
  22:52:50 STEP 1863 H= 465:45 +CPU= 27.093
  22:53:15 STEP 1864 H= 466:00 +CPU= 25.395
  22:53:42 STEP 1865 H= 466:15 +CPU= 26.269
  22:54:07 STEP 1866 H= 466:30 +CPU= 25.633
  22:54:34 STEP 1867 H= 466:45 +CPU= 26.503
  22:55:01 STEP 1868 H= 467:00 +CPU= 26.829
  22:55:27 STEP 1869 H= 467:15 +CPU= 25.955
  22:55:52 STEP 1870 H= 467:30 +CPU= 24.477
  22:56:18 STEP 1871 H= 467:45 +CPU= 26.248
  22:56:54 STEP 1872 H= 468:00 +CPU= 35.768
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001872
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001872
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001872
  22:57:19 STEP 1873 H= 468:15 +CPU= 24.928
  22:57:42 STEP 1874 H= 468:30 +CPU= 23.667
  22:58:08 STEP 1875 H= 468:45 +CPU= 25.765
  22:58:36 STEP 1876 H= 469:00 +CPU= 27.326
  22:59:03 STEP 1877 H= 469:15 +CPU= 27.211
  22:59:29 STEP 1878 H= 469:30 +CPU= 26.031
  22:59:54 STEP 1879 H= 469:45 +CPU= 25.181
  23:00:20 STEP 1880 H= 470:00 +CPU= 25.833
  23:00:46 STEP 1881 H= 470:15 +CPU= 25.748
  23:01:12 STEP 1882 H= 470:30 +CPU= 26.255
  23:01:38 STEP 1883 H= 470:45 +CPU= 24.973
  23:02:08 STEP 1884 H= 471:00 +CPU= 30.547
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001884
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001884
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001884
  23:02:35 STEP 1885 H= 471:15 +CPU= 27.213
  23:03:03 STEP 1886 H= 471:30 +CPU= 27.878
  23:03:29 STEP 1887 H= 471:45 +CPU= 25.793
  23:03:55 STEP 1888 H= 472:00 +CPU= 25.946
  23:04:20 STEP 1889 H= 472:15 +CPU= 24.951
  23:04:45 STEP 1890 H= 472:30 +CPU= 24.711
  23:05:10 STEP 1891 H= 472:45 +CPU= 25.206
  23:05:36 STEP 1892 H= 473:00 +CPU= 25.976
  23:06:01 STEP 1893 H= 473:15 +CPU= 24.998
  23:06:25 STEP 1894 H= 473:30 +CPU= 23.863
  23:06:50 STEP 1895 H= 473:45 +CPU= 25.076
  23:07:23 STEP 1896 H= 474:00 +CPU= 32.165
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001896
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001896
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001896
  23:07:46 STEP 1897 H= 474:15 +CPU= 23.339
  23:08:13 STEP 1898 H= 474:30 +CPU= 26.775
  23:08:39 STEP 1899 H= 474:45 +CPU= 26.113
  23:09:06 STEP 1900 H= 475:00 +CPU= 27.007
  23:09:33 STEP 1901 H= 475:15 +CPU= 26.880
  23:10:00 STEP 1902 H= 475:30 +CPU= 27.098
  23:10:28 STEP 1903 H= 475:45 +CPU= 27.185
  23:10:55 STEP 1904 H= 476:00 +CPU= 27.243
  23:11:19 STEP 1905 H= 476:15 +CPU= 24.388
  23:11:44 STEP 1906 H= 476:30 +CPU= 24.665
  23:12:11 STEP 1907 H= 476:45 +CPU= 26.694
  23:12:43 STEP 1908 H= 477:00 +CPU= 32.135
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001908
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001908
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001908
  23:13:11 STEP 1909 H= 477:15 +CPU= 28.030
  23:13:39 STEP 1910 H= 477:30 +CPU= 27.275
  23:14:05 STEP 1911 H= 477:45 +CPU= 26.213
  23:14:33 STEP 1912 H= 478:00 +CPU= 28.084
  23:15:00 STEP 1913 H= 478:15 +CPU= 26.533
  23:15:25 STEP 1914 H= 478:30 +CPU= 25.814
  23:15:50 STEP 1915 H= 478:45 +CPU= 24.711
  23:16:16 STEP 1916 H= 479:00 +CPU= 25.544
  23:16:41 STEP 1917 H= 479:15 +CPU= 24.642
  23:17:06 STEP 1918 H= 479:30 +CPU= 25.493
  23:17:33 STEP 1919 H= 479:45 +CPU= 26.379
  23:18:05 STEP 1920 H= 480:00 +CPU= 32.441
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.7662921008E-21      I       0.8109244868E-05      I
ICloud fractionI      0.0000000000E+00      I       0.6751700054E-01      I
I  Relat. Hum. I      0.5661153643E-01      I       0.2551764583E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.8496344083E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    328216996 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS820542.5 Mbytes/s, TOTAL TIME=    0.0080( 5.0%)
 23:18:05 STEP 1920 H= 480:00 +CPU= 32.441
 END CNT3
 NSTEP =  1920 CNT0   000000000  24.442  24.442  24.513   0.997               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        57546/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1008    0        0            0    57546 ************    84.0     0.0    84.0     0.0   182.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.99      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 2082    4671.5    4671.5     18.92      0.00
   9 SPCM     - SPECTRAL COMP.                    1920     861.8     861.8      3.22      0.00
  10 SCAN2M - PHYSICS                             1921   12119.8   12119.8     45.29      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                161     742.9     742.9      0.23      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 83      61.6      61.6      0.01      0.00
  14 SUINIF                                          1    4432.5    4432.5      0.01      0.00
  17 GRIDFPOS IN CNT4                              161      29.8      29.8      0.01      0.00
  18 SUSPECG                                         1    2483.1    2483.1      0.00      0.00
  19 SUSPEC                                          1    2491.1    2491.1      0.00      0.00
  24 SUGRIDU                                         1    1534.6    1534.6      0.00      0.00
  25 SPECRT                                          1     229.6     229.6      0.00      0.00
  26 SUGRIDF                                         1     177.2     177.2      0.00      0.00
  27 RESTART FILES - WRITING                        40    2990.2    2990.2      0.23      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               161       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                               161    5538.7    5538.7      1.73      0.00
  31 POSDDH IN STEPO                                 9       2.8       2.8      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1921    5665.3    5665.3     21.17      0.00
  39 SU0YOMB                                         1     332.8     332.8      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1921      24.6      24.6      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1920     421.2     421.2      1.57      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1920      58.0      58.0      0.22      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 83       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     4165     358.1     358.1      2.90      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      4006     215.1     215.1      1.68      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       4006      80.5      80.5      0.63      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      4165     105.8     105.8      0.86      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      42.6      42.6      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           4165      54.6      54.6      0.44      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           4006      28.9      28.9      0.23      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           4165      77.3      77.3      0.63      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           4006      36.8      36.8      0.29      0.00
 400 GSTATS                                     223224       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                207696       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    51408.7 CPU TIME   51218.7 VECTOR TIME    51218.7


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       33.6
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001920
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001920
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001920
Adding to the zip: /home/pdw/Mo/1/slots/0/NODE.001_01
Adding to the zip: /home/pdw/Mo/1/slots/0/ifs.stat
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001860
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001860
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMUAh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMSHh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12289646/ICMGGh7zg+001860
Zipping up the final file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0_r1787494828_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
23:21:34 (167553): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
15 Jun 2024 22:20:26 1522755 22443233 oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0 1,727,100 51,218 0.0297
15 Jun 2024 15:36:47 1522755 22443233 oifs_43r3_bl_a10n_2016092300_20_1018_12289646_0 864,000 27,088 0.0314


©2024 climateprediction.net