climateprediction.net home page
Task 22449500

Task 22449500

Name oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1
Workunit 12291829
Created 16 Jun 2024, 18:40:51 UTC
Sent 16 Jun 2024, 18:42:07 UTC
Report deadline 15 Aug 2024, 18:42:07 UTC
Received 21 Jun 2024, 17:51:20 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1522755
Run time 14 hours 21 min 50 sec
CPU time 14 hours 15 min 16 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 7.59 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,022.62 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,357.28 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
2:56:58 STEP 1275 H= 318:45 +CPU= 26.814
  12:57:25 STEP 1276 H= 319:00 +CPU= 27.159
  12:57:51 STEP 1277 H= 319:15 +CPU= 26.214
  12:58:19 STEP 1278 H= 319:30 +CPU= 27.453
  12:58:47 STEP 1279 H= 319:45 +CPU= 27.782
  12:59:15 STEP 1280 H= 320:00 +CPU= 28.126
  12:59:42 STEP 1281 H= 320:15 +CPU= 27.216
  13:00:09 STEP 1282 H= 320:30 +CPU= 26.789
  13:00:37 STEP 1283 H= 320:45 +CPU= 27.737
  13:01:10 STEP 1284 H= 321:00 +CPU= 33.438
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001284
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001284
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001284
  13:01:36 STEP 1285 H= 321:15 +CPU= 25.783
  13:02:03 STEP 1286 H= 321:30 +CPU= 26.672
  13:02:29 STEP 1287 H= 321:45 +CPU= 25.742
  13:02:56 STEP 1288 H= 322:00 +CPU= 27.512
  13:03:23 STEP 1289 H= 322:15 +CPU= 26.680
  13:03:50 STEP 1290 H= 322:30 +CPU= 26.995
  13:04:16 STEP 1291 H= 322:45 +CPU= 25.639
  13:04:41 STEP 1292 H= 323:00 +CPU= 24.608
  13:05:06 STEP 1293 H= 323:15 +CPU= 24.891
  13:05:31 STEP 1294 H= 323:30 +CPU= 25.566
  13:05:56 STEP 1295 H= 323:45 +CPU= 24.698
  13:06:30 STEP 1296 H= 324:00 +CPU= 33.449
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001296
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001296
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001296
  13:06:55 STEP 1297 H= 324:15 +CPU= 25.183
  13:07:21 STEP 1298 H= 324:30 +CPU= 25.836
  13:07:48 STEP 1299 H= 324:45 +CPU= 26.795
  13:08:16 STEP 1300 H= 325:00 +CPU= 27.997
  13:08:42 STEP 1301 H= 325:15 +CPU= 25.742
  13:09:07 STEP 1302 H= 325:30 +CPU= 25.800
  13:09:33 STEP 1303 H= 325:45 +CPU= 25.345
  13:10:00 STEP 1304 H= 326:00 +CPU= 26.615
  13:10:25 STEP 1305 H= 326:15 +CPU= 25.909
  13:10:52 STEP 1306 H= 326:30 +CPU= 26.832
  13:11:19 STEP 1307 H= 326:45 +CPU= 26.038
  13:11:51 STEP 1308 H= 327:00 +CPU= 32.452
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001308
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001308
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001308
  13:12:17 STEP 1309 H= 327:15 +CPU= 25.715
  13:12:44 STEP 1310 H= 327:30 +CPU= 26.632
  13:13:09 STEP 1311 H= 327:45 +CPU= 25.343
  13:13:36 STEP 1312 H= 328:00 +CPU= 26.579
  13:14:03 STEP 1313 H= 328:15 +CPU= 27.074
  13:14:28 STEP 1314 H= 328:30 +CPU= 25.344
  13:14:54 STEP 1315 H= 328:45 +CPU= 25.129
  13:15:21 STEP 1316 H= 329:00 +CPU= 27.019
  13:15:46 STEP 1317 H= 329:15 +CPU= 25.580
  13:16:11 STEP 1318 H= 329:30 +CPU= 24.442
  13:16:37 STEP 1319 H= 329:45 +CPU= 25.859
  13:17:10 STEP 1320 H= 330:00 +CPU= 32.883
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001320
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001320
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001320
  13:17:36 STEP 1321 H= 330:15 +CPU= 25.774
  13:18:02 STEP 1322 H= 330:30 +CPU= 26.559
  13:18:28 STEP 1323 H= 330:45 +CPU= 25.741
  13:18:54 STEP 1324 H= 331:00 +CPU= 25.479
  13:19:19 STEP 1325 H= 331:15 +CPU= 25.706
  13:19:47 STEP 1326 H= 331:30 +CPU= 26.977
  13:20:12 STEP 1327 H= 331:45 +CPU= 25.627
  13:20:40 STEP 1328 H= 332:00 +CPU= 27.481
  13:21:07 STEP 1329 H= 332:15 +CPU= 26.762
  13:21:34 STEP 1330 H= 332:30 +CPU= 27.090
  13:22:00 STEP 1331 H= 332:45 +CPU= 25.645
  13:22:33 STEP 1332 H= 333:00 +CPU= 33.342
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001332
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001332
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001332
  13:23:00 STEP 1333 H= 333:15 +CPU= 27.265
  13:23:27 STEP 1334 H= 333:30 +CPU= 26.257
  13:23:54 STEP 1335 H= 333:45 +CPU= 27.061
  13:24:22 STEP 1336 H= 334:00 +CPU= 27.733
  13:24:49 STEP 1337 H= 334:15 +CPU= 27.537
  13:25:16 STEP 1338 H= 334:30 +CPU= 26.616
  13:25:43 STEP 1339 H= 334:45 +CPU= 26.569
  13:26:08 STEP 1340 H= 335:00 +CPU= 25.486
  13:26:34 STEP 1341 H= 335:15 +CPU= 25.686
  13:27:00 STEP 1342 H= 335:30 +CPU= 26.240
  13:27:26 STEP 1343 H= 335:45 +CPU= 25.907
  13:28:01 STEP 1344 H= 336:00 +CPU= 34.532
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001344
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001344
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001248
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001260
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001272
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001284
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001296
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001308
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001320
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001332
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001332
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001332
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1_r1433445274_13.zip
Uploading the intermediate file: upload_file_13.zip
  13:28:27 STEP 1345 H= 336:15 +CPU= 26.005
  13:28:54 STEP 1346 H= 336:30 +CPU= 26.513
  13:29:19 STEP 1347 H= 336:45 +CPU= 24.847
  13:29:44 STEP 1348 H= 337:00 +CPU= 25.254
  13:30:10 STEP 1349 H= 337:15 +CPU= 25.829
  13:30:35 STEP 1350 H= 337:30 +CPU= 25.453
  13:31:01 STEP 1351 H= 337:45 +CPU= 25.035
  13:31:26 STEP 1352 H= 338:00 +CPU= 25.653
  13:31:52 STEP 1353 H= 338:15 +CPU= 25.254
  13:32:16 STEP 1354 H= 338:30 +CPU= 24.518
  13:32:42 STEP 1355 H= 338:45 +CPU= 25.942
  13:33:16 STEP 1356 H= 339:00 +CPU= 33.781
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001356
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001356
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001356
  13:33:43 STEP 1357 H= 339:15 +CPU= 26.960
  13:34:10 STEP 1358 H= 339:30 +CPU= 27.071
  13:34:38 STEP 1359 H= 339:45 +CPU= 27.822
  13:35:06 STEP 1360 H= 340:00 +CPU= 27.170
  13:35:32 STEP 1361 H= 340:15 +CPU= 26.036
  13:35:57 STEP 1362 H= 340:30 +CPU= 24.780
  13:36:22 STEP 1363 H= 340:45 +CPU= 25.613
  13:36:49 STEP 1364 H= 341:00 +CPU= 26.229
  13:37:13 STEP 1365 H= 341:15 +CPU= 24.826
  13:37:40 STEP 1366 H= 341:30 +CPU= 25.994
  13:38:05 STEP 1367 H= 341:45 +CPU= 25.257
  13:38:38 STEP 1368 H= 342:00 +CPU= 33.102
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001368
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001368
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001368
  13:39:05 STEP 1369 H= 342:15 +CPU= 27.158
  13:39:32 STEP 1370 H= 342:30 +CPU= 26.475
  13:39:59 STEP 1371 H= 342:45 +CPU= 26.728
  13:40:26 STEP 1372 H= 343:00 +CPU= 27.352
  13:40:53 STEP 1373 H= 343:15 +CPU= 26.208
  13:41:19 STEP 1374 H= 343:30 +CPU= 26.437
  13:41:45 STEP 1375 H= 343:45 +CPU= 25.923
  13:42:11 STEP 1376 H= 344:00 +CPU= 25.911
  13:42:37 STEP 1377 H= 344:15 +CPU= 26.302
  13:43:04 STEP 1378 H= 344:30 +CPU= 26.738
  13:43:31 STEP 1379 H= 344:45 +CPU= 26.985
  13:44:04 STEP 1380 H= 345:00 +CPU= 32.470
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001380
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001380
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001380
  13:44:31 STEP 1381 H= 345:15 +CPU= 26.695
  13:44:57 STEP 1382 H= 345:30 +CPU= 26.656
  13:45:25 STEP 1383 H= 345:45 +CPU= 27.495
  13:45:52 STEP 1384 H= 346:00 +CPU= 27.135
  13:46:19 STEP 1385 H= 346:15 +CPU= 26.569
  13:46:44 STEP 1386 H= 346:30 +CPU= 25.478
  13:47:10 STEP 1387 H= 346:45 +CPU= 25.668
  13:47:37 STEP 1388 H= 347:00 +CPU= 26.423
  13:48:02 STEP 1389 H= 347:15 +CPU= 24.769
  13:48:27 STEP 1390 H= 347:30 +CPU= 25.839
  13:48:53 STEP 1391 H= 347:45 +CPU= 25.676
  13:49:28 STEP 1392 H= 348:00 +CPU= 34.274
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001392
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001392
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001392
  13:49:54 STEP 1393 H= 348:15 +CPU= 26.332
  13:50:20 STEP 1394 H= 348:30 +CPU= 25.945
  13:50:46 STEP 1395 H= 348:45 +CPU= 26.150
  13:51:14 STEP 1396 H= 349:00 +CPU= 27.532
  13:51:40 STEP 1397 H= 349:15 +CPU= 26.061
  13:52:08 STEP 1398 H= 349:30 +CPU= 27.490
  13:52:35 STEP 1399 H= 349:45 +CPU= 27.397
  13:53:02 STEP 1400 H= 350:00 +CPU= 26.919
  13:53:29 STEP 1401 H= 350:15 +CPU= 26.927
  13:53:57 STEP 1402 H= 350:30 +CPU= 27.484
  13:54:24 STEP 1403 H= 350:45 +CPU= 27.571
  13:54:58 STEP 1404 H= 351:00 +CPU= 33.101
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001404
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001404
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001404
  13:55:24 STEP 1405 H= 351:15 +CPU= 26.249
  13:55:52 STEP 1406 H= 351:30 +CPU= 27.949
  13:56:19 STEP 1407 H= 351:45 +CPU= 26.625
  13:56:46 STEP 1408 H= 352:00 +CPU= 27.331
  13:57:14 STEP 1409 H= 352:15 +CPU= 27.543
  13:57:40 STEP 1410 H= 352:30 +CPU= 26.019
  13:58:05 STEP 1411 H= 352:45 +CPU= 25.080
  13:58:31 STEP 1412 H= 353:00 +CPU= 26.150
  13:58:58 STEP 1413 H= 353:15 +CPU= 26.539
  13:59:24 STEP 1414 H= 353:30 +CPU= 25.745
  13:59:49 STEP 1415 H= 353:45 +CPU= 25.053
  14:00:22 STEP 1416 H= 354:00 +CPU= 33.205
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001416
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001416
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001416
  14:00:50 STEP 1417 H= 354:15 +CPU= 27.373
  14:01:17 STEP 1418 H= 354:30 +CPU= 27.089
  14:01:44 STEP 1419 H= 354:45 +CPU= 26.646
  14:02:13 STEP 1420 H= 355:00 +CPU= 29.074
  14:02:40 STEP 1421 H= 355:15 +CPU= 27.098
  14:03:07 STEP 1422 H= 355:30 +CPU= 27.137
  14:03:35 STEP 1423 H= 355:45 +CPU= 27.796
  14:04:02 STEP 1424 H= 356:00 +CPU= 26.690
  14:04:29 STEP 1425 H= 356:15 +CPU= 27.259
  14:04:54 STEP 1426 H= 356:30 +CPU= 24.584
  14:05:19 STEP 1427 H= 356:45 +CPU= 24.743
  14:05:50 STEP 1428 H= 357:00 +CPU= 31.547
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001428
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001428
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001428
  14:06:17 STEP 1429 H= 357:15 +CPU= 26.278
  14:06:42 STEP 1430 H= 357:30 +CPU= 25.013
  14:07:08 STEP 1431 H= 357:45 +CPU= 26.262
  14:07:35 STEP 1432 H= 358:00 +CPU= 27.059
  14:08:02 STEP 1433 H= 358:15 +CPU= 26.132
  14:08:27 STEP 1434 H= 358:30 +CPU= 25.694
  14:08:52 STEP 1435 H= 358:45 +CPU= 24.266
  14:09:19 STEP 1436 H= 359:00 +CPU= 27.166
  14:09:46 STEP 1437 H= 359:15 +CPU= 26.682
  14:10:13 STEP 1438 H= 359:30 +CPU= 26.599
  14:10:38 STEP 1439 H= 359:45 +CPU= 25.585
  14:11:14 STEP 1440 H= 360:00 +CPU= 35.751
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001440
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001440
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001344
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001356
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001368
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001380
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001392
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001404
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001416
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001428
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001428
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001428
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1_r1433445274_14.zip
Uploading the intermediate file: upload_file_14.zip
  14:11:42 STEP 1441 H= 360:15 +CPU= 27.396
  14:12:08 STEP 1442 H= 360:30 +CPU= 26.752
  14:12:36 STEP 1443 H= 360:45 +CPU= 27.767
  14:13:04 STEP 1444 H= 361:00 +CPU= 27.740
  14:13:31 STEP 1445 H= 361:15 +CPU= 27.142
  14:13:59 STEP 1446 H= 361:30 +CPU= 27.583
  14:14:27 STEP 1447 H= 361:45 +CPU= 27.440
  14:14:56 STEP 1448 H= 362:00 +CPU= 29.039
  14:15:23 STEP 1449 H= 362:15 +CPU= 27.080
  14:15:50 STEP 1450 H= 362:30 +CPU= 27.181
  14:16:16 STEP 1451 H= 362:45 +CPU= 26.138
  14:16:47 STEP 1452 H= 363:00 +CPU= 30.884
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001452
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001452
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001452
  14:17:12 STEP 1453 H= 363:15 +CPU= 24.336
  14:17:38 STEP 1454 H= 363:30 +CPU= 26.006
  14:18:05 STEP 1455 H= 363:45 +CPU= 27.091
  14:18:32 STEP 1456 H= 364:00 +CPU= 26.866
  14:18:58 STEP 1457 H= 364:15 +CPU= 26.356
  14:19:26 STEP 1458 H= 364:30 +CPU= 26.936
  14:19:51 STEP 1459 H= 364:45 +CPU= 25.243
  14:20:19 STEP 1460 H= 365:00 +CPU= 28.396
  14:20:47 STEP 1461 H= 365:15 +CPU= 27.885
  14:21:15 STEP 1462 H= 365:30 +CPU= 27.970
  14:21:41 STEP 1463 H= 365:45 +CPU= 25.577
  14:22:14 STEP 1464 H= 366:00 +CPU= 32.648
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001464
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001464
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001464
  14:22:41 STEP 1465 H= 366:15 +CPU= 26.697
  14:23:07 STEP 1466 H= 366:30 +CPU= 26.548
  14:23:33 STEP 1467 H= 366:45 +CPU= 25.830
  14:23:59 STEP 1468 H= 367:00 +CPU= 25.385
  14:24:24 STEP 1469 H= 367:15 +CPU= 25.771
  14:24:51 STEP 1470 H= 367:30 +CPU= 26.558
  14:25:19 STEP 1471 H= 367:45 +CPU= 27.942
  14:25:45 STEP 1472 H= 368:00 +CPU= 26.046
  14:26:12 STEP 1473 H= 368:15 +CPU= 26.361
  14:26:38 STEP 1474 H= 368:30 +CPU= 26.159
  14:27:04 STEP 1475 H= 368:45 +CPU= 25.806
  14:27:36 STEP 1476 H= 369:00 +CPU= 31.851
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001476
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001476
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001476
  14:28:02 STEP 1477 H= 369:15 +CPU= 26.346
  14:28:27 STEP 1478 H= 369:30 +CPU= 24.266
  14:28:52 STEP 1479 H= 369:45 +CPU= 25.161
  14:29:19 STEP 1480 H= 370:00 +CPU= 27.560
  14:29:47 STEP 1481 H= 370:15 +CPU= 27.729
  14:30:13 STEP 1482 H= 370:30 +CPU= 25.976
  14:30:39 STEP 1483 H= 370:45 +CPU= 25.666
  14:31:06 STEP 1484 H= 371:00 +CPU= 27.157
  14:31:33 STEP 1485 H= 371:15 +CPU= 26.053
  14:32:00 STEP 1486 H= 371:30 +CPU= 27.321
  14:32:26 STEP 1487 H= 371:45 +CPU= 26.044
  14:33:03 STEP 1488 H= 372:00 +CPU= 36.864
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001488
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001488
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001488
  14:33:29 STEP 1489 H= 372:15 +CPU= 25.552
  14:33:55 STEP 1490 H= 372:30 +CPU= 25.851
  14:34:21 STEP 1491 H= 372:45 +CPU= 26.312
  14:34:48 STEP 1492 H= 373:00 +CPU= 27.241
  14:35:15 STEP 1493 H= 373:15 +CPU= 26.536
  14:35:41 STEP 1494 H= 373:30 +CPU= 26.093
  14:36:07 STEP 1495 H= 373:45 +CPU= 25.812
  14:36:34 STEP 1496 H= 374:00 +CPU= 26.465
  14:37:00 STEP 1497 H= 374:15 +CPU= 26.503
  14:37:28 STEP 1498 H= 374:30 +CPU= 28.037
  14:37:55 STEP 1499 H= 374:45 +CPU= 26.605
  14:38:26 STEP 1500 H= 375:00 +CPU= 31.381
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001500
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001500
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001500
  14:38:52 STEP 1501 H= 375:15 +CPU= 25.893
  14:39:19 STEP 1502 H= 375:30 +CPU= 26.522
  14:39:46 STEP 1503 H= 375:45 +CPU= 26.922
  14:40:14 STEP 1504 H= 376:00 +CPU= 28.054
  14:40:41 STEP 1505 H= 376:15 +CPU= 27.081
  14:41:07 STEP 1506 H= 376:30 +CPU= 26.033
  14:41:34 STEP 1507 H= 376:45 +CPU= 26.161
  14:42:01 STEP 1508 H= 377:00 +CPU= 27.284
  14:42:28 STEP 1509 H= 377:15 +CPU= 26.844
  14:42:54 STEP 1510 H= 377:30 +CPU= 26.067
  14:43:20 STEP 1511 H= 377:45 +CPU= 26.217
  14:43:54 STEP 1512 H= 378:00 +CPU= 33.692
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001512
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001512
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001512
  14:44:20 STEP 1513 H= 378:15 +CPU= 25.916
  14:44:47 STEP 1514 H= 378:30 +CPU= 26.470
  14:45:12 STEP 1515 H= 378:45 +CPU= 25.429
  14:45:39 STEP 1516 H= 379:00 +CPU= 26.912
  14:46:05 STEP 1517 H= 379:15 +CPU= 25.687
  14:46:31 STEP 1518 H= 379:30 +CPU= 25.342
  14:46:57 STEP 1519 H= 379:45 +CPU= 26.403
  14:47:24 STEP 1520 H= 380:00 +CPU= 27.149
  14:47:50 STEP 1521 H= 380:15 +CPU= 25.485
  14:48:16 STEP 1522 H= 380:30 +CPU= 26.286
  14:48:44 STEP 1523 H= 380:45 +CPU= 27.265
  14:49:16 STEP 1524 H= 381:00 +CPU= 32.774
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001524
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001524
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001524
  14:49:44 STEP 1525 H= 381:15 +CPU= 27.739
  14:50:11 STEP 1526 H= 381:30 +CPU= 26.434
  14:50:38 STEP 1527 H= 381:45 +CPU= 27.127
  14:51:05 STEP 1528 H= 382:00 +CPU= 27.335
  14:51:32 STEP 1529 H= 382:15 +CPU= 26.464
  14:51:59 STEP 1530 H= 382:30 +CPU= 27.349
  14:52:25 STEP 1531 H= 382:45 +CPU= 25.752
  14:52:52 STEP 1532 H= 383:00 +CPU= 26.239
  14:53:17 STEP 1533 H= 383:15 +CPU= 25.406
  14:53:43 STEP 1534 H= 383:30 +CPU= 26.096
  14:54:10 STEP 1535 H= 383:45 +CPU= 26.653
  14:54:46 STEP 1536 H= 384:00 +CPU= 36.140
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001536
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001536
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001440
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001452
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001464
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001476
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001488
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001500
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001512
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001524
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001524
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001524
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1_r1433445274_15.zip
Uploading the intermediate file: upload_file_15.zip
  14:55:13 STEP 1537 H= 384:15 +CPU= 26.886
  14:55:39 STEP 1538 H= 384:30 +CPU= 25.428
  14:56:05 STEP 1539 H= 384:45 +CPU= 26.017
  14:56:31 STEP 1540 H= 385:00 +CPU= 26.188
  14:56:57 STEP 1541 H= 385:15 +CPU= 26.125
  14:57:23 STEP 1542 H= 385:30 +CPU= 25.718
  14:57:48 STEP 1543 H= 385:45 +CPU= 24.593
  14:58:14 STEP 1544 H= 386:00 +CPU= 26.007
  14:58:40 STEP 1545 H= 386:15 +CPU= 25.586
  14:59:06 STEP 1546 H= 386:30 +CPU= 26.272
  14:59:32 STEP 1547 H= 386:45 +CPU= 25.799
  15:00:05 STEP 1548 H= 387:00 +CPU= 32.572
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001548
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001548
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001548
  15:00:30 STEP 1549 H= 387:15 +CPU= 24.908
  15:00:54 STEP 1550 H= 387:30 +CPU= 24.864
  15:01:20 STEP 1551 H= 387:45 +CPU= 25.637
  15:01:47 STEP 1552 H= 388:00 +CPU= 26.988
  15:02:13 STEP 1553 H= 388:15 +CPU= 25.646
  15:02:40 STEP 1554 H= 388:30 +CPU= 26.958
  15:03:07 STEP 1555 H= 388:45 +CPU= 27.036
  15:03:34 STEP 1556 H= 389:00 +CPU= 27.031
  15:04:02 STEP 1557 H= 389:15 +CPU= 27.530
  15:04:29 STEP 1558 H= 389:30 +CPU= 26.530
  15:04:55 STEP 1559 H= 389:45 +CPU= 26.497
  15:05:28 STEP 1560 H= 390:00 +CPU= 33.203
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001560
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001560
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001560
  15:05:57 STEP 1561 H= 390:15 +CPU= 28.216
  15:06:24 STEP 1562 H= 390:30 +CPU= 27.009
  15:06:51 STEP 1563 H= 390:45 +CPU= 27.361
  15:07:20 STEP 1564 H= 391:00 +CPU= 28.379
  15:07:47 STEP 1565 H= 391:15 +CPU= 27.128
  15:08:13 STEP 1566 H= 391:30 +CPU= 25.901
  15:08:40 STEP 1567 H= 391:45 +CPU= 27.092
  15:09:09 STEP 1568 H= 392:00 +CPU= 28.201
  15:09:35 STEP 1569 H= 392:15 +CPU= 26.709
  15:10:02 STEP 1570 H= 392:30 +CPU= 26.986
  15:10:29 STEP 1571 H= 392:45 +CPU= 26.432
  15:11:03 STEP 1572 H= 393:00 +CPU= 33.518
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001572
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001572
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001572
  15:11:30 STEP 1573 H= 393:15 +CPU= 27.575
  15:11:58 STEP 1574 H= 393:30 +CPU= 27.584
  15:12:26 STEP 1575 H= 393:45 +CPU= 27.924
  15:12:54 STEP 1576 H= 394:00 +CPU= 27.957
  15:13:21 STEP 1577 H= 394:15 +CPU= 26.831
  15:13:46 STEP 1578 H= 394:30 +CPU= 24.815
  15:14:12 STEP 1579 H= 394:45 +CPU= 26.491
  15:14:39 STEP 1580 H= 395:00 +CPU= 26.188
  15:15:04 STEP 1581 H= 395:15 +CPU= 24.951
  15:15:30 STEP 1582 H= 395:30 +CPU= 25.875
  15:15:56 STEP 1583 H= 395:45 +CPU= 26.570
  15:16:33 STEP 1584 H= 396:00 +CPU= 36.469
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001584
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001584
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001584
  15:17:01 STEP 1585 H= 396:15 +CPU= 27.598
  15:17:29 STEP 1586 H= 396:30 +CPU= 28.326
  15:17:56 STEP 1587 H= 396:45 +CPU= 27.226
  15:18:24 STEP 1588 H= 397:00 +CPU= 27.674
  15:18:51 STEP 1589 H= 397:15 +CPU= 27.184
  15:19:18 STEP 1590 H= 397:30 +CPU= 26.513
  15:19:44 STEP 1591 H= 397:45 +CPU= 26.040
  15:20:12 STEP 1592 H= 398:00 +CPU= 27.747
  15:20:40 STEP 1593 H= 398:15 +CPU= 28.007
  15:21:04 STEP 1594 H= 398:30 +CPU= 24.285
  15:21:28 STEP 1595 H= 398:45 +CPU= 24.006
  15:22:00 STEP 1596 H= 399:00 +CPU= 31.014
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001596
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001596
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001596
  15:22:28 STEP 1597 H= 399:15 +CPU= 28.734
  15:22:56 STEP 1598 H= 399:30 +CPU= 27.411
  15:23:23 STEP 1599 H= 399:45 +CPU= 27.281
  15:23:51 STEP 1600 H= 400:00 +CPU= 27.583
  15:24:17 STEP 1601 H= 400:15 +CPU= 25.904
  15:24:44 STEP 1602 H= 400:30 +CPU= 27.111
  15:25:10 STEP 1603 H= 400:45 +CPU= 25.235
  15:25:36 STEP 1604 H= 401:00 +CPU= 26.140
  15:26:00 STEP 1605 H= 401:15 +CPU= 24.429
  15:26:27 STEP 1606 H= 401:30 +CPU= 26.288
  15:26:52 STEP 1607 H= 401:45 +CPU= 25.336
  15:27:24 STEP 1608 H= 402:00 +CPU= 31.963
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001608
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001608
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001608
  15:27:52 STEP 1609 H= 402:15 +CPU= 27.588
  15:28:18 STEP 1610 H= 402:30 +CPU= 26.037
  15:28:44 STEP 1611 H= 402:45 +CPU= 26.328
  15:29:12 STEP 1612 H= 403:00 +CPU= 27.583
  15:29:39 STEP 1613 H= 403:15 +CPU= 27.211
  15:30:06 STEP 1614 H= 403:30 +CPU= 27.047
  15:30:33 STEP 1615 H= 403:45 +CPU= 26.097
  15:31:01 STEP 1616 H= 404:00 +CPU= 27.782
  15:31:28 STEP 1617 H= 404:15 +CPU= 27.051
  15:31:54 STEP 1618 H= 404:30 +CPU= 26.430
  15:32:21 STEP 1619 H= 404:45 +CPU= 26.275
  15:32:53 STEP 1620 H= 405:00 +CPU= 32.845
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001620
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001620
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001620
  15:33:19 STEP 1621 H= 405:15 +CPU= 25.059
  15:33:43 STEP 1622 H= 405:30 +CPU= 24.723
  15:34:10 STEP 1623 H= 405:45 +CPU= 26.117
  15:34:37 STEP 1624 H= 406:00 +CPU= 27.756
  15:35:04 STEP 1625 H= 406:15 +CPU= 26.213
  15:35:30 STEP 1626 H= 406:30 +CPU= 26.461
  15:35:57 STEP 1627 H= 406:45 +CPU= 26.495
  15:36:25 STEP 1628 H= 407:00 +CPU= 28.330
  15:36:53 STEP 1629 H= 407:15 +CPU= 27.436
  15:37:20 STEP 1630 H= 407:30 +CPU= 27.240
  15:37:48 STEP 1631 H= 407:45 +CPU= 28.020
  15:38:25 STEP 1632 H= 408:00 +CPU= 36.613
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001632
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001632
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001536
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001548
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001560
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001572
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001584
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001596
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001608
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001620
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001620
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001620
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1_r1433445274_16.zip
Uploading the intermediate file: upload_file_16.zip
  15:38:52 STEP 1633 H= 408:15 +CPU= 26.741
  15:39:19 STEP 1634 H= 408:30 +CPU= 26.641
  15:39:46 STEP 1635 H= 408:45 +CPU= 27.037
  15:40:13 STEP 1636 H= 409:00 +CPU= 27.393
  15:40:40 STEP 1637 H= 409:15 +CPU= 26.310
  15:41:06 STEP 1638 H= 409:30 +CPU= 26.061
  15:41:32 STEP 1639 H= 409:45 +CPU= 25.808
  15:41:58 STEP 1640 H= 410:00 +CPU= 26.026
  15:42:24 STEP 1641 H= 410:15 +CPU= 25.990
  15:42:50 STEP 1642 H= 410:30 +CPU= 25.703
  15:43:16 STEP 1643 H= 410:45 +CPU= 26.148
  15:43:48 STEP 1644 H= 411:00 +CPU= 31.766
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001644
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001644
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001644
  15:44:13 STEP 1645 H= 411:15 +CPU= 25.578
  15:44:40 STEP 1646 H= 411:30 +CPU= 26.530
  15:45:06 STEP 1647 H= 411:45 +CPU= 26.181
  15:45:33 STEP 1648 H= 412:00 +CPU= 27.141
  15:46:01 STEP 1649 H= 412:15 +CPU= 27.216
  15:46:28 STEP 1650 H= 412:30 +CPU= 27.523
  15:46:55 STEP 1651 H= 412:45 +CPU= 26.924
  15:47:23 STEP 1652 H= 413:00 +CPU= 27.931
  15:47:49 STEP 1653 H= 413:15 +CPU= 26.087
  15:48:16 STEP 1654 H= 413:30 +CPU= 26.750
  15:48:43 STEP 1655 H= 413:45 +CPU= 26.722
  15:49:17 STEP 1656 H= 414:00 +CPU= 33.723
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001656
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001656
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001656
  15:49:44 STEP 1657 H= 414:15 +CPU= 26.819
  15:50:10 STEP 1658 H= 414:30 +CPU= 26.108
  15:50:35 STEP 1659 H= 414:45 +CPU= 24.886
  15:51:00 STEP 1660 H= 415:00 +CPU= 25.286
  15:51:27 STEP 1661 H= 415:15 +CPU= 26.044
  15:51:52 STEP 1662 H= 415:30 +CPU= 25.478
  15:52:18 STEP 1663 H= 415:45 +CPU= 25.976
  15:52:45 STEP 1664 H= 416:00 +CPU= 26.831
  15:53:12 STEP 1665 H= 416:15 +CPU= 26.498
  15:53:39 STEP 1666 H= 416:30 +CPU= 27.093
  15:54:06 STEP 1667 H= 416:45 +CPU= 26.722
  15:54:39 STEP 1668 H= 417:00 +CPU= 33.082
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001668
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001668
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001668
  15:55:05 STEP 1669 H= 417:15 +CPU= 26.494
  15:55:32 STEP 1670 H= 417:30 +CPU= 26.795
  15:56:00 STEP 1671 H= 417:45 +CPU= 27.406
  15:56:26 STEP 1672 H= 418:00 +CPU= 26.601
  15:56:53 STEP 1673 H= 418:15 +CPU= 26.203
  15:57:20 STEP 1674 H= 418:30 +CPU= 26.757
  15:57:47 STEP 1675 H= 418:45 +CPU= 26.850
  15:58:13 STEP 1676 H= 419:00 +CPU= 26.842
  15:58:40 STEP 1677 H= 419:15 +CPU= 26.118
  15:59:07 STEP 1678 H= 419:30 +CPU= 27.301
  15:59:33 STEP 1679 H= 419:45 +CPU= 25.998
  16:00:09 STEP 1680 H= 420:00 +CPU= 36.096
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001680
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001680
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001680
  16:00:37 STEP 1681 H= 420:15 +CPU= 27.442
  16:01:04 STEP 1682 H= 420:30 +CPU= 26.952
  16:01:30 STEP 1683 H= 420:45 +CPU= 26.124
  16:01:57 STEP 1684 H= 421:00 +CPU= 26.699
  16:02:23 STEP 1685 H= 421:15 +CPU= 25.792
  16:02:50 STEP 1686 H= 421:30 +CPU= 27.056
  16:03:17 STEP 1687 H= 421:45 +CPU= 27.371
  16:03:46 STEP 1688 H= 422:00 +CPU= 28.599
  16:04:12 STEP 1689 H= 422:15 +CPU= 25.877
  16:04:39 STEP 1690 H= 422:30 +CPU= 26.630
  16:05:05 STEP 1691 H= 422:45 +CPU= 26.225
  16:05:38 STEP 1692 H= 423:00 +CPU= 32.614
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001692
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001692
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001692
  16:06:03 STEP 1693 H= 423:15 +CPU= 25.108
  16:06:29 STEP 1694 H= 423:30 +CPU= 25.620
  16:06:55 STEP 1695 H= 423:45 +CPU= 26.108
  16:07:21 STEP 1696 H= 424:00 +CPU= 25.688
  16:07:46 STEP 1697 H= 424:15 +CPU= 25.429
  16:08:12 STEP 1698 H= 424:30 +CPU= 26.211
  16:08:38 STEP 1699 H= 424:45 +CPU= 25.914
  16:09:06 STEP 1700 H= 425:00 +CPU= 27.312
  16:09:33 STEP 1701 H= 425:15 +CPU= 26.942
  16:09:59 STEP 1702 H= 425:30 +CPU= 25.627
  16:10:25 STEP 1703 H= 425:45 +CPU= 26.562
  16:10:59 STEP 1704 H= 426:00 +CPU= 33.044
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001704
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001704
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001704
  16:11:26 STEP 1705 H= 426:15 +CPU= 27.014
  16:11:53 STEP 1706 H= 426:30 +CPU= 26.962
  16:12:20 STEP 1707 H= 426:45 +CPU= 27.342
  16:12:49 STEP 1708 H= 427:00 +CPU= 28.806
  16:13:16 STEP 1709 H= 427:15 +CPU= 26.693
  16:13:42 STEP 1710 H= 427:30 +CPU= 26.463
  16:14:09 STEP 1711 H= 427:45 +CPU= 26.568
  16:14:37 STEP 1712 H= 428:00 +CPU= 27.496
  16:15:04 STEP 1713 H= 428:15 +CPU= 27.467
  16:15:31 STEP 1714 H= 428:30 +CPU= 27.145
  16:15:58 STEP 1715 H= 428:45 +CPU= 26.393
  16:16:31 STEP 1716 H= 429:00 +CPU= 32.803
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001716
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001716
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001716
  16:16:59 STEP 1717 H= 429:15 +CPU= 27.657
  16:17:24 STEP 1718 H= 429:30 +CPU= 25.421
  16:17:50 STEP 1719 H= 429:45 +CPU= 25.806
  16:18:16 STEP 1720 H= 430:00 +CPU= 26.015
  16:18:43 STEP 1721 H= 430:15 +CPU= 26.721
  16:19:10 STEP 1722 H= 430:30 +CPU= 27.285
  16:19:34 STEP 1723 H= 430:45 +CPU= 24.050
  16:20:01 STEP 1724 H= 431:00 +CPU= 26.150
  16:20:28 STEP 1725 H= 431:15 +CPU= 27.576
  16:20:54 STEP 1726 H= 431:30 +CPU= 25.982
  16:21:20 STEP 1727 H= 431:45 +CPU= 25.819
  16:21:56 STEP 1728 H= 432:00 +CPU= 35.598
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001728
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001728
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001632
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001644
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001656
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001668
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001680
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001692
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001704
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001716
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001716
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001716
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1_r1433445274_17.zip
Uploading the intermediate file: upload_file_17.zip
  16:22:23 STEP 1729 H= 432:15 +CPU= 27.395
  16:22:51 STEP 1730 H= 432:30 +CPU= 27.158
  16:23:18 STEP 1731 H= 432:45 +CPU= 26.650
  16:23:44 STEP 1732 H= 433:00 +CPU= 26.781
  16:24:10 STEP 1733 H= 433:15 +CPU= 25.939
  16:24:37 STEP 1734 H= 433:30 +CPU= 26.437
  16:25:04 STEP 1735 H= 433:45 +CPU= 26.455
  16:25:30 STEP 1736 H= 434:00 +CPU= 26.013
  16:25:54 STEP 1737 H= 434:15 +CPU= 24.647
  16:26:19 STEP 1738 H= 434:30 +CPU= 24.966
  16:26:46 STEP 1739 H= 434:45 +CPU= 25.959
  16:27:18 STEP 1740 H= 435:00 +CPU= 32.017
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001740
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001740
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001740
  16:27:44 STEP 1741 H= 435:15 +CPU= 26.538
  16:28:11 STEP 1742 H= 435:30 +CPU= 26.262
  16:28:37 STEP 1743 H= 435:45 +CPU= 25.930
  16:29:04 STEP 1744 H= 436:00 +CPU= 27.205
  16:29:30 STEP 1745 H= 436:15 +CPU= 26.282
  16:29:57 STEP 1746 H= 436:30 +CPU= 26.252
  16:30:23 STEP 1747 H= 436:45 +CPU= 26.029
  16:30:51 STEP 1748 H= 437:00 +CPU= 27.925
  16:31:18 STEP 1749 H= 437:15 +CPU= 26.968
  16:31:44 STEP 1750 H= 437:30 +CPU= 25.737
  16:32:11 STEP 1751 H= 437:45 +CPU= 26.882
  16:32:44 STEP 1752 H= 438:00 +CPU= 33.072
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001752
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001752
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001752
  16:33:10 STEP 1753 H= 438:15 +CPU= 26.381
  16:33:37 STEP 1754 H= 438:30 +CPU= 26.644
  16:34:04 STEP 1755 H= 438:45 +CPU= 26.941
  16:34:32 STEP 1756 H= 439:00 +CPU= 28.310
  16:34:59 STEP 1757 H= 439:15 +CPU= 26.888
  16:35:26 STEP 1758 H= 439:30 +CPU= 26.853
  16:35:53 STEP 1759 H= 439:45 +CPU= 26.686
  16:36:20 STEP 1760 H= 440:00 +CPU= 27.082
  16:36:48 STEP 1761 H= 440:15 +CPU= 27.110
  16:37:15 STEP 1762 H= 440:30 +CPU= 27.092
  16:37:40 STEP 1763 H= 440:45 +CPU= 25.266
  16:38:12 STEP 1764 H= 441:00 +CPU= 31.644
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001764
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001764
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001764
  16:38:37 STEP 1765 H= 441:15 +CPU= 25.581
  16:39:03 STEP 1766 H= 441:30 +CPU= 25.194
  16:39:30 STEP 1767 H= 441:45 +CPU= 26.793
  16:39:56 STEP 1768 H= 442:00 +CPU= 26.483
  16:40:22 STEP 1769 H= 442:15 +CPU= 25.955
  16:40:50 STEP 1770 H= 442:30 +CPU= 27.690
  16:41:17 STEP 1771 H= 442:45 +CPU= 27.067
  16:41:44 STEP 1772 H= 443:00 +CPU= 26.975
  16:42:11 STEP 1773 H= 443:15 +CPU= 26.667
  16:42:38 STEP 1774 H= 443:30 +CPU= 26.579
  16:43:03 STEP 1775 H= 443:45 +CPU= 25.589
  16:43:39 STEP 1776 H= 444:00 +CPU= 35.631
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001776
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001776
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001776
  16:44:05 STEP 1777 H= 444:15 +CPU= 25.400
  16:44:29 STEP 1778 H= 444:30 +CPU= 24.517
  16:44:55 STEP 1779 H= 444:45 +CPU= 25.330
  16:45:21 STEP 1780 H= 445:00 +CPU= 25.792
  16:45:47 STEP 1781 H= 445:15 +CPU= 25.940
  16:46:13 STEP 1782 H= 445:30 +CPU= 26.546
  16:46:40 STEP 1783 H= 445:45 +CPU= 26.256
  16:47:07 STEP 1784 H= 446:00 +CPU= 27.520
  16:47:35 STEP 1785 H= 446:15 +CPU= 27.896
  16:48:02 STEP 1786 H= 446:30 +CPU= 26.746
  16:48:30 STEP 1787 H= 446:45 +CPU= 27.365
  16:49:03 STEP 1788 H= 447:00 +CPU= 33.380
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001788
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001788
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001788
  16:49:29 STEP 1789 H= 447:15 +CPU= 26.026
  16:49:56 STEP 1790 H= 447:30 +CPU= 26.432
  16:50:22 STEP 1791 H= 447:45 +CPU= 26.585
  16:50:50 STEP 1792 H= 448:00 +CPU= 27.496
  16:51:16 STEP 1793 H= 448:15 +CPU= 25.852
  16:51:42 STEP 1794 H= 448:30 +CPU= 25.842
  16:52:08 STEP 1795 H= 448:45 +CPU= 25.688
  16:52:34 STEP 1796 H= 449:00 +CPU= 26.708
  16:53:02 STEP 1797 H= 449:15 +CPU= 27.467
  16:53:29 STEP 1798 H= 449:30 +CPU= 27.035
  16:53:56 STEP 1799 H= 449:45 +CPU= 27.042
  16:54:30 STEP 1800 H= 450:00 +CPU= 33.523
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001800
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001800
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001800
  16:54:57 STEP 1801 H= 450:15 +CPU= 26.656
  16:55:23 STEP 1802 H= 450:30 +CPU= 26.187
  16:55:50 STEP 1803 H= 450:45 +CPU= 27.182
  16:56:18 STEP 1804 H= 451:00 +CPU= 27.754
  16:56:45 STEP 1805 H= 451:15 +CPU= 26.528
  16:57:12 STEP 1806 H= 451:30 +CPU= 27.123
  16:57:38 STEP 1807 H= 451:45 +CPU= 25.861
  16:58:04 STEP 1808 H= 452:00 +CPU= 26.347
  16:58:32 STEP 1809 H= 452:15 +CPU= 27.240
  16:58:58 STEP 1810 H= 452:30 +CPU= 26.325
  16:59:25 STEP 1811 H= 452:45 +CPU= 27.091
  17:00:00 STEP 1812 H= 453:00 +CPU= 34.507
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001812
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001812
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001812
  17:00:27 STEP 1813 H= 453:15 +CPU= 27.370
  17:00:54 STEP 1814 H= 453:30 +CPU= 26.625
  17:01:20 STEP 1815 H= 453:45 +CPU= 25.650
  17:01:46 STEP 1816 H= 454:00 +CPU= 26.785
  17:02:12 STEP 1817 H= 454:15 +CPU= 25.563
  17:02:39 STEP 1818 H= 454:30 +CPU= 26.863
  17:03:04 STEP 1819 H= 454:45 +CPU= 24.944
  17:03:30 STEP 1820 H= 455:00 +CPU= 25.326
  17:03:57 STEP 1821 H= 455:15 +CPU= 27.741
  17:04:25 STEP 1822 H= 455:30 +CPU= 27.670
  17:04:53 STEP 1823 H= 455:45 +CPU= 27.848
  17:05:30 STEP 1824 H= 456:00 +CPU= 36.861
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001824
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001824
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001728
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001740
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001752
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001764
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001776
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001788
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001800
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001812
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001812
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001812
Zipping up the intermediate file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1_r1433445274_18.zip
Uploading the intermediate file: upload_file_18.zip
  17:05:57 STEP 1825 H= 456:15 +CPU= 27.108
  17:06:23 STEP 1826 H= 456:30 +CPU= 25.924
  17:06:49 STEP 1827 H= 456:45 +CPU= 25.285
  17:07:15 STEP 1828 H= 457:00 +CPU= 26.642
  17:07:42 STEP 1829 H= 457:15 +CPU= 26.708
  17:08:09 STEP 1830 H= 457:30 +CPU= 26.942
  17:08:35 STEP 1831 H= 457:45 +CPU= 26.073
  17:09:01 STEP 1832 H= 458:00 +CPU= 25.841
  17:09:27 STEP 1833 H= 458:15 +CPU= 25.739
  17:09:52 STEP 1834 H= 458:30 +CPU= 24.908
  17:10:18 STEP 1835 H= 458:45 +CPU= 25.863
  17:10:50 STEP 1836 H= 459:00 +CPU= 31.371
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001836
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001836
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001836
  17:11:16 STEP 1837 H= 459:15 +CPU= 25.822
  17:11:42 STEP 1838 H= 459:30 +CPU= 26.558
  17:12:09 STEP 1839 H= 459:45 +CPU= 26.746
  17:12:37 STEP 1840 H= 460:00 +CPU= 27.456
  17:13:04 STEP 1841 H= 460:15 +CPU= 27.496
  17:13:31 STEP 1842 H= 460:30 +CPU= 26.484
  17:13:58 STEP 1843 H= 460:45 +CPU= 27.459
  17:14:26 STEP 1844 H= 461:00 +CPU= 27.239
  17:14:53 STEP 1845 H= 461:15 +CPU= 27.068
  17:15:20 STEP 1846 H= 461:30 +CPU= 26.986
  17:15:47 STEP 1847 H= 461:45 +CPU= 27.496
  17:16:21 STEP 1848 H= 462:00 +CPU= 33.320
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001848
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001848
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001848
  17:16:48 STEP 1849 H= 462:15 +CPU= 26.602
  17:17:15 STEP 1850 H= 462:30 +CPU= 27.016
  17:17:41 STEP 1851 H= 462:45 +CPU= 26.679
  17:18:10 STEP 1852 H= 463:00 +CPU= 28.271
  17:18:37 STEP 1853 H= 463:15 +CPU= 27.000
  17:19:05 STEP 1854 H= 463:30 +CPU= 28.105
  17:19:32 STEP 1855 H= 463:45 +CPU= 27.190
  17:20:00 STEP 1856 H= 464:00 +CPU= 27.211
  17:20:27 STEP 1857 H= 464:15 +CPU= 27.173
  17:20:54 STEP 1858 H= 464:30 +CPU= 27.332
  17:21:21 STEP 1859 H= 464:45 +CPU= 26.922
  17:21:54 STEP 1860 H= 465:00 +CPU= 32.981
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001860
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001860
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001860
  17:22:22 STEP 1861 H= 465:15 +CPU= 26.967
  17:22:49 STEP 1862 H= 465:30 +CPU= 26.994
  17:23:16 STEP 1863 H= 465:45 +CPU= 27.343
  17:23:43 STEP 1864 H= 466:00 +CPU= 26.879
  17:24:09 STEP 1865 H= 466:15 +CPU= 25.782
  17:24:36 STEP 1866 H= 466:30 +CPU= 26.543
  17:25:02 STEP 1867 H= 466:45 +CPU= 26.376
  17:25:29 STEP 1868 H= 467:00 +CPU= 27.149
  17:25:56 STEP 1869 H= 467:15 +CPU= 26.257
  17:26:21 STEP 1870 H= 467:30 +CPU= 25.770
  17:26:49 STEP 1871 H= 467:45 +CPU= 27.244
  17:27:24 STEP 1872 H= 468:00 +CPU= 35.242
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001872
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001872
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001872
  17:27:51 STEP 1873 H= 468:15 +CPU= 26.529
  17:28:15 STEP 1874 H= 468:30 +CPU= 24.513
  17:28:40 STEP 1875 H= 468:45 +CPU= 25.087
  17:29:05 STEP 1876 H= 469:00 +CPU= 24.523
  17:29:29 STEP 1877 H= 469:15 +CPU= 23.868
  17:29:53 STEP 1878 H= 469:30 +CPU= 23.946
  17:30:17 STEP 1879 H= 469:45 +CPU= 24.344
  17:30:44 STEP 1880 H= 470:00 +CPU= 26.674
  17:31:11 STEP 1881 H= 470:15 +CPU= 26.437
  17:31:37 STEP 1882 H= 470:30 +CPU= 26.036
  17:32:03 STEP 1883 H= 470:45 +CPU= 25.812
  17:32:35 STEP 1884 H= 471:00 +CPU= 32.534
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001884
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001884
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001884
  17:33:03 STEP 1885 H= 471:15 +CPU= 27.233
  17:33:30 STEP 1886 H= 471:30 +CPU= 26.894
  17:33:56 STEP 1887 H= 471:45 +CPU= 26.503
  17:34:23 STEP 1888 H= 472:00 +CPU= 27.010
  17:34:50 STEP 1889 H= 472:15 +CPU= 26.174
  17:35:16 STEP 1890 H= 472:30 +CPU= 26.809
  17:35:42 STEP 1891 H= 472:45 +CPU= 25.367
  17:36:09 STEP 1892 H= 473:00 +CPU= 26.927
  17:36:36 STEP 1893 H= 473:15 +CPU= 26.538
  17:37:02 STEP 1894 H= 473:30 +CPU= 26.150
  17:37:28 STEP 1895 H= 473:45 +CPU= 26.551
  17:38:02 STEP 1896 H= 474:00 +CPU= 33.559
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001896
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001896
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001896
  17:38:28 STEP 1897 H= 474:15 +CPU= 25.602
  17:38:54 STEP 1898 H= 474:30 +CPU= 26.147
  17:39:21 STEP 1899 H= 474:45 +CPU= 26.743
  17:39:50 STEP 1900 H= 475:00 +CPU= 29.309
  17:40:17 STEP 1901 H= 475:15 +CPU= 26.967
  17:40:44 STEP 1902 H= 475:30 +CPU= 26.852
  17:41:12 STEP 1903 H= 475:45 +CPU= 28.086
  17:41:41 STEP 1904 H= 476:00 +CPU= 28.529
  17:42:09 STEP 1905 H= 476:15 +CPU= 27.411
  17:42:37 STEP 1906 H= 476:30 +CPU= 28.110
  17:43:03 STEP 1907 H= 476:45 +CPU= 25.608
  17:43:35 STEP 1908 H= 477:00 +CPU= 32.635
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001908
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001908
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001908
  17:44:02 STEP 1909 H= 477:15 +CPU= 26.702
  17:44:28 STEP 1910 H= 477:30 +CPU= 26.171
  17:44:54 STEP 1911 H= 477:45 +CPU= 25.914
  17:45:21 STEP 1912 H= 478:00 +CPU= 26.080
  17:45:47 STEP 1913 H= 478:15 +CPU= 26.102
  17:46:13 STEP 1914 H= 478:30 +CPU= 26.104
  17:46:39 STEP 1915 H= 478:45 +CPU= 25.767
  17:47:06 STEP 1916 H= 479:00 +CPU= 27.332
  17:47:33 STEP 1917 H= 479:15 +CPU= 26.287
  17:48:00 STEP 1918 H= 479:30 +CPU= 27.137
  17:48:28 STEP 1919 H= 479:45 +CPU= 27.805
  17:49:02 STEP 1920 H= 480:00 +CPU= 34.330
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.6909844106E-21      I      -0.1066214236E-04      I
ICloud fractionI      0.0000000000E+00      I       0.7998673382E-01      I
I  Relat. Hum. I      0.3087620424E+00      I       0.4102506466E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.9903475711E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    316662180 BYTES IN      7 RECORDS TRANSFERRED IN    0.0007 SECONDS452374.5 Mbytes/s, TOTAL TIME=    0.0141( 5.0%)
 17:49:02 STEP 1920 H= 480:00 +CPU= 34.330
 END CNT3
 NSTEP =  1920 CNT0   000000000  25.825  25.825  25.924   0.996               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ            0/           0 WRITTEN        57546/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1   1008    0        0            0    57546 ************    85.1     0.0    85.1     0.0   185.3
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.99      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                 2082    4784.6    4784.6     19.34      0.00
   9 SPCM     - SPECTRAL COMP.                    1920     904.6     904.6      3.37      0.00
  10 SCAN2M - PHYSICS                             1921   11980.6   11980.6     44.69      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                161     768.0     768.0      0.24      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 83      64.5      64.5      0.01      0.00
  14 SUINIF                                          1    4542.6    4542.6      0.01      0.00
  17 GRIDFPOS IN CNT4                              161      29.9      29.9      0.01      0.00
  18 SUSPECG                                         1    2199.9    2199.9      0.00      0.00
  19 SUSPEC                                          1    2206.5    2206.5      0.00      0.00
  24 SUGRIDU                                         1    1755.7    1755.7      0.00      0.00
  25 SPECRT                                          1     188.8     188.8      0.00      0.00
  26 SUGRIDF                                         1     391.5     391.5      0.00      0.00
  27 RESTART FILES - WRITING                        40    3091.3    3091.3      0.24      0.00
  28 RESTART FILES - READING                         1       0.0       0.0      0.00      0.00
  29 SU4FPOS IN CNT4                               161       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                               161    5480.0    5480.0      1.71      0.00
  31 POSDDH IN STEPO                                 9       3.1       3.1      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                   1921    5671.5    5671.5     21.16      0.00
  39 SU0YOMB                                         1     363.7     363.7      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                   1921      25.6      25.6      0.10      0.00
  54 SPCM     - M TO S/S TO M TRANSP.             1920     446.1     446.1      1.66      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.             1920      60.3      60.3      0.22      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 83       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     4165     354.8     354.8      2.87      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      4006     203.2     203.2      1.58      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       4006      78.9      78.9      0.61      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      4165     103.2     103.2      0.83      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      53.1      53.1      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           4165      56.6      56.6      0.46      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           4006      30.3      30.3      0.24      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           4165      82.1      82.1      0.66      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           4006      39.5      39.5      0.31      0.00
 400 GSTATS                                     223224       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                207696       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    51495.1 CPU TIME   51316.8 VECTOR TIME    51316.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       33.6
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMGGh7zg+001920
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMSHh7zg+001920
Moving to projects directory: /home/pdw/Mo/1/slots/0/ICMUAh7zg+001920
Adding to the zip: /home/pdw/Mo/1/slots/0/NODE.001_01
Adding to the zip: /home/pdw/Mo/1/slots/0/ifs.stat
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001896
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001860
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001824
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001860
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001920
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001872
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001908
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001848
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMUAh7zg+001836
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMSHh7zg+001884
Adding to the zip: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_12291829/ICMGGh7zg+001860
Zipping up the final file: /home/pdw/Mo/1/projects/climateprediction.net/oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1_r1433445274_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
17:52:33 (261717): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
21 Jun 2024 16:51:28 1522755 22449500 oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1 1,727,100 51,316 0.0297
21 Jun 2024 09:37:54 1522755 22449500 oifs_43r3_bl_a2pa_2016092300_20_1018_12291829_1 864,000 25,410 0.0294


©2024 climateprediction.net