climateprediction.net home page
Task 22446299

Task 22446299

Name oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0
Workunit 12292690
Created 13 Jun 2024, 2:20:29 UTC
Sent 13 Jun 2024, 23:41:45 UTC
Report deadline 12 Aug 2024, 23:41:45 UTC
Received 14 Jun 2024, 20:50:24 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1507812
Run time 19 hours 45 min 54 sec
CPU time 17 hours 19 min 54 sec
Validate state Valid
Credit 2,635.55
Device peak FLOPS 5.39 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,509.97 MB
Peak swap size 5,981.08 MB
Peak disk usage 1,370.19 MB
Stderr
<core_client_version>7.24.1</core_client_version>
<![CDATA[
<stderr_txt>

  15:45:14 STEP 1405 H= 351:15 +CPU= 35.715
  15:45:53 STEP 1406 H= 351:30 +CPU= 35.770
  15:46:32 STEP 1407 H= 351:45 +CPU= 35.152
  15:47:12 STEP 1408 H= 352:00 +CPU= 35.957
  15:47:51 STEP 1409 H= 352:15 +CPU= 34.835
  15:48:31 STEP 1410 H= 352:30 +CPU= 35.110
  15:49:10 STEP 1411 H= 352:45 +CPU= 35.297
  15:49:49 STEP 1412 H= 353:00 +CPU= 35.363
  15:50:27 STEP 1413 H= 353:15 +CPU= 33.937
  15:51:08 STEP 1414 H= 353:30 +CPU= 34.832
  15:51:49 STEP 1415 H= 353:45 +CPU= 34.440
  15:52:40 STEP 1416 H= 354:00 +CPU= 45.895
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001416
  15:53:22 STEP 1417 H= 354:15 +CPU= 37.420
  15:54:02 STEP 1418 H= 354:30 +CPU= 36.777
  15:54:43 STEP 1419 H= 354:45 +CPU= 36.931
  15:55:25 STEP 1420 H= 355:00 +CPU= 37.438
  15:56:03 STEP 1421 H= 355:15 +CPU= 34.547
  15:56:43 STEP 1422 H= 355:30 +CPU= 36.129
  15:57:23 STEP 1423 H= 355:45 +CPU= 36.101
  15:58:03 STEP 1424 H= 356:00 +CPU= 36.563
  15:58:45 STEP 1425 H= 356:15 +CPU= 37.428
  15:59:24 STEP 1426 H= 356:30 +CPU= 35.458
  16:00:04 STEP 1427 H= 356:45 +CPU= 36.009
  16:00:56 STEP 1428 H= 357:00 +CPU= 45.815
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001428
  16:01:35 STEP 1429 H= 357:15 +CPU= 34.967
  16:02:16 STEP 1430 H= 357:30 +CPU= 37.451
  16:02:56 STEP 1431 H= 357:45 +CPU= 35.670
  16:03:38 STEP 1432 H= 358:00 +CPU= 36.650
  16:04:17 STEP 1433 H= 358:15 +CPU= 35.356
  16:04:59 STEP 1434 H= 358:30 +CPU= 35.340
  16:05:39 STEP 1435 H= 358:45 +CPU= 34.466
  16:06:21 STEP 1436 H= 359:00 +CPU= 36.649
  16:07:01 STEP 1437 H= 359:15 +CPU= 35.594
  16:07:40 STEP 1438 H= 359:30 +CPU= 35.317
  16:08:20 STEP 1439 H= 359:45 +CPU= 35.225
  16:09:34 STEP 1440 H= 360:00 +CPU= 47.646
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001428
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0_r842175855_14.zip
Uploading the intermediate file: upload_file_14.zip
  16:10:14 STEP 1441 H= 360:15 +CPU= 34.405
  16:10:53 STEP 1442 H= 360:30 +CPU= 34.997
  16:11:32 STEP 1443 H= 360:45 +CPU= 35.562
  16:12:13 STEP 1444 H= 361:00 +CPU= 36.772
  16:12:52 STEP 1445 H= 361:15 +CPU= 35.566
  16:13:35 STEP 1446 H= 361:30 +CPU= 36.787
  16:14:16 STEP 1447 H= 361:45 +CPU= 35.244
  16:14:58 STEP 1448 H= 362:00 +CPU= 37.772
  16:15:40 STEP 1449 H= 362:15 +CPU= 37.292
  16:16:21 STEP 1450 H= 362:30 +CPU= 37.312
  16:17:01 STEP 1451 H= 362:45 +CPU= 36.823
  16:17:50 STEP 1452 H= 363:00 +CPU= 44.479
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001452
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001452
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001452
  16:18:29 STEP 1453 H= 363:15 +CPU= 35.499
  16:19:08 STEP 1454 H= 363:30 +CPU= 36.044
  16:19:48 STEP 1455 H= 363:45 +CPU= 35.899
  16:20:29 STEP 1456 H= 364:00 +CPU= 36.640
  16:21:09 STEP 1457 H= 364:15 +CPU= 35.742
  16:21:50 STEP 1458 H= 364:30 +CPU= 36.314
  16:22:29 STEP 1459 H= 364:45 +CPU= 34.705
  16:23:07 STEP 1460 H= 365:00 +CPU= 34.357
  16:23:48 STEP 1461 H= 365:15 +CPU= 35.635
  16:24:29 STEP 1462 H= 365:30 +CPU= 36.727
  16:25:07 STEP 1463 H= 365:45 +CPU= 34.926
  16:25:56 STEP 1464 H= 366:00 +CPU= 44.350
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001464
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001464
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001464
  16:26:36 STEP 1465 H= 366:15 +CPU= 35.370
  16:27:16 STEP 1466 H= 366:30 +CPU= 36.132
  16:27:55 STEP 1467 H= 366:45 +CPU= 35.539
  16:28:35 STEP 1468 H= 367:00 +CPU= 37.151
  16:29:14 STEP 1469 H= 367:15 +CPU= 35.299
  16:29:55 STEP 1470 H= 367:30 +CPU= 36.933
  16:30:36 STEP 1471 H= 367:45 +CPU= 36.746
  16:31:18 STEP 1472 H= 368:00 +CPU= 37.445
  16:32:00 STEP 1473 H= 368:15 +CPU= 37.240
  16:32:40 STEP 1474 H= 368:30 +CPU= 36.352
  16:33:21 STEP 1475 H= 368:45 +CPU= 36.844
  16:34:12 STEP 1476 H= 369:00 +CPU= 45.789
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001476
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001476
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001476
  16:34:52 STEP 1477 H= 369:15 +CPU= 36.779
  16:35:33 STEP 1478 H= 369:30 +CPU= 37.144
  16:36:14 STEP 1479 H= 369:45 +CPU= 37.236
  16:36:54 STEP 1480 H= 370:00 +CPU= 36.083
  16:37:35 STEP 1481 H= 370:15 +CPU= 35.957
  16:38:15 STEP 1482 H= 370:30 +CPU= 35.958
  16:38:55 STEP 1483 H= 370:45 +CPU= 36.403
  16:39:37 STEP 1484 H= 371:00 +CPU= 37.826
  16:40:18 STEP 1485 H= 371:15 +CPU= 36.938
  16:40:58 STEP 1486 H= 371:30 +CPU= 36.621
  16:41:40 STEP 1487 H= 371:45 +CPU= 36.750
  16:42:44 STEP 1488 H= 372:00 +CPU= 48.574
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001488
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001488
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001488
  16:43:25 STEP 1489 H= 372:15 +CPU= 36.290
  16:44:05 STEP 1490 H= 372:30 +CPU= 36.195
  16:44:45 STEP 1491 H= 372:45 +CPU= 35.416
  16:45:26 STEP 1492 H= 373:00 +CPU= 36.385
  16:46:08 STEP 1493 H= 373:15 +CPU= 37.283
  16:46:48 STEP 1494 H= 373:30 +CPU= 36.834
  16:47:30 STEP 1495 H= 373:45 +CPU= 37.017
  16:48:13 STEP 1496 H= 374:00 +CPU= 38.826
  16:48:54 STEP 1497 H= 374:15 +CPU= 37.369
  16:49:35 STEP 1498 H= 374:30 +CPU= 37.086
  16:50:15 STEP 1499 H= 374:45 +CPU= 36.436
  16:51:06 STEP 1500 H= 375:00 +CPU= 44.654
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001500
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001500
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001500
  16:51:47 STEP 1501 H= 375:15 +CPU= 35.369
  16:52:26 STEP 1502 H= 375:30 +CPU= 35.611
  16:53:06 STEP 1503 H= 375:45 +CPU= 35.875
  16:53:46 STEP 1504 H= 376:00 +CPU= 35.933
  16:54:25 STEP 1505 H= 376:15 +CPU= 34.526
  16:55:07 STEP 1506 H= 376:30 +CPU= 36.621
  16:55:46 STEP 1507 H= 376:45 +CPU= 33.390
  16:56:28 STEP 1508 H= 377:00 +CPU= 36.108
  16:57:09 STEP 1509 H= 377:15 +CPU= 36.603
  16:57:50 STEP 1510 H= 377:30 +CPU= 36.564
  16:58:30 STEP 1511 H= 377:45 +CPU= 36.160
  16:59:22 STEP 1512 H= 378:00 +CPU= 46.485
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001512
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001512
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001512
  17:00:03 STEP 1513 H= 378:15 +CPU= 37.332
  17:00:43 STEP 1514 H= 378:30 +CPU= 35.372
  17:01:23 STEP 1515 H= 378:45 +CPU= 36.363
  17:02:04 STEP 1516 H= 379:00 +CPU= 36.018
  17:02:44 STEP 1517 H= 379:15 +CPU= 35.863
  17:03:25 STEP 1518 H= 379:30 +CPU= 37.179
  17:04:08 STEP 1519 H= 379:45 +CPU= 38.036
  17:04:50 STEP 1520 H= 380:00 +CPU= 37.164
  17:05:32 STEP 1521 H= 380:15 +CPU= 37.349
  17:06:11 STEP 1522 H= 380:30 +CPU= 35.653
  17:06:51 STEP 1523 H= 380:45 +CPU= 35.818
  17:07:42 STEP 1524 H= 381:00 +CPU= 45.075
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001524
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001524
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001524
  17:08:22 STEP 1525 H= 381:15 +CPU= 36.161
  17:09:02 STEP 1526 H= 381:30 +CPU= 35.789
  17:09:42 STEP 1527 H= 381:45 +CPU= 35.321
  17:10:23 STEP 1528 H= 382:00 +CPU= 36.829
  17:11:03 STEP 1529 H= 382:15 +CPU= 35.301
  17:11:43 STEP 1530 H= 382:30 +CPU= 34.762
  17:12:23 STEP 1531 H= 382:45 +CPU= 34.227
  17:13:04 STEP 1532 H= 383:00 +CPU= 36.319
  17:13:43 STEP 1533 H= 383:15 +CPU= 35.583
  17:14:23 STEP 1534 H= 383:30 +CPU= 36.063
  17:15:04 STEP 1535 H= 383:45 +CPU= 35.627
  17:16:07 STEP 1536 H= 384:00 +CPU= 49.695
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001536
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001536
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001452
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001464
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001476
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001488
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001500
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001512
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001524
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001524
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0_r842175855_15.zip
Uploading the intermediate file: upload_file_15.zip
  17:16:48 STEP 1537 H= 384:15 +CPU= 35.554
  17:17:29 STEP 1538 H= 384:30 +CPU= 35.695
  17:18:11 STEP 1539 H= 384:45 +CPU= 36.839
  17:18:52 STEP 1540 H= 385:00 +CPU= 34.317
  17:19:33 STEP 1541 H= 385:15 +CPU= 36.121
  17:20:14 STEP 1542 H= 385:30 +CPU= 36.728
  17:20:54 STEP 1543 H= 385:45 +CPU= 35.284
  17:21:34 STEP 1544 H= 386:00 +CPU= 36.956
  17:22:16 STEP 1545 H= 386:15 +CPU= 37.368
  17:22:55 STEP 1546 H= 386:30 +CPU= 35.728
  17:23:34 STEP 1547 H= 386:45 +CPU= 35.644
  17:24:23 STEP 1548 H= 387:00 +CPU= 44.514
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001548
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001548
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001548
  17:25:03 STEP 1549 H= 387:15 +CPU= 36.777
  17:25:43 STEP 1550 H= 387:30 +CPU= 35.925
  17:26:22 STEP 1551 H= 387:45 +CPU= 35.658
  17:27:02 STEP 1552 H= 388:00 +CPU= 36.144
  17:27:42 STEP 1553 H= 388:15 +CPU= 35.689
  17:28:17 STEP 1554 H= 388:30 +CPU= 32.441
  17:28:57 STEP 1555 H= 388:45 +CPU= 35.836
  17:29:39 STEP 1556 H= 389:00 +CPU= 37.959
  17:30:20 STEP 1557 H= 389:15 +CPU= 37.377
  17:31:01 STEP 1558 H= 389:30 +CPU= 36.409
  17:31:42 STEP 1559 H= 389:45 +CPU= 35.929
  17:32:33 STEP 1560 H= 390:00 +CPU= 44.231
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001560
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001560
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001560
  17:33:12 STEP 1561 H= 390:15 +CPU= 35.302
  17:33:52 STEP 1562 H= 390:30 +CPU= 34.487
  17:34:32 STEP 1563 H= 390:45 +CPU= 34.930
  17:35:13 STEP 1564 H= 391:00 +CPU= 35.587
  17:35:53 STEP 1565 H= 391:15 +CPU= 34.883
  17:36:35 STEP 1566 H= 391:30 +CPU= 36.180
  17:37:17 STEP 1567 H= 391:45 +CPU= 36.014
  17:38:02 STEP 1568 H= 392:00 +CPU= 37.075
  17:38:42 STEP 1569 H= 392:15 +CPU= 34.685
  17:39:23 STEP 1570 H= 392:30 +CPU= 34.971
  17:40:02 STEP 1571 H= 392:45 +CPU= 34.625
  17:40:53 STEP 1572 H= 393:00 +CPU= 43.670
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001572
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001572
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001572
  17:41:34 STEP 1573 H= 393:15 +CPU= 35.953
  17:42:15 STEP 1574 H= 393:30 +CPU= 34.854
  17:42:56 STEP 1575 H= 393:45 +CPU= 36.202
  17:43:37 STEP 1576 H= 394:00 +CPU= 35.840
  17:44:19 STEP 1577 H= 394:15 +CPU= 36.233
  17:45:00 STEP 1578 H= 394:30 +CPU= 34.901
  17:45:40 STEP 1579 H= 394:45 +CPU= 34.680
  17:46:21 STEP 1580 H= 395:00 +CPU= 35.823
  17:47:01 STEP 1581 H= 395:15 +CPU= 34.726
  17:47:41 STEP 1582 H= 395:30 +CPU= 34.913
  17:48:21 STEP 1583 H= 395:45 +CPU= 35.239
  17:49:22 STEP 1584 H= 396:00 +CPU= 48.438
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001584
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001584
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001584
  17:50:04 STEP 1585 H= 396:15 +CPU= 36.878
  17:50:45 STEP 1586 H= 396:30 +CPU= 35.247
  17:51:29 STEP 1587 H= 396:45 +CPU= 36.759
  17:52:11 STEP 1588 H= 397:00 +CPU= 36.174
  17:52:52 STEP 1589 H= 397:15 +CPU= 35.727
  17:53:33 STEP 1590 H= 397:30 +CPU= 34.776
  17:54:16 STEP 1591 H= 397:45 +CPU= 36.798
  17:55:01 STEP 1592 H= 398:00 +CPU= 38.060
  17:55:45 STEP 1593 H= 398:15 +CPU= 37.637
  17:56:27 STEP 1594 H= 398:30 +CPU= 36.662
  17:57:09 STEP 1595 H= 398:45 +CPU= 37.031
  17:58:01 STEP 1596 H= 399:00 +CPU= 45.829
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001596
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001596
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001596
  17:58:44 STEP 1597 H= 399:15 +CPU= 36.254
  17:59:27 STEP 1598 H= 399:30 +CPU= 36.601
  18:00:07 STEP 1599 H= 399:45 +CPU= 35.056
  18:00:49 STEP 1600 H= 400:00 +CPU= 34.782
  18:01:30 STEP 1601 H= 400:15 +CPU= 34.274
  18:02:12 STEP 1602 H= 400:30 +CPU= 34.132
  18:02:54 STEP 1603 H= 400:45 +CPU= 35.830
  18:03:36 STEP 1604 H= 401:00 +CPU= 36.001
  18:04:19 STEP 1605 H= 401:15 +CPU= 36.713
  18:05:01 STEP 1606 H= 401:30 +CPU= 35.968
  18:05:42 STEP 1607 H= 401:45 +CPU= 36.311
  18:06:35 STEP 1608 H= 402:00 +CPU= 46.488
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001608
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001608
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001608
  18:07:19 STEP 1609 H= 402:15 +CPU= 37.909
  18:08:02 STEP 1610 H= 402:30 +CPU= 37.281
  18:08:43 STEP 1611 H= 402:45 +CPU= 35.580
  18:09:24 STEP 1612 H= 403:00 +CPU= 36.466
  18:10:05 STEP 1613 H= 403:15 +CPU= 35.401
  18:10:46 STEP 1614 H= 403:30 +CPU= 35.411
  18:11:28 STEP 1615 H= 403:45 +CPU= 37.709
  18:12:12 STEP 1616 H= 404:00 +CPU= 38.353
  18:12:55 STEP 1617 H= 404:15 +CPU= 37.418
  18:13:35 STEP 1618 H= 404:30 +CPU= 35.661
  18:14:15 STEP 1619 H= 404:45 +CPU= 35.682
  18:15:08 STEP 1620 H= 405:00 +CPU= 45.884
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001620
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001620
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001620
  18:15:49 STEP 1621 H= 405:15 +CPU= 36.840
  18:16:32 STEP 1622 H= 405:30 +CPU= 37.503
  18:17:14 STEP 1623 H= 405:45 +CPU= 37.061
  18:17:56 STEP 1624 H= 406:00 +CPU= 37.536
  18:18:38 STEP 1625 H= 406:15 +CPU= 37.168
  18:19:18 STEP 1626 H= 406:30 +CPU= 34.573
  18:20:00 STEP 1627 H= 406:45 +CPU= 36.972
  18:20:43 STEP 1628 H= 407:00 +CPU= 36.850
  18:21:25 STEP 1629 H= 407:15 +CPU= 37.486
  18:22:05 STEP 1630 H= 407:30 +CPU= 35.423
  18:22:46 STEP 1631 H= 407:45 +CPU= 35.772
  18:23:46 STEP 1632 H= 408:00 +CPU= 48.290
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001632
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001632
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001536
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001548
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001560
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001572
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001584
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001596
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001608
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001620
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001620
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0_r842175855_16.zip
Uploading the intermediate file: upload_file_16.zip
  18:24:24 STEP 1633 H= 408:15 +CPU= 32.675
  18:25:09 STEP 1634 H= 408:30 +CPU= 35.969
  18:25:50 STEP 1635 H= 408:45 +CPU= 36.154
  18:26:29 STEP 1636 H= 409:00 +CPU= 35.265
  18:27:09 STEP 1637 H= 409:15 +CPU= 35.097
  18:27:49 STEP 1638 H= 409:30 +CPU= 35.819
  18:28:29 STEP 1639 H= 409:45 +CPU= 36.519
  18:29:09 STEP 1640 H= 410:00 +CPU= 35.569
  18:29:50 STEP 1641 H= 410:15 +CPU= 36.209
  18:30:31 STEP 1642 H= 410:30 +CPU= 35.843
  18:31:10 STEP 1643 H= 410:45 +CPU= 35.109
  18:32:01 STEP 1644 H= 411:00 +CPU= 44.934
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001644
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001644
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001644
  18:32:41 STEP 1645 H= 411:15 +CPU= 35.813
  18:33:20 STEP 1646 H= 411:30 +CPU= 36.042
  18:34:02 STEP 1647 H= 411:45 +CPU= 37.083
  18:34:44 STEP 1648 H= 412:00 +CPU= 36.752
  18:35:26 STEP 1649 H= 412:15 +CPU= 37.442
  18:36:08 STEP 1650 H= 412:30 +CPU= 37.671
  18:36:49 STEP 1651 H= 412:45 +CPU= 36.666
  18:37:32 STEP 1652 H= 413:00 +CPU= 37.872
  18:38:11 STEP 1653 H= 413:15 +CPU= 35.606
  18:38:52 STEP 1654 H= 413:30 +CPU= 37.227
  18:39:34 STEP 1655 H= 413:45 +CPU= 37.396
  18:40:25 STEP 1656 H= 414:00 +CPU= 45.164
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001656
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001656
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001656
  18:41:04 STEP 1657 H= 414:15 +CPU= 35.041
  18:41:44 STEP 1658 H= 414:30 +CPU= 35.345
  18:42:24 STEP 1659 H= 414:45 +CPU= 35.959
  18:43:04 STEP 1660 H= 415:00 +CPU= 36.680
  18:43:46 STEP 1661 H= 415:15 +CPU= 37.446
  18:44:26 STEP 1662 H= 415:30 +CPU= 36.550
  18:45:07 STEP 1663 H= 415:45 +CPU= 36.015
  18:45:47 STEP 1664 H= 416:00 +CPU= 36.285
  18:46:27 STEP 1665 H= 416:15 +CPU= 35.927
  18:47:07 STEP 1666 H= 416:30 +CPU= 36.644
  18:47:48 STEP 1667 H= 416:45 +CPU= 36.790
  18:48:39 STEP 1668 H= 417:00 +CPU= 45.625
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001668
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001668
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001668
  18:49:18 STEP 1669 H= 417:15 +CPU= 35.517
  18:49:58 STEP 1670 H= 417:30 +CPU= 35.715
  18:50:37 STEP 1671 H= 417:45 +CPU= 35.062
  18:51:19 STEP 1672 H= 418:00 +CPU= 37.004
  18:52:00 STEP 1673 H= 418:15 +CPU= 36.531
  18:52:40 STEP 1674 H= 418:30 +CPU= 37.113
  18:53:23 STEP 1675 H= 418:45 +CPU= 37.303
  18:54:04 STEP 1676 H= 419:00 +CPU= 37.103
  18:54:45 STEP 1677 H= 419:15 +CPU= 37.168
  18:55:27 STEP 1678 H= 419:30 +CPU= 37.140
  18:56:06 STEP 1679 H= 419:45 +CPU= 36.200
  18:57:03 STEP 1680 H= 420:00 +CPU= 45.669
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001680
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001680
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001680
  18:57:45 STEP 1681 H= 420:15 +CPU= 36.940
  18:58:25 STEP 1682 H= 420:30 +CPU= 36.178
  18:59:04 STEP 1683 H= 420:45 +CPU= 35.467
  18:59:46 STEP 1684 H= 421:00 +CPU= 37.232
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a3d7
(argv4) batchid: 1018
(argv5) wuid: 12292690
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/3
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/3/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_dc0134b60837b6b9430392c15fa294df to: /var/lib/boinc/slots/3/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/3/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690.zip
ic_ancil_file: ic_ancil_12292690
ifsdata_file: ifsdata_12292690
climate_data_file: clim_data_12292690
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/3/ic_ancil_12292690.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/3/ic_ancil_12292690.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ceea424ba667776af72eb83c7fbbe435 to: /var/lib/boinc/slots/3/ifsdata/ifsdata_12292690.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/3/ifsdata/ifsdata_12292690.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/3/159l_2/clim_data_12292690.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/3/159l_2/clim_data_12292690.zip
Checking for progress XML file: /var/lib/boinc/slots/3/progress_file_12292690.xml
Opened progress file ok : /var/lib/boinc/slots/3/progress_file_12292690.xml
-- Model is restarting --
Adjusting last_iter, 1685, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/3/progress_file_12292690.xml
last_cpu_time: 53581
upload_file_number: 17
last_iter: 1681
last_upload: 1468800
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0_r842175855
The child process has been launched with process id: 377824
Executing the command: /var/lib/boinc/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:377824:377824] [20240614:190136:1718384496:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  19:04:18 STEP 1680 H= 420:00 +CPU= 37.423
  19:04:59 STEP 1681 H= 420:15 +CPU= 37.005
  19:05:40 STEP 1682 H= 420:30 +CPU= 37.307
  19:06:19 STEP 1683 H= 420:45 +CPU= 35.546
  19:07:01 STEP 1684 H= 421:00 +CPU= 37.958
  19:07:41 STEP 1685 H= 421:15 +CPU= 35.662
  19:08:20 STEP 1686 H= 421:30 +CPU= 36.146
  19:09:02 STEP 1687 H= 421:45 +CPU= 37.042
  19:09:42 STEP 1688 H= 422:00 +CPU= 36.264
  19:10:20 STEP 1689 H= 422:15 +CPU= 34.742
  19:11:01 STEP 1690 H= 422:30 +CPU= 36.200
  19:11:42 STEP 1691 H= 422:45 +CPU= 36.710
  19:12:33 STEP 1692 H= 423:00 +CPU= 45.839
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001692
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001692
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001692
  19:13:13 STEP 1693 H= 423:15 +CPU= 36.477
  19:13:54 STEP 1694 H= 423:30 +CPU= 37.295
  19:14:35 STEP 1695 H= 423:45 +CPU= 37.503
  19:15:16 STEP 1696 H= 424:00 +CPU= 37.295
  19:15:56 STEP 1697 H= 424:15 +CPU= 36.962
  19:16:37 STEP 1698 H= 424:30 +CPU= 37.536
  19:17:17 STEP 1699 H= 424:45 +CPU= 35.789
  19:17:59 STEP 1700 H= 425:00 +CPU= 38.254
  19:18:40 STEP 1701 H= 425:15 +CPU= 37.127
  19:19:20 STEP 1702 H= 425:30 +CPU= 36.708
  19:20:02 STEP 1703 H= 425:45 +CPU= 37.778
  19:20:51 STEP 1704 H= 426:00 +CPU= 44.063
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001704
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001704
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001704
  19:21:30 STEP 1705 H= 426:15 +CPU= 35.719
  19:22:10 STEP 1706 H= 426:30 +CPU= 34.892
  19:22:50 STEP 1707 H= 426:45 +CPU= 36.224
  19:23:32 STEP 1708 H= 427:00 +CPU= 37.326
  19:24:11 STEP 1709 H= 427:15 +CPU= 35.371
  19:24:51 STEP 1710 H= 427:30 +CPU= 36.689
  19:25:31 STEP 1711 H= 427:45 +CPU= 36.050
  19:26:14 STEP 1712 H= 428:00 +CPU= 37.766
  19:26:54 STEP 1713 H= 428:15 +CPU= 37.156
  19:27:35 STEP 1714 H= 428:30 +CPU= 36.693
  19:28:15 STEP 1715 H= 428:45 +CPU= 36.266
  19:29:06 STEP 1716 H= 429:00 +CPU= 45.655
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001716
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001716
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001716
  19:29:48 STEP 1717 H= 429:15 +CPU= 37.401
  19:30:29 STEP 1718 H= 429:30 +CPU= 36.981
  19:31:09 STEP 1719 H= 429:45 +CPU= 36.101
  19:31:50 STEP 1720 H= 430:00 +CPU= 36.218
  19:32:30 STEP 1721 H= 430:15 +CPU= 36.605
  19:33:11 STEP 1722 H= 430:30 +CPU= 37.340
  19:33:51 STEP 1723 H= 430:45 +CPU= 36.508
  19:34:31 STEP 1724 H= 431:00 +CPU= 36.174
  19:35:02 STEP 1725 H= 431:15 +CPU= 28.807
  19:35:41 STEP 1726 H= 431:30 +CPU= 32.588
  19:36:20 STEP 1727 H= 431:45 +CPU= 31.180
  19:37:25 STEP 1728 H= 432:00 +CPU= 47.878
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001728
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001728
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001632
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001644
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001656
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001668
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001680
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001692
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001704
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001716
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001716
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0_r842175855_17.zip
Uploading the intermediate file: upload_file_17.zip
  19:38:08 STEP 1729 H= 432:15 +CPU= 36.521
  19:38:49 STEP 1730 H= 432:30 +CPU= 37.374
  19:39:30 STEP 1731 H= 432:45 +CPU= 36.831
  19:40:12 STEP 1732 H= 433:00 +CPU= 38.185
  19:40:52 STEP 1733 H= 433:15 +CPU= 35.844
  19:41:32 STEP 1734 H= 433:30 +CPU= 35.373
  19:42:12 STEP 1735 H= 433:45 +CPU= 35.100
  19:42:52 STEP 1736 H= 434:00 +CPU= 36.101
  19:43:33 STEP 1737 H= 434:15 +CPU= 37.284
  19:44:13 STEP 1738 H= 434:30 +CPU= 37.054
  19:44:54 STEP 1739 H= 434:45 +CPU= 37.373
  19:45:45 STEP 1740 H= 435:00 +CPU= 45.698
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001740
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001740
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001740
  19:46:24 STEP 1741 H= 435:15 +CPU= 36.530
  19:47:04 STEP 1742 H= 435:30 +CPU= 36.651
  19:47:44 STEP 1743 H= 435:45 +CPU= 35.817
  19:48:25 STEP 1744 H= 436:00 +CPU= 37.263
  19:49:04 STEP 1745 H= 436:15 +CPU= 36.245
  19:49:44 STEP 1746 H= 436:30 +CPU= 35.468
  19:50:23 STEP 1747 H= 436:45 +CPU= 35.951
  19:51:03 STEP 1748 H= 437:00 +CPU= 36.224
  19:51:44 STEP 1749 H= 437:15 +CPU= 36.054
  19:52:25 STEP 1750 H= 437:30 +CPU= 37.422
  19:53:06 STEP 1751 H= 437:45 +CPU= 37.439
  19:53:59 STEP 1752 H= 438:00 +CPU= 46.475
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001752
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001752
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001752
  19:54:40 STEP 1753 H= 438:15 +CPU= 37.182
  19:55:21 STEP 1754 H= 438:30 +CPU= 37.192
  19:56:00 STEP 1755 H= 438:45 +CPU= 36.040
  19:56:40 STEP 1756 H= 439:00 +CPU= 36.627
  19:57:21 STEP 1757 H= 439:15 +CPU= 36.661
  19:58:03 STEP 1758 H= 439:30 +CPU= 37.717
  19:58:43 STEP 1759 H= 439:45 +CPU= 36.697
  19:59:24 STEP 1760 H= 440:00 +CPU= 36.734
  20:00:02 STEP 1761 H= 440:15 +CPU= 34.649
  20:00:42 STEP 1762 H= 440:30 +CPU= 34.935
  20:01:21 STEP 1763 H= 440:45 +CPU= 34.965
  20:02:12 STEP 1764 H= 441:00 +CPU= 45.975
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001764
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001764
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001764
  20:02:51 STEP 1765 H= 441:15 +CPU= 35.255
  20:03:33 STEP 1766 H= 441:30 +CPU= 36.808
  20:04:13 STEP 1767 H= 441:45 +CPU= 37.168
  20:04:54 STEP 1768 H= 442:00 +CPU= 37.618
  20:05:35 STEP 1769 H= 442:15 +CPU= 37.028
  20:06:16 STEP 1770 H= 442:30 +CPU= 37.296
  20:06:55 STEP 1771 H= 442:45 +CPU= 35.846
  20:07:24 STEP 1772 H= 443:00 +CPU= 26.920
  20:07:50 STEP 1773 H= 443:15 +CPU= 24.551
  20:08:15 STEP 1774 H= 443:30 +CPU= 23.614
  20:08:40 STEP 1775 H= 443:45 +CPU= 23.835
  20:09:56 STEP 1776 H= 444:00 +CPU= 47.176
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001776
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001776
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001776
  20:10:37 STEP 1777 H= 444:15 +CPU= 36.645
  20:11:18 STEP 1778 H= 444:30 +CPU= 37.036
  20:11:59 STEP 1779 H= 444:45 +CPU= 36.477
  20:12:40 STEP 1780 H= 445:00 +CPU= 37.930
  20:13:20 STEP 1781 H= 445:15 +CPU= 36.685
  20:14:00 STEP 1782 H= 445:30 +CPU= 35.433
  20:14:41 STEP 1783 H= 445:45 +CPU= 36.861
  20:15:21 STEP 1784 H= 446:00 +CPU= 37.095
  20:16:02 STEP 1785 H= 446:15 +CPU= 36.857
  20:16:44 STEP 1786 H= 446:30 +CPU= 37.452
  20:17:25 STEP 1787 H= 446:45 +CPU= 37.497
  20:18:15 STEP 1788 H= 447:00 +CPU= 45.146
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001788
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001788
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001788
  20:18:57 STEP 1789 H= 447:15 +CPU= 37.492
  20:19:36 STEP 1790 H= 447:30 +CPU= 35.943
  20:20:18 STEP 1791 H= 447:45 +CPU= 37.229
  20:20:58 STEP 1792 H= 448:00 +CPU= 36.344
  20:21:39 STEP 1793 H= 448:15 +CPU= 36.632
  20:22:19 STEP 1794 H= 448:30 +CPU= 35.676
  20:22:59 STEP 1795 H= 448:45 +CPU= 36.186
  20:23:39 STEP 1796 H= 449:00 +CPU= 35.918
  20:24:20 STEP 1797 H= 449:15 +CPU= 36.301
  20:25:00 STEP 1798 H= 449:30 +CPU= 36.580
  20:25:40 STEP 1799 H= 449:45 +CPU= 35.297
  20:26:30 STEP 1800 H= 450:00 +CPU= 45.416
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001800
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001800
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001800
  20:27:09 STEP 1801 H= 450:15 +CPU= 36.073
  20:27:49 STEP 1802 H= 450:30 +CPU= 35.823
  20:28:29 STEP 1803 H= 450:45 +CPU= 36.461
  20:29:09 STEP 1804 H= 451:00 +CPU= 36.901
  20:29:51 STEP 1805 H= 451:15 +CPU= 36.826
  20:30:29 STEP 1806 H= 451:30 +CPU= 34.442
  20:31:08 STEP 1807 H= 451:45 +CPU= 35.296
  20:31:49 STEP 1808 H= 452:00 +CPU= 36.429
  20:32:30 STEP 1809 H= 452:15 +CPU= 36.609
  20:33:11 STEP 1810 H= 452:30 +CPU= 37.125
  20:33:51 STEP 1811 H= 452:45 +CPU= 36.096
  20:34:41 STEP 1812 H= 453:00 +CPU= 45.377
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001812
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001812
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001812
  20:35:19 STEP 1813 H= 453:15 +CPU= 34.777
  20:35:59 STEP 1814 H= 453:30 +CPU= 36.084
  20:36:37 STEP 1815 H= 453:45 +CPU= 34.988
  20:37:17 STEP 1816 H= 454:00 +CPU= 35.632
  20:37:56 STEP 1817 H= 454:15 +CPU= 35.209
  20:38:34 STEP 1818 H= 454:30 +CPU= 35.456
  20:39:13 STEP 1819 H= 454:45 +CPU= 34.899
  20:39:52 STEP 1820 H= 455:00 +CPU= 35.316
  20:40:28 STEP 1821 H= 455:15 +CPU= 32.594
  20:41:08 STEP 1822 H= 455:30 +CPU= 33.290
  20:41:49 STEP 1823 H= 455:45 +CPU= 32.577
  20:43:01 STEP 1824 H= 456:00 +CPU= 46.062
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001824
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001824
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001728
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001740
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001752
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001764
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001776
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001788
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001800
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001812
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001812
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0_r842175855_18.zip
Uploading the intermediate file: upload_file_18.zip
  20:43:42 STEP 1825 H= 456:15 +CPU= 32.755
  20:44:22 STEP 1826 H= 456:30 +CPU= 34.926
  20:45:00 STEP 1827 H= 456:45 +CPU= 35.209
  20:45:40 STEP 1828 H= 457:00 +CPU= 35.974
  20:46:19 STEP 1829 H= 457:15 +CPU= 35.676
  20:46:59 STEP 1830 H= 457:30 +CPU= 36.408
  20:47:38 STEP 1831 H= 457:45 +CPU= 34.926
  20:48:20 STEP 1832 H= 458:00 +CPU= 37.723
  20:48:58 STEP 1833 H= 458:15 +CPU= 34.644
  20:49:38 STEP 1834 H= 458:30 +CPU= 35.888
  20:50:18 STEP 1835 H= 458:45 +CPU= 35.567
  20:51:09 STEP 1836 H= 459:00 +CPU= 45.351
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001836
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001836
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001836
  20:51:49 STEP 1837 H= 459:15 +CPU= 34.978
  20:52:29 STEP 1838 H= 459:30 +CPU= 36.152
  20:53:10 STEP 1839 H= 459:45 +CPU= 37.040
  20:53:52 STEP 1840 H= 460:00 +CPU= 37.461
  20:54:33 STEP 1841 H= 460:15 +CPU= 36.339
  20:55:13 STEP 1842 H= 460:30 +CPU= 36.360
  20:55:53 STEP 1843 H= 460:45 +CPU= 36.115
  20:56:35 STEP 1844 H= 461:00 +CPU= 37.466
  20:57:16 STEP 1845 H= 461:15 +CPU= 36.976
  20:57:57 STEP 1846 H= 461:30 +CPU= 36.758
  20:58:38 STEP 1847 H= 461:45 +CPU= 36.508
  20:59:27 STEP 1848 H= 462:00 +CPU= 44.486
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001848
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001848
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001848
  21:00:09 STEP 1849 H= 462:15 +CPU= 36.968
  21:00:50 STEP 1850 H= 462:30 +CPU= 35.864
  21:01:31 STEP 1851 H= 462:45 +CPU= 36.282
  21:02:13 STEP 1852 H= 463:00 +CPU= 37.226
  21:02:54 STEP 1853 H= 463:15 +CPU= 37.004
  21:03:35 STEP 1854 H= 463:30 +CPU= 36.141
  21:04:16 STEP 1855 H= 463:45 +CPU= 37.368
  21:04:58 STEP 1856 H= 464:00 +CPU= 38.151
  21:05:40 STEP 1857 H= 464:15 +CPU= 36.995
  21:06:21 STEP 1858 H= 464:30 +CPU= 37.269
  21:07:01 STEP 1859 H= 464:45 +CPU= 35.272
  21:07:52 STEP 1860 H= 465:00 +CPU= 45.459
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001860
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001860
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001860
  21:08:30 STEP 1861 H= 465:15 +CPU= 34.897
  21:09:10 STEP 1862 H= 465:30 +CPU= 36.752
  21:09:51 STEP 1863 H= 465:45 +CPU= 36.370
  21:10:32 STEP 1864 H= 466:00 +CPU= 37.063
  21:11:12 STEP 1865 H= 466:15 +CPU= 36.439
  21:11:53 STEP 1866 H= 466:30 +CPU= 36.653
  21:12:31 STEP 1867 H= 466:45 +CPU= 34.979
  21:13:00 STEP 1868 H= 467:00 +CPU= 26.727
  21:13:21 STEP 1869 H= 467:15 +CPU= 20.487
  21:13:42 STEP 1870 H= 467:30 +CPU= 19.079
  21:14:08 STEP 1871 H= 467:45 +CPU= 24.660
  21:15:38 STEP 1872 H= 468:00 +CPU= 46.971
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001872
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001872
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001872
  21:16:19 STEP 1873 H= 468:15 +CPU= 36.761
  21:17:00 STEP 1874 H= 468:30 +CPU= 37.133
  21:17:41 STEP 1875 H= 468:45 +CPU= 36.469
  21:18:22 STEP 1876 H= 469:00 +CPU= 37.041
  21:19:02 STEP 1877 H= 469:15 +CPU= 35.504
  21:19:43 STEP 1878 H= 469:30 +CPU= 37.227
  21:20:23 STEP 1879 H= 469:45 +CPU= 35.528
  21:21:04 STEP 1880 H= 470:00 +CPU= 35.569
  21:21:45 STEP 1881 H= 470:15 +CPU= 36.616
  21:22:26 STEP 1882 H= 470:30 +CPU= 36.552
  21:23:08 STEP 1883 H= 470:45 +CPU= 37.084
  21:23:58 STEP 1884 H= 471:00 +CPU= 45.092
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001884
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001884
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001884
  21:24:40 STEP 1885 H= 471:15 +CPU= 37.048
  21:25:18 STEP 1886 H= 471:30 +CPU= 34.767
  21:25:58 STEP 1887 H= 471:45 +CPU= 35.868
  21:26:38 STEP 1888 H= 472:00 +CPU= 36.789
  21:27:19 STEP 1889 H= 472:15 +CPU= 36.838
  21:27:59 STEP 1890 H= 472:30 +CPU= 36.787
  21:28:40 STEP 1891 H= 472:45 +CPU= 36.951
  21:29:21 STEP 1892 H= 473:00 +CPU= 37.441
  21:30:02 STEP 1893 H= 473:15 +CPU= 37.155
  21:30:42 STEP 1894 H= 473:30 +CPU= 35.310
  21:31:23 STEP 1895 H= 473:45 +CPU= 36.861
  21:32:13 STEP 1896 H= 474:00 +CPU= 45.025
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001896
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001896
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001896
  21:32:54 STEP 1897 H= 474:15 +CPU= 37.319
  21:33:33 STEP 1898 H= 474:30 +CPU= 35.807
  21:34:15 STEP 1899 H= 474:45 +CPU= 37.178
  21:34:55 STEP 1900 H= 475:00 +CPU= 37.038
  21:35:36 STEP 1901 H= 475:15 +CPU= 37.339
  21:36:18 STEP 1902 H= 475:30 +CPU= 37.115
  21:36:58 STEP 1903 H= 475:45 +CPU= 37.001
  21:37:39 STEP 1904 H= 476:00 +CPU= 37.768
  21:38:19 STEP 1905 H= 476:15 +CPU= 35.716
  21:38:59 STEP 1906 H= 476:30 +CPU= 36.242
  21:39:40 STEP 1907 H= 476:45 +CPU= 36.818
  21:40:33 STEP 1908 H= 477:00 +CPU= 46.701
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001908
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001908
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001908
  21:41:13 STEP 1909 H= 477:15 +CPU= 36.851
  21:41:54 STEP 1910 H= 477:30 +CPU= 37.062
  21:42:35 STEP 1911 H= 477:45 +CPU= 36.975
  21:43:17 STEP 1912 H= 478:00 +CPU= 37.933
  21:43:57 STEP 1913 H= 478:15 +CPU= 36.788
  21:44:38 STEP 1914 H= 478:30 +CPU= 37.129
  21:45:18 STEP 1915 H= 478:45 +CPU= 36.967
  21:46:00 STEP 1916 H= 479:00 +CPU= 37.669
  21:46:31 STEP 1917 H= 479:15 +CPU= 28.822
  21:46:55 STEP 1918 H= 479:30 +CPU= 22.115
  21:47:21 STEP 1919 H= 479:45 +CPU= 24.233
  21:48:16 STEP 1920 H= 480:00 +CPU= 40.064
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  480.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1372422698E-21      I      -0.5876511777E-05      I
ICloud fractionI      0.0000000000E+00      I       0.8978350963E-01      I
I  Relat. Hum. I      0.7748148159E-01      I       0.2944745944E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.9445529404E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1920 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001920 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001920
 MPI-TASK:   1 -    661380100 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS******** Mbytes/s, TOTAL TIME=    0.0158( 3.2%)
 21:48:16 STEP 1920 H= 480:00 +CPU= 40.064
 END CNT3
 NSTEP =  1920 CNT0   000000000  29.929  29.929  37.198   0.805               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN         7150/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    128    0       46      1044068     7150 ************   159.1   104.3    54.8   105.6    82.7
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 9880366.7 9880366.7     98.80      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  261    6768.5    6768.5     17.66      0.00
   9 SPCM     - SPECTRAL COMP.                     240    1268.9    1268.9      3.05      0.00
  10 SCAN2M - PHYSICS                              241   18901.9   18901.9     45.55      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 20    4223.1    4223.1      0.84      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 11      47.1      47.1      0.01      0.00
  17 GRIDFPOS IN CNT4                               20      39.4      39.4      0.01      0.00
  27 RESTART FILES - WRITING                         5   16892.3   16892.3      0.84      0.00
  28 RESTART FILES - READING                         1  106467.1  106467.1      1.06      0.00
  29 SU4FPOS IN CNT4                                20       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                20    9970.5    9970.5      1.99      0.00
  31 POSDDH IN STEPO                                 2     755.3     755.3      0.02      0.00
  37 CPGLAG   - SL COMPUTATIONS                    241    8359.1    8359.1     20.14      0.00
  39 SU0YOMB                                         1    5275.1    5275.1      0.05      0.00
  51 SCAN2M   - SL COMM. PART 1                    241      34.6      34.6      0.08      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              240     514.0     514.0      1.23      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              240     129.0     129.0      0.31      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 11       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      524     610.6     610.6      3.20      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       500     395.3     395.3      1.98      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        500     115.1     115.1      0.58      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       524     213.3     213.3      1.12      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      36.0      36.0      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            524      69.0      69.0      0.36      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            500      35.4      35.4      0.18      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            524      81.5      81.5      0.43      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            500      37.8      37.8      0.19      0.00
 400 GSTATS                                      28864       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 26802       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    10000.8 CPU TIME    8813.3 VECTOR TIME     8813.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY      174.9
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/3/ICMGGh7zg+001920
Moving to projects directory: /var/lib/boinc/slots/3/ICMSHh7zg+001920
Moving to projects directory: /var/lib/boinc/slots/3/ICMUAh7zg+001920
Adding to the zip: /var/lib/boinc/slots/3/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/3/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMGGh7zg+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMSHh7zg+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12292690/ICMUAh7zg+001920
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0_r842175855_19.zip
Uploading the final file: upload_file_19.zip
Uploading trickle at timestep: 1727100
21:51:58 (377632): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
14 Jun 2024 20:51:30 1507812 22446299 oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0 1,727,100 62,394 0.0361
14 Jun 2024 08:43:29 1507812 22446299 oifs_43r3_bl_a3d7_2016092300_20_1018_12292690_0 864,000 27,166 0.0314


©2024 climateprediction.net