climateprediction.net home page
Task 22294259

Task 22294259

Name oifs_43r3_ps_0909_1990050100_123_959_12176553_1
Workunit 12176553
Created 29 Dec 2022, 9:14:52 UTC
Sent 29 Dec 2022, 9:37:06 UTC
Report deadline 28 Jan 2023, 9:37:06 UTC
Received 31 Dec 2022, 9:48:44 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1498859
Run time 10 hours 28 min 39 sec
CPU time 10 hours 25 min 46 sec
Validate state Invalid
Credit 4,838.40
Device peak FLOPS 3.97 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,184.91 MB
Peak swap size 4,974.12 MB
Peak disk usage 1,826.97 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
ding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000516
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000516
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_21.zip
Uploading the intermediate file: upload_file_21.zip
  16:00:07 STEP  529 H= 529:00 +CPU= 31.525
  16:00:41 STEP  530 H= 530:00 +CPU= 33.901
  16:01:32 STEP  531 H= 531:00 +CPU= 51.183
  16:02:05 STEP  532 H= 532:00 +CPU= 32.116
  16:02:35 STEP  533 H= 533:00 +CPU= 30.255
  16:03:26 STEP  534 H= 534:00 +CPU= 50.828
  16:03:58 STEP  535 H= 535:00 +CPU= 32.256
  16:04:30 STEP  536 H= 536:00 +CPU= 31.624
  16:05:23 STEP  537 H= 537:00 +CPU= 52.930
  16:05:55 STEP  538 H= 538:00 +CPU= 31.973
  16:06:25 STEP  539 H= 539:00 +CPU= 30.313
  16:07:19 STEP  540 H= 540:00 +CPU= 53.853
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000540
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000540
  16:07:49 STEP  541 H= 541:00 +CPU= 29.768
  16:08:21 STEP  542 H= 542:00 +CPU= 31.966
  16:09:07 STEP  543 H= 543:00 +CPU= 46.047
  16:09:40 STEP  544 H= 544:00 +CPU= 32.266
  16:10:09 STEP  545 H= 545:00 +CPU= 29.436
  16:10:59 STEP  546 H= 546:00 +CPU= 49.561
  16:11:28 STEP  547 H= 547:00 +CPU= 29.627
  16:12:00 STEP  548 H= 548:00 +CPU= 31.746
  16:12:49 STEP  549 H= 549:00 +CPU= 48.572
  16:13:18 STEP  550 H= 550:00 +CPU= 29.186
  16:13:47 STEP  551 H= 551:00 +CPU= 29.142
  16:14:44 STEP  552 H= 552:00 +CPU= 56.354
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000552
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000528
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000540
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000540
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_22.zip
Uploading the intermediate file: upload_file_22.zip
  16:15:14 STEP  553 H= 553:00 +CPU= 29.834
  16:15:44 STEP  554 H= 554:00 +CPU= 30.401
  16:16:31 STEP  555 H= 555:00 +CPU= 47.379
  16:17:03 STEP  556 H= 556:00 +CPU= 31.201
  16:17:34 STEP  557 H= 557:00 +CPU= 31.622
  16:18:20 STEP  558 H= 558:00 +CPU= 45.744
  16:18:50 STEP  559 H= 559:00 +CPU= 29.356
  16:19:18 STEP  560 H= 560:00 +CPU= 28.820
  16:20:06 STEP  561 H= 561:00 +CPU= 47.145
  16:20:37 STEP  562 H= 562:00 +CPU= 31.648
  16:21:08 STEP  563 H= 563:00 +CPU= 30.330
  16:22:02 STEP  564 H= 564:00 +CPU= 54.118
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000564
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000564
  16:22:32 STEP  565 H= 565:00 +CPU= 30.498
  16:23:03 STEP  566 H= 566:00 +CPU= 30.513
  16:23:54 STEP  567 H= 567:00 +CPU= 51.435
  16:24:28 STEP  568 H= 568:00 +CPU= 33.699
  16:25:00 STEP  569 H= 569:00 +CPU= 32.036
  16:25:53 STEP  570 H= 570:00 +CPU= 52.989
  16:26:25 STEP  571 H= 571:00 +CPU= 31.531
  16:26:59 STEP  572 H= 572:00 +CPU= 33.874
  16:27:51 STEP  573 H= 573:00 +CPU= 52.534
  16:28:23 STEP  574 H= 574:00 +CPU= 31.517
  16:28:55 STEP  575 H= 575:00 +CPU= 31.883
  16:29:57 STEP  576 H= 576:00 +CPU= 62.354
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000576
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000552
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000564
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000564
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_23.zip
Uploading the intermediate file: upload_file_23.zip
  16:30:28 STEP  577 H= 577:00 +CPU= 30.666
  16:31:00 STEP  578 H= 578:00 +CPU= 31.814
  16:31:47 STEP  579 H= 579:00 +CPU= 47.091
  16:32:17 STEP  580 H= 580:00 +CPU= 29.606
  16:32:45 STEP  581 H= 581:00 +CPU= 27.847
  16:33:31 STEP  582 H= 582:00 +CPU= 45.655
  16:33:59 STEP  583 H= 583:00 +CPU= 28.304
  16:34:27 STEP  584 H= 584:00 +CPU= 28.105
  16:35:13 STEP  585 H= 585:00 +CPU= 45.614
  16:35:43 STEP  586 H= 586:00 +CPU= 30.187
  16:36:11 STEP  587 H= 587:00 +CPU= 28.118
  16:37:03 STEP  588 H= 588:00 +CPU= 51.842
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000588
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000588
  16:37:31 STEP  589 H= 589:00 +CPU= 28.096
  16:38:00 STEP  590 H= 590:00 +CPU= 28.974
  16:38:46 STEP  591 H= 591:00 +CPU= 45.635
  16:39:15 STEP  592 H= 592:00 +CPU= 28.783
  16:39:43 STEP  593 H= 593:00 +CPU= 27.764
  16:40:28 STEP  594 H= 594:00 +CPU= 45.344
  16:40:57 STEP  595 H= 595:00 +CPU= 28.727
  16:41:26 STEP  596 H= 596:00 +CPU= 28.927
  16:42:12 STEP  597 H= 597:00 +CPU= 46.272
  16:42:41 STEP  598 H= 598:00 +CPU= 28.631
  16:43:10 STEP  599 H= 599:00 +CPU= 29.370
  16:44:04 STEP  600 H= 600:00 +CPU= 53.462
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000600
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000576
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000588
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000588
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_24.zip
Uploading the intermediate file: upload_file_24.zip
  16:44:31 STEP  601 H= 601:00 +CPU= 27.453
  16:45:01 STEP  602 H= 602:00 +CPU= 29.743
  16:45:46 STEP  603 H= 603:00 +CPU= 44.787
  16:46:15 STEP  604 H= 604:00 +CPU= 28.436
  16:46:44 STEP  605 H= 605:00 +CPU= 29.515
  16:47:33 STEP  606 H= 606:00 +CPU= 48.849
  16:48:01 STEP  607 H= 607:00 +CPU= 28.213
  16:48:32 STEP  608 H= 608:00 +CPU= 30.185
  16:49:18 STEP  609 H= 609:00 +CPU= 46.911
  16:49:48 STEP  610 H= 610:00 +CPU= 29.238
  16:50:18 STEP  611 H= 611:00 +CPU= 29.891
  16:51:09 STEP  612 H= 612:00 +CPU= 51.406
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000612
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000612
  16:51:38 STEP  613 H= 613:00 +CPU= 28.663
  16:52:07 STEP  614 H= 614:00 +CPU= 28.816
  16:52:52 STEP  615 H= 615:00 +CPU= 45.640
  16:53:23 STEP  616 H= 616:00 +CPU= 30.153
  16:53:51 STEP  617 H= 617:00 +CPU= 28.358
  16:54:36 STEP  618 H= 618:00 +CPU= 44.827
  16:55:03 STEP  619 H= 619:00 +CPU= 26.724
  16:55:31 STEP  620 H= 620:00 +CPU= 27.959
  16:56:17 STEP  621 H= 621:00 +CPU= 45.512
  16:56:45 STEP  622 H= 622:00 +CPU= 28.353
  16:57:13 STEP  623 H= 623:00 +CPU= 27.659
  16:58:08 STEP  624 H= 624:00 +CPU= 54.698
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000624
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000600
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000612
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000612
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_25.zip
Uploading the intermediate file: upload_file_25.zip
  16:58:36 STEP  625 H= 625:00 +CPU= 28.162
  16:59:06 STEP  626 H= 626:00 +CPU= 30.236
  16:59:52 STEP  627 H= 627:00 +CPU= 45.986
  17:00:21 STEP  628 H= 628:00 +CPU= 29.051
  17:00:50 STEP  629 H= 629:00 +CPU= 28.812
  17:01:35 STEP  630 H= 630:00 +CPU= 44.572
  17:02:02 STEP  631 H= 631:00 +CPU= 27.597
  17:02:32 STEP  632 H= 632:00 +CPU= 29.310
  17:03:18 STEP  633 H= 633:00 +CPU= 45.981
  17:03:46 STEP  634 H= 634:00 +CPU= 28.054
  17:04:14 STEP  635 H= 635:00 +CPU= 27.940
  17:05:04 STEP  636 H= 636:00 +CPU= 49.761
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000636
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000636
  17:05:31 STEP  637 H= 637:00 +CPU= 27.934
  17:06:00 STEP  638 H= 638:00 +CPU= 28.237
  17:06:46 STEP  639 H= 639:00 +CPU= 46.049
  17:07:14 STEP  640 H= 640:00 +CPU= 28.373
  17:07:42 STEP  641 H= 641:00 +CPU= 27.443
  17:08:29 STEP  642 H= 642:00 +CPU= 47.084
  17:08:58 STEP  643 H= 643:00 +CPU= 28.741
  17:09:27 STEP  644 H= 644:00 +CPU= 29.157
  17:10:13 STEP  645 H= 645:00 +CPU= 45.517
  17:10:41 STEP  646 H= 646:00 +CPU= 28.702
  17:11:11 STEP  647 H= 647:00 +CPU= 28.921
  17:12:03 STEP  648 H= 648:00 +CPU= 52.421
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000648
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000624
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000636
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000636
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_26.zip
Uploading the intermediate file: upload_file_26.zip
  17:12:31 STEP  649 H= 649:00 +CPU= 27.401
  17:13:02 STEP  650 H= 650:00 +CPU= 30.955
  17:13:47 STEP  651 H= 651:00 +CPU= 45.528
  17:14:17 STEP  652 H= 652:00 +CPU= 30.053
  17:14:46 STEP  653 H= 653:00 +CPU= 28.119
  17:15:31 STEP  654 H= 654:00 +CPU= 45.404
  17:16:00 STEP  655 H= 655:00 +CPU= 28.671
  17:16:29 STEP  656 H= 656:00 +CPU= 29.294
  17:17:17 STEP  657 H= 657:00 +CPU= 47.184
  17:17:45 STEP  658 H= 658:00 +CPU= 28.728
  17:18:14 STEP  659 H= 659:00 +CPU= 28.808
  17:19:04 STEP  660 H= 660:00 +CPU= 49.500
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000660
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000660
  17:19:31 STEP  661 H= 661:00 +CPU= 27.578
  17:20:00 STEP  662 H= 662:00 +CPU= 28.118
  17:20:44 STEP  663 H= 663:00 +CPU= 44.566
  17:21:13 STEP  664 H= 664:00 +CPU= 28.798
  17:21:40 STEP  665 H= 665:00 +CPU= 27.225
  17:22:26 STEP  666 H= 666:00 +CPU= 45.648
  17:22:53 STEP  667 H= 667:00 +CPU= 26.985
  17:23:23 STEP  668 H= 668:00 +CPU= 29.539
  17:24:08 STEP  669 H= 669:00 +CPU= 44.665
  17:24:36 STEP  670 H= 670:00 +CPU= 28.117
  17:25:04 STEP  671 H= 671:00 +CPU= 27.680
  17:25:54 STEP  672 H= 672:00 +CPU= 50.594
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000672
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000648
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000660
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000660
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_27.zip
Uploading the intermediate file: upload_file_27.zip
  17:26:21 STEP  673 H= 673:00 +CPU= 26.628
  17:26:49 STEP  674 H= 674:00 +CPU= 27.281
  17:27:31 STEP  675 H= 675:00 +CPU= 42.543
  17:27:57 STEP  676 H= 676:00 +CPU= 26.224
  17:28:24 STEP  677 H= 677:00 +CPU= 26.405
  17:29:08 STEP  678 H= 678:00 +CPU= 44.393
  17:29:35 STEP  679 H= 679:00 +CPU= 26.762
  17:30:03 STEP  680 H= 680:00 +CPU= 27.533
  17:30:45 STEP  681 H= 681:00 +CPU= 42.735
  17:31:13 STEP  682 H= 682:00 +CPU= 27.391
  17:31:39 STEP  683 H= 683:00 +CPU= 26.482
  17:32:27 STEP  684 H= 684:00 +CPU= 47.151
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000684
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000684
  17:32:53 STEP  685 H= 685:00 +CPU= 26.251
  17:33:20 STEP  686 H= 686:00 +CPU= 26.587
  17:34:03 STEP  687 H= 687:00 +CPU= 42.995
  17:34:29 STEP  688 H= 688:00 +CPU= 26.440
  17:34:55 STEP  689 H= 689:00 +CPU= 25.736
  17:35:38 STEP  690 H= 690:00 +CPU= 42.903
  17:36:04 STEP  691 H= 691:00 +CPU= 25.715
  17:36:31 STEP  692 H= 692:00 +CPU= 27.027
  17:37:14 STEP  693 H= 693:00 +CPU= 43.106
  17:37:41 STEP  694 H= 694:00 +CPU= 26.728
  17:38:10 STEP  695 H= 695:00 +CPU= 28.674
  17:39:08 STEP  696 H= 696:00 +CPU= 58.163
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000696
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000672
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000684
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000684
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_28.zip
Uploading the intermediate file: upload_file_28.zip
  17:39:38 STEP  697 H= 697:00 +CPU= 29.723
  17:40:07 STEP  698 H= 698:00 +CPU= 29.874
  17:40:57 STEP  699 H= 699:00 +CPU= 49.179
  17:41:28 STEP  700 H= 700:00 +CPU= 31.522
  17:42:00 STEP  701 H= 701:00 +CPU= 31.225
  17:42:46 STEP  702 H= 702:00 +CPU= 46.923
  17:43:14 STEP  703 H= 703:00 +CPU= 27.388
  17:43:42 STEP  704 H= 704:00 +CPU= 28.172
  17:44:28 STEP  705 H= 705:00 +CPU= 45.792
  17:44:57 STEP  706 H= 706:00 +CPU= 28.763
  17:45:24 STEP  707 H= 707:00 +CPU= 27.552
  17:46:15 STEP  708 H= 708:00 +CPU= 50.949
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000708
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000708
  17:46:44 STEP  709 H= 709:00 +CPU= 28.133
  17:47:12 STEP  710 H= 710:00 +CPU= 28.691
  17:47:59 STEP  711 H= 711:00 +CPU= 46.287
  17:48:27 STEP  712 H= 712:00 +CPU= 28.565
  17:48:55 STEP  713 H= 713:00 +CPU= 27.776
  17:49:40 STEP  714 H= 714:00 +CPU= 44.416
  17:50:07 STEP  715 H= 715:00 +CPU= 27.261
  17:50:35 STEP  716 H= 716:00 +CPU= 27.724
  17:51:20 STEP  717 H= 717:00 +CPU= 44.703
  17:51:48 STEP  718 H= 718:00 +CPU= 27.910
  17:52:16 STEP  719 H= 719:00 +CPU= 28.217
  17:53:08 STEP  720 H= 720:00 +CPU= 52.368
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000720
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000696
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000708
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000708
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_29.zip
Uploading the intermediate file: upload_file_29.zip
Uploading trickle at timestep: 2592000
  17:53:36 STEP  721 H= 721:00 +CPU= 27.749
  17:54:04 STEP  722 H= 722:00 +CPU= 28.515
  17:54:50 STEP  723 H= 723:00 +CPU= 45.054
  17:55:18 STEP  724 H= 724:00 +CPU= 27.986
  17:55:45 STEP  725 H= 725:00 +CPU= 27.149
  17:56:29 STEP  726 H= 726:00 +CPU= 44.622
  17:56:57 STEP  727 H= 727:00 +CPU= 27.321
  17:57:24 STEP  728 H= 728:00 +CPU= 27.786
  17:58:08 STEP  729 H= 729:00 +CPU= 44.022
  17:58:37 STEP  730 H= 730:00 +CPU= 28.424
  17:59:04 STEP  731 H= 731:00 +CPU= 26.871
  17:59:54 STEP  732 H= 732:00 +CPU= 50.472
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000732
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000732
  18:00:22 STEP  733 H= 733:00 +CPU= 27.188
  18:00:50 STEP  734 H= 734:00 +CPU= 28.199
  18:01:34 STEP  735 H= 735:00 +CPU= 43.863
  18:02:02 STEP  736 H= 736:00 +CPU= 27.913
  18:02:29 STEP  737 H= 737:00 +CPU= 27.091
  18:03:14 STEP  738 H= 738:00 +CPU= 45.009
  18:03:40 STEP  739 H= 739:00 +CPU= 26.557
  18:04:09 STEP  740 H= 740:00 +CPU= 28.608
  18:04:53 STEP  741 H= 741:00 +CPU= 44.453
  18:05:22 STEP  742 H= 742:00 +CPU= 29.040
  18:05:51 STEP  743 H= 743:00 +CPU= 28.074
  18:06:43 STEP  744 H= 744:00 +CPU= 52.024
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000744
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000720
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000732
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000732
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_30.zip
Uploading the intermediate file: upload_file_30.zip
  18:07:10 STEP  745 H= 745:00 +CPU= 27.758
  18:07:38 STEP  746 H= 746:00 +CPU= 27.493
  18:08:21 STEP  747 H= 747:00 +CPU= 42.932
  18:08:49 STEP  748 H= 748:00 +CPU= 27.727
  18:09:15 STEP  749 H= 749:00 +CPU= 26.128
  18:10:01 STEP  750 H= 750:00 +CPU= 45.983
  18:10:27 STEP  751 H= 751:00 +CPU= 26.494
  18:10:54 STEP  752 H= 752:00 +CPU= 26.905
  18:11:38 STEP  753 H= 753:00 +CPU= 43.682
  18:12:05 STEP  754 H= 754:00 +CPU= 27.604
  18:12:32 STEP  755 H= 755:00 +CPU= 26.901
  18:13:21 STEP  756 H= 756:00 +CPU= 49.018
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000756
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000756
  18:13:48 STEP  757 H= 757:00 +CPU= 26.378
  18:14:15 STEP  758 H= 758:00 +CPU= 26.947
  18:14:58 STEP  759 H= 759:00 +CPU= 43.332
  18:15:25 STEP  760 H= 760:00 +CPU= 27.090
  18:15:53 STEP  761 H= 761:00 +CPU= 27.379
  18:16:36 STEP  762 H= 762:00 +CPU= 43.711
  18:17:03 STEP  763 H= 763:00 +CPU= 26.687
  18:17:31 STEP  764 H= 764:00 +CPU= 27.681
  18:18:15 STEP  765 H= 765:00 +CPU= 44.199
  18:18:42 STEP  766 H= 766:00 +CPU= 27.226
  18:19:10 STEP  767 H= 767:00 +CPU= 27.408
  18:20:01 STEP  768 H= 768:00 +CPU= 51.709
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000768
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000744
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000756
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000756
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_31.zip
Uploading the intermediate file: upload_file_31.zip
  18:20:27 STEP  769 H= 769:00 +CPU= 25.794
  18:20:54 STEP  770 H= 770:00 +CPU= 26.499
  18:21:37 STEP  771 H= 771:00 +CPU= 42.844
  18:22:04 STEP  772 H= 772:00 +CPU= 27.088
  18:22:31 STEP  773 H= 773:00 +CPU= 26.916
  18:23:14 STEP  774 H= 774:00 +CPU= 43.299
  18:23:40 STEP  775 H= 775:00 +CPU= 25.770
  18:24:06 STEP  776 H= 776:00 +CPU= 26.361
  18:24:48 STEP  777 H= 777:00 +CPU= 42.049
  18:25:15 STEP  778 H= 778:00 +CPU= 26.355
  18:25:40 STEP  779 H= 779:00 +CPU= 25.564
  18:26:26 STEP  780 H= 780:00 +CPU= 46.037
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000780
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000780
  18:26:52 STEP  781 H= 781:00 +CPU= 25.544
  18:27:18 STEP  782 H= 782:00 +CPU= 25.845
  18:27:59 STEP  783 H= 783:00 +CPU= 40.943
  18:28:25 STEP  784 H= 784:00 +CPU= 26.726
  18:28:51 STEP  785 H= 785:00 +CPU= 25.425
  18:29:33 STEP  786 H= 786:00 +CPU= 42.390
  18:30:00 STEP  787 H= 787:00 +CPU= 26.242
  18:30:26 STEP  788 H= 788:00 +CPU= 26.661
  18:31:09 STEP  789 H= 789:00 +CPU= 42.249
  18:31:35 STEP  790 H= 790:00 +CPU= 26.159
  18:32:01 STEP  791 H= 791:00 +CPU= 26.004
  18:32:50 STEP  792 H= 792:00 +CPU= 48.904
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000792
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000768
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000780
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000780
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_32.zip
Uploading the intermediate file: upload_file_32.zip
  18:33:15 STEP  793 H= 793:00 +CPU= 25.465
  18:33:42 STEP  794 H= 794:00 +CPU= 26.637
  18:34:24 STEP  795 H= 795:00 +CPU= 42.016
  18:34:50 STEP  796 H= 796:00 +CPU= 26.312
  18:35:16 STEP  797 H= 797:00 +CPU= 26.163
  18:36:00 STEP  798 H= 798:00 +CPU= 43.350
  18:36:26 STEP  799 H= 799:00 +CPU= 25.889
  18:36:52 STEP  800 H= 800:00 +CPU= 26.515
  18:37:34 STEP  801 H= 801:00 +CPU= 42.183
  18:38:01 STEP  802 H= 802:00 +CPU= 26.318
  18:38:27 STEP  803 H= 803:00 +CPU= 26.136
  18:39:14 STEP  804 H= 804:00 +CPU= 47.059
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000804
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000804
  18:39:40 STEP  805 H= 805:00 +CPU= 25.927
  18:40:07 STEP  806 H= 806:00 +CPU= 26.933
  18:40:49 STEP  807 H= 807:00 +CPU= 42.424
  18:41:16 STEP  808 H= 808:00 +CPU= 26.416
  18:41:42 STEP  809 H= 809:00 +CPU= 25.657
  18:42:24 STEP  810 H= 810:00 +CPU= 42.901
  18:42:50 STEP  811 H= 811:00 +CPU= 25.799
  18:43:17 STEP  812 H= 812:00 +CPU= 26.287
  18:44:03 STEP  813 H= 813:00 +CPU= 46.106
  18:44:34 STEP  814 H= 814:00 +CPU= 31.286
  18:45:02 STEP  815 H= 815:00 +CPU= 27.903
  18:45:55 STEP  816 H= 816:00 +CPU= 52.934
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000816
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000792
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000804
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000804
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_33.zip
Uploading the intermediate file: upload_file_33.zip
  18:46:22 STEP  817 H= 817:00 +CPU= 27.363
  18:46:51 STEP  818 H= 818:00 +CPU= 28.144
  18:47:34 STEP  819 H= 819:00 +CPU= 43.745
  18:48:02 STEP  820 H= 820:00 +CPU= 28.151
  18:48:30 STEP  821 H= 821:00 +CPU= 27.274
  18:49:14 STEP  822 H= 822:00 +CPU= 44.361
  18:49:41 STEP  823 H= 823:00 +CPU= 26.764
  18:50:08 STEP  824 H= 824:00 +CPU= 27.167
  18:50:53 STEP  825 H= 825:00 +CPU= 44.733
  18:51:22 STEP  826 H= 826:00 +CPU= 28.623
  18:51:49 STEP  827 H= 827:00 +CPU= 27.699
  18:52:39 STEP  828 H= 828:00 +CPU= 49.994
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000828
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000828
  18:53:06 STEP  829 H= 829:00 +CPU= 26.246
  18:53:34 STEP  830 H= 830:00 +CPU= 28.388
  18:54:18 STEP  831 H= 831:00 +CPU= 44.111
  18:54:46 STEP  832 H= 832:00 +CPU= 28.098
  18:55:14 STEP  833 H= 833:00 +CPU= 27.233
  18:55:59 STEP  834 H= 834:00 +CPU= 45.079
  18:56:26 STEP  835 H= 835:00 +CPU= 27.363
  18:56:54 STEP  836 H= 836:00 +CPU= 28.094
  18:57:39 STEP  837 H= 837:00 +CPU= 44.713
  18:58:07 STEP  838 H= 838:00 +CPU= 27.774
  18:58:36 STEP  839 H= 839:00 +CPU= 29.444
  18:59:29 STEP  840 H= 840:00 +CPU= 52.771
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000840
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000816
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000828
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000828
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_34.zip
Uploading the intermediate file: upload_file_34.zip
  18:59:56 STEP  841 H= 841:00 +CPU= 27.037
  19:00:24 STEP  842 H= 842:00 +CPU= 28.276
  19:01:10 STEP  843 H= 843:00 +CPU= 45.383
  19:01:38 STEP  844 H= 844:00 +CPU= 28.106
  19:02:05 STEP  845 H= 845:00 +CPU= 27.301
  19:02:50 STEP  846 H= 846:00 +CPU= 44.825
  19:03:17 STEP  847 H= 847:00 +CPU= 26.663
  19:03:45 STEP  848 H= 848:00 +CPU= 27.833
  19:04:29 STEP  849 H= 849:00 +CPU= 44.469
  19:04:57 STEP  850 H= 850:00 +CPU= 28.059
  19:05:25 STEP  851 H= 851:00 +CPU= 27.677
  19:06:15 STEP  852 H= 852:00 +CPU= 49.674
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000852
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000852
  19:06:42 STEP  853 H= 853:00 +CPU= 27.118
  19:07:09 STEP  854 H= 854:00 +CPU= 27.329
  19:07:55 STEP  855 H= 855:00 +CPU= 45.424
  19:08:23 STEP  856 H= 856:00 +CPU= 28.039
  19:08:50 STEP  857 H= 857:00 +CPU= 27.735
  19:09:36 STEP  858 H= 858:00 +CPU= 45.857
  19:10:03 STEP  859 H= 859:00 +CPU= 26.983
  19:10:31 STEP  860 H= 860:00 +CPU= 27.983
  19:11:16 STEP  861 H= 861:00 +CPU= 44.712
  19:11:44 STEP  862 H= 862:00 +CPU= 28.007
  19:12:13 STEP  863 H= 863:00 +CPU= 28.407
  19:13:05 STEP  864 H= 864:00 +CPU= 52.081
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000864
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000840
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000852
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000852
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_35.zip
Uploading the intermediate file: upload_file_35.zip
  19:13:32 STEP  865 H= 865:00 +CPU= 27.143
  19:14:00 STEP  866 H= 866:00 +CPU= 27.516
  19:14:44 STEP  867 H= 867:00 +CPU= 44.019
  19:15:11 STEP  868 H= 868:00 +CPU= 27.276
  19:15:38 STEP  869 H= 869:00 +CPU= 27.008
  19:16:22 STEP  870 H= 870:00 +CPU= 44.378
  19:16:49 STEP  871 H= 871:00 +CPU= 26.782
  19:17:16 STEP  872 H= 872:00 +CPU= 27.165
  19:18:00 STEP  873 H= 873:00 +CPU= 43.275
  19:18:27 STEP  874 H= 874:00 +CPU= 26.990
  19:18:53 STEP  875 H= 875:00 +CPU= 26.523
  19:19:42 STEP  876 H= 876:00 +CPU= 48.563
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000876
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000876
  19:20:09 STEP  877 H= 877:00 +CPU= 26.873
  19:20:36 STEP  878 H= 878:00 +CPU= 26.942
  19:21:20 STEP  879 H= 879:00 +CPU= 43.890
  19:21:46 STEP  880 H= 880:00 +CPU= 26.897
  19:22:13 STEP  881 H= 881:00 +CPU= 26.549
  19:22:58 STEP  882 H= 882:00 +CPU= 44.604
  19:23:24 STEP  883 H= 883:00 +CPU= 26.434
  19:23:51 STEP  884 H= 884:00 +CPU= 27.227
  19:24:34 STEP  885 H= 885:00 +CPU= 42.614
  19:25:01 STEP  886 H= 886:00 +CPU= 26.862
  19:25:27 STEP  887 H= 887:00 +CPU= 26.603
  19:26:17 STEP  888 H= 888:00 +CPU= 49.537
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000888
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000864
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000876
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000876
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_36.zip
Uploading the intermediate file: upload_file_36.zip
  19:26:43 STEP  889 H= 889:00 +CPU= 25.766
  19:27:09 STEP  890 H= 890:00 +CPU= 26.490
  19:27:52 STEP  891 H= 891:00 +CPU= 42.272
  19:28:19 STEP  892 H= 892:00 +CPU= 26.853
  19:28:44 STEP  893 H= 893:00 +CPU= 25.803
  19:29:28 STEP  894 H= 894:00 +CPU= 43.303
  19:29:54 STEP  895 H= 895:00 +CPU= 26.150
  19:30:21 STEP  896 H= 896:00 +CPU= 26.731
  19:31:03 STEP  897 H= 897:00 +CPU= 42.317
  19:31:30 STEP  898 H= 898:00 +CPU= 27.101
  19:31:56 STEP  899 H= 899:00 +CPU= 25.901
  19:32:42 STEP  900 H= 900:00 +CPU= 46.432
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000900
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000900
  19:33:08 STEP  901 H= 901:00 +CPU= 25.863
  19:33:35 STEP  902 H= 902:00 +CPU= 26.312
  19:34:16 STEP  903 H= 903:00 +CPU= 41.576
  19:34:42 STEP  904 H= 904:00 +CPU= 26.017
  19:35:08 STEP  905 H= 905:00 +CPU= 25.890
  19:35:51 STEP  906 H= 906:00 +CPU= 42.353
  19:36:16 STEP  907 H= 907:00 +CPU= 25.478
  19:36:43 STEP  908 H= 908:00 +CPU= 26.648
  19:37:25 STEP  909 H= 909:00 +CPU= 42.077
  19:37:51 STEP  910 H= 910:00 +CPU= 26.529
  19:38:18 STEP  911 H= 911:00 +CPU= 26.330
  19:39:07 STEP  912 H= 912:00 +CPU= 49.580
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000912
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000888
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000900
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000900
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_37.zip
Uploading the intermediate file: upload_file_37.zip
  19:39:33 STEP  913 H= 913:00 +CPU= 25.741
  19:40:00 STEP  914 H= 914:00 +CPU= 26.793
  19:40:42 STEP  915 H= 915:00 +CPU= 42.595
  19:41:09 STEP  916 H= 916:00 +CPU= 26.314
  19:41:35 STEP  917 H= 917:00 +CPU= 26.015
  19:42:17 STEP  918 H= 918:00 +CPU= 41.917
  19:42:42 STEP  919 H= 919:00 +CPU= 25.520
  19:43:09 STEP  920 H= 920:00 +CPU= 26.647
  19:43:51 STEP  921 H= 921:00 +CPU= 41.425
  19:44:17 STEP  922 H= 922:00 +CPU= 26.332
  19:44:43 STEP  923 H= 923:00 +CPU= 25.695
  19:45:29 STEP  924 H= 924:00 +CPU= 46.402
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000924
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000924
  19:45:55 STEP  925 H= 925:00 +CPU= 25.653
  19:46:21 STEP  926 H= 926:00 +CPU= 26.340
  19:47:04 STEP  927 H= 927:00 +CPU= 43.171
  19:47:31 STEP  928 H= 928:00 +CPU= 26.380
  19:47:57 STEP  929 H= 929:00 +CPU= 26.056
  19:48:39 STEP  930 H= 930:00 +CPU= 42.416
  19:49:05 STEP  931 H= 931:00 +CPU= 25.700
  19:49:31 STEP  932 H= 932:00 +CPU= 26.454
  19:50:14 STEP  933 H= 933:00 +CPU= 42.449
  19:50:40 STEP  934 H= 934:00 +CPU= 26.567
  19:51:07 STEP  935 H= 935:00 +CPU= 26.292
  19:51:56 STEP  936 H= 936:00 +CPU= 48.964
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000936
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000912
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000924
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000924
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_38.zip
Uploading the intermediate file: upload_file_38.zip
  19:52:22 STEP  937 H= 937:00 +CPU= 25.821
  19:52:48 STEP  938 H= 938:00 +CPU= 26.316
  19:53:29 STEP  939 H= 939:00 +CPU= 41.378
  19:53:56 STEP  940 H= 940:00 +CPU= 26.368
  19:54:21 STEP  941 H= 941:00 +CPU= 25.798
  19:55:04 STEP  942 H= 942:00 +CPU= 42.339
  19:55:30 STEP  943 H= 943:00 +CPU= 26.275
  19:55:57 STEP  944 H= 944:00 +CPU= 26.560
  19:56:39 STEP  945 H= 945:00 +CPU= 41.784
  19:57:05 STEP  946 H= 946:00 +CPU= 26.766
  19:57:31 STEP  947 H= 947:00 +CPU= 25.745
  19:58:17 STEP  948 H= 948:00 +CPU= 46.284
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000948
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000948
  19:58:43 STEP  949 H= 949:00 +CPU= 25.868
  19:59:10 STEP  950 H= 950:00 +CPU= 26.282
  19:59:52 STEP  951 H= 951:00 +CPU= 42.121
  20:00:18 STEP  952 H= 952:00 +CPU= 26.426
  20:00:44 STEP  953 H= 953:00 +CPU= 25.855
  20:01:27 STEP  954 H= 954:00 +CPU= 42.520
  20:01:53 STEP  955 H= 955:00 +CPU= 26.316
  20:02:19 STEP  956 H= 956:00 +CPU= 26.345
  20:03:02 STEP  957 H= 957:00 +CPU= 42.585
  20:03:28 STEP  958 H= 958:00 +CPU= 26.476
  20:03:55 STEP  959 H= 959:00 +CPU= 26.596
  20:04:44 STEP  960 H= 960:00 +CPU= 49.268
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000960
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000936
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000948
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000948
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_39.zip
Uploading the intermediate file: upload_file_39.zip
Uploading trickle at timestep: 3456000
  20:05:10 STEP  961 H= 961:00 +CPU= 25.528
  20:05:37 STEP  962 H= 962:00 +CPU= 26.641
  20:06:19 STEP  963 H= 963:00 +CPU= 42.349
  20:06:45 STEP  964 H= 964:00 +CPU= 26.121
  20:07:11 STEP  965 H= 965:00 +CPU= 26.244
  20:07:54 STEP  966 H= 966:00 +CPU= 43.070
  20:08:21 STEP  967 H= 967:00 +CPU= 26.678
  20:08:48 STEP  968 H= 968:00 +CPU= 26.721
  20:09:30 STEP  969 H= 969:00 +CPU= 41.936
  20:09:56 STEP  970 H= 970:00 +CPU= 26.357
  20:10:22 STEP  971 H= 971:00 +CPU= 25.440
  20:11:08 STEP  972 H= 972:00 +CPU= 46.606
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000972
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000972
  20:11:34 STEP  973 H= 973:00 +CPU= 25.864
  20:12:01 STEP  974 H= 974:00 +CPU= 26.449
  20:12:43 STEP  975 H= 975:00 +CPU= 42.149
  20:13:09 STEP  976 H= 976:00 +CPU= 26.475
  20:13:35 STEP  977 H= 977:00 +CPU= 25.636
  20:14:18 STEP  978 H= 978:00 +CPU= 42.793
  20:14:44 STEP  979 H= 979:00 +CPU= 25.888
  20:15:10 STEP  980 H= 980:00 +CPU= 26.390
  20:15:52 STEP  981 H= 981:00 +CPU= 42.110
  20:16:19 STEP  982 H= 982:00 +CPU= 26.408
  20:16:45 STEP  983 H= 983:00 +CPU= 26.700
  20:17:36 STEP  984 H= 984:00 +CPU= 50.169
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000984
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000972
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_40.zip
Uploading the intermediate file: upload_file_40.zip
  20:18:02 STEP  985 H= 985:00 +CPU= 26.113
  20:18:28 STEP  986 H= 986:00 +CPU= 26.313
  20:19:11 STEP  987 H= 987:00 +CPU= 42.434
  20:19:37 STEP  988 H= 988:00 +CPU= 26.338
  20:20:03 STEP  989 H= 989:00 +CPU= 26.066
  20:20:46 STEP  990 H= 990:00 +CPU= 43.027
  20:21:12 STEP  991 H= 991:00 +CPU= 26.100
  20:21:38 STEP  992 H= 992:00 +CPU= 26.179
  20:22:21 STEP  993 H= 993:00 +CPU= 42.982
  20:22:49 STEP  994 H= 994:00 +CPU= 27.174
  20:23:15 STEP  995 H= 995:00 +CPU= 26.556
  20:24:04 STEP  996 H= 996:00 +CPU= 48.455
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+000996
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+000996
  20:24:30 STEP  997 H= 997:00 +CPU= 26.718
  20:24:57 STEP  998 H= 998:00 +CPU= 26.886
  20:25:40 STEP  999 H= 999:00 +CPU= 42.244
  20:26:06 STEP 1000 H=1000:00 +CPU= 26.568
  20:26:32 STEP 1001 H=1001:00 +CPU= 25.823
  20:27:15 STEP 1002 H=1002:00 +CPU= 42.531
  20:27:40 STEP 1003 H=1003:00 +CPU= 25.553
  20:28:08 STEP 1004 H=1004:00 +CPU= 27.837
  20:28:51 STEP 1005 H=1005:00 +CPU= 43.296
  20:29:18 STEP 1006 H=1006:00 +CPU= 26.709
  20:29:45 STEP 1007 H=1007:00 +CPU= 26.488
  20:30:34 STEP 1008 H=1008:00 +CPU= 49.073
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001008
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+000996
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_41.zip
Uploading the intermediate file: upload_file_41.zip
  20:30:59 STEP 1009 H=1009:00 +CPU= 25.370
  20:31:26 STEP 1010 H=1010:00 +CPU= 26.398
  20:32:07 STEP 1011 H=1011:00 +CPU= 41.570
  20:32:34 STEP 1012 H=1012:00 +CPU= 26.909
  20:33:00 STEP 1013 H=1013:00 +CPU= 25.532
  20:33:42 STEP 1014 H=1014:00 +CPU= 42.701
  20:34:08 STEP 1015 H=1015:00 +CPU= 25.730
  20:34:34 STEP 1016 H=1016:00 +CPU= 26.284
  20:35:16 STEP 1017 H=1017:00 +CPU= 41.920
  20:35:43 STEP 1018 H=1018:00 +CPU= 26.358
  20:36:08 STEP 1019 H=1019:00 +CPU= 25.590
  20:36:55 STEP 1020 H=1020:00 +CPU= 46.443
 SMILAG TRAJECTORY UNDERGROUND      1 TIMES.
  POINT=           25  MAX ETADOT VERTICAL VEL.=   1.448615486740199E-007
  LON  =    113.750000000000       degrees
  LAT  =   -38.6913661222016       degrees
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001020
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001020
  20:37:21 STEP 1021 H=1021:00 +CPU= 26.260
  20:37:48 STEP 1022 H=1022:00 +CPU= 26.982
  20:38:30 STEP 1023 H=1023:00 +CPU= 41.738
  20:38:56 STEP 1024 H=1024:00 +CPU= 26.304
  20:39:22 STEP 1025 H=1025:00 +CPU= 26.192
  20:40:05 STEP 1026 H=1026:00 +CPU= 42.919
  20:40:31 STEP 1027 H=1027:00 +CPU= 25.624
  20:40:57 STEP 1028 H=1028:00 +CPU= 26.353
  20:41:40 STEP 1029 H=1029:00 +CPU= 42.533
  20:42:07 STEP 1030 H=1030:00 +CPU= 26.659
  20:42:33 STEP 1031 H=1031:00 +CPU= 26.401
  20:43:23 STEP 1032 H=1032:00 +CPU= 49.770
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001032
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+001020
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_42.zip
Uploading the intermediate file: upload_file_42.zip
  20:43:49 STEP 1033 H=1033:00 +CPU= 25.833
  20:44:15 STEP 1034 H=1034:00 +CPU= 26.327
  20:44:57 STEP 1035 H=1035:00 +CPU= 41.562
  20:45:23 STEP 1036 H=1036:00 +CPU= 26.430
  20:45:49 STEP 1037 H=1037:00 +CPU= 25.771
  20:46:32 STEP 1038 H=1038:00 +CPU= 43.027
  20:46:58 STEP 1039 H=1039:00 +CPU= 26.003
  20:47:25 STEP 1040 H=1040:00 +CPU= 26.464
  20:48:07 STEP 1041 H=1041:00 +CPU= 42.650
  20:48:34 STEP 1042 H=1042:00 +CPU= 26.330
  20:48:59 STEP 1043 H=1043:00 +CPU= 25.727
  20:49:46 STEP 1044 H=1044:00 +CPU= 46.109
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001044
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001044
  20:50:11 STEP 1045 H=1045:00 +CPU= 25.392
  20:50:38 STEP 1046 H=1046:00 +CPU= 26.942
  20:51:20 STEP 1047 H=1047:00 +CPU= 41.600
  20:51:46 STEP 1048 H=1048:00 +CPU= 26.800
  20:52:12 STEP 1049 H=1049:00 +CPU= 25.771
  20:52:57 STEP 1050 H=1050:00 +CPU= 44.416
  20:53:25 STEP 1051 H=1051:00 +CPU= 28.628
  20:53:53 STEP 1052 H=1052:00 +CPU= 28.150
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  07:28:20 STEP 1053 H=1053:00 +CPU= 47.822
  07:28:53 STEP 1054 H=1054:00 +CPU= 32.944
  07:29:27 STEP 1055 H=1055:00 +CPU= 33.731
  07:30:25 STEP 1056 H=1056:00 +CPU= 58.284
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001056
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMGGhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMSHhq0f+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553/ICMUAhq0f+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553_43.zip
Uploading the intermediate file: upload_file_43.zip
  07:30:55 STEP 1057 H=1057:00 +CPU= 29.295
  07:31:26 STEP 1058 H=1058:00 +CPU= 31.684
  07:32:07 STEP 1059 H=1059:00 +CPU= 40.395
  07:32:35 STEP 1060 H=1060:00 +CPU= 28.507
  07:33:03 STEP 1061 H=1061:00 +CPU= 27.741
  07:33:52 STEP 1062 H=1062:00 +CPU= 48.399
  07:34:22 STEP 1063 H=1063:00 +CPU= 30.435
  07:48:21 STEP 1064 H=1064:00 +CPU= 30.171
  07:49:07 STEP 1065 H=1065:00 +CPU= 46.143
  07:49:36 STEP 1066 H=1066:00 +CPU= 29.250
  07:50:04 STEP 1067 H=1067:00 +CPU= 28.021
  07:50:56 STEP 1068 H=1068:00 +CPU= 51.503
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMGGhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMSHhq0f+001068
Moving to projects directory: /var/lib/boinc-client/slots/3/ICMUAhq0f+001068
  07:51:24 STEP 1069 H=1069:00 +CPU= 28.699
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:32:47 STEP 1070 H=1070:00 +CPU= 27.611
  10:33:41 STEP 1071 H=1071:00 +CPU= 50.673
  10:34:16 STEP 1072 H=1072:00 +CPU= 33.758
  10:34:47 STEP 1073 H=1073:00 +CPU= 30.785
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1990050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0909
(argv4) batchid: 959
(argv5) wuid: 12176553
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/3
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12176553
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2a662c3165dc1d5d3430ddf108342ad1 to: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0909_1990050100_123_959_12176553.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/3/oifs_43r3_ps_0909_1990050100_123_959_12176553.zip
ic_ancil_file: ic_ancil_12176553
ifsdata_file: ifsdata_12176553
climate_data_file: clim_data_12176553
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_8a4e40bab53bac03d3d7c5b7afbcff22 to: /var/lib/boinc-client/slots/3/ic_ancil_12176553.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/3/ic_ancil_12176553.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_65647bf993e059a40f36f0067acdb722 to: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12176553.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/3/ifsdata/ifsdata_12176553.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/3/159l_2/clim_data_12176553.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/3/159l_2/clim_data_12176553.zip
Checking for progress XML file: /var/lib/boinc-client/slots/3/progress_file_12176553.xml
Opened progress file ok : /var/lib/boinc-client/slots/3/progress_file_12176553.xml
-- Model is restarting --
Adjusting last_iter, 1074, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/3/progress_file_12176553.xml
last_cpu_time: 37455
upload_file_number: 44
last_iter: 1055
last_upload: 3801600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0909_1990050100_123_959_12176553_1_r1610887553
The child process has been launched with process id: 488263
Executing the command: /var/lib/boinc-client/slots/3/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:dbgrensenh27vm:1:1:488263:488263] [20221231:103636:1672479396:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  10:37:29 STEP 1056 H=1056:00 +CPU= 48.072
  10:38:01 STEP 1057 H=1057:00 +CPU= 30.623
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:01:58 1498859 22294259 oifs_43r3_ps_0909_1990050100_123_959_12176553_1 3,456,000 33,721 0.0098
09 May 2023 16:01:47 1498859 22294259 oifs_43r3_ps_0909_1990050100_123_959_12176553_1 2,592,000 25,832 0.0100
09 May 2023 16:01:39 1498859 22294259 oifs_43r3_ps_0909_1990050100_123_959_12176553_1 1,728,000 17,243 0.0100
09 May 2023 16:01:32 1498859 22294259 oifs_43r3_ps_0909_1990050100_123_959_12176553_1 864,000 8,599 0.0100


©2024 climateprediction.net