climateprediction.net home page
Task 22301642

Task 22301642

Name oifs_43r3_ps_0228_2004050100_123_973_12189872_1
Workunit 12189872
Created 22 Jan 2023, 7:45:17 UTC
Sent 22 Jan 2023, 7:53:57 UTC
Report deadline 23 Mar 2023, 7:53:57 UTC
Received 24 Jan 2023, 4:20:45 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 148 (0x00000094) Unknown error code
Computer ID 1526251
Run time 17 hours 53 min 25 sec
CPU time 17 hours 50 min 34 sec
Validate state Invalid
Credit 10,886.40
Device peak FLOPS 6.37 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,413.23 MB
Peak swap size 4,978.07 MB
Peak disk usage 4,135.18 MB
Stderr
<core_client_version>7.16.6</core_client_version>
<![CDATA[
<message>
process exited with code 148 (0x94, -108)</message>
<stderr_txt>
123:181800:1674490680:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:181800:1674490680:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  18:18:51 STEP 1680 H=1680:00 +CPU= 42.522
  18:19:11 STEP 1681 H=1681:00 +CPU= 19.418
  18:19:24 STEP 1682 H=1682:00 +CPU= 13.427
  18:19:43 STEP 1683 H=1683:00 +CPU= 18.757
  18:20:02 STEP 1684 H=1684:00 +CPU= 19.065
  18:20:21 STEP 1685 H=1685:00 +CPU= 19.289
  18:20:50 STEP 1686 H=1686:00 +CPU= 28.734
  18:21:09 STEP 1687 H=1687:00 +CPU= 18.297
  18:21:27 STEP 1688 H=1688:00 +CPU= 18.413
  18:21:54 STEP 1689 H=1689:00 +CPU= 26.603
  18:22:12 STEP 1690 H=1690:00 +CPU= 17.814
  18:22:29 STEP 1691 H=1691:00 +CPU= 17.654
  18:23:00 STEP 1692 H=1692:00 +CPU= 30.133
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001692
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001692
  18:23:17 STEP 1693 H=1693:00 +CPU= 17.406
  18:23:34 STEP 1694 H=1694:00 +CPU= 17.373
  18:24:01 STEP 1695 H=1695:00 +CPU= 26.838
  18:24:19 STEP 1696 H=1696:00 +CPU= 18.136
  18:24:37 STEP 1697 H=1697:00 +CPU= 17.585
  18:25:03 STEP 1698 H=1698:00 +CPU= 26.062
  18:25:21 STEP 1699 H=1699:00 +CPU= 17.658
  18:25:40 STEP 1700 H=1700:00 +CPU= 18.614
  18:26:06 STEP 1701 H=1701:00 +CPU= 26.512
  18:26:26 STEP 1702 H=1702:00 +CPU= 19.417
  18:26:44 STEP 1703 H=1703:00 +CPU= 18.702
  18:27:23 STEP 1704 H=1704:00 +CPU= 38.409
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001704
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001680
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001692
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001692
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_70.zip
Uploading the intermediate file: upload_file_70.zip
  18:27:46 STEP 1705 H=1705:00 +CPU= 23.260
  18:28:10 STEP 1706 H=1706:00 +CPU= 23.874
  18:28:46 STEP 1707 H=1707:00 +CPU= 35.865
  18:29:11 STEP 1708 H=1708:00 +CPU= 24.975
  18:29:35 STEP 1709 H=1709:00 +CPU= 23.872
  18:30:12 STEP 1710 H=1710:00 +CPU= 36.848
  18:30:37 STEP 1711 H=1711:00 +CPU= 24.666
  18:31:01 STEP 1712 H=1712:00 +CPU= 23.848
  18:31:37 STEP 1713 H=1713:00 +CPU= 35.841
  18:32:01 STEP 1714 H=1714:00 +CPU= 23.799
  18:32:25 STEP 1715 H=1715:00 +CPU= 24.252
  18:33:05 STEP 1716 H=1716:00 +CPU= 39.299
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001716
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001716
  18:33:30 STEP 1717 H=1717:00 +CPU= 24.975
  18:33:54 STEP 1718 H=1718:00 +CPU= 24.740
  18:34:31 STEP 1719 H=1719:00 +CPU= 36.448
  18:34:55 STEP 1720 H=1720:00 +CPU= 24.005
  18:35:19 STEP 1721 H=1721:00 +CPU= 24.080
  18:35:55 STEP 1722 H=1722:00 +CPU= 35.801
  18:36:18 STEP 1723 H=1723:00 +CPU= 23.164
  18:36:43 STEP 1724 H=1724:00 +CPU= 24.467
  18:37:17 STEP 1725 H=1725:00 +CPU= 34.436
  18:37:41 STEP 1726 H=1726:00 +CPU= 23.218
  18:38:04 STEP 1727 H=1727:00 +CPU= 23.740
  18:38:46 STEP 1728 H=1728:00 +CPU= 41.786
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001728
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001704
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001716
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001716
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_71.zip
Uploading the intermediate file: upload_file_71.zip
  18:39:10 STEP 1729 H=1729:00 +CPU= 23.837
  18:39:35 STEP 1730 H=1730:00 +CPU= 24.429
  18:40:09 STEP 1731 H=1731:00 +CPU= 34.270
  18:40:33 STEP 1732 H=1732:00 +CPU= 24.278
  18:40:57 STEP 1733 H=1733:00 +CPU= 23.128
  18:41:31 STEP 1734 H=1734:00 +CPU= 34.553
  18:41:54 STEP 1735 H=1735:00 +CPU= 22.732
  18:42:18 STEP 1736 H=1736:00 +CPU= 23.663
  18:42:53 STEP 1737 H=1737:00 +CPU= 35.029
  18:43:17 STEP 1738 H=1738:00 +CPU= 24.188
  18:43:41 STEP 1739 H=1739:00 +CPU= 23.755
  18:44:20 STEP 1740 H=1740:00 +CPU= 39.567
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001740
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001740
  18:44:44 STEP 1741 H=1741:00 +CPU= 23.651
  18:45:09 STEP 1742 H=1742:00 +CPU= 24.305
  18:45:44 STEP 1743 H=1743:00 +CPU= 35.636
  18:46:09 STEP 1744 H=1744:00 +CPU= 24.364
  18:46:33 STEP 1745 H=1745:00 +CPU= 24.262
  18:47:09 STEP 1746 H=1746:00 +CPU= 36.233
  18:47:34 STEP 1747 H=1747:00 +CPU= 24.322
  18:47:57 STEP 1748 H=1748:00 +CPU= 23.624
  18:48:34 STEP 1749 H=1749:00 +CPU= 36.221
  18:48:59 STEP 1750 H=1750:00 +CPU= 24.920
  18:49:24 STEP 1751 H=1751:00 +CPU= 24.917
  18:50:06 STEP 1752 H=1752:00 +CPU= 42.683
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001752
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001728
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001740
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001740
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_72.zip
Uploading the intermediate file: upload_file_72.zip
  18:50:30 STEP 1753 H=1753:00 +CPU= 23.189
  18:50:53 STEP 1754 H=1754:00 +CPU= 23.812
  18:51:30 STEP 1755 H=1755:00 +CPU= 36.189
  18:51:54 STEP 1756 H=1756:00 +CPU= 23.924
  18:52:17 STEP 1757 H=1757:00 +CPU= 22.988
  18:52:52 STEP 1758 H=1758:00 +CPU= 35.124
  18:53:15 STEP 1759 H=1759:00 +CPU= 23.523
  18:53:40 STEP 1760 H=1760:00 +CPU= 24.334
  18:54:15 STEP 1761 H=1761:00 +CPU= 35.646
  18:54:40 STEP 1762 H=1762:00 +CPU= 24.678
  18:55:05 STEP 1763 H=1763:00 +CPU= 24.259
  18:55:45 STEP 1764 H=1764:00 +CPU= 40.239
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001764
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001764
  18:56:10 STEP 1765 H=1765:00 +CPU= 25.400
  18:56:36 STEP 1766 H=1766:00 +CPU= 25.451
  18:57:12 STEP 1767 H=1767:00 +CPU= 36.092
  18:57:37 STEP 1768 H=1768:00 +CPU= 25.244
  18:58:02 STEP 1769 H=1769:00 +CPU= 25.352
  18:58:40 STEP 1770 H=1770:00 +CPU= 37.466
  18:59:05 STEP 1771 H=1771:00 +CPU= 25.205
  18:59:30 STEP 1772 H=1772:00 +CPU= 24.993
  19:00:08 STEP 1773 H=1773:00 +CPU= 37.246
  19:00:36 STEP 1774 H=1774:00 +CPU= 27.775
  19:01:03 STEP 1775 H=1775:00 +CPU= 26.345
  19:01:47 STEP 1776 H=1776:00 +CPU= 43.798
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001776
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001752
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001764
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001764
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_73.zip
Uploading the intermediate file: upload_file_73.zip
  19:02:10 STEP 1777 H=1777:00 +CPU= 23.549
  19:02:35 STEP 1778 H=1778:00 +CPU= 24.683
  19:03:12 STEP 1779 H=1779:00 +CPU= 36.623
  19:03:37 STEP 1780 H=1780:00 +CPU= 25.018
  19:04:02 STEP 1781 H=1781:00 +CPU= 25.023
  19:04:39 STEP 1782 H=1782:00 +CPU= 36.640
  19:05:02 STEP 1783 H=1783:00 +CPU= 22.707
  19:05:25 STEP 1784 H=1784:00 +CPU= 23.596
  19:06:00 STEP 1785 H=1785:00 +CPU= 34.844
  19:06:24 STEP 1786 H=1786:00 +CPU= 24.126
  19:06:47 STEP 1787 H=1787:00 +CPU= 22.700
  19:07:25 STEP 1788 H=1788:00 +CPU= 38.245
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001788
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001788
  19:07:48 STEP 1789 H=1789:00 +CPU= 22.791
  19:08:12 STEP 1790 H=1790:00 +CPU= 24.017
  19:08:49 STEP 1791 H=1791:00 +CPU= 37.063
  19:09:13 STEP 1792 H=1792:00 +CPU= 23.807
  19:09:37 STEP 1793 H=1793:00 +CPU= 24.084
  19:10:13 STEP 1794 H=1794:00 +CPU= 35.761
  19:10:36 STEP 1795 H=1795:00 +CPU= 22.741
  19:11:00 STEP 1796 H=1796:00 +CPU= 23.461
  19:11:34 STEP 1797 H=1797:00 +CPU= 34.475
  19:11:58 STEP 1798 H=1798:00 +CPU= 23.525
  19:12:21 STEP 1799 H=1799:00 +CPU= 23.098
  19:13:02 STEP 1800 H=1800:00 +CPU= 41.159
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001800
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001776
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001788
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001788
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_74.zip
Uploading the intermediate file: upload_file_74.zip
  19:13:26 STEP 1801 H=1801:00 +CPU= 23.861
  19:13:50 STEP 1802 H=1802:00 +CPU= 24.031
  19:14:25 STEP 1803 H=1803:00 +CPU= 34.548
  19:14:49 STEP 1804 H=1804:00 +CPU= 24.006
  19:15:12 STEP 1805 H=1805:00 +CPU= 23.237
  19:15:40 STEP 1806 H=1806:00 +CPU= 28.371
  19:15:59 STEP 1807 H=1807:00 +CPU= 19.068
  19:16:28 STEP 1808 H=1808:00 +CPU= 28.811
  19:17:09 STEP 1809 H=1809:00 +CPU= 40.425
  19:17:37 STEP 1810 H=1810:00 +CPU= 27.870
  19:18:05 STEP 1811 H=1811:00 +CPU= 27.734
  19:18:51 STEP 1812 H=1812:00 +CPU= 46.038
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001812
  19:19:20 STEP 1813 H=1813:00 +CPU= 29.042
  19:19:49 STEP 1814 H=1814:00 +CPU= 28.908
  19:20:25 STEP 1815 H=1815:00 +CPU= 35.968
  19:20:53 STEP 1816 H=1816:00 +CPU= 27.917
  19:21:21 STEP 1817 H=1817:00 +CPU= 27.740
  19:22:02 STEP 1818 H=1818:00 +CPU= 41.287
  19:22:31 STEP 1819 H=1819:00 +CPU= 29.084
  19:23:00 STEP 1820 H=1820:00 +CPU= 28.692
  19:23:41 STEP 1821 H=1821:00 +CPU= 41.291
  19:24:10 STEP 1822 H=1822:00 +CPU= 28.181
  19:24:38 STEP 1823 H=1823:00 +CPU= 28.318
  19:25:12 STEP 1824 H=1824:00 +CPU= 33.923
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_75.zip
Uploading the intermediate file: upload_file_75.zip
  19:25:36 STEP 1825 H=1825:00 +CPU= 24.054
  19:26:01 STEP 1826 H=1826:00 +CPU= 24.466
  19:26:37 STEP 1827 H=1827:00 +CPU= 35.891
  19:27:00 STEP 1828 H=1828:00 +CPU= 23.467
  19:27:23 STEP 1829 H=1829:00 +CPU= 23.047
  19:27:59 STEP 1830 H=1830:00 +CPU= 35.719
  19:28:23 STEP 1831 H=1831:00 +CPU= 24.153
  19:28:48 STEP 1832 H=1832:00 +CPU= 24.659
  19:29:22 STEP 1833 H=1833:00 +CPU= 34.623
  19:29:46 STEP 1834 H=1834:00 +CPU= 23.680
  19:30:09 STEP 1835 H=1835:00 +CPU= 23.166
  19:30:48 STEP 1836 H=1836:00 +CPU= 38.287
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001836
  19:31:12 STEP 1837 H=1837:00 +CPU= 23.675
  19:31:37 STEP 1838 H=1838:00 +CPU= 25.081
  19:32:12 STEP 1839 H=1839:00 +CPU= 35.362
  19:32:36 STEP 1840 H=1840:00 +CPU= 23.807
  19:33:00 STEP 1841 H=1841:00 +CPU= 23.809
  19:33:35 STEP 1842 H=1842:00 +CPU= 35.425
  19:33:58 STEP 1843 H=1843:00 +CPU= 22.980
  19:34:22 STEP 1844 H=1844:00 +CPU= 23.809
  19:34:57 STEP 1845 H=1845:00 +CPU= 34.782
  19:35:20 STEP 1846 H=1846:00 +CPU= 23.490
  19:35:45 STEP 1847 H=1847:00 +CPU= 24.909
  19:36:28 STEP 1848 H=1848:00 +CPU= 42.207
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_76.zip
Uploading the intermediate file: upload_file_76.zip
  19:36:51 STEP 1849 H=1849:00 +CPU= 23.281
  19:37:14 STEP 1850 H=1850:00 +CPU= 23.199
  19:37:49 STEP 1851 H=1851:00 +CPU= 34.736
  19:38:13 STEP 1852 H=1852:00 +CPU= 23.886
  19:38:36 STEP 1853 H=1853:00 +CPU= 23.086
  19:39:12 STEP 1854 H=1854:00 +CPU= 35.634
  19:39:36 STEP 1855 H=1855:00 +CPU= 24.549
  19:40:01 STEP 1856 H=1856:00 +CPU= 24.595
  19:40:36 STEP 1857 H=1857:00 +CPU= 35.481
  19:41:01 STEP 1858 H=1858:00 +CPU= 24.010
  19:41:25 STEP 1859 H=1859:00 +CPU= 24.574
  19:42:06 STEP 1860 H=1860:00 +CPU= 40.914
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001860
  19:42:29 STEP 1861 H=1861:00 +CPU= 22.865
  19:42:54 STEP 1862 H=1862:00 +CPU= 25.005
  19:43:30 STEP 1863 H=1863:00 +CPU= 36.282
  19:43:56 STEP 1864 H=1864:00 +CPU= 25.074
  19:44:20 STEP 1865 H=1865:00 +CPU= 24.748
  19:44:59 STEP 1866 H=1866:00 +CPU= 38.139
  19:45:23 STEP 1867 H=1867:00 +CPU= 24.936
  19:45:48 STEP 1868 H=1868:00 +CPU= 24.668
  19:46:24 STEP 1869 H=1869:00 +CPU= 35.641
  19:46:49 STEP 1870 H=1870:00 +CPU= 24.922
  19:47:14 STEP 1871 H=1871:00 +CPU= 24.850
  19:47:56 STEP 1872 H=1872:00 +CPU= 41.964
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_77.zip
Uploading the intermediate file: upload_file_77.zip
  19:48:19 STEP 1873 H=1873:00 +CPU= 23.089
  19:48:43 STEP 1874 H=1874:00 +CPU= 23.783
  19:49:21 STEP 1875 H=1875:00 +CPU= 37.778
  19:49:48 STEP 1876 H=1876:00 +CPU= 27.106
  19:50:14 STEP 1877 H=1877:00 +CPU= 26.297
  19:50:51 STEP 1878 H=1878:00 +CPU= 36.486
  19:51:15 STEP 1879 H=1879:00 +CPU= 23.735
  19:51:39 STEP 1880 H=1880:00 +CPU= 24.524
  19:52:15 STEP 1881 H=1881:00 +CPU= 36.160
  19:52:39 STEP 1882 H=1882:00 +CPU= 24.066
  19:53:04 STEP 1883 H=1883:00 +CPU= 24.502
  19:53:43 STEP 1884 H=1884:00 +CPU= 39.190
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001884
  19:54:06 STEP 1885 H=1885:00 +CPU= 22.873
  19:54:30 STEP 1886 H=1886:00 +CPU= 23.574
  19:55:05 STEP 1887 H=1887:00 +CPU= 35.558
  19:55:30 STEP 1888 H=1888:00 +CPU= 24.783
  19:55:54 STEP 1889 H=1889:00 +CPU= 23.576
  19:56:30 STEP 1890 H=1890:00 +CPU= 35.740
  19:56:53 STEP 1891 H=1891:00 +CPU= 23.049
  19:57:17 STEP 1892 H=1892:00 +CPU= 23.834
  19:57:51 STEP 1893 H=1893:00 +CPU= 33.972
  19:58:14 STEP 1894 H=1894:00 +CPU= 23.375
  19:58:38 STEP 1895 H=1895:00 +CPU= 23.664
  19:59:21 STEP 1896 H=1896:00 +CPU= 43.062
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_78.zip
Uploading the intermediate file: upload_file_78.zip
  19:59:46 STEP 1897 H=1897:00 +CPU= 24.522
  20:00:09 STEP 1898 H=1898:00 +CPU= 23.784
  20:00:45 STEP 1899 H=1899:00 +CPU= 35.127
  20:01:08 STEP 1900 H=1900:00 +CPU= 23.644
  20:01:31 STEP 1901 H=1901:00 +CPU= 22.845
  20:02:08 STEP 1902 H=1902:00 +CPU= 36.837
  20:02:33 STEP 1903 H=1903:00 +CPU= 24.375
  20:02:56 STEP 1904 H=1904:00 +CPU= 23.734
  20:03:32 STEP 1905 H=1905:00 +CPU= 35.432
  20:03:56 STEP 1906 H=1906:00 +CPU= 24.148
  20:04:19 STEP 1907 H=1907:00 +CPU= 23.274
  20:04:58 STEP 1908 H=1908:00 +CPU= 39.025
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001908
  20:05:22 STEP 1909 H=1909:00 +CPU= 23.924
  20:05:46 STEP 1910 H=1910:00 +CPU= 24.078
  20:06:22 STEP 1911 H=1911:00 +CPU= 35.617
  20:06:46 STEP 1912 H=1912:00 +CPU= 23.616
  20:07:09 STEP 1913 H=1913:00 +CPU= 23.222
  20:07:45 STEP 1914 H=1914:00 +CPU= 35.726
  20:08:08 STEP 1915 H=1915:00 +CPU= 23.548
  20:08:33 STEP 1916 H=1916:00 +CPU= 24.487
  20:09:09 STEP 1917 H=1917:00 +CPU= 36.266
  20:09:34 STEP 1918 H=1918:00 +CPU= 24.804
  20:09:58 STEP 1919 H=1919:00 +CPU= 24.377
  20:10:43 STEP 1920 H=1920:00 +CPU= 44.668
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_79.zip
Uploading the intermediate file: upload_file_79.zip
  20:11:08 STEP 1921 H=1921:00 +CPU= 24.337
Uploading trickle at timestep: 6912000
  20:11:32 STEP 1922 H=1922:00 +CPU= 24.544
  20:12:09 STEP 1923 H=1923:00 +CPU= 36.852
  20:12:34 STEP 1924 H=1924:00 +CPU= 24.559
  20:12:58 STEP 1925 H=1925:00 +CPU= 24.333
  20:13:37 STEP 1926 H=1926:00 +CPU= 38.680
  20:14:03 STEP 1927 H=1927:00 +CPU= 25.920
  20:14:29 STEP 1928 H=1928:00 +CPU= 26.631
  20:15:08 STEP 1929 H=1929:00 +CPU= 38.516
  20:15:35 STEP 1930 H=1930:00 +CPU= 26.511
  20:16:00 STEP 1931 H=1931:00 +CPU= 25.831
  20:16:43 STEP 1932 H=1932:00 +CPU= 42.515
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001932
  20:17:09 STEP 1933 H=1933:00 +CPU= 25.761
  20:17:36 STEP 1934 H=1934:00 +CPU= 27.100
  20:18:14 STEP 1935 H=1935:00 +CPU= 38.334
  20:18:41 STEP 1936 H=1936:00 +CPU= 26.938
  20:19:07 STEP 1937 H=1937:00 +CPU= 26.040
  20:19:47 STEP 1938 H=1938:00 +CPU= 39.185
  20:20:13 STEP 1939 H=1939:00 +CPU= 26.215
  20:20:40 STEP 1940 H=1940:00 +CPU= 27.162
  20:21:12 STEP 1941 H=1941:00 +CPU= 31.708
  20:21:26 STEP 1942 H=1942:00 +CPU= 13.783
  20:21:51 STEP 1943 H=1943:00 +CPU= 25.055
  20:22:36 STEP 1944 H=1944:00 +CPU= 45.389
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_80.zip
Uploading the intermediate file: upload_file_80.zip
  20:23:03 STEP 1945 H=1945:00 +CPU= 26.691
  20:23:29 STEP 1946 H=1946:00 +CPU= 25.518
  20:24:06 STEP 1947 H=1947:00 +CPU= 37.527
  20:24:30 STEP 1948 H=1948:00 +CPU= 24.026
  20:24:53 STEP 1949 H=1949:00 +CPU= 22.560
  20:25:27 STEP 1950 H=1950:00 +CPU= 33.632
  20:25:47 STEP 1951 H=1951:00 +CPU= 20.379
  20:26:08 STEP 1952 H=1952:00 +CPU= 21.062
  20:26:39 STEP 1953 H=1953:00 +CPU= 30.701
  20:27:00 STEP 1954 H=1954:00 +CPU= 21.368
  20:27:21 STEP 1955 H=1955:00 +CPU= 20.429
  20:27:53 STEP 1956 H=1956:00 +CPU= 32.081
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001956
  20:28:12 STEP 1957 H=1957:00 +CPU= 19.309
  20:28:32 STEP 1958 H=1958:00 +CPU= 19.843
  20:29:02 STEP 1959 H=1959:00 +CPU= 29.912
  20:29:22 STEP 1960 H=1960:00 +CPU= 19.669
  20:29:40 STEP 1961 H=1961:00 +CPU= 18.730
  20:30:10 STEP 1962 H=1962:00 +CPU= 29.658
  20:30:35 STEP 1963 H=1963:00 +CPU= 24.423
  20:31:00 STEP 1964 H=1964:00 +CPU= 24.878
  20:31:36 STEP 1965 H=1965:00 +CPU= 36.371
  20:32:01 STEP 1966 H=1966:00 +CPU= 24.596
  20:32:26 STEP 1967 H=1967:00 +CPU= 25.288
  20:33:09 STEP 1968 H=1968:00 +CPU= 43.189
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_81.zip
Uploading the intermediate file: upload_file_81.zip
  20:33:34 STEP 1969 H=1969:00 +CPU= 24.559
  20:33:59 STEP 1970 H=1970:00 +CPU= 25.648
  20:34:36 STEP 1971 H=1971:00 +CPU= 36.637
  20:35:01 STEP 1972 H=1972:00 +CPU= 24.477
  20:35:25 STEP 1973 H=1973:00 +CPU= 24.242
  20:36:02 STEP 1974 H=1974:00 +CPU= 37.063
  20:36:27 STEP 1975 H=1975:00 +CPU= 25.056
  20:36:53 STEP 1976 H=1976:00 +CPU= 25.349
  20:37:29 STEP 1977 H=1977:00 +CPU= 36.492
  20:37:54 STEP 1978 H=1978:00 +CPU= 24.504
  20:38:18 STEP 1979 H=1979:00 +CPU= 24.274
  20:38:59 STEP 1980 H=1980:00 +CPU= 40.610
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001980
  20:39:23 STEP 1981 H=1981:00 +CPU= 24.357
  20:39:48 STEP 1982 H=1982:00 +CPU= 25.158
  20:40:25 STEP 1983 H=1983:00 +CPU= 36.793
  20:40:50 STEP 1984 H=1984:00 +CPU= 24.831
  20:41:14 STEP 1985 H=1985:00 +CPU= 24.098
  20:41:51 STEP 1986 H=1986:00 +CPU= 37.414
  20:42:16 STEP 1987 H=1987:00 +CPU= 24.266
  20:42:41 STEP 1988 H=1988:00 +CPU= 25.004
  20:43:18 STEP 1989 H=1989:00 +CPU= 37.028
  20:43:43 STEP 1990 H=1990:00 +CPU= 25.285
  20:44:08 STEP 1991 H=1991:00 +CPU= 25.103
  20:44:52 STEP 1992 H=1992:00 +CPU= 43.790
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_82.zip
Uploading the intermediate file: upload_file_82.zip
  20:45:17 STEP 1993 H=1993:00 +CPU= 25.014
  20:45:43 STEP 1994 H=1994:00 +CPU= 25.055
  20:46:20 STEP 1995 H=1995:00 +CPU= 37.037
  20:46:45 STEP 1996 H=1996:00 +CPU= 25.329
  20:47:09 STEP 1997 H=1997:00 +CPU= 24.296
  20:47:47 STEP 1998 H=1998:00 +CPU= 37.111
  20:48:11 STEP 1999 H=1999:00 +CPU= 24.469
  20:48:36 STEP 2000 H=2000:00 +CPU= 25.240
  20:49:14 STEP 2001 H=2001:00 +CPU= 37.131
  20:49:39 STEP 2002 H=2002:00 +CPU= 24.938
  20:50:04 STEP 2003 H=2003:00 +CPU= 24.908
  20:50:45 STEP 2004 H=2004:00 +CPU= 41.384
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002004
  20:51:10 STEP 2005 H=2005:00 +CPU= 24.503
  20:51:35 STEP 2006 H=2006:00 +CPU= 25.079
  20:52:12 STEP 2007 H=2007:00 +CPU= 36.754
  20:52:37 STEP 2008 H=2008:00 +CPU= 25.153
  20:53:01 STEP 2009 H=2009:00 +CPU= 24.313
  20:53:39 STEP 2010 H=2010:00 +CPU= 37.923
  20:54:04 STEP 2011 H=2011:00 +CPU= 24.691
  20:54:29 STEP 2012 H=2012:00 +CPU= 25.165
  20:55:06 STEP 2013 H=2013:00 +CPU= 37.067
  20:55:31 STEP 2014 H=2014:00 +CPU= 25.134
  20:55:57 STEP 2015 H=2015:00 +CPU= 25.331
  20:56:41 STEP 2016 H=2016:00 +CPU= 43.945
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_83.zip
Uploading the intermediate file: upload_file_83.zip
  20:57:05 STEP 2017 H=2017:00 +CPU= 24.395
  20:57:30 STEP 2018 H=2018:00 +CPU= 24.329
  20:58:06 STEP 2019 H=2019:00 +CPU= 36.606
  20:58:30 STEP 2020 H=2020:00 +CPU= 23.694
  20:58:53 STEP 2021 H=2021:00 +CPU= 23.175
  20:59:29 STEP 2022 H=2022:00 +CPU= 35.512
  20:59:52 STEP 2023 H=2023:00 +CPU= 23.143
  21:00:15 STEP 2024 H=2024:00 +CPU= 23.260
  21:00:50 STEP 2025 H=2025:00 +CPU= 35.096
  21:01:13 STEP 2026 H=2026:00 +CPU= 23.213
  21:01:36 STEP 2027 H=2027:00 +CPU= 22.799
  21:02:17 STEP 2028 H=2028:00 +CPU= 40.475
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002028
  21:02:45 STEP 2029 H=2029:00 +CPU= 27.821
  21:03:12 STEP 2030 H=2030:00 +CPU= 27.239
  21:03:48 STEP 2031 H=2031:00 +CPU= 36.481
  21:04:13 STEP 2032 H=2032:00 +CPU= 24.604
  21:04:37 STEP 2033 H=2033:00 +CPU= 24.100
  21:05:13 STEP 2034 H=2034:00 +CPU= 36.270
  21:05:37 STEP 2035 H=2035:00 +CPU= 23.811
  21:06:02 STEP 2036 H=2036:00 +CPU= 24.558
  21:06:37 STEP 2037 H=2037:00 +CPU= 35.386
  21:07:02 STEP 2038 H=2038:00 +CPU= 24.213
  21:07:25 STEP 2039 H=2039:00 +CPU= 23.300
  21:08:07 STEP 2040 H=2040:00 +CPU= 41.709
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_84.zip
Uploading the intermediate file: upload_file_84.zip
  21:08:30 STEP 2041 H=2041:00 +CPU= 23.214
  21:08:54 STEP 2042 H=2042:00 +CPU= 23.960
  21:09:29 STEP 2043 H=2043:00 +CPU= 35.204
  21:09:49 STEP 2044 H=2044:00 +CPU= 19.657
  21:10:08 STEP 2045 H=2045:00 +CPU= 18.624
  21:10:36 STEP 2046 H=2046:00 +CPU= 28.050
  21:10:54 STEP 2047 H=2047:00 +CPU= 18.579
  21:11:13 STEP 2048 H=2048:00 +CPU= 18.448
  21:11:41 STEP 2049 H=2049:00 +CPU= 28.186
  21:12:01 STEP 2050 H=2050:00 +CPU= 19.714
  21:12:20 STEP 2051 H=2051:00 +CPU= 18.782
  21:12:50 STEP 2052 H=2052:00 +CPU= 30.533
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002052
  21:13:09 STEP 2053 H=2053:00 +CPU= 18.117
  21:13:27 STEP 2054 H=2054:00 +CPU= 18.061
  21:13:53 STEP 2055 H=2055:00 +CPU= 26.002
  21:14:11 STEP 2056 H=2056:00 +CPU= 18.324
  21:14:29 STEP 2057 H=2057:00 +CPU= 17.465
  21:14:55 STEP 2058 H=2058:00 +CPU= 26.880
  21:15:13 STEP 2059 H=2059:00 +CPU= 17.188
  21:15:31 STEP 2060 H=2060:00 +CPU= 17.858
  21:15:58 STEP 2061 H=2061:00 +CPU= 26.974
  21:16:14 STEP 2062 H=2062:00 +CPU= 16.422
  21:16:31 STEP 2063 H=2063:00 +CPU= 17.350
  21:16:57 STEP 2064 H=2064:00 +CPU= 25.704
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_85.zip
Uploading the intermediate file: upload_file_85.zip
  21:17:08 STEP 2065 H=2065:00 +CPU= 10.561
  21:17:24 STEP 2066 H=2066:00 +CPU= 16.401
  21:17:52 STEP 2067 H=2067:00 +CPU= 27.496
  21:18:11 STEP 2068 H=2068:00 +CPU= 19.633
  21:18:30 STEP 2069 H=2069:00 +CPU= 18.759
  21:19:00 STEP 2070 H=2070:00 +CPU= 29.414
  21:19:18 STEP 2071 H=2071:00 +CPU= 18.866
  21:19:40 STEP 2072 H=2072:00 +CPU= 21.178
  21:20:17 STEP 2073 H=2073:00 +CPU= 37.570
  21:20:44 STEP 2074 H=2074:00 +CPU= 26.417
  21:21:08 STEP 2075 H=2075:00 +CPU= 24.785
  21:21:48 STEP 2076 H=2076:00 +CPU= 39.861
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002076
  21:22:13 STEP 2077 H=2077:00 +CPU= 24.589
  21:22:39 STEP 2078 H=2078:00 +CPU= 25.571
  21:23:15 STEP 2079 H=2079:00 +CPU= 36.308
  21:23:40 STEP 2080 H=2080:00 +CPU= 24.674
  21:24:05 STEP 2081 H=2081:00 +CPU= 24.864
  21:24:42 STEP 2082 H=2082:00 +CPU= 37.184
  21:25:07 STEP 2083 H=2083:00 +CPU= 24.682
  21:25:32 STEP 2084 H=2084:00 +CPU= 25.195
  21:26:09 STEP 2085 H=2085:00 +CPU= 36.644
  21:26:33 STEP 2086 H=2086:00 +CPU= 24.688
  21:27:00 STEP 2087 H=2087:00 +CPU= 26.199
  21:27:44 STEP 2088 H=2088:00 +CPU= 44.522
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_86.zip
Uploading the intermediate file: upload_file_86.zip
  21:28:09 STEP 2089 H=2089:00 +CPU= 24.660
  21:28:35 STEP 2090 H=2090:00 +CPU= 25.760
  21:29:12 STEP 2091 H=2091:00 +CPU= 36.692
  21:29:37 STEP 2092 H=2092:00 +CPU= 24.874
  21:30:02 STEP 2093 H=2093:00 +CPU= 24.912
  21:30:42 STEP 2094 H=2094:00 +CPU= 39.902
  21:31:09 STEP 2095 H=2095:00 +CPU= 27.753
  21:31:36 STEP 2096 H=2096:00 +CPU= 27.056
  21:32:14 STEP 2097 H=2097:00 +CPU= 37.509
  21:32:39 STEP 2098 H=2098:00 +CPU= 25.329
  21:33:04 STEP 2099 H=2099:00 +CPU= 25.017
  21:33:46 STEP 2100 H=2100:00 +CPU= 41.223
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002100
  21:34:10 STEP 2101 H=2101:00 +CPU= 24.334
  21:34:35 STEP 2102 H=2102:00 +CPU= 25.056
  21:35:13 STEP 2103 H=2103:00 +CPU= 37.987
  21:35:39 STEP 2104 H=2104:00 +CPU= 25.907
  21:36:04 STEP 2105 H=2105:00 +CPU= 25.026
  21:36:42 STEP 2106 H=2106:00 +CPU= 37.949
  21:37:08 STEP 2107 H=2107:00 +CPU= 25.367
  21:37:34 STEP 2108 H=2108:00 +CPU= 26.488
  21:38:10 STEP 2109 H=2109:00 +CPU= 36.054
  21:38:36 STEP 2110 H=2110:00 +CPU= 25.486
  21:39:01 STEP 2111 H=2111:00 +CPU= 24.923
  21:39:45 STEP 2112 H=2112:00 +CPU= 44.485
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_87.zip
Uploading the intermediate file: upload_file_87.zip
  21:40:10 STEP 2113 H=2113:00 +CPU= 24.774
  21:40:36 STEP 2114 H=2114:00 +CPU= 25.481
  21:41:13 STEP 2115 H=2115:00 +CPU= 36.560
  21:41:39 STEP 2116 H=2116:00 +CPU= 26.408
  21:42:04 STEP 2117 H=2117:00 +CPU= 24.981
  21:42:42 STEP 2118 H=2118:00 +CPU= 37.983
  21:43:07 STEP 2119 H=2119:00 +CPU= 24.936
  21:43:33 STEP 2120 H=2120:00 +CPU= 25.459
  21:44:10 STEP 2121 H=2121:00 +CPU= 37.299
  21:44:35 STEP 2122 H=2122:00 +CPU= 24.870
  21:45:01 STEP 2123 H=2123:00 +CPU= 26.080
  21:45:44 STEP 2124 H=2124:00 +CPU= 42.340
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002124
  21:46:09 STEP 2125 H=2125:00 +CPU= 24.731
  21:46:34 STEP 2126 H=2126:00 +CPU= 25.523
  21:47:13 STEP 2127 H=2127:00 +CPU= 38.391
  21:47:39 STEP 2128 H=2128:00 +CPU= 26.302
  21:48:03 STEP 2129 H=2129:00 +CPU= 24.300
  21:48:40 STEP 2130 H=2130:00 +CPU= 36.403
  21:49:03 STEP 2131 H=2131:00 +CPU= 23.723
  21:49:28 STEP 2132 H=2132:00 +CPU= 24.058
  21:50:03 STEP 2133 H=2133:00 +CPU= 35.347
  21:50:27 STEP 2134 H=2134:00 +CPU= 24.323
  21:50:52 STEP 2135 H=2135:00 +CPU= 24.804
  21:51:35 STEP 2136 H=2136:00 +CPU= 42.351
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_88.zip
Uploading the intermediate file: upload_file_88.zip
  21:51:58 STEP 2137 H=2137:00 +CPU= 23.814
  21:52:23 STEP 2138 H=2138:00 +CPU= 24.305
  21:52:58 STEP 2139 H=2139:00 +CPU= 35.501
  21:53:23 STEP 2140 H=2140:00 +CPU= 24.218
  21:53:47 STEP 2141 H=2141:00 +CPU= 23.912
  21:54:23 STEP 2142 H=2142:00 +CPU= 36.132
  21:54:47 STEP 2143 H=2143:00 +CPU= 24.314
  21:55:12 STEP 2144 H=2144:00 +CPU= 24.353
  21:55:48 STEP 2145 H=2145:00 +CPU= 35.793
  21:56:12 STEP 2146 H=2146:00 +CPU= 24.430
  21:56:37 STEP 2147 H=2147:00 +CPU= 24.407
  21:57:16 STEP 2148 H=2148:00 +CPU= 39.215
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002148
  21:57:40 STEP 2149 H=2149:00 +CPU= 24.588
  21:58:05 STEP 2150 H=2150:00 +CPU= 24.778
  21:58:42 STEP 2151 H=2151:00 +CPU= 36.280
  21:59:06 STEP 2152 H=2152:00 +CPU= 24.603
  21:59:30 STEP 2153 H=2153:00 +CPU= 23.852
  22:00:06 STEP 2154 H=2154:00 +CPU= 36.214
  22:00:30 STEP 2155 H=2155:00 +CPU= 23.911
  22:00:56 STEP 2156 H=2156:00 +CPU= 25.230
  22:01:32 STEP 2157 H=2157:00 +CPU= 36.321
  22:01:57 STEP 2158 H=2158:00 +CPU= 24.851
  22:02:21 STEP 2159 H=2159:00 +CPU= 24.634
  22:03:05 STEP 2160 H=2160:00 +CPU= 42.923
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_89.zip
Uploading the intermediate file: upload_file_89.zip
  22:03:29 STEP 2161 H=2161:00 +CPU= 24.275
Uploading trickle at timestep: 7776000
  22:03:53 STEP 2162 H=2162:00 +CPU= 24.551
  22:04:29 STEP 2163 H=2163:00 +CPU= 35.390
  22:04:53 STEP 2164 H=2164:00 +CPU= 23.773
  22:05:16 STEP 2165 H=2165:00 +CPU= 23.621
  22:05:53 STEP 2166 H=2166:00 +CPU= 36.917
  22:06:17 STEP 2167 H=2167:00 +CPU= 23.950
  22:06:42 STEP 2168 H=2168:00 +CPU= 24.599
  22:07:19 STEP 2169 H=2169:00 +CPU= 37.045
  22:07:45 STEP 2170 H=2170:00 +CPU= 25.641
  22:08:09 STEP 2171 H=2171:00 +CPU= 24.362
  22:08:50 STEP 2172 H=2172:00 +CPU= 40.669
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002172
  22:09:15 STEP 2173 H=2173:00 +CPU= 25.278
  22:09:41 STEP 2174 H=2174:00 +CPU= 25.601
  22:10:19 STEP 2175 H=2175:00 +CPU= 37.758
  22:10:44 STEP 2176 H=2176:00 +CPU= 25.193
  22:11:09 STEP 2177 H=2177:00 +CPU= 25.099
  22:11:47 STEP 2178 H=2178:00 +CPU= 37.426
  22:12:12 STEP 2179 H=2179:00 +CPU= 24.925
  22:12:37 STEP 2180 H=2180:00 +CPU= 25.293
  22:13:15 STEP 2181 H=2181:00 +CPU= 37.448
  22:13:40 STEP 2182 H=2182:00 +CPU= 25.135
  22:14:01 STEP 2183 H=2183:00 +CPU= 20.614
  22:14:38 STEP 2184 H=2184:00 +CPU= 37.069
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_90.zip
Uploading the intermediate file: upload_file_90.zip
  22:15:06 STEP 2185 H=2185:00 +CPU= 28.628
  22:15:36 STEP 2186 H=2186:00 +CPU= 29.233
  22:16:18 STEP 2187 H=2187:00 +CPU= 42.226
  22:16:48 STEP 2188 H=2188:00 +CPU= 29.726
  22:17:16 STEP 2189 H=2189:00 +CPU= 28.051
  22:17:58 STEP 2190 H=2190:00 +CPU= 42.091
  22:18:26 STEP 2191 H=2191:00 +CPU= 28.027
  22:18:41 STEP 2192 H=2192:00 +CPU= 14.326
  22:19:17 STEP 2193 H=2193:00 +CPU= 36.671
  22:19:42 STEP 2194 H=2194:00 +CPU= 24.959
  22:20:07 STEP 2195 H=2195:00 +CPU= 24.336
  22:20:46 STEP 2196 H=2196:00 +CPU= 39.317
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002196
  22:21:10 STEP 2197 H=2197:00 +CPU= 24.075
  22:21:35 STEP 2198 H=2198:00 +CPU= 24.718
  22:22:11 STEP 2199 H=2199:00 +CPU= 36.247
  22:22:36 STEP 2200 H=2200:00 +CPU= 24.247
  22:23:01 STEP 2201 H=2201:00 +CPU= 25.120
  22:23:39 STEP 2202 H=2202:00 +CPU= 37.788
  22:24:03 STEP 2203 H=2203:00 +CPU= 23.997
  22:24:27 STEP 2204 H=2204:00 +CPU= 24.313
  22:25:04 STEP 2205 H=2205:00 +CPU= 36.963
  22:25:30 STEP 2206 H=2206:00 +CPU= 25.879
  22:25:55 STEP 2207 H=2207:00 +CPU= 24.732
  22:26:40 STEP 2208 H=2208:00 +CPU= 44.955
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_91.zip
Uploading the intermediate file: upload_file_91.zip
  22:27:05 STEP 2209 H=2209:00 +CPU= 25.095
  22:27:30 STEP 2210 H=2210:00 +CPU= 25.039
  22:28:08 STEP 2211 H=2211:00 +CPU= 37.520
  22:28:33 STEP 2212 H=2212:00 +CPU= 25.367
  22:28:58 STEP 2213 H=2213:00 +CPU= 25.070
  22:29:36 STEP 2214 H=2214:00 +CPU= 37.670
  22:30:00 STEP 2215 H=2215:00 +CPU= 24.337
  22:30:26 STEP 2216 H=2216:00 +CPU= 25.678
  22:31:05 STEP 2217 H=2217:00 +CPU= 38.381
  22:31:30 STEP 2218 H=2218:00 +CPU= 25.498
  22:31:55 STEP 2219 H=2219:00 +CPU= 24.761
  22:32:36 STEP 2220 H=2220:00 +CPU= 40.672
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002220
  22:33:01 STEP 2221 H=2221:00 +CPU= 24.910
  22:33:26 STEP 2222 H=2222:00 +CPU= 25.454
  22:34:03 STEP 2223 H=2223:00 +CPU= 36.679
  22:34:29 STEP 2224 H=2224:00 +CPU= 26.151
  22:34:54 STEP 2225 H=2225:00 +CPU= 24.740
  22:35:32 STEP 2226 H=2226:00 +CPU= 37.668
  22:35:56 STEP 2227 H=2227:00 +CPU= 24.119
  22:36:21 STEP 2228 H=2228:00 +CPU= 24.979
  22:36:59 STEP 2229 H=2229:00 +CPU= 37.763
  22:37:24 STEP 2230 H=2230:00 +CPU= 24.915
  22:37:49 STEP 2231 H=2231:00 +CPU= 25.362
  22:38:34 STEP 2232 H=2232:00 +CPU= 44.306
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/2/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12189872/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0228_2004050100_123_973_12189872_1_r2031231547_92.zip
Uploading the intermediate file: upload_file_92.zip
  22:39:00 STEP 2233 H=2233:00 +CPU= 26.566
  22:39:26 STEP 2234 H=2234:00 +CPU= 25.767
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.054] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1538] Received signal#15 (SIGTERM) :: 4746MB (heap), 4416MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.060] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.060] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x81f3c0 (old at (nil))
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.060] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1617] Signal#15 was caused by unrecognized si_code [memaddr=0x1], nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.060] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#15, nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.060] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 4746 MB (maxheap), 4416 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.060] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            CALL_SL_HEAP 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             CALL_SL 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              >OMP-CALL_SL     1        (1004) 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               LAPINEA 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                LARMES 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 LARCINA 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223943:1674506383:15703.063] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                  LASCAW 
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223944:1674506384:15704.063] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:LINUX-VM-12700K:1:1:1901:1901] [20230123:223944:1674506384:15704.063] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000E01CE9  lascaw_                   494  lascaw.F90
oifs_43r3_model.e  0000000000DED525  larcina_                  319  larcina.F90
oifs_43r3_model.e  0000000000DEABF8  larmes_                   603  larmes.F90
oifs_43r3_model.e  0000000000DE50FD  lapinea_                  270  lapinea.F90
oifs_43r3_model.e  0000000000D7A4EC  call_sl_                  300  call_sl.F90
oifs_43r3_model.e  0000000000D784FD  call_sl_heap_              94  call_sl_heap.F90
oifs_43r3_model.e  0000000000BF9958  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:38:46 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 7,776,000 62,044 0.0080
09 May 2023 16:38:06 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 6,912,000 55,317 0.0080
09 May 2023 16:36:14 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 5,184,000 41,330 0.0080
09 May 2023 16:36:04 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 4,320,000 35,052 0.0081
09 May 2023 16:35:54 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 3,456,000 27,808 0.0080
09 May 2023 16:35:43 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 2,592,000 20,591 0.0079
09 May 2023 16:35:38 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 1,728,000 13,923 0.0081
09 May 2023 16:35:25 1526251 22301642 oifs_43r3_ps_0228_2004050100_123_973_12189872_1 864,000 7,195 0.0083


©2024 climateprediction.net