climateprediction.net home page
Task 22251694

Task 22251694

Name oifs_43r3_ps_0830_1981050100_123_950_12167474_0
Workunit 12167474
Created 20 Dec 2022, 20:32:39 UTC
Sent 20 Dec 2022, 20:38:01 UTC
Report deadline 19 Jan 2023, 20:38:01 UTC
Received 21 Dec 2022, 14:14:19 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1532374
Run time 15 hours 39 min 46 sec
CPU time 15 hours 29 min 39 sec
Validate state Invalid
Credit 12,096.00
Device peak FLOPS 5.40 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,406.76 MB
Peak swap size 4,974.33 MB
Peak disk usage 3,141.00 MB
Stderr
<core_client_version>7.20.2</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
hq0f+001836
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001836
  08:34:17 STEP 1837 H=1837:00 +CPU= 17.655
  08:34:35 STEP 1838 H=1838:00 +CPU= 18.232
  08:35:03 STEP 1839 H=1839:00 +CPU= 27.505
  08:35:21 STEP 1840 H=1840:00 +CPU= 18.070
  08:35:38 STEP 1841 H=1841:00 +CPU= 17.363
  08:36:07 STEP 1842 H=1842:00 +CPU= 28.384
  08:36:25 STEP 1843 H=1843:00 +CPU= 18.047
  08:36:44 STEP 1844 H=1844:00 +CPU= 18.151
  08:37:12 STEP 1845 H=1845:00 +CPU= 28.326
  08:37:31 STEP 1846 H=1846:00 +CPU= 18.183
  08:37:49 STEP 1847 H=1847:00 +CPU= 18.347
  08:38:23 STEP 1848 H=1848:00 +CPU= 33.256
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_76.zip
Uploading the intermediate file: upload_file_76.zip
  08:38:40 STEP 1849 H=1849:00 +CPU= 17.391
  08:38:58 STEP 1850 H=1850:00 +CPU= 17.994
  08:39:26 STEP 1851 H=1851:00 +CPU= 27.318
  08:39:44 STEP 1852 H=1852:00 +CPU= 18.012
  08:40:02 STEP 1853 H=1853:00 +CPU= 17.441
  08:40:29 STEP 1854 H=1854:00 +CPU= 27.549
  08:40:48 STEP 1855 H=1855:00 +CPU= 18.078
  08:41:06 STEP 1856 H=1856:00 +CPU= 17.892
  08:41:34 STEP 1857 H=1857:00 +CPU= 27.698
  08:41:52 STEP 1858 H=1858:00 +CPU= 18.765
  08:42:10 STEP 1859 H=1859:00 +CPU= 17.787
  08:42:41 STEP 1860 H=1860:00 +CPU= 30.495
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001860
  08:42:58 STEP 1861 H=1861:00 +CPU= 17.197
  08:43:17 STEP 1862 H=1862:00 +CPU= 18.097
  08:43:45 STEP 1863 H=1863:00 +CPU= 27.748
  08:44:03 STEP 1864 H=1864:00 +CPU= 18.014
  08:44:21 STEP 1865 H=1865:00 +CPU= 17.791
  08:44:49 STEP 1866 H=1866:00 +CPU= 27.620
  08:45:07 STEP 1867 H=1867:00 +CPU= 18.205
  08:45:25 STEP 1868 H=1868:00 +CPU= 18.310
  08:45:54 STEP 1869 H=1869:00 +CPU= 27.825
  08:46:12 STEP 1870 H=1870:00 +CPU= 17.980
  08:46:31 STEP 1871 H=1871:00 +CPU= 19.012
  08:47:05 STEP 1872 H=1872:00 +CPU= 33.479
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_77.zip
Uploading the intermediate file: upload_file_77.zip
  08:47:22 STEP 1873 H=1873:00 +CPU= 17.473
  08:47:41 STEP 1874 H=1874:00 +CPU= 18.299
  08:48:08 STEP 1875 H=1875:00 +CPU= 27.338
  08:48:26 STEP 1876 H=1876:00 +CPU= 17.435
  08:48:43 STEP 1877 H=1877:00 +CPU= 17.312
  08:49:11 STEP 1878 H=1878:00 +CPU= 27.586
  08:49:29 STEP 1879 H=1879:00 +CPU= 17.920
  08:49:47 STEP 1880 H=1880:00 +CPU= 18.014
  08:50:15 STEP 1881 H=1881:00 +CPU= 27.642
  08:50:34 STEP 1882 H=1882:00 +CPU= 18.366
  08:50:52 STEP 1883 H=1883:00 +CPU= 17.998
  08:51:23 STEP 1884 H=1884:00 +CPU= 30.590
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001884
  08:51:41 STEP 1885 H=1885:00 +CPU= 17.685
  08:51:58 STEP 1886 H=1886:00 +CPU= 17.366
  08:52:26 STEP 1887 H=1887:00 +CPU= 27.184
  08:52:44 STEP 1888 H=1888:00 +CPU= 17.842
  08:53:01 STEP 1889 H=1889:00 +CPU= 17.487
  08:53:30 STEP 1890 H=1890:00 +CPU= 28.238
  08:53:48 STEP 1891 H=1891:00 +CPU= 18.144
  08:54:07 STEP 1892 H=1892:00 +CPU= 18.597
  08:54:35 STEP 1893 H=1893:00 +CPU= 27.658
  08:54:53 STEP 1894 H=1894:00 +CPU= 18.257
  08:55:11 STEP 1895 H=1895:00 +CPU= 18.181
  08:55:45 STEP 1896 H=1896:00 +CPU= 33.244
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_78.zip
Uploading the intermediate file: upload_file_78.zip
  08:56:03 STEP 1897 H=1897:00 +CPU= 17.460
  08:56:20 STEP 1898 H=1898:00 +CPU= 17.610
  08:56:48 STEP 1899 H=1899:00 +CPU= 27.282
  08:57:06 STEP 1900 H=1900:00 +CPU= 18.095
  08:57:24 STEP 1901 H=1901:00 +CPU= 17.334
  08:57:52 STEP 1902 H=1902:00 +CPU= 27.879
  08:58:09 STEP 1903 H=1903:00 +CPU= 17.349
  08:58:28 STEP 1904 H=1904:00 +CPU= 18.138
  08:58:56 STEP 1905 H=1905:00 +CPU= 28.266
  08:59:15 STEP 1906 H=1906:00 +CPU= 18.335
  08:59:32 STEP 1907 H=1907:00 +CPU= 17.455
  09:00:02 STEP 1908 H=1908:00 +CPU= 29.994
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001908
  09:00:20 STEP 1909 H=1909:00 +CPU= 17.726
  09:00:38 STEP 1910 H=1910:00 +CPU= 17.484
  09:01:06 STEP 1911 H=1911:00 +CPU= 27.766
  09:01:25 STEP 1912 H=1912:00 +CPU= 18.534
  09:01:42 STEP 1913 H=1913:00 +CPU= 17.462
  09:02:10 STEP 1914 H=1914:00 +CPU= 27.633
  09:02:28 STEP 1915 H=1915:00 +CPU= 17.526
  09:02:46 STEP 1916 H=1916:00 +CPU= 17.881
  09:03:15 STEP 1917 H=1917:00 +CPU= 28.205
  09:03:34 STEP 1918 H=1918:00 +CPU= 18.740
  09:03:52 STEP 1919 H=1919:00 +CPU= 18.255
  09:04:25 STEP 1920 H=1920:00 +CPU= 32.770
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_79.zip
Uploading the intermediate file: upload_file_79.zip
  09:04:42 STEP 1921 H=1921:00 +CPU= 17.152
Uploading trickle at timestep: 6912000
  09:05:00 STEP 1922 H=1922:00 +CPU= 17.567
  09:05:28 STEP 1923 H=1923:00 +CPU= 27.125
  09:05:46 STEP 1924 H=1924:00 +CPU= 18.175
  09:06:04 STEP 1925 H=1925:00 +CPU= 17.539
  09:06:32 STEP 1926 H=1926:00 +CPU= 27.665
  09:06:50 STEP 1927 H=1927:00 +CPU= 17.807
  09:07:07 STEP 1928 H=1928:00 +CPU= 17.583
  09:07:35 STEP 1929 H=1929:00 +CPU= 27.990
  09:07:54 STEP 1930 H=1930:00 +CPU= 18.226
  09:08:12 STEP 1931 H=1931:00 +CPU= 17.782
  09:08:42 STEP 1932 H=1932:00 +CPU= 30.340
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001932
  09:09:00 STEP 1933 H=1933:00 +CPU= 17.745
  09:09:18 STEP 1934 H=1934:00 +CPU= 18.018
  09:09:46 STEP 1935 H=1935:00 +CPU= 27.654
  09:10:05 STEP 1936 H=1936:00 +CPU= 18.608
  09:10:23 STEP 1937 H=1937:00 +CPU= 17.776
  09:10:52 STEP 1938 H=1938:00 +CPU= 28.298
  09:11:09 STEP 1939 H=1939:00 +CPU= 17.583
  09:11:28 STEP 1940 H=1940:00 +CPU= 18.024
  09:11:56 STEP 1941 H=1941:00 +CPU= 27.772
  09:12:14 STEP 1942 H=1942:00 +CPU= 18.346
  09:12:33 STEP 1943 H=1943:00 +CPU= 18.814
  09:13:06 STEP 1944 H=1944:00 +CPU= 32.701
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_80.zip
Uploading the intermediate file: upload_file_80.zip
  09:13:24 STEP 1945 H=1945:00 +CPU= 18.082
  09:13:43 STEP 1946 H=1946:00 +CPU= 18.368
  09:14:12 STEP 1947 H=1947:00 +CPU= 29.002
  09:14:32 STEP 1948 H=1948:00 +CPU= 19.421
  09:14:49 STEP 1949 H=1949:00 +CPU= 17.428
  09:15:18 STEP 1950 H=1950:00 +CPU= 28.194
  09:15:36 STEP 1951 H=1951:00 +CPU= 17.756
  09:15:54 STEP 1952 H=1952:00 +CPU= 18.690
  09:16:23 STEP 1953 H=1953:00 +CPU= 28.369
  09:16:42 STEP 1954 H=1954:00 +CPU= 18.576
  09:17:00 STEP 1955 H=1955:00 +CPU= 17.963
  09:17:31 STEP 1956 H=1956:00 +CPU= 31.059
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001956
  09:17:49 STEP 1957 H=1957:00 +CPU= 17.948
  09:18:08 STEP 1958 H=1958:00 +CPU= 18.236
  09:18:36 STEP 1959 H=1959:00 +CPU= 28.308
  09:18:55 STEP 1960 H=1960:00 +CPU= 18.329
  09:19:13 STEP 1961 H=1961:00 +CPU= 17.717
  09:19:42 STEP 1962 H=1962:00 +CPU= 28.875
  09:20:01 STEP 1963 H=1963:00 +CPU= 18.654
  09:20:20 STEP 1964 H=1964:00 +CPU= 19.601
  09:20:49 STEP 1965 H=1965:00 +CPU= 27.978
  09:21:07 STEP 1966 H=1966:00 +CPU= 18.210
  09:21:25 STEP 1967 H=1967:00 +CPU= 18.273
  09:22:00 STEP 1968 H=1968:00 +CPU= 33.872
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_81.zip
Uploading the intermediate file: upload_file_81.zip
  09:22:17 STEP 1969 H=1969:00 +CPU= 17.284
  09:22:36 STEP 1970 H=1970:00 +CPU= 18.364
  09:23:04 STEP 1971 H=1971:00 +CPU= 28.415
  09:23:23 STEP 1972 H=1972:00 +CPU= 18.544
  09:23:40 STEP 1973 H=1973:00 +CPU= 17.229
  09:24:09 STEP 1974 H=1974:00 +CPU= 28.812
  09:24:28 STEP 1975 H=1975:00 +CPU= 18.537
  09:24:47 STEP 1976 H=1976:00 +CPU= 18.914
  09:25:16 STEP 1977 H=1977:00 +CPU= 28.604
  09:25:35 STEP 1978 H=1978:00 +CPU= 18.902
  09:25:53 STEP 1979 H=1979:00 +CPU= 18.219
  09:26:25 STEP 1980 H=1980:00 +CPU= 30.885
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001980
  09:26:43 STEP 1981 H=1981:00 +CPU= 18.326
  09:27:02 STEP 1982 H=1982:00 +CPU= 18.497
  09:27:29 STEP 1983 H=1983:00 +CPU= 27.462
  09:27:48 STEP 1984 H=1984:00 +CPU= 18.334
  09:28:06 STEP 1985 H=1985:00 +CPU= 18.006
  09:28:35 STEP 1986 H=1986:00 +CPU= 28.778
  09:28:53 STEP 1987 H=1987:00 +CPU= 17.675
  09:29:11 STEP 1988 H=1988:00 +CPU= 18.167
  09:29:40 STEP 1989 H=1989:00 +CPU= 28.375
  09:29:58 STEP 1990 H=1990:00 +CPU= 17.951
  09:30:17 STEP 1991 H=1991:00 +CPU= 18.890
  09:30:51 STEP 1992 H=1992:00 +CPU= 33.720
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_82.zip
Uploading the intermediate file: upload_file_82.zip
  09:31:09 STEP 1993 H=1993:00 +CPU= 17.884
  09:31:28 STEP 1994 H=1994:00 +CPU= 17.973
  09:31:56 STEP 1995 H=1995:00 +CPU= 28.029
  09:32:15 STEP 1996 H=1996:00 +CPU= 18.427
  09:32:33 STEP 1997 H=1997:00 +CPU= 17.771
  09:33:01 STEP 1998 H=1998:00 +CPU= 28.677
  09:33:20 STEP 1999 H=1999:00 +CPU= 17.980
  09:33:38 STEP 2000 H=2000:00 +CPU= 18.000
  09:34:06 STEP 2001 H=2001:00 +CPU= 28.066
  09:34:25 STEP 2002 H=2002:00 +CPU= 18.643
  09:34:44 STEP 2003 H=2003:00 +CPU= 18.516
  09:35:15 STEP 2004 H=2004:00 +CPU= 30.808
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002004
  09:35:33 STEP 2005 H=2005:00 +CPU= 18.095
  09:35:51 STEP 2006 H=2006:00 +CPU= 18.066
  09:36:20 STEP 2007 H=2007:00 +CPU= 28.783
  09:36:39 STEP 2008 H=2008:00 +CPU= 18.761
  09:36:58 STEP 2009 H=2009:00 +CPU= 17.919
  09:37:27 STEP 2010 H=2010:00 +CPU= 28.993
  09:37:45 STEP 2011 H=2011:00 +CPU= 18.022
  09:38:04 STEP 2012 H=2012:00 +CPU= 18.555
  09:38:32 STEP 2013 H=2013:00 +CPU= 27.776
  09:38:51 STEP 2014 H=2014:00 +CPU= 19.349
  09:39:11 STEP 2015 H=2015:00 +CPU= 19.041
  09:39:44 STEP 2016 H=2016:00 +CPU= 33.489
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_83.zip
Uploading the intermediate file: upload_file_83.zip
  09:40:02 STEP 2017 H=2017:00 +CPU= 17.695
  09:40:20 STEP 2018 H=2018:00 +CPU= 18.075
  09:40:49 STEP 2019 H=2019:00 +CPU= 27.988
  09:41:08 STEP 2020 H=2020:00 +CPU= 18.735
  09:41:26 STEP 2021 H=2021:00 +CPU= 17.855
  09:41:55 STEP 2022 H=2022:00 +CPU= 28.685
  09:42:13 STEP 2023 H=2023:00 +CPU= 18.152
  09:42:32 STEP 2024 H=2024:00 +CPU= 18.735
  09:43:01 STEP 2025 H=2025:00 +CPU= 28.959
  09:43:20 STEP 2026 H=2026:00 +CPU= 18.550
  09:43:38 STEP 2027 H=2027:00 +CPU= 18.265
  09:44:09 STEP 2028 H=2028:00 +CPU= 30.930
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002028
  09:44:27 STEP 2029 H=2029:00 +CPU= 17.728
  09:44:46 STEP 2030 H=2030:00 +CPU= 18.529
  09:45:15 STEP 2031 H=2031:00 +CPU= 29.097
  09:45:34 STEP 2032 H=2032:00 +CPU= 18.604
  09:45:53 STEP 2033 H=2033:00 +CPU= 18.557
  09:46:22 STEP 2034 H=2034:00 +CPU= 29.115
  09:46:40 STEP 2035 H=2035:00 +CPU= 17.872
  09:46:58 STEP 2036 H=2036:00 +CPU= 18.126
  09:47:27 STEP 2037 H=2037:00 +CPU= 28.513
  09:47:47 STEP 2038 H=2038:00 +CPU= 19.275
  09:48:06 STEP 2039 H=2039:00 +CPU= 19.406
  09:48:40 STEP 2040 H=2040:00 +CPU= 33.389
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_84.zip
Uploading the intermediate file: upload_file_84.zip
  09:48:58 STEP 2041 H=2041:00 +CPU= 18.108
  09:49:16 STEP 2042 H=2042:00 +CPU= 18.050
  09:49:45 STEP 2043 H=2043:00 +CPU= 28.349
  09:50:04 STEP 2044 H=2044:00 +CPU= 18.494
  09:50:22 STEP 2045 H=2045:00 +CPU= 18.353
  09:50:50 STEP 2046 H=2046:00 +CPU= 28.043
  09:51:08 STEP 2047 H=2047:00 +CPU= 17.794
  09:51:27 STEP 2048 H=2048:00 +CPU= 18.324
  09:51:55 STEP 2049 H=2049:00 +CPU= 28.237
  09:52:14 STEP 2050 H=2050:00 +CPU= 18.360
  09:52:33 STEP 2051 H=2051:00 +CPU= 18.837
  09:53:04 STEP 2052 H=2052:00 +CPU= 30.980
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002052
  09:53:23 STEP 2053 H=2053:00 +CPU= 18.874
  09:53:41 STEP 2054 H=2054:00 +CPU= 17.982
  09:54:09 STEP 2055 H=2055:00 +CPU= 27.954
  09:54:28 STEP 2056 H=2056:00 +CPU= 18.332
  09:54:46 STEP 2057 H=2057:00 +CPU= 18.286
  09:55:16 STEP 2058 H=2058:00 +CPU= 28.954
  09:55:34 STEP 2059 H=2059:00 +CPU= 17.984
  09:55:53 STEP 2060 H=2060:00 +CPU= 18.626
  09:56:22 STEP 2061 H=2061:00 +CPU= 28.942
  09:56:41 STEP 2062 H=2062:00 +CPU= 18.874
  09:57:00 STEP 2063 H=2063:00 +CPU= 18.828
  09:57:33 STEP 2064 H=2064:00 +CPU= 33.222
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_85.zip
Uploading the intermediate file: upload_file_85.zip
  09:57:51 STEP 2065 H=2065:00 +CPU= 17.794
  09:58:09 STEP 2066 H=2066:00 +CPU= 17.850
  09:58:37 STEP 2067 H=2067:00 +CPU= 27.767
  09:58:56 STEP 2068 H=2068:00 +CPU= 18.828
  09:59:15 STEP 2069 H=2069:00 +CPU= 18.199
  09:59:44 STEP 2070 H=2070:00 +CPU= 28.483
  10:00:02 STEP 2071 H=2071:00 +CPU= 18.140
  10:00:20 STEP 2072 H=2072:00 +CPU= 18.376
  10:00:49 STEP 2073 H=2073:00 +CPU= 28.221
  10:01:08 STEP 2074 H=2074:00 +CPU= 19.220
  10:01:27 STEP 2075 H=2075:00 +CPU= 18.454
  10:01:59 STEP 2076 H=2076:00 +CPU= 31.304
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002076
  10:02:17 STEP 2077 H=2077:00 +CPU= 17.984
  10:02:35 STEP 2078 H=2078:00 +CPU= 18.503
  10:03:03 STEP 2079 H=2079:00 +CPU= 27.650
  10:03:22 STEP 2080 H=2080:00 +CPU= 18.857
  10:03:41 STEP 2081 H=2081:00 +CPU= 18.144
  10:04:10 STEP 2082 H=2082:00 +CPU= 28.761
  10:04:28 STEP 2083 H=2083:00 +CPU= 18.272
  10:04:47 STEP 2084 H=2084:00 +CPU= 18.377
  10:05:15 STEP 2085 H=2085:00 +CPU= 28.069
  10:05:34 STEP 2086 H=2086:00 +CPU= 19.084
  10:05:53 STEP 2087 H=2087:00 +CPU= 18.151
  10:06:27 STEP 2088 H=2088:00 +CPU= 33.585
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_86.zip
Uploading the intermediate file: upload_file_86.zip
  10:06:44 STEP 2089 H=2089:00 +CPU= 17.390
  10:07:02 STEP 2090 H=2090:00 +CPU= 17.797
  10:07:31 STEP 2091 H=2091:00 +CPU= 28.281
  10:07:49 STEP 2092 H=2092:00 +CPU= 18.285
  10:08:07 STEP 2093 H=2093:00 +CPU= 17.890
  10:08:36 STEP 2094 H=2094:00 +CPU= 28.515
  10:08:55 STEP 2095 H=2095:00 +CPU= 18.286
  10:09:13 STEP 2096 H=2096:00 +CPU= 18.429
  10:09:42 STEP 2097 H=2097:00 +CPU= 27.981
  10:10:00 STEP 2098 H=2098:00 +CPU= 18.702
  10:10:18 STEP 2099 H=2099:00 +CPU= 17.726
  10:10:50 STEP 2100 H=2100:00 +CPU= 30.961
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002100
  10:11:08 STEP 2101 H=2101:00 +CPU= 18.147
  10:11:27 STEP 2102 H=2102:00 +CPU= 18.798
  10:11:56 STEP 2103 H=2103:00 +CPU= 28.531
  10:12:14 STEP 2104 H=2104:00 +CPU= 18.500
  10:12:33 STEP 2105 H=2105:00 +CPU= 17.937
  10:13:02 STEP 2106 H=2106:00 +CPU= 28.789
  10:13:20 STEP 2107 H=2107:00 +CPU= 18.318
  10:13:39 STEP 2108 H=2108:00 +CPU= 18.877
  10:14:09 STEP 2109 H=2109:00 +CPU= 29.288
  10:14:28 STEP 2110 H=2110:00 +CPU= 18.915
  10:14:47 STEP 2111 H=2111:00 +CPU= 18.911
  10:15:21 STEP 2112 H=2112:00 +CPU= 33.409
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_87.zip
Uploading the intermediate file: upload_file_87.zip
  10:15:39 STEP 2113 H=2113:00 +CPU= 18.072
  10:15:57 STEP 2114 H=2114:00 +CPU= 17.889
  10:16:25 STEP 2115 H=2115:00 +CPU= 27.765
  10:16:44 STEP 2116 H=2116:00 +CPU= 18.386
  10:17:01 STEP 2117 H=2117:00 +CPU= 17.769
  10:17:30 STEP 2118 H=2118:00 +CPU= 28.113
  10:17:48 STEP 2119 H=2119:00 +CPU= 17.890
  10:18:06 STEP 2120 H=2120:00 +CPU= 18.254
  10:18:36 STEP 2121 H=2121:00 +CPU= 29.126
  10:18:54 STEP 2122 H=2122:00 +CPU= 18.159
  10:19:12 STEP 2123 H=2123:00 +CPU= 18.234
  10:19:44 STEP 2124 H=2124:00 +CPU= 31.415
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002124
  10:20:03 STEP 2125 H=2125:00 +CPU= 18.351
  10:20:21 STEP 2126 H=2126:00 +CPU= 18.493
  10:20:49 STEP 2127 H=2127:00 +CPU= 27.622
  10:21:08 STEP 2128 H=2128:00 +CPU= 18.595
  10:21:26 STEP 2129 H=2129:00 +CPU= 18.356
  10:21:56 STEP 2130 H=2130:00 +CPU= 28.929
  10:22:14 STEP 2131 H=2131:00 +CPU= 18.074
  10:22:32 STEP 2132 H=2132:00 +CPU= 18.051
  10:23:01 STEP 2133 H=2133:00 +CPU= 28.553
  10:23:20 STEP 2134 H=2134:00 +CPU= 18.701
  10:23:38 STEP 2135 H=2135:00 +CPU= 18.268
  10:24:13 STEP 2136 H=2136:00 +CPU= 34.345
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_88.zip
Uploading the intermediate file: upload_file_88.zip
  10:24:31 STEP 2137 H=2137:00 +CPU= 17.936
  10:24:50 STEP 2138 H=2138:00 +CPU= 18.797
  10:25:18 STEP 2139 H=2139:00 +CPU= 27.809
  10:25:36 STEP 2140 H=2140:00 +CPU= 18.307
  10:25:55 STEP 2141 H=2141:00 +CPU= 18.204
  10:26:24 STEP 2142 H=2142:00 +CPU= 29.427
  10:26:43 STEP 2143 H=2143:00 +CPU= 18.358
  10:27:02 STEP 2144 H=2144:00 +CPU= 18.663
  10:27:31 STEP 2145 H=2145:00 +CPU= 28.778
  10:27:50 STEP 2146 H=2146:00 +CPU= 18.694
  10:28:08 STEP 2147 H=2147:00 +CPU= 18.216
  10:28:40 STEP 2148 H=2148:00 +CPU= 31.412
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002148
  10:28:58 STEP 2149 H=2149:00 +CPU= 17.863
  10:29:16 STEP 2150 H=2150:00 +CPU= 18.016
  10:29:45 STEP 2151 H=2151:00 +CPU= 28.963
  10:30:04 STEP 2152 H=2152:00 +CPU= 18.994
  10:30:22 STEP 2153 H=2153:00 +CPU= 17.847
  10:30:52 STEP 2154 H=2154:00 +CPU= 29.496
  10:31:10 STEP 2155 H=2155:00 +CPU= 17.810
  10:31:29 STEP 2156 H=2156:00 +CPU= 18.274
  10:31:57 STEP 2157 H=2157:00 +CPU= 28.147
  10:32:16 STEP 2158 H=2158:00 +CPU= 18.874
  10:32:35 STEP 2159 H=2159:00 +CPU= 18.436
  10:33:08 STEP 2160 H=2160:00 +CPU= 33.154
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_89.zip
Uploading the intermediate file: upload_file_89.zip
  10:33:26 STEP 2161 H=2161:00 +CPU= 17.589
Uploading trickle at timestep: 7776000
  10:33:45 STEP 2162 H=2162:00 +CPU= 18.453
  10:34:13 STEP 2163 H=2163:00 +CPU= 28.021
  10:34:32 STEP 2164 H=2164:00 +CPU= 18.786
  10:34:50 STEP 2165 H=2165:00 +CPU= 18.158
  10:35:19 STEP 2166 H=2166:00 +CPU= 28.252
  10:35:37 STEP 2167 H=2167:00 +CPU= 17.782
  10:35:55 STEP 2168 H=2168:00 +CPU= 18.237
  10:36:24 STEP 2169 H=2169:00 +CPU= 28.113
  10:36:43 STEP 2170 H=2170:00 +CPU= 18.755
  10:37:01 STEP 2171 H=2171:00 +CPU= 18.177
  10:37:32 STEP 2172 H=2172:00 +CPU= 31.210
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002172
  10:37:51 STEP 2173 H=2173:00 +CPU= 18.353
  10:38:10 STEP 2174 H=2174:00 +CPU= 18.820
  10:38:38 STEP 2175 H=2175:00 +CPU= 27.965
  10:38:57 STEP 2176 H=2176:00 +CPU= 18.516
  10:39:16 STEP 2177 H=2177:00 +CPU= 18.354
  10:39:45 STEP 2178 H=2178:00 +CPU= 29.093
  10:40:04 STEP 2179 H=2179:00 +CPU= 18.406
  10:40:22 STEP 2180 H=2180:00 +CPU= 18.711
  10:40:51 STEP 2181 H=2181:00 +CPU= 28.436
  10:41:10 STEP 2182 H=2182:00 +CPU= 18.393
  10:41:29 STEP 2183 H=2183:00 +CPU= 18.753
  10:42:03 STEP 2184 H=2184:00 +CPU= 33.797
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_90.zip
Uploading the intermediate file: upload_file_90.zip
  10:42:21 STEP 2185 H=2185:00 +CPU= 17.449
  10:42:39 STEP 2186 H=2186:00 +CPU= 18.178
  10:43:07 STEP 2187 H=2187:00 +CPU= 27.716
  10:43:26 STEP 2188 H=2188:00 +CPU= 18.824
  10:43:44 STEP 2189 H=2189:00 +CPU= 17.885
  10:44:13 STEP 2190 H=2190:00 +CPU= 28.675
  10:44:32 STEP 2191 H=2191:00 +CPU= 18.587
  10:44:50 STEP 2192 H=2192:00 +CPU= 18.569
  10:45:19 STEP 2193 H=2193:00 +CPU= 28.446
  10:45:38 STEP 2194 H=2194:00 +CPU= 18.489
  10:45:56 STEP 2195 H=2195:00 +CPU= 17.851
  10:46:28 STEP 2196 H=2196:00 +CPU= 31.615
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002196
  10:46:46 STEP 2197 H=2197:00 +CPU= 18.010
  10:47:04 STEP 2198 H=2198:00 +CPU= 17.725
  10:47:32 STEP 2199 H=2199:00 +CPU= 27.824
  10:47:51 STEP 2200 H=2200:00 +CPU= 18.874
  10:48:08 STEP 2201 H=2201:00 +CPU= 17.280
  10:48:38 STEP 2202 H=2202:00 +CPU= 29.051
  10:48:55 STEP 2203 H=2203:00 +CPU= 17.622
  10:49:14 STEP 2204 H=2204:00 +CPU= 18.849
  10:49:43 STEP 2205 H=2205:00 +CPU= 28.267
  10:50:02 STEP 2206 H=2206:00 +CPU= 18.900
  10:50:21 STEP 2207 H=2207:00 +CPU= 18.403
  10:50:55 STEP 2208 H=2208:00 +CPU= 34.364
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_91.zip
Uploading the intermediate file: upload_file_91.zip
  10:51:13 STEP 2209 H=2209:00 +CPU= 17.640
  10:51:31 STEP 2210 H=2210:00 +CPU= 17.735
  10:51:59 STEP 2211 H=2211:00 +CPU= 28.247
  10:52:18 STEP 2212 H=2212:00 +CPU= 18.261
  10:52:35 STEP 2213 H=2213:00 +CPU= 17.290
  10:53:05 STEP 2214 H=2214:00 +CPU= 28.939
  10:53:23 STEP 2215 H=2215:00 +CPU= 18.162
  10:53:41 STEP 2216 H=2216:00 +CPU= 18.171
  10:54:10 STEP 2217 H=2217:00 +CPU= 28.193
  10:54:29 STEP 2218 H=2218:00 +CPU= 18.701
  10:54:47 STEP 2219 H=2219:00 +CPU= 18.255
  10:55:18 STEP 2220 H=2220:00 +CPU= 31.048
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002220
  10:55:36 STEP 2221 H=2221:00 +CPU= 18.013
  10:55:55 STEP 2222 H=2222:00 +CPU= 18.629
  10:56:24 STEP 2223 H=2223:00 +CPU= 28.286
  10:56:42 STEP 2224 H=2224:00 +CPU= 18.314
  10:57:00 STEP 2225 H=2225:00 +CPU= 18.052
  10:57:30 STEP 2226 H=2226:00 +CPU= 29.337
  10:57:48 STEP 2227 H=2227:00 +CPU= 17.975
  10:58:06 STEP 2228 H=2228:00 +CPU= 17.324
  10:58:34 STEP 2229 H=2229:00 +CPU= 27.777
  10:58:52 STEP 2230 H=2230:00 +CPU= 18.466
  10:59:11 STEP 2231 H=2231:00 +CPU= 18.777
  10:59:45 STEP 2232 H=2232:00 +CPU= 33.801
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_92.zip
Uploading the intermediate file: upload_file_92.zip
  11:00:03 STEP 2233 H=2233:00 +CPU= 17.670
  11:00:22 STEP 2234 H=2234:00 +CPU= 18.161
  11:00:49 STEP 2235 H=2235:00 +CPU= 27.364
  11:01:07 STEP 2236 H=2236:00 +CPU= 18.008
  11:01:25 STEP 2237 H=2237:00 +CPU= 17.619
  11:01:54 STEP 2238 H=2238:00 +CPU= 28.726
  11:02:13 STEP 2239 H=2239:00 +CPU= 18.198
  11:02:31 STEP 2240 H=2240:00 +CPU= 18.544
  11:03:00 STEP 2241 H=2241:00 +CPU= 28.175
  11:03:18 STEP 2242 H=2242:00 +CPU= 18.329
  11:03:37 STEP 2243 H=2243:00 +CPU= 18.825
  11:04:09 STEP 2244 H=2244:00 +CPU= 31.364
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002244
  11:04:27 STEP 2245 H=2245:00 +CPU= 18.065
  11:04:46 STEP 2246 H=2246:00 +CPU= 18.231
  11:05:15 STEP 2247 H=2247:00 +CPU= 28.584
  11:05:34 STEP 2248 H=2248:00 +CPU= 18.636
  11:05:52 STEP 2249 H=2249:00 +CPU= 17.995
  11:06:21 STEP 2250 H=2250:00 +CPU= 28.444
  11:06:40 STEP 2251 H=2251:00 +CPU= 18.749
  11:06:58 STEP 2252 H=2252:00 +CPU= 18.554
  11:07:27 STEP 2253 H=2253:00 +CPU= 28.243
  11:07:46 STEP 2254 H=2254:00 +CPU= 18.906
  11:08:05 STEP 2255 H=2255:00 +CPU= 18.526
  11:08:39 STEP 2256 H=2256:00 +CPU= 33.828
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_93.zip
Uploading the intermediate file: upload_file_93.zip
  11:08:56 STEP 2257 H=2257:00 +CPU= 17.292
  11:09:15 STEP 2258 H=2258:00 +CPU= 18.368
  11:09:43 STEP 2259 H=2259:00 +CPU= 27.445
  11:10:01 STEP 2260 H=2260:00 +CPU= 18.204
  11:10:19 STEP 2261 H=2261:00 +CPU= 17.686
  11:10:48 STEP 2262 H=2262:00 +CPU= 28.823
  11:11:07 STEP 2263 H=2263:00 +CPU= 18.685
  11:11:26 STEP 2264 H=2264:00 +CPU= 18.826
  11:11:55 STEP 2265 H=2265:00 +CPU= 28.729
  11:12:14 STEP 2266 H=2266:00 +CPU= 18.663
  11:12:32 STEP 2267 H=2267:00 +CPU= 17.995
  11:13:04 STEP 2268 H=2268:00 +CPU= 31.380
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002268
  11:13:22 STEP 2269 H=2269:00 +CPU= 18.548
  11:13:41 STEP 2270 H=2270:00 +CPU= 18.082
  11:14:10 STEP 2271 H=2271:00 +CPU= 28.632
  11:14:28 STEP 2272 H=2272:00 +CPU= 18.597
  11:14:47 STEP 2273 H=2273:00 +CPU= 18.420
  11:15:16 STEP 2274 H=2274:00 +CPU= 28.617
  11:15:35 STEP 2275 H=2275:00 +CPU= 18.575
  11:15:54 STEP 2276 H=2276:00 +CPU= 18.757
  11:16:23 STEP 2277 H=2277:00 +CPU= 28.674
  11:16:42 STEP 2278 H=2278:00 +CPU= 18.935
  11:17:01 STEP 2279 H=2279:00 +CPU= 19.278
  11:17:35 STEP 2280 H=2280:00 +CPU= 33.724
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_94.zip
Uploading the intermediate file: upload_file_94.zip
  11:17:54 STEP 2281 H=2281:00 +CPU= 18.282
  11:18:12 STEP 2282 H=2282:00 +CPU= 18.156
  11:18:40 STEP 2283 H=2283:00 +CPU= 27.940
  11:18:59 STEP 2284 H=2284:00 +CPU= 18.750
  11:19:18 STEP 2285 H=2285:00 +CPU= 18.087
  11:19:46 STEP 2286 H=2286:00 +CPU= 28.343
  11:20:05 STEP 2287 H=2287:00 +CPU= 18.423
  11:20:23 STEP 2288 H=2288:00 +CPU= 18.514
  11:20:52 STEP 2289 H=2289:00 +CPU= 28.425
  11:21:11 STEP 2290 H=2290:00 +CPU= 18.848
  11:21:30 STEP 2291 H=2291:00 +CPU= 18.376
  11:22:01 STEP 2292 H=2292:00 +CPU= 31.271
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002292
  11:22:20 STEP 2293 H=2293:00 +CPU= 18.693
  11:22:39 STEP 2294 H=2294:00 +CPU= 18.687
  11:23:09 STEP 2295 H=2295:00 +CPU= 29.364
  11:23:28 STEP 2296 H=2296:00 +CPU= 19.065
  11:23:47 STEP 2297 H=2297:00 +CPU= 18.505
  11:24:16 STEP 2298 H=2298:00 +CPU= 29.006
  11:24:34 STEP 2299 H=2299:00 +CPU= 17.525
  11:24:52 STEP 2300 H=2300:00 +CPU= 18.629
  11:25:21 STEP 2301 H=2301:00 +CPU= 28.469
  11:25:41 STEP 2302 H=2302:00 +CPU= 19.563
  11:25:59 STEP 2303 H=2303:00 +CPU= 18.132
  11:26:33 STEP 2304 H=2304:00 +CPU= 33.721
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_95.zip
Uploading the intermediate file: upload_file_95.zip
  11:26:51 STEP 2305 H=2305:00 +CPU= 17.356
  11:27:09 STEP 2306 H=2306:00 +CPU= 18.210
  11:27:38 STEP 2307 H=2307:00 +CPU= 28.387
  11:27:57 STEP 2308 H=2308:00 +CPU= 18.519
  11:28:14 STEP 2309 H=2309:00 +CPU= 17.489
  11:28:42 STEP 2310 H=2310:00 +CPU= 27.835
  11:29:01 STEP 2311 H=2311:00 +CPU= 18.854
  11:29:21 STEP 2312 H=2312:00 +CPU= 19.495
  11:29:50 STEP 2313 H=2313:00 +CPU= 29.123
  11:30:10 STEP 2314 H=2314:00 +CPU= 19.245
  11:30:28 STEP 2315 H=2315:00 +CPU= 18.083
  11:31:00 STEP 2316 H=2316:00 +CPU= 31.867
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002316
  11:31:19 STEP 2317 H=2317:00 +CPU= 18.523
  11:31:39 STEP 2318 H=2318:00 +CPU= 19.436
  11:32:08 STEP 2319 H=2319:00 +CPU= 28.631
  11:32:28 STEP 2320 H=2320:00 +CPU= 19.838
  11:32:47 STEP 2321 H=2321:00 +CPU= 19.123
  11:33:17 STEP 2322 H=2322:00 +CPU= 29.932
  11:33:36 STEP 2323 H=2323:00 +CPU= 18.251
  11:33:55 STEP 2324 H=2324:00 +CPU= 18.915
  11:34:25 STEP 2325 H=2325:00 +CPU= 30.009
  11:34:44 STEP 2326 H=2326:00 +CPU= 19.008
  11:35:04 STEP 2327 H=2327:00 +CPU= 19.238
  11:35:40 STEP 2328 H=2328:00 +CPU= 35.870
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_96.zip
Uploading the intermediate file: upload_file_96.zip
  11:35:59 STEP 2329 H=2329:00 +CPU= 18.746
  11:36:18 STEP 2330 H=2330:00 +CPU= 18.763
  11:36:49 STEP 2331 H=2331:00 +CPU= 30.463
  11:37:08 STEP 2332 H=2332:00 +CPU= 18.882
  11:37:26 STEP 2333 H=2333:00 +CPU= 18.652
  11:37:58 STEP 2334 H=2334:00 +CPU= 30.792
  11:38:17 STEP 2335 H=2335:00 +CPU= 18.886
  11:38:36 STEP 2336 H=2336:00 +CPU= 19.163
  11:39:06 STEP 2337 H=2337:00 +CPU= 29.805
  11:39:25 STEP 2338 H=2338:00 +CPU= 18.441
  11:39:44 STEP 2339 H=2339:00 +CPU= 18.844
  11:40:17 STEP 2340 H=2340:00 +CPU= 32.660
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002340
  11:40:36 STEP 2341 H=2341:00 +CPU= 18.625
  11:40:55 STEP 2342 H=2342:00 +CPU= 19.564
  11:41:25 STEP 2343 H=2343:00 +CPU= 29.262
  11:41:45 STEP 2344 H=2344:00 +CPU= 20.229
  11:42:05 STEP 2345 H=2345:00 +CPU= 19.386
  11:42:35 STEP 2346 H=2346:00 +CPU= 29.863
  11:42:54 STEP 2347 H=2347:00 +CPU= 18.896
  11:43:15 STEP 2348 H=2348:00 +CPU= 20.201
  11:43:44 STEP 2349 H=2349:00 +CPU= 29.105
  11:44:03 STEP 2350 H=2350:00 +CPU= 19.202
  11:44:23 STEP 2351 H=2351:00 +CPU= 19.104
  11:45:00 STEP 2352 H=2352:00 +CPU= 36.582
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_97.zip
Uploading the intermediate file: upload_file_97.zip
  11:45:18 STEP 2353 H=2353:00 +CPU= 18.495
  11:45:38 STEP 2354 H=2354:00 +CPU= 19.265
  11:46:07 STEP 2355 H=2355:00 +CPU= 28.967
  11:46:27 STEP 2356 H=2356:00 +CPU= 19.627
  11:46:45 STEP 2357 H=2357:00 +CPU= 18.317
  11:47:16 STEP 2358 H=2358:00 +CPU= 30.779
  11:47:36 STEP 2359 H=2359:00 +CPU= 19.125
  11:47:55 STEP 2360 H=2360:00 +CPU= 18.775
  11:48:24 STEP 2361 H=2361:00 +CPU= 29.184
  11:48:43 STEP 2362 H=2362:00 +CPU= 18.900
  11:49:03 STEP 2363 H=2363:00 +CPU= 19.107
  11:49:36 STEP 2364 H=2364:00 +CPU= 32.702
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002364
  11:49:55 STEP 2365 H=2365:00 +CPU= 18.887
  11:50:14 STEP 2366 H=2366:00 +CPU= 18.880
  11:50:44 STEP 2367 H=2367:00 +CPU= 29.547
  11:51:03 STEP 2368 H=2368:00 +CPU= 19.259
  11:51:23 STEP 2369 H=2369:00 +CPU= 19.497
  11:51:53 STEP 2370 H=2370:00 +CPU= 30.173
  11:52:12 STEP 2371 H=2371:00 +CPU= 18.191
  11:52:31 STEP 2372 H=2372:00 +CPU= 19.307
  11:53:01 STEP 2373 H=2373:00 +CPU= 29.231
  11:53:21 STEP 2374 H=2374:00 +CPU= 20.149
  11:53:40 STEP 2375 H=2375:00 +CPU= 19.205
  11:54:16 STEP 2376 H=2376:00 +CPU= 35.311
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_98.zip
Uploading the intermediate file: upload_file_98.zip
  11:54:34 STEP 2377 H=2377:00 +CPU= 18.066
  11:54:53 STEP 2378 H=2378:00 +CPU= 18.610
  11:55:24 STEP 2379 H=2379:00 +CPU= 30.376
  11:55:44 STEP 2380 H=2380:00 +CPU= 19.993
  11:56:03 STEP 2381 H=2381:00 +CPU= 18.698
  11:56:33 STEP 2382 H=2382:00 +CPU= 30.094
  11:56:52 STEP 2383 H=2383:00 +CPU= 18.322
  11:57:11 STEP 2384 H=2384:00 +CPU= 19.353
  11:57:41 STEP 2385 H=2385:00 +CPU= 29.345
  11:58:01 STEP 2386 H=2386:00 +CPU= 19.444
  11:58:20 STEP 2387 H=2387:00 +CPU= 19.343
  11:58:53 STEP 2388 H=2388:00 +CPU= 32.660
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002388
  11:59:12 STEP 2389 H=2389:00 +CPU= 18.257
  11:59:30 STEP 2390 H=2390:00 +CPU= 18.687
  12:00:01 STEP 2391 H=2391:00 +CPU= 29.964
  12:00:21 STEP 2392 H=2392:00 +CPU= 19.702
  12:00:40 STEP 2393 H=2393:00 +CPU= 18.785
  12:01:10 STEP 2394 H=2394:00 +CPU= 29.572
  12:01:29 STEP 2395 H=2395:00 +CPU= 18.898
  12:01:49 STEP 2396 H=2396:00 +CPU= 19.694
  12:02:18 STEP 2397 H=2397:00 +CPU= 29.216
  12:02:37 STEP 2398 H=2398:00 +CPU= 19.137
  12:02:57 STEP 2399 H=2399:00 +CPU= 19.611
  12:03:33 STEP 2400 H=2400:00 +CPU= 34.974
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_99.zip
Uploading the intermediate file: upload_file_99.zip
  12:03:51 STEP 2401 H=2401:00 +CPU= 18.465
Uploading trickle at timestep: 8640000
  12:04:10 STEP 2402 H=2402:00 +CPU= 18.583
  12:04:39 STEP 2403 H=2403:00 +CPU= 29.089
  12:04:58 STEP 2404 H=2404:00 +CPU= 18.879
  12:05:18 STEP 2405 H=2405:00 +CPU= 19.414
  12:05:48 STEP 2406 H=2406:00 +CPU= 30.039
  12:06:07 STEP 2407 H=2407:00 +CPU= 18.197
  12:06:26 STEP 2408 H=2408:00 +CPU= 19.192
  12:06:56 STEP 2409 H=2409:00 +CPU= 29.375
  12:07:16 STEP 2410 H=2410:00 +CPU= 19.907
  12:07:35 STEP 2411 H=2411:00 +CPU= 19.407
  12:08:08 STEP 2412 H=2412:00 +CPU= 32.160
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002412
  12:08:26 STEP 2413 H=2413:00 +CPU= 18.032
  12:08:45 STEP 2414 H=2414:00 +CPU= 19.040
  12:09:14 STEP 2415 H=2415:00 +CPU= 28.442
  12:09:34 STEP 2416 H=2416:00 +CPU= 19.309
  12:09:52 STEP 2417 H=2417:00 +CPU= 18.128
  12:10:21 STEP 2418 H=2418:00 +CPU= 29.233
  12:10:40 STEP 2419 H=2419:00 +CPU= 18.528
  12:11:00 STEP 2420 H=2420:00 +CPU= 19.777
  12:11:30 STEP 2421 H=2421:00 +CPU= 29.829
  12:11:51 STEP 2422 H=2422:00 +CPU= 20.163
  12:12:11 STEP 2423 H=2423:00 +CPU= 20.003
  12:12:45 STEP 2424 H=2424:00 +CPU= 33.990
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_12167474/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_ps_0830_1981050100_123_950_12167474_0_r978337860_100.zip
Uploading the intermediate file: upload_file_100.zip
  12:13:03 STEP 2425 H=2425:00 +CPU= 18.004
  12:13:22 STEP 2426 H=2426:00 +CPU= 18.863
  12:13:53 STEP 2427 H=2427:00 +CPU= 29.859
  12:14:12 STEP 2428 H=2428:00 +CPU= 18.907
  12:14:30 STEP 2429 H=2429:00 +CPU= 18.000
  12:15:00 STEP 2430 H=2430:00 +CPU= 29.417
  12:15:18 STEP 2431 H=2431:00 +CPU= 18.134
  12:15:37 STEP 2432 H=2432:00 +CPU= 19.339
  12:16:07 STEP 2433 H=2433:00 +CPU= 29.720
  12:16:27 STEP 2434 H=2434:00 +CPU= 19.589
  12:16:46 STEP 2435 H=2435:00 +CPU= 18.352
  12:17:18 STEP 2436 H=2436:00 +CPU= 31.996
Moving to projects directory: /var/lib/boinc/slots/11/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/11/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc/slots/11/ICMUAhq0f+002436
  12:17:36 STEP 2437 H=2437:00 +CPU= 18.153
  12:17:55 STEP 2438 H=2438:00 +CPU= 18.764
  12:18:25 STEP 2439 H=2439:00 +CPU= 28.922
  12:18:44 STEP 2440 H=2440:00 +CPU= 19.571
  12:19:03 STEP 2441 H=2441:00 +CPU= 18.250
  12:19:33 STEP 2442 H=2442:00 +CPU= 29.397
  12:19:51 STEP 2443 H=2443:00 +CPU= 18.603
  12:20:11 STEP 2444 H=2444:00 +CPU= 19.364
  12:20:40 STEP 2445 H=2445:00 +CPU= 28.864
  12:20:59 STEP 2446 H=2446:00 +CPU= 19.089
  12:21:19 STEP 2447 H=2447:00 +CPU= 19.005
ECCODES ERROR   :  IO ERROR: No space left on device (No space left on device)
 GRIB_WRITE_BYTES_INT        50003  FAILED         -11
 GRIB_API ERROR MSG: Input output problem
MPL_ABORT: CALLED FROM PROCESSOR      1 THRD     1
 MPL_ABORT: THRD           1   GRIB_WRITE_BYTES_REAL8 FAILED
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.994] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3843] 2594 MB (maxheap), 4415 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         IOPACK 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          WRRESF 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           IOSTREAM_MIX:IO_PUT 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            IOSTREAM_MIX:WRITE_RECORD 
[EC_DRHOOK:sekundaer:1:1:615095:615095] [20221221:122123:1671625283:56317.995] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             GRIB_API:IGRIB_WRITE_BYTES_REAL8 
 SDL_TRACEBACK: Calling INTEL_TRBK, THRD =            1
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc/slots/11/oifs_43r3_model.exe' : sigcontextptr=0x7ffea9238a20
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 19) with addr2line-cmd
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc/slots/11/oifs_43r3_model.exe' 0x839273 0x839879 0x8c2957 0x8c3229 0xa050aa 0x9d0105 0x9c2a6a 0x5f34e2 0x5db0e2 0x579b82 0x55d8d6 0x5471c9 0x546412 0x545e28 0x40708d 0x40220f 0x4021a2 0x1dca390 0x40206e
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x839273] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x839879] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x8c2957] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x8c3229] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0xa050aa] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x9d0105] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x9c2a6a] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x5f34e2] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0x5db0e2] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0x579b82] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0x55d8d6] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x5471c9] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x546412] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x545e28] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x40708d] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x40220f] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4021a2] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x1dca390] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x40206e] : ??() at ??:0
  [LinuxTraceBack@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 SDL_TRACEBACK: Done INTEL_TRBK, THRD =            1
abort: 
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 15:45:52 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 8,640,000 54,722 0.0063
09 May 2023 15:45:52 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 8,640,000 54,722 0.0063
09 May 2023 15:45:47 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 7,776,000 49,359 0.0063
09 May 2023 15:45:47 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 7,776,000 49,359 0.0063
09 May 2023 15:45:39 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 6,912,000 44,082 0.0064
09 May 2023 15:45:34 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 6,048,000 38,923 0.0064
09 May 2023 15:45:34 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 6,048,000 38,923 0.0064
09 May 2023 15:45:35 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 5,184,000 33,704 0.0065
09 May 2023 15:45:23 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 4,320,000 27,564 0.0064
09 May 2023 15:45:17 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 3,456,000 21,164 0.0061
09 May 2023 15:45:15 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 2,592,000 14,987 0.0058
09 May 2023 15:45:10 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 1,728,000 9,452 0.0055
09 May 2023 15:45:09 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 864,000 4,450 0.0052
09 May 2023 15:45:08 1532374 22251694 oifs_43r3_ps_0830_1981050100_123_950_12167474_0 864,000 4,450 0.0052


©2024 climateprediction.net