climateprediction.net home page
Task 22303570

Task 22303570

Name oifs_43r3_ps_0184_2001050100_123_970_12186828_1
Workunit 12186828
Created 25 Jan 2023, 16:20:43 UTC
Sent 25 Jan 2023, 16:30:09 UTC
Report deadline 26 Mar 2023, 16:30:09 UTC
Received 6 Feb 2023, 17:38:09 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1537829
Run time 11 hours 49 min 12 sec
CPU time 4 hours 31 min 32 sec
Validate state Invalid
Credit 5,201.28
Device peak FLOPS 4.14 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.32 MB
Peak swap size 4,978.07 MB
Peak disk usage 7,179.69 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230205:095656:1675587416:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:57:46 STEP  792 H= 792:00 +CPU= 43.010
  09:58:10 STEP  793 H= 793:00 +CPU= 23.396
  09:58:35 STEP  794 H= 794:00 +CPU= 24.168
  09:59:14 STEP  795 H= 795:00 +CPU= 38.614
  09:59:39 STEP  796 H= 796:00 +CPU= 24.126
  10:00:03 STEP  797 H= 797:00 +CPU= 23.445
  10:00:43 STEP  798 H= 798:00 +CPU= 39.456
  10:01:07 STEP  799 H= 799:00 +CPU= 23.445
  10:01:31 STEP  800 H= 800:00 +CPU= 24.228
  10:02:11 STEP  801 H= 801:00 +CPU= 38.743
  10:02:36 STEP  802 H= 802:00 +CPU= 24.232
  10:02:59 STEP  803 H= 803:00 +CPU= 23.453
  10:03:43 STEP  804 H= 804:00 +CPU= 42.818
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000804
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000804
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000804
  10:04:07 STEP  805 H= 805:00 +CPU= 23.433
  10:04:32 STEP  806 H= 806:00 +CPU= 24.245
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:05:33 STEP  807 H= 807:00 +CPU= 38.781
  10:05:58 STEP  808 H= 808:00 +CPU= 24.280
  10:06:21 STEP  809 H= 809:00 +CPU= 23.443
  10:07:02 STEP  810 H= 810:00 +CPU= 39.566
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:08:07 STEP  811 H= 811:00 +CPU= 23.487
  10:08:32 STEP  812 H= 812:00 +CPU= 24.199
  10:09:11 STEP  813 H= 813:00 +CPU= 38.735
  10:09:37 STEP  814 H= 814:00 +CPU= 24.271
  10:10:02 STEP  815 H= 815:00 +CPU= 24.248
  10:10:48 STEP  816 H= 816:00 +CPU= 44.644
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000816
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000816
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000816
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000792
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000792
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000792
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000804
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000804
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000804
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_1.zip
Uploading the intermediate file: upload_file_1.zip
  10:11:12 STEP  817 H= 817:00 +CPU= 23.511
  10:11:36 STEP  818 H= 818:00 +CPU= 24.207
  10:12:16 STEP  819 H= 819:00 +CPU= 38.850
  10:12:41 STEP  820 H= 820:00 +CPU= 24.280
  10:13:05 STEP  821 H= 821:00 +CPU= 23.583
  10:13:46 STEP  822 H= 822:00 +CPU= 39.681
  10:14:09 STEP  823 H= 823:00 +CPU= 23.555
  10:14:34 STEP  824 H= 824:00 +CPU= 24.296
  10:15:14 STEP  825 H= 825:00 +CPU= 38.939
  10:15:39 STEP  826 H= 826:00 +CPU= 24.338
  10:16:03 STEP  827 H= 827:00 +CPU= 23.545
  10:16:47 STEP  828 H= 828:00 +CPU= 42.958
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000828
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000828
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000828
  10:17:11 STEP  829 H= 829:00 +CPU= 23.636
  10:17:36 STEP  830 H= 830:00 +CPU= 24.372
  10:18:16 STEP  831 H= 831:00 +CPU= 38.931
  10:18:41 STEP  832 H= 832:00 +CPU= 24.377
  10:19:05 STEP  833 H= 833:00 +CPU= 23.592
  10:19:46 STEP  834 H= 834:00 +CPU= 39.718
  10:20:11 STEP  835 H= 835:00 +CPU= 23.686
  10:20:37 STEP  836 H= 836:00 +CPU= 24.467
  10:21:16 STEP  837 H= 837:00 +CPU= 38.877
  10:21:42 STEP  838 H= 838:00 +CPU= 24.469
  10:22:09 STEP  839 H= 839:00 +CPU= 24.445
  10:22:56 STEP  840 H= 840:00 +CPU= 45.008
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000840
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000840
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000840
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000816
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000816
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000816
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000828
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000828
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000828
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_2.zip
Uploading the intermediate file: upload_file_2.zip
  10:23:22 STEP  841 H= 841:00 +CPU= 23.718
  10:23:47 STEP  842 H= 842:00 +CPU= 24.423
  10:24:27 STEP  843 H= 843:00 +CPU= 38.896
  10:24:52 STEP  844 H= 844:00 +CPU= 24.439
  10:25:18 STEP  845 H= 845:00 +CPU= 23.721
  10:25:58 STEP  846 H= 846:00 +CPU= 39.784
  10:26:23 STEP  847 H= 847:00 +CPU= 23.628
  10:26:47 STEP  848 H= 848:00 +CPU= 24.297
  10:27:28 STEP  849 H= 849:00 +CPU= 39.015
  10:27:53 STEP  850 H= 850:00 +CPU= 24.348
  10:28:17 STEP  851 H= 851:00 +CPU= 23.614
  10:29:06 STEP  852 H= 852:00 +CPU= 43.556
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000852
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000852
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000852
  10:29:32 STEP  853 H= 853:00 +CPU= 23.787
  10:29:58 STEP  854 H= 854:00 +CPU= 24.456
  10:30:39 STEP  855 H= 855:00 +CPU= 39.120
  10:31:05 STEP  856 H= 856:00 +CPU= 24.455
  10:31:30 STEP  857 H= 857:00 +CPU= 23.796
  10:32:12 STEP  858 H= 858:00 +CPU= 39.932
  10:32:38 STEP  859 H= 859:00 +CPU= 23.770
  10:33:02 STEP  860 H= 860:00 +CPU= 24.246
  10:33:43 STEP  861 H= 861:00 +CPU= 38.922
  10:34:08 STEP  862 H= 862:00 +CPU= 24.298
  10:34:32 STEP  863 H= 863:00 +CPU= 24.248
  10:35:19 STEP  864 H= 864:00 +CPU= 44.995
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000864
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000864
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000864
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000840
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000840
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000840
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000852
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000852
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000852
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_3.zip
Uploading the intermediate file: upload_file_3.zip
  10:35:44 STEP  865 H= 865:00 +CPU= 23.617
  10:36:09 STEP  866 H= 866:00 +CPU= 24.340
  10:36:50 STEP  867 H= 867:00 +CPU= 39.014
  10:37:15 STEP  868 H= 868:00 +CPU= 24.328
  10:37:39 STEP  869 H= 869:00 +CPU= 23.545
  10:38:20 STEP  870 H= 870:00 +CPU= 39.707
  10:38:44 STEP  871 H= 871:00 +CPU= 23.632
  10:39:09 STEP  872 H= 872:00 +CPU= 24.364
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  10:40:10 STEP  873 H= 873:00 +CPU= 38.997
  10:40:34 STEP  874 H= 874:00 +CPU= 24.506
  10:40:58 STEP  875 H= 875:00 +CPU= 23.672
  10:41:42 STEP  876 H= 876:00 +CPU= 43.222
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000876
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000876
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000876
  10:42:06 STEP  877 H= 877:00 +CPU= 23.688
  10:42:31 STEP  878 H= 878:00 +CPU= 24.450
  10:43:10 STEP  879 H= 879:00 +CPU= 39.105
  10:43:35 STEP  880 H= 880:00 +CPU= 24.469
  10:43:59 STEP  881 H= 881:00 +CPU= 23.709
  10:44:39 STEP  882 H= 882:00 +CPU= 39.819
  10:45:03 STEP  883 H= 883:00 +CPU= 23.689
  10:45:27 STEP  884 H= 884:00 +CPU= 24.322
  10:46:07 STEP  885 H= 885:00 +CPU= 39.078
  10:46:32 STEP  886 H= 886:00 +CPU= 24.424
  10:46:56 STEP  887 H= 887:00 +CPU= 24.291
  10:47:41 STEP  888 H= 888:00 +CPU= 44.982
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000888
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000888
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000888
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000864
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000864
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000864
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000876
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000876
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000876
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_4.zip
Uploading the intermediate file: upload_file_4.zip
  10:48:05 STEP  889 H= 889:00 +CPU= 23.636
  10:48:30 STEP  890 H= 890:00 +CPU= 24.373
  10:49:10 STEP  891 H= 891:00 +CPU= 38.983
  10:49:34 STEP  892 H= 892:00 +CPU= 24.344
  10:49:58 STEP  893 H= 893:00 +CPU= 23.591
  10:50:38 STEP  894 H= 894:00 +CPU= 39.920
  10:51:02 STEP  895 H= 895:00 +CPU= 23.638
  10:51:26 STEP  896 H= 896:00 +CPU= 24.442
  10:52:05 STEP  897 H= 897:00 +CPU= 39.055
  10:52:30 STEP  898 H= 898:00 +CPU= 24.465
  10:52:54 STEP  899 H= 899:00 +CPU= 23.719
  10:53:37 STEP  900 H= 900:00 +CPU= 43.310
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000900
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000900
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000900
  10:54:01 STEP  901 H= 901:00 +CPU= 23.721
  10:54:26 STEP  902 H= 902:00 +CPU= 24.492
  10:55:05 STEP  903 H= 903:00 +CPU= 39.176
  10:55:29 STEP  904 H= 904:00 +CPU= 24.387
  10:55:53 STEP  905 H= 905:00 +CPU= 23.684
  10:56:33 STEP  906 H= 906:00 +CPU= 39.912
  10:56:57 STEP  907 H= 907:00 +CPU= 23.563
  10:57:21 STEP  908 H= 908:00 +CPU= 24.396
  10:58:00 STEP  909 H= 909:00 +CPU= 39.099
  10:58:25 STEP  910 H= 910:00 +CPU= 24.405
  10:58:49 STEP  911 H= 911:00 +CPU= 24.333
  10:59:35 STEP  912 H= 912:00 +CPU= 44.949
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000912
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000912
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000912
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000888
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000888
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000888
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000900
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000900
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000900
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_5.zip
Uploading the intermediate file: upload_file_5.zip
  10:59:59 STEP  913 H= 913:00 +CPU= 23.604
  11:00:23 STEP  914 H= 914:00 +CPU= 24.342
  11:01:02 STEP  915 H= 915:00 +CPU= 38.930
  11:01:27 STEP  916 H= 916:00 +CPU= 24.377
  11:01:51 STEP  917 H= 917:00 +CPU= 23.714
  11:02:31 STEP  918 H= 918:00 +CPU= 39.931
  11:02:55 STEP  919 H= 919:00 +CPU= 23.719
  11:03:19 STEP  920 H= 920:00 +CPU= 24.499
  11:03:59 STEP  921 H= 921:00 +CPU= 39.044
  11:04:23 STEP  922 H= 922:00 +CPU= 24.360
  11:04:47 STEP  923 H= 923:00 +CPU= 23.638
  11:05:31 STEP  924 H= 924:00 +CPU= 43.304
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000924
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000924
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000924
  11:05:57 STEP  925 H= 925:00 +CPU= 24.054
  11:06:23 STEP  926 H= 926:00 +CPU= 24.617
  11:07:03 STEP  927 H= 927:00 +CPU= 39.159
  11:07:28 STEP  928 H= 928:00 +CPU= 24.561
  11:07:52 STEP  929 H= 929:00 +CPU= 23.713
  11:08:33 STEP  930 H= 930:00 +CPU= 39.922
  11:08:57 STEP  931 H= 931:00 +CPU= 23.798
  11:09:23 STEP  932 H= 932:00 +CPU= 24.589
  11:10:03 STEP  933 H= 933:00 +CPU= 39.258
  11:10:28 STEP  934 H= 934:00 +CPU= 24.515
  11:10:53 STEP  935 H= 935:00 +CPU= 24.384
  11:11:39 STEP  936 H= 936:00 +CPU= 45.049
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000936
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000936
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000936
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000912
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000912
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000912
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000924
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000924
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000924
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_6.zip
Uploading the intermediate file: upload_file_6.zip
  11:12:03 STEP  937 H= 937:00 +CPU= 23.677
  11:12:28 STEP  938 H= 938:00 +CPU= 24.475
  11:13:07 STEP  939 H= 939:00 +CPU= 39.068
  11:13:32 STEP  940 H= 940:00 +CPU= 24.416
  11:13:56 STEP  941 H= 941:00 +CPU= 23.712
  11:14:36 STEP  942 H= 942:00 +CPU= 39.993
  11:15:01 STEP  943 H= 943:00 +CPU= 23.766
  11:15:25 STEP  944 H= 944:00 +CPU= 24.554
  11:16:05 STEP  945 H= 945:00 +CPU= 39.174
  11:16:30 STEP  946 H= 946:00 +CPU= 24.443
  11:16:53 STEP  947 H= 947:00 +CPU= 23.736
  11:17:38 STEP  948 H= 948:00 +CPU= 43.393
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000948
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000948
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000948
  11:18:02 STEP  949 H= 949:00 +CPU= 23.782
  11:18:26 STEP  950 H= 950:00 +CPU= 24.487
  11:19:06 STEP  951 H= 951:00 +CPU= 39.108
  11:19:30 STEP  952 H= 952:00 +CPU= 24.497
  11:19:54 STEP  953 H= 953:00 +CPU= 23.746
  11:20:35 STEP  954 H= 954:00 +CPU= 40.004
  11:20:59 STEP  955 H= 955:00 +CPU= 23.661
  11:21:23 STEP  956 H= 956:00 +CPU= 24.446
  11:22:03 STEP  957 H= 957:00 +CPU= 39.082
  11:22:27 STEP  958 H= 958:00 +CPU= 24.443
  11:22:52 STEP  959 H= 959:00 +CPU= 24.353
  11:23:38 STEP  960 H= 960:00 +CPU= 45.054
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000960
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000960
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000960
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000936
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000936
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000936
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000948
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000948
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000948
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_7.zip
Uploading the intermediate file: upload_file_7.zip
  11:24:23 STEP  961 H= 961:00 +CPU= 23.760
  11:24:47 STEP  962 H= 962:00 +CPU= 24.461
  11:25:27 STEP  963 H= 963:00 +CPU= 39.225
  11:25:52 STEP  964 H= 964:00 +CPU= 24.490
  11:26:16 STEP  965 H= 965:00 +CPU= 23.726
  11:26:56 STEP  966 H= 966:00 +CPU= 40.091
  11:27:20 STEP  967 H= 967:00 +CPU= 23.808
  11:27:45 STEP  968 H= 968:00 +CPU= 24.592
  11:28:25 STEP  969 H= 969:00 +CPU= 39.321
  11:28:49 STEP  970 H= 970:00 +CPU= 24.589
  11:29:13 STEP  971 H= 971:00 +CPU= 23.776
  11:29:57 STEP  972 H= 972:00 +CPU= 43.426
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000972
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000972
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000972
  11:30:21 STEP  973 H= 973:00 +CPU= 23.836
  11:30:46 STEP  974 H= 974:00 +CPU= 24.611
  11:31:25 STEP  975 H= 975:00 +CPU= 39.302
  11:31:50 STEP  976 H= 976:00 +CPU= 24.479
  11:32:14 STEP  977 H= 977:00 +CPU= 23.810
  11:32:54 STEP  978 H= 978:00 +CPU= 40.164
  11:33:18 STEP  979 H= 979:00 +CPU= 23.739
  11:33:43 STEP  980 H= 980:00 +CPU= 24.419
  11:34:22 STEP  981 H= 981:00 +CPU= 39.107
  11:34:46 STEP  982 H= 982:00 +CPU= 24.466
  11:35:11 STEP  983 H= 983:00 +CPU= 24.393
  11:35:57 STEP  984 H= 984:00 +CPU= 45.098
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000984
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000984
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000984
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000960
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000960
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000960
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000972
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000972
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000972
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_8.zip
Uploading the intermediate file: upload_file_8.zip
  11:36:21 STEP  985 H= 985:00 +CPU= 23.799
  11:36:49 STEP  986 H= 986:00 +CPU= 24.783
  11:37:30 STEP  987 H= 987:00 +CPU= 39.467
  11:37:55 STEP  988 H= 988:00 +CPU= 24.572
  11:38:19 STEP  989 H= 989:00 +CPU= 23.820
  11:39:00 STEP  990 H= 990:00 +CPU= 40.088
  11:39:24 STEP  991 H= 991:00 +CPU= 23.933
  11:39:50 STEP  992 H= 992:00 +CPU= 24.652
  11:40:30 STEP  993 H= 993:00 +CPU= 39.349
  11:40:55 STEP  994 H= 994:00 +CPU= 24.545
  11:41:19 STEP  995 H= 995:00 +CPU= 23.867
  11:42:03 STEP  996 H= 996:00 +CPU= 43.443
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+000996
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+000996
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+000996
  11:42:29 STEP  997 H= 997:00 +CPU= 24.081
  11:42:54 STEP  998 H= 998:00 +CPU= 24.710
  11:43:35 STEP  999 H= 999:00 +CPU= 39.470
  11:44:00 STEP 1000 H=1000:00 +CPU= 24.581
  11:44:26 STEP 1001 H=1001:00 +CPU= 24.024
  11:45:07 STEP 1002 H=1002:00 +CPU= 40.115
  11:45:30 STEP 1003 H=1003:00 +CPU= 23.742
  11:45:55 STEP 1004 H=1004:00 +CPU= 24.592
  11:46:35 STEP 1005 H=1005:00 +CPU= 39.333
  11:47:00 STEP 1006 H=1006:00 +CPU= 24.525
  11:47:25 STEP 1007 H=1007:00 +CPU= 24.418
  11:48:11 STEP 1008 H=1008:00 +CPU= 45.203
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001008
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001008
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001008
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000984
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000984
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000984
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+000996
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+000996
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+000996
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_9.zip
Uploading the intermediate file: upload_file_9.zip
  11:48:36 STEP 1009 H=1009:00 +CPU= 23.842
  11:49:01 STEP 1010 H=1010:00 +CPU= 24.548
  11:49:40 STEP 1011 H=1011:00 +CPU= 39.234
  11:50:05 STEP 1012 H=1012:00 +CPU= 24.497
  11:50:30 STEP 1013 H=1013:00 +CPU= 23.812
  11:51:10 STEP 1014 H=1014:00 +CPU= 40.182
  11:51:34 STEP 1015 H=1015:00 +CPU= 23.813
  11:51:59 STEP 1016 H=1016:00 +CPU= 24.541
  11:52:39 STEP 1017 H=1017:00 +CPU= 39.293
  11:53:04 STEP 1018 H=1018:00 +CPU= 24.599
  11:53:28 STEP 1019 H=1019:00 +CPU= 23.764
  11:54:12 STEP 1020 H=1020:00 +CPU= 43.487
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001020
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001020
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001020
  11:54:36 STEP 1021 H=1021:00 +CPU= 23.879
  11:55:01 STEP 1022 H=1022:00 +CPU= 24.650
  11:55:41 STEP 1023 H=1023:00 +CPU= 39.308
  11:56:06 STEP 1024 H=1024:00 +CPU= 24.610
  11:56:31 STEP 1025 H=1025:00 +CPU= 23.850
  11:57:12 STEP 1026 H=1026:00 +CPU= 40.133
  11:57:36 STEP 1027 H=1027:00 +CPU= 23.774
  11:58:01 STEP 1028 H=1028:00 +CPU= 24.585
  11:58:41 STEP 1029 H=1029:00 +CPU= 39.334
  11:59:06 STEP 1030 H=1030:00 +CPU= 24.464
  11:59:31 STEP 1031 H=1031:00 +CPU= 24.466
  12:00:20 STEP 1032 H=1032:00 +CPU= 45.446
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001032
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001032
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001032
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001008
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001008
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001008
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001020
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001020
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001020
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_10.zip
Uploading the intermediate file: upload_file_10.zip
  12:00:47 STEP 1033 H=1033:00 +CPU= 24.009
Uploading trickle at timestep: 3715200
  12:01:12 STEP 1034 H=1034:00 +CPU= 24.641
  12:01:53 STEP 1035 H=1035:00 +CPU= 39.380
  12:02:18 STEP 1036 H=1036:00 +CPU= 24.682
  12:02:43 STEP 1037 H=1037:00 +CPU= 23.895
  12:03:25 STEP 1038 H=1038:00 +CPU= 40.324
  12:03:50 STEP 1039 H=1039:00 +CPU= 23.993
  12:04:15 STEP 1040 H=1040:00 +CPU= 24.727
  12:04:56 STEP 1041 H=1041:00 +CPU= 39.450
  12:05:22 STEP 1042 H=1042:00 +CPU= 24.796
  12:05:46 STEP 1043 H=1043:00 +CPU= 23.872
  12:06:33 STEP 1044 H=1044:00 +CPU= 43.745
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001044
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001044
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001044
  12:06:58 STEP 1045 H=1045:00 +CPU= 23.983
  12:07:23 STEP 1046 H=1046:00 +CPU= 24.686
  12:08:04 STEP 1047 H=1047:00 +CPU= 39.461
  12:08:29 STEP 1048 H=1048:00 +CPU= 24.681
  12:08:54 STEP 1049 H=1049:00 +CPU= 23.924
  12:09:38 STEP 1050 H=1050:00 +CPU= 40.427
  12:10:02 STEP 1051 H=1051:00 +CPU= 23.900
  12:10:28 STEP 1052 H=1052:00 +CPU= 24.708
  12:11:09 STEP 1053 H=1053:00 +CPU= 39.346
  12:11:34 STEP 1054 H=1054:00 +CPU= 24.522
  12:11:58 STEP 1055 H=1055:00 +CPU= 24.521
  12:12:45 STEP 1056 H=1056:00 +CPU= 45.415
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001056
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001056
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001056
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001032
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001032
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001032
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001044
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001044
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001044
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_11.zip
Uploading the intermediate file: upload_file_11.zip
  12:13:10 STEP 1057 H=1057:00 +CPU= 23.785
  12:13:34 STEP 1058 H=1058:00 +CPU= 24.498
  12:14:14 STEP 1059 H=1059:00 +CPU= 39.276
  12:14:39 STEP 1060 H=1060:00 +CPU= 24.606
  12:15:03 STEP 1061 H=1061:00 +CPU= 23.827
  12:15:44 STEP 1062 H=1062:00 +CPU= 40.279
  12:16:09 STEP 1063 H=1063:00 +CPU= 23.930
  12:16:34 STEP 1064 H=1064:00 +CPU= 24.679
  12:17:15 STEP 1065 H=1065:00 +CPU= 39.590
  12:17:40 STEP 1066 H=1066:00 +CPU= 24.666
  12:18:05 STEP 1067 H=1067:00 +CPU= 23.946
  12:18:50 STEP 1068 H=1068:00 +CPU= 43.581
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001068
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001068
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001068
  12:19:14 STEP 1069 H=1069:00 +CPU= 23.950
  12:19:39 STEP 1070 H=1070:00 +CPU= 24.656
  12:20:19 STEP 1071 H=1071:00 +CPU= 39.374
  12:20:45 STEP 1072 H=1072:00 +CPU= 24.697
  12:21:09 STEP 1073 H=1073:00 +CPU= 23.876
  12:21:50 STEP 1074 H=1074:00 +CPU= 40.314
  12:22:15 STEP 1075 H=1075:00 +CPU= 23.900
  12:22:40 STEP 1076 H=1076:00 +CPU= 24.587
  12:23:20 STEP 1077 H=1077:00 +CPU= 39.393
  12:23:47 STEP 1078 H=1078:00 +CPU= 24.747
  12:24:13 STEP 1079 H=1079:00 +CPU= 24.623
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:25:20 STEP 1080 H=1080:00 +CPU= 45.427
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001080
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001080
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001080
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001056
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001056
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001056
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001068
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001068
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001068
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_12.zip
Uploading the intermediate file: upload_file_12.zip
  12:25:45 STEP 1081 H=1081:00 +CPU= 23.824
  12:26:11 STEP 1082 H=1082:00 +CPU= 24.645
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:27:13 STEP 1083 H=1083:00 +CPU= 39.482
  12:27:38 STEP 1084 H=1084:00 +CPU= 24.700
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:28:24 STEP 1085 H=1085:00 +CPU= 23.903
  12:29:06 STEP 1086 H=1086:00 +CPU= 40.357
  12:29:32 STEP 1087 H=1087:00 +CPU= 24.162
  12:29:58 STEP 1088 H=1088:00 +CPU= 24.703
  12:30:40 STEP 1089 H=1089:00 +CPU= 39.579
  12:31:06 STEP 1090 H=1090:00 +CPU= 24.744
  12:31:31 STEP 1091 H=1091:00 +CPU= 24.060
  12:32:17 STEP 1092 H=1092:00 +CPU= 43.611
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001092
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001092
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001092
  12:32:42 STEP 1093 H=1093:00 +CPU= 24.007
  12:33:07 STEP 1094 H=1094:00 +CPU= 24.704
  12:33:47 STEP 1095 H=1095:00 +CPU= 39.377
  12:34:13 STEP 1096 H=1096:00 +CPU= 24.697
  12:34:39 STEP 1097 H=1097:00 +CPU= 23.961
  12:35:20 STEP 1098 H=1098:00 +CPU= 40.256
  12:35:45 STEP 1099 H=1099:00 +CPU= 23.893
  12:36:12 STEP 1100 H=1100:00 +CPU= 24.811
  12:36:54 STEP 1101 H=1101:00 +CPU= 39.640
  12:37:20 STEP 1102 H=1102:00 +CPU= 24.728
  12:37:45 STEP 1103 H=1103:00 +CPU= 24.587
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  12:38:53 STEP 1104 H=1104:00 +CPU= 45.237
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001104
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001104
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001104
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001080
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001080
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001080
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001092
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001092
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001092
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_13.zip
Uploading the intermediate file: upload_file_13.zip
  12:39:18 STEP 1105 H=1105:00 +CPU= 23.967
  12:39:43 STEP 1106 H=1106:00 +CPU= 24.739
  12:40:24 STEP 1107 H=1107:00 +CPU= 39.471
  12:40:49 STEP 1108 H=1108:00 +CPU= 24.718
  12:41:14 STEP 1109 H=1109:00 +CPU= 24.007
  12:41:59 STEP 1110 H=1110:00 +CPU= 40.718
  12:42:25 STEP 1111 H=1111:00 +CPU= 24.171
  12:42:51 STEP 1112 H=1112:00 +CPU= 24.940
  12:43:33 STEP 1113 H=1113:00 +CPU= 39.715
  12:43:59 STEP 1114 H=1114:00 +CPU= 24.893
  12:44:24 STEP 1115 H=1115:00 +CPU= 24.092
  12:45:10 STEP 1116 H=1116:00 +CPU= 43.947
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001116
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001116
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001116
  12:45:37 STEP 1117 H=1117:00 +CPU= 24.210
  12:46:04 STEP 1118 H=1118:00 +CPU= 25.051
  12:46:46 STEP 1119 H=1119:00 +CPU= 39.778
  12:47:12 STEP 1120 H=1120:00 +CPU= 24.865
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2001050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0184
(argv4) batchid: 970
(argv5) wuid: 12186828
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /home/kuba/slots/5
Project directory is: /home/kuba/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828
..mkdir for temp folder for results failed
Copying: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/kuba/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/kuba/slots/5/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_acff872e97c176530e908b74118fbaac to: /home/kuba/slots/5/oifs_43r3_ps_0184_2001050100_123_970_12186828.zip
Unzipping the namelist zip file: /home/kuba/slots/5/oifs_43r3_ps_0184_2001050100_123_970_12186828.zip
ic_ancil_file: ic_ancil_12186828
ifsdata_file: ifsdata_12186828
climate_data_file: clim_data_12186828
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_59ea3c8daa94d8370a8d527cf5808c72 to: /home/kuba/slots/5/ic_ancil_12186828.zip
Unzipping the IC ancils zip file: /home/kuba/slots/5/ic_ancil_12186828.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_3a831086924e6b09320ddfb27d2eefee to: /home/kuba/slots/5/ifsdata/ifsdata_12186828.zip
Unzipping the ifsdata_zip file: /home/kuba/slots/5/ifsdata/ifsdata_12186828.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/kuba/slots/5/159l_2/clim_data_12186828.zip
Unzipping the climate data zip file: /home/kuba/slots/5/159l_2/clim_data_12186828.zip
Checking for progress XML file: /home/kuba/slots/5/progress_file_12186828.xml
Opened progress file ok : /home/kuba/slots/5/progress_file_12186828.xml
-- Model is restarting --
Adjusting last_iter, 1121, to previous model restart step.
Creating progress file: /home/kuba/slots/5/progress_file_12186828.xml
last_cpu_time: 9774
upload_file_number: 14
last_iter: 1103
last_upload: 3974400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841
The child process has been launched with process id: 74
Executing the command: /home/kuba/slots/5/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ElementaryBlack:1:1:74:74] [20230206:164609:1675698369:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  16:46:59 STEP 1104 H=1104:00 +CPU= 43.741
  16:47:24 STEP 1105 H=1105:00 +CPU= 23.958
  16:47:49 STEP 1106 H=1106:00 +CPU= 24.715
  16:48:28 STEP 1107 H=1107:00 +CPU= 39.513
  16:48:53 STEP 1108 H=1108:00 +CPU= 24.717
  16:49:18 STEP 1109 H=1109:00 +CPU= 23.962
  16:49:58 STEP 1110 H=1110:00 +CPU= 40.320
  16:50:22 STEP 1111 H=1111:00 +CPU= 23.997
  16:50:48 STEP 1112 H=1112:00 +CPU= 24.736
  16:51:28 STEP 1113 H=1113:00 +CPU= 39.558
  16:51:53 STEP 1114 H=1114:00 +CPU= 24.740
  16:52:17 STEP 1115 H=1115:00 +CPU= 24.026
  16:53:01 STEP 1116 H=1116:00 +CPU= 43.625
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001116
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001116
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001116
  16:53:25 STEP 1117 H=1117:00 +CPU= 24.016
  16:53:50 STEP 1118 H=1118:00 +CPU= 24.795
  16:54:30 STEP 1119 H=1119:00 +CPU= 39.454
  16:54:55 STEP 1120 H=1120:00 +CPU= 24.666
  16:55:20 STEP 1121 H=1121:00 +CPU= 23.960
  16:56:00 STEP 1122 H=1122:00 +CPU= 40.264
  16:56:25 STEP 1123 H=1123:00 +CPU= 23.926
  16:56:50 STEP 1124 H=1124:00 +CPU= 24.667
  16:57:29 STEP 1125 H=1125:00 +CPU= 39.455
  16:57:54 STEP 1126 H=1126:00 +CPU= 24.729
  16:58:19 STEP 1127 H=1127:00 +CPU= 24.645
  16:59:05 STEP 1128 H=1128:00 +CPU= 45.463
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001128
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001128
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001128
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001104
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001104
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001104
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001116
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001116
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001116
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_14.zip
Uploading the intermediate file: upload_file_14.zip
  16:59:30 STEP 1129 H=1129:00 +CPU= 23.918
  16:59:54 STEP 1130 H=1130:00 +CPU= 24.711
  17:00:34 STEP 1131 H=1131:00 +CPU= 39.533
  17:00:59 STEP 1132 H=1132:00 +CPU= 24.768
  17:01:23 STEP 1133 H=1133:00 +CPU= 23.985
  17:02:04 STEP 1134 H=1134:00 +CPU= 40.472
  17:02:28 STEP 1135 H=1135:00 +CPU= 24.075
  17:02:53 STEP 1136 H=1136:00 +CPU= 24.816
  17:03:33 STEP 1137 H=1137:00 +CPU= 39.519
  17:03:58 STEP 1138 H=1138:00 +CPU= 24.699
  17:04:22 STEP 1139 H=1139:00 +CPU= 24.057
  17:05:06 STEP 1140 H=1140:00 +CPU= 43.781
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001140
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001140
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001140
  17:05:30 STEP 1141 H=1141:00 +CPU= 24.068
  17:05:56 STEP 1142 H=1142:00 +CPU= 24.870
  17:06:35 STEP 1143 H=1143:00 +CPU= 39.575
  17:07:00 STEP 1144 H=1144:00 +CPU= 24.771
  17:07:25 STEP 1145 H=1145:00 +CPU= 24.054
  17:08:05 STEP 1146 H=1146:00 +CPU= 40.396
  17:08:30 STEP 1147 H=1147:00 +CPU= 23.981
  17:08:54 STEP 1148 H=1148:00 +CPU= 24.787
  17:09:34 STEP 1149 H=1149:00 +CPU= 39.632
  17:09:59 STEP 1150 H=1150:00 +CPU= 24.784
  17:10:24 STEP 1151 H=1151:00 +CPU= 24.727
  17:11:10 STEP 1152 H=1152:00 +CPU= 45.491
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001152
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001152
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001152
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001128
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001128
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001128
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001140
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001140
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001140
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_15.zip
Uploading the intermediate file: upload_file_15.zip
  17:11:34 STEP 1153 H=1153:00 +CPU= 23.901
  17:11:59 STEP 1154 H=1154:00 +CPU= 24.700
  17:12:39 STEP 1155 H=1155:00 +CPU= 39.521
  17:13:04 STEP 1156 H=1156:00 +CPU= 24.733
  17:13:28 STEP 1157 H=1157:00 +CPU= 24.004
  17:14:09 STEP 1158 H=1158:00 +CPU= 40.348
  17:14:33 STEP 1159 H=1159:00 +CPU= 24.061
  17:14:58 STEP 1160 H=1160:00 +CPU= 24.811
  17:15:38 STEP 1161 H=1161:00 +CPU= 39.593
  17:16:03 STEP 1162 H=1162:00 +CPU= 24.823
  17:16:27 STEP 1163 H=1163:00 +CPU= 24.117
  17:17:11 STEP 1164 H=1164:00 +CPU= 43.748
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001164
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001164
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001164
  17:17:36 STEP 1165 H=1165:00 +CPU= 24.119
  17:18:01 STEP 1166 H=1166:00 +CPU= 24.801
  17:18:41 STEP 1167 H=1167:00 +CPU= 39.608
  17:19:06 STEP 1168 H=1168:00 +CPU= 24.815
  17:19:30 STEP 1169 H=1169:00 +CPU= 24.109
  17:20:11 STEP 1170 H=1170:00 +CPU= 40.475
  17:20:35 STEP 1171 H=1171:00 +CPU= 24.095
  17:21:00 STEP 1172 H=1172:00 +CPU= 24.829
  17:21:40 STEP 1173 H=1173:00 +CPU= 39.544
  17:22:05 STEP 1174 H=1174:00 +CPU= 24.735
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:22:51 STEP 1175 H=1175:00 +CPU= 24.511
  17:23:37 STEP 1176 H=1176:00 +CPU= 45.179
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001176
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001176
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001176
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001152
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001152
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001152
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001164
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001164
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001164
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_16.zip
Uploading the intermediate file: upload_file_16.zip
  17:24:01 STEP 1177 H=1177:00 +CPU= 23.784
  17:24:25 STEP 1178 H=1178:00 +CPU= 24.556
  17:25:05 STEP 1179 H=1179:00 +CPU= 39.290
  17:25:30 STEP 1180 H=1180:00 +CPU= 24.623
  17:25:54 STEP 1181 H=1181:00 +CPU= 23.913
  17:26:34 STEP 1182 H=1182:00 +CPU= 40.261
  17:26:59 STEP 1183 H=1183:00 +CPU= 23.947
  17:27:24 STEP 1184 H=1184:00 +CPU= 24.685
  17:28:03 STEP 1185 H=1185:00 +CPU= 39.422
  17:28:28 STEP 1186 H=1186:00 +CPU= 24.668
  17:28:52 STEP 1187 H=1187:00 +CPU= 23.952
  17:29:36 STEP 1188 H=1188:00 +CPU= 43.483
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001188
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001188
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001188
  17:30:00 STEP 1189 H=1189:00 +CPU= 23.973
  17:30:25 STEP 1190 H=1190:00 +CPU= 24.738
  17:31:05 STEP 1191 H=1191:00 +CPU= 39.382
  17:31:29 STEP 1192 H=1192:00 +CPU= 24.650
  17:31:53 STEP 1193 H=1193:00 +CPU= 23.874
  17:32:34 STEP 1194 H=1194:00 +CPU= 40.191
  17:32:58 STEP 1195 H=1195:00 +CPU= 23.861
  17:33:23 STEP 1196 H=1196:00 +CPU= 24.567
  17:34:02 STEP 1197 H=1197:00 +CPU= 39.241
  17:34:27 STEP 1198 H=1198:00 +CPU= 24.562
  17:34:52 STEP 1199 H=1199:00 +CPU= 24.505
  17:35:38 STEP 1200 H=1200:00 +CPU= 45.264
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001200
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001200
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001200
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001176
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001176
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001176
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001188
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001188
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001188
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_17.zip
Uploading the intermediate file: upload_file_17.zip
  17:36:02 STEP 1201 H=1201:00 +CPU= 23.809
  17:36:27 STEP 1202 H=1202:00 +CPU= 24.580
  17:37:06 STEP 1203 H=1203:00 +CPU= 39.256
  17:37:31 STEP 1204 H=1204:00 +CPU= 24.521
  17:37:55 STEP 1205 H=1205:00 +CPU= 23.817
  17:38:35 STEP 1206 H=1206:00 +CPU= 40.236
  17:38:59 STEP 1207 H=1207:00 +CPU= 23.847
  17:39:24 STEP 1208 H=1208:00 +CPU= 24.671
  17:40:04 STEP 1209 H=1209:00 +CPU= 39.390
  17:40:29 STEP 1210 H=1210:00 +CPU= 24.694
  17:40:53 STEP 1211 H=1211:00 +CPU= 23.960
  17:41:37 STEP 1212 H=1212:00 +CPU= 43.498
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001212
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001212
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001212
  17:42:01 STEP 1213 H=1213:00 +CPU= 23.898
  17:42:26 STEP 1214 H=1214:00 +CPU= 24.625
  17:43:05 STEP 1215 H=1215:00 +CPU= 39.313
  17:43:30 STEP 1216 H=1216:00 +CPU= 24.642
  17:43:54 STEP 1217 H=1217:00 +CPU= 23.900
  17:44:35 STEP 1218 H=1218:00 +CPU= 40.164
  17:44:59 STEP 1219 H=1219:00 +CPU= 23.831
  17:45:24 STEP 1220 H=1220:00 +CPU= 24.571
  17:46:03 STEP 1221 H=1221:00 +CPU= 39.268
  17:46:28 STEP 1222 H=1222:00 +CPU= 24.622
  17:46:53 STEP 1223 H=1223:00 +CPU= 24.535
  17:47:39 STEP 1224 H=1224:00 +CPU= 45.212
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001224
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001224
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001224
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001200
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001200
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001200
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001212
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001212
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001212
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_18.zip
Uploading the intermediate file: upload_file_18.zip
  17:48:03 STEP 1225 H=1225:00 +CPU= 23.828
  17:48:27 STEP 1226 H=1226:00 +CPU= 24.494
  17:49:07 STEP 1227 H=1227:00 +CPU= 39.174
  17:49:32 STEP 1228 H=1228:00 +CPU= 24.515
  17:49:56 STEP 1229 H=1229:00 +CPU= 23.874
  17:50:36 STEP 1230 H=1230:00 +CPU= 40.167
  17:51:00 STEP 1231 H=1231:00 +CPU= 23.759
  17:51:24 STEP 1232 H=1232:00 +CPU= 24.595
  17:52:04 STEP 1233 H=1233:00 +CPU= 39.343
  17:52:29 STEP 1234 H=1234:00 +CPU= 24.636
  17:52:53 STEP 1235 H=1235:00 +CPU= 23.879
  17:53:37 STEP 1236 H=1236:00 +CPU= 43.526
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001236
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001236
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001236
  17:54:01 STEP 1237 H=1237:00 +CPU= 23.915
  17:54:26 STEP 1238 H=1238:00 +CPU= 24.636
  17:55:05 STEP 1239 H=1239:00 +CPU= 39.259
  17:55:30 STEP 1240 H=1240:00 +CPU= 24.619
  17:55:54 STEP 1241 H=1241:00 +CPU= 23.848
  17:56:35 STEP 1242 H=1242:00 +CPU= 40.113
  17:56:59 STEP 1243 H=1243:00 +CPU= 23.768
  17:57:24 STEP 1244 H=1244:00 +CPU= 24.495
  17:58:03 STEP 1245 H=1245:00 +CPU= 39.258
  17:58:28 STEP 1246 H=1246:00 +CPU= 24.562
  17:58:52 STEP 1247 H=1247:00 +CPU= 24.498
  17:59:38 STEP 1248 H=1248:00 +CPU= 45.174
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001248
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001248
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001248
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001224
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001224
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001224
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001236
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001236
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001236
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_19.zip
Uploading the intermediate file: upload_file_19.zip
  18:00:02 STEP 1249 H=1249:00 +CPU= 23.808
  18:00:27 STEP 1250 H=1250:00 +CPU= 24.559
  18:01:06 STEP 1251 H=1251:00 +CPU= 39.250
  18:01:31 STEP 1252 H=1252:00 +CPU= 24.559
  18:01:55 STEP 1253 H=1253:00 +CPU= 23.852
  18:02:36 STEP 1254 H=1254:00 +CPU= 40.090
  18:03:00 STEP 1255 H=1255:00 +CPU= 23.823
  18:03:24 STEP 1256 H=1256:00 +CPU= 24.654
  18:04:04 STEP 1257 H=1257:00 +CPU= 39.293
  18:04:29 STEP 1258 H=1258:00 +CPU= 24.634
  18:04:53 STEP 1259 H=1259:00 +CPU= 23.846
  18:05:37 STEP 1260 H=1260:00 +CPU= 43.364
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001260
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001260
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001260
  18:06:01 STEP 1261 H=1261:00 +CPU= 23.848
  18:06:26 STEP 1262 H=1262:00 +CPU= 24.569
  18:07:05 STEP 1263 H=1263:00 +CPU= 39.146
  18:07:30 STEP 1264 H=1264:00 +CPU= 24.573
  18:07:54 STEP 1265 H=1265:00 +CPU= 23.786
  18:08:34 STEP 1266 H=1266:00 +CPU= 39.974
  18:08:58 STEP 1267 H=1267:00 +CPU= 23.766
  18:09:23 STEP 1268 H=1268:00 +CPU= 24.480
  18:10:06 STEP 1269 H=1269:00 +CPU= 39.239
  18:10:35 STEP 1270 H=1270:00 +CPU= 24.727
  18:11:00 STEP 1271 H=1271:00 +CPU= 24.477
  18:11:46 STEP 1272 H=1272:00 +CPU= 45.184
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001272
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001272
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001272
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001248
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001248
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001248
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001260
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001260
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001260
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_20.zip
Uploading the intermediate file: upload_file_20.zip
  18:12:10 STEP 1273 H=1273:00 +CPU= 23.749
  18:12:35 STEP 1274 H=1274:00 +CPU= 24.553
  18:13:14 STEP 1275 H=1275:00 +CPU= 39.243
  18:13:39 STEP 1276 H=1276:00 +CPU= 24.541
  18:14:03 STEP 1277 H=1277:00 +CPU= 23.833
  18:14:43 STEP 1278 H=1278:00 +CPU= 40.127
  18:15:07 STEP 1279 H=1279:00 +CPU= 23.821
  18:15:32 STEP 1280 H=1280:00 +CPU= 24.615
  18:16:11 STEP 1281 H=1281:00 +CPU= 39.214
  18:16:36 STEP 1282 H=1282:00 +CPU= 24.547
  18:17:00 STEP 1283 H=1283:00 +CPU= 23.767
  18:17:44 STEP 1284 H=1284:00 +CPU= 43.361
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001284
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001284
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001284
  18:18:08 STEP 1285 H=1285:00 +CPU= 23.817
  18:18:32 STEP 1286 H=1286:00 +CPU= 24.513
  18:19:12 STEP 1287 H=1287:00 +CPU= 39.171
  18:19:37 STEP 1288 H=1288:00 +CPU= 24.546
  18:20:01 STEP 1289 H=1289:00 +CPU= 23.837
  18:20:41 STEP 1290 H=1290:00 +CPU= 40.138
  18:21:05 STEP 1291 H=1291:00 +CPU= 23.731
  18:21:30 STEP 1292 H=1292:00 +CPU= 24.522
  18:22:09 STEP 1293 H=1293:00 +CPU= 39.118
  18:22:34 STEP 1294 H=1294:00 +CPU= 24.422
  18:22:58 STEP 1295 H=1295:00 +CPU= 24.436
  18:23:44 STEP 1296 H=1296:00 +CPU= 45.230
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001296
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001296
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001296
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001272
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001272
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001272
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001284
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001284
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001284
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_21.zip
Uploading the intermediate file: upload_file_21.zip
  18:24:08 STEP 1297 H=1297:00 +CPU= 23.722
  18:24:33 STEP 1298 H=1298:00 +CPU= 24.537
  18:25:12 STEP 1299 H=1299:00 +CPU= 39.120
  18:25:37 STEP 1300 H=1300:00 +CPU= 24.454
  18:26:01 STEP 1301 H=1301:00 +CPU= 23.810
  18:26:41 STEP 1302 H=1302:00 +CPU= 40.101
  18:27:05 STEP 1303 H=1303:00 +CPU= 23.759
  18:27:30 STEP 1304 H=1304:00 +CPU= 24.590
  18:28:09 STEP 1305 H=1305:00 +CPU= 39.313
  18:28:34 STEP 1306 H=1306:00 +CPU= 24.528
  18:28:58 STEP 1307 H=1307:00 +CPU= 23.802
  18:29:42 STEP 1308 H=1308:00 +CPU= 43.314
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001308
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001308
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001308
  18:30:06 STEP 1309 H=1309:00 +CPU= 23.758
  18:30:30 STEP 1310 H=1310:00 +CPU= 24.507
  18:31:10 STEP 1311 H=1311:00 +CPU= 39.320
  18:31:34 STEP 1312 H=1312:00 +CPU= 24.592
  18:31:58 STEP 1313 H=1313:00 +CPU= 23.797
  18:32:39 STEP 1314 H=1314:00 +CPU= 40.091
  18:33:03 STEP 1315 H=1315:00 +CPU= 23.764
  18:33:27 STEP 1316 H=1316:00 +CPU= 24.570
  18:34:07 STEP 1317 H=1317:00 +CPU= 39.228
  18:34:31 STEP 1318 H=1318:00 +CPU= 24.483
  18:34:56 STEP 1319 H=1319:00 +CPU= 24.390
  18:35:42 STEP 1320 H=1320:00 +CPU= 45.141
Moving to projects directory: /home/kuba/slots/5/ICMGGhq0f+001320
Moving to projects directory: /home/kuba/slots/5/ICMSHhq0f+001320
Moving to projects directory: /home/kuba/slots/5/ICMUAhq0f+001320
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001296
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001296
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001296
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMGGhq0f+001308
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMSHhq0f+001308
Adding to the zip: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_12186828/ICMUAhq0f+001308
Zipping up the intermediate file: /home/kuba/projects/climateprediction.net/oifs_43r3_ps_0184_2001050100_123_970_12186828_1_r224808841_22.zip
Uploading the intermediate file: upload_file_22.zip
  18:36:06 STEP 1321 H=1321:00 +CPU= 23.761
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:48:11 1537829 22303570 oifs_43r3_ps_0184_2001050100_123_970_12186828_1 3,715,200 7,153 0.0019
09 May 2023 16:47:03 1537829 22303570 oifs_43r3_ps_0184_2001050100_123_970_12186828_1 1,641,600 14,651 0.0089


©2024 climateprediction.net