climateprediction.net home page
Task 22438178

Task 22438178

Name oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0
Workunit 12286557
Created 7 Jun 2024, 17:41:01 UTC
Sent 8 Jun 2024, 1:04:54 UTC
Report deadline 7 Aug 2024, 1:04:54 UTC
Received 8 Jun 2024, 18:18:06 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 0 (0x00000000)
Computer ID 1507812
Run time 14 hours 17 min 10 sec
CPU time 12 hours 21 min 20 sec
Validate state Invalid
Credit 1,976.32
Device peak FLOPS 5.39 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,502.25 MB
Peak swap size 5,980.80 MB
Peak disk usage 2,013.84 MB
Stderr
<core_client_version>7.24.1</core_client_version>
<![CDATA[
<stderr_txt>
ng_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.001] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:zion.faskatech.net:1:1:1659328:1659328] [20240608:133622:1717846582:0.001] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  13:37:31 STEP  864 H= 216:00 +CPU= 24.899
  13:37:59 STEP  865 H= 216:15 +CPU= 23.841
  13:38:26 STEP  866 H= 216:30 +CPU= 23.747
  13:38:54 STEP  867 H= 216:45 +CPU= 24.156
  13:39:22 STEP  868 H= 217:00 +CPU= 23.411
  13:39:51 STEP  869 H= 217:15 +CPU= 24.921
  13:40:21 STEP  870 H= 217:30 +CPU= 24.603
  13:40:49 STEP  871 H= 217:45 +CPU= 24.463
  13:41:18 STEP  872 H= 218:00 +CPU= 25.171
  13:41:47 STEP  873 H= 218:15 +CPU= 24.147
  13:42:15 STEP  874 H= 218:30 +CPU= 23.815
  13:42:43 STEP  875 H= 218:45 +CPU= 23.832
  13:43:17 STEP  876 H= 219:00 +CPU= 29.304
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000876
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000876
  13:43:46 STEP  877 H= 219:15 +CPU= 23.704
  13:44:13 STEP  878 H= 219:30 +CPU= 23.213
  13:44:40 STEP  879 H= 219:45 +CPU= 23.173
  13:45:07 STEP  880 H= 220:00 +CPU= 23.780
  13:45:37 STEP  881 H= 220:15 +CPU= 25.017
  13:46:06 STEP  882 H= 220:30 +CPU= 24.643
  13:46:36 STEP  883 H= 220:45 +CPU= 24.762
  13:47:05 STEP  884 H= 221:00 +CPU= 25.030
  13:47:35 STEP  885 H= 221:15 +CPU= 24.460
  13:48:03 STEP  886 H= 221:30 +CPU= 24.527
  13:48:31 STEP  887 H= 221:45 +CPU= 24.071
  13:49:04 STEP  888 H= 222:00 +CPU= 28.980
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000888
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000888
  13:49:33 STEP  889 H= 222:15 +CPU= 23.208
  13:49:59 STEP  890 H= 222:30 +CPU= 23.069
  13:50:27 STEP  891 H= 222:45 +CPU= 23.997
  13:50:56 STEP  892 H= 223:00 +CPU= 24.390
  13:51:24 STEP  893 H= 223:15 +CPU= 24.723
  13:51:54 STEP  894 H= 223:30 +CPU= 24.687
  13:52:22 STEP  895 H= 223:45 +CPU= 24.611
  13:52:51 STEP  896 H= 224:00 +CPU= 25.037
  13:53:19 STEP  897 H= 224:15 +CPU= 24.589
  13:53:49 STEP  898 H= 224:30 +CPU= 24.709
  13:54:17 STEP  899 H= 224:45 +CPU= 24.611
  13:54:52 STEP  900 H= 225:00 +CPU= 29.344
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000900
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000900
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000900
  13:55:19 STEP  901 H= 225:15 +CPU= 23.236
  13:55:47 STEP  902 H= 225:30 +CPU= 23.322
  13:56:14 STEP  903 H= 225:45 +CPU= 23.888
  13:56:44 STEP  904 H= 226:00 +CPU= 24.387
  13:57:13 STEP  905 H= 226:15 +CPU= 24.705
  13:57:42 STEP  906 H= 226:30 +CPU= 24.624
  13:58:11 STEP  907 H= 226:45 +CPU= 24.579
  13:58:40 STEP  908 H= 227:00 +CPU= 24.988
  13:59:10 STEP  909 H= 227:15 +CPU= 25.393
  13:59:40 STEP  910 H= 227:30 +CPU= 24.731
  14:00:10 STEP  911 H= 227:45 +CPU= 24.729
  14:01:00 STEP  912 H= 228:00 +CPU= 33.855
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000912
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000912
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000912
  14:01:31 STEP  913 H= 228:15 +CPU= 25.366
  14:02:01 STEP  914 H= 228:30 +CPU= 24.763
  14:02:30 STEP  915 H= 228:45 +CPU= 24.642
  14:03:01 STEP  916 H= 229:00 +CPU= 24.830
  14:03:31 STEP  917 H= 229:15 +CPU= 24.337
  14:04:01 STEP  918 H= 229:30 +CPU= 24.527
  14:04:30 STEP  919 H= 229:45 +CPU= 24.462
  14:05:00 STEP  920 H= 230:00 +CPU= 24.959
  14:05:30 STEP  921 H= 230:15 +CPU= 24.545
  14:06:00 STEP  922 H= 230:30 +CPU= 24.841
  14:06:31 STEP  923 H= 230:45 +CPU= 24.863
  14:07:10 STEP  924 H= 231:00 +CPU= 31.656
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000924
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000924
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000924
  14:07:42 STEP  925 H= 231:15 +CPU= 25.879
  14:08:14 STEP  926 H= 231:30 +CPU= 25.960
  14:08:45 STEP  927 H= 231:45 +CPU= 25.629
  14:09:16 STEP  928 H= 232:00 +CPU= 25.971
  14:09:46 STEP  929 H= 232:15 +CPU= 25.596
  14:10:17 STEP  930 H= 232:30 +CPU= 25.690
  14:10:48 STEP  931 H= 232:45 +CPU= 25.378
  14:11:19 STEP  932 H= 233:00 +CPU= 25.555
  14:11:49 STEP  933 H= 233:15 +CPU= 24.758
  14:12:18 STEP  934 H= 233:30 +CPU= 24.424
  14:12:47 STEP  935 H= 233:45 +CPU= 24.295
  14:13:20 STEP  936 H= 234:00 +CPU= 29.153
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000936
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000936
  14:13:44 STEP  937 H= 234:15 +CPU= 22.512
  14:14:07 STEP  938 H= 234:30 +CPU= 20.788
  14:14:27 STEP  939 H= 234:45 +CPU= 19.200
  14:14:49 STEP  940 H= 235:00 +CPU= 20.520
  14:15:14 STEP  941 H= 235:15 +CPU= 23.120
  14:15:51 STEP  942 H= 235:30 +CPU= 30.699
  14:16:30 STEP  943 H= 235:45 +CPU= 32.294
  14:17:10 STEP  944 H= 236:00 +CPU= 32.764
  14:17:50 STEP  945 H= 236:15 +CPU= 32.106
  14:18:25 STEP  946 H= 236:30 +CPU= 30.368
  14:18:58 STEP  947 H= 236:45 +CPU= 30.107
  14:19:41 STEP  948 H= 237:00 +CPU= 37.568
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000948
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000948
  14:20:18 STEP  949 H= 237:15 +CPU= 31.192
  14:20:56 STEP  950 H= 237:30 +CPU= 32.323
  14:21:35 STEP  951 H= 237:45 +CPU= 32.060
  14:22:13 STEP  952 H= 238:00 +CPU= 32.251
  14:22:50 STEP  953 H= 238:15 +CPU= 31.398
  14:23:28 STEP  954 H= 238:30 +CPU= 32.174
  14:24:03 STEP  955 H= 238:45 +CPU= 30.930
  14:24:36 STEP  956 H= 239:00 +CPU= 30.058
  14:25:12 STEP  957 H= 239:15 +CPU= 30.237
  14:25:47 STEP  958 H= 239:30 +CPU= 30.418
  14:26:24 STEP  959 H= 239:45 +CPU= 31.289
  14:27:27 STEP  960 H= 240:00 +CPU= 40.377
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000960
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000864
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000876
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000888
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000900
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000912
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000924
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000936
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000948
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000948
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_9.zip
Uploading the intermediate file: upload_file_9.zip
  14:28:03 STEP  961 H= 240:15 +CPU= 28.897
  14:28:37 STEP  962 H= 240:30 +CPU= 29.203
  14:29:12 STEP  963 H= 240:45 +CPU= 29.833
  14:29:47 STEP  964 H= 241:00 +CPU= 29.977
  14:30:21 STEP  965 H= 241:15 +CPU= 28.747
  14:30:56 STEP  966 H= 241:30 +CPU= 29.186
  14:31:29 STEP  967 H= 241:45 +CPU= 29.330
  14:32:06 STEP  968 H= 242:00 +CPU= 30.659
  14:32:40 STEP  969 H= 242:15 +CPU= 29.607
  14:33:17 STEP  970 H= 242:30 +CPU= 31.283
  14:33:53 STEP  971 H= 242:45 +CPU= 30.082
  14:34:37 STEP  972 H= 243:00 +CPU= 37.529
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000972
  14:35:11 STEP  973 H= 243:15 +CPU= 29.712
  14:35:46 STEP  974 H= 243:30 +CPU= 29.529
  14:36:20 STEP  975 H= 243:45 +CPU= 29.314
  14:36:56 STEP  976 H= 244:00 +CPU= 29.779
  14:37:29 STEP  977 H= 244:15 +CPU= 28.278
  14:38:05 STEP  978 H= 244:30 +CPU= 30.288
  14:38:41 STEP  979 H= 244:45 +CPU= 30.639
  14:39:18 STEP  980 H= 245:00 +CPU= 31.550
  14:39:53 STEP  981 H= 245:15 +CPU= 30.434
  14:40:30 STEP  982 H= 245:30 +CPU= 31.345
  14:41:04 STEP  983 H= 245:45 +CPU= 29.551
  14:41:47 STEP  984 H= 246:00 +CPU= 36.739
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000984
  14:42:21 STEP  985 H= 246:15 +CPU= 29.893
  14:42:57 STEP  986 H= 246:30 +CPU= 30.092
  14:43:32 STEP  987 H= 246:45 +CPU= 29.836
  14:44:05 STEP  988 H= 247:00 +CPU= 28.572
  14:44:28 STEP  989 H= 247:15 +CPU= 20.934
  14:44:48 STEP  990 H= 247:30 +CPU= 19.294
  14:45:17 STEP  991 H= 247:45 +CPU= 25.358
  14:45:57 STEP  992 H= 248:00 +CPU= 30.182
  14:46:38 STEP  993 H= 248:15 +CPU= 30.754
  14:47:14 STEP  994 H= 248:30 +CPU= 30.234
  14:47:49 STEP  995 H= 248:45 +CPU= 29.920
  14:48:31 STEP  996 H= 249:00 +CPU= 36.803
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+000996
  14:49:06 STEP  997 H= 249:15 +CPU= 29.990
  14:49:40 STEP  998 H= 249:30 +CPU= 30.330
  14:50:13 STEP  999 H= 249:45 +CPU= 28.905
  14:50:48 STEP 1000 H= 250:00 +CPU= 31.058
  14:51:21 STEP 1001 H= 250:15 +CPU= 29.300
  14:51:56 STEP 1002 H= 250:30 +CPU= 30.699
  14:52:32 STEP 1003 H= 250:45 +CPU= 30.947
  14:53:07 STEP 1004 H= 251:00 +CPU= 31.105
  14:53:41 STEP 1005 H= 251:15 +CPU= 29.742
  14:54:14 STEP 1006 H= 251:30 +CPU= 29.319
  14:54:48 STEP 1007 H= 251:45 +CPU= 29.172
  14:55:42 STEP 1008 H= 252:00 +CPU= 40.034
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001008
  14:56:17 STEP 1009 H= 252:15 +CPU= 30.213
  14:56:52 STEP 1010 H= 252:30 +CPU= 30.073
  14:57:26 STEP 1011 H= 252:45 +CPU= 29.737
  14:58:02 STEP 1012 H= 253:00 +CPU= 31.069
  14:58:36 STEP 1013 H= 253:15 +CPU= 30.290
  14:59:10 STEP 1014 H= 253:30 +CPU= 29.856
  14:59:44 STEP 1015 H= 253:45 +CPU= 29.571
  15:00:19 STEP 1016 H= 254:00 +CPU= 30.538
  15:00:53 STEP 1017 H= 254:15 +CPU= 29.926
  15:01:27 STEP 1018 H= 254:30 +CPU= 29.397
  15:02:00 STEP 1019 H= 254:45 +CPU= 29.600
  15:02:42 STEP 1020 H= 255:00 +CPU= 37.220
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001020
  15:03:15 STEP 1021 H= 255:15 +CPU= 28.979
  15:03:50 STEP 1022 H= 255:30 +CPU= 30.488
  15:04:24 STEP 1023 H= 255:45 +CPU= 29.492
  15:05:00 STEP 1024 H= 256:00 +CPU= 31.291
  15:05:34 STEP 1025 H= 256:15 +CPU= 29.921
  15:06:10 STEP 1026 H= 256:30 +CPU= 31.585
  15:06:45 STEP 1027 H= 256:45 +CPU= 29.892
  15:07:20 STEP 1028 H= 257:00 +CPU= 30.031
  15:07:54 STEP 1029 H= 257:15 +CPU= 29.587
  15:08:28 STEP 1030 H= 257:30 +CPU= 29.841
  15:09:02 STEP 1031 H= 257:45 +CPU= 29.835
  15:09:44 STEP 1032 H= 258:00 +CPU= 37.431
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001032
  15:10:18 STEP 1033 H= 258:15 +CPU= 29.604
  15:10:53 STEP 1034 H= 258:30 +CPU= 30.529
  15:11:28 STEP 1035 H= 258:45 +CPU= 30.854
  15:12:05 STEP 1036 H= 259:00 +CPU= 31.880
  15:12:41 STEP 1037 H= 259:15 +CPU= 31.550
  15:13:11 STEP 1038 H= 259:30 +CPU= 26.765
  15:13:30 STEP 1039 H= 259:45 +CPU= 18.023
  15:14:01 STEP 1040 H= 260:00 +CPU= 21.125
  15:14:33 STEP 1041 H= 260:15 +CPU= 27.280
  15:15:06 STEP 1042 H= 260:30 +CPU= 27.889
  15:15:38 STEP 1043 H= 260:45 +CPU= 28.274
  15:16:20 STEP 1044 H= 261:00 +CPU= 36.128
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001044
  15:16:53 STEP 1045 H= 261:15 +CPU= 28.551
  15:17:26 STEP 1046 H= 261:30 +CPU= 28.373
  15:18:00 STEP 1047 H= 261:45 +CPU= 28.491
  15:18:33 STEP 1048 H= 262:00 +CPU= 28.822
  15:19:05 STEP 1049 H= 262:15 +CPU= 27.892
  15:19:38 STEP 1050 H= 262:30 +CPU= 28.731
  15:20:12 STEP 1051 H= 262:45 +CPU= 29.328
  15:20:46 STEP 1052 H= 263:00 +CPU= 30.155
  15:21:20 STEP 1053 H= 263:15 +CPU= 29.885
  15:21:53 STEP 1054 H= 263:30 +CPU= 28.851
  15:22:26 STEP 1055 H= 263:45 +CPU= 28.720
  15:23:30 STEP 1056 H= 264:00 +CPU= 39.104
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_10.zip
Uploading the intermediate file: upload_file_10.zip
  15:24:04 STEP 1057 H= 264:15 +CPU= 28.353
  15:24:38 STEP 1058 H= 264:30 +CPU= 29.473
  15:25:11 STEP 1059 H= 264:45 +CPU= 29.063
  15:25:46 STEP 1060 H= 265:00 +CPU= 30.375
  15:26:19 STEP 1061 H= 265:15 +CPU= 29.662
  15:26:55 STEP 1062 H= 265:30 +CPU= 30.507
  15:27:29 STEP 1063 H= 265:45 +CPU= 29.204
  15:28:04 STEP 1064 H= 266:00 +CPU= 31.289
  15:28:38 STEP 1065 H= 266:15 +CPU= 29.955
  15:29:11 STEP 1066 H= 266:30 +CPU= 28.690
  15:29:45 STEP 1067 H= 266:45 +CPU= 29.435
  15:30:26 STEP 1068 H= 267:00 +CPU= 35.693
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001068
  15:30:58 STEP 1069 H= 267:15 +CPU= 28.513
  15:31:32 STEP 1070 H= 267:30 +CPU= 28.979
  15:32:05 STEP 1071 H= 267:45 +CPU= 29.127
  15:32:40 STEP 1072 H= 268:00 +CPU= 31.280
  15:33:15 STEP 1073 H= 268:15 +CPU= 30.498
  15:33:50 STEP 1074 H= 268:30 +CPU= 30.388
  15:34:24 STEP 1075 H= 268:45 +CPU= 29.899
  15:34:59 STEP 1076 H= 269:00 +CPU= 30.105
  15:35:33 STEP 1077 H= 269:15 +CPU= 28.734
  15:36:08 STEP 1078 H= 269:30 +CPU= 30.136
  15:36:43 STEP 1079 H= 269:45 +CPU= 29.704
  15:37:29 STEP 1080 H= 270:00 +CPU= 37.605
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001080
  15:38:03 STEP 1081 H= 270:15 +CPU= 30.602
  15:38:37 STEP 1082 H= 270:30 +CPU= 29.781
  15:39:14 STEP 1083 H= 270:45 +CPU= 32.353
  15:39:49 STEP 1084 H= 271:00 +CPU= 31.968
  15:40:23 STEP 1085 H= 271:15 +CPU= 29.242
  15:40:59 STEP 1086 H= 271:30 +CPU= 30.176
  15:41:30 STEP 1087 H= 271:45 +CPU= 27.357
  15:42:02 STEP 1088 H= 272:00 +CPU= 26.659
  15:42:25 STEP 1089 H= 272:15 +CPU= 20.999
  15:43:01 STEP 1090 H= 272:30 +CPU= 27.587
  15:43:38 STEP 1091 H= 272:45 +CPU= 27.505
  15:44:19 STEP 1092 H= 273:00 +CPU= 35.974
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001092
  15:44:52 STEP 1093 H= 273:15 +CPU= 30.077
  15:45:28 STEP 1094 H= 273:30 +CPU= 29.723
  15:46:03 STEP 1095 H= 273:45 +CPU= 30.508
  15:46:41 STEP 1096 H= 274:00 +CPU= 31.580
  15:47:15 STEP 1097 H= 274:15 +CPU= 28.675
  15:47:49 STEP 1098 H= 274:30 +CPU= 28.732
  15:48:22 STEP 1099 H= 274:45 +CPU= 28.914
  15:48:57 STEP 1100 H= 275:00 +CPU= 30.320
  15:49:36 STEP 1101 H= 275:15 +CPU= 32.303
  15:50:15 STEP 1102 H= 275:30 +CPU= 32.463
  15:50:51 STEP 1103 H= 275:45 +CPU= 31.285
  15:52:08 STEP 1104 H= 276:00 +CPU= 40.703
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001104
  15:52:44 STEP 1105 H= 276:15 +CPU= 31.383
  15:53:25 STEP 1106 H= 276:30 +CPU= 33.745
  15:54:03 STEP 1107 H= 276:45 +CPU= 32.610
  15:54:38 STEP 1108 H= 277:00 +CPU= 30.869
  15:55:10 STEP 1109 H= 277:15 +CPU= 29.374
  15:55:43 STEP 1110 H= 277:30 +CPU= 28.870
  15:56:16 STEP 1111 H= 277:45 +CPU= 28.502
  15:56:49 STEP 1112 H= 278:00 +CPU= 27.777
  15:57:25 STEP 1113 H= 278:15 +CPU= 29.305
  15:57:58 STEP 1114 H= 278:30 +CPU= 29.285
  15:58:33 STEP 1115 H= 278:45 +CPU= 29.873
  15:59:16 STEP 1116 H= 279:00 +CPU= 36.423
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001116
  15:59:51 STEP 1117 H= 279:15 +CPU= 29.468
  16:00:26 STEP 1118 H= 279:30 +CPU= 28.941
  16:01:00 STEP 1119 H= 279:45 +CPU= 28.872
  16:01:34 STEP 1120 H= 280:00 +CPU= 29.192
  16:02:06 STEP 1121 H= 280:15 +CPU= 28.641
  16:02:40 STEP 1122 H= 280:30 +CPU= 28.416
  16:03:13 STEP 1123 H= 280:45 +CPU= 28.386
  16:03:49 STEP 1124 H= 281:00 +CPU= 29.732
  16:04:26 STEP 1125 H= 281:15 +CPU= 29.689
  16:05:02 STEP 1126 H= 281:30 +CPU= 29.972
  16:05:36 STEP 1127 H= 281:45 +CPU= 29.550
  16:06:21 STEP 1128 H= 282:00 +CPU= 37.080
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001128
  16:06:59 STEP 1129 H= 282:15 +CPU= 30.406
  16:07:34 STEP 1130 H= 282:30 +CPU= 29.609
  16:08:08 STEP 1131 H= 282:45 +CPU= 28.194
  16:08:44 STEP 1132 H= 283:00 +CPU= 29.740
  16:09:19 STEP 1133 H= 283:15 +CPU= 29.587
  16:09:52 STEP 1134 H= 283:30 +CPU= 28.550
  16:10:25 STEP 1135 H= 283:45 +CPU= 27.460
  16:10:53 STEP 1136 H= 284:00 +CPU= 24.149
  16:11:16 STEP 1137 H= 284:15 +CPU= 21.822
  16:11:52 STEP 1138 H= 284:30 +CPU= 28.925
  16:12:29 STEP 1139 H= 284:45 +CPU= 30.161
  16:13:14 STEP 1140 H= 285:00 +CPU= 37.707
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001140
  16:13:51 STEP 1141 H= 285:15 +CPU= 30.175
  16:14:27 STEP 1142 H= 285:30 +CPU= 29.212
  16:15:00 STEP 1143 H= 285:45 +CPU= 29.526
  16:15:36 STEP 1144 H= 286:00 +CPU= 30.113
  16:16:10 STEP 1145 H= 286:15 +CPU= 29.604
  16:16:45 STEP 1146 H= 286:30 +CPU= 29.678
  16:17:20 STEP 1147 H= 286:45 +CPU= 29.135
  16:17:54 STEP 1148 H= 287:00 +CPU= 29.787
  16:18:29 STEP 1149 H= 287:15 +CPU= 29.376
  16:19:04 STEP 1150 H= 287:30 +CPU= 29.841
  16:19:40 STEP 1151 H= 287:45 +CPU= 30.105
  16:20:40 STEP 1152 H= 288:00 +CPU= 40.323
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_11.zip
Uploading the intermediate file: upload_file_11.zip
  16:21:14 STEP 1153 H= 288:15 +CPU= 27.916
  16:21:49 STEP 1154 H= 288:30 +CPU= 29.063
  16:22:23 STEP 1155 H= 288:45 +CPU= 28.462
  16:22:57 STEP 1156 H= 289:00 +CPU= 29.754
  16:23:30 STEP 1157 H= 289:15 +CPU= 28.615
  16:24:05 STEP 1158 H= 289:30 +CPU= 29.058
  16:24:39 STEP 1159 H= 289:45 +CPU= 28.590
  16:25:14 STEP 1160 H= 290:00 +CPU= 30.030
  16:25:50 STEP 1161 H= 290:15 +CPU= 30.139
  16:26:26 STEP 1162 H= 290:30 +CPU= 30.087
  16:27:02 STEP 1163 H= 290:45 +CPU= 30.035
  16:27:47 STEP 1164 H= 291:00 +CPU= 36.369
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001164
  16:28:20 STEP 1165 H= 291:15 +CPU= 27.824
  16:28:55 STEP 1166 H= 291:30 +CPU= 29.047
  16:29:27 STEP 1167 H= 291:45 +CPU= 28.092
  16:30:03 STEP 1168 H= 292:00 +CPU= 29.597
  16:30:39 STEP 1169 H= 292:15 +CPU= 29.447
  16:31:12 STEP 1170 H= 292:30 +CPU= 28.095
  16:31:48 STEP 1171 H= 292:45 +CPU= 29.696
  16:32:22 STEP 1172 H= 293:00 +CPU= 29.610
  16:32:57 STEP 1173 H= 293:15 +CPU= 29.879
  16:33:33 STEP 1174 H= 293:30 +CPU= 30.415
  16:34:09 STEP 1175 H= 293:45 +CPU= 29.904
  16:34:50 STEP 1176 H= 294:00 +CPU= 35.823
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001176
  16:35:23 STEP 1177 H= 294:15 +CPU= 28.960
  16:35:58 STEP 1178 H= 294:30 +CPU= 29.568
  16:36:32 STEP 1179 H= 294:45 +CPU= 28.723
  16:37:06 STEP 1180 H= 295:00 +CPU= 29.294
  16:37:40 STEP 1181 H= 295:15 +CPU= 28.825
  16:38:12 STEP 1182 H= 295:30 +CPU= 28.357
  16:38:45 STEP 1183 H= 295:45 +CPU= 28.593
  16:39:20 STEP 1184 H= 296:00 +CPU= 29.135
  16:39:47 STEP 1185 H= 296:15 +CPU= 24.985
  16:40:13 STEP 1186 H= 296:30 +CPU= 23.862
  16:40:46 STEP 1187 H= 296:45 +CPU= 26.699
  16:41:28 STEP 1188 H= 297:00 +CPU= 34.177
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001188
  16:42:00 STEP 1189 H= 297:15 +CPU= 27.965
  16:42:31 STEP 1190 H= 297:30 +CPU= 27.643
  16:43:02 STEP 1191 H= 297:45 +CPU= 27.675
  16:43:35 STEP 1192 H= 298:00 +CPU= 28.653
  16:44:08 STEP 1193 H= 298:15 +CPU= 29.571
  16:44:40 STEP 1194 H= 298:30 +CPU= 28.298
  16:45:13 STEP 1195 H= 298:45 +CPU= 29.061
  16:45:47 STEP 1196 H= 299:00 +CPU= 29.881
  16:46:21 STEP 1197 H= 299:15 +CPU= 29.307
  16:46:55 STEP 1198 H= 299:30 +CPU= 29.794
  16:47:30 STEP 1199 H= 299:45 +CPU= 29.251
  16:48:44 STEP 1200 H= 300:00 +CPU= 39.568
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001200
  16:49:18 STEP 1201 H= 300:15 +CPU= 29.183
  16:49:52 STEP 1202 H= 300:30 +CPU= 29.182
  16:50:26 STEP 1203 H= 300:45 +CPU= 28.748
  16:51:00 STEP 1204 H= 301:00 +CPU= 29.638
  16:51:34 STEP 1205 H= 301:15 +CPU= 29.241
  16:52:08 STEP 1206 H= 301:30 +CPU= 29.252
  16:52:43 STEP 1207 H= 301:45 +CPU= 30.261
  16:53:17 STEP 1208 H= 302:00 +CPU= 30.477
  16:53:50 STEP 1209 H= 302:15 +CPU= 29.101
  16:54:23 STEP 1210 H= 302:30 +CPU= 29.197
  16:54:56 STEP 1211 H= 302:45 +CPU= 28.994
  16:55:37 STEP 1212 H= 303:00 +CPU= 35.699
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001212
  16:56:10 STEP 1213 H= 303:15 +CPU= 28.765
  16:56:44 STEP 1214 H= 303:30 +CPU= 28.261
  16:57:18 STEP 1215 H= 303:45 +CPU= 29.054
  16:57:52 STEP 1216 H= 304:00 +CPU= 30.101
  16:58:25 STEP 1217 H= 304:15 +CPU= 29.054
  16:59:01 STEP 1218 H= 304:30 +CPU= 29.945
  16:59:36 STEP 1219 H= 304:45 +CPU= 30.342
  17:00:10 STEP 1220 H= 305:00 +CPU= 29.719
  17:00:41 STEP 1221 H= 305:15 +CPU= 28.367
  17:01:14 STEP 1222 H= 305:30 +CPU= 28.786
  17:01:48 STEP 1223 H= 305:45 +CPU= 28.921
  17:02:31 STEP 1224 H= 306:00 +CPU= 36.147
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001224
  17:03:04 STEP 1225 H= 306:15 +CPU= 28.927
  17:03:37 STEP 1226 H= 306:30 +CPU= 29.050
  17:04:11 STEP 1227 H= 306:45 +CPU= 28.646
  17:04:46 STEP 1228 H= 307:00 +CPU= 29.518
  17:05:21 STEP 1229 H= 307:15 +CPU= 29.569
  17:05:54 STEP 1230 H= 307:30 +CPU= 28.685
  17:06:27 STEP 1231 H= 307:45 +CPU= 28.723
  17:07:00 STEP 1232 H= 308:00 +CPU= 28.330
  17:07:33 STEP 1233 H= 308:15 +CPU= 27.880
  17:08:05 STEP 1234 H= 308:30 +CPU= 27.960
  17:08:34 STEP 1235 H= 308:45 +CPU= 25.586
  17:09:16 STEP 1236 H= 309:00 +CPU= 36.467
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001236
  17:09:50 STEP 1237 H= 309:15 +CPU= 29.480
  17:10:24 STEP 1238 H= 309:30 +CPU= 29.631
  17:10:58 STEP 1239 H= 309:45 +CPU= 29.670
  17:11:34 STEP 1240 H= 310:00 +CPU= 30.795
  17:12:10 STEP 1241 H= 310:15 +CPU= 30.388
  17:12:46 STEP 1242 H= 310:30 +CPU= 30.498
  17:13:20 STEP 1243 H= 310:45 +CPU= 29.423
  17:13:54 STEP 1244 H= 311:00 +CPU= 29.577
  17:14:29 STEP 1245 H= 311:15 +CPU= 29.116
  17:15:04 STEP 1246 H= 311:30 +CPU= 29.454
  17:15:38 STEP 1247 H= 311:45 +CPU= 29.156
  17:16:52 STEP 1248 H= 312:00 +CPU= 40.004
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_12.zip
Uploading the intermediate file: upload_file_12.zip
  17:17:30 STEP 1249 H= 312:15 +CPU= 29.456
  17:18:05 STEP 1250 H= 312:30 +CPU= 30.126
  17:18:40 STEP 1251 H= 312:45 +CPU= 30.040
  17:19:18 STEP 1252 H= 313:00 +CPU= 31.142
  17:19:53 STEP 1253 H= 313:15 +CPU= 29.512
  17:20:30 STEP 1254 H= 313:30 +CPU= 29.389
  17:21:03 STEP 1255 H= 313:45 +CPU= 28.963
  17:21:38 STEP 1256 H= 314:00 +CPU= 29.829
  17:22:12 STEP 1257 H= 314:15 +CPU= 29.160
  17:22:48 STEP 1258 H= 314:30 +CPU= 29.289
  17:23:25 STEP 1259 H= 314:45 +CPU= 29.682
  17:24:08 STEP 1260 H= 315:00 +CPU= 37.131
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001260
  17:24:42 STEP 1261 H= 315:15 +CPU= 29.746
  17:25:17 STEP 1262 H= 315:30 +CPU= 30.029
  17:25:54 STEP 1263 H= 315:45 +CPU= 30.470
  17:26:29 STEP 1264 H= 316:00 +CPU= 30.445
  17:27:02 STEP 1265 H= 316:15 +CPU= 29.361
  17:27:38 STEP 1266 H= 316:30 +CPU= 29.137
  17:28:13 STEP 1267 H= 316:45 +CPU= 29.441
  17:28:49 STEP 1268 H= 317:00 +CPU= 29.862
  17:29:24 STEP 1269 H= 317:15 +CPU= 29.083
  17:29:59 STEP 1270 H= 317:30 +CPU= 29.099
  17:30:37 STEP 1271 H= 317:45 +CPU= 29.830
  17:31:21 STEP 1272 H= 318:00 +CPU= 37.380
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001272
  17:31:57 STEP 1273 H= 318:15 +CPU= 30.082
  17:32:36 STEP 1274 H= 318:30 +CPU= 30.716
  17:33:13 STEP 1275 H= 318:45 +CPU= 30.294
  17:33:50 STEP 1276 H= 319:00 +CPU= 30.243
  17:34:27 STEP 1277 H= 319:15 +CPU= 29.734
  17:34:58 STEP 1278 H= 319:30 +CPU= 27.511
  17:35:33 STEP 1279 H= 319:45 +CPU= 28.794
  17:36:09 STEP 1280 H= 320:00 +CPU= 28.926
  17:36:42 STEP 1281 H= 320:15 +CPU= 28.109
  17:37:13 STEP 1282 H= 320:30 +CPU= 27.545
  17:37:43 STEP 1283 H= 320:45 +CPU= 26.323
  17:38:27 STEP 1284 H= 321:00 +CPU= 36.786
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001284
  17:39:04 STEP 1285 H= 321:15 +CPU= 29.559
  17:39:41 STEP 1286 H= 321:30 +CPU= 30.487
  17:40:17 STEP 1287 H= 321:45 +CPU= 29.414
  17:40:53 STEP 1288 H= 322:00 +CPU= 29.780
  17:41:27 STEP 1289 H= 322:15 +CPU= 29.288
  17:42:02 STEP 1290 H= 322:30 +CPU= 29.250
  17:42:36 STEP 1291 H= 322:45 +CPU= 29.112
  17:43:11 STEP 1292 H= 323:00 +CPU= 29.625
  17:43:49 STEP 1293 H= 323:15 +CPU= 30.180
  17:44:24 STEP 1294 H= 323:30 +CPU= 29.573
  17:45:01 STEP 1295 H= 323:45 +CPU= 30.358
  17:46:02 STEP 1296 H= 324:00 +CPU= 41.048
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001296
  17:46:38 STEP 1297 H= 324:15 +CPU= 30.075
  17:47:11 STEP 1298 H= 324:30 +CPU= 28.925
  17:47:44 STEP 1299 H= 324:45 +CPU= 28.932
  17:48:20 STEP 1300 H= 325:00 +CPU= 30.178
  17:48:55 STEP 1301 H= 325:15 +CPU= 29.629
  17:49:29 STEP 1302 H= 325:30 +CPU= 29.438
  17:50:05 STEP 1303 H= 325:45 +CPU= 29.739
  17:50:41 STEP 1304 H= 326:00 +CPU= 30.330
  17:51:16 STEP 1305 H= 326:15 +CPU= 29.810
  17:51:51 STEP 1306 H= 326:30 +CPU= 29.998
  17:52:28 STEP 1307 H= 326:45 +CPU= 30.230
  17:53:12 STEP 1308 H= 327:00 +CPU= 38.025
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001308
  17:53:47 STEP 1309 H= 327:15 +CPU= 29.315
  17:54:22 STEP 1310 H= 327:30 +CPU= 29.591
  17:54:57 STEP 1311 H= 327:45 +CPU= 29.164
  17:55:33 STEP 1312 H= 328:00 +CPU= 30.231
  17:56:08 STEP 1313 H= 328:15 +CPU= 29.484
  17:56:44 STEP 1314 H= 328:30 +CPU= 29.481
  17:57:19 STEP 1315 H= 328:45 +CPU= 29.578
  17:57:56 STEP 1316 H= 329:00 +CPU= 30.342
  17:58:33 STEP 1317 H= 329:15 +CPU= 30.264
  17:59:10 STEP 1318 H= 329:30 +CPU= 30.346
  17:59:47 STEP 1319 H= 329:45 +CPU= 30.807
  18:00:32 STEP 1320 H= 330:00 +CPU= 37.252
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001320
  18:01:09 STEP 1321 H= 330:15 +CPU= 29.587
  18:01:45 STEP 1322 H= 330:30 +CPU= 29.574
  18:02:21 STEP 1323 H= 330:45 +CPU= 29.640
  18:02:59 STEP 1324 H= 331:00 +CPU= 30.472
  18:03:36 STEP 1325 H= 331:15 +CPU= 29.753
  18:04:11 STEP 1326 H= 331:30 +CPU= 29.280
  18:04:48 STEP 1327 H= 331:45 +CPU= 29.840
  18:05:25 STEP 1328 H= 332:00 +CPU= 30.942
  18:06:00 STEP 1329 H= 332:15 +CPU= 29.449
  18:06:37 STEP 1330 H= 332:30 +CPU= 30.292
  18:07:08 STEP 1331 H= 332:45 +CPU= 27.775
  18:07:59 STEP 1332 H= 333:00 +CPU= 35.901
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001332
  18:08:35 STEP 1333 H= 333:15 +CPU= 29.537
  18:09:11 STEP 1334 H= 333:30 +CPU= 29.695
  18:09:47 STEP 1335 H= 333:45 +CPU= 29.787
  18:10:24 STEP 1336 H= 334:00 +CPU= 30.281
  18:11:00 STEP 1337 H= 334:15 +CPU= 29.939
  18:11:37 STEP 1338 H= 334:30 +CPU= 30.014
  18:12:13 STEP 1339 H= 334:45 +CPU= 30.234
  18:12:52 STEP 1340 H= 335:00 +CPU= 31.380
  18:13:30 STEP 1341 H= 335:15 +CPU= 30.832
  18:14:09 STEP 1342 H= 335:30 +CPU= 30.155
  18:14:48 STEP 1343 H= 335:45 +CPU= 29.915
  18:15:56 STEP 1344 H= 336:00 +CPU= 39.988
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_13.zip
Uploading the intermediate file: upload_file_13.zip
  18:16:32 STEP 1345 H= 336:15 +CPU= 29.097
  18:17:10 STEP 1346 H= 336:30 +CPU= 29.726
  18:17:46 STEP 1347 H= 336:45 +CPU= 29.670
  18:18:24 STEP 1348 H= 337:00 +CPU= 30.423
  18:18:59 STEP 1349 H= 337:15 +CPU= 29.978
  18:19:34 STEP 1350 H= 337:30 +CPU= 30.251
  18:20:09 STEP 1351 H= 337:45 +CPU= 30.337
  18:20:45 STEP 1352 H= 338:00 +CPU= 30.907
  18:21:20 STEP 1353 H= 338:15 +CPU= 29.568
  18:21:54 STEP 1354 H= 338:30 +CPU= 29.438
  18:22:29 STEP 1355 H= 338:45 +CPU= 29.335
  18:23:11 STEP 1356 H= 339:00 +CPU= 36.395
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001356
  18:23:44 STEP 1357 H= 339:15 +CPU= 29.236
  18:24:17 STEP 1358 H= 339:30 +CPU= 29.662
  18:24:51 STEP 1359 H= 339:45 +CPU= 29.726
  18:25:26 STEP 1360 H= 340:00 +CPU= 30.349
  18:25:59 STEP 1361 H= 340:15 +CPU= 30.023
  18:26:35 STEP 1362 H= 340:30 +CPU= 30.039
  18:27:12 STEP 1363 H= 340:45 +CPU= 30.206
  18:27:47 STEP 1364 H= 341:00 +CPU= 30.202
  18:28:21 STEP 1365 H= 341:15 +CPU= 29.506
  18:28:56 STEP 1366 H= 341:30 +CPU= 29.585
  18:29:31 STEP 1367 H= 341:45 +CPU= 29.252
  18:30:12 STEP 1368 H= 342:00 +CPU= 36.517
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001368
  18:30:45 STEP 1369 H= 342:15 +CPU= 29.339
  18:31:20 STEP 1370 H= 342:30 +CPU= 29.879
  18:31:56 STEP 1371 H= 342:45 +CPU= 29.954
  18:32:31 STEP 1372 H= 343:00 +CPU= 30.622
  18:33:05 STEP 1373 H= 343:15 +CPU= 29.292
  18:33:38 STEP 1374 H= 343:30 +CPU= 29.414
  18:34:13 STEP 1375 H= 343:45 +CPU= 29.106
  18:34:48 STEP 1376 H= 344:00 +CPU= 29.171
  18:35:21 STEP 1377 H= 344:15 +CPU= 28.997
  18:35:56 STEP 1378 H= 344:30 +CPU= 28.739
  18:36:28 STEP 1379 H= 344:45 +CPU= 28.150
  18:37:14 STEP 1380 H= 345:00 +CPU= 36.146
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001380
  18:37:51 STEP 1381 H= 345:15 +CPU= 29.500
  18:38:26 STEP 1382 H= 345:30 +CPU= 29.895
  18:39:01 STEP 1383 H= 345:45 +CPU= 30.298
  18:39:38 STEP 1384 H= 346:00 +CPU= 30.881
  18:40:14 STEP 1385 H= 346:15 +CPU= 30.500
  18:40:48 STEP 1386 H= 346:30 +CPU= 29.818
  18:41:23 STEP 1387 H= 346:45 +CPU= 29.799
  18:41:58 STEP 1388 H= 347:00 +CPU= 30.139
  18:42:31 STEP 1389 H= 347:15 +CPU= 29.397
  18:43:06 STEP 1390 H= 347:30 +CPU= 29.633
  18:43:39 STEP 1391 H= 347:45 +CPU= 29.663
  18:44:57 STEP 1392 H= 348:00 +CPU= 40.800
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001392
  18:45:33 STEP 1393 H= 348:15 +CPU= 30.145
  18:46:09 STEP 1394 H= 348:30 +CPU= 30.375
  18:46:46 STEP 1395 H= 348:45 +CPU= 30.335
  18:47:22 STEP 1396 H= 349:00 +CPU= 30.051
  18:47:56 STEP 1397 H= 349:15 +CPU= 29.281
  18:48:30 STEP 1398 H= 349:30 +CPU= 29.542
  18:49:04 STEP 1399 H= 349:45 +CPU= 29.457
  18:49:40 STEP 1400 H= 350:00 +CPU= 30.020
  18:50:14 STEP 1401 H= 350:15 +CPU= 29.381
  18:50:48 STEP 1402 H= 350:30 +CPU= 29.734
  18:51:23 STEP 1403 H= 350:45 +CPU= 29.825
  18:52:06 STEP 1404 H= 351:00 +CPU= 37.776
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001404
  18:52:42 STEP 1405 H= 351:15 +CPU= 30.381
  18:53:17 STEP 1406 H= 351:30 +CPU= 30.584
  18:53:52 STEP 1407 H= 351:45 +CPU= 29.907
  18:54:26 STEP 1408 H= 352:00 +CPU= 29.889
  18:55:00 STEP 1409 H= 352:15 +CPU= 29.334
  18:55:34 STEP 1410 H= 352:30 +CPU= 29.411
  18:56:09 STEP 1411 H= 352:45 +CPU= 29.642
  18:56:43 STEP 1412 H= 353:00 +CPU= 29.584
  18:57:18 STEP 1413 H= 353:15 +CPU= 29.405
  18:57:52 STEP 1414 H= 353:30 +CPU= 29.905
  18:58:27 STEP 1415 H= 353:45 +CPU= 29.988
  18:59:12 STEP 1416 H= 354:00 +CPU= 37.925
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001416
  18:59:48 STEP 1417 H= 354:15 +CPU= 30.664
  19:00:22 STEP 1418 H= 354:30 +CPU= 29.698
  19:00:56 STEP 1419 H= 354:45 +CPU= 29.491
  19:01:31 STEP 1420 H= 355:00 +CPU= 29.464
  19:02:02 STEP 1421 H= 355:15 +CPU= 28.129
  19:02:37 STEP 1422 H= 355:30 +CPU= 29.382
  19:03:11 STEP 1423 H= 355:45 +CPU= 29.371
  19:03:42 STEP 1424 H= 356:00 +CPU= 28.614
  19:04:15 STEP 1425 H= 356:15 +CPU= 29.311
  19:04:45 STEP 1426 H= 356:30 +CPU= 26.978
  19:05:17 STEP 1427 H= 356:45 +CPU= 28.847
  19:06:10 STEP 1428 H= 357:00 +CPU= 38.237
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001428
  19:06:47 STEP 1429 H= 357:15 +CPU= 31.668
  19:07:26 STEP 1430 H= 357:30 +CPU= 31.729
  19:08:03 STEP 1431 H= 357:45 +CPU= 31.772
  19:08:36 STEP 1432 H= 358:00 +CPU= 29.990
  19:09:09 STEP 1433 H= 358:15 +CPU= 29.254
  19:09:42 STEP 1434 H= 358:30 +CPU= 29.827
  19:10:19 STEP 1435 H= 358:45 +CPU= 31.678
  19:10:58 STEP 1436 H= 359:00 +CPU= 32.958
  19:11:35 STEP 1437 H= 359:15 +CPU= 32.252
  19:12:09 STEP 1438 H= 359:30 +CPU= 30.558
  19:12:42 STEP 1439 H= 359:45 +CPU= 30.114
  19:13:50 STEP 1440 H= 360:00 +CPU= 36.789
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.3652317990E-21      I       0.1790833901E-06      I
ICloud fractionI      0.0000000000E+00      I       0.2762732233E-01      I
I  Relat. Hum. I      0.1499706154E-01      I       0.7312114518E-01      I
I  PASS 01     I      0.0000000000E+00      I       0.1041900971E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    760804148 BYTES IN      7 RECORDS TRANSFERRED IN    0.0004 SECONDS******** Mbytes/s, TOTAL TIME=    0.0162( 2.5%)
 19:13:50 STEP 1440 H= 360:00 +CPU= 36.789
 END CNT3
 NSTEP =  1440 CNT0   000000000  27.209  27.209  32.116   0.847               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           46/  1069125708 WRITTEN        17160/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    303    0       46      1044068    17160 ************   142.4    36.0   106.4    36.5   172.1
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.81      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  625    5828.9    5828.9     17.99      0.00
   9 SPCM     - SPECTRAL COMP.                     576    1030.3    1030.3      2.93      0.00
  10 SCAN2M - PHYSICS                              577   15520.9   15520.9     44.23      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 48    4867.2    4867.2      1.15      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 25      36.6      36.6      0.00      0.00
  17 GRIDFPOS IN CNT4                               48      35.2      35.2      0.01      0.00
  27 RESTART FILES - WRITING                        12   19313.3   19313.3      1.14      0.00
  28 RESTART FILES - READING                         1   37057.7   37057.7      0.18      0.00
  29 SU4FPOS IN CNT4                                48       0.1       0.1      0.00      0.00
  30 DYNFPOS IN CNT4                                48    8292.4    8292.4      1.97      0.00
  31 POSDDH IN STEPO                                 3     230.6     230.6      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    577    7536.0    7536.0     21.47      0.00
  39 SU0YOMB                                         1     215.6     215.6      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    577      32.8      32.8      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              576     473.3     473.3      1.35      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              576     125.0     125.0      0.36      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 25       0.3       0.3      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1252     536.4     536.4      3.32      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1200     345.0     345.0      2.04      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1200     105.4     105.4      0.62      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1252     177.1     177.1      1.09      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      37.6      37.6      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1252      67.9      67.9      0.42      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1200      34.1      34.1      0.20      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1252      67.0      67.0      0.41      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1200      28.8      28.8      0.17      0.00
 400 GSTATS                                      67705       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 62941       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    20248.4 CPU TIME   17004.3 VECTOR TIME    17004.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       64.1
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/0/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/0/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/0/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/slots/0/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/0/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12286557/ICMUAh7zg+001440
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
19:17:31 (1659252): called boinc_finish(0)

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_15.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_16.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_17.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_18.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
<file_xfer_error>
  <file_name>oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0_r1205867204_19.zip</file_name>
  <error_code>-161 (not found)</error_code>
</file_xfer_error>
</message>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
08 Jun 2024 17:16:28 1507812 22438178 oifs_43r3_bl_a3ne_2016092300_20_1017_12286557_0 1,295,100 44,480 0.0343


©2024 climateprediction.net