climateprediction.net home page
Task 22285064

Task 22285064

Name oifs_43r3_ps_0289_2013050100_123_982_12198933_0
Workunit 12198933
Created 24 Dec 2022, 7:11:21 UTC
Sent 19 Jan 2023, 14:15:25 UTC
Report deadline 20 Mar 2023, 14:15:25 UTC
Received 23 Jan 2023, 12:14:01 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1410588
Run time 13 hours 23 min 44 sec
CPU time 13 hours 21 min 16 sec
Validate state Invalid
Credit 12,821.76
Device peak FLOPS 6.86 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.28 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,594.10 MB
Stderr
<core_client_version>7.16.14</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:8603:8603] [20230123:092715:1674462435:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:8603:8603] [20230123:092715:1674462435:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  09:27:39 STEP 2064 H=2064:00 +CPU= 21.560
  09:27:54 STEP 2065 H=2065:00 +CPU= 14.906
  09:28:09 STEP 2066 H=2066:00 +CPU= 15.391
  09:28:33 STEP 2067 H=2067:00 +CPU= 23.883
  09:28:49 STEP 2068 H=2068:00 +CPU= 15.331
  09:29:04 STEP 2069 H=2069:00 +CPU= 15.052
  09:29:28 STEP 2070 H=2070:00 +CPU= 24.409
  09:29:43 STEP 2071 H=2071:00 +CPU= 14.921
  09:29:59 STEP 2072 H=2072:00 +CPU= 15.562
  09:30:22 STEP 2073 H=2073:00 +CPU= 23.903
  09:30:38 STEP 2074 H=2074:00 +CPU= 15.538
  09:30:53 STEP 2075 H=2075:00 +CPU= 15.014
  09:31:18 STEP 2076 H=2076:00 +CPU= 25.056
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002076
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002076
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002076
  09:31:29 STEP 2077 H=2077:00 +CPU= 10.917
  09:31:40 STEP 2078 H=2078:00 +CPU= 11.227
  09:31:58 STEP 2079 H=2079:00 +CPU= 17.379
  09:32:09 STEP 2080 H=2080:00 +CPU= 11.089
  09:32:19 STEP 2081 H=2081:00 +CPU= 10.706
  09:32:37 STEP 2082 H=2082:00 +CPU= 17.813
  09:32:48 STEP 2083 H=2083:00 +CPU= 10.873
  09:32:59 STEP 2084 H=2084:00 +CPU= 11.013
  09:33:16 STEP 2085 H=2085:00 +CPU= 17.372
  09:33:28 STEP 2086 H=2086:00 +CPU= 11.154
  09:33:41 STEP 2087 H=2087:00 +CPU= 13.669
  09:34:09 STEP 2088 H=2088:00 +CPU= 27.804
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002088
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002088
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002088
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002064
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002064
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002064
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002076
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002076
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002076
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_86.zip
Uploading the intermediate file: upload_file_86.zip
  09:34:24 STEP 2089 H=2089:00 +CPU= 15.050
  09:34:40 STEP 2090 H=2090:00 +CPU= 15.537
  09:35:04 STEP 2091 H=2091:00 +CPU= 24.274
  09:35:19 STEP 2092 H=2092:00 +CPU= 15.417
  09:35:35 STEP 2093 H=2093:00 +CPU= 15.172
  09:35:59 STEP 2094 H=2094:00 +CPU= 24.555
  09:36:14 STEP 2095 H=2095:00 +CPU= 14.979
  09:36:30 STEP 2096 H=2096:00 +CPU= 15.474
  09:36:54 STEP 2097 H=2097:00 +CPU= 24.025
  09:37:09 STEP 2098 H=2098:00 +CPU= 15.478
  09:37:24 STEP 2099 H=2099:00 +CPU= 15.142
  09:37:51 STEP 2100 H=2100:00 +CPU= 26.879
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002100
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002100
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002100
  09:38:06 STEP 2101 H=2101:00 +CPU= 14.853
  09:38:22 STEP 2102 H=2102:00 +CPU= 15.535
  09:38:46 STEP 2103 H=2103:00 +CPU= 24.059
  09:39:01 STEP 2104 H=2104:00 +CPU= 15.419
  09:39:16 STEP 2105 H=2105:00 +CPU= 15.020
  09:39:40 STEP 2106 H=2106:00 +CPU= 24.366
  09:39:56 STEP 2107 H=2107:00 +CPU= 15.156
  09:40:11 STEP 2108 H=2108:00 +CPU= 15.419
  09:40:35 STEP 2109 H=2109:00 +CPU= 23.935
  09:40:50 STEP 2110 H=2110:00 +CPU= 15.371
  09:41:06 STEP 2111 H=2111:00 +CPU= 15.532
  09:41:34 STEP 2112 H=2112:00 +CPU= 27.956
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002112
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002112
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002112
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002088
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002088
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002088
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002100
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002100
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002100
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_87.zip
Uploading the intermediate file: upload_file_87.zip
  09:41:49 STEP 2113 H=2113:00 +CPU= 14.854
  09:42:04 STEP 2114 H=2114:00 +CPU= 15.512
  09:42:28 STEP 2115 H=2115:00 +CPU= 23.971
  09:42:44 STEP 2116 H=2116:00 +CPU= 15.525
  09:42:59 STEP 2117 H=2117:00 +CPU= 14.944
  09:43:23 STEP 2118 H=2118:00 +CPU= 24.399
  09:43:38 STEP 2119 H=2119:00 +CPU= 14.954
  09:43:54 STEP 2120 H=2120:00 +CPU= 15.643
  09:44:18 STEP 2121 H=2121:00 +CPU= 23.943
  09:44:33 STEP 2122 H=2122:00 +CPU= 15.497
  09:44:48 STEP 2123 H=2123:00 +CPU= 14.896
  09:45:15 STEP 2124 H=2124:00 +CPU= 26.603
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002124
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002124
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002124
  09:45:29 STEP 2125 H=2125:00 +CPU= 14.950
  09:45:45 STEP 2126 H=2126:00 +CPU= 15.527
  09:46:09 STEP 2127 H=2127:00 +CPU= 23.768
  09:46:24 STEP 2128 H=2128:00 +CPU= 15.449
  09:46:39 STEP 2129 H=2129:00 +CPU= 14.896
  09:47:04 STEP 2130 H=2130:00 +CPU= 24.441
  09:47:18 STEP 2131 H=2131:00 +CPU= 14.836
  09:47:34 STEP 2132 H=2132:00 +CPU= 15.738
  09:47:58 STEP 2133 H=2133:00 +CPU= 23.794
  09:48:13 STEP 2134 H=2134:00 +CPU= 15.479
  09:48:29 STEP 2135 H=2135:00 +CPU= 15.327
  09:48:57 STEP 2136 H=2136:00 +CPU= 27.992
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002136
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002136
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002136
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002112
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002112
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002112
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002124
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002124
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002124
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_88.zip
Uploading the intermediate file: upload_file_88.zip
  09:49:12 STEP 2137 H=2137:00 +CPU= 15.056
  09:49:27 STEP 2138 H=2138:00 +CPU= 15.368
  09:49:51 STEP 2139 H=2139:00 +CPU= 23.907
  09:50:07 STEP 2140 H=2140:00 +CPU= 15.497
  09:50:22 STEP 2141 H=2141:00 +CPU= 15.213
  09:50:46 STEP 2142 H=2142:00 +CPU= 24.611
  09:51:01 STEP 2143 H=2143:00 +CPU= 15.035
  09:51:17 STEP 2144 H=2144:00 +CPU= 15.609
  09:51:41 STEP 2145 H=2145:00 +CPU= 24.057
  09:51:57 STEP 2146 H=2146:00 +CPU= 15.473
  09:52:12 STEP 2147 H=2147:00 +CPU= 15.082
  09:52:39 STEP 2148 H=2148:00 +CPU= 27.087
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002148
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002148
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002148
  09:52:54 STEP 2149 H=2149:00 +CPU= 15.175
  09:53:09 STEP 2150 H=2150:00 +CPU= 15.367
  09:53:33 STEP 2151 H=2151:00 +CPU= 24.232
  09:53:49 STEP 2152 H=2152:00 +CPU= 15.485
  09:54:04 STEP 2153 H=2153:00 +CPU= 14.929
  09:54:28 STEP 2154 H=2154:00 +CPU= 24.360
  09:54:43 STEP 2155 H=2155:00 +CPU= 15.139
  09:54:59 STEP 2156 H=2156:00 +CPU= 15.473
  09:55:23 STEP 2157 H=2157:00 +CPU= 23.766
  09:55:38 STEP 2158 H=2158:00 +CPU= 15.472
  09:55:54 STEP 2159 H=2159:00 +CPU= 15.429
  09:56:21 STEP 2160 H=2160:00 +CPU= 27.946
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002160
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002160
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002160
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002136
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002136
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002136
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002148
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002148
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002148
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_89.zip
Uploading the intermediate file: upload_file_89.zip
  09:56:36 STEP 2161 H=2161:00 +CPU= 14.976
  09:56:52 STEP 2162 H=2162:00 +CPU= 15.470
  09:57:16 STEP 2163 H=2163:00 +CPU= 23.903
  09:57:31 STEP 2164 H=2164:00 +CPU= 15.416
  09:57:46 STEP 2165 H=2165:00 +CPU= 15.035
  09:58:11 STEP 2166 H=2166:00 +CPU= 24.358
  09:58:26 STEP 2167 H=2167:00 +CPU= 15.171
  09:58:41 STEP 2168 H=2168:00 +CPU= 15.515
  09:59:05 STEP 2169 H=2169:00 +CPU= 23.841
  09:59:21 STEP 2170 H=2170:00 +CPU= 15.540
  09:59:36 STEP 2171 H=2171:00 +CPU= 15.018
  10:00:02 STEP 2172 H=2172:00 +CPU= 26.677
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002172
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002172
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002172
  10:00:17 STEP 2173 H=2173:00 +CPU= 14.951
  10:00:33 STEP 2174 H=2174:00 +CPU= 15.506
  10:00:57 STEP 2175 H=2175:00 +CPU= 23.902
  10:01:12 STEP 2176 H=2176:00 +CPU= 15.531
  10:01:27 STEP 2177 H=2177:00 +CPU= 14.902
  10:01:52 STEP 2178 H=2178:00 +CPU= 24.492
  10:02:07 STEP 2179 H=2179:00 +CPU= 14.910
  10:02:22 STEP 2180 H=2180:00 +CPU= 15.667
  10:02:46 STEP 2181 H=2181:00 +CPU= 23.949
  10:03:02 STEP 2182 H=2182:00 +CPU= 15.479
  10:03:17 STEP 2183 H=2183:00 +CPU= 15.288
  10:03:45 STEP 2184 H=2184:00 +CPU= 28.096
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002184
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002184
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002184
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002160
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002160
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002160
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002172
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002172
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002172
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_90.zip
Uploading the intermediate file: upload_file_90.zip
  10:04:00 STEP 2185 H=2185:00 +CPU= 14.878
  10:04:15 STEP 2186 H=2186:00 +CPU= 15.347
  10:04:39 STEP 2187 H=2187:00 +CPU= 23.641
  10:04:55 STEP 2188 H=2188:00 +CPU= 15.595
  10:05:10 STEP 2189 H=2189:00 +CPU= 15.023
  10:05:34 STEP 2190 H=2190:00 +CPU= 24.361
  10:05:49 STEP 2191 H=2191:00 +CPU= 15.010
  10:06:04 STEP 2192 H=2192:00 +CPU= 15.459
  10:06:28 STEP 2193 H=2193:00 +CPU= 23.793
  10:06:44 STEP 2194 H=2194:00 +CPU= 15.572
  10:06:59 STEP 2195 H=2195:00 +CPU= 15.007
  10:07:26 STEP 2196 H=2196:00 +CPU= 26.844
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002196
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002196
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002196
  10:07:40 STEP 2197 H=2197:00 +CPU= 14.890
  10:07:56 STEP 2198 H=2198:00 +CPU= 15.380
  10:08:20 STEP 2199 H=2199:00 +CPU= 23.749
  10:08:35 STEP 2200 H=2200:00 +CPU= 15.439
  10:08:50 STEP 2201 H=2201:00 +CPU= 15.015
  10:09:14 STEP 2202 H=2202:00 +CPU= 24.391
  10:09:29 STEP 2203 H=2203:00 +CPU= 14.860
  10:09:45 STEP 2204 H=2204:00 +CPU= 15.332
  10:10:08 STEP 2205 H=2205:00 +CPU= 23.731
  10:10:24 STEP 2206 H=2206:00 +CPU= 15.981
  10:10:39 STEP 2207 H=2207:00 +CPU= 14.949
  10:11:04 STEP 2208 H=2208:00 +CPU= 24.823
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002208
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002208
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002208
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002184
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002184
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002184
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002196
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002196
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002196
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_91.zip
Uploading the intermediate file: upload_file_91.zip
  10:11:19 STEP 2209 H=2209:00 +CPU= 14.975
  10:11:35 STEP 2210 H=2210:00 +CPU= 15.459
  10:11:58 STEP 2211 H=2211:00 +CPU= 23.831
  10:12:14 STEP 2212 H=2212:00 +CPU= 15.556
  10:12:29 STEP 2213 H=2213:00 +CPU= 14.927
  10:12:53 STEP 2214 H=2214:00 +CPU= 24.206
  10:13:08 STEP 2215 H=2215:00 +CPU= 14.887
  10:13:24 STEP 2216 H=2216:00 +CPU= 15.531
  10:13:47 STEP 2217 H=2217:00 +CPU= 23.764
  10:14:03 STEP 2218 H=2218:00 +CPU= 15.409
  10:14:18 STEP 2219 H=2219:00 +CPU= 14.925
  10:14:44 STEP 2220 H=2220:00 +CPU= 26.650
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002220
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002220
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002220
  10:14:59 STEP 2221 H=2221:00 +CPU= 14.899
  10:15:15 STEP 2222 H=2222:00 +CPU= 15.477
  10:15:39 STEP 2223 H=2223:00 +CPU= 23.878
  10:15:54 STEP 2224 H=2224:00 +CPU= 15.761
  10:16:10 STEP 2225 H=2225:00 +CPU= 15.253
  10:16:35 STEP 2226 H=2226:00 +CPU= 25.222
  10:16:50 STEP 2227 H=2227:00 +CPU= 15.210
  10:17:06 STEP 2228 H=2228:00 +CPU= 16.248
  10:17:31 STEP 2229 H=2229:00 +CPU= 24.643
  10:17:47 STEP 2230 H=2230:00 +CPU= 16.114
  10:18:03 STEP 2231 H=2231:00 +CPU= 15.707
  10:18:31 STEP 2232 H=2232:00 +CPU= 28.363
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002232
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002232
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002232
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002208
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002208
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002208
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002220
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002220
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002220
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_92.zip
Uploading the intermediate file: upload_file_92.zip
  10:18:46 STEP 2233 H=2233:00 +CPU= 15.012
  10:19:02 STEP 2234 H=2234:00 +CPU= 15.460
  10:19:26 STEP 2235 H=2235:00 +CPU= 24.011
  10:19:42 STEP 2236 H=2236:00 +CPU= 16.065
  10:19:57 STEP 2237 H=2237:00 +CPU= 15.124
  10:20:22 STEP 2238 H=2238:00 +CPU= 24.808
  10:20:37 STEP 2239 H=2239:00 +CPU= 15.249
  10:20:52 STEP 2240 H=2240:00 +CPU= 15.611
  10:21:16 STEP 2241 H=2241:00 +CPU= 23.915
  10:21:32 STEP 2242 H=2242:00 +CPU= 15.579
  10:21:47 STEP 2243 H=2243:00 +CPU= 15.013
  10:22:14 STEP 2244 H=2244:00 +CPU= 26.799
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002244
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002244
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002244
  10:22:29 STEP 2245 H=2245:00 +CPU= 15.069
  10:22:44 STEP 2246 H=2246:00 +CPU= 15.511
  10:23:08 STEP 2247 H=2247:00 +CPU= 24.083
  10:23:24 STEP 2248 H=2248:00 +CPU= 15.755
  10:23:39 STEP 2249 H=2249:00 +CPU= 15.010
  10:24:04 STEP 2250 H=2250:00 +CPU= 24.422
  10:24:19 STEP 2251 H=2251:00 +CPU= 15.183
  10:24:34 STEP 2252 H=2252:00 +CPU= 15.436
  10:24:58 STEP 2253 H=2253:00 +CPU= 24.082
  10:25:14 STEP 2254 H=2254:00 +CPU= 15.905
  10:25:30 STEP 2255 H=2255:00 +CPU= 15.713
  10:25:58 STEP 2256 H=2256:00 +CPU= 28.328
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002256
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002256
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002256
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002232
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002232
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002232
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002244
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002244
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002244
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_93.zip
Uploading the intermediate file: upload_file_93.zip
  10:26:13 STEP 2257 H=2257:00 +CPU= 14.892
  10:26:29 STEP 2258 H=2258:00 +CPU= 15.522
  10:26:53 STEP 2259 H=2259:00 +CPU= 24.187
  10:27:08 STEP 2260 H=2260:00 +CPU= 15.674
  10:27:24 STEP 2261 H=2261:00 +CPU= 15.120
  10:27:48 STEP 2262 H=2262:00 +CPU= 24.668
  10:28:03 STEP 2263 H=2263:00 +CPU= 15.093
  10:28:19 STEP 2264 H=2264:00 +CPU= 15.480
  10:28:43 STEP 2265 H=2265:00 +CPU= 23.998
  10:28:59 STEP 2266 H=2266:00 +CPU= 15.769
  10:29:14 STEP 2267 H=2267:00 +CPU= 15.172
  10:29:41 STEP 2268 H=2268:00 +CPU= 27.194
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002268
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002268
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002268
  10:29:57 STEP 2269 H=2269:00 +CPU= 15.850
  10:30:11 STEP 2270 H=2270:00 +CPU= 13.931
  10:30:34 STEP 2271 H=2271:00 +CPU= 23.085
  10:30:49 STEP 2272 H=2272:00 +CPU= 15.223
  10:31:01 STEP 2273 H=2273:00 +CPU= 11.533
  10:31:20 STEP 2274 H=2274:00 +CPU= 18.761
  10:31:31 STEP 2275 H=2275:00 +CPU= 11.304
  10:31:42 STEP 2276 H=2276:00 +CPU= 11.578
  10:32:01 STEP 2277 H=2277:00 +CPU= 18.135
  10:32:12 STEP 2278 H=2278:00 +CPU= 11.497
  10:32:24 STEP 2279 H=2279:00 +CPU= 11.487
  10:32:45 STEP 2280 H=2280:00 +CPU= 21.237
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002280
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002280
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002280
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002256
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002256
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002256
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002268
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002268
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002268
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_94.zip
Uploading the intermediate file: upload_file_94.zip
  10:32:56 STEP 2281 H=2281:00 +CPU= 11.449
  10:33:08 STEP 2282 H=2282:00 +CPU= 11.804
  10:33:32 STEP 2283 H=2283:00 +CPU= 24.082
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:12:24 STEP 2284 H=2284:00 +CPU= 15.883
  11:12:38 STEP 2285 H=2285:00 +CPU= 14.517
  11:13:03 STEP 2286 H=2286:00 +CPU= 25.052
  11:13:17 STEP 2287 H=2287:00 +CPU= 13.656
  11:13:31 STEP 2288 H=2288:00 +CPU= 14.552
  11:13:55 STEP 2289 H=2289:00 +CPU= 23.260
  11:14:11 STEP 2290 H=2290:00 +CPU= 15.888
  11:14:26 STEP 2291 H=2291:00 +CPU= 15.685
  11:14:55 STEP 2292 H=2292:00 +CPU= 28.197
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002292
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002292
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002292
  11:15:10 STEP 2293 H=2293:00 +CPU= 15.831
  11:15:27 STEP 2294 H=2294:00 +CPU= 16.732
  11:15:53 STEP 2295 H=2295:00 +CPU= 25.445
  11:16:09 STEP 2296 H=2296:00 +CPU= 16.471
  11:16:25 STEP 2297 H=2297:00 +CPU= 15.920
  11:16:51 STEP 2298 H=2298:00 +CPU= 25.910
  11:17:06 STEP 2299 H=2299:00 +CPU= 15.529
  11:17:21 STEP 2300 H=2300:00 +CPU= 14.671
  11:17:39 STEP 2301 H=2301:00 +CPU= 18.339
  11:17:51 STEP 2302 H=2302:00 +CPU= 11.734
  11:18:03 STEP 2303 H=2303:00 +CPU= 11.590
  11:18:24 STEP 2304 H=2304:00 +CPU= 21.395
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002304
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002304
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002304
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002280
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002280
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002280
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002292
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002292
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002292
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_95.zip
Uploading the intermediate file: upload_file_95.zip
  11:18:35 STEP 2305 H=2305:00 +CPU= 11.168
  11:18:47 STEP 2306 H=2306:00 +CPU= 11.350
Uploading trickle at timestep: 8294400
  11:19:05 STEP 2307 H=2307:00 +CPU= 17.951
  11:19:17 STEP 2308 H=2308:00 +CPU= 12.111
  11:19:28 STEP 2309 H=2309:00 +CPU= 11.380
  11:19:47 STEP 2310 H=2310:00 +CPU= 18.526
  11:19:58 STEP 2311 H=2311:00 +CPU= 11.062
  11:20:10 STEP 2312 H=2312:00 +CPU= 11.884
  11:20:27 STEP 2313 H=2313:00 +CPU= 17.944
  11:20:39 STEP 2314 H=2314:00 +CPU= 11.520
  11:20:50 STEP 2315 H=2315:00 +CPU= 11.234
  11:21:10 STEP 2316 H=2316:00 +CPU= 20.006
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002316
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002316
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002316
  11:21:22 STEP 2317 H=2317:00 +CPU= 11.391
  11:21:36 STEP 2318 H=2318:00 +CPU= 14.404
  11:22:01 STEP 2319 H=2319:00 +CPU= 25.195
  11:22:17 STEP 2320 H=2320:00 +CPU= 16.047
  11:22:33 STEP 2321 H=2321:00 +CPU= 15.814
  11:22:59 STEP 2322 H=2322:00 +CPU= 25.934
  11:23:15 STEP 2323 H=2323:00 +CPU= 16.243
  11:23:32 STEP 2324 H=2324:00 +CPU= 16.367
  11:23:57 STEP 2325 H=2325:00 +CPU= 25.550
  11:24:14 STEP 2326 H=2326:00 +CPU= 16.712
  11:24:30 STEP 2327 H=2327:00 +CPU= 16.076
  11:24:59 STEP 2328 H=2328:00 +CPU= 28.650
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002328
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002328
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002328
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002304
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002304
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002304
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002316
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002316
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002316
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_96.zip
Uploading the intermediate file: upload_file_96.zip
  11:25:14 STEP 2329 H=2329:00 +CPU= 14.979
  11:25:30 STEP 2330 H=2330:00 +CPU= 15.921
  11:25:54 STEP 2331 H=2331:00 +CPU= 23.945
  11:26:10 STEP 2332 H=2332:00 +CPU= 16.139
  11:26:25 STEP 2333 H=2333:00 +CPU= 15.231
  11:26:44 STEP 2334 H=2334:00 +CPU= 19.056
  11:26:59 STEP 2335 H=2335:00 +CPU= 14.401
  11:27:15 STEP 2336 H=2336:00 +CPU= 16.338
  11:27:41 STEP 2337 H=2337:00 +CPU= 25.247
  11:27:57 STEP 2338 H=2338:00 +CPU= 16.544
  11:28:13 STEP 2339 H=2339:00 +CPU= 15.819
  11:28:41 STEP 2340 H=2340:00 +CPU= 28.618
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002340
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002340
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002340
  11:28:58 STEP 2341 H=2341:00 +CPU= 16.032
  11:29:14 STEP 2342 H=2342:00 +CPU= 16.428
  11:29:38 STEP 2343 H=2343:00 +CPU= 24.498
  11:29:50 STEP 2344 H=2344:00 +CPU= 11.386
  11:30:01 STEP 2345 H=2345:00 +CPU= 11.436
  11:30:20 STEP 2346 H=2346:00 +CPU= 18.413
  11:30:31 STEP 2347 H=2347:00 +CPU= 11.267
  11:30:43 STEP 2348 H=2348:00 +CPU= 12.508
  11:31:03 STEP 2349 H=2349:00 +CPU= 19.121
  11:31:14 STEP 2350 H=2350:00 +CPU= 11.506
  11:31:26 STEP 2351 H=2351:00 +CPU= 11.495
  11:31:48 STEP 2352 H=2352:00 +CPU= 22.290
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002352
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002352
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002352
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002328
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002328
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002328
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002340
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002340
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002340
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_97.zip
Uploading the intermediate file: upload_file_97.zip
  11:31:59 STEP 2353 H=2353:00 +CPU= 11.370
  11:32:12 STEP 2354 H=2354:00 +CPU= 12.562
  11:32:35 STEP 2355 H=2355:00 +CPU= 22.711
  11:32:51 STEP 2356 H=2356:00 +CPU= 16.636
  11:33:07 STEP 2357 H=2357:00 +CPU= 15.765
  11:33:33 STEP 2358 H=2358:00 +CPU= 26.217
  11:33:47 STEP 2359 H=2359:00 +CPU= 13.709
  11:34:03 STEP 2360 H=2360:00 +CPU= 16.493
  11:34:26 STEP 2361 H=2361:00 +CPU= 22.498
  11:34:38 STEP 2362 H=2362:00 +CPU= 11.779
  11:34:50 STEP 2363 H=2363:00 +CPU= 12.204
  11:35:11 STEP 2364 H=2364:00 +CPU= 20.785
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002364
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002364
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002364
  11:35:22 STEP 2365 H=2365:00 +CPU= 11.588
  11:35:36 STEP 2366 H=2366:00 +CPU= 13.263
  11:36:00 STEP 2367 H=2367:00 +CPU= 24.366
  11:36:16 STEP 2368 H=2368:00 +CPU= 15.870
  11:36:31 STEP 2369 H=2369:00 +CPU= 15.258
  11:36:52 STEP 2370 H=2370:00 +CPU= 20.546
  11:37:08 STEP 2371 H=2371:00 +CPU= 15.824
  11:37:24 STEP 2372 H=2372:00 +CPU= 16.435
  11:37:49 STEP 2373 H=2373:00 +CPU= 25.152
  11:38:05 STEP 2374 H=2374:00 +CPU= 16.287
  11:38:22 STEP 2375 H=2375:00 +CPU= 16.292
  11:38:50 STEP 2376 H=2376:00 +CPU= 28.732
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002376
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002376
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002376
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002352
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002352
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002352
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002364
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002364
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002364
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_98.zip
Uploading the intermediate file: upload_file_98.zip
  11:39:04 STEP 2377 H=2377:00 +CPU= 13.894
  11:39:21 STEP 2378 H=2378:00 +CPU= 16.936
  11:39:43 STEP 2379 H=2379:00 +CPU= 21.856
  11:39:55 STEP 2380 H=2380:00 +CPU= 12.142
  11:40:07 STEP 2381 H=2381:00 +CPU= 11.842
  11:40:26 STEP 2382 H=2382:00 +CPU= 19.165
  11:40:40 STEP 2383 H=2383:00 +CPU= 13.480
  11:40:55 STEP 2384 H=2384:00 +CPU= 15.173
  11:41:21 STEP 2385 H=2385:00 +CPU= 25.704
  11:41:37 STEP 2386 H=2386:00 +CPU= 16.383
  11:41:53 STEP 2387 H=2387:00 +CPU= 15.931
  11:42:21 STEP 2388 H=2388:00 +CPU= 28.255
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002388
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002388
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002388
  11:42:37 STEP 2389 H=2389:00 +CPU= 15.758
  11:42:53 STEP 2390 H=2390:00 +CPU= 16.034
  11:43:18 STEP 2391 H=2391:00 +CPU= 24.684
  11:43:34 STEP 2392 H=2392:00 +CPU= 15.974
  11:43:49 STEP 2393 H=2393:00 +CPU= 15.687
  11:44:15 STEP 2394 H=2394:00 +CPU= 25.977
  11:44:31 STEP 2395 H=2395:00 +CPU= 15.887
  11:44:47 STEP 2396 H=2396:00 +CPU= 15.920
  11:45:13 STEP 2397 H=2397:00 +CPU= 25.438
  11:45:29 STEP 2398 H=2398:00 +CPU= 16.724
  11:45:46 STEP 2399 H=2399:00 +CPU= 16.555
  11:46:16 STEP 2400 H=2400:00 +CPU= 29.889
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002400
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002400
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002400
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002376
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002376
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002376
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002388
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002388
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002388
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_99.zip
Uploading the intermediate file: upload_file_99.zip
  11:46:32 STEP 2401 H=2401:00 +CPU= 15.662
  11:46:47 STEP 2402 H=2402:00 +CPU= 15.881
  11:47:11 STEP 2403 H=2403:00 +CPU= 23.850
  11:47:23 STEP 2404 H=2404:00 +CPU= 11.486
  11:47:37 STEP 2405 H=2405:00 +CPU= 13.734
  11:48:02 STEP 2406 H=2406:00 +CPU= 25.413
  11:48:18 STEP 2407 H=2407:00 +CPU= 16.045
  11:48:34 STEP 2408 H=2408:00 +CPU= 15.882
  11:48:59 STEP 2409 H=2409:00 +CPU= 24.684
  11:49:15 STEP 2410 H=2410:00 +CPU= 15.988
  11:49:30 STEP 2411 H=2411:00 +CPU= 15.785
  11:49:58 STEP 2412 H=2412:00 +CPU= 27.653
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002412
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002412
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002412
  11:50:13 STEP 2413 H=2413:00 +CPU= 15.492
  11:50:29 STEP 2414 H=2414:00 +CPU= 15.962
  11:50:48 STEP 2415 H=2415:00 +CPU= 18.788
  11:51:03 STEP 2416 H=2416:00 +CPU= 14.469
  11:51:15 STEP 2417 H=2417:00 +CPU= 12.232
  11:51:41 STEP 2418 H=2418:00 +CPU= 25.607
  11:51:57 STEP 2419 H=2419:00 +CPU= 16.372
  11:52:14 STEP 2420 H=2420:00 +CPU= 17.254
  11:52:40 STEP 2421 H=2421:00 +CPU= 26.109
  11:52:57 STEP 2422 H=2422:00 +CPU= 16.757
  11:53:14 STEP 2423 H=2423:00 +CPU= 17.004
  11:53:44 STEP 2424 H=2424:00 +CPU= 30.145
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002424
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002424
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002424
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002400
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002400
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002400
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002412
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002412
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002412
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_100.zip
Uploading the intermediate file: upload_file_100.zip
  11:54:01 STEP 2425 H=2425:00 +CPU= 16.509
  11:54:18 STEP 2426 H=2426:00 +CPU= 16.765
  11:54:43 STEP 2427 H=2427:00 +CPU= 25.760
  11:55:01 STEP 2428 H=2428:00 +CPU= 17.086
  11:55:17 STEP 2429 H=2429:00 +CPU= 16.524
  11:55:44 STEP 2430 H=2430:00 +CPU= 26.764
  11:56:00 STEP 2431 H=2431:00 +CPU= 16.410
  11:56:17 STEP 2432 H=2432:00 +CPU= 17.105
  11:56:43 STEP 2433 H=2433:00 +CPU= 25.782
  11:56:56 STEP 2434 H=2434:00 +CPU= 13.348
  11:57:09 STEP 2435 H=2435:00 +CPU= 12.630
  11:57:30 STEP 2436 H=2436:00 +CPU= 21.152
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002436
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002436
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002436
  11:57:43 STEP 2437 H=2437:00 +CPU= 12.421
  11:57:59 STEP 2438 H=2438:00 +CPU= 16.717
  11:58:25 STEP 2439 H=2439:00 +CPU= 25.841
  11:58:42 STEP 2440 H=2440:00 +CPU= 16.656
  11:58:59 STEP 2441 H=2441:00 +CPU= 17.217
  11:59:22 STEP 2442 H=2442:00 +CPU= 22.406
  11:59:34 STEP 2443 H=2443:00 +CPU= 11.963
  11:59:46 STEP 2444 H=2444:00 +CPU= 12.107
  12:00:05 STEP 2445 H=2445:00 +CPU= 19.707
  12:00:21 STEP 2446 H=2446:00 +CPU= 15.571
  12:00:33 STEP 2447 H=2447:00 +CPU= 12.113
  12:00:55 STEP 2448 H=2448:00 +CPU= 22.351
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002448
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002448
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002448
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002424
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002424
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002424
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002436
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002436
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002436
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_101.zip
Uploading the intermediate file: upload_file_101.zip
  12:01:08 STEP 2449 H=2449:00 +CPU= 12.350
  12:01:25 STEP 2450 H=2450:00 +CPU= 16.828
  12:01:48 STEP 2451 H=2451:00 +CPU= 23.115
  12:02:00 STEP 2452 H=2452:00 +CPU= 11.895
  12:02:11 STEP 2453 H=2453:00 +CPU= 11.824
  12:02:34 STEP 2454 H=2454:00 +CPU= 22.546
  12:02:50 STEP 2455 H=2455:00 +CPU= 16.303
  12:03:07 STEP 2456 H=2456:00 +CPU= 16.749
  12:03:33 STEP 2457 H=2457:00 +CPU= 25.459
  12:03:49 STEP 2458 H=2458:00 +CPU= 16.866
  12:04:04 STEP 2459 H=2459:00 +CPU= 14.529
  12:04:25 STEP 2460 H=2460:00 +CPU= 20.812
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002460
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002460
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002460
  12:04:38 STEP 2461 H=2461:00 +CPU= 13.328
  12:04:55 STEP 2462 H=2462:00 +CPU= 17.182
  12:05:22 STEP 2463 H=2463:00 +CPU= 26.155
  12:05:38 STEP 2464 H=2464:00 +CPU= 16.329
  12:05:55 STEP 2465 H=2465:00 +CPU= 16.839
  12:06:22 STEP 2466 H=2466:00 +CPU= 27.211
  12:06:39 STEP 2467 H=2467:00 +CPU= 16.487
  12:06:56 STEP 2468 H=2468:00 +CPU= 17.004
  12:07:22 STEP 2469 H=2469:00 +CPU= 26.351
  12:07:39 STEP 2470 H=2470:00 +CPU= 17.529
  12:07:57 STEP 2471 H=2471:00 +CPU= 17.407
  12:08:27 STEP 2472 H=2472:00 +CPU= 30.072
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002472
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002472
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002472
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002448
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002448
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002448
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002460
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002460
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002460
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_102.zip
Uploading the intermediate file: upload_file_102.zip
  12:08:43 STEP 2473 H=2473:00 +CPU= 16.452
  12:08:59 STEP 2474 H=2474:00 +CPU= 15.577
  12:09:18 STEP 2475 H=2475:00 +CPU= 18.937
  12:09:30 STEP 2476 H=2476:00 +CPU= 12.168
  12:09:42 STEP 2477 H=2477:00 +CPU= 11.935
  12:10:02 STEP 2478 H=2478:00 +CPU= 19.766
  12:10:16 STEP 2479 H=2479:00 +CPU= 14.418
  12:10:34 STEP 2480 H=2480:00 +CPU= 17.319
  12:11:00 STEP 2481 H=2481:00 +CPU= 25.997
  12:11:13 STEP 2482 H=2482:00 +CPU= 13.709
  12:11:25 STEP 2483 H=2483:00 +CPU= 11.882
  12:11:46 STEP 2484 H=2484:00 +CPU= 21.004
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002484
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002484
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002484
  12:11:58 STEP 2485 H=2485:00 +CPU= 11.728
  12:12:15 STEP 2486 H=2486:00 +CPU= 16.769
  12:12:40 STEP 2487 H=2487:00 +CPU= 25.501
  12:12:57 STEP 2488 H=2488:00 +CPU= 16.653
  12:13:13 STEP 2489 H=2489:00 +CPU= 16.180
  12:13:37 STEP 2490 H=2490:00 +CPU= 23.656
  12:13:48 STEP 2491 H=2491:00 +CPU= 11.752
  12:14:01 STEP 2492 H=2492:00 +CPU= 12.181
  12:14:19 STEP 2493 H=2493:00 +CPU= 18.647
  12:14:31 STEP 2494 H=2494:00 +CPU= 11.730
  12:14:43 STEP 2495 H=2495:00 +CPU= 12.112
  12:15:05 STEP 2496 H=2496:00 +CPU= 22.151
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002496
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002496
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002496
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002472
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002472
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002472
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002484
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002484
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002484
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_103.zip
Uploading the intermediate file: upload_file_103.zip
  12:15:17 STEP 2497 H=2497:00 +CPU= 11.261
  12:15:29 STEP 2498 H=2498:00 +CPU= 12.128
  12:15:47 STEP 2499 H=2499:00 +CPU= 18.528
  12:15:59 STEP 2500 H=2500:00 +CPU= 12.143
  12:16:11 STEP 2501 H=2501:00 +CPU= 11.685
  12:16:34 STEP 2502 H=2502:00 +CPU= 22.675
  12:16:50 STEP 2503 H=2503:00 +CPU= 16.659
  12:17:05 STEP 2504 H=2504:00 +CPU= 14.720
  12:17:24 STEP 2505 H=2505:00 +CPU= 18.716
  12:17:38 STEP 2506 H=2506:00 +CPU= 13.850
  12:17:52 STEP 2507 H=2507:00 +CPU= 14.003
  12:18:13 STEP 2508 H=2508:00 +CPU= 21.445
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002508
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002508
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002508
  12:18:30 STEP 2509 H=2509:00 +CPU= 17.045
  12:18:47 STEP 2510 H=2510:00 +CPU= 16.955
  12:19:13 STEP 2511 H=2511:00 +CPU= 26.286
  12:19:30 STEP 2512 H=2512:00 +CPU= 17.044
  12:19:47 STEP 2513 H=2513:00 +CPU= 16.418
  12:20:14 STEP 2514 H=2514:00 +CPU= 27.366
  12:20:27 STEP 2515 H=2515:00 +CPU= 12.555
  12:20:39 STEP 2516 H=2516:00 +CPU= 12.563
  12:20:58 STEP 2517 H=2517:00 +CPU= 18.943
  12:21:13 STEP 2518 H=2518:00 +CPU= 14.894
  12:21:31 STEP 2519 H=2519:00 +CPU= 17.325
  12:22:02 STEP 2520 H=2520:00 +CPU= 31.190
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002520
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002520
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002520
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002496
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002496
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002496
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002508
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002508
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002508
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_104.zip
Uploading the intermediate file: upload_file_104.zip
  12:22:19 STEP 2521 H=2521:00 +CPU= 16.858
  12:22:35 STEP 2522 H=2522:00 +CPU= 15.763
  12:22:53 STEP 2523 H=2523:00 +CPU= 18.484
  12:23:05 STEP 2524 H=2524:00 +CPU= 11.799
  12:23:17 STEP 2525 H=2525:00 +CPU= 11.627
  12:23:36 STEP 2526 H=2526:00 +CPU= 18.960
  12:23:47 STEP 2527 H=2527:00 +CPU= 11.485
  12:23:59 STEP 2528 H=2528:00 +CPU= 12.070
  12:24:19 STEP 2529 H=2529:00 +CPU= 19.416
  12:24:31 STEP 2530 H=2530:00 +CPU= 12.268
  12:24:43 STEP 2531 H=2531:00 +CPU= 12.098
  12:25:03 STEP 2532 H=2532:00 +CPU= 20.422
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002532
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002532
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002532
  12:25:15 STEP 2533 H=2533:00 +CPU= 11.773
  12:25:30 STEP 2534 H=2534:00 +CPU= 14.961
  12:25:56 STEP 2535 H=2535:00 +CPU= 25.599
  12:26:09 STEP 2536 H=2536:00 +CPU= 13.045
  12:26:22 STEP 2537 H=2537:00 +CPU= 13.050
  12:26:41 STEP 2538 H=2538:00 +CPU= 19.371
  12:26:53 STEP 2539 H=2539:00 +CPU= 11.652
  12:27:05 STEP 2540 H=2540:00 +CPU= 12.260
  12:27:24 STEP 2541 H=2541:00 +CPU= 19.288
  12:27:39 STEP 2542 H=2542:00 +CPU= 14.101
  12:27:52 STEP 2543 H=2543:00 +CPU= 13.294
  12:28:16 STEP 2544 H=2544:00 +CPU= 24.505
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002544
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002544
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002544
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002520
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002520
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002520
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002532
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002532
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002532
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_105.zip
Uploading the intermediate file: upload_file_105.zip
  12:28:28 STEP 2545 H=2545:00 +CPU= 11.872
Uploading trickle at timestep: 9158400
  12:28:41 STEP 2546 H=2546:00 +CPU= 12.691
  12:29:08 STEP 2547 H=2547:00 +CPU= 26.610
  12:29:25 STEP 2548 H=2548:00 +CPU= 16.931
  12:29:41 STEP 2549 H=2549:00 +CPU= 16.531
  12:30:08 STEP 2550 H=2550:00 +CPU= 26.858
  12:30:25 STEP 2551 H=2551:00 +CPU= 16.615
  12:30:41 STEP 2552 H=2552:00 +CPU= 16.907
  12:31:09 STEP 2553 H=2553:00 +CPU= 27.669
  12:31:28 STEP 2554 H=2554:00 +CPU= 18.643
  12:31:45 STEP 2555 H=2555:00 +CPU= 17.231
  12:32:15 STEP 2556 H=2556:00 +CPU= 29.719
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002556
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002556
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002556
  12:32:31 STEP 2557 H=2557:00 +CPU= 16.216
  12:32:48 STEP 2558 H=2558:00 +CPU= 16.988
  12:33:14 STEP 2559 H=2559:00 +CPU= 26.085
  12:33:32 STEP 2560 H=2560:00 +CPU= 17.563
  12:33:48 STEP 2561 H=2561:00 +CPU= 16.796
  12:34:15 STEP 2562 H=2562:00 +CPU= 26.385
  12:34:31 STEP 2563 H=2563:00 +CPU= 16.440
  12:34:49 STEP 2564 H=2564:00 +CPU= 17.338
  12:35:15 STEP 2565 H=2565:00 +CPU= 25.961
  12:35:31 STEP 2566 H=2566:00 +CPU= 16.804
  12:35:48 STEP 2567 H=2567:00 +CPU= 16.965
  12:36:18 STEP 2568 H=2568:00 +CPU= 29.341
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002568
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002568
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002568
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002544
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002544
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002544
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002556
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002556
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002556
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_106.zip
Uploading the intermediate file: upload_file_106.zip
  12:36:34 STEP 2569 H=2569:00 +CPU= 16.204
  12:36:51 STEP 2570 H=2570:00 +CPU= 17.215
  12:37:17 STEP 2571 H=2571:00 +CPU= 25.861
  12:37:34 STEP 2572 H=2572:00 +CPU= 16.881
  12:37:50 STEP 2573 H=2573:00 +CPU= 16.582
  12:38:17 STEP 2574 H=2574:00 +CPU= 26.690
  12:38:31 STEP 2575 H=2575:00 +CPU= 14.201
  12:38:45 STEP 2576 H=2576:00 +CPU= 13.252
  12:39:04 STEP 2577 H=2577:00 +CPU= 19.769
  12:39:17 STEP 2578 H=2578:00 +CPU= 12.784
  12:39:30 STEP 2579 H=2579:00 +CPU= 12.681
  12:39:52 STEP 2580 H=2580:00 +CPU= 22.032
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002580
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002580
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002580
  12:40:04 STEP 2581 H=2581:00 +CPU= 12.368
  12:40:17 STEP 2582 H=2582:00 +CPU= 13.026
  12:40:37 STEP 2583 H=2583:00 +CPU= 19.460
  12:40:50 STEP 2584 H=2584:00 +CPU= 13.335
  12:41:03 STEP 2585 H=2585:00 +CPU= 12.819
  12:41:27 STEP 2586 H=2586:00 +CPU= 24.013
  12:41:45 STEP 2587 H=2587:00 +CPU= 17.645
  12:42:02 STEP 2588 H=2588:00 +CPU= 17.899
  12:42:30 STEP 2589 H=2589:00 +CPU= 27.872
  12:42:49 STEP 2590 H=2590:00 +CPU= 18.567
  12:43:07 STEP 2591 H=2591:00 +CPU= 18.021
Quit request received from BOINC client, ending the child process
..Failed, model did not complete successfully
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2013050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0289
(argv4) batchid: 982
(argv5) wuid: 12198933
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /home/alez/boinc/slots/11
Project directory is: /home/alez/boinc/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933
..mkdir for temp folder for results failed
Copying: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /home/alez/boinc/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /home/alez/boinc/slots/11/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_61c381c0be019d74216981e017338d65 to: /home/alez/boinc/slots/11/oifs_43r3_ps_0289_2013050100_123_982_12198933.zip
Unzipping the namelist zip file: /home/alez/boinc/slots/11/oifs_43r3_ps_0289_2013050100_123_982_12198933.zip
ic_ancil_file: ic_ancil_12198933
ifsdata_file: ifsdata_12198933
climate_data_file: clim_data_12198933
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_7cbd78a819a0fdd0cbee17f4191083cc to: /home/alez/boinc/slots/11/ic_ancil_12198933.zip
Unzipping the IC ancils zip file: /home/alez/boinc/slots/11/ic_ancil_12198933.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_174b6d66713973119b979dea38a7185b to: /home/alez/boinc/slots/11/ifsdata/ifsdata_12198933.zip
Unzipping the ifsdata_zip file: /home/alez/boinc/slots/11/ifsdata/ifsdata_12198933.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /home/alez/boinc/slots/11/159l_2/clim_data_12198933.zip
Unzipping the climate data zip file: /home/alez/boinc/slots/11/159l_2/clim_data_12198933.zip
Checking for progress XML file: /home/alez/boinc/slots/11/progress_file_12198933.xml
Opened progress file ok : /home/alez/boinc/slots/11/progress_file_12198933.xml
-- Model is restarting --
Adjusting last_iter, 2592, to previous model restart step.
Creating progress file: /home/alez/boinc/slots/11/progress_file_12198933.xml
last_cpu_time: 47477
upload_file_number: 107
last_iter: 2591
last_upload: 9244800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042
The child process has been launched with process id: 16007
Executing the command: /home/alez/boinc/slots/11/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:ichigo:1:1:16007:16007] [20230123:124424:1674474264:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  12:44:59 STEP 2592 H=2592:00 +CPU= 32.683
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002592
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002592
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002592
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002568
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002568
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002568
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002580
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002580
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002580
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_107.zip
Uploading the intermediate file: upload_file_107.zip
  12:45:13 STEP 2593 H=2593:00 +CPU= 13.904
  12:45:26 STEP 2594 H=2594:00 +CPU= 12.956
  12:45:50 STEP 2595 H=2595:00 +CPU= 24.097
  12:46:08 STEP 2596 H=2596:00 +CPU= 17.575
  12:46:25 STEP 2597 H=2597:00 +CPU= 16.848
  12:46:53 STEP 2598 H=2598:00 +CPU= 28.144
  12:47:11 STEP 2599 H=2599:00 +CPU= 17.702
  12:47:25 STEP 2600 H=2600:00 +CPU= 14.289
  12:47:44 STEP 2601 H=2601:00 +CPU= 19.447
  12:47:57 STEP 2602 H=2602:00 +CPU= 12.198
  12:48:09 STEP 2603 H=2603:00 +CPU= 12.488
  12:48:36 STEP 2604 H=2604:00 +CPU= 26.651
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002604
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002604
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002604
  12:48:49 STEP 2605 H=2605:00 +CPU= 12.865
  12:49:03 STEP 2606 H=2606:00 +CPU= 14.158
  12:49:23 STEP 2607 H=2607:00 +CPU= 20.672
  12:49:36 STEP 2608 H=2608:00 +CPU= 12.359
  12:49:48 STEP 2609 H=2609:00 +CPU= 12.033
  12:50:07 STEP 2610 H=2610:00 +CPU= 19.708
  12:50:21 STEP 2611 H=2611:00 +CPU= 13.149
  12:50:39 STEP 2612 H=2612:00 +CPU= 18.324
  12:51:01 STEP 2613 H=2613:00 +CPU= 22.291
  12:51:14 STEP 2614 H=2614:00 +CPU= 12.538
  12:51:26 STEP 2615 H=2615:00 +CPU= 12.561
  12:51:49 STEP 2616 H=2616:00 +CPU= 23.057
Moving to projects directory: /home/alez/boinc/slots/11/ICMGGhq0f+002616
Moving to projects directory: /home/alez/boinc/slots/11/ICMSHhq0f+002616
Moving to projects directory: /home/alez/boinc/slots/11/ICMUAhq0f+002616
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002592
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002592
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002592
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMGGhq0f+002604
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMSHhq0f+002604
Adding to the zip: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_12198933/ICMUAhq0f+002604
Zipping up the intermediate file: /home/alez/boinc/projects/climateprediction.net/oifs_43r3_ps_0289_2013050100_123_982_12198933_0_r238173042_108.zip
Uploading the intermediate file: upload_file_108.zip
  12:52:02 STEP 2617 H=2617:00 +CPU= 12.877
  12:52:15 STEP 2618 H=2618:00 +CPU= 12.591
  12:52:35 STEP 2619 H=2619:00 +CPU= 19.681
  12:52:47 STEP 2620 H=2620:00 +CPU= 12.707
  12:52:59 STEP 2621 H=2621:00 +CPU= 11.933
  12:53:23 STEP 2622 H=2622:00 +CPU= 24.320
  12:53:36 STEP 2623 H=2623:00 +CPU= 12.147
  12:53:49 STEP 2624 H=2624:00 +CPU= 13.501
  12:54:09 STEP 2625 H=2625:00 +CPU= 19.754
  12:54:21 STEP 2626 H=2626:00 +CPU= 12.578
Abort request received from BOINC client, ending the child process

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:37:35 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 9,158,400 46,575 0.0051
09 May 2023 16:37:30 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 8,294,400 42,391 0.0051
09 May 2023 16:36:20 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 6,739,200 34,865 0.0052
09 May 2023 16:36:02 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 5,875,200 30,724 0.0052
09 May 2023 16:35:57 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 5,011,200 26,306 0.0052
09 May 2023 16:35:44 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 3,715,200 19,191 0.0052
09 May 2023 16:35:44 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 3,715,200 19,191 0.0052
09 May 2023 16:35:18 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 2,678,400 13,621 0.0051
09 May 2023 16:34:04 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 1,814,400 9,751 0.0054
09 May 2023 16:33:58 1410588 22285064 oifs_43r3_ps_0289_2013050100_123_982_12198933_0 950,400 5,085 0.0054


©2024 climateprediction.net