climateprediction.net home page
Task 22268058

Task 22268058

Name oifs_43r3_ps_0267_1997050100_123_966_12182911_0
Workunit 12182911
Created 22 Dec 2022, 13:09:17 UTC
Sent 29 Dec 2022, 13:33:47 UTC
Report deadline 28 Jan 2023, 13:33:47 UTC
Received 30 Dec 2022, 11:06:21 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1534826
Run time 1 hours 46 min 1 sec
CPU time 1 hours 34 min 13 sec
Validate state Invalid
Credit 1,209.60
Device peak FLOPS 1.00 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,778.01 MB
Peak swap size 4,974.25 MB
Peak disk usage 1,419.71 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000168
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000180
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000180
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000180
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_7.zip
Uploading the intermediate file: upload_file_7.zip
  15:30:50 STEP  193 H= 193:00 +CPU= 11.318
  15:31:01 STEP  194 H= 194:00 +CPU= 11.026
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:31:22 STEP  195 H= 195:00 +CPU= 17.239
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:31:36 STEP  196 H= 196:00 +CPU= 11.258
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:31:50 STEP  197 H= 197:00 +CPU= 10.750
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:32:12 STEP  198 H= 198:00 +CPU= 17.889
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:32:25 STEP  199 H= 199:00 +CPU= 10.657
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:32:38 STEP  200 H= 200:00 +CPU= 11.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:33:02 STEP  201 H= 201:00 +CPU= 17.360
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:33:15 STEP  202 H= 202:00 +CPU= 11.372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:33:28 STEP  203 H= 203:00 +CPU= 10.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:33:54 STEP  204 H= 204:00 +CPU= 19.980
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000204
Moving to projects directory: /root/slots/2/ICMSHhq0f+000204
Moving to projects directory: /root/slots/2/ICMUAhq0f+000204
  15:34:07 STEP  205 H= 205:00 +CPU= 11.061
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:34:23 STEP  206 H= 206:00 +CPU= 11.516
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:34:44 STEP  207 H= 207:00 +CPU= 17.238
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:34:57 STEP  208 H= 208:00 +CPU= 11.271
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:35:11 STEP  209 H= 209:00 +CPU= 10.220
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:35:32 STEP  210 H= 210:00 +CPU= 16.787
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:35:44 STEP  211 H= 211:00 +CPU= 10.255
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:35:57 STEP  212 H= 212:00 +CPU= 10.779
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:36:21 STEP  213 H= 213:00 +CPU= 17.404
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:36:33 STEP  214 H= 214:00 +CPU= 10.786
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:36:46 STEP  215 H= 215:00 +CPU= 10.919
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:37:13 STEP  216 H= 216:00 +CPU= 20.534
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000216
Moving to projects directory: /root/slots/2/ICMSHhq0f+000216
Moving to projects directory: /root/slots/2/ICMUAhq0f+000216
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000192
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000192
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000192
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000204
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000204
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000204
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_8.zip
Uploading the intermediate file: upload_file_8.zip
  15:37:26 STEP  217 H= 217:00 +CPU= 10.891
  15:37:37 STEP  218 H= 218:00 +CPU= 11.201
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:37:56 STEP  219 H= 219:00 +CPU= 16.713
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:38:11 STEP  220 H= 220:00 +CPU= 11.216
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:38:23 STEP  221 H= 221:00 +CPU= 10.487
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:38:45 STEP  222 H= 222:00 +CPU= 17.351
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:38:58 STEP  223 H= 223:00 +CPU= 10.975
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:39:13 STEP  224 H= 224:00 +CPU= 11.023
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:39:33 STEP  225 H= 225:00 +CPU= 16.028
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:39:46 STEP  226 H= 226:00 +CPU= 11.224
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:40:01 STEP  227 H= 227:00 +CPU= 11.029
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:40:24 STEP  228 H= 228:00 +CPU= 19.045
Moving to projects directory: /root/slots/2/ICMGGhq0f+000228
Moving to projects directory: /root/slots/2/ICMSHhq0f+000228
Moving to projects directory: /root/slots/2/ICMUAhq0f+000228
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:40:37 STEP  229 H= 229:00 +CPU= 10.895
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:40:53 STEP  230 H= 230:00 +CPU= 11.448
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:41:14 STEP  231 H= 231:00 +CPU= 17.444
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:41:27 STEP  232 H= 232:00 +CPU= 11.230
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:41:42 STEP  233 H= 233:00 +CPU= 11.058
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:42:04 STEP  234 H= 234:00 +CPU= 17.227
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:42:17 STEP  235 H= 235:00 +CPU= 10.908
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:42:31 STEP  236 H= 236:00 +CPU= 10.691
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:42:52 STEP  237 H= 237:00 +CPU= 16.692
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:43:05 STEP  238 H= 238:00 +CPU= 11.120
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:43:18 STEP  239 H= 239:00 +CPU= 10.485
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:43:43 STEP  240 H= 240:00 +CPU= 19.925
Moving to projects directory: /root/slots/2/ICMGGhq0f+000240
Moving to projects directory: /root/slots/2/ICMSHhq0f+000240
Moving to projects directory: /root/slots/2/ICMUAhq0f+000240
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000216
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000216
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000216
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000228
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000228
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000228
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_9.zip
Uploading the intermediate file: upload_file_9.zip
  15:43:55 STEP  241 H= 241:00 +CPU= 11.141
  15:44:06 STEP  242 H= 242:00 +CPU= 11.136
Uploading trickle at timestep: 864000
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:44:26 STEP  243 H= 243:00 +CPU= 16.491
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:44:42 STEP  244 H= 244:00 +CPU= 11.427
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:44:55 STEP  245 H= 245:00 +CPU= 11.221
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:45:17 STEP  246 H= 246:00 +CPU= 17.729
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:45:31 STEP  247 H= 247:00 +CPU= 10.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:45:45 STEP  248 H= 248:00 +CPU= 11.227
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:46:07 STEP  249 H= 249:00 +CPU= 18.092
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:46:22 STEP  250 H= 250:00 +CPU= 11.477
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:46:35 STEP  251 H= 251:00 +CPU= 10.888
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:46:58 STEP  252 H= 252:00 +CPU= 18.558
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000252
Moving to projects directory: /root/slots/2/ICMSHhq0f+000252
Moving to projects directory: /root/slots/2/ICMUAhq0f+000252
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:47:12 STEP  253 H= 253:00 +CPU= 10.711
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:47:25 STEP  254 H= 254:00 +CPU= 10.675
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:47:46 STEP  255 H= 255:00 +CPU= 16.983
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:48:01 STEP  256 H= 256:00 +CPU= 11.111
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:48:14 STEP  257 H= 257:00 +CPU= 10.847
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:48:36 STEP  258 H= 258:00 +CPU= 17.504
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:48:48 STEP  259 H= 259:00 +CPU= 10.648
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:49:04 STEP  260 H= 260:00 +CPU= 11.287
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:49:25 STEP  261 H= 261:00 +CPU= 17.679
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:49:38 STEP  262 H= 262:00 +CPU= 10.840
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:49:54 STEP  263 H= 263:00 +CPU= 11.393
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:50:17 STEP  264 H= 264:00 +CPU= 19.941
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000264
Moving to projects directory: /root/slots/2/ICMSHhq0f+000264
Moving to projects directory: /root/slots/2/ICMUAhq0f+000264
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000240
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000240
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000240
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000252
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000252
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000252
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_10.zip
Uploading the intermediate file: upload_file_10.zip
  15:50:30 STEP  265 H= 265:00 +CPU= 10.730
  15:50:41 STEP  266 H= 266:00 +CPU= 10.820
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:51:02 STEP  267 H= 267:00 +CPU= 17.025
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:51:15 STEP  268 H= 268:00 +CPU= 11.294
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:51:30 STEP  269 H= 269:00 +CPU= 10.784
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:51:51 STEP  270 H= 270:00 +CPU= 16.996
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:52:04 STEP  271 H= 271:00 +CPU= 10.560
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:52:17 STEP  272 H= 272:00 +CPU= 11.067
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:52:38 STEP  273 H= 273:00 +CPU= 17.089
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:52:53 STEP  274 H= 274:00 +CPU= 10.628
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:53:06 STEP  275 H= 275:00 +CPU= 11.064
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:53:31 STEP  276 H= 276:00 +CPU= 19.683
Moving to projects directory: /root/slots/2/ICMGGhq0f+000276
Moving to projects directory: /root/slots/2/ICMSHhq0f+000276
Moving to projects directory: /root/slots/2/ICMUAhq0f+000276
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:53:44 STEP  277 H= 277:00 +CPU= 11.068
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:53:58 STEP  278 H= 278:00 +CPU= 11.375
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:54:21 STEP  279 H= 279:00 +CPU= 17.445
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:54:34 STEP  280 H= 280:00 +CPU= 10.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:54:47 STEP  281 H= 281:00 +CPU= 10.822
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:55:11 STEP  282 H= 282:00 +CPU= 17.850
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:55:24 STEP  283 H= 283:00 +CPU= 10.798
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:55:36 STEP  284 H= 284:00 +CPU= 10.482
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:55:57 STEP  285 H= 285:00 +CPU= 16.754
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:56:12 STEP  286 H= 286:00 +CPU= 11.267
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:56:25 STEP  287 H= 287:00 +CPU= 10.767
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:56:51 STEP  288 H= 288:00 +CPU= 19.828
Moving to projects directory: /root/slots/2/ICMGGhq0f+000288
Moving to projects directory: /root/slots/2/ICMSHhq0f+000288
Moving to projects directory: /root/slots/2/ICMUAhq0f+000288
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000264
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000264
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000264
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000276
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000276
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000276
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_11.zip
Uploading the intermediate file: upload_file_11.zip
  15:57:01 STEP  289 H= 289:00 +CPU= 10.462
  15:57:13 STEP  290 H= 290:00 +CPU= 11.399
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:57:32 STEP  291 H= 291:00 +CPU= 17.279
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:57:45 STEP  292 H= 292:00 +CPU= 10.898
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:57:57 STEP  293 H= 293:00 +CPU= 10.396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:21 STEP  294 H= 294:00 +CPU= 17.231
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:33 STEP  295 H= 295:00 +CPU= 10.388
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:46 STEP  296 H= 296:00 +CPU= 11.143
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:59:08 STEP  297 H= 297:00 +CPU= 17.771
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:59:24 STEP  298 H= 298:00 +CPU= 11.734
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:59:37 STEP  299 H= 299:00 +CPU= 11.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:04 STEP  300 H= 300:00 +CPU= 20.549
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000300
Moving to projects directory: /root/slots/2/ICMSHhq0f+000300
Moving to projects directory: /root/slots/2/ICMUAhq0f+000300
  16:00:17 STEP  301 H= 301:00 +CPU= 11.177
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:33 STEP  302 H= 302:00 +CPU= 11.533
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:00:55 STEP  303 H= 303:00 +CPU= 18.085
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:01:10 STEP  304 H= 304:00 +CPU= 11.556
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:01:24 STEP  305 H= 305:00 +CPU= 11.248
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:01:46 STEP  306 H= 306:00 +CPU= 18.506
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:02:02 STEP  307 H= 307:00 +CPU= 11.455
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:02:15 STEP  308 H= 308:00 +CPU= 11.514
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:02:38 STEP  309 H= 309:00 +CPU= 18.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:02:54 STEP  310 H= 310:00 +CPU= 11.760
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:03:13 STEP  311 H= 311:00 +CPU= 13.039
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:03:55 STEP  312 H= 312:00 +CPU= 25.786
Moving to projects directory: /root/slots/2/ICMGGhq0f+000312
Moving to projects directory: /root/slots/2/ICMSHhq0f+000312
Moving to projects directory: /root/slots/2/ICMUAhq0f+000312
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000288
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000288
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000288
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000300
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000300
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000300
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_12.zip
Uploading the intermediate file: upload_file_12.zip
  16:04:13 STEP  313 H= 313:00 +CPU= 13.602
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:04:27 STEP  314 H= 314:00 +CPU= 11.793
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:04:53 STEP  315 H= 315:00 +CPU= 18.288
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:05:06 STEP  316 H= 316:00 +CPU= 10.966
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:05:21 STEP  317 H= 317:00 +CPU= 10.428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:05:44 STEP  318 H= 318:00 +CPU= 17.509
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:05:57 STEP  319 H= 319:00 +CPU= 10.416
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:06:13 STEP  320 H= 320:00 +CPU= 11.045
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:06:35 STEP  321 H= 321:00 +CPU= 17.011
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:06:48 STEP  322 H= 322:00 +CPU= 10.910
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:07:03 STEP  323 H= 323:00 +CPU= 10.544
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:07:27 STEP  324 H= 324:00 +CPU= 18.979
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000324
Moving to projects directory: /root/slots/2/ICMSHhq0f+000324
Moving to projects directory: /root/slots/2/ICMUAhq0f+000324
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:07:42 STEP  325 H= 325:00 +CPU= 10.551
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:07:56 STEP  326 H= 326:00 +CPU= 10.896
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:08:18 STEP  327 H= 327:00 +CPU= 17.114
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:08:34 STEP  328 H= 328:00 +CPU= 10.963
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:08:47 STEP  329 H= 329:00 +CPU= 10.601
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:09:11 STEP  330 H= 330:00 +CPU= 17.330
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:09:24 STEP  331 H= 331:00 +CPU= 10.443
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:09:37 STEP  332 H= 332:00 +CPU= 10.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:10:02 STEP  333 H= 333:00 +CPU= 17.251
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:10:15 STEP  334 H= 334:00 +CPU= 10.720
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:10:28 STEP  335 H= 335:00 +CPU= 10.902
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:10:56 STEP  336 H= 336:00 +CPU= 20.074
Moving to projects directory: /root/slots/2/ICMGGhq0f+000336
Moving to projects directory: /root/slots/2/ICMSHhq0f+000336
Moving to projects directory: /root/slots/2/ICMUAhq0f+000336
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000312
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000312
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000312
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000324
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000324
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000324
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_13.zip
Uploading the intermediate file: upload_file_13.zip
  16:11:07 STEP  337 H= 337:00 +CPU= 10.658
  16:11:18 STEP  338 H= 338:00 +CPU= 10.746
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:11:41 STEP  339 H= 339:00 +CPU= 17.208
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:11:54 STEP  340 H= 340:00 +CPU= 10.860
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:12:07 STEP  341 H= 341:00 +CPU= 10.750
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:12:32 STEP  342 H= 342:00 +CPU= 17.410
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:12:45 STEP  343 H= 343:00 +CPU= 10.635
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:13:01 STEP  344 H= 344:00 +CPU= 10.873
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:13:23 STEP  345 H= 345:00 +CPU= 17.167
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:13:37 STEP  346 H= 346:00 +CPU= 11.006
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:13:52 STEP  347 H= 347:00 +CPU= 10.717
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:16 STEP  348 H= 348:00 +CPU= 18.962
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000348
Moving to projects directory: /root/slots/2/ICMSHhq0f+000348
Moving to projects directory: /root/slots/2/ICMUAhq0f+000348
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:31 STEP  349 H= 349:00 +CPU= 10.841
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:14:45 STEP  350 H= 350:00 +CPU= 10.871
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:06 STEP  351 H= 351:00 +CPU= 16.938
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:22 STEP  352 H= 352:00 +CPU= 11.065
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:35 STEP  353 H= 353:00 +CPU= 10.569
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:15:58 STEP  354 H= 354:00 +CPU= 17.614
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:13 STEP  355 H= 355:00 +CPU= 10.511
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:27 STEP  356 H= 356:00 +CPU= 11.017
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:16:49 STEP  357 H= 357:00 +CPU= 17.080
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:04 STEP  358 H= 358:00 +CPU= 10.823
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:18 STEP  359 H= 359:00 +CPU= 10.943
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:17:45 STEP  360 H= 360:00 +CPU= 20.081
Moving to projects directory: /root/slots/2/ICMGGhq0f+000360
Moving to projects directory: /root/slots/2/ICMSHhq0f+000360
Moving to projects directory: /root/slots/2/ICMUAhq0f+000360
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000336
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000336
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000336
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000348
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000348
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000348
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_14.zip
Uploading the intermediate file: upload_file_14.zip
  16:17:56 STEP  361 H= 361:00 +CPU= 10.527
  16:18:07 STEP  362 H= 362:00 +CPU= 10.932
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:31 STEP  363 H= 363:00 +CPU= 17.295
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:45 STEP  364 H= 364:00 +CPU= 10.689
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:18:58 STEP  365 H= 365:00 +CPU= 10.435
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:22 STEP  366 H= 366:00 +CPU= 17.619
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:35 STEP  367 H= 367:00 +CPU= 10.446
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:19:50 STEP  368 H= 368:00 +CPU= 10.865
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:13 STEP  369 H= 369:00 +CPU= 17.297
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:26 STEP  370 H= 370:00 +CPU= 10.851
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:20:41 STEP  371 H= 371:00 +CPU= 10.428
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:06 STEP  372 H= 372:00 +CPU= 19.199
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000372
Moving to projects directory: /root/slots/2/ICMSHhq0f+000372
Moving to projects directory: /root/slots/2/ICMUAhq0f+000372
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:20 STEP  373 H= 373:00 +CPU= 10.454
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:34 STEP  374 H= 374:00 +CPU= 10.946
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:21:56 STEP  375 H= 375:00 +CPU= 17.175
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:11 STEP  376 H= 376:00 +CPU= 10.658
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:25 STEP  377 H= 377:00 +CPU= 10.733
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:22:47 STEP  378 H= 378:00 +CPU= 17.417
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:03 STEP  379 H= 379:00 +CPU= 10.811
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:16 STEP  380 H= 380:00 +CPU= 10.715
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:38 STEP  381 H= 381:00 +CPU= 17.183
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:23:54 STEP  382 H= 382:00 +CPU= 11.010
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:24:07 STEP  383 H= 383:00 +CPU= 10.790
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:24:35 STEP  384 H= 384:00 +CPU= 20.214
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000384
Moving to projects directory: /root/slots/2/ICMSHhq0f+000384
Moving to projects directory: /root/slots/2/ICMUAhq0f+000384
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000360
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000360
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000360
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMGGhq0f+000372
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMSHhq0f+000372
Adding to the zip: /root/projects/climateprediction.net/oifs_43r3_ps_12182911/ICMUAhq0f+000372
Zipping up the intermediate file: /root/projects/climateprediction.net/oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984_15.zip
Uploading the intermediate file: upload_file_15.zip
  16:24:48 STEP  385 H= 385:00 +CPU= 10.587
  16:24:59 STEP  386 H= 386:00 +CPU= 10.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:21 STEP  387 H= 387:00 +CPU= 17.091
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:35 STEP  388 H= 388:00 +CPU= 10.795
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:48 STEP  389 H= 389:00 +CPU= 10.610
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:13 STEP  390 H= 390:00 +CPU= 17.574
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:26 STEP  391 H= 391:00 +CPU= 10.646
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:26:41 STEP  392 H= 392:00 +CPU= 10.929
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:03 STEP  393 H= 393:00 +CPU= 17.147
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:17 STEP  394 H= 394:00 +CPU= 10.909
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:32 STEP  395 H= 395:00 +CPU= 10.753
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:27:56 STEP  396 H= 396:00 +CPU= 18.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Moving to projects directory: /root/slots/2/ICMGGhq0f+000396
Moving to projects directory: /root/slots/2/ICMSHhq0f+000396
Moving to projects directory: /root/slots/2/ICMUAhq0f+000396
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:11 STEP  397 H= 397:00 +CPU= 10.613
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:25 STEP  398 H= 398:00 +CPU= 11.002
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:28:47 STEP  399 H= 399:00 +CPU= 17.280
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:03 STEP  400 H= 400:00 +CPU= 11.047
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:16 STEP  401 H= 401:00 +CPU= 10.560
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:41 STEP  402 H= 402:00 +CPU= 17.707
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:29:54 STEP  403 H= 403:00 +CPU= 10.484
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 1997050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0267
(argv4) batchid: 966
(argv5) wuid: 12182911
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /root/slots/2
Project directory is: /root/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /root/projects/climateprediction.net/oifs_43r3_ps_12182911
..mkdir for temp folder for results failed
Copying: /root/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /root/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /root/slots/2/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f698d18b7b0ff1182a0cbc3a52934038 to: /root/slots/2/oifs_43r3_ps_0267_1997050100_123_966_12182911.zip
Unzipping the namelist zip file: /root/slots/2/oifs_43r3_ps_0267_1997050100_123_966_12182911.zip
ic_ancil_file: ic_ancil_12182911
ifsdata_file: ifsdata_12182911
climate_data_file: clim_data_12182911
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_29610d087272e6b3280a3d33787a23ae to: /root/slots/2/ic_ancil_12182911.zip
Unzipping the IC ancils zip file: /root/slots/2/ic_ancil_12182911.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_b689c0d740d92a7fcdfeec9b71323c05 to: /root/slots/2/ifsdata/ifsdata_12182911.zip
Unzipping the ifsdata_zip file: /root/slots/2/ifsdata/ifsdata_12182911.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /root/slots/2/159l_2/clim_data_12182911.zip
Unzipping the climate data zip file: /root/slots/2/159l_2/clim_data_12182911.zip
Checking for progress XML file: /root/slots/2/progress_file_12182911.xml
Opened progress file ok : /root/slots/2/progress_file_12182911.xml
-- Model is restarting --
Adjusting last_iter, 404, to previous model restart step.
Creating progress file: /root/slots/2/progress_file_12182911.xml
last_cpu_time: 5539
upload_file_number: 16
last_iter: 383
last_upload: 1382400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0267_1997050100_123_966_12182911_0_r1004952984
The child process has been launched with process id: 9997
Executing the command: /root/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:50-privat:1:1:9997:9997] [20221230:105921:1672394361:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
The child process terminated with status: 0
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:01:42 1534826 22268058 oifs_43r3_ps_0267_1997050100_123_966_12182911_0 864,000 3,374 0.0039


©2024 climateprediction.net