climateprediction.net home page
Task 22280523

Task 22280523

Name oifs_43r3_ps_0019_2009050100_123_978_12194663_0
Workunit 12194663
Created 23 Dec 2022, 19:57:35 UTC
Sent 23 Dec 2022, 20:05:58 UTC
Report deadline 22 Jan 2023, 20:05:58 UTC
Received 2 Jan 2023, 23:44:55 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1520317
Run time 22 hours 31 min
CPU time 20 hours 29 min 42 sec
Validate state Invalid
Credit 14,031.36
Device peak FLOPS 8.55 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.25 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,171.07 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
e: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/36/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/36/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_2baf20b7ff899e93ee6763dbd26cbbca to: /var/lib/boinc-client/slots/36/oifs_43r3_ps_0019_2009050100_123_978_12194663.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/36/oifs_43r3_ps_0019_2009050100_123_978_12194663.zip
ic_ancil_file: ic_ancil_12194663
ifsdata_file: ifsdata_12194663
climate_data_file: clim_data_12194663
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_5fb64121cb11cd3349e901f12cced230 to: /var/lib/boinc-client/slots/36/ic_ancil_12194663.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/36/ic_ancil_12194663.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_ef3a54b107bc33bcf90a0f6ff63a9fb3 to: /var/lib/boinc-client/slots/36/ifsdata/ifsdata_12194663.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/36/ifsdata/ifsdata_12194663.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/36/159l_2/clim_data_12194663.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/36/159l_2/clim_data_12194663.zip
Checking for progress XML file: /var/lib/boinc-client/slots/36/progress_file_12194663.xml
Opened progress file ok : /var/lib/boinc-client/slots/36/progress_file_12194663.xml
-- Model is restarting --
Adjusting last_iter, 2326, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/36/progress_file_12194663.xml
last_cpu_time: 55309
upload_file_number: 96
last_iter: 2303
last_upload: 8294400
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262
The child process has been launched with process id: 60577
Executing the command: /var/lib/boinc-client/slots/36/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:60577:60577] [20230102:053047:1672597847:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:31:36 STEP 2304 H=2304:00 +CPU= 43.524
  05:32:01 STEP 2305 H=2305:00 +CPU= 23.349
  05:32:26 STEP 2306 H=2306:00 +CPU= 23.968
  05:33:08 STEP 2307 H=2307:00 +CPU= 39.417
  05:33:36 STEP 2308 H=2308:00 +CPU= 25.901
  05:34:04 STEP 2309 H=2309:00 +CPU= 25.306
  05:34:43 STEP 2310 H=2310:00 +CPU= 37.588
  05:35:09 STEP 2311 H=2311:00 +CPU= 24.723
  05:35:37 STEP 2312 H=2312:00 +CPU= 25.720
  05:36:16 STEP 2313 H=2313:00 +CPU= 37.309
  05:36:46 STEP 2314 H=2314:00 +CPU= 27.931
  05:37:14 STEP 2315 H=2315:00 +CPU= 26.399
  05:37:57 STEP 2316 H=2316:00 +CPU= 41.028
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002316
  05:38:24 STEP 2317 H=2317:00 +CPU= 25.256
  05:38:50 STEP 2318 H=2318:00 +CPU= 24.174
  05:39:30 STEP 2319 H=2319:00 +CPU= 37.042
  05:39:56 STEP 2320 H=2320:00 +CPU= 25.380
  05:40:21 STEP 2321 H=2321:00 +CPU= 22.967
  05:40:58 STEP 2322 H=2322:00 +CPU= 35.012
  05:41:21 STEP 2323 H=2323:00 +CPU= 21.715
  05:41:44 STEP 2324 H=2324:00 +CPU= 22.294
  05:42:20 STEP 2325 H=2325:00 +CPU= 34.367
  05:42:44 STEP 2326 H=2326:00 +CPU= 22.465
  05:43:08 STEP 2327 H=2327:00 +CPU= 22.874
  05:43:51 STEP 2328 H=2328:00 +CPU= 40.715
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_96.zip
Uploading the intermediate file: upload_file_96.zip
  05:44:14 STEP 2329 H=2329:00 +CPU= 21.697
  05:44:38 STEP 2330 H=2330:00 +CPU= 22.798
  05:45:15 STEP 2331 H=2331:00 +CPU= 34.850
  05:45:39 STEP 2332 H=2332:00 +CPU= 22.632
  05:46:01 STEP 2333 H=2333:00 +CPU= 21.452
  05:46:37 STEP 2334 H=2334:00 +CPU= 34.480
  05:47:01 STEP 2335 H=2335:00 +CPU= 22.538
  05:47:24 STEP 2336 H=2336:00 +CPU= 22.292
  05:48:00 STEP 2337 H=2337:00 +CPU= 33.682
  05:48:24 STEP 2338 H=2338:00 +CPU= 23.142
  05:48:48 STEP 2339 H=2339:00 +CPU= 22.025
  05:49:30 STEP 2340 H=2340:00 +CPU= 38.172
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002340
  05:49:54 STEP 2341 H=2341:00 +CPU= 22.983
  05:50:18 STEP 2342 H=2342:00 +CPU= 23.597
  05:50:53 STEP 2343 H=2343:00 +CPU= 34.316
  05:51:18 STEP 2344 H=2344:00 +CPU= 23.802
  05:51:41 STEP 2345 H=2345:00 +CPU= 23.035
  05:52:18 STEP 2346 H=2346:00 +CPU= 35.673
  05:52:42 STEP 2347 H=2347:00 +CPU= 22.633
  05:53:08 STEP 2348 H=2348:00 +CPU= 23.912
  05:53:47 STEP 2349 H=2349:00 +CPU= 35.765
  05:54:11 STEP 2350 H=2350:00 +CPU= 22.965
  05:54:35 STEP 2351 H=2351:00 +CPU= 22.922
  05:55:21 STEP 2352 H=2352:00 +CPU= 43.108
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_97.zip
Uploading the intermediate file: upload_file_97.zip
  05:55:44 STEP 2353 H=2353:00 +CPU= 21.841
  05:56:09 STEP 2354 H=2354:00 +CPU= 23.841
  05:56:47 STEP 2355 H=2355:00 +CPU= 35.269
  05:57:12 STEP 2356 H=2356:00 +CPU= 24.084
  05:57:37 STEP 2357 H=2357:00 +CPU= 23.799
  05:58:16 STEP 2358 H=2358:00 +CPU= 36.219
  05:58:38 STEP 2359 H=2359:00 +CPU= 21.770
  05:59:02 STEP 2360 H=2360:00 +CPU= 22.945
  05:59:36 STEP 2361 H=2361:00 +CPU= 33.753
  06:00:01 STEP 2362 H=2362:00 +CPU= 22.980
  06:00:25 STEP 2363 H=2363:00 +CPU= 22.981
  06:01:05 STEP 2364 H=2364:00 +CPU= 38.667
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002364
  06:01:29 STEP 2365 H=2365:00 +CPU= 22.938
  06:01:53 STEP 2366 H=2366:00 +CPU= 22.419
  06:02:29 STEP 2367 H=2367:00 +CPU= 34.989
  06:02:53 STEP 2368 H=2368:00 +CPU= 22.880
  06:03:18 STEP 2369 H=2369:00 +CPU= 23.137
  06:03:54 STEP 2370 H=2370:00 +CPU= 34.440
  06:04:18 STEP 2371 H=2371:00 +CPU= 23.014
  06:04:43 STEP 2372 H=2372:00 +CPU= 23.972
  06:05:19 STEP 2373 H=2373:00 +CPU= 33.959
  06:05:45 STEP 2374 H=2374:00 +CPU= 25.049
  06:06:11 STEP 2375 H=2375:00 +CPU= 25.079
  06:07:01 STEP 2376 H=2376:00 +CPU= 46.570
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_98.zip
Uploading the intermediate file: upload_file_98.zip
  06:07:28 STEP 2377 H=2377:00 +CPU= 25.494
  06:07:55 STEP 2378 H=2378:00 +CPU= 25.886
  06:08:37 STEP 2379 H=2379:00 +CPU= 38.667
  06:09:03 STEP 2380 H=2380:00 +CPU= 26.168
  06:09:29 STEP 2381 H=2381:00 +CPU= 24.951
  06:10:11 STEP 2382 H=2382:00 +CPU= 38.989
  06:10:35 STEP 2383 H=2383:00 +CPU= 23.968
  06:11:02 STEP 2384 H=2384:00 +CPU= 24.976
  06:11:41 STEP 2385 H=2385:00 +CPU= 37.617
  06:12:09 STEP 2386 H=2386:00 +CPU= 25.461
  06:12:33 STEP 2387 H=2387:00 +CPU= 23.541
  06:13:17 STEP 2388 H=2388:00 +CPU= 41.739
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002388
  06:13:44 STEP 2389 H=2389:00 +CPU= 24.648
  06:14:10 STEP 2390 H=2390:00 +CPU= 25.474
  06:14:53 STEP 2391 H=2391:00 +CPU= 39.874
  06:15:25 STEP 2392 H=2392:00 +CPU= 30.389
  06:15:55 STEP 2393 H=2393:00 +CPU= 28.243
  06:16:47 STEP 2394 H=2394:00 +CPU= 48.494
  06:17:23 STEP 2395 H=2395:00 +CPU= 34.100
  06:18:01 STEP 2396 H=2396:00 +CPU= 35.047
  06:18:55 STEP 2397 H=2397:00 +CPU= 50.097
  06:19:32 STEP 2398 H=2398:00 +CPU= 34.318
  06:20:10 STEP 2399 H=2399:00 +CPU= 34.466
  06:21:24 STEP 2400 H=2400:00 +CPU= 66.561
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_99.zip
Uploading the intermediate file: upload_file_99.zip
  06:22:04 STEP 2401 H=2401:00 +CPU= 36.390
  06:22:45 STEP 2402 H=2402:00 +CPU= 37.127
  06:23:41 STEP 2403 H=2403:00 +CPU= 54.049
  06:24:22 STEP 2404 H=2404:00 +CPU= 37.902
  06:25:03 STEP 2405 H=2405:00 +CPU= 37.474
  06:26:01 STEP 2406 H=2406:00 +CPU= 53.880
  06:26:40 STEP 2407 H=2407:00 +CPU= 36.476
  06:27:19 STEP 2408 H=2408:00 +CPU= 37.667
  06:28:17 STEP 2409 H=2409:00 +CPU= 54.063
  06:29:00 STEP 2410 H=2410:00 +CPU= 40.078
  06:29:39 STEP 2411 H=2411:00 +CPU= 36.375
  06:30:41 STEP 2412 H=2412:00 +CPU= 56.976
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002412
  06:31:19 STEP 2413 H=2413:00 +CPU= 35.985
  06:31:58 STEP 2414 H=2414:00 +CPU= 35.554
  06:32:56 STEP 2415 H=2415:00 +CPU= 54.084
  06:33:35 STEP 2416 H=2416:00 +CPU= 37.030
  06:34:14 STEP 2417 H=2417:00 +CPU= 36.109
  06:35:12 STEP 2418 H=2418:00 +CPU= 52.915
  06:35:50 STEP 2419 H=2419:00 +CPU= 36.052
  06:36:27 STEP 2420 H=2420:00 +CPU= 34.707
  06:37:22 STEP 2421 H=2421:00 +CPU= 52.383
  06:38:02 STEP 2422 H=2422:00 +CPU= 37.043
  06:38:39 STEP 2423 H=2423:00 +CPU= 34.604
  06:39:44 STEP 2424 H=2424:00 +CPU= 60.384
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_100.zip
Uploading the intermediate file: upload_file_100.zip
  06:40:21 STEP 2425 H=2425:00 +CPU= 34.080
  06:41:01 STEP 2426 H=2426:00 +CPU= 36.771
  06:42:01 STEP 2427 H=2427:00 +CPU= 54.134
  06:42:40 STEP 2428 H=2428:00 +CPU= 35.821
  06:43:17 STEP 2429 H=2429:00 +CPU= 34.543
  06:44:11 STEP 2430 H=2430:00 +CPU= 49.925
  06:44:46 STEP 2431 H=2431:00 +CPU= 33.048
  06:45:29 STEP 2432 H=2432:00 +CPU= 38.326
  06:46:27 STEP 2433 H=2433:00 +CPU= 53.349
  06:47:05 STEP 2434 H=2434:00 +CPU= 35.035
  06:47:45 STEP 2435 H=2435:00 +CPU= 36.505
  06:48:50 STEP 2436 H=2436:00 +CPU= 59.719
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002436
  06:49:28 STEP 2437 H=2437:00 +CPU= 35.213
  06:50:07 STEP 2438 H=2438:00 +CPU= 35.380
  06:51:05 STEP 2439 H=2439:00 +CPU= 54.827
  06:51:47 STEP 2440 H=2440:00 +CPU= 38.697
  06:52:24 STEP 2441 H=2441:00 +CPU= 35.404
  06:53:21 STEP 2442 H=2442:00 +CPU= 53.743
  06:53:56 STEP 2443 H=2443:00 +CPU= 33.505
  06:54:34 STEP 2444 H=2444:00 +CPU= 36.176
  06:55:30 STEP 2445 H=2445:00 +CPU= 52.137
  06:56:09 STEP 2446 H=2446:00 +CPU= 36.793
  06:56:48 STEP 2447 H=2447:00 +CPU= 35.809
  06:57:52 STEP 2448 H=2448:00 +CPU= 60.579
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_101.zip
Uploading the intermediate file: upload_file_101.zip
  06:58:29 STEP 2449 H=2449:00 +CPU= 34.253
  06:59:07 STEP 2450 H=2450:00 +CPU= 34.459
  07:00:01 STEP 2451 H=2451:00 +CPU= 51.270
  07:00:37 STEP 2452 H=2452:00 +CPU= 33.683
  07:01:04 STEP 2453 H=2453:00 +CPU= 25.236
  07:01:46 STEP 2454 H=2454:00 +CPU= 38.067
  07:02:12 STEP 2455 H=2455:00 +CPU= 23.994
  07:02:38 STEP 2456 H=2456:00 +CPU= 24.856
  07:03:19 STEP 2457 H=2457:00 +CPU= 38.164
  07:03:46 STEP 2458 H=2458:00 +CPU= 25.082
  07:04:12 STEP 2459 H=2459:00 +CPU= 24.319
  07:04:57 STEP 2460 H=2460:00 +CPU= 41.852
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002460
  07:05:24 STEP 2461 H=2461:00 +CPU= 24.930
  07:05:50 STEP 2462 H=2462:00 +CPU= 23.875
  07:06:29 STEP 2463 H=2463:00 +CPU= 36.176
  07:06:56 STEP 2464 H=2464:00 +CPU= 25.532
  07:07:23 STEP 2465 H=2465:00 +CPU= 24.487
  07:08:03 STEP 2466 H=2466:00 +CPU= 37.155
  07:08:28 STEP 2467 H=2467:00 +CPU= 23.416
  07:08:53 STEP 2468 H=2468:00 +CPU= 23.616
  07:09:32 STEP 2469 H=2469:00 +CPU= 35.484
  07:09:57 STEP 2470 H=2470:00 +CPU= 23.605
  07:10:23 STEP 2471 H=2471:00 +CPU= 24.047
  07:11:06 STEP 2472 H=2472:00 +CPU= 41.057
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_102.zip
Uploading the intermediate file: upload_file_102.zip
  07:11:30 STEP 2473 H=2473:00 +CPU= 22.795
  07:11:54 STEP 2474 H=2474:00 +CPU= 22.691
  07:12:31 STEP 2475 H=2475:00 +CPU= 34.849
  07:12:56 STEP 2476 H=2476:00 +CPU= 23.371
  07:13:19 STEP 2477 H=2477:00 +CPU= 22.121
  07:13:53 STEP 2478 H=2478:00 +CPU= 32.534
  07:14:18 STEP 2479 H=2479:00 +CPU= 22.515
  07:14:42 STEP 2480 H=2480:00 +CPU= 22.380
  07:15:17 STEP 2481 H=2481:00 +CPU= 33.466
  07:15:41 STEP 2482 H=2482:00 +CPU= 22.408
  07:16:05 STEP 2483 H=2483:00 +CPU= 21.931
  07:16:42 STEP 2484 H=2484:00 +CPU= 36.056
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002484
  07:17:05 STEP 2485 H=2485:00 +CPU= 21.948
  07:17:30 STEP 2486 H=2486:00 +CPU= 22.776
  07:18:05 STEP 2487 H=2487:00 +CPU= 33.596
  07:18:30 STEP 2488 H=2488:00 +CPU= 22.347
  07:18:53 STEP 2489 H=2489:00 +CPU= 21.853
  07:19:26 STEP 2490 H=2490:00 +CPU= 32.923
  07:19:49 STEP 2491 H=2491:00 +CPU= 21.459
  07:20:12 STEP 2492 H=2492:00 +CPU= 22.280
  07:20:48 STEP 2493 H=2493:00 +CPU= 33.555
  07:21:11 STEP 2494 H=2494:00 +CPU= 21.812
  07:21:33 STEP 2495 H=2495:00 +CPU= 21.650
  07:22:15 STEP 2496 H=2496:00 +CPU= 39.546
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_103.zip
Uploading the intermediate file: upload_file_103.zip
  07:22:38 STEP 2497 H=2497:00 +CPU= 21.629
  07:23:01 STEP 2498 H=2498:00 +CPU= 22.087
  07:23:36 STEP 2499 H=2499:00 +CPU= 32.686
  07:23:59 STEP 2500 H=2500:00 +CPU= 22.011
  07:24:22 STEP 2501 H=2501:00 +CPU= 21.580
  07:24:57 STEP 2502 H=2502:00 +CPU= 33.163
  07:25:19 STEP 2503 H=2503:00 +CPU= 21.193
  07:25:42 STEP 2504 H=2504:00 +CPU= 22.033
  07:26:18 STEP 2505 H=2505:00 +CPU= 33.605
  07:26:39 STEP 2506 H=2506:00 +CPU= 21.231
  07:27:01 STEP 2507 H=2507:00 +CPU= 21.420
  07:27:42 STEP 2508 H=2508:00 +CPU= 38.043
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002508
  07:28:06 STEP 2509 H=2509:00 +CPU= 23.043
  07:28:29 STEP 2510 H=2510:00 +CPU= 22.529
  07:29:02 STEP 2511 H=2511:00 +CPU= 32.497
  07:29:27 STEP 2512 H=2512:00 +CPU= 22.384
  07:29:49 STEP 2513 H=2513:00 +CPU= 22.115
  07:30:25 STEP 2514 H=2514:00 +CPU= 35.710
  07:30:47 STEP 2515 H=2515:00 +CPU= 22.031
  07:31:10 STEP 2516 H=2516:00 +CPU= 22.764
  07:31:45 STEP 2517 H=2517:00 +CPU= 33.053
  07:32:08 STEP 2518 H=2518:00 +CPU= 22.365
  07:32:31 STEP 2519 H=2519:00 +CPU= 22.733
  07:33:14 STEP 2520 H=2520:00 +CPU= 40.483
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_104.zip
Uploading the intermediate file: upload_file_104.zip
  07:33:38 STEP 2521 H=2521:00 +CPU= 22.332
  07:34:03 STEP 2522 H=2522:00 +CPU= 22.722
  07:34:39 STEP 2523 H=2523:00 +CPU= 34.299
  07:35:03 STEP 2524 H=2524:00 +CPU= 22.616
  07:35:26 STEP 2525 H=2525:00 +CPU= 21.760
  07:36:04 STEP 2526 H=2526:00 +CPU= 36.107
  07:36:28 STEP 2527 H=2527:00 +CPU= 22.436
  07:36:52 STEP 2528 H=2528:00 +CPU= 23.196
  07:37:29 STEP 2529 H=2529:00 +CPU= 35.199
  07:37:53 STEP 2530 H=2530:00 +CPU= 22.337
  07:38:16 STEP 2531 H=2531:00 +CPU= 22.286
  07:38:57 STEP 2532 H=2532:00 +CPU= 38.628
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002532
  07:39:22 STEP 2533 H=2533:00 +CPU= 22.799
  07:39:47 STEP 2534 H=2534:00 +CPU= 23.181
  07:40:23 STEP 2535 H=2535:00 +CPU= 34.151
  07:40:50 STEP 2536 H=2536:00 +CPU= 24.128
  07:41:15 STEP 2537 H=2537:00 +CPU= 23.146
  07:41:54 STEP 2538 H=2538:00 +CPU= 36.559
  07:42:16 STEP 2539 H=2539:00 +CPU= 21.890
  07:42:40 STEP 2540 H=2540:00 +CPU= 23.196
  07:43:18 STEP 2541 H=2541:00 +CPU= 34.417
  07:43:45 STEP 2542 H=2542:00 +CPU= 24.544
  07:44:11 STEP 2543 H=2543:00 +CPU= 24.049
  07:44:53 STEP 2544 H=2544:00 +CPU= 41.250
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_105.zip
Uploading the intermediate file: upload_file_105.zip
  07:45:18 STEP 2545 H=2545:00 +CPU= 23.186
Uploading trickle at timestep: 9158400
  07:45:42 STEP 2546 H=2546:00 +CPU= 23.338
  07:46:22 STEP 2547 H=2547:00 +CPU= 36.714
  07:46:49 STEP 2548 H=2548:00 +CPU= 23.887
  07:47:14 STEP 2549 H=2549:00 +CPU= 22.677
  07:47:51 STEP 2550 H=2550:00 +CPU= 36.477
  07:48:15 STEP 2551 H=2551:00 +CPU= 22.620
  07:48:38 STEP 2552 H=2552:00 +CPU= 22.178
  07:49:15 STEP 2553 H=2553:00 +CPU= 35.092
  07:49:39 STEP 2554 H=2554:00 +CPU= 22.706
  07:50:03 STEP 2555 H=2555:00 +CPU= 23.128
  07:50:43 STEP 2556 H=2556:00 +CPU= 38.640
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002556
  07:51:07 STEP 2557 H=2557:00 +CPU= 22.308
  07:51:32 STEP 2558 H=2558:00 +CPU= 23.456
  07:52:11 STEP 2559 H=2559:00 +CPU= 35.718
  07:52:35 STEP 2560 H=2560:00 +CPU= 23.539
  07:52:58 STEP 2561 H=2561:00 +CPU= 21.474
  07:53:36 STEP 2562 H=2562:00 +CPU= 35.680
  07:54:00 STEP 2563 H=2563:00 +CPU= 22.216
  07:54:23 STEP 2564 H=2564:00 +CPU= 21.683
  07:54:59 STEP 2565 H=2565:00 +CPU= 33.611
  07:55:22 STEP 2566 H=2566:00 +CPU= 22.264
  07:55:46 STEP 2567 H=2567:00 +CPU= 23.077
  07:56:32 STEP 2568 H=2568:00 +CPU= 41.731
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_106.zip
Uploading the intermediate file: upload_file_106.zip
  07:56:57 STEP 2569 H=2569:00 +CPU= 23.312
  07:57:25 STEP 2570 H=2570:00 +CPU= 26.033
  07:58:14 STEP 2571 H=2571:00 +CPU= 44.100
  07:58:44 STEP 2572 H=2572:00 +CPU= 27.304
  07:59:16 STEP 2573 H=2573:00 +CPU= 28.595
  08:00:04 STEP 2574 H=2574:00 +CPU= 42.776
  08:00:34 STEP 2575 H=2575:00 +CPU= 27.852
  08:01:05 STEP 2576 H=2576:00 +CPU= 28.584
  08:01:51 STEP 2577 H=2577:00 +CPU= 41.613
  08:02:19 STEP 2578 H=2578:00 +CPU= 27.235
  08:02:48 STEP 2579 H=2579:00 +CPU= 27.352
  08:03:37 STEP 2580 H=2580:00 +CPU= 45.312
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002580
  08:04:06 STEP 2581 H=2581:00 +CPU= 27.224
  08:04:35 STEP 2582 H=2582:00 +CPU= 27.392
  08:05:18 STEP 2583 H=2583:00 +CPU= 41.269
  08:05:47 STEP 2584 H=2584:00 +CPU= 27.597
  08:06:16 STEP 2585 H=2585:00 +CPU= 26.979
  08:07:01 STEP 2586 H=2586:00 +CPU= 41.544
  08:07:28 STEP 2587 H=2587:00 +CPU= 26.101
  08:07:59 STEP 2588 H=2588:00 +CPU= 28.801
  08:08:44 STEP 2589 H=2589:00 +CPU= 41.823
  08:09:12 STEP 2590 H=2590:00 +CPU= 26.103
  08:09:41 STEP 2591 H=2591:00 +CPU= 27.380
  08:10:33 STEP 2592 H=2592:00 +CPU= 48.670
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_107.zip
Uploading the intermediate file: upload_file_107.zip
  08:11:02 STEP 2593 H=2593:00 +CPU= 26.411
  08:11:30 STEP 2594 H=2594:00 +CPU= 26.813
  08:12:14 STEP 2595 H=2595:00 +CPU= 40.556
  08:12:43 STEP 2596 H=2596:00 +CPU= 27.007
  08:13:14 STEP 2597 H=2597:00 +CPU= 26.413
  08:14:01 STEP 2598 H=2598:00 +CPU= 42.327
  08:14:29 STEP 2599 H=2599:00 +CPU= 27.185
  08:14:59 STEP 2600 H=2600:00 +CPU= 28.933
  08:15:48 STEP 2601 H=2601:00 +CPU= 42.190
  08:16:18 STEP 2602 H=2602:00 +CPU= 28.450
  08:16:47 STEP 2603 H=2603:00 +CPU= 25.906
  08:17:34 STEP 2604 H=2604:00 +CPU= 44.837
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002604
  08:18:02 STEP 2605 H=2605:00 +CPU= 26.747
  08:18:34 STEP 2606 H=2606:00 +CPU= 28.302
  08:19:17 STEP 2607 H=2607:00 +CPU= 40.526
  08:19:43 STEP 2608 H=2608:00 +CPU= 26.171
  08:20:12 STEP 2609 H=2609:00 +CPU= 27.030
  08:20:57 STEP 2610 H=2610:00 +CPU= 42.240
  08:21:25 STEP 2611 H=2611:00 +CPU= 25.849
  08:21:53 STEP 2612 H=2612:00 +CPU= 26.626
  08:22:37 STEP 2613 H=2613:00 +CPU= 41.195
  08:23:06 STEP 2614 H=2614:00 +CPU= 27.057
  08:23:34 STEP 2615 H=2615:00 +CPU= 26.833
  08:24:25 STEP 2616 H=2616:00 +CPU= 47.572
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_108.zip
Uploading the intermediate file: upload_file_108.zip
  08:24:52 STEP 2617 H=2617:00 +CPU= 25.609
  08:25:20 STEP 2618 H=2618:00 +CPU= 27.450
  08:26:08 STEP 2619 H=2619:00 +CPU= 43.492
  08:26:35 STEP 2620 H=2620:00 +CPU= 25.835
  08:27:04 STEP 2621 H=2621:00 +CPU= 26.591
  08:27:48 STEP 2622 H=2622:00 +CPU= 41.345
  08:28:18 STEP 2623 H=2623:00 +CPU= 27.191
  08:28:48 STEP 2624 H=2624:00 +CPU= 26.943
  08:29:35 STEP 2625 H=2625:00 +CPU= 41.997
  08:30:04 STEP 2626 H=2626:00 +CPU= 27.237
  08:30:30 STEP 2627 H=2627:00 +CPU= 25.314
  08:31:15 STEP 2628 H=2628:00 +CPU= 43.604
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002628
  08:31:43 STEP 2629 H=2629:00 +CPU= 26.142
  08:32:13 STEP 2630 H=2630:00 +CPU= 27.524
  08:32:53 STEP 2631 H=2631:00 +CPU= 39.223
  08:33:25 STEP 2632 H=2632:00 +CPU= 29.440
  08:33:54 STEP 2633 H=2633:00 +CPU= 27.572
  08:34:37 STEP 2634 H=2634:00 +CPU= 39.940
  08:35:05 STEP 2635 H=2635:00 +CPU= 27.054
  08:35:34 STEP 2636 H=2636:00 +CPU= 27.678
  08:36:16 STEP 2637 H=2637:00 +CPU= 40.249
  08:36:46 STEP 2638 H=2638:00 +CPU= 27.713
  08:37:15 STEP 2639 H=2639:00 +CPU= 27.386
  08:38:06 STEP 2640 H=2640:00 +CPU= 48.613
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002640
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002628
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002628
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_109.zip
Uploading the intermediate file: upload_file_109.zip
  08:38:36 STEP 2641 H=2641:00 +CPU= 27.924
  08:39:05 STEP 2642 H=2642:00 +CPU= 27.798
  08:39:48 STEP 2643 H=2643:00 +CPU= 41.015
  08:40:19 STEP 2644 H=2644:00 +CPU= 28.272
  08:40:49 STEP 2645 H=2645:00 +CPU= 28.375
  08:41:33 STEP 2646 H=2646:00 +CPU= 42.111
  08:42:03 STEP 2647 H=2647:00 +CPU= 28.678
  08:42:31 STEP 2648 H=2648:00 +CPU= 26.843
  08:43:14 STEP 2649 H=2649:00 +CPU= 40.388
  08:43:44 STEP 2650 H=2650:00 +CPU= 27.994
  08:44:15 STEP 2651 H=2651:00 +CPU= 27.757
  08:45:05 STEP 2652 H=2652:00 +CPU= 45.806
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002652
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002652
  08:45:33 STEP 2653 H=2653:00 +CPU= 27.985
  08:46:06 STEP 2654 H=2654:00 +CPU= 30.029
  08:46:48 STEP 2655 H=2655:00 +CPU= 40.776
  08:47:18 STEP 2656 H=2656:00 +CPU= 29.020
  08:47:48 STEP 2657 H=2657:00 +CPU= 27.625
  08:48:29 STEP 2658 H=2658:00 +CPU= 39.959
  08:48:59 STEP 2659 H=2659:00 +CPU= 28.650
  08:49:29 STEP 2660 H=2660:00 +CPU= 28.116
  08:50:15 STEP 2661 H=2661:00 +CPU= 43.589
  08:50:48 STEP 2662 H=2662:00 +CPU= 29.743
  08:51:18 STEP 2663 H=2663:00 +CPU= 28.910
  08:52:14 STEP 2664 H=2664:00 +CPU= 51.363
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002664
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002640
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002652
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002652
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_110.zip
Uploading the intermediate file: upload_file_110.zip
  08:52:46 STEP 2665 H=2665:00 +CPU= 29.284
  08:53:17 STEP 2666 H=2666:00 +CPU= 28.538
  08:54:08 STEP 2667 H=2667:00 +CPU= 43.880
  08:54:42 STEP 2668 H=2668:00 +CPU= 30.316
  08:55:15 STEP 2669 H=2669:00 +CPU= 28.982
  08:56:03 STEP 2670 H=2670:00 +CPU= 43.996
  08:56:35 STEP 2671 H=2671:00 +CPU= 28.938
  08:57:06 STEP 2672 H=2672:00 +CPU= 29.168
  08:57:49 STEP 2673 H=2673:00 +CPU= 40.894
  08:58:21 STEP 2674 H=2674:00 +CPU= 29.316
  08:58:50 STEP 2675 H=2675:00 +CPU= 27.427
  08:59:44 STEP 2676 H=2676:00 +CPU= 48.320
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002676
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002676
  09:00:16 STEP 2677 H=2677:00 +CPU= 29.956
  09:00:49 STEP 2678 H=2678:00 +CPU= 29.666
  09:01:36 STEP 2679 H=2679:00 +CPU= 42.145
  09:02:08 STEP 2680 H=2680:00 +CPU= 28.710
  09:02:39 STEP 2681 H=2681:00 +CPU= 28.604
  09:03:26 STEP 2682 H=2682:00 +CPU= 43.288
  09:03:59 STEP 2683 H=2683:00 +CPU= 28.866
  09:04:29 STEP 2684 H=2684:00 +CPU= 28.421
  09:05:16 STEP 2685 H=2685:00 +CPU= 43.863
  09:05:46 STEP 2686 H=2686:00 +CPU= 28.886
  09:06:19 STEP 2687 H=2687:00 +CPU= 30.752
  09:07:16 STEP 2688 H=2688:00 +CPU= 56.600
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002688
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002664
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002676
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002676
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_111.zip
Uploading the intermediate file: upload_file_111.zip
  09:07:55 STEP 2689 H=2689:00 +CPU= 34.775
  09:08:33 STEP 2690 H=2690:00 +CPU= 35.263
  09:09:26 STEP 2691 H=2691:00 +CPU= 49.066
  09:10:02 STEP 2692 H=2692:00 +CPU= 33.027
  09:10:38 STEP 2693 H=2693:00 +CPU= 33.361
  09:11:31 STEP 2694 H=2694:00 +CPU= 49.621
  09:12:11 STEP 2695 H=2695:00 +CPU= 35.371
  09:12:52 STEP 2696 H=2696:00 +CPU= 36.107
  09:13:50 STEP 2697 H=2697:00 +CPU= 50.870
  09:14:26 STEP 2698 H=2698:00 +CPU= 34.088
  09:15:04 STEP 2699 H=2699:00 +CPU= 34.730
  09:16:03 STEP 2700 H=2700:00 +CPU= 54.237
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002700
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002700
  09:16:40 STEP 2701 H=2701:00 +CPU= 32.439
  09:17:15 STEP 2702 H=2702:00 +CPU= 34.717
  09:18:11 STEP 2703 H=2703:00 +CPU= 52.232
  09:18:52 STEP 2704 H=2704:00 +CPU= 36.564
  09:19:29 STEP 2705 H=2705:00 +CPU= 33.075
  09:20:25 STEP 2706 H=2706:00 +CPU= 51.344
  09:21:01 STEP 2707 H=2707:00 +CPU= 33.959
  09:21:38 STEP 2708 H=2708:00 +CPU= 33.899
  09:22:35 STEP 2709 H=2709:00 +CPU= 51.198
  09:23:10 STEP 2710 H=2710:00 +CPU= 32.469
  09:23:49 STEP 2711 H=2711:00 +CPU= 34.073
  09:24:58 STEP 2712 H=2712:00 +CPU= 60.385
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002712
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002688
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002700
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002700
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_112.zip
Uploading the intermediate file: upload_file_112.zip
  09:25:37 STEP 2713 H=2713:00 +CPU= 33.977
  09:26:12 STEP 2714 H=2714:00 +CPU= 32.896
  09:27:04 STEP 2715 H=2715:00 +CPU= 48.332
  09:27:41 STEP 2716 H=2716:00 +CPU= 35.616
  09:28:19 STEP 2717 H=2717:00 +CPU= 33.272
  09:29:16 STEP 2718 H=2718:00 +CPU= 50.685
  09:29:47 STEP 2719 H=2719:00 +CPU= 30.373
  09:30:24 STEP 2720 H=2720:00 +CPU= 33.449
  09:31:24 STEP 2721 H=2721:00 +CPU= 52.430
  09:32:00 STEP 2722 H=2722:00 +CPU= 32.808
  09:32:35 STEP 2723 H=2723:00 +CPU= 32.224
  09:33:36 STEP 2724 H=2724:00 +CPU= 56.090
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002724
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002724
  09:34:12 STEP 2725 H=2725:00 +CPU= 33.155
  09:34:49 STEP 2726 H=2726:00 +CPU= 33.911
  09:35:43 STEP 2727 H=2727:00 +CPU= 49.856
  09:36:20 STEP 2728 H=2728:00 +CPU= 34.515
  09:36:56 STEP 2729 H=2729:00 +CPU= 33.438
  09:37:51 STEP 2730 H=2730:00 +CPU= 50.107
  09:38:27 STEP 2731 H=2731:00 +CPU= 32.975
  09:39:05 STEP 2732 H=2732:00 +CPU= 34.851
  09:39:59 STEP 2733 H=2733:00 +CPU= 49.355
  09:40:38 STEP 2734 H=2734:00 +CPU= 33.995
  09:41:15 STEP 2735 H=2735:00 +CPU= 34.721
  09:42:19 STEP 2736 H=2736:00 +CPU= 58.678
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002736
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002712
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002724
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002724
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_113.zip
Uploading the intermediate file: upload_file_113.zip
  09:42:53 STEP 2737 H=2737:00 +CPU= 31.884
  09:43:30 STEP 2738 H=2738:00 +CPU= 34.494
  09:44:20 STEP 2739 H=2739:00 +CPU= 47.377
  09:44:57 STEP 2740 H=2740:00 +CPU= 34.554
  09:45:33 STEP 2741 H=2741:00 +CPU= 32.525
  09:46:27 STEP 2742 H=2742:00 +CPU= 51.156
  09:47:03 STEP 2743 H=2743:00 +CPU= 32.765
  09:47:37 STEP 2744 H=2744:00 +CPU= 32.664
  09:48:32 STEP 2745 H=2745:00 +CPU= 50.241
  09:49:09 STEP 2746 H=2746:00 +CPU= 33.571
  09:49:44 STEP 2747 H=2747:00 +CPU= 32.301
  09:50:42 STEP 2748 H=2748:00 +CPU= 53.339
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002748
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002748
  09:51:16 STEP 2749 H=2749:00 +CPU= 31.232
  09:51:52 STEP 2750 H=2750:00 +CPU= 32.632
  09:52:48 STEP 2751 H=2751:00 +CPU= 50.126
  09:53:24 STEP 2752 H=2752:00 +CPU= 32.592
  09:54:01 STEP 2753 H=2753:00 +CPU= 32.947
  09:54:55 STEP 2754 H=2754:00 +CPU= 49.037
  09:55:26 STEP 2755 H=2755:00 +CPU= 29.748
  09:56:03 STEP 2756 H=2756:00 +CPU= 31.738
  09:56:50 STEP 2757 H=2757:00 +CPU= 45.825
  09:57:23 STEP 2758 H=2758:00 +CPU= 31.801
  09:57:56 STEP 2759 H=2759:00 +CPU= 31.317
  09:58:53 STEP 2760 H=2760:00 +CPU= 54.944
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002760
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002736
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002748
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002748
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_114.zip
Uploading the intermediate file: upload_file_114.zip
  09:59:26 STEP 2761 H=2761:00 +CPU= 30.606
  09:59:58 STEP 2762 H=2762:00 +CPU= 31.556
  10:00:49 STEP 2763 H=2763:00 +CPU= 47.242
  10:01:21 STEP 2764 H=2764:00 +CPU= 30.502
  10:01:51 STEP 2765 H=2765:00 +CPU= 28.674
  10:02:42 STEP 2766 H=2766:00 +CPU= 47.307
  10:03:14 STEP 2767 H=2767:00 +CPU= 30.177
  10:03:49 STEP 2768 H=2768:00 +CPU= 31.915
  10:04:35 STEP 2769 H=2769:00 +CPU= 44.681
  10:05:08 STEP 2770 H=2770:00 +CPU= 32.017
  10:05:39 STEP 2771 H=2771:00 +CPU= 30.945
  10:06:32 STEP 2772 H=2772:00 +CPU= 49.796
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002772
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002772
  10:07:07 STEP 2773 H=2773:00 +CPU= 31.816
  10:07:44 STEP 2774 H=2774:00 +CPU= 32.708
  10:08:35 STEP 2775 H=2775:00 +CPU= 46.376
  10:09:09 STEP 2776 H=2776:00 +CPU= 31.688
  10:09:44 STEP 2777 H=2777:00 +CPU= 31.636
  10:10:31 STEP 2778 H=2778:00 +CPU= 45.011
  10:11:03 STEP 2779 H=2779:00 +CPU= 30.457
  10:11:35 STEP 2780 H=2780:00 +CPU= 30.017
  10:12:22 STEP 2781 H=2781:00 +CPU= 44.223
  10:12:55 STEP 2782 H=2782:00 +CPU= 30.735
  10:13:26 STEP 2783 H=2783:00 +CPU= 29.769
  10:14:23 STEP 2784 H=2784:00 +CPU= 53.594
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002784
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002760
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002772
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002772
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_115.zip
Uploading the intermediate file: upload_file_115.zip
Uploading trickle at timestep: 10022400
  10:14:56 STEP 2785 H=2785:00 +CPU= 29.669
  10:15:28 STEP 2786 H=2786:00 +CPU= 30.798
  10:16:16 STEP 2787 H=2787:00 +CPU= 44.423
  10:16:51 STEP 2788 H=2788:00 +CPU= 31.312
  10:17:23 STEP 2789 H=2789:00 +CPU= 30.504
  10:18:13 STEP 2790 H=2790:00 +CPU= 46.211
  10:18:44 STEP 2791 H=2791:00 +CPU= 29.088
  10:19:16 STEP 2792 H=2792:00 +CPU= 29.351
  10:20:05 STEP 2793 H=2793:00 +CPU= 44.466
  10:20:39 STEP 2794 H=2794:00 +CPU= 31.726
  10:21:11 STEP 2795 H=2795:00 +CPU= 29.602
  10:22:04 STEP 2796 H=2796:00 +CPU= 49.414
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002796
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002796
  10:22:36 STEP 2797 H=2797:00 +CPU= 30.395
  10:23:09 STEP 2798 H=2798:00 +CPU= 30.504
  10:23:56 STEP 2799 H=2799:00 +CPU= 45.260
  10:24:30 STEP 2800 H=2800:00 +CPU= 31.718
  10:25:02 STEP 2801 H=2801:00 +CPU= 29.549
  10:25:50 STEP 2802 H=2802:00 +CPU= 44.612
  10:26:20 STEP 2803 H=2803:00 +CPU= 28.756
  10:26:50 STEP 2804 H=2804:00 +CPU= 29.150
  10:27:37 STEP 2805 H=2805:00 +CPU= 44.044
  10:28:09 STEP 2806 H=2806:00 +CPU= 29.492
  10:28:41 STEP 2807 H=2807:00 +CPU= 29.767
  10:29:37 STEP 2808 H=2808:00 +CPU= 51.756
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002808
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002784
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002796
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002796
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_116.zip
Uploading the intermediate file: upload_file_116.zip
  10:30:07 STEP 2809 H=2809:00 +CPU= 29.076
  10:30:37 STEP 2810 H=2810:00 +CPU= 29.111
  10:31:22 STEP 2811 H=2811:00 +CPU= 42.685
  10:31:53 STEP 2812 H=2812:00 +CPU= 28.985
  10:32:24 STEP 2813 H=2813:00 +CPU= 28.828
  10:33:11 STEP 2814 H=2814:00 +CPU= 44.855
  10:33:42 STEP 2815 H=2815:00 +CPU= 28.806
  10:34:14 STEP 2816 H=2816:00 +CPU= 29.808
  10:35:01 STEP 2817 H=2817:00 +CPU= 43.554
  10:35:33 STEP 2818 H=2818:00 +CPU= 29.702
  10:36:04 STEP 2819 H=2819:00 +CPU= 28.629
  10:36:55 STEP 2820 H=2820:00 +CPU= 48.174
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002820
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002820
  10:37:27 STEP 2821 H=2821:00 +CPU= 29.566
  10:37:58 STEP 2822 H=2822:00 +CPU= 29.041
  10:38:44 STEP 2823 H=2823:00 +CPU= 43.978
  10:39:15 STEP 2824 H=2824:00 +CPU= 29.371
  10:39:46 STEP 2825 H=2825:00 +CPU= 28.796
  10:40:32 STEP 2826 H=2826:00 +CPU= 43.602
  10:41:03 STEP 2827 H=2827:00 +CPU= 28.872
  10:41:35 STEP 2828 H=2828:00 +CPU= 29.636
  10:42:21 STEP 2829 H=2829:00 +CPU= 43.611
  10:42:52 STEP 2830 H=2830:00 +CPU= 29.673
  10:43:25 STEP 2831 H=2831:00 +CPU= 30.227
  10:44:20 STEP 2832 H=2832:00 +CPU= 51.460
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002832
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002808
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002820
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002820
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_117.zip
Uploading the intermediate file: upload_file_117.zip
  10:44:50 STEP 2833 H=2833:00 +CPU= 28.297
  10:45:22 STEP 2834 H=2834:00 +CPU= 29.248
  10:46:07 STEP 2835 H=2835:00 +CPU= 42.998
  10:46:37 STEP 2836 H=2836:00 +CPU= 29.150
  10:47:08 STEP 2837 H=2837:00 +CPU= 28.587
  10:47:53 STEP 2838 H=2838:00 +CPU= 43.173
  10:48:24 STEP 2839 H=2839:00 +CPU= 28.731
  10:48:55 STEP 2840 H=2840:00 +CPU= 29.618
  10:49:40 STEP 2841 H=2841:00 +CPU= 42.466
  10:50:12 STEP 2842 H=2842:00 +CPU= 29.659
  10:50:42 STEP 2843 H=2843:00 +CPU= 28.450
  10:51:33 STEP 2844 H=2844:00 +CPU= 47.068
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002844
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002844
  10:52:04 STEP 2845 H=2845:00 +CPU= 29.430
  10:52:36 STEP 2846 H=2846:00 +CPU= 29.858
  10:53:23 STEP 2847 H=2847:00 +CPU= 43.916
  10:53:54 STEP 2848 H=2848:00 +CPU= 29.213
  10:54:25 STEP 2849 H=2849:00 +CPU= 28.988
  10:55:13 STEP 2850 H=2850:00 +CPU= 44.474
  10:55:44 STEP 2851 H=2851:00 +CPU= 29.250
  10:56:15 STEP 2852 H=2852:00 +CPU= 29.467
  10:57:01 STEP 2853 H=2853:00 +CPU= 43.112
  10:57:32 STEP 2854 H=2854:00 +CPU= 29.741
  10:58:04 STEP 2855 H=2855:00 +CPU= 29.770
  10:59:00 STEP 2856 H=2856:00 +CPU= 52.073
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMGGhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMSHhq0f+002856
Moving to projects directory: /var/lib/boinc-client/slots/36/ICMUAhq0f+002856
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002832
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMGGhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMSHhq0f+002844
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12194663/ICMUAhq0f+002844
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0019_2009050100_123_978_12194663_0_r450412262_118.zip
Uploading the intermediate file: upload_file_118.zip
  10:59:30 STEP 2857 H=2857:00 +CPU= 28.274
  11:00:01 STEP 2858 H=2858:00 +CPU= 29.728
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:05:30 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 10,022,400 71,197 0.0071
09 May 2023 16:05:25 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 9,158,400 62,878 0.0069
09 May 2023 16:05:25 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 9,158,400 62,878 0.0069
09 May 2023 16:05:18 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 7,689,600 50,903 0.0066
09 May 2023 15:53:45 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 5,875,200 40,227 0.0068
09 May 2023 15:53:15 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 5,011,200 34,519 0.0069
09 May 2023 15:53:01 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 4,147,200 28,932 0.0070
09 May 2023 15:52:49 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 3,283,200 22,929 0.0070
09 May 2023 15:52:36 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 2,419,200 16,340 0.0068
09 May 2023 15:52:02 1520317 22280523 oifs_43r3_ps_0019_2009050100_123_978_12194663_0 864,000 5,725 0.0066


©2024 climateprediction.net