climateprediction.net home page
Task 22310884

Task 22310884

Name oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0
Workunit 12210662
Created 13 Feb 2023, 19:08:42 UTC
Sent 13 Feb 2023, 19:10:49 UTC
Report deadline 14 Apr 2023, 19:10:49 UTC
Received 15 Feb 2023, 17:24:55 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1533094
Run time 1 days 1 hours 56 min 58 sec
CPU time 1 days 1 hours 29 min 27 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 8.06 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,075.97 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,590.71 MB
Stderr
<core_client_version>7.22.0</core_client_version>
<![CDATA[
<stderr_txt>
3634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                CLOUD_LAYER 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113634:1676457394:673.206] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 CLOUDSC 
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113635:1676457395:674.206] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:antares:1:1:4848:4848] [20230215:113635:1676457395:674.206] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000011E2552  cloudsc_                  925  cloudsc.F90
oifs_43r3_model.e  00000000011D9EE7  cloud_layer_              131  cloud_layer.F90
oifs_43r3_model.e  00000000011730D0  callpar_                 1223  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a2zn
(argv4) batchid: 991
(argv5) wuid: 12210662
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc/slots/21
Project directory is: /var/lib/boinc/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662
..mkdir for temp folder for results failed
Copying: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc/slots/21/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc/slots/21/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c348b945a438a52f48c5d8993ddd7a56 to: /var/lib/boinc/slots/21/oifs_43r3_bl_a2zn_2016092300_15_991_12210662.zip
Unzipping the namelist zip file: /var/lib/boinc/slots/21/oifs_43r3_bl_a2zn_2016092300_15_991_12210662.zip
ic_ancil_file: ic_ancil_12210662
ifsdata_file: ifsdata_12210662
climate_data_file: clim_data_12210662
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc/slots/21/ic_ancil_12210662.zip
Unzipping the IC ancils zip file: /var/lib/boinc/slots/21/ic_ancil_12210662.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_723f772bbec8d12fbb800ea1e5ba6337 to: /var/lib/boinc/slots/21/ifsdata/ifsdata_12210662.zip
Unzipping the ifsdata_zip file: /var/lib/boinc/slots/21/ifsdata/ifsdata_12210662.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc/slots/21/159l_2/clim_data_12210662.zip
Unzipping the climate data zip file: /var/lib/boinc/slots/21/159l_2/clim_data_12210662.zip
Checking for progress XML file: /var/lib/boinc/slots/21/progress_file_12210662.xml
Opened progress file ok : /var/lib/boinc/slots/21/progress_file_12210662.xml
-- Model is restarting --
Adjusting last_iter, 1019, to previous model restart step.
Creating progress file: /var/lib/boinc/slots/21/progress_file_12210662.xml
last_cpu_time: 70336
upload_file_number: 10
last_iter: 1009
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0_r1348356034
The child process has been launched with process id: 8873
Executing the command: /var/lib/boinc/slots/21/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:antares:1:1:8873:8873] [20230215:113934:1676457574:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  11:40:26 STEP 1008 H= 252:00 +CPU= 40.853
  11:40:57 STEP 1009 H= 252:15 +CPU= 29.309
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  11:42:05 STEP 1010 H= 252:30 +CPU= 45.120
  11:43:01 STEP 1011 H= 252:45 +CPU= 52.600
  11:43:53 STEP 1012 H= 253:00 +CPU= 49.730
  11:44:47 STEP 1013 H= 253:15 +CPU= 50.719
  11:45:39 STEP 1014 H= 253:30 +CPU= 51.100
  11:46:36 STEP 1015 H= 253:45 +CPU= 54.297
  11:47:36 STEP 1016 H= 254:00 +CPU= 57.832
  11:48:30 STEP 1017 H= 254:15 +CPU= 51.480
  11:49:25 STEP 1018 H= 254:30 +CPU= 52.688
  11:50:22 STEP 1019 H= 254:45 +CPU= 54.566
  11:51:22 STEP 1020 H= 255:00 +CPU= 58.361
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001020
  11:52:14 STEP 1021 H= 255:15 +CPU= 50.341
  11:53:07 STEP 1022 H= 255:30 +CPU= 52.538
  11:54:04 STEP 1023 H= 255:45 +CPU= 55.107
  11:54:56 STEP 1024 H= 256:00 +CPU= 51.251
  11:55:47 STEP 1025 H= 256:15 +CPU= 49.814
  11:56:43 STEP 1026 H= 256:30 +CPU= 53.758
  11:57:36 STEP 1027 H= 256:45 +CPU= 51.137
  11:58:34 STEP 1028 H= 257:00 +CPU= 55.940
  11:59:28 STEP 1029 H= 257:15 +CPU= 53.169
  12:00:21 STEP 1030 H= 257:30 +CPU= 51.913
  12:01:13 STEP 1031 H= 257:45 +CPU= 50.472
  12:02:21 STEP 1032 H= 258:00 +CPU= 65.947
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001032
  12:03:15 STEP 1033 H= 258:15 +CPU= 53.034
  12:04:09 STEP 1034 H= 258:30 +CPU= 52.717
  12:04:58 STEP 1035 H= 258:45 +CPU= 48.034
  12:05:48 STEP 1036 H= 259:00 +CPU= 49.156
  12:06:40 STEP 1037 H= 259:15 +CPU= 50.378
  12:07:32 STEP 1038 H= 259:30 +CPU= 50.374
  12:08:22 STEP 1039 H= 259:45 +CPU= 49.145
  12:09:17 STEP 1040 H= 260:00 +CPU= 53.530
  12:10:09 STEP 1041 H= 260:15 +CPU= 51.511
  12:11:03 STEP 1042 H= 260:30 +CPU= 52.860
  12:11:55 STEP 1043 H= 260:45 +CPU= 50.642
  12:13:00 STEP 1044 H= 261:00 +CPU= 62.966
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001044
  12:13:51 STEP 1045 H= 261:15 +CPU= 50.367
  12:14:43 STEP 1046 H= 261:30 +CPU= 50.795
  12:15:34 STEP 1047 H= 261:45 +CPU= 50.211
  12:16:29 STEP 1048 H= 262:00 +CPU= 53.842
  12:17:22 STEP 1049 H= 262:15 +CPU= 51.445
  12:18:15 STEP 1050 H= 262:30 +CPU= 51.982
  12:19:09 STEP 1051 H= 262:45 +CPU= 52.932
  12:20:05 STEP 1052 H= 263:00 +CPU= 53.934
  12:21:00 STEP 1053 H= 263:15 +CPU= 54.241
  12:21:56 STEP 1054 H= 263:30 +CPU= 53.656
  12:22:47 STEP 1055 H= 263:45 +CPU= 50.861
  12:24:02 STEP 1056 H= 264:00 +CPU= 72.142
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0_r1348356034_10.zip
Uploading the intermediate file: upload_file_10.zip
  12:24:55 STEP 1057 H= 264:15 +CPU= 51.291
  12:25:55 STEP 1058 H= 264:30 +CPU= 58.521
  12:26:45 STEP 1059 H= 264:45 +CPU= 48.625
  12:27:41 STEP 1060 H= 265:00 +CPU= 53.413
  12:28:35 STEP 1061 H= 265:15 +CPU= 52.989
  12:29:27 STEP 1062 H= 265:30 +CPU= 51.299
  12:30:22 STEP 1063 H= 265:45 +CPU= 52.542
  12:31:16 STEP 1064 H= 266:00 +CPU= 53.553
  12:32:12 STEP 1065 H= 266:15 +CPU= 54.871
  12:32:58 STEP 1066 H= 266:30 +CPU= 44.885
  12:33:57 STEP 1067 H= 266:45 +CPU= 57.393
  12:35:01 STEP 1068 H= 267:00 +CPU= 63.071
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001068
  12:35:59 STEP 1069 H= 267:15 +CPU= 56.455
  12:36:54 STEP 1070 H= 267:30 +CPU= 54.319
  12:37:52 STEP 1071 H= 267:45 +CPU= 56.044
  12:38:49 STEP 1072 H= 268:00 +CPU= 55.324
  12:39:46 STEP 1073 H= 268:15 +CPU= 55.118
  12:40:40 STEP 1074 H= 268:30 +CPU= 52.775
  12:41:35 STEP 1075 H= 268:45 +CPU= 54.548
  12:42:33 STEP 1076 H= 269:00 +CPU= 56.326
  12:43:33 STEP 1077 H= 269:15 +CPU= 58.339
  12:44:39 STEP 1078 H= 269:30 +CPU= 63.692
  12:45:45 STEP 1079 H= 269:45 +CPU= 63.883
  12:47:13 STEP 1080 H= 270:00 +CPU= 83.980
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001080
  12:48:20 STEP 1081 H= 270:15 +CPU= 65.612
  12:49:30 STEP 1082 H= 270:30 +CPU= 67.091
  12:50:43 STEP 1083 H= 270:45 +CPU= 69.105
  12:51:54 STEP 1084 H= 271:00 +CPU= 67.676
  12:53:07 STEP 1085 H= 271:15 +CPU= 71.131
  12:54:21 STEP 1086 H= 271:30 +CPU= 70.296
  12:55:33 STEP 1087 H= 271:45 +CPU= 69.230
  12:56:48 STEP 1088 H= 272:00 +CPU= 69.070
  12:57:58 STEP 1089 H= 272:15 +CPU= 65.830
  12:59:13 STEP 1090 H= 272:30 +CPU= 68.029
  13:00:26 STEP 1091 H= 272:45 +CPU= 67.636
  13:01:52 STEP 1092 H= 273:00 +CPU= 80.195
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001092
  13:03:05 STEP 1093 H= 273:15 +CPU= 66.697
  13:04:18 STEP 1094 H= 273:30 +CPU= 67.479
  13:05:30 STEP 1095 H= 273:45 +CPU= 67.152
  13:06:43 STEP 1096 H= 274:00 +CPU= 69.273
  13:07:56 STEP 1097 H= 274:15 +CPU= 67.796
  13:09:10 STEP 1098 H= 274:30 +CPU= 68.874
  13:10:22 STEP 1099 H= 274:45 +CPU= 67.191
  13:11:35 STEP 1100 H= 275:00 +CPU= 68.144
  13:12:45 STEP 1101 H= 275:15 +CPU= 66.347
  13:13:59 STEP 1102 H= 275:30 +CPU= 67.894
  13:15:11 STEP 1103 H= 275:45 +CPU= 65.983
  13:16:53 STEP 1104 H= 276:00 +CPU= 91.041
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001104
  13:18:11 STEP 1105 H= 276:15 +CPU= 67.293
  13:19:25 STEP 1106 H= 276:30 +CPU= 65.621
  13:20:39 STEP 1107 H= 276:45 +CPU= 63.467
  13:21:50 STEP 1108 H= 277:00 +CPU= 64.416
  13:23:00 STEP 1109 H= 277:15 +CPU= 64.249
  13:24:08 STEP 1110 H= 277:30 +CPU= 62.474
  13:25:17 STEP 1111 H= 277:45 +CPU= 64.374
  13:26:27 STEP 1112 H= 278:00 +CPU= 65.676
  13:27:36 STEP 1113 H= 278:15 +CPU= 63.487
  13:28:44 STEP 1114 H= 278:30 +CPU= 64.142
  13:29:55 STEP 1115 H= 278:45 +CPU= 65.534
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  13:31:38 STEP 1116 H= 279:00 +CPU= 76.102
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001116
  13:32:48 STEP 1117 H= 279:15 +CPU= 64.682
  13:41:38 STEP 1118 H= 279:30 +CPU=141.952
  13:42:49 STEP 1119 H= 279:45 +CPU= 68.517
  13:44:00 STEP 1120 H= 280:00 +CPU= 69.541
  13:45:08 STEP 1121 H= 280:15 +CPU= 64.934
  13:46:16 STEP 1122 H= 280:30 +CPU= 66.549
  13:47:24 STEP 1123 H= 280:45 +CPU= 65.635
  13:48:30 STEP 1124 H= 281:00 +CPU= 63.952
  13:49:40 STEP 1125 H= 281:15 +CPU= 67.642
  13:50:49 STEP 1126 H= 281:30 +CPU= 66.187
  13:51:57 STEP 1127 H= 281:45 +CPU= 66.069
  13:53:14 STEP 1128 H= 282:00 +CPU= 74.858
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001128
  13:54:26 STEP 1129 H= 282:15 +CPU= 69.667
  13:55:40 STEP 1130 H= 282:30 +CPU= 69.795
  13:56:50 STEP 1131 H= 282:45 +CPU= 66.558
  13:58:03 STEP 1132 H= 283:00 +CPU= 69.351
  13:59:13 STEP 1133 H= 283:15 +CPU= 67.903
  14:00:24 STEP 1134 H= 283:30 +CPU= 68.053
  14:01:34 STEP 1135 H= 283:45 +CPU= 66.111
  14:02:45 STEP 1136 H= 284:00 +CPU= 67.767
  14:03:51 STEP 1137 H= 284:15 +CPU= 63.783
  14:05:01 STEP 1138 H= 284:30 +CPU= 65.832
  14:06:09 STEP 1139 H= 284:45 +CPU= 64.980
  14:07:33 STEP 1140 H= 285:00 +CPU= 80.938
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001140
  14:08:46 STEP 1141 H= 285:15 +CPU= 69.656
  14:09:53 STEP 1142 H= 285:30 +CPU= 63.883
  14:11:00 STEP 1143 H= 285:45 +CPU= 64.946
  14:12:12 STEP 1144 H= 286:00 +CPU= 68.436
  14:13:27 STEP 1145 H= 286:15 +CPU= 71.000
  14:14:35 STEP 1146 H= 286:30 +CPU= 65.359
  14:15:45 STEP 1147 H= 286:45 +CPU= 67.510
  14:17:01 STEP 1148 H= 287:00 +CPU= 71.682
  14:18:09 STEP 1149 H= 287:15 +CPU= 65.088
  14:19:21 STEP 1150 H= 287:30 +CPU= 69.666
  14:20:32 STEP 1151 H= 287:45 +CPU= 67.996
  14:22:11 STEP 1152 H= 288:00 +CPU= 93.324
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0_r1348356034_11.zip
Uploading the intermediate file: upload_file_11.zip
  14:23:21 STEP 1153 H= 288:15 +CPU= 65.572
  14:24:32 STEP 1154 H= 288:30 +CPU= 66.887
  14:25:42 STEP 1155 H= 288:45 +CPU= 66.702
  14:26:54 STEP 1156 H= 289:00 +CPU= 68.548
  14:28:02 STEP 1157 H= 289:15 +CPU= 65.468
  14:29:11 STEP 1158 H= 289:30 +CPU= 65.433
  14:30:17 STEP 1159 H= 289:45 +CPU= 62.259
  14:31:33 STEP 1160 H= 290:00 +CPU= 66.500
  14:32:46 STEP 1161 H= 290:15 +CPU= 66.738
  14:33:58 STEP 1162 H= 290:30 +CPU= 66.939
  14:35:11 STEP 1163 H= 290:45 +CPU= 67.649
  14:36:38 STEP 1164 H= 291:00 +CPU= 81.158
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001164
  14:37:50 STEP 1165 H= 291:15 +CPU= 67.674
  14:39:02 STEP 1166 H= 291:30 +CPU= 66.943
  14:40:16 STEP 1167 H= 291:45 +CPU= 67.799
  14:41:31 STEP 1168 H= 292:00 +CPU= 68.752
  14:42:43 STEP 1169 H= 292:15 +CPU= 67.035
  14:43:57 STEP 1170 H= 292:30 +CPU= 67.064
  14:45:07 STEP 1171 H= 292:45 +CPU= 65.361
  14:46:20 STEP 1172 H= 293:00 +CPU= 67.596
  14:47:36 STEP 1173 H= 293:15 +CPU= 67.305
  14:48:52 STEP 1174 H= 293:30 +CPU= 68.395
  14:50:09 STEP 1175 H= 293:45 +CPU= 70.718
  14:51:39 STEP 1176 H= 294:00 +CPU= 81.981
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001176
  14:52:49 STEP 1177 H= 294:15 +CPU= 66.655
  14:53:57 STEP 1178 H= 294:30 +CPU= 64.330
  14:55:02 STEP 1179 H= 294:45 +CPU= 63.220
  14:56:08 STEP 1180 H= 295:00 +CPU= 62.799
  14:57:12 STEP 1181 H= 295:15 +CPU= 61.979
  14:58:19 STEP 1182 H= 295:30 +CPU= 64.008
  14:59:20 STEP 1183 H= 295:45 +CPU= 58.985
  15:00:30 STEP 1184 H= 296:00 +CPU= 65.845
  15:01:37 STEP 1185 H= 296:15 +CPU= 63.284
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:03:01 STEP 1186 H= 296:30 +CPU= 61.114
  15:04:08 STEP 1187 H= 296:45 +CPU= 63.725
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:05:47 STEP 1188 H= 297:00 +CPU= 74.991
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001188
  15:06:59 STEP 1189 H= 297:15 +CPU= 68.398
  15:08:09 STEP 1190 H= 297:30 +CPU= 67.422
  15:09:19 STEP 1191 H= 297:45 +CPU= 67.117
  15:10:31 STEP 1192 H= 298:00 +CPU= 67.786
  15:11:41 STEP 1193 H= 298:15 +CPU= 66.251
  15:12:51 STEP 1194 H= 298:30 +CPU= 67.231
  15:14:03 STEP 1195 H= 298:45 +CPU= 67.558
  15:15:16 STEP 1196 H= 299:00 +CPU= 69.402
  15:16:26 STEP 1197 H= 299:15 +CPU= 66.517
  15:17:36 STEP 1198 H= 299:30 +CPU= 65.834
  15:18:44 STEP 1199 H= 299:45 +CPU= 65.524
  15:20:18 STEP 1200 H= 300:00 +CPU= 89.210
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001200
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:21:47 STEP 1201 H= 300:15 +CPU= 66.579
  15:22:56 STEP 1202 H= 300:30 +CPU= 66.311
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:24:22 STEP 1203 H= 300:45 +CPU= 63.418
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:25:45 STEP 1204 H= 301:00 +CPU= 61.124
  15:26:44 STEP 1205 H= 301:15 +CPU= 58.156
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:28:03 STEP 1206 H= 301:30 +CPU= 56.729
  15:28:57 STEP 1207 H= 301:45 +CPU= 53.055
  15:30:06 STEP 1208 H= 302:00 +CPU= 67.858
  15:31:05 STEP 1209 H= 302:15 +CPU= 56.671
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:32:15 STEP 1210 H= 302:30 +CPU= 49.041
  15:33:18 STEP 1211 H= 302:45 +CPU= 62.645
  15:34:29 STEP 1212 H= 303:00 +CPU= 70.367
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001212
  15:35:28 STEP 1213 H= 303:15 +CPU= 56.809
  15:36:31 STEP 1214 H= 303:30 +CPU= 59.109
  15:37:27 STEP 1215 H= 303:45 +CPU= 54.269
  15:38:26 STEP 1216 H= 304:00 +CPU= 56.065
  15:39:15 STEP 1217 H= 304:15 +CPU= 47.397
  15:40:09 STEP 1218 H= 304:30 +CPU= 51.746
  15:41:05 STEP 1219 H= 304:45 +CPU= 53.124
  15:42:02 STEP 1220 H= 305:00 +CPU= 55.056
  15:42:57 STEP 1221 H= 305:15 +CPU= 52.788
  15:43:54 STEP 1222 H= 305:30 +CPU= 55.410
  15:44:48 STEP 1223 H= 305:45 +CPU= 50.317
  15:45:52 STEP 1224 H= 306:00 +CPU= 61.391
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001224
  15:46:48 STEP 1225 H= 306:15 +CPU= 53.691
  15:47:40 STEP 1226 H= 306:30 +CPU= 50.940
  15:48:34 STEP 1227 H= 306:45 +CPU= 52.319
  15:49:31 STEP 1228 H= 307:00 +CPU= 54.538
  15:50:31 STEP 1229 H= 307:15 +CPU= 56.009
  15:51:26 STEP 1230 H= 307:30 +CPU= 53.374
  15:52:25 STEP 1231 H= 307:45 +CPU= 55.368
  15:53:22 STEP 1232 H= 308:00 +CPU= 55.057
  15:54:15 STEP 1233 H= 308:15 +CPU= 50.558
  15:55:11 STEP 1234 H= 308:30 +CPU= 53.118
  15:56:03 STEP 1235 H= 308:45 +CPU= 49.291
  15:57:07 STEP 1236 H= 309:00 +CPU= 62.365
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001236
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:58:20 STEP 1237 H= 309:15 +CPU= 49.881
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  15:59:40 STEP 1238 H= 309:30 +CPU= 56.299
  16:00:39 STEP 1239 H= 309:45 +CPU= 55.883
  16:01:26 STEP 1240 H= 310:00 +CPU= 46.284
  16:02:22 STEP 1241 H= 310:15 +CPU= 54.106
  16:03:11 STEP 1242 H= 310:30 +CPU= 47.152
  16:04:15 STEP 1243 H= 310:45 +CPU= 60.010
  16:05:15 STEP 1244 H= 311:00 +CPU= 57.945
  16:06:07 STEP 1245 H= 311:15 +CPU= 49.727
  16:07:08 STEP 1246 H= 311:30 +CPU= 58.734
  16:08:09 STEP 1247 H= 311:45 +CPU= 58.427
  16:09:29 STEP 1248 H= 312:00 +CPU= 75.202
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0_r1348356034_12.zip
Uploading the intermediate file: upload_file_12.zip
  16:10:26 STEP 1249 H= 312:15 +CPU= 53.220
  16:11:26 STEP 1250 H= 312:30 +CPU= 57.089
  16:12:20 STEP 1251 H= 312:45 +CPU= 52.953
  16:13:18 STEP 1252 H= 313:00 +CPU= 54.908
  16:14:13 STEP 1253 H= 313:15 +CPU= 52.453
  16:15:12 STEP 1254 H= 313:30 +CPU= 55.521
  16:16:10 STEP 1255 H= 313:45 +CPU= 55.202
  16:17:13 STEP 1256 H= 314:00 +CPU= 60.621
  16:18:11 STEP 1257 H= 314:15 +CPU= 55.129
  16:19:05 STEP 1258 H= 314:30 +CPU= 52.292
  16:19:59 STEP 1259 H= 314:45 +CPU= 52.106
  16:21:01 STEP 1260 H= 315:00 +CPU= 59.428
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001260
  16:21:51 STEP 1261 H= 315:15 +CPU= 48.109
  16:22:43 STEP 1262 H= 315:30 +CPU= 50.802
  16:23:33 STEP 1263 H= 315:45 +CPU= 48.177
  16:24:24 STEP 1264 H= 316:00 +CPU= 48.972
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:25:33 STEP 1265 H= 316:15 +CPU= 47.668
  16:26:23 STEP 1266 H= 316:30 +CPU= 48.895
  16:27:19 STEP 1267 H= 316:45 +CPU= 54.437
  16:28:09 STEP 1268 H= 317:00 +CPU= 48.666
  16:29:02 STEP 1269 H= 317:15 +CPU= 51.097
  16:29:56 STEP 1270 H= 317:30 +CPU= 52.503
  16:30:50 STEP 1271 H= 317:45 +CPU= 51.235
  16:31:58 STEP 1272 H= 318:00 +CPU= 65.859
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001272
  16:32:49 STEP 1273 H= 318:15 +CPU= 49.001
  16:33:37 STEP 1274 H= 318:30 +CPU= 47.187
  16:34:28 STEP 1275 H= 318:45 +CPU= 49.767
  16:35:29 STEP 1276 H= 319:00 +CPU= 58.015
  16:36:27 STEP 1277 H= 319:15 +CPU= 55.484
  16:37:17 STEP 1278 H= 319:30 +CPU= 48.636
  16:38:11 STEP 1279 H= 319:45 +CPU= 52.212
  16:39:04 STEP 1280 H= 320:00 +CPU= 50.116
  16:39:51 STEP 1281 H= 320:15 +CPU= 46.262
  16:40:45 STEP 1282 H= 320:30 +CPU= 51.366
  16:41:37 STEP 1283 H= 320:45 +CPU= 50.367
  16:42:36 STEP 1284 H= 321:00 +CPU= 56.429
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001284
  16:43:22 STEP 1285 H= 321:15 +CPU= 45.213
  16:44:06 STEP 1286 H= 321:30 +CPU= 42.061
  16:44:45 STEP 1287 H= 321:45 +CPU= 38.151
  16:45:34 STEP 1288 H= 322:00 +CPU= 47.928
  16:46:20 STEP 1289 H= 322:15 +CPU= 44.699
  16:46:59 STEP 1290 H= 322:30 +CPU= 37.424
  16:47:34 STEP 1291 H= 322:45 +CPU= 34.298
  16:48:22 STEP 1292 H= 323:00 +CPU= 46.402
  16:48:58 STEP 1293 H= 323:15 +CPU= 34.772
  16:49:46 STEP 1294 H= 323:30 +CPU= 47.018
  16:50:28 STEP 1295 H= 323:45 +CPU= 39.772
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:51:39 STEP 1296 H= 324:00 +CPU= 49.897
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001296
  16:52:07 STEP 1297 H= 324:15 +CPU= 27.223
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:53:20 STEP 1298 H= 324:30 +CPU= 31.134
  16:53:57 STEP 1299 H= 324:45 +CPU= 35.452
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:54:45 STEP 1300 H= 325:00 +CPU= 27.527
  16:55:15 STEP 1301 H= 325:15 +CPU= 29.240
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:56:22 STEP 1302 H= 325:30 +CPU= 25.181
  16:56:52 STEP 1303 H= 325:45 +CPU= 29.416
  16:57:21 STEP 1304 H= 326:00 +CPU= 28.329
  16:57:54 STEP 1305 H= 326:15 +CPU= 32.696
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  16:58:47 STEP 1306 H= 326:30 +CPU= 31.423
  16:59:21 STEP 1307 H= 326:45 +CPU= 34.043
  17:00:01 STEP 1308 H= 327:00 +CPU= 39.198
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001308
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:00:49 STEP 1309 H= 327:15 +CPU= 26.911
  17:01:19 STEP 1310 H= 327:30 +CPU= 29.095
  17:01:47 STEP 1311 H= 327:45 +CPU= 28.477
  17:02:17 STEP 1312 H= 328:00 +CPU= 29.288
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:03:13 STEP 1313 H= 328:15 +CPU= 34.552
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:04:04 STEP 1314 H= 328:30 +CPU= 29.147
  17:04:36 STEP 1315 H= 328:45 +CPU= 30.661
  17:05:10 STEP 1316 H= 329:00 +CPU= 33.036
  17:05:41 STEP 1317 H= 329:15 +CPU= 30.635
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:06:37 STEP 1318 H= 329:30 +CPU= 34.013
  17:07:10 STEP 1319 H= 329:45 +CPU= 32.305
  17:07:53 STEP 1320 H= 330:00 +CPU= 41.132
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001320
  17:08:25 STEP 1321 H= 330:15 +CPU= 30.753
  17:08:59 STEP 1322 H= 330:30 +CPU= 33.309
  17:09:27 STEP 1323 H= 330:45 +CPU= 27.150
  17:09:54 STEP 1324 H= 331:00 +CPU= 26.110
  17:10:24 STEP 1325 H= 331:15 +CPU= 29.120
  17:10:49 STEP 1326 H= 331:30 +CPU= 25.066
  17:11:15 STEP 1327 H= 331:45 +CPU= 24.982
  17:11:49 STEP 1328 H= 332:00 +CPU= 32.190
  17:12:19 STEP 1329 H= 332:15 +CPU= 30.244
  17:12:48 STEP 1330 H= 332:30 +CPU= 27.445
  17:13:16 STEP 1331 H= 332:45 +CPU= 27.277
  17:13:50 STEP 1332 H= 333:00 +CPU= 33.684
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001332
  17:14:17 STEP 1333 H= 333:15 +CPU= 26.094
  17:14:44 STEP 1334 H= 333:30 +CPU= 26.823
  17:15:15 STEP 1335 H= 333:45 +CPU= 30.036
  17:15:49 STEP 1336 H= 334:00 +CPU= 32.801
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:16:39 STEP 1337 H= 334:15 +CPU= 28.577
  17:17:04 STEP 1338 H= 334:30 +CPU= 24.324
  17:17:31 STEP 1339 H= 334:45 +CPU= 25.738
  17:17:57 STEP 1340 H= 335:00 +CPU= 25.575
  17:18:28 STEP 1341 H= 335:15 +CPU= 30.675
  17:18:57 STEP 1342 H= 335:30 +CPU= 28.865
  17:19:26 STEP 1343 H= 335:45 +CPU= 27.670
  17:20:05 STEP 1344 H= 336:00 +CPU= 38.209
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0_r1348356034_13.zip
Uploading the intermediate file: upload_file_13.zip
  17:20:35 STEP 1345 H= 336:15 +CPU= 28.056
  17:21:02 STEP 1346 H= 336:30 +CPU= 25.952
  17:21:29 STEP 1347 H= 336:45 +CPU= 26.677
  17:21:57 STEP 1348 H= 337:00 +CPU= 27.675
  17:22:25 STEP 1349 H= 337:15 +CPU= 26.948
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:23:12 STEP 1350 H= 337:30 +CPU= 25.043
  17:23:42 STEP 1351 H= 337:45 +CPU= 29.490
  17:24:13 STEP 1352 H= 338:00 +CPU= 29.961
  17:24:43 STEP 1353 H= 338:15 +CPU= 29.517
  17:25:13 STEP 1354 H= 338:30 +CPU= 29.340
  17:25:41 STEP 1355 H= 338:45 +CPU= 27.631
  17:26:14 STEP 1356 H= 339:00 +CPU= 32.110
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001356
  17:26:44 STEP 1357 H= 339:15 +CPU= 29.313
  17:27:14 STEP 1358 H= 339:30 +CPU= 29.476
  17:27:45 STEP 1359 H= 339:45 +CPU= 30.776
  17:28:12 STEP 1360 H= 340:00 +CPU= 25.863
  17:28:42 STEP 1361 H= 340:15 +CPU= 29.214
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:29:31 STEP 1362 H= 340:30 +CPU= 28.028
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:30:17 STEP 1363 H= 340:45 +CPU= 24.426
  17:30:47 STEP 1364 H= 341:00 +CPU= 28.938
  17:31:18 STEP 1365 H= 341:15 +CPU= 29.491
  17:31:47 STEP 1366 H= 341:30 +CPU= 27.661
  17:32:17 STEP 1367 H= 341:45 +CPU= 29.385
  17:32:54 STEP 1368 H= 342:00 +CPU= 35.978
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001368
  17:33:18 STEP 1369 H= 342:15 +CPU= 23.714
  17:33:47 STEP 1370 H= 342:30 +CPU= 28.226
  17:34:15 STEP 1371 H= 342:45 +CPU= 27.628
  17:34:42 STEP 1372 H= 343:00 +CPU= 26.617
  17:35:08 STEP 1373 H= 343:15 +CPU= 25.476
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:35:54 STEP 1374 H= 343:30 +CPU= 24.831
  17:36:23 STEP 1375 H= 343:45 +CPU= 27.839
  17:36:51 STEP 1376 H= 344:00 +CPU= 28.283
  17:37:21 STEP 1377 H= 344:15 +CPU= 28.441
  17:37:48 STEP 1378 H= 344:30 +CPU= 26.259
  17:38:17 STEP 1379 H= 344:45 +CPU= 28.620
  17:38:53 STEP 1380 H= 345:00 +CPU= 34.672
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001380
  17:39:17 STEP 1381 H= 345:15 +CPU= 23.743
  17:39:43 STEP 1382 H= 345:30 +CPU= 25.359
  17:40:10 STEP 1383 H= 345:45 +CPU= 27.059
  17:40:38 STEP 1384 H= 346:00 +CPU= 27.499
  17:41:06 STEP 1385 H= 346:15 +CPU= 26.748
  17:41:33 STEP 1386 H= 346:30 +CPU= 26.893
  17:42:01 STEP 1387 H= 346:45 +CPU= 27.193
  17:42:28 STEP 1388 H= 347:00 +CPU= 26.435
  17:42:57 STEP 1389 H= 347:15 +CPU= 28.331
  17:43:24 STEP 1390 H= 347:30 +CPU= 26.215
  17:43:54 STEP 1391 H= 347:45 +CPU= 29.861
  17:44:34 STEP 1392 H= 348:00 +CPU= 38.960
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001392
  17:45:06 STEP 1393 H= 348:15 +CPU= 31.690
  17:45:34 STEP 1394 H= 348:30 +CPU= 27.557
  17:46:05 STEP 1395 H= 348:45 +CPU= 29.546
  17:46:35 STEP 1396 H= 349:00 +CPU= 29.404
  17:47:05 STEP 1397 H= 349:15 +CPU= 28.463
  17:47:34 STEP 1398 H= 349:30 +CPU= 27.986
  17:48:08 STEP 1399 H= 349:45 +CPU= 33.463
  17:48:37 STEP 1400 H= 350:00 +CPU= 27.487
  17:49:03 STEP 1401 H= 350:15 +CPU= 25.875
  17:49:32 STEP 1402 H= 350:30 +CPU= 27.564
  17:50:00 STEP 1403 H= 350:45 +CPU= 28.024
  17:50:35 STEP 1404 H= 351:00 +CPU= 33.520
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001404
  17:51:03 STEP 1405 H= 351:15 +CPU= 27.376
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:51:56 STEP 1406 H= 351:30 +CPU= 30.246
  17:52:27 STEP 1407 H= 351:45 +CPU= 29.848
  17:52:56 STEP 1408 H= 352:00 +CPU= 28.800
  17:53:28 STEP 1409 H= 352:15 +CPU= 30.917
  17:53:59 STEP 1410 H= 352:30 +CPU= 30.682
  17:54:28 STEP 1411 H= 352:45 +CPU= 28.479
  17:54:58 STEP 1412 H= 353:00 +CPU= 29.178
  17:55:28 STEP 1413 H= 353:15 +CPU= 28.834
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  17:56:17 STEP 1414 H= 353:30 +CPU= 27.584
  17:56:48 STEP 1415 H= 353:45 +CPU= 29.902
  17:57:24 STEP 1416 H= 354:00 +CPU= 35.835
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001416
  17:57:53 STEP 1417 H= 354:15 +CPU= 28.011
  17:58:21 STEP 1418 H= 354:30 +CPU= 27.437
  17:58:49 STEP 1419 H= 354:45 +CPU= 26.907
  17:59:21 STEP 1420 H= 355:00 +CPU= 31.660
  17:59:50 STEP 1421 H= 355:15 +CPU= 27.868
  18:00:18 STEP 1422 H= 355:30 +CPU= 27.252
  18:00:47 STEP 1423 H= 355:45 +CPU= 27.683
  18:01:13 STEP 1424 H= 356:00 +CPU= 26.022
  18:01:40 STEP 1425 H= 356:15 +CPU= 26.403
  18:02:09 STEP 1426 H= 356:30 +CPU= 28.143
  18:02:38 STEP 1427 H= 356:45 +CPU= 28.592
  18:03:13 STEP 1428 H= 357:00 +CPU= 33.850
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001428
  18:03:40 STEP 1429 H= 357:15 +CPU= 26.530
  18:04:08 STEP 1430 H= 357:30 +CPU= 27.892
  18:04:36 STEP 1431 H= 357:45 +CPU= 27.217
  18:05:04 STEP 1432 H= 358:00 +CPU= 27.278
  18:05:31 STEP 1433 H= 358:15 +CPU= 26.510
  18:05:59 STEP 1434 H= 358:30 +CPU= 26.727
  18:06:29 STEP 1435 H= 358:45 +CPU= 29.304
  18:07:00 STEP 1436 H= 359:00 +CPU= 30.197
  18:07:26 STEP 1437 H= 359:15 +CPU= 25.708
  18:07:53 STEP 1438 H= 359:30 +CPU= 26.584
  18:08:22 STEP 1439 H= 359:45 +CPU= 27.618
  18:08:57 STEP 1440 H= 360:00 +CPU= 34.393
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I     -0.1879718801E-20      I       0.2384897334E-07      I
ICloud fractionI      0.0000000000E+00      I       0.4661236342E-01      I
I  Relat. Hum. I      0.1671856472E+00      I       0.2226597597E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.1712280381E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    296262369 BYTES IN      7 RECORDS TRANSFERRED IN    0.0010 SECONDS296262.4 Mbytes/s, TOTAL TIME=    0.0131( 7.6%)
 18:08:57 STEP 1440 H= 360:00 +CPU= 34.393
 END CNT3
 NSTEP =  1440 CNT0   000000000  23.387  23.387  23.767   0.984               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN        12762/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    228    0       34      1044068    12762 ************    46.3     4.7    41.6     7.6    85.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.96      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  469   11125.0   11125.0     22.33      0.00
   9 SPCM     - SPECTRAL COMP.                     432    2356.7    2356.7      4.36      0.00
  10 SCAN2M - PHYSICS                              433   21299.5   21299.5     39.48      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 36    1822.2    1822.2      0.28      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 19     114.1     114.1      0.01      0.00
  17 GRIDFPOS IN CNT4                               36      49.8      49.8      0.01      0.00
  27 RESTART FILES - WRITING                         9    7287.5    7287.5      0.28      0.00
  28 RESTART FILES - READING                         1   10134.8   10134.8      0.04      0.00
  29 SU4FPOS IN CNT4                                36       0.4       0.4      0.00      0.00
  30 DYNFPOS IN CNT4                                36   10285.3   10285.3      1.58      0.00
  31 POSDDH IN STEPO                                 2       2.5       2.5      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    433   10390.4   10390.4     19.26      0.00
  39 SU0YOMB                                         1     181.6     181.6      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    433      48.6      48.6      0.09      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              432    1133.3    1133.3      2.10      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              432     181.6     181.6      0.34      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 19       1.0       1.0      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM      940     744.2     744.2      2.99      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM       900     380.1     380.1      1.46      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM        900     144.6     144.6      0.56      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM       940     716.4     716.4      2.88      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      20.1      20.1      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION            940     179.0     179.0      0.72      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION            900      94.6      94.6      0.36      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION            940     200.7     200.7      0.81      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION            900      87.5      87.5      0.34      0.00
 400 GSTATS                                      50912       8.9       8.9      1.93      0.00
 401 GSTATS HOOK                                 47318       0.0       0.0      0.01      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    23362.6 CPU TIME   21431.3 VECTOR TIME    21431.3


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       55.5
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc/slots/21/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/21/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc/slots/21/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc/slots/21/NODE.001_01
Adding to the zip: /var/lib/boinc/slots/21/ifs.stat
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_12210662/ICMUAh7zg+001440
Zipping up the final file: /var/lib/boinc/projects/climateprediction.net/oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0_r1348356034_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
18:12:26 (8842): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:51:43 1533094 22310884 oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0 1,295,100 91,767 0.0709
09 May 2023 16:51:38 1533094 22310884 oifs_43r3_bl_a2zn_2016092300_15_991_12210662_0 864,000 65,144 0.0754


©2024 climateprediction.net