climateprediction.net home page
Task 22309914

Task 22309914

Name oifs_43r3_bl_a29b_2016092300_15_991_12209714_0
Workunit 12209714
Created 13 Feb 2023, 17:23:51 UTC
Sent 13 Feb 2023, 17:29:39 UTC
Report deadline 14 Apr 2023, 17:29:39 UTC
Received 15 Feb 2023, 0:15:30 UTC
Server state Over
Outcome Success
Client state Done
Exit status 0 (0x00000000)
Computer ID 1536331
Run time 12 hours 25 min 45 sec
CPU time 12 hours 25 min 45 sec
Validate state Valid
Credit 1,813.14
Device peak FLOPS 6.13 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.11
x86_64-pc-linux-gnu
Peak working set size 5,074.93 MB
Peak swap size 5,930.79 MB
Peak disk usage 1,590.08 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<stderr_txt>
       GP_MODEL 
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210100:1676404860:5616.119] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210100:1676404860:5616.119] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS CLDPP T/S    (1002) 
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210100:1676404860:5616.119] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210100:1676404860:5616.119] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210100:1676404860:5616.119] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                CLOUD_LAYER 
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210100:1676404860:5616.119] [c_drhook_print_@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:3897] :                 CLOUDSC 
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210101:1676404861:5617.123] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:W32Ub22:1:1:1776:1776] [20230214:210101:1676404861:5617.123] [signal_drhook@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x401140 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  000000000081FCF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8BB20  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000011F8636  cloudsc_                 2994  cloudsc.F90
oifs_43r3_model.e  00000000011D9EE7  cloud_layer_              131  cloud_layer.F90
oifs_43r3_model.e  00000000011730D0  callpar_                 1223  callpar.F90
oifs_43r3_model.e  00000000010B682F  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  00000000010A915C  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000E680C3  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000015C1E62  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  00000000015AC40E  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  00000000006620B4  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004A0E7C  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  000000000049DC29  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  000000000049CE72  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049CD68  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  00000000004078AD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040276F  MAIN__                     96  master.F90
oifs_43r3_model.e  000000000040270D  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D8287A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001D840D7  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000004025E5  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.11_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a29b
(argv4) batchid: 991
(argv5) wuid: 12209714
(argv6) fclen: 15
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/7
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.11
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/7/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/7/oifs_43r3_bl_app_1.11_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_0bfe35727062412ecd7c2b0b913b1531 to: /var/lib/boinc-client/slots/7/oifs_43r3_bl_a29b_2016092300_15_991_12209714.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/7/oifs_43r3_bl_a29b_2016092300_15_991_12209714.zip
ic_ancil_file: ic_ancil_12209714
ifsdata_file: ifsdata_12209714
climate_data_file: clim_data_12209714
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/7/ic_ancil_12209714.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/7/ic_ancil_12209714.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_90d6b49b57b0b8fd641aa3c31912fde1 to: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12209714.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/7/ifsdata/ifsdata_12209714.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/7/159l_2/clim_data_12209714.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/7/159l_2/clim_data_12209714.zip
Checking for progress XML file: /var/lib/boinc-client/slots/7/progress_file_12209714.xml
Opened progress file ok : /var/lib/boinc-client/slots/7/progress_file_12209714.xml
-- Model is restarting --
Adjusting last_iter, 996, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/7/progress_file_12209714.xml
last_cpu_time: 30221
upload_file_number: 10
last_iter: 961
last_upload: 864000
model_completed: 0
total_length_of_simulation: 1296000
result_base_name: oifs_43r3_bl_a29b_2016092300_15_991_12209714_0_r1988314655
The child process has been launched with process id: 1786
Executing the command: /var/lib/boinc-client/slots/7/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2091] fp = 0x30e4a00
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [process_options@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [ignore_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [restore_default_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f520 (old at (nil))
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f520 (old at (nil))
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [trapfpe_treatment@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f520 (old at (nil))
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f520 (old at 0x401140)
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f520 (old at (nil))
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f520 (old at (nil))
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [signal_drhook_init@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f520 (old at (nil))
[EC_DRHOOK:W32Ub22:1:1:1786:1786] [20230214:210232:1676404952:0.000] [catch_signals@/home/abowery/Desktop/OpenIFS/oifs_43r3_bl/gc_oifs43r3-feature-cpdn/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  21:03:31 STEP  960 H= 240:00 +CPU= 41.170
  21:04:01 STEP  961 H= 240:15 +CPU= 30.479
  21:04:31 STEP  962 H= 240:30 +CPU= 29.944
  21:05:01 STEP  963 H= 240:45 +CPU= 30.199
  21:05:31 STEP  964 H= 241:00 +CPU= 30.056
  21:06:01 STEP  965 H= 241:15 +CPU= 29.607
  21:06:31 STEP  966 H= 241:30 +CPU= 29.640
  21:07:00 STEP  967 H= 241:45 +CPU= 29.692
  21:07:31 STEP  968 H= 242:00 +CPU= 30.331
  21:08:01 STEP  969 H= 242:15 +CPU= 30.069
  21:08:31 STEP  970 H= 242:30 +CPU= 30.001
  21:09:01 STEP  971 H= 242:45 +CPU= 29.747
  21:09:37 STEP  972 H= 243:00 +CPU= 36.229
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000972
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000972
  21:10:07 STEP  973 H= 243:15 +CPU= 30.106
  21:10:37 STEP  974 H= 243:30 +CPU= 30.409
  21:11:07 STEP  975 H= 243:45 +CPU= 29.914
  21:11:38 STEP  976 H= 244:00 +CPU= 30.491
  21:12:08 STEP  977 H= 244:15 +CPU= 30.134
  21:12:38 STEP  978 H= 244:30 +CPU= 29.962
  21:13:08 STEP  979 H= 244:45 +CPU= 30.016
  21:13:39 STEP  980 H= 245:00 +CPU= 30.842
  21:14:09 STEP  981 H= 245:15 +CPU= 30.189
  21:14:40 STEP  982 H= 245:30 +CPU= 30.346
  21:15:09 STEP  983 H= 245:45 +CPU= 29.771
  21:15:46 STEP  984 H= 246:00 +CPU= 36.414
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000984
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000984
  21:16:16 STEP  985 H= 246:15 +CPU= 30.468
  21:16:46 STEP  986 H= 246:30 +CPU= 29.986
  21:17:16 STEP  987 H= 246:45 +CPU= 29.799
  21:17:47 STEP  988 H= 247:00 +CPU= 30.523
  21:18:17 STEP  989 H= 247:15 +CPU= 29.997
  21:18:46 STEP  990 H= 247:30 +CPU= 29.781
  21:19:17 STEP  991 H= 247:45 +CPU= 30.476
  21:19:47 STEP  992 H= 248:00 +CPU= 30.349
  21:20:17 STEP  993 H= 248:15 +CPU= 29.739
  21:20:47 STEP  994 H= 248:30 +CPU= 29.640
  21:21:17 STEP  995 H= 248:45 +CPU= 30.090
  21:21:53 STEP  996 H= 249:00 +CPU= 36.366
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+000996
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+000996
  21:22:24 STEP  997 H= 249:15 +CPU= 30.526
  21:22:54 STEP  998 H= 249:30 +CPU= 30.151
  21:23:24 STEP  999 H= 249:45 +CPU= 30.043
  21:23:56 STEP 1000 H= 250:00 +CPU= 32.228
  21:24:26 STEP 1001 H= 250:15 +CPU= 29.846
  21:24:56 STEP 1002 H= 250:30 +CPU= 29.921
  21:25:26 STEP 1003 H= 250:45 +CPU= 30.011
  21:25:56 STEP 1004 H= 251:00 +CPU= 30.195
  21:26:27 STEP 1005 H= 251:15 +CPU= 30.128
  21:26:56 STEP 1006 H= 251:30 +CPU= 29.685
  21:27:26 STEP 1007 H= 251:45 +CPU= 29.855
  21:28:12 STEP 1008 H= 252:00 +CPU= 45.543
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001008
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001008
  21:28:44 STEP 1009 H= 252:15 +CPU= 32.541
  21:29:15 STEP 1010 H= 252:30 +CPU= 30.561
  21:29:45 STEP 1011 H= 252:45 +CPU= 30.352
  21:30:16 STEP 1012 H= 253:00 +CPU= 31.094
  21:30:47 STEP 1013 H= 253:15 +CPU= 30.388
  21:31:17 STEP 1014 H= 253:30 +CPU= 30.335
  21:31:47 STEP 1015 H= 253:45 +CPU= 30.210
  21:32:18 STEP 1016 H= 254:00 +CPU= 30.496
  21:32:48 STEP 1017 H= 254:15 +CPU= 30.297
  21:33:19 STEP 1018 H= 254:30 +CPU= 30.350
  21:33:49 STEP 1019 H= 254:45 +CPU= 30.224
  21:34:26 STEP 1020 H= 255:00 +CPU= 36.921
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001020
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001020
  21:34:56 STEP 1021 H= 255:15 +CPU= 29.957
  21:35:26 STEP 1022 H= 255:30 +CPU= 29.903
  21:35:56 STEP 1023 H= 255:45 +CPU= 30.103
  21:36:26 STEP 1024 H= 256:00 +CPU= 30.462
  21:36:57 STEP 1025 H= 256:15 +CPU= 30.862
  21:37:27 STEP 1026 H= 256:30 +CPU= 30.324
  21:37:58 STEP 1027 H= 256:45 +CPU= 30.188
  21:38:29 STEP 1028 H= 257:00 +CPU= 30.782
  21:38:58 STEP 1029 H= 257:15 +CPU= 29.717
  21:39:28 STEP 1030 H= 257:30 +CPU= 29.975
  21:39:58 STEP 1031 H= 257:45 +CPU= 29.916
  21:40:35 STEP 1032 H= 258:00 +CPU= 37.070
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001032
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001032
  21:41:06 STEP 1033 H= 258:15 +CPU= 30.297
  21:41:36 STEP 1034 H= 258:30 +CPU= 30.188
  21:42:06 STEP 1035 H= 258:45 +CPU= 30.119
  21:42:37 STEP 1036 H= 259:00 +CPU= 30.512
  21:43:07 STEP 1037 H= 259:15 +CPU= 30.552
  21:43:37 STEP 1038 H= 259:30 +CPU= 30.051
  21:44:08 STEP 1039 H= 259:45 +CPU= 30.322
  21:44:38 STEP 1040 H= 260:00 +CPU= 30.363
  21:45:08 STEP 1041 H= 260:15 +CPU= 29.954
  21:45:38 STEP 1042 H= 260:30 +CPU= 29.809
  21:46:08 STEP 1043 H= 260:45 +CPU= 30.037
  21:46:45 STEP 1044 H= 261:00 +CPU= 36.739
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001044
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001044
  21:47:15 STEP 1045 H= 261:15 +CPU= 30.178
  21:47:45 STEP 1046 H= 261:30 +CPU= 30.090
  21:48:15 STEP 1047 H= 261:45 +CPU= 30.029
  21:48:46 STEP 1048 H= 262:00 +CPU= 30.879
  21:49:16 STEP 1049 H= 262:15 +CPU= 30.067
  21:49:46 STEP 1050 H= 262:30 +CPU= 30.245
  21:50:17 STEP 1051 H= 262:45 +CPU= 30.729
  21:50:48 STEP 1052 H= 263:00 +CPU= 30.974
  21:51:18 STEP 1053 H= 263:15 +CPU= 30.343
  21:51:48 STEP 1054 H= 263:30 +CPU= 29.674
  21:52:18 STEP 1055 H= 263:45 +CPU= 30.286
  21:53:05 STEP 1056 H= 264:00 +CPU= 46.422
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001056
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+000960
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+000972
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+000984
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+000996
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001008
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001020
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001032
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001044
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001044
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29b_2016092300_15_991_12209714_0_r1988314655_10.zip
Uploading the intermediate file: upload_file_10.zip
  21:53:41 STEP 1057 H= 264:15 +CPU= 36.447
  21:54:12 STEP 1058 H= 264:30 +CPU= 31.138
  21:54:44 STEP 1059 H= 264:45 +CPU= 31.319
  21:55:14 STEP 1060 H= 265:00 +CPU= 30.497
  21:55:45 STEP 1061 H= 265:15 +CPU= 30.534
  21:56:15 STEP 1062 H= 265:30 +CPU= 30.478
  21:56:46 STEP 1063 H= 265:45 +CPU= 31.069
  21:57:17 STEP 1064 H= 266:00 +CPU= 30.987
  21:57:48 STEP 1065 H= 266:15 +CPU= 30.815
  21:58:20 STEP 1066 H= 266:30 +CPU= 31.992
  21:58:50 STEP 1067 H= 266:45 +CPU= 30.032
  21:59:27 STEP 1068 H= 267:00 +CPU= 36.757
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001068
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001068
  21:59:57 STEP 1069 H= 267:15 +CPU= 30.397
  22:00:28 STEP 1070 H= 267:30 +CPU= 30.258
  22:00:58 STEP 1071 H= 267:45 +CPU= 30.244
  22:01:29 STEP 1072 H= 268:00 +CPU= 31.321
  22:02:00 STEP 1073 H= 268:15 +CPU= 30.808
  22:02:31 STEP 1074 H= 268:30 +CPU= 30.396
  22:03:01 STEP 1075 H= 268:45 +CPU= 30.338
  22:03:32 STEP 1076 H= 269:00 +CPU= 30.582
  22:04:02 STEP 1077 H= 269:15 +CPU= 30.184
  22:04:32 STEP 1078 H= 269:30 +CPU= 30.539
  22:05:03 STEP 1079 H= 269:45 +CPU= 30.457
  22:05:40 STEP 1080 H= 270:00 +CPU= 36.683
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001080
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001080
  22:06:10 STEP 1081 H= 270:15 +CPU= 30.497
  22:06:40 STEP 1082 H= 270:30 +CPU= 30.015
  22:07:10 STEP 1083 H= 270:45 +CPU= 30.048
  22:07:41 STEP 1084 H= 271:00 +CPU= 30.871
  22:08:11 STEP 1085 H= 271:15 +CPU= 30.073
  22:08:42 STEP 1086 H= 271:30 +CPU= 30.375
  22:09:12 STEP 1087 H= 271:45 +CPU= 30.288
  22:09:43 STEP 1088 H= 272:00 +CPU= 30.741
  22:10:13 STEP 1089 H= 272:15 +CPU= 30.443
  22:10:43 STEP 1090 H= 272:30 +CPU= 30.073
  22:11:13 STEP 1091 H= 272:45 +CPU= 29.945
  22:11:50 STEP 1092 H= 273:00 +CPU= 36.584
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001092
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001092
  22:12:20 STEP 1093 H= 273:15 +CPU= 30.462
  22:12:51 STEP 1094 H= 273:30 +CPU= 30.790
  22:13:21 STEP 1095 H= 273:45 +CPU= 30.246
  22:13:52 STEP 1096 H= 274:00 +CPU= 30.481
  22:14:22 STEP 1097 H= 274:15 +CPU= 30.139
  22:14:52 STEP 1098 H= 274:30 +CPU= 30.211
  22:15:22 STEP 1099 H= 274:45 +CPU= 29.925
  22:15:53 STEP 1100 H= 275:00 +CPU= 30.349
  22:16:23 STEP 1101 H= 275:15 +CPU= 30.528
  22:16:53 STEP 1102 H= 275:30 +CPU= 30.032
  22:17:23 STEP 1103 H= 275:45 +CPU= 30.045
  22:18:10 STEP 1104 H= 276:00 +CPU= 46.257
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001104
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001104
  22:18:41 STEP 1105 H= 276:15 +CPU= 31.234
  22:19:12 STEP 1106 H= 276:30 +CPU= 30.782
  22:19:42 STEP 1107 H= 276:45 +CPU= 30.593
  22:20:13 STEP 1108 H= 277:00 +CPU= 31.167
  22:20:44 STEP 1109 H= 277:15 +CPU= 30.383
  22:21:14 STEP 1110 H= 277:30 +CPU= 30.327
  22:21:45 STEP 1111 H= 277:45 +CPU= 30.570
  22:22:16 STEP 1112 H= 278:00 +CPU= 31.100
  22:22:47 STEP 1113 H= 278:15 +CPU= 30.784
  22:23:17 STEP 1114 H= 278:30 +CPU= 30.572
  22:23:47 STEP 1115 H= 278:45 +CPU= 29.894
  22:24:24 STEP 1116 H= 279:00 +CPU= 36.537
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001116
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001116
  22:24:55 STEP 1117 H= 279:15 +CPU= 30.838
  22:25:25 STEP 1118 H= 279:30 +CPU= 30.615
  22:25:56 STEP 1119 H= 279:45 +CPU= 30.419
  22:26:26 STEP 1120 H= 280:00 +CPU= 30.836
  22:26:57 STEP 1121 H= 280:15 +CPU= 30.297
  22:27:28 STEP 1122 H= 280:30 +CPU= 30.981
  22:27:58 STEP 1123 H= 280:45 +CPU= 30.222
  22:28:29 STEP 1124 H= 281:00 +CPU= 30.524
  22:28:59 STEP 1125 H= 281:15 +CPU= 30.043
  22:29:28 STEP 1126 H= 281:30 +CPU= 29.845
  22:29:58 STEP 1127 H= 281:45 +CPU= 29.691
  22:30:34 STEP 1128 H= 282:00 +CPU= 36.348
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001128
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001128
  22:31:05 STEP 1129 H= 282:15 +CPU= 30.167
  22:31:34 STEP 1130 H= 282:30 +CPU= 29.847
  22:32:04 STEP 1131 H= 282:45 +CPU= 29.789
  22:32:35 STEP 1132 H= 283:00 +CPU= 30.570
  22:33:05 STEP 1133 H= 283:15 +CPU= 29.633
  22:33:34 STEP 1134 H= 283:30 +CPU= 29.938
  22:34:05 STEP 1135 H= 283:45 +CPU= 30.083
  22:34:35 STEP 1136 H= 284:00 +CPU= 30.658
  22:35:05 STEP 1137 H= 284:15 +CPU= 30.127
  22:35:35 STEP 1138 H= 284:30 +CPU= 29.716
  22:36:05 STEP 1139 H= 284:45 +CPU= 30.110
  22:36:42 STEP 1140 H= 285:00 +CPU= 37.042
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001140
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001140
  22:37:12 STEP 1141 H= 285:15 +CPU= 30.043
  22:37:42 STEP 1142 H= 285:30 +CPU= 29.957
  22:38:12 STEP 1143 H= 285:45 +CPU= 30.216
  22:38:43 STEP 1144 H= 286:00 +CPU= 30.827
  22:39:14 STEP 1145 H= 286:15 +CPU= 30.435
  22:39:44 STEP 1146 H= 286:30 +CPU= 30.378
  22:40:14 STEP 1147 H= 286:45 +CPU= 30.328
  22:40:45 STEP 1148 H= 287:00 +CPU= 30.679
  22:41:15 STEP 1149 H= 287:15 +CPU= 30.116
  22:41:45 STEP 1150 H= 287:30 +CPU= 29.811
  22:42:15 STEP 1151 H= 287:45 +CPU= 29.930
  22:43:01 STEP 1152 H= 288:00 +CPU= 46.055
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001152
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001056
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001068
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001080
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001092
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001104
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001116
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001128
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001140
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001140
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29b_2016092300_15_991_12209714_0_r1988314655_11.zip
Uploading the intermediate file: upload_file_11.zip
  22:43:37 STEP 1153 H= 288:15 +CPU= 35.364
  22:44:07 STEP 1154 H= 288:30 +CPU= 30.926
  22:44:38 STEP 1155 H= 288:45 +CPU= 30.657
  22:45:09 STEP 1156 H= 289:00 +CPU= 30.989
  22:45:40 STEP 1157 H= 289:15 +CPU= 30.912
  22:46:11 STEP 1158 H= 289:30 +CPU= 31.166
  22:46:42 STEP 1159 H= 289:45 +CPU= 30.956
  22:47:15 STEP 1160 H= 290:00 +CPU= 32.497
  22:47:45 STEP 1161 H= 290:15 +CPU= 30.620
  22:48:16 STEP 1162 H= 290:30 +CPU= 30.464
  22:48:45 STEP 1163 H= 290:45 +CPU= 29.496
  22:49:22 STEP 1164 H= 291:00 +CPU= 36.505
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001164
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001164
  22:49:52 STEP 1165 H= 291:15 +CPU= 30.512
  22:50:23 STEP 1166 H= 291:30 +CPU= 30.745
  22:50:54 STEP 1167 H= 291:45 +CPU= 30.563
  22:51:25 STEP 1168 H= 292:00 +CPU= 31.084
  22:51:55 STEP 1169 H= 292:15 +CPU= 30.561
  22:52:27 STEP 1170 H= 292:30 +CPU= 31.407
  22:52:58 STEP 1171 H= 292:45 +CPU= 30.903
  22:53:29 STEP 1172 H= 293:00 +CPU= 31.049
  22:53:59 STEP 1173 H= 293:15 +CPU= 30.432
  22:54:30 STEP 1174 H= 293:30 +CPU= 30.473
  22:55:00 STEP 1175 H= 293:45 +CPU= 30.642
  22:55:38 STEP 1176 H= 294:00 +CPU= 37.604
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001176
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001176
  22:56:08 STEP 1177 H= 294:15 +CPU= 30.571
  22:56:39 STEP 1178 H= 294:30 +CPU= 30.337
  22:57:09 STEP 1179 H= 294:45 +CPU= 30.312
  22:57:40 STEP 1180 H= 295:00 +CPU= 30.711
  22:58:10 STEP 1181 H= 295:15 +CPU= 30.274
  22:58:40 STEP 1182 H= 295:30 +CPU= 30.024
  22:59:11 STEP 1183 H= 295:45 +CPU= 30.435
  22:59:41 STEP 1184 H= 296:00 +CPU= 30.845
  23:00:12 STEP 1185 H= 296:15 +CPU= 30.719
  23:00:43 STEP 1186 H= 296:30 +CPU= 30.573
  23:01:13 STEP 1187 H= 296:45 +CPU= 30.102
  23:01:50 STEP 1188 H= 297:00 +CPU= 36.862
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001188
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001188
  23:02:20 STEP 1189 H= 297:15 +CPU= 30.156
  23:02:50 STEP 1190 H= 297:30 +CPU= 30.307
  23:03:21 STEP 1191 H= 297:45 +CPU= 30.623
  23:03:51 STEP 1192 H= 298:00 +CPU= 30.494
  23:04:22 STEP 1193 H= 298:15 +CPU= 30.487
  23:04:52 STEP 1194 H= 298:30 +CPU= 30.147
  23:05:22 STEP 1195 H= 298:45 +CPU= 30.048
  23:05:53 STEP 1196 H= 299:00 +CPU= 30.647
  23:06:23 STEP 1197 H= 299:15 +CPU= 30.067
  23:06:53 STEP 1198 H= 299:30 +CPU= 30.305
  23:07:23 STEP 1199 H= 299:45 +CPU= 30.207
  23:08:10 STEP 1200 H= 300:00 +CPU= 46.294
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001200
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001200
  23:08:42 STEP 1201 H= 300:15 +CPU= 32.835
  23:09:13 STEP 1202 H= 300:30 +CPU= 30.869
  23:09:44 STEP 1203 H= 300:45 +CPU= 30.560
  23:10:15 STEP 1204 H= 301:00 +CPU= 31.420
  23:10:46 STEP 1205 H= 301:15 +CPU= 30.630
  23:11:16 STEP 1206 H= 301:30 +CPU= 30.044
  23:11:46 STEP 1207 H= 301:45 +CPU= 30.301
  23:12:17 STEP 1208 H= 302:00 +CPU= 31.034
  23:12:48 STEP 1209 H= 302:15 +CPU= 30.171
  23:13:17 STEP 1210 H= 302:30 +CPU= 29.672
  23:13:46 STEP 1211 H= 302:45 +CPU= 29.219
  23:14:23 STEP 1212 H= 303:00 +CPU= 36.843
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001212
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001212
  23:14:53 STEP 1213 H= 303:15 +CPU= 29.946
  23:15:23 STEP 1214 H= 303:30 +CPU= 30.028
  23:15:53 STEP 1215 H= 303:45 +CPU= 29.976
  23:16:24 STEP 1216 H= 304:00 +CPU= 30.533
  23:16:54 STEP 1217 H= 304:15 +CPU= 30.455
  23:17:24 STEP 1218 H= 304:30 +CPU= 29.918
  23:17:55 STEP 1219 H= 304:45 +CPU= 30.538
  23:18:25 STEP 1220 H= 305:00 +CPU= 30.675
  23:18:55 STEP 1221 H= 305:15 +CPU= 30.027
  23:19:26 STEP 1222 H= 305:30 +CPU= 30.341
  23:19:56 STEP 1223 H= 305:45 +CPU= 30.084
  23:20:33 STEP 1224 H= 306:00 +CPU= 37.045
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001224
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001224
  23:21:03 STEP 1225 H= 306:15 +CPU= 30.032
  23:21:33 STEP 1226 H= 306:30 +CPU= 29.780
  23:22:03 STEP 1227 H= 306:45 +CPU= 29.987
  23:22:33 STEP 1228 H= 307:00 +CPU= 30.153
  23:23:03 STEP 1229 H= 307:15 +CPU= 30.049
  23:23:33 STEP 1230 H= 307:30 +CPU= 30.298
  23:24:03 STEP 1231 H= 307:45 +CPU= 30.106
  23:24:34 STEP 1232 H= 308:00 +CPU= 30.669
  23:25:04 STEP 1233 H= 308:15 +CPU= 29.768
  23:25:34 STEP 1234 H= 308:30 +CPU= 30.122
  23:26:04 STEP 1235 H= 308:45 +CPU= 30.296
  23:26:41 STEP 1236 H= 309:00 +CPU= 36.890
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001236
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001236
  23:27:11 STEP 1237 H= 309:15 +CPU= 30.119
  23:27:41 STEP 1238 H= 309:30 +CPU= 30.009
  23:28:11 STEP 1239 H= 309:45 +CPU= 29.998
  23:28:42 STEP 1240 H= 310:00 +CPU= 30.531
  23:29:12 STEP 1241 H= 310:15 +CPU= 30.166
  23:29:42 STEP 1242 H= 310:30 +CPU= 30.387
  23:30:12 STEP 1243 H= 310:45 +CPU= 29.932
  23:30:43 STEP 1244 H= 311:00 +CPU= 30.465
  23:31:13 STEP 1245 H= 311:15 +CPU= 30.310
  23:31:43 STEP 1246 H= 311:30 +CPU= 30.007
  23:32:14 STEP 1247 H= 311:45 +CPU= 30.333
  23:33:00 STEP 1248 H= 312:00 +CPU= 46.279
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001248
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001152
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001164
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001176
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001188
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001200
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001212
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001224
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001236
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001236
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29b_2016092300_15_991_12209714_0_r1988314655_12.zip
Uploading the intermediate file: upload_file_12.zip
  23:33:33 STEP 1249 H= 312:15 +CPU= 33.340
  23:34:03 STEP 1250 H= 312:30 +CPU= 29.938
  23:34:32 STEP 1251 H= 312:45 +CPU= 29.214
  23:35:01 STEP 1252 H= 313:00 +CPU= 28.924
  23:35:30 STEP 1253 H= 313:15 +CPU= 29.208
  23:36:00 STEP 1254 H= 313:30 +CPU= 29.726
  23:36:30 STEP 1255 H= 313:45 +CPU= 29.425
  23:36:58 STEP 1256 H= 314:00 +CPU= 28.734
  23:37:27 STEP 1257 H= 314:15 +CPU= 28.292
  23:37:55 STEP 1258 H= 314:30 +CPU= 28.721
  23:38:23 STEP 1259 H= 314:45 +CPU= 28.072
  23:38:58 STEP 1260 H= 315:00 +CPU= 34.663
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001260
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001260
  23:39:26 STEP 1261 H= 315:15 +CPU= 28.083
  23:39:54 STEP 1262 H= 315:30 +CPU= 28.119
  23:40:23 STEP 1263 H= 315:45 +CPU= 28.316
  23:40:51 STEP 1264 H= 316:00 +CPU= 28.755
  23:41:21 STEP 1265 H= 316:15 +CPU= 29.068
  23:41:49 STEP 1266 H= 316:30 +CPU= 28.709
  23:42:19 STEP 1267 H= 316:45 +CPU= 29.554
  23:42:48 STEP 1268 H= 317:00 +CPU= 29.355
  23:43:16 STEP 1269 H= 317:15 +CPU= 28.057
  23:43:44 STEP 1270 H= 317:30 +CPU= 28.036
  23:44:12 STEP 1271 H= 317:45 +CPU= 28.128
  23:44:47 STEP 1272 H= 318:00 +CPU= 34.582
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001272
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001272
  23:45:15 STEP 1273 H= 318:15 +CPU= 27.566
  23:45:42 STEP 1274 H= 318:30 +CPU= 27.433
  23:46:10 STEP 1275 H= 318:45 +CPU= 27.579
  23:46:38 STEP 1276 H= 319:00 +CPU= 28.062
  23:47:05 STEP 1277 H= 319:15 +CPU= 27.804
  23:47:33 STEP 1278 H= 319:30 +CPU= 27.550
  23:48:01 STEP 1279 H= 319:45 +CPU= 27.765
  23:48:29 STEP 1280 H= 320:00 +CPU= 27.973
  23:48:56 STEP 1281 H= 320:15 +CPU= 27.678
  23:49:24 STEP 1282 H= 320:30 +CPU= 27.430
  23:49:52 STEP 1283 H= 320:45 +CPU= 27.628
  23:50:25 STEP 1284 H= 321:00 +CPU= 33.875
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001284
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001284
  23:50:53 STEP 1285 H= 321:15 +CPU= 27.999
  23:51:22 STEP 1286 H= 321:30 +CPU= 28.160
  23:51:49 STEP 1287 H= 321:45 +CPU= 27.676
  23:52:17 STEP 1288 H= 322:00 +CPU= 28.159
  23:52:45 STEP 1289 H= 322:15 +CPU= 27.904
  23:53:13 STEP 1290 H= 322:30 +CPU= 27.558
  23:53:40 STEP 1291 H= 322:45 +CPU= 27.613
  23:54:08 STEP 1292 H= 323:00 +CPU= 27.917
  23:54:36 STEP 1293 H= 323:15 +CPU= 27.800
  23:55:05 STEP 1294 H= 323:30 +CPU= 28.293
  23:55:32 STEP 1295 H= 323:45 +CPU= 27.811
  23:56:13 STEP 1296 H= 324:00 +CPU= 41.033
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001296
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001296
  23:56:42 STEP 1297 H= 324:15 +CPU= 28.623
  23:57:10 STEP 1298 H= 324:30 +CPU= 27.989
  23:57:38 STEP 1299 H= 324:45 +CPU= 28.494
  23:58:07 STEP 1300 H= 325:00 +CPU= 28.066
  23:58:34 STEP 1301 H= 325:15 +CPU= 27.883
  23:59:02 STEP 1302 H= 325:30 +CPU= 27.811
  23:59:30 STEP 1303 H= 325:45 +CPU= 28.193
  23:59:59 STEP 1304 H= 326:00 +CPU= 28.351
  00:00:27 STEP 1305 H= 326:15 +CPU= 27.875
  00:00:55 STEP 1306 H= 326:30 +CPU= 28.147
  00:01:23 STEP 1307 H= 326:45 +CPU= 27.946
  00:01:57 STEP 1308 H= 327:00 +CPU= 33.812
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001308
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001308
  00:02:24 STEP 1309 H= 327:15 +CPU= 27.724
  00:02:52 STEP 1310 H= 327:30 +CPU= 28.082
  00:03:20 STEP 1311 H= 327:45 +CPU= 27.731
  00:03:48 STEP 1312 H= 328:00 +CPU= 28.100
  00:04:16 STEP 1313 H= 328:15 +CPU= 27.831
  00:04:44 STEP 1314 H= 328:30 +CPU= 27.804
  00:05:12 STEP 1315 H= 328:45 +CPU= 27.986
  00:05:40 STEP 1316 H= 329:00 +CPU= 28.349
  00:06:08 STEP 1317 H= 329:15 +CPU= 27.675
  00:06:36 STEP 1318 H= 329:30 +CPU= 27.979
  00:07:04 STEP 1319 H= 329:45 +CPU= 27.841
  00:07:38 STEP 1320 H= 330:00 +CPU= 34.222
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001320
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001320
  00:08:06 STEP 1321 H= 330:15 +CPU= 27.678
  00:08:34 STEP 1322 H= 330:30 +CPU= 27.827
  00:09:01 STEP 1323 H= 330:45 +CPU= 27.742
  00:09:30 STEP 1324 H= 331:00 +CPU= 28.324
  00:09:57 STEP 1325 H= 331:15 +CPU= 27.736
  00:10:25 STEP 1326 H= 331:30 +CPU= 27.758
  00:10:54 STEP 1327 H= 331:45 +CPU= 28.340
  00:11:22 STEP 1328 H= 332:00 +CPU= 28.078
  00:11:50 STEP 1329 H= 332:15 +CPU= 28.055
  00:12:17 STEP 1330 H= 332:30 +CPU= 27.644
  00:12:45 STEP 1331 H= 332:45 +CPU= 27.960
  00:13:19 STEP 1332 H= 333:00 +CPU= 34.000
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001332
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001332
  00:13:47 STEP 1333 H= 333:15 +CPU= 27.811
  00:14:15 STEP 1334 H= 333:30 +CPU= 27.777
  00:14:43 STEP 1335 H= 333:45 +CPU= 27.867
  00:15:11 STEP 1336 H= 334:00 +CPU= 28.309
  00:15:39 STEP 1337 H= 334:15 +CPU= 27.717
  00:16:07 STEP 1338 H= 334:30 +CPU= 27.868
  00:16:34 STEP 1339 H= 334:45 +CPU= 27.477
  00:17:02 STEP 1340 H= 335:00 +CPU= 28.123
  00:17:30 STEP 1341 H= 335:15 +CPU= 27.414
  00:17:58 STEP 1342 H= 335:30 +CPU= 27.843
  00:18:25 STEP 1343 H= 335:45 +CPU= 27.608
  00:19:06 STEP 1344 H= 336:00 +CPU= 40.917
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001344
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001248
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001260
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001272
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001284
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001296
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001308
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001320
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001332
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001332
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29b_2016092300_15_991_12209714_0_r1988314655_13.zip
Uploading the intermediate file: upload_file_13.zip
  00:19:38 STEP 1345 H= 336:15 +CPU= 31.428
  00:20:06 STEP 1346 H= 336:30 +CPU= 28.198
  00:20:34 STEP 1347 H= 336:45 +CPU= 27.940
  00:21:02 STEP 1348 H= 337:00 +CPU= 28.343
  00:21:30 STEP 1349 H= 337:15 +CPU= 28.284
  00:21:59 STEP 1350 H= 337:30 +CPU= 28.233
  00:22:26 STEP 1351 H= 337:45 +CPU= 27.760
  00:22:55 STEP 1352 H= 338:00 +CPU= 28.772
  00:23:23 STEP 1353 H= 338:15 +CPU= 27.921
  00:23:52 STEP 1354 H= 338:30 +CPU= 28.701
  00:24:20 STEP 1355 H= 338:45 +CPU= 28.053
  00:24:54 STEP 1356 H= 339:00 +CPU= 34.349
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001356
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001356
  00:25:22 STEP 1357 H= 339:15 +CPU= 28.046
  00:25:51 STEP 1358 H= 339:30 +CPU= 28.372
  00:26:19 STEP 1359 H= 339:45 +CPU= 28.212
  00:26:47 STEP 1360 H= 340:00 +CPU= 28.514
  00:27:15 STEP 1361 H= 340:15 +CPU= 27.908
  00:27:43 STEP 1362 H= 340:30 +CPU= 27.991
  00:28:12 STEP 1363 H= 340:45 +CPU= 28.456
  00:28:40 STEP 1364 H= 341:00 +CPU= 28.315
  00:29:08 STEP 1365 H= 341:15 +CPU= 28.115
  00:29:36 STEP 1366 H= 341:30 +CPU= 27.482
  00:30:03 STEP 1367 H= 341:45 +CPU= 27.061
  00:30:36 STEP 1368 H= 342:00 +CPU= 33.813
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001368
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001368
  00:31:04 STEP 1369 H= 342:15 +CPU= 27.667
  00:31:32 STEP 1370 H= 342:30 +CPU= 27.554
  00:31:59 STEP 1371 H= 342:45 +CPU= 27.423
  00:32:27 STEP 1372 H= 343:00 +CPU= 28.144
  00:32:55 STEP 1373 H= 343:15 +CPU= 27.448
  00:33:22 STEP 1374 H= 343:30 +CPU= 27.414
  00:33:49 STEP 1375 H= 343:45 +CPU= 27.134
  00:34:17 STEP 1376 H= 344:00 +CPU= 27.767
  00:34:45 STEP 1377 H= 344:15 +CPU= 27.406
  00:35:12 STEP 1378 H= 344:30 +CPU= 27.725
  00:35:40 STEP 1379 H= 344:45 +CPU= 27.716
  00:36:14 STEP 1380 H= 345:00 +CPU= 33.897
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001380
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001380
  00:36:42 STEP 1381 H= 345:15 +CPU= 27.692
  00:37:09 STEP 1382 H= 345:30 +CPU= 27.507
  00:37:36 STEP 1383 H= 345:45 +CPU= 27.216
  00:38:05 STEP 1384 H= 346:00 +CPU= 28.247
  00:38:32 STEP 1385 H= 346:15 +CPU= 27.665
  00:39:00 STEP 1386 H= 346:30 +CPU= 27.788
  00:39:28 STEP 1387 H= 346:45 +CPU= 28.062
  00:39:56 STEP 1388 H= 347:00 +CPU= 28.274
  00:40:24 STEP 1389 H= 347:15 +CPU= 27.609
  00:40:52 STEP 1390 H= 347:30 +CPU= 27.845
  00:41:20 STEP 1391 H= 347:45 +CPU= 28.698
  00:42:03 STEP 1392 H= 348:00 +CPU= 42.854
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001392
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001392
  00:42:32 STEP 1393 H= 348:15 +CPU= 28.731
  00:43:01 STEP 1394 H= 348:30 +CPU= 28.718
  00:43:29 STEP 1395 H= 348:45 +CPU= 28.101
  00:43:58 STEP 1396 H= 349:00 +CPU= 28.604
  00:44:25 STEP 1397 H= 349:15 +CPU= 27.690
  00:44:54 STEP 1398 H= 349:30 +CPU= 28.773
  00:45:22 STEP 1399 H= 349:45 +CPU= 27.996
  00:45:50 STEP 1400 H= 350:00 +CPU= 28.114
  00:46:18 STEP 1401 H= 350:15 +CPU= 27.678
  00:46:46 STEP 1402 H= 350:30 +CPU= 28.178
  00:47:14 STEP 1403 H= 350:45 +CPU= 27.819
  00:47:48 STEP 1404 H= 351:00 +CPU= 34.174
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001404
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001404
  00:48:16 STEP 1405 H= 351:15 +CPU= 28.099
  00:48:44 STEP 1406 H= 351:30 +CPU= 27.792
  00:49:12 STEP 1407 H= 351:45 +CPU= 27.779
  00:49:40 STEP 1408 H= 352:00 +CPU= 28.476
  00:50:08 STEP 1409 H= 352:15 +CPU= 27.730
  00:50:36 STEP 1410 H= 352:30 +CPU= 27.861
  00:51:03 STEP 1411 H= 352:45 +CPU= 27.493
  00:51:31 STEP 1412 H= 353:00 +CPU= 27.685
  00:51:59 STEP 1413 H= 353:15 +CPU= 27.638
  00:52:26 STEP 1414 H= 353:30 +CPU= 27.675
  00:52:54 STEP 1415 H= 353:45 +CPU= 27.211
  00:53:28 STEP 1416 H= 354:00 +CPU= 34.553
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001416
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001416
  00:53:57 STEP 1417 H= 354:15 +CPU= 28.928
  00:54:24 STEP 1418 H= 354:30 +CPU= 27.350
  00:54:52 STEP 1419 H= 354:45 +CPU= 27.922
  00:55:20 STEP 1420 H= 355:00 +CPU= 27.905
  00:55:48 STEP 1421 H= 355:15 +CPU= 27.458
  00:56:15 STEP 1422 H= 355:30 +CPU= 27.403
  00:56:42 STEP 1423 H= 355:45 +CPU= 27.379
  00:57:11 STEP 1424 H= 356:00 +CPU= 28.191
  00:57:39 STEP 1425 H= 356:15 +CPU= 27.844
  00:58:06 STEP 1426 H= 356:30 +CPU= 27.469
  00:58:33 STEP 1427 H= 356:45 +CPU= 27.326
  00:59:07 STEP 1428 H= 357:00 +CPU= 33.535
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001428
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001428
  00:59:35 STEP 1429 H= 357:15 +CPU= 27.673
  01:00:02 STEP 1430 H= 357:30 +CPU= 27.819
  01:00:30 STEP 1431 H= 357:45 +CPU= 27.893
  01:00:58 STEP 1432 H= 358:00 +CPU= 28.030
  01:01:26 STEP 1433 H= 358:15 +CPU= 27.984
  01:01:54 STEP 1434 H= 358:30 +CPU= 27.998
  01:02:22 STEP 1435 H= 358:45 +CPU= 27.687
  01:02:50 STEP 1436 H= 359:00 +CPU= 28.086
  01:03:18 STEP 1437 H= 359:15 +CPU= 27.756
  01:03:45 STEP 1438 H= 359:30 +CPU= 27.531
  01:04:13 STEP 1439 H= 359:45 +CPU= 27.785
  01:04:54 STEP 1440 H= 360:00 +CPU= 41.099
..The child process terminated with status: 0
>>> Printing last 70 lines from file: NODE.001_01

 
DDH------------------------------------------------------------------------
I                                                                         I
I TIME =  360.000H DDH * Diagnostic Var.,     AREA                GLOBAL  I
I                                                                         I
I-------------------------------------------------------------------------I
I   VARIABLE   I      Initial value         I        Current value        I
I-------------------------------------------------------------------------I
I  OMEGA (P/S) I      0.1144021588E-20      I       0.1053932895E-05      I
ICloud fractionI      0.0000000000E+00      I       0.3994410046E-01      I
I  Relat. Hum. I      0.4420188244E+00      I       0.4816736023E+00      I
I  PASS 01     I      0.0000000000E+00      I       0.3302199617E+03      I
DDH------------------------------------------------------------------------
  
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =  1440 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+001440 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+001440
 MPI-TASK:   1 -    311998529 BYTES IN      7 RECORDS TRANSFERRED IN    0.0005 SECONDS623997.1 Mbytes/s, TOTAL TIME=    0.0140( 3.6%)
 01:04:54 STEP 1440 H= 360:00 +CPU= 41.099
 END CNT3
 NSTEP =  1440 CNT0   000000000  30.665  30.665  30.680   1.000               0               0               0               0               0
IO-STREAM STATISTICS, TOTAL NO OF REC/BYTES READ           34/  1069125708 WRITTEN        14180/************
 -TASK-OPENED-OPEN-RECS IN -KBYTE IN    -RECS OUT-KBYTE OUT-WALL   -WALL IN-WALL
  OU-TOT IN -TOT OUT
   1    253    0       34      1044068    14180 ************    49.9    14.1    35.8    15.4    67.0
===-=== START OF TIMING STATISTICS ===-===
 
STATS FOR ALL TASKS
 NUM ROUTINE                                     CALLS  MEAN(ms)   MAX(ms)   FRAC(%)  UNBAL(%)
   0 CNT0     - COMPLETE EXECUTION                   1 ********* *********    100.00      0.00
   1 CNT4     - FORWARD INTEGRATION                  1 ********* *********     99.88      0.00
   8 SCAN2M - GRID-POINT DYNAMICS                  521    6004.8    6004.8     21.51      0.00
   9 SPCM     - SPECTRAL COMP.                     480     911.6     911.6      3.01      0.00
  10 SCAN2M - PHYSICS                              481   13392.8   13392.8     44.30      0.00
  11 IOPACK   - OUTPUT P.P. RESULTS                 40    1330.7    1330.7      0.37      0.00
  12 SPNORM   - SPECTRAL NORM COMP.                 21      62.7      62.7      0.01      0.00
  17 GRIDFPOS IN CNT4                               40      32.5      32.5      0.01      0.00
  27 RESTART FILES - WRITING                        10    5322.6    5322.6      0.37      0.00
  28 RESTART FILES - READING                         1   16560.9   16560.9      0.11      0.00
  29 SU4FPOS IN CNT4                                40       0.0       0.0      0.00      0.00
  30 DYNFPOS IN CNT4                                40    6111.2    6111.2      1.68      0.00
  31 POSDDH IN STEPO                                 3       0.9       0.9      0.00      0.00
  37 CPGLAG   - SL COMPUTATIONS                    481    5705.0    5705.0     18.87      0.00
  39 SU0YOMB                                         1     716.6     716.6      0.00      0.00
  51 SCAN2M   - SL COMM. PART 1                    481      35.0      35.0      0.12      0.00
  54 SPCM     - M TO S/S TO M TRANSP.              480     346.5     346.5      1.14      0.00
  55 SPCIMPF  - S TO M/M TO S TRANSP.              480      79.0      79.0      0.26      0.00
  56 SPNORM   - SPECTRAL NORM COMM.                 21       0.2       0.2      0.00      0.00
 102 LTINV_CTL   - INVERSE LEGENDRE TRANSFORM     1044     495.0     495.0      3.55      0.00
 103 LTDIR_CTL   - DIRECT LEGENDRE TRANSFORM      1000     297.3     297.3      2.04      0.00
 106 FTDIR_CTL   - DIRECT FOURIER TRANSFORM       1000      95.4      95.4      0.66      0.00
 107 FTINV_CTL   - INVERSE FOURIER TRANSFORM      1044     141.2     141.2      1.01      0.00
 140 SULEG       - COMP. OF LEGENDRE POL.            1      75.6      75.6      0.00      0.00
 152 LTINV_CTL   - M TO L TRANSPOSITION           1044      72.8      72.8      0.52      0.00
 153 LTDIR_CTL   - L TO M TRANSPOSITION           1000      42.4      42.4      0.29      0.00
 157 FTINV_CTL   - L TO G TRANSPOSITION           1044      62.2      62.2      0.45      0.00
 158 FTDIR_CTL   - G TO L TRANSPOSITION           1000      36.9      36.9      0.25      0.00
 400 GSTATS                                      56453       0.0       0.0      0.00      0.00
 401 GSTATS HOOK                                 52473       0.0       0.0      0.00      0.00
TOTAL MEASURED IMBALANCE =       0.0 SECONDS,  0.0 PERCENT
TOTAL WALLCLOCK TIME    14542.5 CPU TIME   14523.8 VECTOR TIME    14523.8


===-=== END   OF TIMING STATISTICS ===-===
FORECAST DAYS PER DAY       89.2
  *** END CNT0 *** 
------------------------------------------------
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMGGh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMSHh7zg+001440
Moving to projects directory: /var/lib/boinc-client/slots/7/ICMUAh7zg+001440
Adding to the zip: /var/lib/boinc-client/slots/7/NODE.001_01
Adding to the zip: /var/lib/boinc-client/slots/7/ifs.stat
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001428
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001440
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001404
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001416
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001356
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001344
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMGGh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001380
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001368
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMSHh7zg+001392
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12209714/ICMUAh7zg+001440
Zipping up the final file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a29b_2016092300_15_991_12209714_0_r1988314655_14.zip
Uploading the final file: upload_file_14.zip
Uploading trickle at timestep: 1295100
01:08:25 (1660): called boinc_finish(0)

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:51:27 1536331 22309914 oifs_43r3_bl_a29b_2016092300_15_991_12209714_0 1,295,100 44,745 0.0345


©2024 climateprediction.net