climateprediction.net home page
Task 22276736

Task 22276736

Name oifs_43r3_ps_0430_2005050100_123_974_12191074_0
Workunit 12191074
Created 23 Dec 2022, 10:33:09 UTC
Sent 23 Dec 2022, 10:46:15 UTC
Report deadline 22 Jan 2023, 10:46:15 UTC
Received 1 Jan 2023, 19:44:17 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1520317
Run time 22 hours 22 min 58 sec
CPU time 17 hours 29 min 13 sec
Validate state Invalid
Credit 13,184.64
Device peak FLOPS 8.55 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,419.28 MB
Peak swap size 4,978.08 MB
Peak disk usage 7,288.16 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
ux/support/drhook.c:3843] 4749 MB (maxheap), 4419 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :            CALL_SL_HEAP 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :             CALL_SL 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :              >OMP-CALL_SL     2        (1005) 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :               LAPINEB 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                LARCINB 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                 LAITRE_GFL 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160038:1672290038:6982.073] [c_drhook_print_@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:3897] :                  LAITRI 
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160039:1672290039:6983.073] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#15, nsigs = 1
[EC_DRHOOK:roo:1:1:10529:10529] [20221229:160039:1672290039:6983.073] [signal_drhook@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1ce8bb0 for signal#15, nsigs = 1
forrtl: error (78): process killed (SIGTERM)
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CE916B  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000081FFF1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DC9090  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000DD4534  laitri_                   406  laitri.F90
oifs_43r3_model.e  0000000000DCACDB  laitre_gfl_               289  laitre_gfl.F90
oifs_43r3_model.e  0000000000DC6671  larcinb_                  865  larcinb.F90
oifs_43r3_model.e  0000000000D7EBA8  lapineb_                  451  lapineb.F90
oifs_43r3_model.e  0000000000D7B935  call_sl_                  363  call_sl.F90
oifs_43r3_model.e  0000000000D784FD  call_sl_heap_              94  call_sl_heap.F90
oifs_43r3_model.e  0000000000BF9958  gp_model_                 533  gp_model.F90
oifs_43r3_model.e  00000000012E9482  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BBC1CE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  000000000057A664  stepo_                    327  stepo.F90
oifs_43r3_model.e  000000000055DEEC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000005471C9  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  0000000000546412  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  0000000000545E28  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  000000000040708D  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  000000000040220F  MAIN__                     96  master.F90
oifs_43r3_model.e  00000000004021A2  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCA390  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000040206E  Unknown               Unknown  Unknown
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2005050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0430
(argv4) batchid: 974
(argv5) wuid: 12191074
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/1
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_c75c7502b5eee744ebbd1abc6dfce10c to: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0430_2005050100_123_974_12191074.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/1/oifs_43r3_ps_0430_2005050100_123_974_12191074.zip
ic_ancil_file: ic_ancil_12191074
ifsdata_file: ifsdata_12191074
climate_data_file: clim_data_12191074
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_ca417744e42b7dae45aab4d8ad39f288 to: /var/lib/boinc-client/slots/1/ic_ancil_12191074.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/1/ic_ancil_12191074.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_7131ea9e8649bbae398455889183686c to: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12191074.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/1/ifsdata/ifsdata_12191074.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/1/159l_2/clim_data_12191074.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/1/159l_2/clim_data_12191074.zip
Checking for progress XML file: /var/lib/boinc-client/slots/1/progress_file_12191074.xml
Opened progress file ok : /var/lib/boinc-client/slots/1/progress_file_12191074.xml
-- Model is restarting --
Adjusting last_iter, 2137, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/1/progress_file_12191074.xml
last_cpu_time: 51674
upload_file_number: 89
last_iter: 2135
last_upload: 7689600
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189
The child process has been launched with process id: 29614
Executing the command: /var/lib/boinc-client/slots/1/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:29614:29614] [20230101:173015:1672554615:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  17:30:51 STEP 2136 H=2136:00 +CPU= 28.737
  17:31:09 STEP 2137 H=2137:00 +CPU= 16.492
  17:31:27 STEP 2138 H=2138:00 +CPU= 16.892
  17:31:55 STEP 2139 H=2139:00 +CPU= 25.685
  17:32:14 STEP 2140 H=2140:00 +CPU= 16.878
  17:32:31 STEP 2141 H=2141:00 +CPU= 16.284
  17:33:00 STEP 2142 H=2142:00 +CPU= 25.777
  17:33:18 STEP 2143 H=2143:00 +CPU= 16.275
  17:33:36 STEP 2144 H=2144:00 +CPU= 16.665
  17:34:03 STEP 2145 H=2145:00 +CPU= 25.031
  17:34:21 STEP 2146 H=2146:00 +CPU= 16.745
  17:34:39 STEP 2147 H=2147:00 +CPU= 16.243
  17:35:10 STEP 2148 H=2148:00 +CPU= 28.183
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002148
  17:35:28 STEP 2149 H=2149:00 +CPU= 16.180
  17:35:46 STEP 2150 H=2150:00 +CPU= 16.827
  17:36:14 STEP 2151 H=2151:00 +CPU= 25.253
  17:36:32 STEP 2152 H=2152:00 +CPU= 16.773
  17:36:50 STEP 2153 H=2153:00 +CPU= 16.047
  17:37:18 STEP 2154 H=2154:00 +CPU= 25.782
  17:37:36 STEP 2155 H=2155:00 +CPU= 16.268
  17:37:54 STEP 2156 H=2156:00 +CPU= 16.512
  17:38:21 STEP 2157 H=2157:00 +CPU= 25.366
  17:38:40 STEP 2158 H=2158:00 +CPU= 16.710
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  22:06:26 STEP 2159 H=2159:00 +CPU= 16.533
  22:07:01 STEP 2160 H=2160:00 +CPU= 30.546
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_89.zip
Uploading the intermediate file: upload_file_89.zip
  22:07:19 STEP 2161 H=2161:00 +CPU= 16.740
  22:07:38 STEP 2162 H=2162:00 +CPU= 17.136
  22:08:08 STEP 2163 H=2163:00 +CPU= 26.529
  22:08:27 STEP 2164 H=2164:00 +CPU= 17.456
  22:08:47 STEP 2165 H=2165:00 +CPU= 16.724
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  01:36:51 STEP 2166 H=2166:00 +CPU= 25.725
  01:37:08 STEP 2167 H=2167:00 +CPU= 16.043
  01:37:25 STEP 2168 H=2168:00 +CPU= 16.375
  01:37:50 STEP 2169 H=2169:00 +CPU= 24.916
  01:38:07 STEP 2170 H=2170:00 +CPU= 16.354
  01:38:23 STEP 2171 H=2171:00 +CPU= 15.601
  01:38:52 STEP 2172 H=2172:00 +CPU= 27.505
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002172
  01:39:09 STEP 2173 H=2173:00 +CPU= 15.972
  01:39:26 STEP 2174 H=2174:00 +CPU= 16.116
  01:39:52 STEP 2175 H=2175:00 +CPU= 24.879
  01:40:08 STEP 2176 H=2176:00 +CPU= 16.227
  01:40:25 STEP 2177 H=2177:00 +CPU= 15.848
  01:40:51 STEP 2178 H=2178:00 +CPU= 25.198
  01:41:07 STEP 2179 H=2179:00 +CPU= 15.637
  01:41:25 STEP 2180 H=2180:00 +CPU= 16.324
  01:41:51 STEP 2181 H=2181:00 +CPU= 24.557
  01:42:07 STEP 2182 H=2182:00 +CPU= 16.236
  01:42:25 STEP 2183 H=2183:00 +CPU= 16.272
  01:42:56 STEP 2184 H=2184:00 +CPU= 29.990
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_90.zip
Uploading the intermediate file: upload_file_90.zip
  01:43:13 STEP 2185 H=2185:00 +CPU= 16.021
  01:43:30 STEP 2186 H=2186:00 +CPU= 16.287
  01:43:57 STEP 2187 H=2187:00 +CPU= 25.501
  01:44:14 STEP 2188 H=2188:00 +CPU= 17.200
  01:44:31 STEP 2189 H=2189:00 +CPU= 15.914
Suspend request received from the BOINC client, suspending the child process
Resuming the child process
  02:57:19 STEP 2190 H=2190:00 +CPU= 25.106
  02:57:35 STEP 2191 H=2191:00 +CPU= 15.404
  02:57:51 STEP 2192 H=2192:00 +CPU= 16.067
  02:58:17 STEP 2193 H=2193:00 +CPU= 24.444
  02:58:33 STEP 2194 H=2194:00 +CPU= 16.168
  02:58:49 STEP 2195 H=2195:00 +CPU= 15.344
  02:59:17 STEP 2196 H=2196:00 +CPU= 27.320
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002196
  02:59:34 STEP 2197 H=2197:00 +CPU= 15.565
  02:59:50 STEP 2198 H=2198:00 +CPU= 15.998
  03:00:15 STEP 2199 H=2199:00 +CPU= 24.588
  03:00:32 STEP 2200 H=2200:00 +CPU= 15.874
  03:00:48 STEP 2201 H=2201:00 +CPU= 15.355
  03:01:14 STEP 2202 H=2202:00 +CPU= 24.967
  03:01:31 STEP 2203 H=2203:00 +CPU= 15.765
  03:01:47 STEP 2204 H=2204:00 +CPU= 15.925
  03:02:13 STEP 2205 H=2205:00 +CPU= 24.414
  03:02:29 STEP 2206 H=2206:00 +CPU= 15.990
  03:02:46 STEP 2207 H=2207:00 +CPU= 16.082
  03:03:17 STEP 2208 H=2208:00 +CPU= 29.778
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_91.zip
Uploading the intermediate file: upload_file_91.zip
  03:03:33 STEP 2209 H=2209:00 +CPU= 15.420
  03:03:49 STEP 2210 H=2210:00 +CPU= 16.095
  03:04:15 STEP 2211 H=2211:00 +CPU= 24.669
  03:04:32 STEP 2212 H=2212:00 +CPU= 16.159
  03:04:48 STEP 2213 H=2213:00 +CPU= 15.646
  03:05:14 STEP 2214 H=2214:00 +CPU= 25.373
  03:05:30 STEP 2215 H=2215:00 +CPU= 15.624
  03:05:47 STEP 2216 H=2216:00 +CPU= 16.173
  03:06:12 STEP 2217 H=2217:00 +CPU= 24.418
  03:06:29 STEP 2218 H=2218:00 +CPU= 16.087
  03:06:45 STEP 2219 H=2219:00 +CPU= 15.513
  03:07:13 STEP 2220 H=2220:00 +CPU= 27.291
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002220
  03:07:29 STEP 2221 H=2221:00 +CPU= 15.803
  03:07:46 STEP 2222 H=2222:00 +CPU= 16.093
  03:08:11 STEP 2223 H=2223:00 +CPU= 24.527
  03:08:27 STEP 2224 H=2224:00 +CPU= 16.030
  03:08:44 STEP 2225 H=2225:00 +CPU= 15.659
  03:09:09 STEP 2226 H=2226:00 +CPU= 24.947
  03:09:25 STEP 2227 H=2227:00 +CPU= 15.415
  03:09:42 STEP 2228 H=2228:00 +CPU= 15.975
  03:10:07 STEP 2229 H=2229:00 +CPU= 24.984
  03:10:24 STEP 2230 H=2230:00 +CPU= 16.066
  03:10:41 STEP 2231 H=2231:00 +CPU= 15.784
  03:11:11 STEP 2232 H=2232:00 +CPU= 29.325
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_92.zip
Uploading the intermediate file: upload_file_92.zip
  03:11:28 STEP 2233 H=2233:00 +CPU= 15.323
  03:11:44 STEP 2234 H=2234:00 +CPU= 15.661
  03:12:10 STEP 2235 H=2235:00 +CPU= 24.502
  03:12:26 STEP 2236 H=2236:00 +CPU= 15.909
  03:12:42 STEP 2237 H=2237:00 +CPU= 15.501
  03:13:09 STEP 2238 H=2238:00 +CPU= 25.240
  03:13:25 STEP 2239 H=2239:00 +CPU= 15.649
  03:13:42 STEP 2240 H=2240:00 +CPU= 16.497
  03:14:08 STEP 2241 H=2241:00 +CPU= 25.042
  03:14:25 STEP 2242 H=2242:00 +CPU= 16.330
  03:14:41 STEP 2243 H=2243:00 +CPU= 15.879
  03:15:10 STEP 2244 H=2244:00 +CPU= 27.769
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002244
  03:15:26 STEP 2245 H=2245:00 +CPU= 15.462
  03:15:42 STEP 2246 H=2246:00 +CPU= 16.152
  03:16:08 STEP 2247 H=2247:00 +CPU= 24.790
  03:16:24 STEP 2248 H=2248:00 +CPU= 15.978
  03:16:40 STEP 2249 H=2249:00 +CPU= 15.457
  03:17:06 STEP 2250 H=2250:00 +CPU= 25.083
  03:17:22 STEP 2251 H=2251:00 +CPU= 15.632
  03:17:39 STEP 2252 H=2252:00 +CPU= 16.016
  03:18:04 STEP 2253 H=2253:00 +CPU= 24.320
  03:18:21 STEP 2254 H=2254:00 +CPU= 15.863
  03:18:38 STEP 2255 H=2255:00 +CPU= 16.312
  03:19:08 STEP 2256 H=2256:00 +CPU= 29.277
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_93.zip
Uploading the intermediate file: upload_file_93.zip
  03:19:24 STEP 2257 H=2257:00 +CPU= 15.531
  03:19:41 STEP 2258 H=2258:00 +CPU= 16.120
  03:20:07 STEP 2259 H=2259:00 +CPU= 24.463
  03:20:23 STEP 2260 H=2260:00 +CPU= 16.146
  03:20:40 STEP 2261 H=2261:00 +CPU= 15.622
  03:21:06 STEP 2262 H=2262:00 +CPU= 25.506
  03:21:23 STEP 2263 H=2263:00 +CPU= 15.897
  03:21:40 STEP 2264 H=2264:00 +CPU= 16.198
  03:22:05 STEP 2265 H=2265:00 +CPU= 24.522
  03:22:22 STEP 2266 H=2266:00 +CPU= 16.006
  03:22:38 STEP 2267 H=2267:00 +CPU= 15.594
  03:23:06 STEP 2268 H=2268:00 +CPU= 27.361
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002268
  03:23:23 STEP 2269 H=2269:00 +CPU= 15.945
  03:23:40 STEP 2270 H=2270:00 +CPU= 16.220
  03:24:05 STEP 2271 H=2271:00 +CPU= 24.636
  03:24:21 STEP 2272 H=2272:00 +CPU= 16.023
  03:24:37 STEP 2273 H=2273:00 +CPU= 15.584
  03:25:03 STEP 2274 H=2274:00 +CPU= 24.732
  03:25:18 STEP 2275 H=2275:00 +CPU= 15.268
  03:25:35 STEP 2276 H=2276:00 +CPU= 15.993
  03:26:00 STEP 2277 H=2277:00 +CPU= 24.233
  03:26:16 STEP 2278 H=2278:00 +CPU= 15.872
  03:26:33 STEP 2279 H=2279:00 +CPU= 15.818
  03:27:02 STEP 2280 H=2280:00 +CPU= 28.974
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_94.zip
Uploading the intermediate file: upload_file_94.zip
  03:27:18 STEP 2281 H=2281:00 +CPU= 15.613
  03:27:35 STEP 2282 H=2282:00 +CPU= 15.932
  03:28:00 STEP 2283 H=2283:00 +CPU= 24.500
  03:28:17 STEP 2284 H=2284:00 +CPU= 16.367
  03:28:33 STEP 2285 H=2285:00 +CPU= 15.488
  03:28:59 STEP 2286 H=2286:00 +CPU= 25.042
  03:29:16 STEP 2287 H=2287:00 +CPU= 15.652
  03:29:32 STEP 2288 H=2288:00 +CPU= 16.005
  03:29:58 STEP 2289 H=2289:00 +CPU= 24.552
  03:30:15 STEP 2290 H=2290:00 +CPU= 15.965
  03:30:31 STEP 2291 H=2291:00 +CPU= 15.383
  03:31:00 STEP 2292 H=2292:00 +CPU= 27.631
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002292
  03:31:17 STEP 2293 H=2293:00 +CPU= 15.837
  03:31:33 STEP 2294 H=2294:00 +CPU= 16.405
  03:31:59 STEP 2295 H=2295:00 +CPU= 24.602
  03:32:16 STEP 2296 H=2296:00 +CPU= 16.468
  03:32:33 STEP 2297 H=2297:00 +CPU= 15.951
  03:32:59 STEP 2298 H=2298:00 +CPU= 25.191
  03:33:15 STEP 2299 H=2299:00 +CPU= 15.653
  03:33:31 STEP 2300 H=2300:00 +CPU= 16.090
  03:33:56 STEP 2301 H=2301:00 +CPU= 24.333
  03:34:13 STEP 2302 H=2302:00 +CPU= 15.732
  03:34:30 STEP 2303 H=2303:00 +CPU= 16.318
  03:35:00 STEP 2304 H=2304:00 +CPU= 29.780
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_95.zip
Uploading the intermediate file: upload_file_95.zip
  03:35:16 STEP 2305 H=2305:00 +CPU= 15.313
  03:35:33 STEP 2306 H=2306:00 +CPU= 16.043
  03:35:58 STEP 2307 H=2307:00 +CPU= 24.487
  03:36:15 STEP 2308 H=2308:00 +CPU= 15.890
  03:36:31 STEP 2309 H=2309:00 +CPU= 15.579
  03:36:57 STEP 2310 H=2310:00 +CPU= 24.744
  03:37:13 STEP 2311 H=2311:00 +CPU= 15.597
  03:37:30 STEP 2312 H=2312:00 +CPU= 15.790
  03:37:56 STEP 2313 H=2313:00 +CPU= 24.457
  03:38:12 STEP 2314 H=2314:00 +CPU= 15.980
  03:38:28 STEP 2315 H=2315:00 +CPU= 15.662
  03:38:58 STEP 2316 H=2316:00 +CPU= 27.795
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002316
  03:39:14 STEP 2317 H=2317:00 +CPU= 15.682
  03:39:31 STEP 2318 H=2318:00 +CPU= 16.320
  03:39:57 STEP 2319 H=2319:00 +CPU= 25.059
  03:40:15 STEP 2320 H=2320:00 +CPU= 16.774
  03:40:31 STEP 2321 H=2321:00 +CPU= 15.805
  03:40:57 STEP 2322 H=2322:00 +CPU= 25.229
  03:41:12 STEP 2323 H=2323:00 +CPU= 15.449
  03:41:29 STEP 2324 H=2324:00 +CPU= 15.983
  03:41:55 STEP 2325 H=2325:00 +CPU= 24.779
  03:42:11 STEP 2326 H=2326:00 +CPU= 15.973
  03:42:28 STEP 2327 H=2327:00 +CPU= 15.920
  03:42:58 STEP 2328 H=2328:00 +CPU= 29.478
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_96.zip
Uploading the intermediate file: upload_file_96.zip
  03:43:15 STEP 2329 H=2329:00 +CPU= 15.606
  03:43:31 STEP 2330 H=2330:00 +CPU= 16.049
  03:43:57 STEP 2331 H=2331:00 +CPU= 24.481
  03:44:13 STEP 2332 H=2332:00 +CPU= 15.808
  03:44:29 STEP 2333 H=2333:00 +CPU= 15.323
  03:44:55 STEP 2334 H=2334:00 +CPU= 24.649
  03:45:11 STEP 2335 H=2335:00 +CPU= 15.710
  03:45:28 STEP 2336 H=2336:00 +CPU= 15.863
  03:45:53 STEP 2337 H=2337:00 +CPU= 24.317
  03:46:10 STEP 2338 H=2338:00 +CPU= 16.611
  03:46:27 STEP 2339 H=2339:00 +CPU= 16.136
  03:46:56 STEP 2340 H=2340:00 +CPU= 27.734
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002340
  03:47:12 STEP 2341 H=2341:00 +CPU= 16.163
  03:47:29 STEP 2342 H=2342:00 +CPU= 16.136
  03:47:54 STEP 2343 H=2343:00 +CPU= 24.513
  03:48:11 STEP 2344 H=2344:00 +CPU= 16.085
  03:48:27 STEP 2345 H=2345:00 +CPU= 15.628
  03:48:53 STEP 2346 H=2346:00 +CPU= 25.052
  03:49:09 STEP 2347 H=2347:00 +CPU= 15.580
  03:49:25 STEP 2348 H=2348:00 +CPU= 15.770
  03:49:51 STEP 2349 H=2349:00 +CPU= 24.435
  03:50:07 STEP 2350 H=2350:00 +CPU= 16.007
  03:50:23 STEP 2351 H=2351:00 +CPU= 15.699
  03:50:54 STEP 2352 H=2352:00 +CPU= 29.345
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002352
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002340
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002340
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_97.zip
Uploading the intermediate file: upload_file_97.zip
  03:51:10 STEP 2353 H=2353:00 +CPU= 15.626
  03:51:26 STEP 2354 H=2354:00 +CPU= 16.000
  03:51:52 STEP 2355 H=2355:00 +CPU= 24.562
  03:52:09 STEP 2356 H=2356:00 +CPU= 16.351
  03:52:25 STEP 2357 H=2357:00 +CPU= 15.364
  03:52:51 STEP 2358 H=2358:00 +CPU= 25.121
  03:53:07 STEP 2359 H=2359:00 +CPU= 15.803
  03:53:24 STEP 2360 H=2360:00 +CPU= 16.107
  03:53:50 STEP 2361 H=2361:00 +CPU= 24.262
  03:54:07 STEP 2362 H=2362:00 +CPU= 16.097
  03:54:23 STEP 2363 H=2363:00 +CPU= 15.755
  03:54:52 STEP 2364 H=2364:00 +CPU= 27.311
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002364
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002364
  03:55:08 STEP 2365 H=2365:00 +CPU= 15.762
  03:55:25 STEP 2366 H=2366:00 +CPU= 16.021
  03:55:50 STEP 2367 H=2367:00 +CPU= 24.523
  03:56:07 STEP 2368 H=2368:00 +CPU= 15.958
  03:56:23 STEP 2369 H=2369:00 +CPU= 15.743
  03:56:49 STEP 2370 H=2370:00 +CPU= 25.174
  03:57:05 STEP 2371 H=2371:00 +CPU= 15.889
  03:57:22 STEP 2372 H=2372:00 +CPU= 16.294
  03:57:47 STEP 2373 H=2373:00 +CPU= 24.287
  03:58:04 STEP 2374 H=2374:00 +CPU= 16.098
  03:58:20 STEP 2375 H=2375:00 +CPU= 16.267
  03:58:51 STEP 2376 H=2376:00 +CPU= 29.774
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002376
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002352
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002364
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002364
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_98.zip
Uploading the intermediate file: upload_file_98.zip
  03:59:08 STEP 2377 H=2377:00 +CPU= 15.631
Uploading trickle at timestep: 8553600
  03:59:24 STEP 2378 H=2378:00 +CPU= 16.165
  03:59:49 STEP 2379 H=2379:00 +CPU= 24.185
  04:00:06 STEP 2380 H=2380:00 +CPU= 15.738
  04:00:22 STEP 2381 H=2381:00 +CPU= 15.600
  04:00:48 STEP 2382 H=2382:00 +CPU= 25.229
  04:01:04 STEP 2383 H=2383:00 +CPU= 15.705
  04:01:20 STEP 2384 H=2384:00 +CPU= 15.780
  04:01:46 STEP 2385 H=2385:00 +CPU= 24.564
  04:02:03 STEP 2386 H=2386:00 +CPU= 16.023
  04:02:18 STEP 2387 H=2387:00 +CPU= 15.394
  04:02:46 STEP 2388 H=2388:00 +CPU= 27.471
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002388
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002388
  04:03:03 STEP 2389 H=2389:00 +CPU= 15.650
  04:03:19 STEP 2390 H=2390:00 +CPU= 16.220
  04:03:45 STEP 2391 H=2391:00 +CPU= 24.486
  04:04:02 STEP 2392 H=2392:00 +CPU= 16.249
  04:04:18 STEP 2393 H=2393:00 +CPU= 15.693
  04:04:44 STEP 2394 H=2394:00 +CPU= 25.317
  04:05:01 STEP 2395 H=2395:00 +CPU= 15.529
  04:05:18 STEP 2396 H=2396:00 +CPU= 16.146
  04:05:43 STEP 2397 H=2397:00 +CPU= 24.506
  04:06:00 STEP 2398 H=2398:00 +CPU= 15.767
  04:06:16 STEP 2399 H=2399:00 +CPU= 15.832
  04:06:47 STEP 2400 H=2400:00 +CPU= 30.432
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002400
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002376
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002388
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002388
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_99.zip
Uploading the intermediate file: upload_file_99.zip
  04:07:04 STEP 2401 H=2401:00 +CPU= 16.340
  04:07:21 STEP 2402 H=2402:00 +CPU= 16.139
  04:07:46 STEP 2403 H=2403:00 +CPU= 24.464
  04:08:03 STEP 2404 H=2404:00 +CPU= 15.971
  04:08:19 STEP 2405 H=2405:00 +CPU= 15.649
  04:08:45 STEP 2406 H=2406:00 +CPU= 25.059
  04:09:01 STEP 2407 H=2407:00 +CPU= 15.357
  04:09:17 STEP 2408 H=2408:00 +CPU= 15.906
  04:09:42 STEP 2409 H=2409:00 +CPU= 24.189
  04:09:59 STEP 2410 H=2410:00 +CPU= 16.153
  04:10:15 STEP 2411 H=2411:00 +CPU= 15.214
  04:10:44 STEP 2412 H=2412:00 +CPU= 28.062
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002412
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002412
  04:11:02 STEP 2413 H=2413:00 +CPU= 17.628
  04:11:21 STEP 2414 H=2414:00 +CPU= 17.779
  04:11:46 STEP 2415 H=2415:00 +CPU= 24.667
  04:12:01 STEP 2416 H=2416:00 +CPU= 14.938
  04:12:16 STEP 2417 H=2417:00 +CPU= 15.055
  04:12:40 STEP 2418 H=2418:00 +CPU= 23.187
  04:12:58 STEP 2419 H=2419:00 +CPU= 17.294
  04:13:18 STEP 2420 H=2420:00 +CPU= 18.613
  04:13:47 STEP 2421 H=2421:00 +CPU= 27.683
  04:14:06 STEP 2422 H=2422:00 +CPU= 18.032
  04:14:25 STEP 2423 H=2423:00 +CPU= 19.120
  04:15:02 STEP 2424 H=2424:00 +CPU= 36.027
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002424
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002400
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002412
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002412
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_100.zip
Uploading the intermediate file: upload_file_100.zip
  04:15:20 STEP 2425 H=2425:00 +CPU= 17.637
  04:15:39 STEP 2426 H=2426:00 +CPU= 18.595
  04:16:07 STEP 2427 H=2427:00 +CPU= 27.191
  04:16:26 STEP 2428 H=2428:00 +CPU= 18.024
  04:16:44 STEP 2429 H=2429:00 +CPU= 17.795
  04:17:13 STEP 2430 H=2430:00 +CPU= 27.825
  04:17:31 STEP 2431 H=2431:00 +CPU= 17.789
  04:17:50 STEP 2432 H=2432:00 +CPU= 18.138
  04:18:17 STEP 2433 H=2433:00 +CPU= 26.628
  04:18:37 STEP 2434 H=2434:00 +CPU= 18.229
  04:18:55 STEP 2435 H=2435:00 +CPU= 17.270
  04:19:26 STEP 2436 H=2436:00 +CPU= 29.704
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002436
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002436
  04:19:44 STEP 2437 H=2437:00 +CPU= 16.900
  04:20:03 STEP 2438 H=2438:00 +CPU= 18.111
  04:20:32 STEP 2439 H=2439:00 +CPU= 27.503
  04:20:51 STEP 2440 H=2440:00 +CPU= 17.948
  04:21:10 STEP 2441 H=2441:00 +CPU= 17.691
  04:21:35 STEP 2442 H=2442:00 +CPU= 25.114
  04:21:55 STEP 2443 H=2443:00 +CPU= 19.232
  04:22:16 STEP 2444 H=2444:00 +CPU= 18.623
  04:22:47 STEP 2445 H=2445:00 +CPU= 29.276
  04:23:10 STEP 2446 H=2446:00 +CPU= 20.070
  04:23:31 STEP 2447 H=2447:00 +CPU= 19.849
  04:24:06 STEP 2448 H=2448:00 +CPU= 33.742
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002448
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002424
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002436
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002436
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_101.zip
Uploading the intermediate file: upload_file_101.zip
  04:24:26 STEP 2449 H=2449:00 +CPU= 18.797
  04:24:46 STEP 2450 H=2450:00 +CPU= 19.919
  04:25:14 STEP 2451 H=2451:00 +CPU= 26.865
  04:25:32 STEP 2452 H=2452:00 +CPU= 17.671
  04:25:49 STEP 2453 H=2453:00 +CPU= 16.840
  04:26:17 STEP 2454 H=2454:00 +CPU= 27.433
  04:26:32 STEP 2455 H=2455:00 +CPU= 15.518
  04:26:48 STEP 2456 H=2456:00 +CPU= 15.453
  04:27:13 STEP 2457 H=2457:00 +CPU= 24.797
  04:27:32 STEP 2458 H=2458:00 +CPU= 18.182
  04:27:51 STEP 2459 H=2459:00 +CPU= 18.726
  04:28:26 STEP 2460 H=2460:00 +CPU= 32.355
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002460
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002460
  04:28:46 STEP 2461 H=2461:00 +CPU= 19.329
  04:29:07 STEP 2462 H=2462:00 +CPU= 18.935
  04:29:38 STEP 2463 H=2463:00 +CPU= 28.990
  04:29:59 STEP 2464 H=2464:00 +CPU= 19.445
  04:30:18 STEP 2465 H=2465:00 +CPU= 18.319
  04:30:48 STEP 2466 H=2466:00 +CPU= 28.585
  04:31:06 STEP 2467 H=2467:00 +CPU= 17.895
  04:31:25 STEP 2468 H=2468:00 +CPU= 18.935
  04:31:53 STEP 2469 H=2469:00 +CPU= 27.443
  04:32:12 STEP 2470 H=2470:00 +CPU= 18.532
  04:32:32 STEP 2471 H=2471:00 +CPU= 19.560
  04:33:05 STEP 2472 H=2472:00 +CPU= 33.346
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002472
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002448
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002460
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002460
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_102.zip
Uploading the intermediate file: upload_file_102.zip
  04:33:22 STEP 2473 H=2473:00 +CPU= 16.885
  04:33:41 STEP 2474 H=2474:00 +CPU= 18.101
  04:34:08 STEP 2475 H=2475:00 +CPU= 26.706
  04:34:26 STEP 2476 H=2476:00 +CPU= 17.753
  04:34:43 STEP 2477 H=2477:00 +CPU= 16.874
  04:35:11 STEP 2478 H=2478:00 +CPU= 27.928
  04:35:29 STEP 2479 H=2479:00 +CPU= 17.761
  04:35:48 STEP 2480 H=2480:00 +CPU= 18.043
  04:36:19 STEP 2481 H=2481:00 +CPU= 30.027
  04:36:40 STEP 2482 H=2482:00 +CPU= 20.091
  04:36:59 STEP 2483 H=2483:00 +CPU= 18.613
  04:37:33 STEP 2484 H=2484:00 +CPU= 32.739
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002484
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002484
  04:37:53 STEP 2485 H=2485:00 +CPU= 19.349
  04:38:13 STEP 2486 H=2486:00 +CPU= 18.972
  04:38:40 STEP 2487 H=2487:00 +CPU= 26.255
  04:38:59 STEP 2488 H=2488:00 +CPU= 17.765
  04:39:17 STEP 2489 H=2489:00 +CPU= 17.401
  04:39:47 STEP 2490 H=2490:00 +CPU= 27.813
  04:40:04 STEP 2491 H=2491:00 +CPU= 17.265
  04:40:24 STEP 2492 H=2492:00 +CPU= 17.905
  04:40:52 STEP 2493 H=2493:00 +CPU= 26.860
  04:41:09 STEP 2494 H=2494:00 +CPU= 16.750
  04:41:27 STEP 2495 H=2495:00 +CPU= 17.244
  04:42:02 STEP 2496 H=2496:00 +CPU= 32.524
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002496
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002472
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002484
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002484
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_103.zip
Uploading the intermediate file: upload_file_103.zip
  04:42:20 STEP 2497 H=2497:00 +CPU= 17.539
  04:42:38 STEP 2498 H=2498:00 +CPU= 17.714
  04:43:08 STEP 2499 H=2499:00 +CPU= 28.225
  04:43:27 STEP 2500 H=2500:00 +CPU= 18.447
  04:43:45 STEP 2501 H=2501:00 +CPU= 17.020
  04:44:16 STEP 2502 H=2502:00 +CPU= 29.854
  04:44:35 STEP 2503 H=2503:00 +CPU= 18.396
  04:44:54 STEP 2504 H=2504:00 +CPU= 18.698
  04:45:25 STEP 2505 H=2505:00 +CPU= 29.807
  04:45:44 STEP 2506 H=2506:00 +CPU= 19.408
  04:46:05 STEP 2507 H=2507:00 +CPU= 19.868
  04:46:43 STEP 2508 H=2508:00 +CPU= 35.669
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002508
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002508
  04:47:06 STEP 2509 H=2509:00 +CPU= 20.900
  04:47:26 STEP 2510 H=2510:00 +CPU= 19.998
  04:47:59 STEP 2511 H=2511:00 +CPU= 31.803
  04:48:21 STEP 2512 H=2512:00 +CPU= 20.906
  04:48:41 STEP 2513 H=2513:00 +CPU= 19.143
  04:49:12 STEP 2514 H=2514:00 +CPU= 30.552
  04:49:32 STEP 2515 H=2515:00 +CPU= 19.517
  04:49:51 STEP 2516 H=2516:00 +CPU= 18.739
  04:50:23 STEP 2517 H=2517:00 +CPU= 31.141
  04:50:43 STEP 2518 H=2518:00 +CPU= 20.033
  04:51:06 STEP 2519 H=2519:00 +CPU= 20.858
  04:51:45 STEP 2520 H=2520:00 +CPU= 37.596
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002520
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002496
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002508
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002508
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_104.zip
Uploading the intermediate file: upload_file_104.zip
  04:52:05 STEP 2521 H=2521:00 +CPU= 19.215
  04:52:24 STEP 2522 H=2522:00 +CPU= 19.367
  04:52:56 STEP 2523 H=2523:00 +CPU= 30.496
  04:53:16 STEP 2524 H=2524:00 +CPU= 20.319
  04:53:36 STEP 2525 H=2525:00 +CPU= 19.119
  04:54:06 STEP 2526 H=2526:00 +CPU= 29.581
  04:54:24 STEP 2527 H=2527:00 +CPU= 18.255
  04:54:44 STEP 2528 H=2528:00 +CPU= 19.471
  04:55:14 STEP 2529 H=2529:00 +CPU= 29.544
  04:55:35 STEP 2530 H=2530:00 +CPU= 20.840
  04:55:55 STEP 2531 H=2531:00 +CPU= 20.054
  04:56:37 STEP 2532 H=2532:00 +CPU= 39.831
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002532
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002532
  04:57:01 STEP 2533 H=2533:00 +CPU= 23.536
  04:57:27 STEP 2534 H=2534:00 +CPU= 24.469
  04:58:10 STEP 2535 H=2535:00 +CPU= 41.718
  04:58:41 STEP 2536 H=2536:00 +CPU= 29.407
  04:59:11 STEP 2537 H=2537:00 +CPU= 27.680
  04:59:58 STEP 2538 H=2538:00 +CPU= 43.683
  05:00:29 STEP 2539 H=2539:00 +CPU= 27.654
  05:01:01 STEP 2540 H=2540:00 +CPU= 28.949
  05:01:41 STEP 2541 H=2541:00 +CPU= 38.026
  05:02:06 STEP 2542 H=2542:00 +CPU= 24.315
  05:02:33 STEP 2543 H=2543:00 +CPU= 26.090
  05:03:25 STEP 2544 H=2544:00 +CPU= 47.445
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002544
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002520
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002532
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002532
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_105.zip
Uploading the intermediate file: upload_file_105.zip
  05:03:56 STEP 2545 H=2545:00 +CPU= 28.022
  05:04:26 STEP 2546 H=2546:00 +CPU= 28.078
  05:05:11 STEP 2547 H=2547:00 +CPU= 42.963
  05:05:43 STEP 2548 H=2548:00 +CPU= 30.651
  05:06:12 STEP 2549 H=2549:00 +CPU= 27.260
  05:07:02 STEP 2550 H=2550:00 +CPU= 49.142
  05:07:31 STEP 2551 H=2551:00 +CPU= 27.838
  05:08:02 STEP 2552 H=2552:00 +CPU= 29.025
  05:08:46 STEP 2553 H=2553:00 +CPU= 41.635
  05:09:15 STEP 2554 H=2554:00 +CPU= 27.357
  05:09:44 STEP 2555 H=2555:00 +CPU= 27.850
  05:10:31 STEP 2556 H=2556:00 +CPU= 44.838
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002556
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002556
  05:11:02 STEP 2557 H=2557:00 +CPU= 28.395
  05:11:31 STEP 2558 H=2558:00 +CPU= 27.052
  05:12:11 STEP 2559 H=2559:00 +CPU= 38.189
  05:12:38 STEP 2560 H=2560:00 +CPU= 25.557
  05:13:06 STEP 2561 H=2561:00 +CPU= 25.840
  05:13:48 STEP 2562 H=2562:00 +CPU= 39.564
  05:14:17 STEP 2563 H=2563:00 +CPU= 27.672
  05:14:48 STEP 2564 H=2564:00 +CPU= 29.823
  05:15:27 STEP 2565 H=2565:00 +CPU= 37.709
  05:15:56 STEP 2566 H=2566:00 +CPU= 27.710
  05:16:23 STEP 2567 H=2567:00 +CPU= 23.921
  05:17:15 STEP 2568 H=2568:00 +CPU= 48.995
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002568
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002544
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002556
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002556
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_106.zip
Uploading the intermediate file: upload_file_106.zip
  05:17:46 STEP 2569 H=2569:00 +CPU= 28.037
  05:18:19 STEP 2570 H=2570:00 +CPU= 31.134
  05:19:06 STEP 2571 H=2571:00 +CPU= 43.968
  05:19:41 STEP 2572 H=2572:00 +CPU= 30.584
  05:20:15 STEP 2573 H=2573:00 +CPU= 30.545
  05:21:04 STEP 2574 H=2574:00 +CPU= 44.772
  05:21:35 STEP 2575 H=2575:00 +CPU= 28.868
  05:22:10 STEP 2576 H=2576:00 +CPU= 31.961
  05:22:56 STEP 2577 H=2577:00 +CPU= 43.129
  05:23:28 STEP 2578 H=2578:00 +CPU= 29.948
  05:24:00 STEP 2579 H=2579:00 +CPU= 28.674
  05:24:52 STEP 2580 H=2580:00 +CPU= 49.249
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002580
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002580
  05:25:21 STEP 2581 H=2581:00 +CPU= 27.082
  05:25:50 STEP 2582 H=2582:00 +CPU= 27.493
  05:26:34 STEP 2583 H=2583:00 +CPU= 41.054
  05:27:01 STEP 2584 H=2584:00 +CPU= 25.644
  05:27:28 STEP 2585 H=2585:00 +CPU= 26.218
  05:28:12 STEP 2586 H=2586:00 +CPU= 41.236
  05:28:41 STEP 2587 H=2587:00 +CPU= 26.800
  05:29:12 STEP 2588 H=2588:00 +CPU= 29.494
  05:29:55 STEP 2589 H=2589:00 +CPU= 40.049
  05:30:24 STEP 2590 H=2590:00 +CPU= 26.015
  05:30:51 STEP 2591 H=2591:00 +CPU= 26.004
  05:31:38 STEP 2592 H=2592:00 +CPU= 44.395
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002592
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002568
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002580
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002580
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_107.zip
Uploading the intermediate file: upload_file_107.zip
  05:32:03 STEP 2593 H=2593:00 +CPU= 23.770
  05:32:29 STEP 2594 H=2594:00 +CPU= 24.348
  05:33:10 STEP 2595 H=2595:00 +CPU= 38.238
  05:33:37 STEP 2596 H=2596:00 +CPU= 25.243
  05:34:04 STEP 2597 H=2597:00 +CPU= 25.118
  05:34:45 STEP 2598 H=2598:00 +CPU= 37.851
  05:35:15 STEP 2599 H=2599:00 +CPU= 26.577
  05:35:44 STEP 2600 H=2600:00 +CPU= 27.168
  05:36:29 STEP 2601 H=2601:00 +CPU= 42.655
  05:36:59 STEP 2602 H=2602:00 +CPU= 27.479
  05:37:25 STEP 2603 H=2603:00 +CPU= 24.945
  05:38:09 STEP 2604 H=2604:00 +CPU= 41.224
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002604
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002604
  05:38:35 STEP 2605 H=2605:00 +CPU= 24.110
  05:38:59 STEP 2606 H=2606:00 +CPU= 22.577
  05:39:35 STEP 2607 H=2607:00 +CPU= 34.632
  05:40:00 STEP 2608 H=2608:00 +CPU= 23.232
  05:40:24 STEP 2609 H=2609:00 +CPU= 22.300
  05:41:01 STEP 2610 H=2610:00 +CPU= 34.894
  05:41:24 STEP 2611 H=2611:00 +CPU= 21.581
  05:41:48 STEP 2612 H=2612:00 +CPU= 22.623
  05:42:23 STEP 2613 H=2613:00 +CPU= 33.578
  05:42:48 STEP 2614 H=2614:00 +CPU= 23.718
  05:43:12 STEP 2615 H=2615:00 +CPU= 22.763
  05:43:55 STEP 2616 H=2616:00 +CPU= 40.588
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002616
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002616
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002592
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMGGhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMSHhq0f+002604
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12191074/ICMUAhq0f+002604
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0430_2005050100_123_974_12191074_0_r1464522189_108.zip
Uploading the intermediate file: upload_file_108.zip
  05:44:17 STEP 2617 H=2617:00 +CPU= 21.550
Uploading trickle at timestep: 9417600
  05:44:41 STEP 2618 H=2618:00 +CPU= 22.784
  05:45:18 STEP 2619 H=2619:00 +CPU= 34.714
  05:45:43 STEP 2620 H=2620:00 +CPU= 23.280
  05:46:05 STEP 2621 H=2621:00 +CPU= 21.510
  05:46:44 STEP 2622 H=2622:00 +CPU= 36.255
  05:47:08 STEP 2623 H=2623:00 +CPU= 22.365
  05:47:32 STEP 2624 H=2624:00 +CPU= 23.234
  05:48:09 STEP 2625 H=2625:00 +CPU= 34.668
  05:48:33 STEP 2626 H=2626:00 +CPU= 23.028
  05:48:58 STEP 2627 H=2627:00 +CPU= 22.810
  05:49:40 STEP 2628 H=2628:00 +CPU= 38.853
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMGGhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMSHhq0f+002628
Moving to projects directory: /var/lib/boinc-client/slots/1/ICMUAhq0f+002628
  05:50:04 STEP 2629 H=2629:00 +CPU= 22.994
  05:50:27 STEP 2630 H=2630:00 +CPU= 22.800
  05:51:02 STEP 2631 H=2631:00 +CPU= 33.579
  05:51:27 STEP 2632 H=2632:00 +CPU= 23.272
  05:51:51 STEP 2633 H=2633:00 +CPU= 23.389
  05:52:29 STEP 2634 H=2634:00 +CPU= 35.996
  05:52:53 STEP 2635 H=2635:00 +CPU= 23.007
  05:53:18 STEP 2636 H=2636:00 +CPU= 22.964
  05:53:56 STEP 2637 H=2637:00 +CPU= 35.599
  05:54:22 STEP 2638 H=2638:00 +CPU= 24.325
  05:54:48 STEP 2639 H=2639:00 +CPU= 24.275
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:05:21 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 9,417,600 62,339 0.0066
09 May 2023 16:05:21 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 9,417,600 62,339 0.0066
09 May 2023 16:05:18 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 8,553,600 56,335 0.0066
09 May 2023 15:53:31 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 6,048,000 41,403 0.0068
09 May 2023 15:53:01 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 5,184,000 35,786 0.0069
09 May 2023 15:52:55 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 4,320,000 29,795 0.0069
09 May 2023 15:52:43 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 3,456,000 23,364 0.0068
09 May 2023 15:52:14 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 2,419,200 15,437 0.0064
09 May 2023 15:52:02 1520317 22276736 oifs_43r3_ps_0430_2005050100_123_974_12191074_0 1,555,200 10,043 0.0065


©2024 climateprediction.net