climateprediction.net home page
Task 22447756

Task 22447756

Name oifs_43r3_bl_a4hg_2016092300_20_1018_12294139_0
Workunit 12294139
Created 13 Jun 2024, 4:25:42 UTC
Sent 13 Jun 2024, 12:45:36 UTC
Report deadline 12 Aug 2024, 12:45:36 UTC
Received 14 Jun 2024, 23:50:34 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1536980
Run time 25 min 47 sec
CPU time 24 min 41 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 9.95 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,415.90 MB
Peak swap size 5,981.07 MB
Peak disk usage 1,359.74 MB
Stderr
<core_client_version>7.24.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4hg
(argv4) batchid: 1018
(argv5) wuid: 12294139
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2
Project directory is: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139
Copying: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_990636e16f47d90888f86529f09e5402 to: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_bl_a4hg_2016092300_20_1018_12294139.zip
Unzipping the namelist zip file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_bl_a4hg_2016092300_20_1018_12294139.zip
ic_ancil_file: ic_ancil_12294139
ifsdata_file: ifsdata_12294139
climate_data_file: clim_data_12294139
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ic_ancil_12294139.zip
Unzipping the IC ancils zip file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ic_ancil_12294139.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_55f30777bb1895f60ceeda7c95a7b65b to: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ifsdata/ifsdata_12294139.zip
Unzipping the ifsdata_zip file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ifsdata/ifsdata_12294139.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/159l_2/clim_data_12294139.zip
Unzipping the climate data zip file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/159l_2/clim_data_12294139.zip
Checking for progress XML file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/progress_file_12294139.xml
Creating progress file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/progress_file_12294139.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4hg_2016092300_20_1018_12294139_0_r1630973064
The child process has been launched with process id: 243298
Executing the command: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:185411:1718405651:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  18:54:22 STEP    0 H=   0:00 +CPU=  9.436
  18:54:30 STEP    1 H=   0:15 +CPU=  7.072
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000000
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000000
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000000
  18:54:36 STEP    2 H=   0:30 +CPU=  6.890
  18:54:43 STEP    3 H=   0:45 +CPU=  7.031
  18:54:51 STEP    4 H=   1:00 +CPU=  7.087
  18:54:58 STEP    5 H=   1:15 +CPU=  6.946
  18:55:04 STEP    6 H=   1:30 +CPU=  6.610
  18:55:11 STEP    7 H=   1:45 +CPU=  6.743
  18:55:18 STEP    8 H=   2:00 +CPU=  7.261
  18:55:25 STEP    9 H=   2:15 +CPU=  7.130
  18:55:32 STEP   10 H=   2:30 +CPU=  7.070
  18:55:39 STEP   11 H=   2:45 +CPU=  7.093
  18:55:49 STEP   12 H=   3:00 +CPU=  9.075
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000012
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000012
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000012
  18:55:55 STEP   13 H=   3:15 +CPU=  6.636
  18:56:02 STEP   14 H=   3:30 +CPU=  6.683
  18:56:09 STEP   15 H=   3:45 +CPU=  6.625
  18:56:16 STEP   16 H=   4:00 +CPU=  7.209
  18:56:23 STEP   17 H=   4:15 +CPU=  7.012
  18:56:30 STEP   18 H=   4:30 +CPU=  6.999
  18:56:37 STEP   19 H=   4:45 +CPU=  6.961
  18:56:44 STEP   20 H=   5:00 +CPU=  7.339
  18:56:51 STEP   21 H=   5:15 +CPU=  7.273
  18:56:59 STEP   22 H=   5:30 +CPU=  7.138
  18:57:06 STEP   23 H=   5:45 +CPU=  7.116
  18:57:15 STEP   24 H=   6:00 +CPU=  9.154
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000024
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000024
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000024
  18:57:22 STEP   25 H=   6:15 +CPU=  7.041
  18:57:29 STEP   26 H=   6:30 +CPU=  6.934
  18:57:36 STEP   27 H=   6:45 +CPU=  7.147
  18:57:43 STEP   28 H=   7:00 +CPU=  7.404
  18:57:50 STEP   29 H=   7:15 +CPU=  6.849
  18:57:57 STEP   30 H=   7:30 +CPU=  6.690
  18:58:04 STEP   31 H=   7:45 +CPU=  7.035
  18:58:11 STEP   32 H=   8:00 +CPU=  7.320
  18:58:19 STEP   33 H=   8:15 +CPU=  7.156
  18:58:25 STEP   34 H=   8:30 +CPU=  6.907
  18:58:32 STEP   35 H=   8:45 +CPU=  6.617
  18:58:41 STEP   36 H=   9:00 +CPU=  9.014
  18:58:48 STEP   37 H=   9:15 +CPU=  7.302
  18:58:56 STEP   38 H=   9:30 +CPU=  7.300
  18:59:03 STEP   39 H=   9:45 +CPU=  7.176
  18:59:10 STEP   40 H=  10:00 +CPU=  7.482
  18:59:17 STEP   41 H=  10:15 +CPU=  7.075
  18:59:25 STEP   42 H=  10:30 +CPU=  7.066
  18:59:31 STEP   43 H=  10:45 +CPU=  6.757
  18:59:38 STEP   44 H=  11:00 +CPU=  6.854
  18:59:45 STEP   45 H=  11:15 +CPU=  6.728
  18:59:52 STEP   46 H=  11:30 +CPU=  7.009
  18:59:59 STEP   47 H=  11:45 +CPU=  6.982
  19:00:09 STEP   48 H=  12:00 +CPU= 10.218
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000048
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000048
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000048
  19:00:16 STEP   49 H=  12:15 +CPU=  6.926
  19:00:23 STEP   50 H=  12:30 +CPU=  7.093
  19:00:30 STEP   51 H=  12:45 +CPU=  7.084
  19:00:38 STEP   52 H=  13:00 +CPU=  7.242
  19:00:44 STEP   53 H=  13:15 +CPU=  6.640
  19:00:51 STEP   54 H=  13:30 +CPU=  6.815
  19:00:58 STEP   55 H=  13:45 +CPU=  6.689
  19:01:05 STEP   56 H=  14:00 +CPU=  6.991
  19:01:12 STEP   57 H=  14:15 +CPU=  6.796
  19:01:19 STEP   58 H=  14:30 +CPU=  7.286
  19:01:26 STEP   59 H=  14:45 +CPU=  7.167
  19:01:35 STEP   60 H=  15:00 +CPU=  9.330
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000060
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000060
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000060
  19:01:42 STEP   61 H=  15:15 +CPU=  6.905
  19:01:49 STEP   62 H=  15:30 +CPU=  6.895
  19:01:56 STEP   63 H=  15:45 +CPU=  6.871
  19:02:03 STEP   64 H=  16:00 +CPU=  7.329
  19:02:11 STEP   65 H=  16:15 +CPU=  7.126
  19:02:17 STEP   66 H=  16:30 +CPU=  6.677
  19:02:24 STEP   67 H=  16:45 +CPU=  7.023
  19:02:32 STEP   68 H=  17:00 +CPU=  7.504
  19:02:38 STEP   69 H=  17:15 +CPU=  6.679
  19:02:45 STEP   70 H=  17:30 +CPU=  6.932
  19:02:52 STEP   71 H=  17:45 +CPU=  7.032
  19:03:02 STEP   72 H=  18:00 +CPU=  9.491
  19:03:09 STEP   73 H=  18:15 +CPU=  6.958
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000072
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000072
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000072
  19:03:16 STEP   74 H=  18:30 +CPU=  6.989
  19:03:23 STEP   75 H=  18:45 +CPU=  7.043
  19:03:30 STEP   76 H=  19:00 +CPU=  7.221
  19:03:38 STEP   77 H=  19:15 +CPU=  7.277
  19:03:45 STEP   78 H=  19:30 +CPU=  7.122
  19:03:51 STEP   79 H=  19:45 +CPU=  6.833
  19:03:59 STEP   80 H=  20:00 +CPU=  7.182
  19:04:06 STEP   81 H=  20:15 +CPU=  6.987
  19:04:13 STEP   82 H=  20:30 +CPU=  6.838
  19:04:20 STEP   83 H=  20:45 +CPU=  7.302
  19:04:29 STEP   84 H=  21:00 +CPU=  9.496
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000084
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000084
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000084
  19:04:36 STEP   85 H=  21:15 +CPU=  6.770
  19:04:43 STEP   86 H=  21:30 +CPU=  7.013
  19:04:50 STEP   87 H=  21:45 +CPU=  6.880
  19:04:57 STEP   88 H=  22:00 +CPU=  7.056
  19:05:04 STEP   89 H=  22:15 +CPU=  6.731
  19:05:11 STEP   90 H=  22:30 +CPU=  6.762
  19:05:17 STEP   91 H=  22:45 +CPU=  6.690
  19:05:24 STEP   92 H=  23:00 +CPU=  6.992
  19:05:31 STEP   93 H=  23:15 +CPU=  6.814
  19:05:38 STEP   94 H=  23:30 +CPU=  7.131
  19:05:45 STEP   95 H=  23:45 +CPU=  7.217
  19:05:56 STEP   96 H=  24:00 +CPU= 10.387
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000096
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000096
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000096
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000000
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000000
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000000
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000012
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000012
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000012
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000024
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000024
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000024
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000048
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000048
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000048
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000060
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000060
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000060
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000072
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000072
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000072
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000084
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000084
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000084
Zipping up the intermediate file: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_a4hg_2016092300_20_1018_12294139_0_r1630973064_0.zip
  19:06:03 STEP   97 H=  24:15 +CPU=  7.013
Uploading the intermediate file: upload_file_0.zip
  19:06:10 STEP   98 H=  24:30 +CPU=  7.038
  19:06:17 STEP   99 H=  24:45 +CPU=  6.820
  19:06:24 STEP  100 H=  25:00 +CPU=  7.231
  19:06:31 STEP  101 H=  25:15 +CPU=  6.881
  19:06:38 STEP  102 H=  25:30 +CPU=  7.109
  19:06:45 STEP  103 H=  25:45 +CPU=  7.183
  19:06:53 STEP  104 H=  26:00 +CPU=  7.465
  19:07:00 STEP  105 H=  26:15 +CPU=  7.128
  19:07:07 STEP  106 H=  26:30 +CPU=  7.160
  19:07:14 STEP  107 H=  26:45 +CPU=  7.161
  19:07:23 STEP  108 H=  27:00 +CPU=  9.256
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000108
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000108
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000108
  19:07:30 STEP  109 H=  27:15 +CPU=  6.761
  19:07:37 STEP  110 H=  27:30 +CPU=  7.125
  19:07:45 STEP  111 H=  27:45 +CPU=  7.135
  19:07:52 STEP  112 H=  28:00 +CPU=  7.289
  19:07:59 STEP  113 H=  28:15 +CPU=  6.783
  19:08:06 STEP  114 H=  28:30 +CPU=  6.935
  19:08:12 STEP  115 H=  28:45 +CPU=  6.762
  19:08:20 STEP  116 H=  29:00 +CPU=  7.332
  19:08:27 STEP  117 H=  29:15 +CPU=  7.145
  19:08:34 STEP  118 H=  29:30 +CPU=  7.223
  19:08:41 STEP  119 H=  29:45 +CPU=  7.251
  19:08:51 STEP  120 H=  30:00 +CPU=  9.548
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000120
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000120
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000120
  19:08:58 STEP  121 H=  30:15 +CPU=  7.493
  19:09:06 STEP  122 H=  30:30 +CPU=  7.116
  19:09:13 STEP  123 H=  30:45 +CPU=  7.357
  19:09:20 STEP  124 H=  31:00 +CPU=  7.393
  19:09:28 STEP  125 H=  31:15 +CPU=  7.589
  19:09:35 STEP  126 H=  31:30 +CPU=  7.375
  19:09:42 STEP  127 H=  31:45 +CPU=  6.929
  19:09:50 STEP  128 H=  32:00 +CPU=  7.612
  19:09:57 STEP  129 H=  32:15 +CPU=  7.126
  19:10:05 STEP  130 H=  32:30 +CPU=  7.390
  19:10:12 STEP  131 H=  32:45 +CPU=  7.163
  19:10:22 STEP  132 H=  33:00 +CPU=  9.959
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000132
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000132
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000132
  19:10:29 STEP  133 H=  33:15 +CPU=  7.096
  19:10:36 STEP  134 H=  33:30 +CPU=  7.091
  19:10:43 STEP  135 H=  33:45 +CPU=  7.390
  19:10:51 STEP  136 H=  34:00 +CPU=  7.429
  19:10:58 STEP  137 H=  34:15 +CPU=  6.960
  19:11:05 STEP  138 H=  34:30 +CPU=  6.857
  19:11:11 STEP  139 H=  34:45 +CPU=  6.643
  19:11:19 STEP  140 H=  35:00 +CPU=  7.316
  19:11:26 STEP  141 H=  35:15 +CPU=  7.013
  19:11:33 STEP  142 H=  35:30 +CPU=  7.606
  19:11:41 STEP  143 H=  35:45 +CPU=  7.226
  19:11:51 STEP  144 H=  36:00 +CPU= 10.525
  19:11:58 STEP  145 H=  36:15 +CPU=  7.105
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000144
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000144
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000144
  19:12:05 STEP  146 H=  36:30 +CPU=  7.114
  19:12:13 STEP  147 H=  36:45 +CPU=  7.156
  19:12:20 STEP  148 H=  37:00 +CPU=  7.432
  19:12:27 STEP  149 H=  37:15 +CPU=  7.128
  19:12:34 STEP  150 H=  37:30 +CPU=  6.974
  19:12:41 STEP  151 H=  37:45 +CPU=  7.306
  19:12:49 STEP  152 H=  38:00 +CPU=  7.290
  19:12:56 STEP  153 H=  38:15 +CPU=  7.135
  19:13:03 STEP  154 H=  38:30 +CPU=  7.183
  19:13:11 STEP  155 H=  38:45 +CPU=  7.797
  19:13:21 STEP  156 H=  39:00 +CPU=  9.981
  19:13:28 STEP  157 H=  39:15 +CPU=  7.440
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000156
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000156
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000156
  19:13:35 STEP  158 H=  39:30 +CPU=  7.039
  19:13:43 STEP  159 H=  39:45 +CPU=  7.306
  19:13:50 STEP  160 H=  40:00 +CPU=  7.617
  19:13:58 STEP  161 H=  40:15 +CPU=  7.445
  19:14:05 STEP  162 H=  40:30 +CPU=  7.294
  19:14:13 STEP  163 H=  40:45 +CPU=  7.561
  19:14:20 STEP  164 H=  41:00 +CPU=  7.523
  19:14:28 STEP  165 H=  41:15 +CPU=  7.803
  19:14:36 STEP  166 H=  41:30 +CPU=  7.555
  19:14:43 STEP  167 H=  41:45 +CPU=  7.314
  19:14:52 STEP  168 H=  42:00 +CPU=  9.344
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000168
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000168
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000168
  19:15:00 STEP  169 H=  42:15 +CPU=  7.710
  19:15:08 STEP  170 H=  42:30 +CPU=  7.507
  19:15:15 STEP  171 H=  42:45 +CPU=  7.565
  19:15:24 STEP  172 H=  43:00 +CPU=  8.582
  19:15:31 STEP  173 H=  43:15 +CPU=  7.526
  19:15:38 STEP  174 H=  43:30 +CPU=  7.084
  19:15:46 STEP  175 H=  43:45 +CPU=  7.320
  19:15:54 STEP  176 H=  44:00 +CPU=  7.770
  19:16:01 STEP  177 H=  44:15 +CPU=  7.223
  19:16:08 STEP  178 H=  44:30 +CPU=  7.269
  19:16:15 STEP  179 H=  44:45 +CPU=  7.256
  19:16:25 STEP  180 H=  45:00 +CPU=  9.511
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000180
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000180
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000180
  19:16:32 STEP  181 H=  45:15 +CPU=  7.242
  19:16:40 STEP  182 H=  45:30 +CPU=  7.455
  19:16:47 STEP  183 H=  45:45 +CPU=  7.566
  19:16:55 STEP  184 H=  46:00 +CPU=  7.482
  19:17:02 STEP  185 H=  46:15 +CPU=  7.136
  19:17:10 STEP  186 H=  46:30 +CPU=  7.658
  19:17:17 STEP  187 H=  46:45 +CPU=  7.535
  19:17:25 STEP  188 H=  47:00 +CPU=  7.668
  19:17:32 STEP  189 H=  47:15 +CPU=  7.268
  19:17:40 STEP  190 H=  47:30 +CPU=  7.645
  19:17:47 STEP  191 H=  47:45 +CPU=  7.418
  19:17:58 STEP  192 H=  48:00 +CPU= 10.614
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMGGh7zg+000192
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMSHh7zg+000192
Moving to projects directory: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/ICMUAh7zg+000192
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000096
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000096
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000096
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000108
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000108
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000108
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000120
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000120
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000120
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000132
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000132
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000132
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000144
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000144
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000144
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000156
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000156
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000156
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000168
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000168
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000168
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMGGh7zg+000180
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMSHh7zg+000180
Adding to the zip: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_12294139/ICMUAh7zg+000180
Zipping up the intermediate file: /mnt/spin/nobak/boinc/boinc-client-cpdn/projects/climateprediction.net/oifs_43r3_bl_a4hg_2016092300_20_1018_12294139_0_r1630973064_1.zip
Uploading the intermediate file: upload_file_1.zip
  19:18:06 STEP  193 H=  48:15 +CPU=  7.650
  19:18:13 STEP  194 H=  48:30 +CPU=  7.707
  19:18:21 STEP  195 H=  48:45 +CPU=  7.649
  19:18:29 STEP  196 H=  49:00 +CPU=  7.684
  19:18:36 STEP  197 H=  49:15 +CPU=  7.685
  19:18:44 STEP  198 H=  49:30 +CPU=  7.547
  19:18:51 STEP  199 H=  49:45 +CPU=  7.314
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4412MB (heap), 5557MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4412 MB (maxheap), 5557 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191856:1718407136:1484.509] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_model.exe' : sigcontextptr=0x7ffc1aca08d0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.510] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_model.exe' : sigcontextptr=0x7ffc1aca07e0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.530] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:ilia:1:1:243298:243298] [20240614:191857:1718407137:1485.530] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC6692  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000121FD8B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  0000000001207E29  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000011FDCB2  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000FAC003  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.661687659065733E-05 0.000000000000000E+00 0.729533754333928E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.253239407692350E-05 0.000000000000000E+00 0.549413214362265E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.166452019226661E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   192 STEPO   F00000000
 CREATE RESTART FILE ./srf00020000.0001                                                                                                                                                                                                                                              
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./srf00020000.0001 MODE=w
 IO-STREAM CLOSED - ./srf00020000.0001
 MPI-TASK:   1 - ************ BYTES IN     46 RECORDS TRANSFERRED IN    0.5387 SECONDS******** Mbytes/s, TOTAL TIME=    0.6592(81.7%)
 CREATE RESTART CONTROL FILE ./rcf                                                                                                                   
 WRRESF: DELETED PREVIOUS RESTART FILE OK: ./srf00011200.0001
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   192 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000192 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000192
 MPI-TASK:   1 -   1205350116 BYTES IN      6 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0028( 3.6%)
 NSTEP =   192 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000192 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+000192
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0023 SECONDS307564.9 Mbytes/s, TOTAL TIME=    0.0902( 2.5%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000192 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+000192
 MPI-TASK:   1 -  19285601856 BYTES IN    125 RECORDS TRANSFERRED IN    0.0014 SECONDS******** Mbytes/s, TOTAL TIME=    0.0595( 2.4%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000192 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000192
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0000 SECONDS     0.0 Mbytes/s, TOTAL TIME=    0.0014( 0.0%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000192 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000192
 MPI-TASK:   1 -   1196770387 BYTES IN      4 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0021( 4.8%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   192 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   192 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000192 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000192
 MPI-TASK:   1 -   1205350116 BYTES IN      7 RECORDS TRANSFERRED IN    0.0001 SECONDS******** Mbytes/s, TOTAL TIME=    0.0037( 2.7%)
 19:17:58 STEP  192 H=  48:00 +CPU= 10.614
 NSTEP =   193 STEPO   0AAA00AAA
 19:18:06 STEP  193 H=  48:15 +CPU=  7.650
 NSTEP =   194 STEPO   0AAA00AAA
 19:18:13 STEP  194 H=  48:30 +CPU=  7.707
ISTEP=     195 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     195 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   195 STEPO   0AAA00AAA
 19:18:21 STEP  195 H=  48:45 +CPU=  7.649
 NSTEP =   196 STEPO   0AAA00AAA
 19:18:29 STEP  196 H=  49:00 +CPU=  7.684
 NSTEP =   197 STEPO   0AAA00AAA
 19:18:36 STEP  197 H=  49:15 +CPU=  7.685
 NSTEP =   198 STEPO   0AAA00AAA
 19:18:44 STEP  198 H=  49:30 +CPU=  7.547
 NSTEP =   199 STEPO   0AAA00AAA
 19:18:51 STEP  199 H=  49:45 +CPU=  7.314
ISTEP=     200 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     200 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   200 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 19:18:06 0AAA00AAA STEPO      194     7.650    7.650    7.661     23:51     23:54 0.44030252245509E-04       2GB       0MB
 19:18:13 0AAA00AAA STEPO      195     7.707    7.707    7.743     23:59     24:02 0.44030252245509E-04       2GB       0MB
 19:18:21 0AAA00AAA STEPO      196     7.649    7.649    7.656     24:07     24:09 0.44030252245509E-04       2GB       0MB
 19:18:29 0AAA00AAA STEPO      197     7.684    7.684    7.710     24:14     24:17 0.44030252245509E-04       2GB       0MB
 19:18:36 0AAA00AAA STEPO      198     7.685    7.685    7.699     24:22     24:25 0.44030252245509E-04       2GB       0MB
 19:18:44 0AAA00AAA STEPO      199     7.547    7.547    7.562     24:30     24:32 0.44030252245509E-04       2GB       0MB
 19:18:51 0AAA00AAA STEPO      200     7.314    7.314    7.331     24:37     24:40 0.44030252245509E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     192",
 CTIME   = "00020000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         180, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /mnt/spin/nobak/boinc/boinc-client-cpdn/slots/2/progress_file_12294139.xml

<running_values>
  <last_cpu_time>1478.550000</last_cpu_time>
  <upload_file_number>2</upload_file_number>
  <last_iter>200</last_iter>
  <last_upload>172800</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net