climateprediction.net home page
Task 22440038

Task 22440038

Name oifs_43r3_bl_a4hg_2016092300_20_1017_12287639_1
Workunit 12287639
Created 7 Jun 2024, 21:18:08 UTC
Sent 7 Jun 2024, 21:39:56 UTC
Report deadline 6 Aug 2024, 21:39:56 UTC
Received 7 Jun 2024, 22:40:35 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 1 (0x00000001) Unknown error code
Computer ID 1543890
Run time 53 min 36 sec
CPU time 52 min 28 sec
Validate state Invalid
Credit 0.00
Device peak FLOPS 8.53 GFLOPS
Application version OpenIFS 43r3 Baroclinic Lifecycle v1.13
x86_64-pc-linux-gnu
Peak working set size 5,020.63 MB
Peak swap size 5,980.80 MB
Peak disk usage 1,365.45 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
process exited with code 1 (0x1, -255)</message>
<stderr_txt>
(argv0) ../../projects/climateprediction.net/oifs_43r3_bl_1.13_x86_64-pc-linux-gnu
(argv1) start_date: 2016092300
(argv2) exptid: h7zg
(argv3) unique_member_id: a4hg
(argv4) batchid: 1017
(argv5) wuid: 12287639
(argv6) fclen: 20
(argv7) app_name: oifs_43r3_bl
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/8
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_bl
version: 1.13
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_app_1.13_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_f5c66b6c8a90c5b34e54789d7be5213b to: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a4hg_2016092300_20_1017_12287639.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/8/oifs_43r3_bl_a4hg_2016092300_20_1017_12287639.zip
ic_ancil_file: ic_ancil_12287639
ifsdata_file: ifsdata_12287639
climate_data_file: clim_data_12287639
horiz_resolution: 159
vert_resolution: 91
grid_type: l_2
upload_interval: 96
utstep: 900
nfrres: restart dump frequency (steps) 48
Copying IC ancils from: ../../projects/climateprediction.net/jf_f7ecfd27aa1e68bbe7e8d19800e25c73 to: /var/lib/boinc-client/slots/8/ic_ancil_12287639.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/8/ic_ancil_12287639.zip
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_0d2e7c38ee434cead559cf3d5ed1b926 to: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12287639.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/8/ifsdata/ifsdata_12287639.zip
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/8/159l_2/clim_data_12287639.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/8/159l_2/clim_data_12287639.zip
Checking for progress XML file: /var/lib/boinc-client/slots/8/progress_file_12287639.xml
Creating progress file: /var/lib/boinc-client/slots/8/progress_file_12287639.xml
last_cpu_time: 0
upload_file_number: 0
last_iter: 0
last_upload: 0
model_completed: 0
total_length_of_simulation: 1728000
result_base_name: oifs_43r3_bl_a4hg_2016092300_20_1017_12287639_1_r670293892
The child process has been launched with process id: 16850
Executing the command: /var/lib/boinc-client/slots/8/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2091] fp = 0x314f440
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [process_options@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [ignore_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [restore_default_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [trapfpe_treatment@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x820cf0 (old at 0x1cf8da0)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [signal_drhook_init@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x820cf0 (old at (nil))
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240607:234015:1717796415:0.000] [catch_signals@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  23:40:31 STEP    0 H=   0:00 +CPU= 13.457
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000000
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000000
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000000
  23:40:42 STEP    1 H=   0:15 +CPU= 11.485
  23:40:55 STEP    2 H=   0:30 +CPU= 12.150
  23:41:07 STEP    3 H=   0:45 +CPU= 12.263
  23:41:20 STEP    4 H=   1:00 +CPU= 13.173
  23:41:33 STEP    5 H=   1:15 +CPU= 12.856
  23:41:47 STEP    6 H=   1:30 +CPU= 13.797
  23:42:00 STEP    7 H=   1:45 +CPU= 13.596
  23:42:14 STEP    8 H=   2:00 +CPU= 14.154
  23:42:28 STEP    9 H=   2:15 +CPU= 13.792
  23:42:43 STEP   10 H=   2:30 +CPU= 15.025
  23:42:59 STEP   11 H=   2:45 +CPU= 15.333
  23:43:18 STEP   12 H=   3:00 +CPU= 19.262
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000012
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000012
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000012
  23:43:34 STEP   13 H=   3:15 +CPU= 15.542
  23:43:50 STEP   14 H=   3:30 +CPU= 16.337
  23:44:05 STEP   15 H=   3:45 +CPU= 14.972
  23:44:20 STEP   16 H=   4:00 +CPU= 14.570
  23:44:34 STEP   17 H=   4:15 +CPU= 14.376
  23:44:49 STEP   18 H=   4:30 +CPU= 14.890
  23:45:03 STEP   19 H=   4:45 +CPU= 13.972
  23:45:18 STEP   20 H=   5:00 +CPU= 15.453
  23:45:33 STEP   21 H=   5:15 +CPU= 14.384
  23:45:47 STEP   22 H=   5:30 +CPU= 14.359
  23:46:02 STEP   23 H=   5:45 +CPU= 14.807
  23:46:20 STEP   24 H=   6:00 +CPU= 17.932
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000024
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000024
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000024
  23:46:34 STEP   25 H=   6:15 +CPU= 14.390
  23:46:49 STEP   26 H=   6:30 +CPU= 14.286
  23:47:04 STEP   27 H=   6:45 +CPU= 14.978
  23:47:19 STEP   28 H=   7:00 +CPU= 14.733
  23:47:33 STEP   29 H=   7:15 +CPU= 14.054
  23:47:47 STEP   30 H=   7:30 +CPU= 13.983
  23:48:00 STEP   31 H=   7:45 +CPU= 13.725
  23:48:15 STEP   32 H=   8:00 +CPU= 14.571
  23:48:30 STEP   33 H=   8:15 +CPU= 14.880
  23:48:44 STEP   34 H=   8:30 +CPU= 14.588
  23:48:59 STEP   35 H=   8:45 +CPU= 14.967
  23:49:17 STEP   36 H=   9:00 +CPU= 17.464
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000036
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000036
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000036
  23:49:32 STEP   37 H=   9:15 +CPU= 14.723
  23:49:46 STEP   38 H=   9:30 +CPU= 14.238
  23:50:00 STEP   39 H=   9:45 +CPU= 14.160
  23:50:14 STEP   40 H=  10:00 +CPU= 14.048
  23:50:28 STEP   41 H=  10:15 +CPU= 14.218
  23:50:43 STEP   42 H=  10:30 +CPU= 14.060
  23:50:57 STEP   43 H=  10:45 +CPU= 14.760
  23:51:12 STEP   44 H=  11:00 +CPU= 14.837
  23:51:27 STEP   45 H=  11:15 +CPU= 14.934
  23:51:41 STEP   46 H=  11:30 +CPU= 14.344
  23:51:56 STEP   47 H=  11:45 +CPU= 14.872
  23:52:17 STEP   48 H=  12:00 +CPU= 20.148
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000048
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000048
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000048
  23:52:31 STEP   49 H=  12:15 +CPU= 14.427
  23:52:46 STEP   50 H=  12:30 +CPU= 14.847
  23:53:01 STEP   51 H=  12:45 +CPU= 14.825
  23:53:16 STEP   52 H=  13:00 +CPU= 15.037
  23:53:31 STEP   53 H=  13:15 +CPU= 14.821
  23:53:45 STEP   54 H=  13:30 +CPU= 14.480
  23:54:00 STEP   55 H=  13:45 +CPU= 15.155
  23:54:15 STEP   56 H=  14:00 +CPU= 14.433
  23:54:29 STEP   57 H=  14:15 +CPU= 14.041
  23:54:43 STEP   58 H=  14:30 +CPU= 14.462
  23:54:58 STEP   59 H=  14:45 +CPU= 14.712
  23:55:17 STEP   60 H=  15:00 +CPU= 18.586
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000060
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000060
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000060
  23:55:31 STEP   61 H=  15:15 +CPU= 14.518
  23:55:45 STEP   62 H=  15:30 +CPU= 14.148
  23:56:00 STEP   63 H=  15:45 +CPU= 14.900
  23:56:15 STEP   64 H=  16:00 +CPU= 14.636
  23:56:29 STEP   65 H=  16:15 +CPU= 14.129
  23:56:44 STEP   66 H=  16:30 +CPU= 14.417
  23:56:57 STEP   67 H=  16:45 +CPU= 13.694
  23:57:13 STEP   68 H=  17:00 +CPU= 15.256
  23:57:27 STEP   69 H=  17:15 +CPU= 14.905
  23:57:42 STEP   70 H=  17:30 +CPU= 14.234
  23:57:55 STEP   71 H=  17:45 +CPU= 13.572
  23:58:14 STEP   72 H=  18:00 +CPU= 18.838
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000072
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000072
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000072
  23:58:29 STEP   73 H=  18:15 +CPU= 14.528
  23:58:44 STEP   74 H=  18:30 +CPU= 15.310
  23:58:59 STEP   75 H=  18:45 +CPU= 15.159
  23:59:14 STEP   76 H=  19:00 +CPU= 14.807
  23:59:28 STEP   77 H=  19:15 +CPU= 13.771
  23:59:42 STEP   78 H=  19:30 +CPU= 14.214
  23:59:56 STEP   79 H=  19:45 +CPU= 13.988
  00:00:11 STEP   80 H=  20:00 +CPU= 14.615
  00:00:25 STEP   81 H=  20:15 +CPU= 14.173
  00:00:40 STEP   82 H=  20:30 +CPU= 14.962
  00:00:55 STEP   83 H=  20:45 +CPU= 15.051
  00:01:14 STEP   84 H=  21:00 +CPU= 18.468
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000084
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000084
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000084
  00:01:28 STEP   85 H=  21:15 +CPU= 14.491
  00:01:43 STEP   86 H=  21:30 +CPU= 14.434
  00:01:57 STEP   87 H=  21:45 +CPU= 14.533
  00:02:12 STEP   88 H=  22:00 +CPU= 14.844
  00:02:26 STEP   89 H=  22:15 +CPU= 14.323
  00:02:41 STEP   90 H=  22:30 +CPU= 15.030
  00:02:57 STEP   91 H=  22:45 +CPU= 15.237
  00:03:11 STEP   92 H=  23:00 +CPU= 14.726
  00:03:26 STEP   93 H=  23:15 +CPU= 14.456
  00:03:40 STEP   94 H=  23:30 +CPU= 14.578
  00:03:55 STEP   95 H=  23:45 +CPU= 14.900
  00:04:15 STEP   96 H=  24:00 +CPU= 19.230
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000096
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000096
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000000
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000012
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000024
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000036
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000048
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000060
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000072
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000084
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000084
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4hg_2016092300_20_1017_12287639_1_r670293892_0.zip
  00:04:29 STEP   97 H=  24:15 +CPU= 14.273
Uploading the intermediate file: upload_file_0.zip
  00:04:43 STEP   98 H=  24:30 +CPU= 13.863
  00:04:57 STEP   99 H=  24:45 +CPU= 14.485
  00:05:12 STEP  100 H=  25:00 +CPU= 15.215
  00:05:27 STEP  101 H=  25:15 +CPU= 14.137
  00:05:42 STEP  102 H=  25:30 +CPU= 15.385
  00:05:57 STEP  103 H=  25:45 +CPU= 14.611
  00:06:12 STEP  104 H=  26:00 +CPU= 15.386
  00:06:27 STEP  105 H=  26:15 +CPU= 15.048
  00:06:42 STEP  106 H=  26:30 +CPU= 14.547
  00:06:57 STEP  107 H=  26:45 +CPU= 15.401
  00:07:16 STEP  108 H=  27:00 +CPU= 18.500
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000108
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000108
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000108
  00:07:30 STEP  109 H=  27:15 +CPU= 14.358
  00:07:44 STEP  110 H=  27:30 +CPU= 14.348
  00:07:59 STEP  111 H=  27:45 +CPU= 15.001
  00:08:14 STEP  112 H=  28:00 +CPU= 14.857
  00:08:28 STEP  113 H=  28:15 +CPU= 13.516
  00:08:43 STEP  114 H=  28:30 +CPU= 15.589
  00:08:59 STEP  115 H=  28:45 +CPU= 15.028
  00:09:14 STEP  116 H=  29:00 +CPU= 15.317
  00:09:29 STEP  117 H=  29:15 +CPU= 14.716
  00:09:43 STEP  118 H=  29:30 +CPU= 14.839
  00:09:58 STEP  119 H=  29:45 +CPU= 14.899
  00:10:16 STEP  120 H=  30:00 +CPU= 17.969
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000120
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000120
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000120
  00:10:30 STEP  121 H=  30:15 +CPU= 14.060
  00:10:45 STEP  122 H=  30:30 +CPU= 14.059
  00:10:59 STEP  123 H=  30:45 +CPU= 14.544
  00:11:14 STEP  124 H=  31:00 +CPU= 14.901
  00:11:28 STEP  125 H=  31:15 +CPU= 14.029
  00:11:43 STEP  126 H=  31:30 +CPU= 14.521
  00:11:57 STEP  127 H=  31:45 +CPU= 14.194
  00:12:11 STEP  128 H=  32:00 +CPU= 14.191
  00:12:25 STEP  129 H=  32:15 +CPU= 14.274
  00:12:40 STEP  130 H=  32:30 +CPU= 14.586
  00:12:55 STEP  131 H=  32:45 +CPU= 14.951
  00:13:13 STEP  132 H=  33:00 +CPU= 17.671
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000132
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000132
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000132
  00:13:27 STEP  133 H=  33:15 +CPU= 14.017
  00:13:41 STEP  134 H=  33:30 +CPU= 14.462
  00:13:55 STEP  135 H=  33:45 +CPU= 14.243
  00:14:10 STEP  136 H=  34:00 +CPU= 14.678
  00:14:24 STEP  137 H=  34:15 +CPU= 13.588
  00:14:38 STEP  138 H=  34:30 +CPU= 14.021
  00:14:52 STEP  139 H=  34:45 +CPU= 14.412
  00:15:07 STEP  140 H=  35:00 +CPU= 14.443
  00:15:21 STEP  141 H=  35:15 +CPU= 13.853
  00:15:35 STEP  142 H=  35:30 +CPU= 14.336
  00:15:49 STEP  143 H=  35:45 +CPU= 14.376
  00:16:09 STEP  144 H=  36:00 +CPU= 19.514
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000144
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000144
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000144
  00:16:24 STEP  145 H=  36:15 +CPU= 14.768
  00:16:38 STEP  146 H=  36:30 +CPU= 14.247
  00:16:52 STEP  147 H=  36:45 +CPU= 13.978
  00:17:07 STEP  148 H=  37:00 +CPU= 14.831
  00:17:21 STEP  149 H=  37:15 +CPU= 14.249
  00:17:35 STEP  150 H=  37:30 +CPU= 14.239
  00:17:49 STEP  151 H=  37:45 +CPU= 13.913
  00:18:03 STEP  152 H=  38:00 +CPU= 14.246
  00:18:18 STEP  153 H=  38:15 +CPU= 14.564
  00:18:32 STEP  154 H=  38:30 +CPU= 14.250
  00:18:47 STEP  155 H=  38:45 +CPU= 14.442
  00:19:05 STEP  156 H=  39:00 +CPU= 18.203
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000156
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000156
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000156
  00:19:19 STEP  157 H=  39:15 +CPU= 13.930
  00:19:33 STEP  158 H=  39:30 +CPU= 14.435
  00:19:48 STEP  159 H=  39:45 +CPU= 14.139
  00:20:02 STEP  160 H=  40:00 +CPU= 14.537
  00:20:17 STEP  161 H=  40:15 +CPU= 14.515
  00:20:31 STEP  162 H=  40:30 +CPU= 14.332
  00:20:46 STEP  163 H=  40:45 +CPU= 14.661
  00:21:01 STEP  164 H=  41:00 +CPU= 14.835
  00:21:15 STEP  165 H=  41:15 +CPU= 14.548
  00:21:30 STEP  166 H=  41:30 +CPU= 14.537
  00:21:44 STEP  167 H=  41:45 +CPU= 14.676
  00:22:02 STEP  168 H=  42:00 +CPU= 17.841
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000168
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000168
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000168
  00:22:17 STEP  169 H=  42:15 +CPU= 14.431
  00:22:31 STEP  170 H=  42:30 +CPU= 14.607
  00:22:46 STEP  171 H=  42:45 +CPU= 14.317
  00:23:01 STEP  172 H=  43:00 +CPU= 14.976
  00:23:15 STEP  173 H=  43:15 +CPU= 14.611
  00:23:30 STEP  174 H=  43:30 +CPU= 14.864
  00:23:44 STEP  175 H=  43:45 +CPU= 14.214
  00:24:00 STEP  176 H=  44:00 +CPU= 15.279
  00:24:15 STEP  177 H=  44:15 +CPU= 15.111
  00:24:30 STEP  178 H=  44:30 +CPU= 14.825
  00:24:44 STEP  179 H=  44:45 +CPU= 14.659
  00:25:02 STEP  180 H=  45:00 +CPU= 17.996
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000180
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000180
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000180
  00:25:17 STEP  181 H=  45:15 +CPU= 14.435
  00:25:31 STEP  182 H=  45:30 +CPU= 14.500
  00:25:46 STEP  183 H=  45:45 +CPU= 14.507
  00:26:01 STEP  184 H=  46:00 +CPU= 14.989
  00:26:15 STEP  185 H=  46:15 +CPU= 14.622
  00:26:30 STEP  186 H=  46:30 +CPU= 14.701
  00:26:45 STEP  187 H=  46:45 +CPU= 14.857
  00:27:00 STEP  188 H=  47:00 +CPU= 15.112
  00:27:15 STEP  189 H=  47:15 +CPU= 14.706
  00:27:29 STEP  190 H=  47:30 +CPU= 14.089
  00:27:43 STEP  191 H=  47:45 +CPU= 14.357
  00:28:04 STEP  192 H=  48:00 +CPU= 20.751
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000192
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000192
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000192
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000096
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000108
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000120
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000132
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000144
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000156
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000168
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMGGh7zg+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMSHh7zg+000180
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_12287639/ICMUAh7zg+000180
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_bl_a4hg_2016092300_20_1017_12287639_1_r670293892_1.zip
  00:28:19 STEP  193 H=  48:15 +CPU= 14.804
Uploading the intermediate file: upload_file_1.zip
  00:28:34 STEP  194 H=  48:30 +CPU= 14.592
  00:28:48 STEP  195 H=  48:45 +CPU= 14.432
  00:29:03 STEP  196 H=  49:00 +CPU= 14.870
  00:29:17 STEP  197 H=  49:15 +CPU= 14.133
  00:29:32 STEP  198 H=  49:30 +CPU= 14.801
  00:29:47 STEP  199 H=  49:45 +CPU= 14.790
  00:30:02 STEP  200 H=  50:00 +CPU= 15.177
  00:30:18 STEP  201 H=  50:15 +CPU= 15.485
  00:30:33 STEP  202 H=  50:30 +CPU= 14.999
  00:30:48 STEP  203 H=  50:45 +CPU= 15.214
  00:31:07 STEP  204 H=  51:00 +CPU= 18.696
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMGGh7zg+000204
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMSHh7zg+000204
Moving to projects directory: /var/lib/boinc-client/slots/8/ICMUAh7zg+000204
  00:31:21 STEP  205 H=  51:15 +CPU= 14.827
  00:31:36 STEP  206 H=  51:30 +CPU= 14.905
  00:31:51 STEP  207 H=  51:45 +CPU= 14.983
  00:32:07 STEP  208 H=  52:00 +CPU= 15.123
  00:32:22 STEP  209 H=  52:15 +CPU= 15.077
  00:32:37 STEP  210 H=  52:30 +CPU= 15.789
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1538] Received signal#8 (SIGFPE) :: 4411MB (heap), 5024MB (maxrss), 0MB (maxstack), 0 (paging), nsigs = 1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1542] Also activating Harakiri-alarm (SIGALRM=14) to expire after 500s elapsed to prevent hangs, nsigs = 1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1544] Harakiri signal handler 'signal_harakiri' for signal#14 (SIGALRM) installed at 0x820890 (old at (nil))
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1617] Signal#8 was caused by floating-point overflow [memaddr=0x1cc6692], nsigs = 1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1686] Starting DrHook backtrace for signal#8, nsigs = 1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3843] 4411 MB (maxheap), 5024 MB (maxrss), 0 MB (maxstack)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :  MASTER 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :   CNT0 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :    CNT1 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :     CNT2 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :      CNT3 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :       CNT4 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :        STEPO 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :         SCAN2M 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :          GP_MODEL_HEAP 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :           GP_MODEL 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :            EC_PHYS_DRV 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :             >OMP-PHYSICS              (1001) 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :              EC_PHYS 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :               CALLPAR 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                TURBULENCE_LAYER 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                 VDFOUTER 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003248:1717799568:3153.287] [c_drhook_print_@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:3897] :                  VDFMAIN 
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' : sigcontextptr=0x7ffca9c7ea10
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 24) with addr2line-cmd
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x1dd5490] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1cc6692] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x121fd8b] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x1207e29] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x11fdcb2] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0xfac003] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xf5fa6b] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf4dbc4] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xbd7131] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0x12c5002] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0xbacbae] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0x688153] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x4cd1dc] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4b5d85] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b4f52] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x49d638] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x406fbd] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x401e7f] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e12] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x1dcc34a] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcdbe7] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x401cd5] : ??() at ??:0
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.288] [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
 Process            1  thread            1  calling linux_trbk from intel_trbk()
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:240] Backtrace(s) for program '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' : sigcontextptr=0x7ffca9c7e920
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:277] Backtrace (size = 25) with addr2line-cmd
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:302] /usr/bin/addr2line -fs -e '/var/lib/boinc-client/slots/8/oifs_43r3_model.exe' 0x83a8f3 0x83aef9 0x822540 0x1dd5490 0x1cc6692 0x121fd8b 0x1207e29 0x11fdcb2 0xfac003 0xf5fa6b 0xf4dbc4 0xbd7131 0x12c5002 0xbacbae 0x688153 0x4cd1dc 0x4b5d85 0x4b4f52 0x49d638 0x406fbd 0x401e7f 0x401e12 0x1dcc34a 0x1dcdbe7 0x401cd5
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [0]: [0x83a8f3] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [1]: [0x83aef9] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [2]: [0x822540] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [3]: [0x1dd5490] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [4]: [0x1cc6692] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [5]: [0x121fd8b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [6]: [0x1207e29] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [7]: [0x11fdcb2] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [8]: [0xfac003] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [9]: [0xf5fa6b] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [10]: [0xf4dbc4] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [11]: [0xbd7131] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [12]: [0x12c5002] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [13]: [0xbacbae] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [14]: [0x688153] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [15]: [0x4cd1dc] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [16]: [0x4b5d85] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [17]: [0x4b4f52] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [18]: [0x49d638] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [19]: [0x406fbd] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [20]: [0x401e7f] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [21]: [0x401e12] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [22]: [0x1dcc34a] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [23]: [0x1dcdbe7] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:319] [24]: [0x401cd5] : ??() at ??:0
  [LinuxTraceBack@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/utilities/linuxtrbk.c:370] End of backtrace(s)
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.310] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1734] DrHook backtrace done for signal#8, nsigs = 1
[EC_DRHOOK:Ubuntu-KI:1:1:16850:16850] [20240608:003249:1717799569:3154.310] [signal_drhook@/home/abowery/Working_folder/OpenIFS/oifs_43r3_bl/gc_oifs43r3_2/src/ifsaux/support/drhook.c:1785] Calling previous signal handler at 0x1cf8da0 for signal#8, nsigs = 1
forrtl: error (72): floating overflow
Image              PC                Routine            Line        Source             
oifs_43r3_model.e  0000000001CF935B  Unknown               Unknown  Unknown
oifs_43r3_model.e  00000000008214C1  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DD5490  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001CC6692  Unknown               Unknown  Unknown
oifs_43r3_model.e  000000000121FD8B  vdfmain_                  874  vdfmain.F90
oifs_43r3_model.e  0000000001207E29  vdfouter_                 739  vdfouter.F90
oifs_43r3_model.e  00000000011FDCB2  turbulence_layer_         158  turbulence_layer.F90
oifs_43r3_model.e  0000000000FAC003  callpar_                  869  callpar.F90
oifs_43r3_model.e  0000000000F5FA6B  ec_phys_                  670  ec_phys.F90
oifs_43r3_model.e  0000000000F4DBC4  ec_phys_drv_              599  ec_phys_drv.F90
oifs_43r3_model.e  0000000000BD7131  gp_model_                 613  gp_model.F90
oifs_43r3_model.e  00000000012C5002  gp_model_heap_             74  gp_model_heap.F90
oifs_43r3_model.e  0000000000BACBAE  scan2m_                   535  scan2m.F90
oifs_43r3_model.e  0000000000688153  stepo_                    327  stepo.F90
oifs_43r3_model.e  00000000004CD1DC  cnt4_                    1133  cnt4.F90
oifs_43r3_model.e  00000000004B5D85  cnt3_                     267  cnt3.F90
oifs_43r3_model.e  00000000004B4F52  cnt2_                      88  cnt2.F90
oifs_43r3_model.e  000000000049D638  cnt1_                      92  cnt1.F90
oifs_43r3_model.e  0000000000406FBD  cnt0_                     146  cnt0.F90
oifs_43r3_model.e  0000000000401E7F  MAIN__                     96  master.F90
oifs_43r3_model.e  0000000000401E12  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCC34A  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000001DCDBE7  Unknown               Unknown  Unknown
oifs_43r3_model.e  0000000000401CD5  Unknown               Unknown  Unknown
..The child process has been killed with signal: 6
>>> Printing last 70 lines from file: NODE.001_01

         AVE   0.161121157712972E-05 0.000000000000000E+00 0.489793809851461E-02
 GPNORM CIW                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.100947885878641E-04 0.000000000000000E+00 0.917663553707787E-02
 GPNORM CC                   AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.241859638729580E+00 0.000000000000000E+00 0.100000000000000E+01
 GPNORM RFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.253250244768005E-04 0.000000000000000E+00 0.185563469052077E-01
 GPNORM SFL                  AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.182974992624540E-04 0.000000000000000E+00 0.692642042781914E-02
 GPNORM CRFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.436869852567434E-05 0.000000000000000E+00 0.667768696296353E-02
 GPNORM CSFL                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.145079298931540E-05 0.000000000000000E+00 0.544343179617634E-02
 GPNORM PFRC                 AVERAGE               MINIMUM               MAXIMUM        
         AVE   0.146176306727086E+00 0.000000000000000E+00 0.100000000000000E+01
 NSTEP =   204 STEPO   A00000000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   204 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000204 MODE=w
 IO-STREAM CLOSED - ICMGGh7zg+000204
 MPI-TASK:   1 -    319147412 BYTES IN      6 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0056( 5.4%)
 NSTEP =   204 STEPO   0AA000000
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMSHh7zg+000204 MODE=w
 IO-STREAM CLOSED - ./ICMSHh7zg+000204
 MPI-TASK:   1 -    707399250 BYTES IN    202 RECORDS TRANSFERRED IN    0.0064 SECONDS110531.1 Mbytes/s, TOTAL TIME=    0.1552( 4.1%)
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000204 MODE=w
 IO-STREAM CLOSED - ./ICMUAh7zg+000204
 MPI-TASK:   1 -   5106358592 BYTES IN    125 RECORDS TRANSFERRED IN    0.0036 SECONDS******** Mbytes/s, TOTAL TIME=    0.1237( 2.9%)
 STEPO_FPOS WAS : B00000F00E
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000204 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000204
 MPI-TASK:   1 -           10 BYTES IN      2 RECORDS TRANSFERRED IN    0.0001 SECONDS     0.1 Mbytes/s, TOTAL TIME=    0.0025( 4.0%)
 STEPO_FPOS WAS : V00000000Y
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ./ICMUAh7zg+000204 MODE=a
 IO-STREAM CLOSED - ./ICMUAh7zg+000204
 MPI-TASK:   1 -    310567683 BYTES IN      4 RECORDS TRANSFERRED IN    0.0002 SECONDS******** Mbytes/s, TOTAL TIME=    0.0041( 4.9%)
 STEPO_FPOS WAS : T00000000M
 NSTEP =   204 STEPO   0AAA00AAA
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
  MAXGPFV : MAX. VALUE =   0.000000000000000E+000
 NSTEP =   204 SCAN2M_HPOS  P
 IO-STREAM SETUP - IOTYPE =           2  NUMIOPROCS =           1  CPATH =
 ICMGGh7zg+000204 MODE=a
 IO-STREAM CLOSED - ICMGGh7zg+000204
 MPI-TASK:   1 -    319147412 BYTES IN      7 RECORDS TRANSFERRED IN    0.0003 SECONDS******** Mbytes/s, TOTAL TIME=    0.0068( 4.4%)
 00:31:07 STEP  204 H=  51:00 +CPU= 18.696
ISTEP=     205 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     205 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   205 STEPO   0AAA00AAA
 00:31:21 STEP  205 H=  51:15 +CPU= 14.827
 NSTEP =   206 STEPO   0AAA00AAA
 00:31:36 STEP  206 H=  51:30 +CPU= 14.905
 NSTEP =   207 STEPO   0AAA00AAA
 00:31:51 STEP  207 H=  51:45 +CPU= 14.983
 NSTEP =   208 STEPO   0AAA00AAA
 00:32:07 STEP  208 H=  52:00 +CPU= 15.123
 NSTEP =   209 STEPO   0AAA00AAA
 00:32:22 STEP  209 H=  52:15 +CPU= 15.077
ISTEP=     210 RSOLINC= 1361.4482 RI0= 1365.0000 ZI0= 1361.4482 RIP0= 1361.4482
ISTEP=     210 ZI0= 1361.4482 ZSEASON= 1.000000000 REA=149597870000.00 RDEASO=149597870000.00
 NSTEP =   210 STEPO   0AAA00AAA
 00:32:37 STEP  210 H=  52:30 +CPU= 15.789
 NSTEP =   211 STEPO   0AAA00AAA
------------------------------------------------
CNT0 not found; string returned was: 'STEPO'
>>> Printing last 8 lines from file: ifs.stat

 00:31:07 0AAA00AAA STEPO      205    15.354   15.354   15.371     50:47     50:52 0.42543753665463E-04       2GB       0MB
 00:31:21 0AAA00AAA STEPO      206    14.827   14.827   14.843     51:02     51:06 0.42543753665463E-04       2GB       0MB
 00:31:36 0AAA00AAA STEPO      207    14.905   14.905   14.926     51:17     51:21 0.42543753665463E-04       2GB       0MB
 00:31:51 0AAA00AAA STEPO      208    14.983   14.983   14.999     51:31     51:36 0.42543753665463E-04       2GB       0MB
 00:32:07 0AAA00AAA STEPO      209    15.123   15.123   15.140     51:47     51:51 0.42543753665463E-04       2GB       0MB
 00:32:22 0AAA00AAA STEPO      210    15.077   15.077   15.090     52:02     52:07 0.42543753665463E-04       2GB       0MB
 00:32:37 0AAA00AAA STEPO      211    15.789   15.789   15.791     52:17     52:22 0.42543753665463E-04       2GB       0MB
------------------------------------------------
>>> Printing last 11 lines from file: rcf

 CSTEP   = "     192",
 CTIME   = "00020000      ",
 NSTEPLPP        =         201,         202,          49,      228226,      228227, 2*0,         180, 2*0,
 IPRGPNSRES      =           1,
 IPRGPEWRES      =           1,
 IPRTRWRES       =           1,
 IPRTRVRES       =           1,
 GMASS0  =  0.000000000000000E+000,
 GMASSI  =  0.000000000000000E+000
 /
------------------------------------------------
>>> Printing last 8 lines from file: /var/lib/boinc-client/slots/8/progress_file_12287639.xml

<running_values>
  <last_cpu_time>3148.440000</last_cpu_time>
  <upload_file_number>2</upload_file_number>
  <last_iter>211</last_iter>
  <last_upload>172800</last_upload>
  <model_completed>0</model_completed>
</running_values>
------------------------------------------------
..Failed, model did not complete successfully

</stderr_txt>
]]>
No trickles!


©2024 climateprediction.net