climateprediction.net home page
Task 22286289

Task 22286289

Name oifs_43r3_ps_0434_2014050100_123_983_12200078_0
Workunit 12200078
Created 24 Dec 2022, 10:10:07 UTC
Sent 24 Dec 2022, 10:28:23 UTC
Report deadline 23 Jan 2023, 10:28:23 UTC
Received 1 Jan 2023, 23:56:25 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 196 (0x000000C4) EXIT_DISK_LIMIT_EXCEEDED
Computer ID 1520317
Run time 17 hours 44 min 36 sec
CPU time 16 hours 51 min 42 sec
Validate state Invalid
Credit 11,733.12
Device peak FLOPS 8.55 GFLOPS
Application version OpenIFS 43r3 Perturbed Surface v1.05
x86_64-pc-linux-gnu
Peak working set size 4,418.39 MB
Peak swap size 4,978.16 MB
Peak disk usage 7,587.78 MB
Stderr
<core_client_version>7.18.1</core_client_version>
<![CDATA[
<message>
Disk usage limit exceeded</message>
<stderr_txt>
TEP 1802 H=1802:00 +CPU= 17.339
  04:33:51 STEP 1803 H=1803:00 +CPU= 26.983
  04:34:09 STEP 1804 H=1804:00 +CPU= 17.013
  04:34:26 STEP 1805 H=1805:00 +CPU= 16.849
  04:34:53 STEP 1806 H=1806:00 +CPU= 26.469
  04:35:11 STEP 1807 H=1807:00 +CPU= 17.643
  04:35:29 STEP 1808 H=1808:00 +CPU= 18.213
  04:35:57 STEP 1809 H=1809:00 +CPU= 27.562
  04:36:18 STEP 1810 H=1810:00 +CPU= 19.311
  04:36:37 STEP 1811 H=1811:00 +CPU= 19.055
  04:37:11 STEP 1812 H=1812:00 +CPU= 32.063
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001812
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001812
  04:37:30 STEP 1813 H=1813:00 +CPU= 19.024
  04:37:51 STEP 1814 H=1814:00 +CPU= 19.441
  04:38:20 STEP 1815 H=1815:00 +CPU= 27.851
  04:38:40 STEP 1816 H=1816:00 +CPU= 17.859
  04:38:57 STEP 1817 H=1817:00 +CPU= 17.049
  04:39:26 STEP 1818 H=1818:00 +CPU= 27.360
  04:39:44 STEP 1819 H=1819:00 +CPU= 17.022
  04:40:03 STEP 1820 H=1820:00 +CPU= 17.808
  04:40:31 STEP 1821 H=1821:00 +CPU= 26.534
  04:40:49 STEP 1822 H=1822:00 +CPU= 17.606
  04:41:07 STEP 1823 H=1823:00 +CPU= 17.132
  04:41:42 STEP 1824 H=1824:00 +CPU= 32.522
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001824
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001800
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001812
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001812
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_75.zip
Uploading the intermediate file: upload_file_75.zip
  04:42:01 STEP 1825 H=1825:00 +CPU= 17.339
  04:42:19 STEP 1826 H=1826:00 +CPU= 17.106
  04:42:47 STEP 1827 H=1827:00 +CPU= 26.708
  04:43:07 STEP 1828 H=1828:00 +CPU= 18.350
  04:43:25 STEP 1829 H=1829:00 +CPU= 17.340
  04:43:54 STEP 1830 H=1830:00 +CPU= 27.831
  04:44:12 STEP 1831 H=1831:00 +CPU= 17.484
  04:44:31 STEP 1832 H=1832:00 +CPU= 18.287
  04:45:01 STEP 1833 H=1833:00 +CPU= 29.375
  04:45:21 STEP 1834 H=1834:00 +CPU= 19.238
  04:45:40 STEP 1835 H=1835:00 +CPU= 19.153
  04:46:15 STEP 1836 H=1836:00 +CPU= 33.587
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001836
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001836
  04:46:37 STEP 1837 H=1837:00 +CPU= 20.634
  04:46:59 STEP 1838 H=1838:00 +CPU= 20.830
  04:47:32 STEP 1839 H=1839:00 +CPU= 32.244
  04:47:53 STEP 1840 H=1840:00 +CPU= 20.395
  04:48:16 STEP 1841 H=1841:00 +CPU= 21.090
  04:48:48 STEP 1842 H=1842:00 +CPU= 30.630
  04:49:07 STEP 1843 H=1843:00 +CPU= 19.357
  04:49:27 STEP 1844 H=1844:00 +CPU= 19.535
  04:49:58 STEP 1845 H=1845:00 +CPU= 30.062
  04:50:18 STEP 1846 H=1846:00 +CPU= 19.647
  04:50:39 STEP 1847 H=1847:00 +CPU= 20.210
  04:51:19 STEP 1848 H=1848:00 +CPU= 37.135
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001848
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001824
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001836
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001836
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_76.zip
Uploading the intermediate file: upload_file_76.zip
  04:51:39 STEP 1849 H=1849:00 +CPU= 19.172
  04:51:59 STEP 1850 H=1850:00 +CPU= 18.924
  04:52:28 STEP 1851 H=1851:00 +CPU= 28.685
  04:52:48 STEP 1852 H=1852:00 +CPU= 19.945
  04:53:08 STEP 1853 H=1853:00 +CPU= 19.625
  04:53:39 STEP 1854 H=1854:00 +CPU= 29.853
  04:53:57 STEP 1855 H=1855:00 +CPU= 18.561
  04:54:16 STEP 1856 H=1856:00 +CPU= 18.685
  04:54:46 STEP 1857 H=1857:00 +CPU= 29.035
  04:55:05 STEP 1858 H=1858:00 +CPU= 18.911
  04:55:26 STEP 1859 H=1859:00 +CPU= 19.825
  04:56:01 STEP 1860 H=1860:00 +CPU= 34.607
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001860
  04:56:24 STEP 1861 H=1861:00 +CPU= 22.265
  04:56:52 STEP 1862 H=1862:00 +CPU= 27.119
Quit request received from BOINC client, ending the child process
(argv0) ../../projects/climateprediction.net/oifs_43r3_ps_1.05_x86_64-pc-linux-gnu
(argv1) start_date: 2014050100
(argv2) exptid: hq0f
(argv3) unique_member_id: 0434
(argv4) batchid: 983
(argv5) wuid: 12200078
(argv6) fclen: 123
(argv7) app_name: oifs_43r3_ps
(argv8) nthreads: 1
Working directory is: /var/lib/boinc-client/slots/12
Project directory is: /var/lib/boinc-client/projects/climateprediction.net/
app name: oifs_43r3_ps
version: 1.05
Location of temp folder: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078
..mkdir for temp folder for results failed
Copying: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Unzipping the app zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_app_1.05_x86_64-pc-linux-gnu.zip
Copying the namelist files from: ../../projects/climateprediction.net/jf_df8aed3f341dd81e99b868bf5573d272 to: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0434_2014050100_123_983_12200078.zip
Unzipping the namelist zip file: /var/lib/boinc-client/slots/12/oifs_43r3_ps_0434_2014050100_123_983_12200078.zip
ic_ancil_file: ic_ancil_12200078
ifsdata_file: ifsdata_12200078
climate_data_file: clim_data_12200078
horiz_resolution: 159
vert_resolution: 60
grid_type: l_2
upload_interval: 24
utstep: 3600
nfrres: restart dump frequency (steps) 24
Copying IC ancils from: ../../projects/climateprediction.net/jf_a9a6e706ee7b9ced7a8a62e373027dd4 to: /var/lib/boinc-client/slots/12/ic_ancil_12200078.zip
Unzipping the IC ancils zip file: /var/lib/boinc-client/slots/12/ic_ancil_12200078.zip
..mkdir for ifsdata folder failed
Copying the ifsdata_file from: ../../projects/climateprediction.net/jf_8c26e774f8e717ffff4d2186b8b5e1e3 to: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12200078.zip
Unzipping the ifsdata_zip file: /var/lib/boinc-client/slots/12/ifsdata/ifsdata_12200078.zip
..mkdir for the climate data folder failed
Copying the climate data file from: ../../projects/climateprediction.net/jf_f32c668cf7829426ffd94699b48b9a2e to: /var/lib/boinc-client/slots/12/159l_2/clim_data_12200078.zip
Unzipping the climate data zip file: /var/lib/boinc-client/slots/12/159l_2/clim_data_12200078.zip
Checking for progress XML file: /var/lib/boinc-client/slots/12/progress_file_12200078.xml
Opened progress file ok : /var/lib/boinc-client/slots/12/progress_file_12200078.xml
-- Model is restarting --
Adjusting last_iter, 1863, to previous model restart step.
Creating progress file: /var/lib/boinc-client/slots/12/progress_file_12200078.xml
last_cpu_time: 44060
upload_file_number: 77
last_iter: 1847
last_upload: 6652800
model_completed: 0
total_length_of_simulation: 10627200
result_base_name: oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652
The child process has been launched with process id: 59893
Executing the command: /var/lib/boinc-client/slots/12/oifs_43r3_model.exe
[EC_DRHOOK:hostname:myproc:omptid:pid:unixtid] [YYYYMMDD:HHMMSS:epoch:walltime] [function@file:lineno] -- Max OpenMP threads = 1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2091] fp = 0x3129fe0
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2098] DR_HOOK_ALLOW_COREDUMP=-1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2104] Hardlimit for core file is now 0 (0x0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2122] DR_HOOK_PROFILE_PROC=-1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2128] DR_HOOK_PROFILE_LIMIT=-10.000
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2194] DR_HOOK_RANDOM_MEMSTAT=0  (RAND_MAX=2147483647)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2205] DR_HOOK_HASHBITS=16
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2213] DR_HOOK_NCALLSTACK=0
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2221] DR_HOOK_HARAKIRI_TIMEOUT=500
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2228] DR_HOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2235] DR_HOOK_TRAPFPE_INVALID=1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2242] DR_HOOK_TRAPFPE_DIVBYZERO=1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [process_options@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:2249] DR_HOOK_TRAPFPE_OVERFLOW=1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [ignore_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1227] DR_HOOK_IGNORE_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [restore_default_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1178] DR_HOOK_RESTORE_DEFAULT_SIGNALS=<undef>
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1937] New signal handler 'signal_drhook' for signal#6 (SIGABRT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1938] New signal handler 'signal_drhook' for signal#7 (SIGBUS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1939] New signal handler 'signal_drhook' for signal#11 (SIGSEGV) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1944] New signal handler 'signal_drhook' for signal#16 (SIGSTKFLT) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [trapfpe_treatment@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1149] DR_HOOK enables SIGFPE-related floating point trapping since DRHOOK_TRAPFPE=1
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1948] New signal handler 'signal_drhook' for signal#8 (SIGFPE) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1949] New signal handler 'signal_drhook' for signal#4 (SIGILL) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1951] New signal handler 'signal_drhook' for signal#5 (SIGTRAP) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1952] New signal handler 'signal_drhook' for signal#2 (SIGINT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1960] New signal handler 'signal_drhook' for signal#3 (SIGQUIT) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1961] New signal handler 'signal_drhook' for signal#15 (SIGTERM) at 0x81f820 (old at 0x1ce8bb0)
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1966] New signal handler 'signal_drhook' for signal#24 (SIGXCPU) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1968] New signal handler 'signal_drhook' for signal#25 (SIGXFSZ) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [signal_drhook_init@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1973] New signal handler 'signal_drhook' for signal#31 (SIGSYS) at 0x81f820 (old at (nil))
[EC_DRHOOK:roo:1:1:59893:59893] [20230102:051319:1672596799:0.000] [catch_signals@/home/glenn/github/jamie_oifs43r3.git/src/ifsaux/support/drhook.c:1111] DR_HOOK_CATCH_SIGNALS=<undef>
  05:14:12 STEP 1848 H=1848:00 +CPU= 46.231
  05:14:40 STEP 1849 H=1849:00 +CPU= 27.381
  05:15:08 STEP 1850 H=1850:00 +CPU= 26.239
  05:15:52 STEP 1851 H=1851:00 +CPU= 42.338
  05:16:19 STEP 1852 H=1852:00 +CPU= 24.849
  05:16:46 STEP 1853 H=1853:00 +CPU= 25.069
  05:17:30 STEP 1854 H=1854:00 +CPU= 42.617
  05:17:59 STEP 1855 H=1855:00 +CPU= 27.009
  05:18:31 STEP 1856 H=1856:00 +CPU= 30.080
  05:19:18 STEP 1857 H=1857:00 +CPU= 44.039
  05:19:50 STEP 1858 H=1858:00 +CPU= 29.250
  05:20:17 STEP 1859 H=1859:00 +CPU= 26.061
  05:21:10 STEP 1860 H=1860:00 +CPU= 50.623
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001860
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001860
  05:21:44 STEP 1861 H=1861:00 +CPU= 30.507
  05:22:15 STEP 1862 H=1862:00 +CPU= 30.657
  05:23:07 STEP 1863 H=1863:00 +CPU= 46.076
  05:23:40 STEP 1864 H=1864:00 +CPU= 29.630
  05:24:08 STEP 1865 H=1865:00 +CPU= 26.339
  05:24:53 STEP 1866 H=1866:00 +CPU= 42.475
  05:25:20 STEP 1867 H=1867:00 +CPU= 26.284
  05:25:51 STEP 1868 H=1868:00 +CPU= 28.367
  05:26:33 STEP 1869 H=1869:00 +CPU= 39.992
  05:27:01 STEP 1870 H=1870:00 +CPU= 26.304
  05:27:27 STEP 1871 H=1871:00 +CPU= 24.719
  05:28:18 STEP 1872 H=1872:00 +CPU= 48.773
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001872
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001848
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001860
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001860
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_77.zip
Uploading the intermediate file: upload_file_77.zip
  05:28:46 STEP 1873 H=1873:00 +CPU= 26.589
  05:29:18 STEP 1874 H=1874:00 +CPU= 29.484
  05:29:59 STEP 1875 H=1875:00 +CPU= 39.016
  05:30:27 STEP 1876 H=1876:00 +CPU= 26.320
  05:30:54 STEP 1877 H=1877:00 +CPU= 25.365
  05:31:34 STEP 1878 H=1878:00 +CPU= 37.968
  05:32:00 STEP 1879 H=1879:00 +CPU= 24.175
  05:32:25 STEP 1880 H=1880:00 +CPU= 24.224
  05:33:09 STEP 1881 H=1881:00 +CPU= 40.399
  05:33:36 STEP 1882 H=1882:00 +CPU= 24.992
  05:34:08 STEP 1883 H=1883:00 +CPU= 28.056
  05:34:53 STEP 1884 H=1884:00 +CPU= 41.572
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001884
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001884
  05:35:21 STEP 1885 H=1885:00 +CPU= 25.392
  05:35:50 STEP 1886 H=1886:00 +CPU= 26.378
  05:36:32 STEP 1887 H=1887:00 +CPU= 39.699
  05:37:02 STEP 1888 H=1888:00 +CPU= 27.549
  05:37:29 STEP 1889 H=1889:00 +CPU= 25.512
  05:38:09 STEP 1890 H=1890:00 +CPU= 37.847
  05:38:34 STEP 1891 H=1891:00 +CPU= 24.517
  05:39:00 STEP 1892 H=1892:00 +CPU= 23.943
  05:39:37 STEP 1893 H=1893:00 +CPU= 34.372
  05:40:00 STEP 1894 H=1894:00 +CPU= 21.875
  05:40:23 STEP 1895 H=1895:00 +CPU= 22.244
  05:41:05 STEP 1896 H=1896:00 +CPU= 40.143
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001896
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001872
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001884
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001884
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_78.zip
Uploading the intermediate file: upload_file_78.zip
  05:41:29 STEP 1897 H=1897:00 +CPU= 22.118
  05:41:52 STEP 1898 H=1898:00 +CPU= 21.800
  05:42:28 STEP 1899 H=1899:00 +CPU= 34.423
  05:42:52 STEP 1900 H=1900:00 +CPU= 22.851
  05:43:14 STEP 1901 H=1901:00 +CPU= 21.472
  05:43:49 STEP 1902 H=1902:00 +CPU= 33.464
  05:44:13 STEP 1903 H=1903:00 +CPU= 22.350
  05:44:36 STEP 1904 H=1904:00 +CPU= 22.210
  05:45:13 STEP 1905 H=1905:00 +CPU= 34.474
  05:45:36 STEP 1906 H=1906:00 +CPU= 22.923
  05:46:00 STEP 1907 H=1907:00 +CPU= 22.839
  05:46:40 STEP 1908 H=1908:00 +CPU= 38.604
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001908
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001908
  05:47:04 STEP 1909 H=1909:00 +CPU= 22.785
  05:47:30 STEP 1910 H=1910:00 +CPU= 24.010
  05:48:05 STEP 1911 H=1911:00 +CPU= 34.743
  05:48:29 STEP 1912 H=1912:00 +CPU= 22.934
  05:48:53 STEP 1913 H=1913:00 +CPU= 22.855
  05:49:33 STEP 1914 H=1914:00 +CPU= 36.507
  05:49:57 STEP 1915 H=1915:00 +CPU= 23.388
  05:50:20 STEP 1916 H=1916:00 +CPU= 22.618
  05:50:55 STEP 1917 H=1917:00 +CPU= 33.425
  05:51:19 STEP 1918 H=1918:00 +CPU= 23.176
  05:51:45 STEP 1919 H=1919:00 +CPU= 23.569
  05:52:30 STEP 1920 H=1920:00 +CPU= 42.113
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001920
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001896
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001908
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001908
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_79.zip
Uploading the intermediate file: upload_file_79.zip
  05:52:53 STEP 1921 H=1921:00 +CPU= 22.151
  05:53:19 STEP 1922 H=1922:00 +CPU= 24.051
  05:53:56 STEP 1923 H=1923:00 +CPU= 35.011
  05:54:20 STEP 1924 H=1924:00 +CPU= 23.695
  05:54:46 STEP 1925 H=1925:00 +CPU= 23.764
  05:55:22 STEP 1926 H=1926:00 +CPU= 34.903
  05:55:46 STEP 1927 H=1927:00 +CPU= 22.700
  05:56:10 STEP 1928 H=1928:00 +CPU= 23.716
  05:56:47 STEP 1929 H=1929:00 +CPU= 35.248
  05:57:12 STEP 1930 H=1930:00 +CPU= 23.979
  05:57:37 STEP 1931 H=1931:00 +CPU= 23.501
  05:58:17 STEP 1932 H=1932:00 +CPU= 38.165
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001932
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001932
  05:58:40 STEP 1933 H=1933:00 +CPU= 22.245
  05:59:03 STEP 1934 H=1934:00 +CPU= 22.766
  05:59:40 STEP 1935 H=1935:00 +CPU= 34.167
  06:00:04 STEP 1936 H=1936:00 +CPU= 23.211
  06:00:28 STEP 1937 H=1937:00 +CPU= 22.854
  06:01:05 STEP 1938 H=1938:00 +CPU= 34.983
  06:01:29 STEP 1939 H=1939:00 +CPU= 23.113
  06:01:54 STEP 1940 H=1940:00 +CPU= 23.675
  06:02:30 STEP 1941 H=1941:00 +CPU= 35.126
  06:02:55 STEP 1942 H=1942:00 +CPU= 23.591
  06:03:21 STEP 1943 H=1943:00 +CPU= 23.877
  06:04:06 STEP 1944 H=1944:00 +CPU= 42.160
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001944
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001920
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001932
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001932
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_80.zip
Uploading the intermediate file: upload_file_80.zip
  06:04:31 STEP 1945 H=1945:00 +CPU= 23.268
  06:04:55 STEP 1946 H=1946:00 +CPU= 23.138
  06:05:35 STEP 1947 H=1947:00 +CPU= 36.997
  06:06:01 STEP 1948 H=1948:00 +CPU= 25.391
  06:06:28 STEP 1949 H=1949:00 +CPU= 25.070
  06:07:08 STEP 1950 H=1950:00 +CPU= 39.422
  06:07:32 STEP 1951 H=1951:00 +CPU= 23.653
  06:07:57 STEP 1952 H=1952:00 +CPU= 24.513
  06:08:36 STEP 1953 H=1953:00 +CPU= 37.773
  06:09:03 STEP 1954 H=1954:00 +CPU= 26.435
  06:09:28 STEP 1955 H=1955:00 +CPU= 24.674
  06:10:17 STEP 1956 H=1956:00 +CPU= 46.236
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001956
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001956
  06:10:44 STEP 1957 H=1957:00 +CPU= 25.502
  06:11:11 STEP 1958 H=1958:00 +CPU= 25.739
  06:11:50 STEP 1959 H=1959:00 +CPU= 37.348
  06:12:18 STEP 1960 H=1960:00 +CPU= 26.238
  06:12:45 STEP 1961 H=1961:00 +CPU= 24.737
  06:13:26 STEP 1962 H=1962:00 +CPU= 38.979
  06:13:51 STEP 1963 H=1963:00 +CPU= 24.477
  06:14:19 STEP 1964 H=1964:00 +CPU= 25.363
  06:15:01 STEP 1965 H=1965:00 +CPU= 40.209
  06:15:33 STEP 1966 H=1966:00 +CPU= 29.851
  06:16:05 STEP 1967 H=1967:00 +CPU= 30.284
  06:17:08 STEP 1968 H=1968:00 +CPU= 58.606
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001968
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001944
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001956
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001956
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_81.zip
Uploading the intermediate file: upload_file_81.zip
  06:17:44 STEP 1969 H=1969:00 +CPU= 33.799
  06:18:23 STEP 1970 H=1970:00 +CPU= 35.765
  06:19:17 STEP 1971 H=1971:00 +CPU= 51.200
  06:19:56 STEP 1972 H=1972:00 +CPU= 35.891
  06:20:35 STEP 1973 H=1973:00 +CPU= 35.958
  06:21:36 STEP 1974 H=1974:00 +CPU= 54.045
  06:22:17 STEP 1975 H=1975:00 +CPU= 36.783
  06:22:53 STEP 1976 H=1976:00 +CPU= 35.141
  06:23:55 STEP 1977 H=1977:00 +CPU= 55.379
  06:24:38 STEP 1978 H=1978:00 +CPU= 38.583
  06:25:13 STEP 1979 H=1979:00 +CPU= 34.304
  06:26:19 STEP 1980 H=1980:00 +CPU= 59.427
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001980
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001980
  06:27:00 STEP 1981 H=1981:00 +CPU= 39.509
  06:27:40 STEP 1982 H=1982:00 +CPU= 37.264
  06:28:36 STEP 1983 H=1983:00 +CPU= 52.367
  06:29:16 STEP 1984 H=1984:00 +CPU= 36.733
  06:29:56 STEP 1985 H=1985:00 +CPU= 36.845
  06:30:55 STEP 1986 H=1986:00 +CPU= 55.135
  06:31:34 STEP 1987 H=1987:00 +CPU= 36.107
  06:32:13 STEP 1988 H=1988:00 +CPU= 36.239
  06:33:09 STEP 1989 H=1989:00 +CPU= 52.344
  06:33:48 STEP 1990 H=1990:00 +CPU= 36.650
  06:34:28 STEP 1991 H=1991:00 +CPU= 37.016
  06:35:35 STEP 1992 H=1992:00 +CPU= 63.012
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+001992
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001968
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001980
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001980
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_82.zip
Uploading the intermediate file: upload_file_82.zip
  06:36:12 STEP 1993 H=1993:00 +CPU= 35.580
  06:36:50 STEP 1994 H=1994:00 +CPU= 34.900
  06:37:45 STEP 1995 H=1995:00 +CPU= 51.514
  06:38:24 STEP 1996 H=1996:00 +CPU= 36.517
  06:39:03 STEP 1997 H=1997:00 +CPU= 35.751
  06:39:56 STEP 1998 H=1998:00 +CPU= 49.270
  06:40:31 STEP 1999 H=1999:00 +CPU= 33.267
  06:41:13 STEP 2000 H=2000:00 +CPU= 38.212
  06:42:11 STEP 2001 H=2001:00 +CPU= 53.430
  06:42:52 STEP 2002 H=2002:00 +CPU= 36.731
  06:43:30 STEP 2003 H=2003:00 +CPU= 34.786
  06:44:31 STEP 2004 H=2004:00 +CPU= 55.184
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002004
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002004
  06:45:06 STEP 2005 H=2005:00 +CPU= 33.839
  06:45:47 STEP 2006 H=2006:00 +CPU= 37.267
  06:46:46 STEP 2007 H=2007:00 +CPU= 54.492
  06:47:26 STEP 2008 H=2008:00 +CPU= 38.503
  06:48:06 STEP 2009 H=2009:00 +CPU= 36.201
  06:49:07 STEP 2010 H=2010:00 +CPU= 56.778
  06:49:47 STEP 2011 H=2011:00 +CPU= 37.323
  06:50:29 STEP 2012 H=2012:00 +CPU= 38.453
  06:51:25 STEP 2013 H=2013:00 +CPU= 51.804
  06:52:07 STEP 2014 H=2014:00 +CPU= 39.300
  06:52:46 STEP 2015 H=2015:00 +CPU= 36.446
  06:53:52 STEP 2016 H=2016:00 +CPU= 61.538
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002016
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+001992
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002004
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002004
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_83.zip
Uploading the intermediate file: upload_file_83.zip
  06:54:30 STEP 2017 H=2017:00 +CPU= 34.871
  06:55:10 STEP 2018 H=2018:00 +CPU= 36.354
  06:56:07 STEP 2019 H=2019:00 +CPU= 52.939
  06:56:45 STEP 2020 H=2020:00 +CPU= 35.399
  06:57:23 STEP 2021 H=2021:00 +CPU= 35.386
  06:58:17 STEP 2022 H=2022:00 +CPU= 50.867
  06:58:54 STEP 2023 H=2023:00 +CPU= 34.063
  06:59:35 STEP 2024 H=2024:00 +CPU= 37.332
  07:00:32 STEP 2025 H=2025:00 +CPU= 51.979
  07:01:00 STEP 2026 H=2026:00 +CPU= 26.018
  07:01:26 STEP 2027 H=2027:00 +CPU= 24.745
  07:02:09 STEP 2028 H=2028:00 +CPU= 39.924
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002028
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002028
  07:02:36 STEP 2029 H=2029:00 +CPU= 25.008
  07:03:04 STEP 2030 H=2030:00 +CPU= 25.968
  07:03:46 STEP 2031 H=2031:00 +CPU= 39.135
  07:04:12 STEP 2032 H=2032:00 +CPU= 24.396
  07:04:40 STEP 2033 H=2033:00 +CPU= 25.072
  07:05:21 STEP 2034 H=2034:00 +CPU= 38.472
  07:05:46 STEP 2035 H=2035:00 +CPU= 23.767
  07:06:13 STEP 2036 H=2036:00 +CPU= 24.668
  07:06:54 STEP 2037 H=2037:00 +CPU= 38.922
  07:07:21 STEP 2038 H=2038:00 +CPU= 24.617
  07:07:46 STEP 2039 H=2039:00 +CPU= 24.425
  07:08:32 STEP 2040 H=2040:00 +CPU= 43.214
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002040
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002016
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002028
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002028
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_84.zip
Uploading the intermediate file: upload_file_84.zip
  07:08:56 STEP 2041 H=2041:00 +CPU= 22.842
  07:09:21 STEP 2042 H=2042:00 +CPU= 23.463
  07:09:57 STEP 2043 H=2043:00 +CPU= 34.561
  07:10:22 STEP 2044 H=2044:00 +CPU= 23.378
  07:10:45 STEP 2045 H=2045:00 +CPU= 22.471
  07:11:23 STEP 2046 H=2046:00 +CPU= 35.596
  07:11:47 STEP 2047 H=2047:00 +CPU= 22.494
  07:12:12 STEP 2048 H=2048:00 +CPU= 22.679
  07:12:49 STEP 2049 H=2049:00 +CPU= 35.413
  07:13:15 STEP 2050 H=2050:00 +CPU= 23.300
  07:13:40 STEP 2051 H=2051:00 +CPU= 22.484
  07:14:23 STEP 2052 H=2052:00 +CPU= 38.039
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002052
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002052
  07:14:47 STEP 2053 H=2053:00 +CPU= 21.658
  07:15:10 STEP 2054 H=2054:00 +CPU= 21.435
  07:15:45 STEP 2055 H=2055:00 +CPU= 32.970
  07:16:08 STEP 2056 H=2056:00 +CPU= 21.403
  07:16:32 STEP 2057 H=2057:00 +CPU= 22.062
  07:17:12 STEP 2058 H=2058:00 +CPU= 35.425
  07:17:37 STEP 2059 H=2059:00 +CPU= 22.039
  07:18:02 STEP 2060 H=2060:00 +CPU= 22.488
  07:18:40 STEP 2061 H=2061:00 +CPU= 33.974
  07:19:03 STEP 2062 H=2062:00 +CPU= 21.914
  07:19:27 STEP 2063 H=2063:00 +CPU= 22.190
  07:20:09 STEP 2064 H=2064:00 +CPU= 39.442
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002064
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002040
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002052
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002052
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_85.zip
Uploading the intermediate file: upload_file_85.zip
  07:20:31 STEP 2065 H=2065:00 +CPU= 21.274
  07:20:54 STEP 2066 H=2066:00 +CPU= 21.970
  07:21:29 STEP 2067 H=2067:00 +CPU= 33.361
  07:21:53 STEP 2068 H=2068:00 +CPU= 22.478
  07:22:16 STEP 2069 H=2069:00 +CPU= 21.094
  07:22:50 STEP 2070 H=2070:00 +CPU= 33.154
  07:23:12 STEP 2071 H=2071:00 +CPU= 21.166
  07:23:36 STEP 2072 H=2072:00 +CPU= 22.196
  07:24:11 STEP 2073 H=2073:00 +CPU= 33.156
  07:24:34 STEP 2074 H=2074:00 +CPU= 22.376
  07:24:57 STEP 2075 H=2075:00 +CPU= 21.437
  07:25:37 STEP 2076 H=2076:00 +CPU= 37.805
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002076
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002076
  07:25:58 STEP 2077 H=2077:00 +CPU= 20.859
  07:26:22 STEP 2078 H=2078:00 +CPU= 22.495
  07:26:58 STEP 2079 H=2079:00 +CPU= 33.738
  07:27:23 STEP 2080 H=2080:00 +CPU= 22.256
  07:27:49 STEP 2081 H=2081:00 +CPU= 22.540
  07:28:24 STEP 2082 H=2082:00 +CPU= 34.441
  07:28:50 STEP 2083 H=2083:00 +CPU= 23.382
  07:29:15 STEP 2084 H=2084:00 +CPU= 23.317
  07:29:50 STEP 2085 H=2085:00 +CPU= 34.499
  07:30:14 STEP 2086 H=2086:00 +CPU= 23.255
  07:30:39 STEP 2087 H=2087:00 +CPU= 22.748
  07:31:22 STEP 2088 H=2088:00 +CPU= 40.670
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002088
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002064
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002076
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002076
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_86.zip
Uploading the intermediate file: upload_file_86.zip
  07:31:46 STEP 2089 H=2089:00 +CPU= 22.708
Uploading trickle at timestep: 7516800
  07:32:14 STEP 2090 H=2090:00 +CPU= 27.487
  07:32:50 STEP 2091 H=2091:00 +CPU= 34.519
  07:33:14 STEP 2092 H=2092:00 +CPU= 22.675
  07:33:39 STEP 2093 H=2093:00 +CPU= 22.529
  07:34:15 STEP 2094 H=2094:00 +CPU= 34.741
  07:34:39 STEP 2095 H=2095:00 +CPU= 22.385
  07:35:03 STEP 2096 H=2096:00 +CPU= 22.450
  07:35:39 STEP 2097 H=2097:00 +CPU= 34.715
  07:36:04 STEP 2098 H=2098:00 +CPU= 23.415
  07:36:28 STEP 2099 H=2099:00 +CPU= 22.652
  07:37:09 STEP 2100 H=2100:00 +CPU= 38.817
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002100
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002100
  07:37:33 STEP 2101 H=2101:00 +CPU= 22.829
  07:37:58 STEP 2102 H=2102:00 +CPU= 23.192
  07:38:34 STEP 2103 H=2103:00 +CPU= 34.510
  07:38:58 STEP 2104 H=2104:00 +CPU= 23.140
  07:39:21 STEP 2105 H=2105:00 +CPU= 21.885
  07:39:58 STEP 2106 H=2106:00 +CPU= 34.903
  07:40:22 STEP 2107 H=2107:00 +CPU= 22.168
  07:40:47 STEP 2108 H=2108:00 +CPU= 23.603
  07:41:22 STEP 2109 H=2109:00 +CPU= 33.670
  07:41:48 STEP 2110 H=2110:00 +CPU= 24.135
  07:42:13 STEP 2111 H=2111:00 +CPU= 23.403
  07:42:57 STEP 2112 H=2112:00 +CPU= 40.896
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002112
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002088
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002100
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002100
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_87.zip
Uploading the intermediate file: upload_file_87.zip
  07:43:24 STEP 2113 H=2113:00 +CPU= 23.610
  07:43:50 STEP 2114 H=2114:00 +CPU= 23.717
  07:44:29 STEP 2115 H=2115:00 +CPU= 36.446
  07:44:56 STEP 2116 H=2116:00 +CPU= 23.876
  07:45:19 STEP 2117 H=2117:00 +CPU= 22.848
  07:45:58 STEP 2118 H=2118:00 +CPU= 36.139
  07:46:21 STEP 2119 H=2119:00 +CPU= 22.158
  07:46:45 STEP 2120 H=2120:00 +CPU= 23.372
  07:47:23 STEP 2121 H=2121:00 +CPU= 35.619
  07:47:47 STEP 2122 H=2122:00 +CPU= 23.553
  07:48:10 STEP 2123 H=2123:00 +CPU= 22.438
  07:48:52 STEP 2124 H=2124:00 +CPU= 39.028
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002124
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002124
  07:49:16 STEP 2125 H=2125:00 +CPU= 22.575
  07:49:40 STEP 2126 H=2126:00 +CPU= 22.935
  07:50:17 STEP 2127 H=2127:00 +CPU= 35.309
  07:50:41 STEP 2128 H=2128:00 +CPU= 22.746
  07:51:04 STEP 2129 H=2129:00 +CPU= 22.590
  07:51:42 STEP 2130 H=2130:00 +CPU= 35.959
  07:52:06 STEP 2131 H=2131:00 +CPU= 22.522
  07:52:31 STEP 2132 H=2132:00 +CPU= 23.166
  07:53:06 STEP 2133 H=2133:00 +CPU= 33.810
  07:53:29 STEP 2134 H=2134:00 +CPU= 21.924
  07:53:53 STEP 2135 H=2135:00 +CPU= 22.423
  07:54:36 STEP 2136 H=2136:00 +CPU= 40.394
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002136
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002112
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002124
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002124
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_88.zip
Uploading the intermediate file: upload_file_88.zip
  07:54:58 STEP 2137 H=2137:00 +CPU= 21.347
  07:55:23 STEP 2138 H=2138:00 +CPU= 22.947
  07:55:59 STEP 2139 H=2139:00 +CPU= 33.776
  07:56:24 STEP 2140 H=2140:00 +CPU= 23.161
  07:56:49 STEP 2141 H=2141:00 +CPU= 22.861
  07:57:34 STEP 2142 H=2142:00 +CPU= 41.078
  07:58:04 STEP 2143 H=2143:00 +CPU= 27.214
  07:58:36 STEP 2144 H=2144:00 +CPU= 28.584
  07:59:19 STEP 2145 H=2145:00 +CPU= 40.585
  07:59:48 STEP 2146 H=2146:00 +CPU= 27.012
  08:00:20 STEP 2147 H=2147:00 +CPU= 28.780
  08:01:13 STEP 2148 H=2148:00 +CPU= 46.529
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002148
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002148
  08:01:43 STEP 2149 H=2149:00 +CPU= 28.173
  08:02:13 STEP 2150 H=2150:00 +CPU= 28.180
  08:02:56 STEP 2151 H=2151:00 +CPU= 40.206
  08:03:24 STEP 2152 H=2152:00 +CPU= 26.786
  08:03:52 STEP 2153 H=2153:00 +CPU= 26.872
  08:04:37 STEP 2154 H=2154:00 +CPU= 41.749
  08:05:05 STEP 2155 H=2155:00 +CPU= 26.744
  08:05:34 STEP 2156 H=2156:00 +CPU= 27.590
  08:06:18 STEP 2157 H=2157:00 +CPU= 41.367
  08:06:48 STEP 2158 H=2158:00 +CPU= 27.764
  08:07:18 STEP 2159 H=2159:00 +CPU= 27.537
  08:08:09 STEP 2160 H=2160:00 +CPU= 48.377
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002160
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002136
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002148
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002148
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_89.zip
Uploading the intermediate file: upload_file_89.zip
  08:08:38 STEP 2161 H=2161:00 +CPU= 26.337
  08:09:07 STEP 2162 H=2162:00 +CPU= 26.856
  08:09:50 STEP 2163 H=2163:00 +CPU= 40.341
  08:10:19 STEP 2164 H=2164:00 +CPU= 27.054
  08:10:47 STEP 2165 H=2165:00 +CPU= 26.995
  08:11:35 STEP 2166 H=2166:00 +CPU= 42.042
  08:12:03 STEP 2167 H=2167:00 +CPU= 26.356
  08:12:31 STEP 2168 H=2168:00 +CPU= 26.790
  08:13:14 STEP 2169 H=2169:00 +CPU= 40.748
  08:13:46 STEP 2170 H=2170:00 +CPU= 28.571
  08:14:14 STEP 2171 H=2171:00 +CPU= 26.771
  08:15:00 STEP 2172 H=2172:00 +CPU= 45.980
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002172
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002172
  08:15:28 STEP 2173 H=2173:00 +CPU= 27.151
  08:15:54 STEP 2174 H=2174:00 +CPU= 24.916
  08:16:38 STEP 2175 H=2175:00 +CPU= 40.972
  08:17:09 STEP 2176 H=2176:00 +CPU= 28.400
  08:17:40 STEP 2177 H=2177:00 +CPU= 28.316
  08:18:23 STEP 2178 H=2178:00 +CPU= 41.789
  08:18:50 STEP 2179 H=2179:00 +CPU= 25.823
  08:19:17 STEP 2180 H=2180:00 +CPU= 26.028
  08:20:01 STEP 2181 H=2181:00 +CPU= 40.791
  08:20:30 STEP 2182 H=2182:00 +CPU= 27.317
  08:20:59 STEP 2183 H=2183:00 +CPU= 27.244
  08:21:51 STEP 2184 H=2184:00 +CPU= 48.230
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002184
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002160
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002172
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002172
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_90.zip
Uploading the intermediate file: upload_file_90.zip
  08:22:19 STEP 2185 H=2185:00 +CPU= 26.652
  08:22:47 STEP 2186 H=2186:00 +CPU= 27.274
  08:23:30 STEP 2187 H=2187:00 +CPU= 40.702
  08:24:01 STEP 2188 H=2188:00 +CPU= 28.793
  08:24:28 STEP 2189 H=2189:00 +CPU= 25.687
  08:25:10 STEP 2190 H=2190:00 +CPU= 39.583
  08:25:39 STEP 2191 H=2191:00 +CPU= 27.970
  08:26:11 STEP 2192 H=2192:00 +CPU= 28.554
  08:26:56 STEP 2193 H=2193:00 +CPU= 40.777
  08:27:24 STEP 2194 H=2194:00 +CPU= 27.284
  08:27:51 STEP 2195 H=2195:00 +CPU= 26.184
  08:28:34 STEP 2196 H=2196:00 +CPU= 42.527
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002196
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002196
  08:29:02 STEP 2197 H=2197:00 +CPU= 27.001
  08:29:33 STEP 2198 H=2198:00 +CPU= 28.129
  08:30:18 STEP 2199 H=2199:00 +CPU= 41.456
  08:30:46 STEP 2200 H=2200:00 +CPU= 27.031
  08:31:18 STEP 2201 H=2201:00 +CPU= 28.371
  08:32:00 STEP 2202 H=2202:00 +CPU= 41.103
  08:32:31 STEP 2203 H=2203:00 +CPU= 28.112
  08:33:04 STEP 2204 H=2204:00 +CPU= 29.473
  08:33:47 STEP 2205 H=2205:00 +CPU= 40.101
  08:34:16 STEP 2206 H=2206:00 +CPU= 27.185
  08:34:45 STEP 2207 H=2207:00 +CPU= 27.113
  08:35:38 STEP 2208 H=2208:00 +CPU= 48.884
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002208
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002184
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002196
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002196
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_91.zip
Uploading the intermediate file: upload_file_91.zip
  08:36:06 STEP 2209 H=2209:00 +CPU= 26.381
  08:36:37 STEP 2210 H=2210:00 +CPU= 28.400
  08:37:21 STEP 2211 H=2211:00 +CPU= 41.133
  08:37:51 STEP 2212 H=2212:00 +CPU= 27.825
  08:38:20 STEP 2213 H=2213:00 +CPU= 27.356
  08:39:04 STEP 2214 H=2214:00 +CPU= 41.814
  08:39:32 STEP 2215 H=2215:00 +CPU= 26.634
  08:40:02 STEP 2216 H=2216:00 +CPU= 28.198
  08:40:46 STEP 2217 H=2217:00 +CPU= 41.069
  08:41:16 STEP 2218 H=2218:00 +CPU= 28.223
  08:41:45 STEP 2219 H=2219:00 +CPU= 27.209
  08:42:35 STEP 2220 H=2220:00 +CPU= 45.994
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002220
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002220
  08:43:06 STEP 2221 H=2221:00 +CPU= 28.397
  08:43:35 STEP 2222 H=2222:00 +CPU= 27.982
  08:44:17 STEP 2223 H=2223:00 +CPU= 40.969
  08:44:48 STEP 2224 H=2224:00 +CPU= 28.344
  08:45:16 STEP 2225 H=2225:00 +CPU= 27.202
  08:46:02 STEP 2226 H=2226:00 +CPU= 44.549
  08:46:32 STEP 2227 H=2227:00 +CPU= 27.402
  08:47:02 STEP 2228 H=2228:00 +CPU= 27.831
  08:47:43 STEP 2229 H=2229:00 +CPU= 40.137
  08:48:10 STEP 2230 H=2230:00 +CPU= 25.680
  08:48:39 STEP 2231 H=2231:00 +CPU= 28.515
  08:49:36 STEP 2232 H=2232:00 +CPU= 50.693
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002232
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002208
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002220
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002220
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_92.zip
Uploading the intermediate file: upload_file_92.zip
  08:50:07 STEP 2233 H=2233:00 +CPU= 28.027
  08:50:38 STEP 2234 H=2234:00 +CPU= 29.361
  08:51:19 STEP 2235 H=2235:00 +CPU= 40.773
  08:51:51 STEP 2236 H=2236:00 +CPU= 29.580
  08:52:24 STEP 2237 H=2237:00 +CPU= 29.118
  08:53:11 STEP 2238 H=2238:00 +CPU= 42.270
  08:53:42 STEP 2239 H=2239:00 +CPU= 28.300
  08:54:13 STEP 2240 H=2240:00 +CPU= 29.007
  08:54:56 STEP 2241 H=2241:00 +CPU= 41.428
  08:55:30 STEP 2242 H=2242:00 +CPU= 30.296
  08:56:03 STEP 2243 H=2243:00 +CPU= 29.338
  08:56:54 STEP 2244 H=2244:00 +CPU= 47.567
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002244
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002244
  08:57:25 STEP 2245 H=2245:00 +CPU= 28.294
  08:57:58 STEP 2246 H=2246:00 +CPU= 29.246
  08:58:46 STEP 2247 H=2247:00 +CPU= 44.405
  08:59:21 STEP 2248 H=2248:00 +CPU= 30.190
  08:59:49 STEP 2249 H=2249:00 +CPU= 27.727
  09:00:38 STEP 2250 H=2250:00 +CPU= 44.919
  09:01:07 STEP 2251 H=2251:00 +CPU= 28.079
  09:01:39 STEP 2252 H=2252:00 +CPU= 28.443
  09:02:22 STEP 2253 H=2253:00 +CPU= 41.189
  09:02:52 STEP 2254 H=2254:00 +CPU= 28.453
  09:03:21 STEP 2255 H=2255:00 +CPU= 28.453
  09:04:19 STEP 2256 H=2256:00 +CPU= 51.450
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002256
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002232
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002244
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002244
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_93.zip
Uploading the intermediate file: upload_file_93.zip
  09:04:49 STEP 2257 H=2257:00 +CPU= 27.530
  09:05:19 STEP 2258 H=2258:00 +CPU= 28.327
  09:06:05 STEP 2259 H=2259:00 +CPU= 44.254
  09:06:37 STEP 2260 H=2260:00 +CPU= 31.063
  09:07:12 STEP 2261 H=2261:00 +CPU= 31.841
  09:08:09 STEP 2262 H=2262:00 +CPU= 51.834
  09:08:47 STEP 2263 H=2263:00 +CPU= 33.750
  09:09:25 STEP 2264 H=2264:00 +CPU= 34.961
  09:10:18 STEP 2265 H=2265:00 +CPU= 49.436
  09:10:55 STEP 2266 H=2266:00 +CPU= 34.465
  09:11:35 STEP 2267 H=2267:00 +CPU= 34.920
  09:12:40 STEP 2268 H=2268:00 +CPU= 56.931
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002268
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002268
  09:13:15 STEP 2269 H=2269:00 +CPU= 32.702
  09:13:53 STEP 2270 H=2270:00 +CPU= 35.144
  09:14:49 STEP 2271 H=2271:00 +CPU= 50.966
  09:15:25 STEP 2272 H=2272:00 +CPU= 33.493
  09:16:02 STEP 2273 H=2273:00 +CPU= 34.295
  09:16:57 STEP 2274 H=2274:00 +CPU= 51.424
  09:17:35 STEP 2275 H=2275:00 +CPU= 33.279
  09:18:12 STEP 2276 H=2276:00 +CPU= 34.166
  09:19:05 STEP 2277 H=2277:00 +CPU= 49.683
  09:19:43 STEP 2278 H=2278:00 +CPU= 33.882
  09:20:23 STEP 2279 H=2279:00 +CPU= 35.143
  09:21:27 STEP 2280 H=2280:00 +CPU= 60.124
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002280
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002256
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002268
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002268
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_94.zip
Uploading the intermediate file: upload_file_94.zip
  09:22:02 STEP 2281 H=2281:00 +CPU= 32.774
  09:22:37 STEP 2282 H=2282:00 +CPU= 33.718
  09:23:32 STEP 2283 H=2283:00 +CPU= 50.731
  09:24:09 STEP 2284 H=2284:00 +CPU= 33.844
  09:24:46 STEP 2285 H=2285:00 +CPU= 33.884
  09:25:40 STEP 2286 H=2286:00 +CPU= 50.317
  09:26:15 STEP 2287 H=2287:00 +CPU= 33.695
  09:26:52 STEP 2288 H=2288:00 +CPU= 33.889
  09:27:44 STEP 2289 H=2289:00 +CPU= 49.265
  09:28:24 STEP 2290 H=2290:00 +CPU= 35.237
  09:28:56 STEP 2291 H=2291:00 +CPU= 31.049
  09:29:55 STEP 2292 H=2292:00 +CPU= 54.180
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002292
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002292
  09:30:33 STEP 2293 H=2293:00 +CPU= 33.602
  09:31:11 STEP 2294 H=2294:00 +CPU= 34.282
  09:32:05 STEP 2295 H=2295:00 +CPU= 49.463
  09:32:41 STEP 2296 H=2296:00 +CPU= 32.690
  09:33:17 STEP 2297 H=2297:00 +CPU= 33.450
  09:34:08 STEP 2298 H=2298:00 +CPU= 49.050
  09:34:42 STEP 2299 H=2299:00 +CPU= 31.870
  09:35:18 STEP 2300 H=2300:00 +CPU= 33.565
  09:36:11 STEP 2301 H=2301:00 +CPU= 49.087
  09:36:48 STEP 2302 H=2302:00 +CPU= 34.277
  09:37:27 STEP 2303 H=2303:00 +CPU= 36.000
  09:38:33 STEP 2304 H=2304:00 +CPU= 61.341
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002304
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002280
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002292
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002292
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_95.zip
Uploading the intermediate file: upload_file_95.zip
  09:39:09 STEP 2305 H=2305:00 +CPU= 33.793
  09:39:46 STEP 2306 H=2306:00 +CPU= 34.446
  09:40:41 STEP 2307 H=2307:00 +CPU= 51.596
  09:41:16 STEP 2308 H=2308:00 +CPU= 32.000
  09:41:50 STEP 2309 H=2309:00 +CPU= 31.815
  09:42:44 STEP 2310 H=2310:00 +CPU= 48.827
  09:43:19 STEP 2311 H=2311:00 +CPU= 33.508
  09:43:53 STEP 2312 H=2312:00 +CPU= 31.758
  09:44:44 STEP 2313 H=2313:00 +CPU= 47.111
  09:45:20 STEP 2314 H=2314:00 +CPU= 33.136
  09:45:54 STEP 2315 H=2315:00 +CPU= 31.891
  09:46:51 STEP 2316 H=2316:00 +CPU= 53.872
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002316
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002316
  09:47:26 STEP 2317 H=2317:00 +CPU= 32.894
  09:48:02 STEP 2318 H=2318:00 +CPU= 33.831
  09:48:57 STEP 2319 H=2319:00 +CPU= 52.559
  09:49:32 STEP 2320 H=2320:00 +CPU= 32.742
  09:50:06 STEP 2321 H=2321:00 +CPU= 32.178
  09:50:58 STEP 2322 H=2322:00 +CPU= 49.517
  09:51:32 STEP 2323 H=2323:00 +CPU= 31.170
  09:52:06 STEP 2324 H=2324:00 +CPU= 32.413
  09:52:59 STEP 2325 H=2325:00 +CPU= 48.391
  09:53:31 STEP 2326 H=2326:00 +CPU= 31.165
  09:54:05 STEP 2327 H=2327:00 +CPU= 31.688
  09:55:07 STEP 2328 H=2328:00 +CPU= 57.464
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002328
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002328
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002304
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMGGhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMSHhq0f+002316
Adding to the zip: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_12200078/ICMUAhq0f+002316
Zipping up the intermediate file: /var/lib/boinc-client/projects/climateprediction.net/oifs_43r3_ps_0434_2014050100_123_983_12200078_0_r492164652_96.zip
Uploading the intermediate file: upload_file_96.zip
Uploading trickle at timestep: 8380800
  09:55:42 STEP 2329 H=2329:00 +CPU= 31.410
  09:56:14 STEP 2330 H=2330:00 +CPU= 30.934
  09:57:03 STEP 2331 H=2331:00 +CPU= 46.635
  09:57:37 STEP 2332 H=2332:00 +CPU= 31.793
  09:58:11 STEP 2333 H=2333:00 +CPU= 30.628
  09:59:01 STEP 2334 H=2334:00 +CPU= 46.233
  09:59:35 STEP 2335 H=2335:00 +CPU= 31.394
  10:00:12 STEP 2336 H=2336:00 +CPU= 31.670
  10:00:59 STEP 2337 H=2337:00 +CPU= 45.062
  10:01:34 STEP 2338 H=2338:00 +CPU= 31.671
  10:02:06 STEP 2339 H=2339:00 +CPU= 29.681
  10:02:59 STEP 2340 H=2340:00 +CPU= 49.454
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMGGhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMSHhq0f+002340
Moving to projects directory: /var/lib/boinc-client/slots/12/ICMUAhq0f+002340
  10:03:35 STEP 2341 H=2341:00 +CPU= 32.124
  10:04:06 STEP 2342 H=2342:00 +CPU= 29.214
  10:04:53 STEP 2343 H=2343:00 +CPU= 45.942
  10:05:30 STEP 2344 H=2344:00 +CPU= 31.884
  10:06:04 STEP 2345 H=2345:00 +CPU= 31.153
  10:07:02 STEP 2346 H=2346:00 +CPU= 50.245
  10:07:38 STEP 2347 H=2347:00 +CPU= 32.456
  10:08:14 STEP 2348 H=2348:00 +CPU= 31.671
  10:09:04 STEP 2349 H=2349:00 +CPU= 46.826
  10:09:38 STEP 2350 H=2350:00 +CPU= 31.368
  10:10:10 STEP 2351 H=2351:00 +CPU= 30.131
Abort request received from BOINC client, ending the child process
..Failed, model did not complete successfully

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
09 May 2023 16:05:28 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 8,380,800 59,877 0.0071
09 May 2023 16:05:28 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 8,380,800 59,877 0.0071
09 May 2023 16:05:23 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 7,516,800 51,830 0.0069
09 May 2023 16:05:18 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 6,134,400 40,701 0.0066
09 May 2023 16:05:18 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 6,134,400 40,701 0.0066
09 May 2023 15:53:38 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 3,801,600 26,756 0.0070
09 May 2023 15:53:08 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 2,937,600 21,076 0.0072
09 May 2023 15:52:55 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 2,073,600 15,250 0.0074
09 May 2023 15:52:43 1520317 22286289 oifs_43r3_ps_0434_2014050100_123_983_12200078_0 1,209,600 8,915 0.0074


©2024 climateprediction.net