Task 16496904

Name	hadcm3n_7fl7_1980_40_008431486_3
Workunit	8582342
Created	15 Apr 2014, 11:35:42 UTC
Sent	15 Apr 2014, 12:20:19 UTC
Report deadline	17 Oct 2023, 17:40:19 UTC
Received	11 May 2014, 11:55:13 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1308817
Run time	14 days 11 hours 3 min 53 sec
CPU time	14 days 7 hours 15 min 36 sec
Validate state	Invalid
Credit	9,020.16
Device peak FLOPS	2.94 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.45</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory 22:00:42 (60806): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory 00:45:20 (61644): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory 00:47:12 (62138): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory 00:47:54 (62177): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory 00:50:31 (62218): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory 00:51:13 (62260): No heartbeat from core client for 30 sec - exiting CPDN Monitor - No 'heartbeat' from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory CPDN Monitor - Quit request from BOINC... Unable to load library hadcm3n_se_6.07_i686-pc-linux-gnu.so dlopen error: libz.so.1: cannot open shared object file: No such file or directory BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 BUFFOUT: Write Failed: No space left on device BUFFOUT: C I/O Error - Return code = 1 Model crashed: WRITDUMP: BAD BUFFOUT OF DATA tmp/pipe_dummy 2048 forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadcm3n_7fl7_1980_40_008431486/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 084554BF Unknown Unknown Unknown hadcm3n_um_6.07_i 08281F64 Unknown Unknown Unknown hadcm3n_um_6.07_i 083350C5 Unknown Unknown Unknown hadcm3n_um_6.07_i 0821D177 Unknown Unknown Unknown hadcm3n_um_6.07_i 08219DA1 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838720B Unknown Unknown Unknown hadcm3n_um_6.07_i 083995C6 Unknown Unknown Unknown hadcm3n_um_6.07_i 0838F8B7 Unknown Unknown Unknown hadcm3n_um_6.07_i 0839BDF8 Unknown Unknown Unknown libc.so.6 001E3D26 Unknown Unknown Unknown hadcm3n_um_6.07_i 0804CB11 Unknown Unknown Unknown Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=26596, iMonCtr=1 Model crash detected, will try to restart... forrtl: No space left on device forrtl: severe (38): error during write, unit 6, file /var/lib/boinc/projects/climateprediction.net/hadcm3n_7fl7_1980_40_008431486/dataout/stdout_um.txt Image PC Routine Line Source hadcm3n_um_6.07_i 0848EB7D Unknown Unknown Unknown hadcm3n_um_6.07_i 0848D975 Unknown Unknown Unknown hadcm3n_um_6.07_i 0845F3CF Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F90D Unknown Unknown Unknown hadcm3n_um_6.07_i 0841F257 Unknown Unknown Unknown hadcm3n_um_6.07_i 084554BF Unknown Unknown Unknown hadcm3n_um_6.07_i 0832327C Unknown Unknown Unknown hadcm3n_um_6.07_i 08198894 Unknown Unknown Unknown haCalled boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
11 May 2014 06:36:01	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	751,680	1,226,815	1.6321
10 May 2014 20:53:34	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	725,760	1,189,772	1.6393
10 May 2014 10:25:26	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	699,840	1,152,113	1.6463
09 May 2014 14:26:03	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	673,920	1,113,897	1.6529
09 May 2014 02:04:26	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	648,000	1,068,286	1.6486
08 May 2014 04:05:47	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	622,080	1,021,917	1.6427
07 May 2014 06:20:51	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	596,160	978,215	1.6409
06 May 2014 08:28:24	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	570,240	934,910	1.6395
05 May 2014 10:34:08	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	544,320	892,088	1.6389
04 May 2014 23:21:54	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	518,400	849,096	1.6379
04 May 2014 11:06:14	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	492,480	807,061	1.6388
03 May 2014 23:23:52	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	466,560	763,499	1.6364
03 May 2014 10:37:44	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	440,640	719,607	1.6331
02 May 2014 15:14:55	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	414,720	675,629	1.6291
02 May 2014 02:07:23	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	388,800	632,055	1.6257
01 May 2014 04:06:33	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	362,880	587,904	1.6201
30 Apr 2014 07:27:02	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	336,960	545,278	1.6182
29 Apr 2014 10:31:31	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	311,040	502,968	1.6171
28 Apr 2014 13:43:03	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	285,120	460,417	1.6148
28 Apr 2014 02:17:44	1308817	16496904	hadcm3n_7fl7_1980_40_008431486_3	259,200	418,272	1.6137