Task 14870069

Name	hadcm3n_yk5f_1980_40_008000047_4
Workunit	8155161
Created	6 Jul 2012, 4:40:26 UTC
Sent	6 Jul 2012, 4:40:30 UTC
Report deadline	5 Oct 2012, 12:07:41 UTC
Received	17 Jul 2012, 0:31:55 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1193970
Run time	10 days 18 hours 45 min 28 sec
CPU time	10 days 16 hours 37 min 30 sec
Validate state	Invalid
Credit	6,842.88
Device peak FLOPS	2.75 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>6.10.58</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Signal 15 received, exiting... Called boinc_finish SIGSEGV: segmentation violation Stack trace (7 frames): ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x80b80df] [0xffffe400] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805067e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805112c] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu[0x805137a] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7449c2e] ../../projects/climateprediction.net/hadcm3n_6.07_i686-pc-linux-gnu(__gxx_personality_v0+0x169)[0x804cb51] Exiting... SIGSEGV: segmentation violation Stack trace (13 frames): /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xfe)[0xf75e5c2e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4333, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xfe)[0xf75bcc2e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4333, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7613c2e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4333, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xfe)[0xf7576c2e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4333, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xfe)[0xf75f5c2e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4333, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] [0xffffe400] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8155092] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8159b01] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81518bf] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x815a74a] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8153291] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807f0e3] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /home/billn/Download/BOINC/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /lib/libc.so.6(__libc_start_main+0xfe)[0xf75efc2e] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=4333, iMonCtr=1 Model crash detected, will try to restart... Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
16 Jul 2012 13:52:26	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	570,240	888,544	1.5582
16 Jul 2012 02:45:02	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	544,320	848,293	1.5584
15 Jul 2012 14:38:22	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	518,400	807,287	1.5573
15 Jul 2012 03:31:20	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	492,480	766,413	1.5562
14 Jul 2012 16:37:07	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	466,560	725,308	1.5546
14 Jul 2012 04:29:11	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	440,640	684,273	1.5529
13 Jul 2012 17:08:58	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	414,720	643,257	1.5511
13 Jul 2012 06:04:00	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	388,800	602,237	1.5490
12 Jul 2012 17:52:59	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	362,880	561,221	1.5466
12 Jul 2012 06:46:28	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	336,960	520,331	1.5442
11 Jul 2012 19:37:27	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	311,040	480,275	1.5441
11 Jul 2012 08:33:37	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	285,120	440,215	1.5440
10 Jul 2012 21:25:55	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	259,200	400,129	1.5437
10 Jul 2012 10:15:46	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	233,280	360,069	1.5435
09 Jul 2012 23:08:24	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	207,360	320,144	1.5439
09 Jul 2012 12:00:29	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	181,440	280,124	1.5439
09 Jul 2012 00:53:25	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	155,520	240,149	1.5442
08 Jul 2012 12:45:44	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	129,600	200,151	1.5444
08 Jul 2012 01:33:42	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	103,680	160,182	1.5450
07 Jul 2012 14:41:18	1193970	14870069	hadcm3n_yk5f_1980_40_008000047_4	77,760	120,181	1.5455