Task 16098938

Name	hadcm3n_49z8_1980_40_008367353_3
Workunit	8518212
Created	28 Nov 2013, 1:00:07 UTC
Sent	28 Nov 2013, 1:00:11 UTC
Report deadline	27 Feb 2014, 8:27:22 UTC
Received	7 Dec 2013, 16:02:18 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1301382
Run time	8 days 19 hours 53 min 39 sec
CPU time	6 days 13 hours 35 min 1 sec
Validate state	Invalid
Credit	8,709.12
Device peak FLOPS	2.15 GFLOPS
Application version	UK Met Office Coupled Model Full Resolution Ocean v6.07 i686-pc-linux-gnu
Stderr	<core_client_version>7.0.65</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1198, iMonCtr=1 Model crash detected, will try to restart... Model crashed: ATM_DYN : INVALID THETA DETECTED. tmp/pipe_dummy 2048 Suspended CPDN Monitor - Suspend request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (9 frames): /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77a1400] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8325acf] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8071809] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837738c] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /usr/lib32/libc.so.6(__libc_start_main+0xf3)[0xf759d7c3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=1080, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SIGSEGV: segmentation violation Stack trace (15 frames): /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x840da8f] linux-gate.so.1(__kernel_sigreturn+0x0)[0xf77c9400] /lib/ld-linux.so.2(+0xe1a9)[0xf77d81a9] /lib/ld-linux.so.2(+0x14560)[0xf77de560] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x83856e0] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x82a92a9] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8140cfb] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x81dadc8] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x8267a6b] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x807b49e] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x837e9f4] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839982e] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x838f8b7] /boinc/main/projects/climateprediction.net/hadcm3n_um_6.07_i686-pc-linux-gnu[0x839bdf8] /usr/lib32/libc.so.6(__libc_start_main+0xf3)[0xf75c57c3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2572, iMonCtr=1 Model crash detected, will try to restart... CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... SETPOS: Seek Failed: Invalid argument SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 SETPOS: Seek Failed: Invalid argument SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 SETPOS: Seek Failed: Invalid argument SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 SETPOS: Seek Failed: Invalid argument SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 SETPOS: Seek Failed: Invalid argument SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Signal 15 received, exiting... Called boinc_finish SETPOS: Seek Failed: Invalid argument SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Model crashed: SETPOS: Unit 63 to Word Address -198 Failed with Error Code -1 Sorry, too many model crashes! :-( Called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 Dec 2013 07:35:00	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	725,760	551,996	0.7606
07 Dec 2013 00:12:41	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	699,840	528,515	0.7552
06 Dec 2013 16:21:06	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	673,920	505,327	0.7498
06 Dec 2013 09:02:50	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	648,000	483,335	0.7459
06 Dec 2013 03:00:21	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	622,080	461,530	0.7419
05 Dec 2013 21:27:33	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	596,160	440,126	0.7383
05 Dec 2013 16:11:13	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	570,240	418,311	0.7336
05 Dec 2013 07:22:05	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	544,320	391,246	0.7188
04 Dec 2013 23:39:48	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	518,400	363,416	0.7010
04 Dec 2013 16:18:45	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	492,480	335,606	0.6815
04 Dec 2013 08:09:55	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	466,560	307,823	0.6598
04 Dec 2013 00:27:29	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	440,640	470,529	1.0678
03 Dec 2013 16:44:09	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	414,720	442,753	1.0676
03 Dec 2013 08:58:26	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	388,800	414,979	1.0673
03 Dec 2013 01:17:10	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	362,880	387,206	1.0670
02 Dec 2013 17:34:31	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	336,960	359,413	1.0666
02 Dec 2013 09:49:12	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	311,040	331,603	1.0661
02 Dec 2013 02:17:17	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	285,120	304,693	1.0686
01 Dec 2013 19:00:28	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	259,200	280,343	1.0816
01 Dec 2013 03:57:35	1301382	16098938	hadcm3n_49z8_1980_40_008367353_3	233,280	253,809	1.0880