Task 16572709

Name	hadam3pm2_e8i4_1991_10_008714063_0
Workunit	8860537
Created	19 Apr 2014, 9:24:06 UTC
Sent	28 Apr 2014, 14:19:51 UTC
Report deadline	10 Apr 2015, 19:39:51 UTC
Received	7 May 2014, 22:09:36 UTC
Server state	Over
Outcome	Computation error
Client state	Compute error
Exit status	22 (0x00000016) Unknown error code
Computer ID	1325999
Run time	9 days 2 hours 43 min 27 sec
CPU time	2 days 1 hours 4 min 14 sec
Validate state	Invalid
Credit	8,900.08
Device peak FLOPS	3.01 GFLOPS
Application version	UK Met Office HadAM3P (global only) with MOSES II landsurface scheme v7.03 i686-pc-linux-gnu
Stderr	<core_client_version>7.2.33</core_client_version> <![CDATA[ <message> process exited with code 22 (0x16, -234) </message> <stderr_txt> CPDN Monitor - Quit request from BOINC... CPDN Monitor - Quit request from BOINC... Suspended CPDN Monitor - Suspend request from BOINC... Suspended CPDN Monitor - Quit request from BOINC... oa.pc\|3dec.nc oa.pc\|4jan.nc oa.pc\|4feb.nc oa.pc\|4mar.nc oa.pc\|4apr.nc oa.pc\|4may.nc oa.pc\|4jun.nc oa.pc\|4jul.nc oa.pc\|4aug.nc oa.pc\|4sep.nc oa.pc\|4oct.nc oa.pc\|4nov.nc oa.pe\|3dec.nc oa.pe\|4jan.nc oa.pe\|4feb.nc oa.pe\|4mar.nc oa.pe\|4apr.nc oa.pe\|4may.nc oa.pe\|4jun.nc oa.pe\|4jul.nc oa.pe\|4aug.nc oa.pe\|4sep.nc oa.pe\|4oct.nc oa.pe\|4nov.nc SIGSEGV: segmentation violation Stack trace (13 frames): /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2127, iMonCtr=1 Model crash detected, will try to restart... SIGSEGV: segmentation violation Stack trace (13 frames): /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu(boinc_catch_signal+0x6f)[0x83bccdf] linux-gate.so.1(__kernel_sigreturn+0x0)[0x55578400] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8165b39] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816b8f8] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8162a76] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x816d014] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8161751] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8085677] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x8094c0a] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x831e514] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x83205f9] /data1/pvh/BOINC/projects/climateprediction.net/hadam3pm2_um_7.03_i686-pc-linux-gnu[0x834c82d] /lib/libc.so.6(__libc_start_main+0xf3)[0x555fc9d3] Exiting... Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=2127, iMonCtr=1 Model crash detected, will try to restart... oa.pc\|4dec.nc oa.pc\|5jan.nc oa.pc\|5feb.nc oa.pc\|5mar.nc oa.pc\|5apr.nc oa.pc\|5may.nc oa.pc\|5jun.nc oa.pc\|5jul.nc oa.pc\|5aug.nc oa.pc\|5sep.nc oa.pc\|5oct.nc oa.pc\|5nov.nc oa.pe\|4dec.nc oa.pe\|5jan.nc oa.pe\|5feb.nc oa.pe\|5mar.nc oa.pe\|5apr.nc oa.pe\|5may.nc oa.pe\|5jun.nc oa.pe\|5jul.nc oa.pe\|5aug.nc oa.pe\|5sep.nc oa.pe\|5oct.nc oa.pe\|5nov.nc Suspended CPDN Monitor - Suspend request from BOINC... oa.pc\|5dec.nc oa.pc\|6jan.nc oa.pc\|6feb.nc oa.pc\|6mar.nc oa.pc\|6apr.nc oa.pc\|6may.nc oa.pc\|6jun.nc oa.pc\|6jul.nc oa.pc\|6aug.nc oa.pc\|6sep.nc oa.pc\|6oct.nc oa.pc\|6nov.nc oa.pe\|5dec.nc oa.pe\|6jan.nc oa.pe\|6feb.nc oa.pe\|6mar.nc oa.pe\|6apr.nc oa.pe\|6may.nc oa.pe\|6jun.nc oa.pe\|6jul.nc oa.pe\|6aug.nc oa.pe\|6sep.nc oa.pe\|6oct.nc oa.pe\|6nov.nc oa.pc\|6dec.nc oa.pc\|7jan.nc oa.pc\|7feb.nc oa.pc\|7mar.nc oa.pc\|7apr.nc oa.pc\|7may.nc oa.pc\|7jun.nc oa.pc\|7jul.nc oa.pc\|7aug.nc oa.pc\|7sep.nc oa.pc\|7oct.nc oa.pc\|7nov.nc oa.pe\|6dec.nc oa.pe\|7jan.nc oa.pe\|7feb.nc oa.pe\|7mar.nc oa.pe\|7apr.nc oa.pe\|7may.nc oa.pe\|7jun.nc oa.pe\|7jul.nc oa.pe\|7aug.nc oa.pe\|7sep.nc oa.pe\|7oct.nc oa.pe\|7nov.nc oa.pc\|7dec.nc oa.pc\|8jan.nc oa.pc\|8feb.nc oa.pc\|8mar.nc oa.pc\|8apr.nc oa.pc\|8may.nc oa.pc\|8jun.nc oa.pc\|8jul.nc oa.pc\|8aug.nc oa.pc\|8sep.nc oa.pc\|8oct.nc oa.pc\|8nov.nc oa.pe\|7dec.nc oa.pe\|8jan.nc oa.pe\|8feb.nc oa.pe\|8mar.nc oa.pe\|8apr.nc oa.pe\|8may.nc oa.pe\|8jun.nc oa.pe\|8jul.nc oa.pe\|8aug.nc oa.pe\|8sep.nc oa.pe\|8oct.nc oa.pe\|8nov.nc oa.pc\|8dec.nc oa.pc\|9jan.nc oa.pc\|9feb.nc oa.pc\|9mar.nc oa.pc\|9apr.nc oa.pc\|9may.nc oa.pc\|9jun.nc oa.pc\|9jul.nc oa.pc\|9aug.nc oa.pc\|9sep.nc oa.pc\|9oct.nc oa.pc\|9nov.nc oa.pe\|8dec.nc oa.pe\|9jan.nc oa.pe\|9feb.nc oa.pe\|9mar.nc oa.pe\|9apr.nc oa.pe\|9may.nc oa.pe\|9jun.nc oa.pe\|9jul.nc oa.pe\|9aug.nc oa.pe\|9sep.nc oa.pe\|9oct.nc oa.pe\|9nov.nc oa.pc\|9dec.nc oa.pc\|0jan.nc oa.pc\|0feb.nc oa.pc\|0mar.nc oa.pc\|0apr.nc oa.pc\|0may.nc oa.pc\|0jun.nc oa.pc\|0jul.nc oa.pc\|0aug.nc oa.pc\|0sep.nc oa.pc\|0oct.nc oa.pc\|0nov.nc oa.pe\|9dec.nc oa.pe\|0jan.nc oa.pe\|0feb.nc oa.pe\|0mar.nc oa.pe\|0apr.nc oa.pe\|0may.nc oa.pe\|0jun.nc oa.pe\|0jul.nc oa.pe\|0aug.nc oa.pe\|0sep.nc oa.pe\|0oct.nc oa.pe\|0nov.nc Model crashed: æM Model crashed: æM Model crashed: æM Model crashed: æM Model crashed: æM Model crashed: æM Sorry, too many model crashes! :-( 00:08:13 (2127): called boinc_finish </stderr_txt> ]]>

Latest Trickles Received
Time Sent (UTC)	Host ID	Result ID	Result Name	Timestep	CPU Time (sec)	Average (sec/TS)
07 May 2014 18:18:27	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	308,228	585,362	1.8991
07 May 2014 16:17:19	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	305,348	578,822	1.8956
07 May 2014 14:25:14	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	302,468	572,290	1.8921
07 May 2014 12:34:40	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	299,588	565,782	1.8885
07 May 2014 11:04:15	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	296,708	559,288	1.8850
07 May 2014 09:03:48	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	293,828	552,759	1.8812
07 May 2014 07:01:00	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	290,948	546,209	1.8773
07 May 2014 05:05:34	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	288,068	539,664	1.8734
07 May 2014 03:15:12	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	285,188	533,118	1.8694
07 May 2014 01:24:46	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	282,308	526,607	1.8654
07 May 2014 00:19:31	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	279,428	520,119	1.8614
06 May 2014 20:36:07	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	273,668	507,074	1.8529
06 May 2014 18:30:32	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	270,788	500,449	1.8481
06 May 2014 16:19:10	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	267,908	493,739	1.8429
06 May 2014 14:26:45	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	265,028	487,025	1.8376
06 May 2014 13:14:37	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	262,148	481,007	1.8349
06 May 2014 11:14:01	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	259,268	476,206	1.8367
06 May 2014 10:13:48	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	256,388	471,415	1.8387
06 May 2014 09:13:35	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	253,508	466,637	1.8407
06 May 2014 07:11:11	1325999	16572709	hadam3pm2_e8i4_1991_10_008714063_0	250,628	461,863	1.8428