climateprediction.net home page
Task 12928826

Task 12928826

Name hadcm3n_o53w_1940_40_007267606_1
Workunit 7465846
Created 3 Jun 2011, 4:28:19 UTC
Sent 3 Jun 2011, 4:28:25 UTC
Report deadline 2 Sep 2011, 11:55:36 UTC
Received 5 Jul 2011, 2:53:23 UTC
Server state Over
Outcome Computation error
Client state Compute error
Exit status 22 (0x00000016) Unknown error code
Computer ID 864106
Run time 19 days 21 hours 44 min 6 sec
CPU time 15 days 2 hours 36 min 19 sec
Validate state Invalid
Credit 11,197.44
Device peak FLOPS 3.47 GFLOPS
Application version UK Met Office Coupled Model Full Resolution Ocean v6.07
i686-apple-darwin
Stderr 
<core_client_version>6.10.17</core_client_version>
<![CDATA[
<message>
process exited with code 22 (0x16, -234)
</message>
<stderr_txt>
CPDN Monitor - Quit request from BOINC...
22:51:30 (91724): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
19:15:24 (10840): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
10:20:56 (6698): No heartbeat from core client for 30 sec - exiting
CPDN Monitor - No 'heartbeat' from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
CPDN Monitor - Quit request from BOINC...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Sun Jul  3 20:11:37 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x92c29c5a ___kill + 10
1   libSystem.B.dylib                   0x92cbc5a5 _raise + 26
2   libSystem.B.dylib                   0x92cd26e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x92bc8c0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x92c50429 _nanosleep + 345
2   libSystem.B.dylib                   0x92c502ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x92bf6259 __pthread_start + 345
6   libSystem.B.dylib                   0x92bf60de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc968 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x92bc8000 - 0x92d6ffff /usr/lib/libSystem.B.dylib
0x965cd000 - 0x965dbfff /usr/lib/libz.1.dylib
0x9992f000 - 0x99932fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30563, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Sun Jul  3 20:11:41 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x92c29c5a ___kill + 10
1   libSystem.B.dylib                   0x92cbc5a5 _raise + 26
2   libSystem.B.dylib                   0x92cd26e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x92bc8c0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x92c50429 _nanosleep + 345
2   libSystem.B.dylib                   0x92c502ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x92bf6259 __pthread_start + 345
6   libSystem.B.dylib                   0x92bf60de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc968 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x92bc8000 - 0x92d6ffff /usr/lib/libSystem.B.dylib
0x965cd000 - 0x965dbfff /usr/lib/libz.1.dylib
0x9992f000 - 0x99932fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30563, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Sun Jul  3 20:11:45 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x92c29c5a ___kill + 10
1   libSystem.B.dylib                   0x92cbc5a5 _raise + 26
2   libSystem.B.dylib                   0x92cd26e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x92bc8c0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x92c50429 _nanosleep + 345
2   libSystem.B.dylib                   0x92c502ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x92bf6259 __pthread_start + 345
6   libSystem.B.dylib                   0x92bf60de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc968 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x92bc8000 - 0x92d6ffff /usr/lib/libSystem.B.dylib
0x965cd000 - 0x965dbfff /usr/lib/libz.1.dylib
0x9992f000 - 0x99932fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30563, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Sun Jul  3 20:11:49 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x92c29c5a ___kill + 10
1   libSystem.B.dylib                   0x92cbc5a5 _raise + 26
2   libSystem.B.dylib                   0x92cd26e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x92bc8c0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x92c50429 _nanosleep + 345
2   libSystem.B.dylib                   0x92c502ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x92bf6259 __pthread_start + 345
6   libSystem.B.dylib                   0x92bf60de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc968 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x92bc8000 - 0x92d6ffff /usr/lib/libSystem.B.dylib
0x965cd000 - 0x965dbfff /usr/lib/libz.1.dylib
0x9992f000 - 0x99932fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30563, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Sun Jul  3 20:11:52 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x92c29c5a ___kill + 10
1   libSystem.B.dylib                   0x92cbc5a5 _raise + 26
2   libSystem.B.dylib                   0x92cd26e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x92bc8c0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x92c50429 _nanosleep + 345
2   libSystem.B.dylib                   0x92c502ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x92bf6259 __pthread_start + 345
6   libSystem.B.dylib                   0x92bf60de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc968 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x92bc8000 - 0x92d6ffff /usr/lib/libSystem.B.dylib
0x965cd000 - 0x965dbfff /usr/lib/libz.1.dylib
0x9992f000 - 0x99932fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=30563, iMonCtr=1
Model crash detected, will try to restart...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Sun Jul  3 20:11:55 2011

Thread 0 Crashed:
CPDN Monitor - Quit request from BOINC...
SIGABRT: abort called

Crashed executable name: hadcm3n_um_6.07_i686-apple-darwin
built using BOINC library version 6.13.0
Machine type Intel 80486 (32-bit executable)
System version: Macintosh OS 10.6.8 build 10K540
Mon Jul  4 21:36:08 2011

Thread 0 Crashed:
0   libSystem.B.dylib                   0x92c29c5a ___kill + 10
1   libSystem.B.dylib                   0x92cbc5a5 _raise + 26
2   libSystem.B.dylib                   0x92cd26e4 _abort + 93
3   hadcm3n_um_6.07_i686-apple-darwin   0x00336cff _um_setup + 127
4   hadcm3n_um_6.07_i686-apple-darwin   0x00389b71 _um_shell + 3953
5   hadcm3n_um_6.07_i686-apple-darwin   0x000048e1 _main + 1153
6   hadcm3n_um_6.07_i686-apple-darwin   0x000027fb __start + 209
7   hadcm3n_um_6.07_i686-apple-darwin   0x00002729 start + 41

Thread 1:
0   libSystem.B.dylib                   0x92bc8c0e _mach_wait_until + 10
1   libSystem.B.dylib                   0x92c50429 _nanosleep + 345
2   libSystem.B.dylib                   0x92c502ca _usleep + 61
3   hadcm3n_um_6.07_i686-apple-darwin   0x00442ec0 __Z11boinc_sleepd + 188
4   hadcm3n_um_6.07_i686-apple-darwin   0x0043865e __Z12timer_threadPv + 78
5   libSystem.B.dylib                   0x92bf6259 __pthread_start + 345
6   libSystem.B.dylib                   0x92bf60de _thread_start + 34

Thread 0 crashed with X86 Thread State (32-bit):
  eax: 0x00000000 ebx: 0x00000000 ecx: 0x00000000 edx: 0x00000000
  edi: 0x00000000 esi: 0x00000000 ebp: 0xbfffc968 esp: 0x00000000
   ss: 0x00000000 efl: 0x00000000 eip: 0x00447dc2  cs: 0x00000000
   ds: 0x00000000  es: 0x00000000  fs: 0x00000000  gs: 0x00000000

Binary Images Description:
    0x1000 -   0x4cbfff /Library/Application Support/BOINC Data/projects/climateprediction.net/hadcm3n_um_6.07_i686-apple-darwin
0x92bc8000 - 0x92d6ffff /usr/lib/libSystem.B.dylib
0x965cd000 - 0x965dbfff /usr/lib/libz.1.dylib
0x9992f000 - 0x99932fff /usr/lib/system/libmathCommon.A.dylib


Exiting...
Controller:: CPDN process is not running, exiting, bRetVal = 1, checkPID=0, selfPID=46288, iMonCtr=1
Model crash detected, will try to restart...
Sorry, too many model crashes! :-(
Called boinc_finish

</stderr_txt>
]]>
Latest Trickles Received
Time Sent (UTC) Host ID Result ID Result Name Timestep CPU Time (sec) Average (sec/TS)
03 Jul 2011 23:26:49 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 933,120 1,302,654 1.3960
03 Jul 2011 07:08:43 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 907,200 1,267,017 1.3966
02 Jul 2011 14:09:04 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 881,280 1,230,024 1.3957
01 Jul 2011 15:52:55 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 855,360 1,193,515 1.3953
01 Jul 2011 01:35:39 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 829,440 1,157,252 1.3952
30 Jun 2011 09:45:07 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 803,520 1,121,049 1.3952
29 Jun 2011 20:24:36 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 777,600 1,084,964 1.3953
29 Jun 2011 06:57:21 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 751,680 1,048,827 1.3953
28 Jun 2011 17:51:20 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 725,760 1,012,671 1.3953
28 Jun 2011 04:31:36 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 699,840 975,941 1.3945
27 Jun 2011 15:41:08 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 673,920 939,781 1.3945
27 Jun 2011 04:32:37 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 648,000 906,285 1.3986
26 Jun 2011 16:07:00 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 622,080 871,215 1.4005
26 Jun 2011 02:29:17 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 596,160 835,073 1.4008
25 Jun 2011 08:47:54 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 570,240 798,745 1.4007
24 Jun 2011 17:02:59 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 544,320 762,326 1.4005
24 Jun 2011 03:53:56 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 518,400 726,429 1.4013
23 Jun 2011 14:59:24 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 492,480 690,451 1.4020
23 Jun 2011 01:44:33 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 466,560 654,280 1.4023
22 Jun 2011 11:02:54 864106 12928826 hadcm3n_o53w_1940_40_007267606_1 440,640 617,974 1.4024


©2024 climateprediction.net