I've got a strange problem, and have seem similar posts here in the
forum, but not quite the same...so I thought I'd post to see what y'all
think.

Last Friday, I had to down one of the nodes in the cluster (2 node
cluster). The 2nd node (I'll call server2) picked up the master ip, as
was expected. When I brought the 1st node (server1) back up, it was no
longer able to join the cluster. It simply hangs on "Joining....".
When I look at the log file, it says "Join retry, some other node
acquired the cluster lock".

In trying to troubleshoot this, I brought down server2, and rebooted
both servers. I let the server1 acquire the Master, and server2 joined
without any issues. I then had server1 leave the cluster, and server2
then became the master. When trying to re-join the cluster with
server1, it would again hang. So basically, if server1 is the Master,
both servers can join. But if server2 is the Master, server1 is unable
to join.

I have verified that there are no communication problems between the
two servers. And that they can both see the sbd partition. My
heartbeat settings are on the installation defaults. I have downed both
servers completely, and it always comes back to the same thing (above
paragraph).
I have also tried reinstalling NCS on both servers (joining an existing
node), and it always fails on the "Join Node" section.

I've seen references to some TID's that are supposed to have some
solutions, but am unable to find the TID's. I've also seen reference to
re-creating the SBD partition, but can't find any good instruction on
how to do that without hosing everything. I've read references on
changing the panning ID, but also can't find instructions on how to do
that.

Can someone please give me some advice. If re-creating the sbd
partition is what I should try next, can you explain how to do it?
Thank you!


--
jessbryant
------------------------------------------------------------------------
jessbryant's Profile: http://forums.novell.com/member.php?userid=13263
View this thread: http://forums.novell.com/showthread.php?t=347994

You delete the SBD partition as you would any other partition (use
NSSMU) and then issue the command SBD INSTALL. TID 10082213 has the
steps, just ignore the Mirrorred bit assuming your SBD is not
mirrorred.

If that doesn't pan out (boomtish*) TID 3075104 tells you about the
panning ID.


--
Andrew C Taubman
Novell Support Forums Volunteer SysOp
http://forums.novell.com/
(Sorry, support is not provided via e-mail)

Opinions expressed above are not
necessarily those of Novell Inc.
------------------------------------------------------------------------
ataubman's Profile: http://forums.novell.com/member.php?userid=34
View this thread: http://forums.novell.com/showthread.php?t=347994

Hello,

maybe that the "panning clusterid" got out of sync. I had that Problem too
and I know tehre was a TID about it, but I cant find it anymore :-(

Klaus



"jessbryant" <jessbryant@no-mx.forums.novell.com> wrote in message
news:jessbryant.3hlhao@no-mx.forums.novell.com...
>
> I've got a strange problem, and have seem similar posts here in the
> forum, but not quite the same...so I thought I'd post to see what y'all
> think.
>
> Last Friday, I had to down one of the nodes in the cluster (2 node
> cluster). The 2nd node (I'll call server2) picked up the master ip, as
> was expected. When I brought the 1st node (server1) back up, it was no
> longer able to join the cluster. It simply hangs on "Joining....".
> When I look at the log file, it says "Join retry, some other node
> acquired the cluster lock".
>
> In trying to troubleshoot this, I brought down server2, and rebooted
> both servers. I let the server1 acquire the Master, and server2 joined
> without any issues. I then had server1 leave the cluster, and server2
> then became the master. When trying to re-join the cluster with
> server1, it would again hang. So basically, if server1 is the Master,
> both servers can join. But if server2 is the Master, server1 is unable
> to join.
>
> I have verified that there are no communication problems between the
> two servers. And that they can both see the sbd partition. My
> heartbeat settings are on the installation defaults. I have downed both
> servers completely, and it always comes back to the same thing (above
> paragraph).
> I have also tried reinstalling NCS on both servers (joining an existing
> node), and it always fails on the "Join Node" section.
>
> I've seen references to some TID's that are supposed to have some
> solutions, but am unable to find the TID's. I've also seen reference to
> re-creating the SBD partition, but can't find any good instruction on
> how to do that without hosing everything. I've read references on
> changing the panning ID, but also can't find instructions on how to do
> that.
>
> Can someone please give me some advice. If re-creating the sbd
> partition is what I should try next, can you explain how to do it?
> Thank you!
>
>
> --
> jessbryant
> ------------------------------------------------------------------------
> jessbryant's Profile: http://forums.novell.com/member.php?userid=13263
> View this thread: http://forums.novell.com/showthread.php?t=347994
>

Well, we've deleted and recreated the sbd partition. That didn't fix
the problem. We have narrowed it down to this:

If we start up the "broken" server, and then turn the firewall off from
a command line, the node immediately joins the cluster.

We've compared the firewall settings with the "good" server, and they
are identical. So I'm not sure what else to check. It doesn't make
sense that the firewall would prevent it from joining the cluster as a
slave. The firewall doesn't prevent it from joining the cluster as long
as it thinks it's the "master". Baffling.

Any ideas to add to this, given my latest findings?

Thanks!!!


--
jessbryant
------------------------------------------------------------------------
jessbryant's Profile: http://forums.novell.com/member.php?userid=13263
View this thread: http://forums.novell.com/showthread.php?t=347994

Just want to add a little bit more information about this strange
behavior:

If the firewall is disabled completely, and then the server is started
up, the node will still hang upon "joining". The firewall has to be
turned off manually from a command line after the server is up and
running. Once this has happened, the node joins the cluster.

I then tried to start the firewall, and as soon as I did that, the
server rebooted itself.

What's up with that?


--
jessbryant
------------------------------------------------------------------------
jessbryant's Profile: http://forums.novell.com/member.php?userid=13263
View this thread: http://forums.novell.com/showthread.php?t=347994

If it seems to deal with the Firewall I would try a packet scan on both
Machines. You can use PKTSCAN.NLM to make a wireshark compatible Trace and
view it with Wireshark. Comare the situations.

Klaus

"jessbryant" <jessbryant@no-mx.forums.novell.com> wrote in message
news:jessbryant.3hn7c0@no-mx.forums.novell.com...
>
> Well, we've deleted and recreated the sbd partition. That didn't fix
> the problem. We have narrowed it down to this:
>
> If we start up the "broken" server, and then turn the firewall off from
> a command line, the node immediately joins the cluster.
>
> We've compared the firewall settings with the "good" server, and they
> are identical. So I'm not sure what else to check. It doesn't make
> sense that the firewall would prevent it from joining the cluster as a
> slave. The firewall doesn't prevent it from joining the cluster as long
> as it thinks it's the "master". Baffling.
>
> Any ideas to add to this, given my latest findings?
>
> Thanks!!!
>
>
> --
> jessbryant
> ------------------------------------------------------------------------
> jessbryant's Profile: http://forums.novell.com/member.php?userid=13263
> View this thread: http://forums.novell.com/showthread.php?t=347994
>

you could use tcpdump ..
On master node :
tcpdump -v v proto 224 and src <master ip>

on other nodes:
tcpdump -v v proto 224 and dst <master ip>


Klaus Arpe;1665635 Wrote:
> If it seems to deal with the Firewall I would try a packet scan on both
> Machines. You can use PKTSCAN.NLM to make a wireshark compatible Trace
> and
> view it with Wireshark. Comare the situations.
>
> Klaus
>
> "jessbryant" <jessbryant@no-mx.forums.novell.com> wrote in message
> news:jessbryant.3hn7c0@no-mx.forums.novell.com...
> >
> > Well, we've deleted and recreated the sbd partition. That didn't
> fix
> > the problem. We have narrowed it down to this:
> >
> > If we start up the "broken" server, and then turn the firewall off
> from
> > a command line, the node immediately joins the cluster.
> >
> > We've compared the firewall settings with the "good" server, and
> they
> > are identical. So I'm not sure what else to check. It doesn't make
> > sense that the firewall would prevent it from joining the cluster as
> a
> > slave. The firewall doesn't prevent it from joining the cluster as
> long
> > as it thinks it's the "master". Baffling.
> >
> > Any ideas to add to this, given my latest findings?
> >
> > Thanks!!!
> >
> >
> > --
> > jessbryant
> >
> ------------------------------------------------------------------------
> > jessbryant's Profile: 'NOVELL FORUMS - View Profile: jessbryant'
> (http://forums.novell.com/member.php?userid=13263)
> > View this thread: 'Strange Node Joining Problem - NOVELL FORUMS'
> (http://forums.novell.com/showthread.php?t=347994)
> >


--
ashwin_pankaj
------------------------------------------------------------------------
ashwin_pankaj's Profile: http://forums.novell.com/member.php?userid=10708
View this thread: http://forums.novell.com/showthread.php?t=347994