| View previous topic :: View next topic |
| Author |
Message |
Ignoramus6517
Guest
|
 Posted: Tue Nov 18, 2008 12:23 am Post subject: Zombie port being used UNKNOWN BY WHAT PROCESS |
 |
|
We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
.... stuff ...
tcp 0 0 *:45764 *:* LISTEN root 12180 -
.... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
--
Due to extreme spam originating from Google Groups, and their inattention
to spammers, I and many others block all articles originating
from Google Groups. If you want your postings to be seen by
more readers you will need to find a different means of
posting on Usenet.
http://improve-usenet.org/ |
|
| |
|
 Back to top |
Cork Soaker
Guest
|
| Posted: Tue Nov 18, 2008 12:43 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
Ignoramus6517 wrote:
| Quote: | We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
... stuff ...
tcp 0 0 *:45764 *:* LISTEN root 12180 -
... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
|
Well, what's the process? top will tell you what 12180 is. |
|
| |
|
| Back to top |
Ignoramus6517
Guest
|
| Posted: Tue Nov 18, 2008 12:44 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
On 2008-11-17, Ignoramus6517 <ignoramus6517@NOSPAM.6517.invalid> wrote:
| Quote: | We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
... stuff ...
tcp 0 0 *:45764 *:* LISTEN root 12180 -
... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
|
One more observation. If I telnet to that port and say "fuck" and hit
ENTER, I get this output in /var/log/messages:
Nov 17 12:29:35 myserver kernel: [131074.840941] RPC: bad TCP reclen 0x6675636b (non-terminal)
i
| Quote: | So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
|
--
Due to extreme spam originating from Google Groups, and their inattention
to spammers, I and many others block all articles originating
from Google Groups. If you want your postings to be seen by
more readers you will need to find a different means of
posting on Usenet.
http://improve-usenet.org/ |
|
| |
|
| Back to top |
Allen Kistler
Guest
|
| Posted: Tue Nov 18, 2008 1:17 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
Ignoramus6517 wrote:
| Quote: | On 2008-11-17, Ignoramus6517 <ignoramus6517@NOSPAM.6517.invalid> wrote:
We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
... stuff ...
tcp 0 0 *:45764 *:* LISTEN root 12180 -
... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
One more observation. If I telnet to that port and say "fuck" and hit
ENTER, I get this output in /var/log/messages:
Nov 17 12:29:35 myserver kernel: [131074.840941] RPC: bad TCP reclen 0x6675636b (non-terminal)
i
So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
|
Try "rpcinfo -p" then. |
|
| |
|
| Back to top |
Ignoramus6517
Guest
|
| Posted: Tue Nov 18, 2008 1:53 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
On 2008-11-17, Allen Kistler <ackistler@oohay.moc> wrote:
| Quote: | Ignoramus6517 wrote:
On 2008-11-17, Ignoramus6517 <ignoramus6517@NOSPAM.6517.invalid> wrote:
We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
... stuff ...
tcp 0 0 *:45764 *:* LISTEN root 12180 -
... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
One more observation. If I telnet to that port and say "fuck" and hit
ENTER, I get this output in /var/log/messages:
Nov 17 12:29:35 myserver kernel: [131074.840941] RPC: bad TCP reclen 0x6675636b (non-terminal)
i
So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
Try "rpcinfo -p" then.
|
Very interesting!
root:~ ==] rpcinfo -p
program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100021 1 tcp 45764 nlockmgr
100021 3 tcp 45764 nlockmgr
100021 4 tcp 45764 nlockmgr
100024 1 udp 32765 status
100024 1 tcp 32765 status
So, how can I make portmap allocate ports from only a range that I
specify?
--
Due to extreme spam originating from Google Groups, and their inattention
to spammers, I and many others block all articles originating
from Google Groups. If you want your postings to be seen by
more readers you will need to find a different means of
posting on Usenet.
http://improve-usenet.org/ |
|
| |
|
| Back to top |
Ignoramus6517
Guest
|
| Posted: Tue Nov 18, 2008 1:53 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
On 2008-11-17, Cork Soaker <Thunderbird@Hardy.invalid> wrote:
| Quote: | Ignoramus6517 wrote:
We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
... stuff ...
tcp 0 0 *:45764 *:* LISTEN root 12180 -
... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
Well, what's the process? top will tell you what 12180 is.
|
There is NO such process. That's the thing. By the way, 12180 is a
"inode number" not a process. Whatever it means.
--
Due to extreme spam originating from Google Groups, and their inattention
to spammers, I and many others block all articles originating
from Google Groups. If you want your postings to be seen by
more readers you will need to find a different means of
posting on Usenet.
http://improve-usenet.org/ |
|
| |
|
| Back to top |
Allen Kistler
Guest
|
| Posted: Tue Nov 18, 2008 2:31 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
Ignoramus6517 wrote:
| Quote: | On 2008-11-17, Allen Kistler <ackistler@oohay.moc> wrote:
Try "rpcinfo -p" then.
Very interesting!
root:~ ==] rpcinfo -p
program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100021 1 tcp 45764 nlockmgr
100021 3 tcp 45764 nlockmgr
100021 4 tcp 45764 nlockmgr
100024 1 udp 32765 status
100024 1 tcp 32765 status
So, how can I make portmap allocate ports from only a range that I
specify?
|
Actually the question is how to make nfs lockd and nfs statd request
only the port that you specify. (If the rpc process does not request a
port or if the requested port is unavailable, then portmap picks a
random open port.) Maybe the first question to answer, though, is why
you're running lockd and statd if you're not sharing anything via nfs.
(I notice mountd and nfsd are not running.) Also a good follow-up
question would be why you're running portmap (sometimes aka rpcbind) if
you're not mounting anything via nfs, if indeed you're not mounting
anything via nfs.
The answer to how to make lockd and statd use fixed ports depends a bit
on distro. In RH and Fedora, I fix the lockd port in /etc/modprobe.conf
and the statd port in /etc/sysconfig/nfs as an argument that gets passed
to statd when it starts.
In /etc/modprobe.conf
options lockd nlm_udpport=2050 nlm_tcpport=2050
In /etc/sysconfig/nfs
STATD_PORT=2052
YMMV |
|
| |
|
| Back to top |
dennis@home
Guest
|
| Posted: Tue Nov 18, 2008 3:13 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
"Ignoramus6517" <ignoramus6517@NOSPAM.6517.invalid> wrote in message
news:cZ-dnXzCufYmVrzUnZ2dnUVZ_obinZ2d@giganews.com...
| Quote: | On 2008-11-17, Cork Soaker <Thunderbird@Hardy.invalid> wrote:
Ignoramus6517 wrote:
We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
... stuff ...
tcp 0 0 *:45764 *:*
LISTEN root 12180 -
... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
Well, what's the process? top will tell you what 12180 is.
There is NO such process. That's the thing. By the way, 12180 is a
"inode number" not a process. Whatever it means.
|
Inodes are what identifies files on a file system.
You could see what file is inode 12180 but I doubt if it will help.
BTW if you listen on that port the application could fail to start anytime.
Ports in that range can be used by things like web browsers and other
applications used by users at "random".
But I am surprised you didn't know that anyway being an expert. |
|
| |
|
| Back to top |
Cork Soaker
Guest
|
| Posted: Tue Nov 18, 2008 3:28 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
Ignoramus6517 wrote:
| Quote: | On 2008-11-17, Cork Soaker <Thunderbird@Hardy.invalid> wrote:
Ignoramus6517 wrote:
We have a server with a variety of processes running and listening to
various ports. They tend to work well.
One in particular, not remarkable in any way, listens on port 45764.
Today it is no longer is able to listen and says listen failed.
My investigation turned up the following facts:
1) If I telnet to this port, connection gets established
2) fuser -t 45764/tcp outputs NOTHING
3) netstat -aep outputs
... stuff ...
tcp 0 0 *:45764 *:* LISTEN root 12180 -
... stuff ...
Most other lines ...stuff... contact the PID and name of the process
in question, but not this one. This one has a "-" for PID/Name
So, we have a rogue zombie TCP port here, any ideas? Who is listening
on it?
Well, what's the process? top will tell you what 12180 is.
There is NO such process. That's the thing. By the way, 12180 is a
"inode number" not a process. Whatever it means.
|
Well then, that's cleared that up! :-)
Also, Wireshark can snoop any traffic that it /may/ be making. |
|
| |
|
| Back to top |
Ignoramus6517
Guest
|
| Posted: Tue Nov 18, 2008 3:53 am Post subject: Re: Zombie port being used UNKNOWN BY WHAT PROCESS |
|
|
On 2008-11-17, Allen Kistler <ackistler@oohay.moc> wrote:
| Quote: | Ignoramus6517 wrote:
On 2008-11-17, Allen Kistler <ackistler@oohay.moc> wrote:
Try "rpcinfo -p" then.
Very interesting!
root:~ ==] rpcinfo -p
program vers proto port
100000 2 tcp 111 portmapper
100000 2 udp 111 portmapper
100021 1 tcp 45764 nlockmgr
100021 3 tcp 45764 nlockmgr
100021 4 tcp 45764 nlockmgr
100024 1 udp 32765 status
100024 1 tcp 32765 status
So, how can I make portmap allocate ports from only a range that I
specify?
Actually the question is how to make nfs lockd and nfs statd request
only the port that you specify. (If the rpc process does not request a
port or if the requested port is unavailable, then portmap picks a
random open port.) Maybe the first question to answer, though, is why
you're running lockd and statd if you're not sharing anything via nfs.
(I notice mountd and nfsd are not running.) Also a good follow-up
question would be why you're running portmap (sometimes aka rpcbind) if
you're not mounting anything via nfs, if indeed you're not mounting
anything via nfs.
|
Actually I am using some NFS shares as a client, but this server is
not sharing anything of its own own.
| Quote: | The answer to how to make lockd and statd use fixed ports depends a bit
on distro. In RH and Fedora, I fix the lockd port in /etc/modprobe.conf
and the statd port in /etc/sysconfig/nfs as an argument that gets passed
to statd when it starts.
In /etc/modprobe.conf
options lockd nlm_udpport=2050 nlm_tcpport=2050
In /etc/sysconfig/nfs
STATD_PORT=2052
YMMV
|
We decided to move some ports, and adjust the ephemeral port range by
setting net.ipv4.ip_local_port_range = 58000 65535, in file
/etc/sysctl.conf. We'll move some more ports around and increase the
range from 58000 start to 51000 start.
--
Due to extreme spam originating from Google Groups, and their inattention
to spammers, I and many others block all articles originating
from Google Groups. If you want your postings to be seen by
more readers you will need to find a different means of
posting on Usenet.
http://improve-usenet.org/ |
|
| |
|
| Back to top |
|
