Developing software using sockets almost always involves starting, stopping, debugging multiple processes, opening and closing sockets. The processes may or may not work properly as differing stages of development creates “interesting” problems. To complicate matters, multiple systems frequently add to this effort.

This development process often encounters sockets left open after crashes, processes not properly closing or terminating, etc. The end result has processes running with unwanted sockets. When debugging resumes, developers encounter the “Address already in use” problem.

Source code is on github.

listeningPort.py – Map port names to process pid

listeningPort.py performs a mapping of a port number to a process id, i.e., a pid. Furthermore, the interface allows a range of ports as input. The output can take several hopefully useful forms.

Additionally, since listeningPort.py maps ports to pids, it has the capability to kill a process using a specific port. This significantly facilitates socket development.

Installation

Some developers will need to install psutils:

        pip install psutils

Copy listeningPort.py to somewhere in your $PATH. A common place would be $HOME/bin.

While copying, you might like to rename it to something like listening

The logic of import listeningPort can be implemented. Assume that listeningPort.py has been copied to $HOME/bin/listening . Then create a symbolic link with a .py extension, possibly add $HOME/bin to your path and import it:

        cd $HOME/bin         # Or wherever you copied listeningPort.py
        ln -s ./listening listening.py

        cd     # Test the "install", go to a dev dir
        python               # Start python
        import os
        HOME_BIN = os.getenv('HOME') + '/bin'
        sys.path.append(HOME_BIN)
        import listening # Assumes you symbolically linked to "listening.py"
        .... 

A brief demo of the problem

The demo server, demoSocket.py, binds to a user selected port and does little else. Its behavior opens a port and keeps it open. demoSocket.py doesn’t do much. It only listens to a socket. It serves its purpose by continuing to listen to a port. Its purpose is to illustrate the utility listeningPort.py and nothing else.

The following examples use ports 5570 through 5580. Port 5570 is the default port used by ZeroMQ.

To illustrate the “Address in use” problem, open two console windows and cd to the directory where this code lives. In one console, start the demo:

./demoSocket.py
Listening on port 5570

Port 5570 is now bound to this process.

In the other console window, start demoSocket.py in an attempt to bind to the same port:

./demoSocket.py
Listening on port 5570
Traceback (most recent call last):
  File "./demoSocket.py", line 57, in 
    main()
  File "./demoSocket.py", line 40, in main
    status = serversocket.bind((socket.gethostname(), port))
  File "/usr/lib64/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use

The second attempt to have another process listen to the same port fails. Only one process at a time may listen to a specific port. Your traceback may be slightly different. The significant portion that easily stops development is: “Address already in use“.

listeningPort.py to determine who uses that port

Source code available at github.

In that same window that display that Address already in use error, determine the pid and command line of the port listener:

./listeningPort.py 5570
Port 5570 : listening thru pid 28235 ./demoSocket.py 5570

Now we can see that port 5570 has pid 28235 and the command line of pid 28235 is “./demoSocket.py 5570“.

But wait, there’s more! Here is the same query with different outputs. Your output will have different pids:

./listeningPort.py --short 5570    # Only significant params
5570 28235 ./demoSocket.py 5570
$ ./listeningPort.py --pid 5570    # Only the pid
28235
$ ./listeningPort.py --proc 5570   # The command line of the process
python ./demoSocket.py 5570
$ ./listeningPort.py --kill 5570   # Kill the process using that port

The command line switch of --short abbreviates the output for ease of parsing using additional utilities such as awk, perl, python, etc.

The --pid flag outputs only the pid of the process holding the port.

The --proc supplies only the command line of the process holding the port. This can be useful in further identifying the source of that port holder. While finding this command line from a pid is trivial, encapsulating this provides yet another easy access to information.

The --kill uses the port to identify the pid and then attempts to kill that process. Depending on permissions, this may or may not succeed.

The –help discussion

The --help flag outputs:

./listeningPort.py --help

List the processes that are listening to a port.
Defaults to ZeroMQ port of 5570.

Use by:
  listeningPort [--help] [--short | --pid | --proc | --kill]
                 [ ...]
e.g.:
  listeningPort 5570             # The ZeroMQ default port
  listeningPort 5570 5571 5572   # Multiple ports may be checked
  listeningPort --short 5570     # Short form of output
  listeningPort --pid 5570       # Output pid only
  listeningPort $(seq 5570 5580) # Ports 5570 through 5580 inclusive.
  listeningPort --kill $(seq 5570 5580) # Kill all processes
                                        # listening to ports 5570 thru 5580.

For the case of a free port, the output is similar to:
  listeningPort 5571
  Port 5571 : Nobody listening

--help = this message

Only one of the following can be supplied:
--short = Output consists of only three space separated fields:
      
    Ports with nobody listening gets ignored for output.
--pid  = Output consists only of a pid
--proc = Output consists only of process names
--kill = Any ports with a listener will be killed with "kill -9 "
         A successful kill has a return code of non-zero.
         A non-zero output communicates the number of processes
         killed. The --kill may kill multiple processes when a
         sequence of ports gets used.
         An unsuccessful kill returns 0.
         A "sudo" may be required to kill some processes.

Return codes:
   255  == Invalid command line.
    0   == Nobody listening to 
  > 0   == The number of ports someone is listening to.
           For a series of port, this value is the number
           of ports with a listener.
           For a single port, this will be 1 is someone
           is listening.


***NOTICE***: This routine does NOT work on OSX!
Replace this with:
    lsof -i | awk '{ print $2; }' | head -2
    PID
    18101
This prints only the pid of the process using this port.
Now use "ps" to find the process:
    ps ax | grep 18191 | grep -v grep
    10191 s001  S+    0:00.00 /usr/bin/python /usr/local/bin/logCollector

Querying multiple ports

In socket development a common scenarios uses a range of ports. The linux utility “seq” easily supplies these value to listeningPort.py. Thus, for our examples that use ports 5570 through 5580, seq would display:

seq -s ' ' 5570 5580
5570 5571 5572 5573 5574 5575 5576 5577 5578 5579 5580

The -s flag uses a space to separate each sequence number. seq has other useful flags. See the man page if interested.

To illustrate multiple ports, in the console window running demoSocket.py, start multiple instance with different sockets. Throw each invocation in the background:

./demoSocket.py 5570 &
[1] 29385
$ Listening on port 5570
./demoSocket.py 5573 &
[2] 29487
$ Listening on port 5573
./demoSocket.py 5574 &
[3] 29492
$ Listening on port 5574
./demoSocket.py 5579 &
[4] 29497
$ Listening on port 5579
jobs
[1]   Running                 ./demoSocket.py 5570 &
[2]   Running                 ./demoSocket.py 5573 &
[3]-  Running                 ./demoSocket.py 5574 &
[4]+  Running                 ./demoSocket.py 5579 &

Now find the pids of these given port numbers:

./listeningPort.py $(seq 5570 5580)
Port 5570 : listening thru pid 29385 ./demoSocket.py 5570
Port 5571 : Nobody listening
Port 5572 : Nobody listening
Port 5573 : listening thru pid 29487 ./demoSocket.py 5573
Port 5574 : listening thru pid 29492 ./demoSocket.py 5574
Port 5575 : Nobody listening
Port 5576 : Nobody listening
Port 5577 : Nobody listening
Port 5578 : Nobody listening
Port 5579 : listening thru pid 29497 ./demoSocket.py 5579
Port 5580 : Nobody listening

Get a more succinct list:

./listeningPort.py --short $(seq 5570 5580)
5570 29385 ./demoSocket.py 5570
5573 29487 ./demoSocket.py 5573
5574 29492 ./demoSocket.py 5574
5579 29497 ./demoSocket.py 5579

You have the idea.

Now let’s kill all those background processes.

./listeningPort.py --kill $(seq 5570 5580)
Port 5571 : Nobody listening
Port 5572 : Nobody listening
Port 5575 : Nobody listening
Port 5576 : Nobody listening
Port 5577 : Nobody listening
Port 5578 : Nobody listening
Port 5580 : Nobody listening

The processes killed do not report anything, only the ports with nobody listening gets reported.

In the console starting the demoSocket.py processes, press the return key:

[1]   Terminated              ./demoSocket.py 5570
[2]   Terminated              ./demoSocket.py 5573
[3]-  Terminated              ./demoSocket.py 5574
[4]+  Terminated              ./demoSocket.py 5579