ENIB S7-NSA 04_sockets - Socket programming

{} Execution environment

{} UDP server

Among the protocols commonly used to develop networked applications, the UDP protocol is very simple to implement.
It involves running a server program that waits for requests arriving on a port number it is bound to.
This server can be accessed by multiple client programs that send requests to the server's IP address and port, and then wait for a reply in return.

From your host system, with the convenience of your usual text editor, create a file named SHARED/prog_udp_server.py with the following initial contents:

#!/usr/bin/env python3

import sys
import socket

def main():
  try:
    port=int(sys.argv[1]) # expect a port number on the command line
  except:
    print(f'usage: {sys.argv[0]} port'); sys.exit(1)
  #
  server=socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # create UDP socket
  server.bind(('0.0.0.0', port)) # bind this socket to the chosen port number
  #
  name=socket.gethostname() # determine the name of the current machine
  print(f'host {name} waiting for UDP messages on port {port}')
  #
  while True:
    pass # the contents of this loop will be changed later on...
  #
  server.close() # close UDP socket once it becomes useless (never reached)

if __name__=='__main__': main()

Make an effort to understand how it works by reading the comments; naturally, the loop is incomplete for the moment.
A socket is a mechanism provided by the operating system to perform communication operations; there are several different types of sockets.
The parameters chosen when creating this socket indicate that it is to be dedicated to the UDP protocol.

From machine A, make this file executable using the command chmod +x SHARED/prog_udp_server.py and then run it with SHARED/prog_udp_server.py.
Observe the message informing you that you need to choose a port number; then run it again with SHARED/prog_udp_server.py 9988.
From another terminal✍ display the system's UDP sockets using ss -panu; you should see a line indicating that your Python program is indeed monitoring port 9988.
Since the loop in this program merely blocks without doing anything, stop it by pressing Ctrl c.

Now that you have verified that your UDP socket is indeed monitoring the chosen port, complete your program's loop as follows:

  while True:
    (data, (client_ip, client_port))=server.recvfrom(1024) # wait for request
    client=f'{client_ip}:{client_port}'
    request=data.decode() # consider request as text
    print(f'from {client}: {request!r}')

As this code suggests, the .recvfrom() operation provides both a received packet of bytes (data) and the details (IP address and port number) of the socket that sent the data.
For the sake of simplicity here, the packet of bytes is treated as text (using the .decode() operation).
A simple display of the sender's details and the received text request is produced to make observation easier.

Run your program again on A with SHARED/prog_udp_server.py 9988 and, from another terminal✍, send it a request with echo 'Hello, myself!' | ncat -u 127.0.0.1 9988 and another with echo 'Hello again, myself!' | ncat -u A 9988.
Each ncat -u command behaves as a UDP client that send its standard input to the server specified on its command line (IP address and port number).
Check, in the terminal running your program, that the displayed output correctly illustrates the messages received.
On machines B and C✍, also try echo 'Hello, neighbour!' | ncat -u A 9988 and verify the successful reception on your UDP server on A.
Notice that, unlike the server for which a port number is explicitly chosen, the port number of the clients contacting the server is completely arbitrary; this will be discussed again very soon.

Start another server now, bound to the same port, on machine B with SHARED/prog_udp_server.py 9988✍.
From machine C, try echo 'Hello, everyone!' | ncat -u 192.168.10.255 9988 and observe the reaction of all your UDP servers.
You have just broadcast a UDP message that caused all the hosts on the subnet listening on the chosen port to react, thanks to the use of the subnet's broadcast address.
You can stop the your UDP servers by pressing Ctrl c.

Complete your program's loop once again by adding these two new lines:

  while True:
    # ...four previous lines...
    #
    reply=f'{len(request)} chars: {request!r}\n' # prepare reply
    server.sendto(reply.encode(), (client_ip, client_port)) # send to client

The first step is to prepare a reply to the received request; here, for the sake of the example, we produce a reply of little practical interest (the received message preceded by its character count), but one that is easy to recognise.
Once this reply is ready, .sendto() will attempt to deliver it to the client that contacted the server.
The reply, which is in text form here, must be converted into a packet of bytes to be transmitted using .encode().
The client's details (IP address and port number), which were obtained when the request was received, are then used to specify the destination of the packet of bytes to be sent.

Run your program again on machine A with SHARED/prog_udp_server.py 9988.
On machines B and C✍, use the command ncat -u A 9988.
The aim is still to send messages to the specified UDP server, but this time in interactive mode: anything typed at the keyboard is sent as a UDP request to the server, and the UDP replies received are displayed in the terminal.
Observe that your keyboard input from either of these machines does indeed trigger the UDP server on A, and that you receive the appropriate reply.

You have now implemented a complete UDP server: it is capable of waiting for a request on a chosen port number and then replying to any client that has contacted it.
You can now see the fundamental difference between a client and a server:

A server must be explicitly bound to a port number because all clients must know in advance which port number the server is listening on.
On the same host, several servers cannot therefore be bound to the same port number, because this is precisely what distinguishes them.
A client does NOT need to be bound to a chosen port number because the system assigns one arbitrarily, and this accompanies the request (just like the IP address).
The server receiving the request discovers this arbitrary port at that point, without needing to know it in advance (not any more than it knows the client's IP address in advance)✍
It is even discouraged to bind the client to a port number, because doing so prevents multiple clients from being launched from the same host.
Indeed, if they were bound to the same port number, there would be no way to determine which client a reply addressed to the host's IP address and the identical port number was intended for.
.

{} UDP client

The generic ncat tool has served as a UDP client to test your UDP server; it is now time to write your own program to act as a UDP client.

From your host system, with the convenience of your usual text editor, create a file named SHARED/prog_udp_client.py with the following contents:

#!/usr/bin/env python3

import sys
import socket

def main():
  try:
    host=sys.argv[1]      # expect an host name/address on the command line
    port=int(sys.argv[2]) # expect a port number on the command line
  except:
    print(f'usage: {sys.argv[0]} server_host server_port'); sys.exit(1)
  #
  client=socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # create UDP socket
  #
  while True:
    print('waiting for input')
    request=sys.stdin.readline() # input text-line (ended with \n)
    if not request: break # EOF on standard-input
    #
    client.sendto(request.encode(), (host, port)) # send to server
    #
    data=client.recv(1024) # wait for reply from server
    reply=data.decode() # consider reply as text
    print(reply)
  #
  client.close() # close UDP socket once it becomes useless

if __name__=='__main__': main()

Make an effort to understand how it works by reading the comments.
You can see that this program expects the server's details on its command line: the server's IP address (or its name, for greater convenience) and the port number to which the server is expected to be bound.
Note that the creation of the UDP socket is identical to that used for the server; however, as explained previously, the .bind() operation is not normally used by the client✍.
When a line of text is entered on the standard input✍, it is sent to the server in exactly the same way that the server sent its reply to the client in your previous program.
Receiving the reply from the server could use .recvfrom() in order to verify that the expected server did indeed produce the received reply.
Here, to keep the example simple, the .recv() operation is perfectly sufficient to obtain the packet of bytes making up the reply.

From one of your virtual machines, make this file executable with the command chmod +x SHARED/prog_udp_client.py.
Start your UDP server on machine A with SHARED/prog_udp_server.py 9988, then on machines B and C✍ use your UDP client with SHARED/prog_udp_client.py A 9988.
Verify that your UDP client programs are able to interact with your UDP server as expected.

On one of your virtual machines running your UDP client, open another terminal✍ in order to launch a second UDP client targeting the same server.
Observe that these two clients on the same machine interact with the same server without any problems; indeed, the assignment of an arbitrary port to each of them allows them to be distinguished.

Stop your UDP server on A by pressing Ctrl c, without stopping your clients.
On just one of your clients, enter a line of text and observe that it waits indefinitely for a reply (since the target server no longer exists).
Restart your UDP server on A and observe that the blocked client remains in this state (you will need to terminate it by pressing Ctrl c).
This is a property of the UDP protocol: sending a message to a non-existent peer causes the message to be lost immediately, without the sender being informed.
On another client that was left running, enter another line of text and observe that, despite the server having been stopped and restarted, this client can continue communicating as before.
This is another property of the UDP protocol: there is no connection when communicating over UDP; what matters is that the receiver is available at the moment the sender transmits its message.

You can now stop all your UDP client and server programs by pressing Ctrl c.

{} TCP server

Among the protocols commonly used to develop networked applications, the TCP protocol is even more widely used than UDP.
It involves running a server program that waits for connections arriving on a port number it is bound to.
This server can be accessed by multiple client programs that connect to the server's IP address and port.
Once a connection has been established, the client and the server use it to exchange bidirectional streams of bytes.

From your host system, with the convenience of your usual text editor, create a file named SHARED/prog_tcp_server.py with the following initial contents:

#!/usr/bin/env python3

import sys
import socket

def main():
  try:
    port=int(sys.argv[1]) # expect a port number on the command line
  except:
    print(f'usage: {sys.argv[0]} port'); sys.exit(1)
  #
  server=socket.socket(socket.AF_INET, socket.SOCK_STREAM) # create TCP socket
  server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) # (optional)
  server.bind(('0.0.0.0', port)) # bind this socket to the chosen port number
  server.listen() # dedicated to accepting connections
  #
  name=socket.gethostname() # determine the name of the current machine
  print(f'host {name} waiting for TCP connections on port {port}')
  # 
  while True:
    (conn, (client_ip, client_port)) = server.accept() # wait for connection
    handle_client(conn, client_ip, client_port)
  #
  server.close() # close TCP listener once it becomes useless (never reached)

def handle_client(conn, client_ip, client_port):
  client=f'{client_ip}:{client_port}'
  print(f'connection from', client)
  #
  conn_file=conn.makefile('rw', buffering=1) # helper for text-lines
  while True:
    request=conn_file.readline() # wait for text-line (ended with \n)
    if not request: break # EOF on TCP socket
    print(f'from {client}: {request!r}')
    #
    reply=f'{len(request)} chars: {request!r}\n' # prepare reply
    conn_file.write(reply) # send text-line to client
  #
  print(f'{client} disconnected')
  conn_file.close() # close text-line helper once it becomes useless
  conn.close() # close TCP connection once it becomes useless

if __name__=='__main__': main()

Make an effort to understand how it works by reading the comments.
The parameters chosen when creating this socket indicate that it is to be dedicated to the TCP protocol.
The SO_REUSEADDR option is not generally necessary, but it makes program development easier✍.
In addition to the .bind() operation, which binds the TCP socket to a port number, the .listen() operation tells the operating system that this TCP socket is to be dedicated to detecting incoming connections.
The main loop of a TCP server then consists of waiting for a connection from a TCP client to be detected, and then communicating with that client through the established connection.
As this code suggests, the .accept() operation provides both a connection (conn, another TCP socket) and the details (IP address and port number) of the client socket TCP that made this connection.
The handle_client() function is responsible for the entire dialogue between the TCP client that has just connected and the TCP server.
For the sake of simplicity, since this example deals only with lines of text, a helper (conn_file) takes care of the conversions between a stream of bytes and lines of text.
As in the case of the UDP server, each request received is used to produce and send back a reply.
This dialogue ends when the TCP client closes the connection, causing the end of file (EOF) to be detected.

From machine A, make this file executable using the command chmod +x SHARED/prog_tcp_server.py and then run it with SHARED/prog_tcp_server.py 9988.
On machines B and C✍, use the command ncat A 9988.
Each ncat command acts as a TCP client connecting to the specified TCP server, and interacting with it; anything typed at the keyboard is sent as a request through the connection, and the replies received through that connection are displayed in the terminal.
On some of these machines, open another terminal✍ and run the command ss -pant; you should see a line indicating that the ncat program has ESTABlished a connection to your TCP server.
On machine A, open another terminal✍ and run the command ss -pant; you should see a line indicating that your Python program is indeed LISTENing port 9988, and some other lines about the connections to this server ESTABlished by the clients, as you just saw on the other machines.

Observe that only the first client connecting to the server actually interacts; the other clients seem to have no incidence on the server.
However, as soon as the first client disconnects (pressing the Ctrl d key combination), observe that the next one is considered by the server, and so on...
As it stands, your TCP server is capable of serving only one client at a time, which is rather disappointing, but understandable.
Indeed, the source code shows that the dialogue with one client (handle_client()) must be completed before the server considers the next connection (.accept()).

To fix this limitation, modify your TCP server as follows:

# ...previous import statements...
import threading

# ...main program...
  while True:
    (conn, (client_ip, client_port)) = server.accept() # wait for connection
    # handle_client(conn, client_ip, client_port)
    threading.Thread(
      target=handle_client,                # invoke handle_client()
      args=(conn, client_ip, client_port), # with these three parameters
      daemon=True).start()                 # don't wait for its termination

The idea is to start a thread that executes the same function as before (it is left commented out for reference), but in parallel with the main program.
As a result, the main program loop immediately returns to waiting for the next connection (.accept()), while the dialogue with the client from the previous connection can continue for as long as necessary in a parallel execution.
Under these conditions, there are as many threads as there are established connections (each executes the handle_client() function to communicate with the TCP client that created the connection), and the main program is always ready to accept new connections from new TCP clients.
Stop your TCP server and your ncat TCP clients by pressing Ctrl c, then restart the newly modified TCP server on machine A with SHARED/prog_tcp_server.py 9988.
On machines B and C✍, use again the command ncat A 9988 to verify that multiple TCP clients can now interact simultaneously, each at its own pace, with your TCP server.

This solution, which consists of using one thread per connection, is sufficient for a small-scale project such as this illustrative example.
However, such a server would not be able to cope with being contacted by a very large number of clients simultaneously, because managing a huge number of threads is generally very demanding in terms of system resources.
Modern solutions combine the use of a small number of threads (to take advantage of the parallelism available on multi-processor and multi-core hardware) with asynchronous programming to distribute the processing associated with multiple connections across those few threads.

{} TCP client

The generic ncat tool has served as a TCP client to test your TCP server; it is now time to write your own program to act as a TCP client.

From your host system, with the convenience of your usual text editor, create a file named SHARED/prog_tcp_client.py with the following contents:

#!/usr/bin/env python3

import sys
import socket

def main():
  try:
    host=sys.argv[1]      # expect an host name/address on the command line
    port=int(sys.argv[2]) # expect a port number on the command line
  except:
    print(f'usage: {sys.argv[0]} server_host server_port'); sys.exit(1)
  #
  conn=socket.socket(socket.AF_INET, socket.SOCK_STREAM) # create TCP socket
  conn.connect((host, port)) # connect this socket to the TCP server
  #
  conn_file=conn.makefile('rw', buffering=1) # helper for text-lines
  while True:
    print('waiting for input')
    request=sys.stdin.readline() # input text-line (ended with \n)
    if not request: break # EOF on standard input
    #
    conn_file.write(request) # send text-line to server
    #
    reply=conn_file.readline() # wait for text-line (ended with \n)
    if not reply: break # EOF on TCP socket
    print(reply)
  #
  conn_file.close() # close text-line helper once it becomes useless
  conn.close() # close TCP connection once it becomes useless

if __name__=='__main__': main()

Make an effort to understand how it works by reading the comments.
You can see that this program expects the server's details on its command line: the server's IP address (or its name, for greater convenience) and the port number to which the server is expected to be bound.
Note that the creation of the TCP socket is identical to that used for the server; however, as explained previously, the .bind() operation is not normally used by the client✍.
To be usable, this socket must connect to the TCP server.
For the sake of simplicity, since this example deals only with lines of text, a helper (conn_file) takes care of the conversions between a stream of bytes and lines of text.
When a line of text is entered on the standard input✍, it is sent to the server in exactly the same way that the server sent its reply to the client in your previous program.
Receiving the reply from the server is also similar to receiving the request on the server side, and the concern about end of file detection is the same.
Indeed, since TCP connections are bidirectional, sending and receiving operations are identical on both ends of the connection.

From one of your virtual machines, make this file executable with the command chmod +x SHARED/prog_tcp_client.py.
Start your TCP server on machine A with SHARED/prog_tcp_server.py 9988, then on machines B and C✍ use your TCP client with SHARED/prog_tcp_client.py A 9988.
Verify that your TCP client programs are able to interact with your TCP server as expected.

On one of your virtual machines running your TCP client, open another terminal✍ in order to launch a second TCP client targeting the same server.
Observe that these two clients on the same machine interact with the same server without any problems; indeed, the assignment of an arbitrary port to each of them allows them to be distinguished.

When you terminate a client by pressing Ctrl d, observe that the server correctly detects that the connection has been closed; the client can, of course, be restarted afterwards.
Stop your TCP server on A by pressing Ctrl c, without stopping your clients.
On each of your clients, enter a line of text and observe that this client terminates immediately; its attempt to receive the reply detects an end of file because the connection has been closed by the server.
When you attempt to restart your clients while the server is stopped, you should observe that the connection fails immediately.
This is a property of the TCP protocol: all communication relies on a connection, and failure to establish or maintain that connection results in an explicit error; transmitted data cannot be lost silently (unlike with UDP).