We will look at four network applications, all running over TCP using sockets, written completely from scratch in Python, presented in order of increasing complexity:
This is perhaps the simplest possible server. It listens on port 59090. When a client connects, the server sends the current datetime to the client. The with
statement ensures that the connection is automatically closed at the end of the block. After closing the connection, the server goes back to waiting for the next client.
# A simple TCP server. When a client connects, it sends the client the current # datetime, then closes the connection. This is arguably the simplest server # you can write. Beware though that a client has to be completely served its # date before the server will be able to handle another client. import socketserver from datetime import datetime class DateHandler(socketserver.StreamRequestHandler): def handle(self): self.wfile.write(f'{datetime.now().isoformat()}\n'.encode('utf-8')) with socketserver.TCPServer(('', 59090), DateHandler) as server: print('The date server is running...') server.serve_forever()
Discussion:
TCPServer
object abstracts the server socket: You pass the server a Handler
class, in our case, a subclass StreamRequestHandler
. You have to override handle
; you can also override setup
and finish
(but call the base class versions if you do). server
, client_address
, and request
. For StreamRequestHandler
s, you get instance variables rfile
and wfile
. When the handle
method ends, wfile
will be automatically flushed.handle
method ends and the communication socket is closed, so in this case, closing the connection is initiated by the server.$ python3 date_server.py The date server is running...
A quick test with nc
:
$ nc localhost 59090 2019-02-17T22:26:21.629324
Here’s how to do the client in Python. It connects, prints the datetime it gets from the server, then exits.
# A command line client for the date server. import sys import socket if len(sys.argv) != 2: print('Pass the server IP as the sole command line argument') else: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: sock.connect((sys.argv[1], 59090)) print(sock.recv(1024).decode('utf-8'))
Discussion:
AF_INET
means the networking layer underneath is IPv4. Other possible values are AF_UNIX
, AF_INET6
, AF_BLUETOOTH
, and others.SOCK_STREAM
is for TCP; there is also SOCK_DGRAM
for UDP among others.socket.makefile
; however, for now it’s fine to use the lower-level socket calls. We know the date will fit in 1024 bytes, so recv
will be fine. (We’ll do the file thing later.)
$ python3 date_client.py localhost 2019-02-17T22:28:14.988791
The previous example was pretty trivial: it did not read any data from the client, and worse, it served only one client at a time.
This next server receives lines of text from a client and sends back the lines uppercased. It efficiently handles multiple clients at once: When a client connects, the server spawns a thread, dedicated to just that client, to read, uppercase, and reply. The server can listen for and serve other clients at the same time, so we have true concurrency. We make use of Python’s ThreadingMixIn
to do this thread spawning automatically:
# A server program which accepts requests from clients to capitalize strings. When # clients connect, a new thread is started to handle a client. The receiving of the # client data, the capitalizing, and the sending back of the data is handled on the # worker thread, allowing much greater throughput because more clients can be handled # concurrently. import socketserver import threading class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer): daemon_threads = True allow_reuse_address = True class CapitalizeHandler(socketserver.StreamRequestHandler): def handle(self): client = f'{self.client_address} on {threading.currentThread().getName()}' print(f'Connected: {client}') while True: data = self.rfile.readline() if not data: break self.wfile.write(data.decode('utf-8').upper().encode('utf-8')) print(f'Closed: {client}') with ThreadedTCPServer(('', 59898), CapitalizeHandler) as server: print(f'The capitalization server is running...') server.serve_forever()
Discussion:
TCPServer
and ThreadingMixIn
. This means that every time a client connects, the handler for it will run on a new thread.StreamRequestHandler
’s wfile
attribute is an unbuffered stream! Therefore, no flush
is necessary after writing the capitalized string.rfile.readLine()
returns the empty string. Note that if the client sends a blank line, it will arrive at the server as a one-character string containing a newline. Only when the client closed its end will readline
return the empty string (with no new line). At this point, the loop exits and the handle
method returns, closing the server-side socket.We can test with nc
. Since the handler repeatedly reads lines from the client until the end of standard input, you need to terminate your nc
session with Ctrl+D or Ctrl+C:
$ nc localhost 59898 you’re not me! YOU’RE NOT ME! Say fellas, did somebody mention the Door to Darkness? SAY FELLAS, DID SOMEBODY MENTION THE DOOR TO DARKNESS? e d g e l o r d E D G E L O R D
Of course, we can write our own simple command line client:
import sys import socket if len(sys.argv) != 2: print('Pass the server IP as the sole command line argument') else: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: sock.connect((sys.argv[1], 59898)) print('Enter lines of text then Ctrl+D or Ctrl+C to quit') while True: line = sys.stdin.readline() if not line: # End of standard input, exit this entire script break sock.sendall(f'{line}'.encode('utf-8')) while True: data = sock.recv(128) print(data.decode("utf-8"), end='') if len(data) < 128: # No more of this message, go back to waiting for next message break
Discussion:
while True
loops but they will do for now.readLine()
returns the empty string. Blank lines from standard input are read in as strings with a single newline character in them.recv
method, which requires us to specify a buffer size. If the server message is larger than that size, we have to repeatedly read. We know we have reached the end of the message when the buffer is not completely full.The client can be used interactively, or we can redirect standard input:
$ python3 capitalize_client.py localhost < hello.py Enter lines of text then Ctrl+D or Ctrl+C to quit PRINT('HELLO, WORLD')
Classwork
Experimentation time. Replace the loop in the client that keeps reading from the receive buffer until we get an empty or partially full buffer with code that just reads a fixed amount of bytes. Arrange for the server to send too much data. How the client behave in this case?
Here is the server for a two-player game. It listens for two clients to connect, and spawns a thread for each: the first is Player X and the second is Player O. The client and server send simple string messages back and forth to each other; messages correspond to the Tic Tac Toe protocol, which I made up for this example.
# A server for a multi-player tic tac toe game. Loosely based on an example in # Deitel and Deitel’s “Java How to Program” book. For this project I created a # new application-level protocol called TTTP (for Tic Tac Toe Protocol), which # is entirely plain text. The messages of TTTP are: # # Client -> Server # MOVE <n> # QUIT # # Server -> Client # WELCOME <char> # VALID_MOVE # OTHER_PLAYER_MOVED <n> # OTHER_PLAYER_LEFT # VICTORY # DEFEAT # TIE # MESSAGE <text> import socketserver import threading class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer): daemon_threads = True allow_reuse_address = True class PlayerHandler(socketserver.StreamRequestHandler): def handle(self): self.opponent = None client = f'{self.client_address} on {threading.currentThread().getName()}' print(f'Connected: {client}') try: self.initialize() self.process_commands() except Exception as e: print(e) finally: try: self.opponent.send('OTHER_PLAYER_LEFT') except: # Hack for when the game ends, not happy about this pass print(f'Closed: {client}') def send(self, message): self.wfile.write(f'{message}\n'.encode('utf-8')) def initialize(self): Game.join(self) self.send('WELCOME ' + self.mark) if self.mark == 'X': self.game.current_player = self self.send('MESSAGE Waiting for opponent to connect') else: self.opponent = self.game.current_player self.opponent.opponent = self self.opponent.send('MESSAGE Your move') def process_commands(self): while True: command = self.rfile.readline() if not command: break command = command.decode('utf-8') if command.startswith('QUIT'): return elif command.startswith('MOVE'): self.process_move_command(int(command[5:])) def process_move_command(self, location): try: self.game.move(location, self) self.send('VALID_MOVE') self.opponent.send(f'OPPONENT_MOVED {location}') if self.game.has_winner(): self.send('VICTORY') self.opponent.send('DEFEAT') elif self.game.board_filled_up(): self.send('TIE') self.opponent.send('TIE') except Exception as e: self.send('MESSAGE ' + str(e)) class Game: next_game = None game_selection_lock = threading.Lock() def __init__(self): self.board = [None] * 9 self.current_player = None self.lock = threading.Lock() def has_winner(self): b = self.board return ((b[0] is not None and b[0] == b[1] and b[0] == b[2]) or (b[3] is not None and b[3] == b[4] and b[3] == b[5]) or (b[6] is not None and b[6] == b[7] and b[6] == b[8]) or (b[0] is not None and b[0] == b[3] and b[0] == b[6]) or (b[1] is not None and b[1] == b[4] and b[1] == b[7]) or (b[2] is not None and b[2] == b[5] and b[2] == b[8]) or (b[0] is not None and b[0] == b[4] and b[0] == b[8]) or (b[2] is not None and b[2] == b[4] and b[2] == b[6])) def board_filled_up(self): return all(cell is not None for cell in self.board) def move(self, location, player): with self.lock: if player != self.current_player: raise ValueError('Not your turn') elif player.opponent is None: raise ValueError('You don’t have an opponent yet') elif self.board[location] is not None: raise ValueError('Cell already occupied') self.board[location] = self.current_player self.current_player = self.current_player.opponent @classmethod def join(cls, player): with cls.game_selection_lock: if cls.next_game is None: cls.next_game = Game() player.game = cls.next_game player.mark = 'X' else: player.mark = 'O' player.game = cls.next_game cls.next_game = None with ThreadedTCPServer(('', 58901), PlayerHandler) as server: print(f'The Tic Tac Toe server is running...') server.serve_forever()
We can skip the client. We can just use a graphical Java client on this page; after all, it’s cool to say you have a game written in multiple programming languages. If you feel like being really old school, you can play the game using nc
.
Sorry, this is a homework assignment.
We’ve covered: