.. title: Echo Server and Client with Twisted
.. slug: echo-server-and-client-with-twisted
.. date: 2015-02-11 00:59:32 UTC
.. tags: python, twisted
.. category:
.. link:
.. description:
.. type: text

Echo Server and Client with Twisted
==========================================

This is an introduction to Twisted using an echo server and client as an example.

.. TEASER_END: Read more

Basic Structure
---------------------

Get all your ducks in a row first. We'll setup a basic structure and then add more code as needed.

::

    import sys
    from twisted.python import log


    def main():
        log.startLogging(sys.stdout)
        log.msg('Start your engines...')


    if __name__ == '__main__':
        main()

It's a simple script that basically does nothing special. We do, however, have imported *log* from Twisted. We'll use this instead of `print` statements to log output or messages as needed.

Add Reactor
------------------------

A reactor is the backbone of Twisted. It's an event loop that looks for any work that needs to be done.

::

    import sys
    from twisted.python import log
    from twisted.internet import reactor


    def main():
        log.startLogging(sys.stdout)
        log.msg('Start your engines...')
        reactor.run()


    if __name__ == '__main__':
        main()

When you run this code you'll see something similar to the following output. However, the process will appear to hang. The reason is that the reactor has started and will continue to run until we stop it. Since our code is still pretty barebones the only way to stop it is to use ctrl+c.

::

    2015-01-15 12:00:58-0800 [-] Log opened.
    2015-01-15 12:00:58-0800 [-] Start your engines...

Introduction to Transport, Protocol, Factory
----------------------------------------------

Transport, in Twisted parlance, is an interface for reading and writing bytes "on the wire" so to speak. It support IPv4, IPv6, UNIX sockets, TCP, UDP, TLS/SSL.

A Protocol builds on top of Transport. Transport generates events and a state machine, Protocol, handles these events. Some examples of events include connection made, data received, connection lost.

Factory creates new Protocol objects and combines them with Transports.

Add Transport
-----------------

Let's add TCP/IPv4 Transport to our code.

::

    import sys
    from twisted.python import log
    from twisted.internet import reactor


    def main():
        log.startLogging(sys.stdout)
        log.msg('Start your engines...')
        reactor.listenTCP(16000, EchoServerFactory())
        reactor.connectTCP('127.0.0.1', 16000, EchoClientFactory())
        reactor.run()


    if __name__ == '__main__':
        main()

Here we have added `listenTCP` and `connectTCP` to our code. These are server and client transports, respectively. You'll also notice `EchoServerFactory` and `EchoClientFactory`. These are Factory classes that we'll develop in the next section.

Note: Our code is now incomplete and cannot be run. Be patient and read the next sections to complete our code.

Add Factory
----------------------

The next thing you need to add is a Factory. A factory basically generates a new object for each connection made. We have a server and client factory available to use.

::

    import sys
    from twisted.python import log
    from twisted.internet import reactor
    from twisted.internet.protocol import ServerFactory, ClientFactory


    class EchoServerFactory(ServerFactory):
        def buildProtocol(self, addr):
            return EchoServerProtocol()


    class EchoClientFactory(ClientFactory):
        def startedConnecting(self, connector):
            log.msg('Started to connect.')

        def buildProtocol(self, addr):
            log.msg('Connected.')
            return EchoClientProtocol()

        def clientConnectionLost(self, connector, reason):
            log.msg('Lost connection. Reason: {}'.format(reason))

        def clientConnectionFailed(self, connector, reason):
            log.msg('Lost failed. Reason: {}'.format(reason))


    def main():
        log.startLogging(sys.stdout)
        log.msg('Start your engines...')
        reactor.listenTCP(16000, EchoServerFactory())
        reactor.connectTCP('127.0.0.1', 16000, EchoClientFactory())
        reactor.run()


    if __name__ == '__main__':
        main()

Server Factory
+++++++++++++++++++++++++++++

This is a very basic Server Factory. We are using the `buildProtocol` method to return a `EchoServerProtocol` object, which is a Protocol (and will be built further in this guide). This code shows how a Factory can glue together a Protocol and Transport.

::

    class EchoServerFactory(ServerFactory):
        def buildProtocol(self, addr):
            return EchoServerProtocol()


Client Factory
+++++++++++++++++++++++++++++

Our Client Factory is a bit more detailed as we take advantage of the methods it provides. Similar to Server Factory we have a `buildProtocol` method that does exactly the same thing but instead of a `EchoServerProtocol` it returns `EchoClientProtocol`.

::

    class EchoClientFactory(ClientFactory):
        def buildProtocol(self, addr):
            return EchoClientProtocol()

        def startedConnecting(self, connector):
            log.msg('Started to connect.')

        def clientConnectionLost(self, connector, reason):
            log.msg('Lost connection. Reason: {}'.format(reason))

        def clientConnectionFailed(self, connector, reason):
            log.msg('Lost failed. Reason: {}'.format(reason))

The other methods are simple enough for you to make sense of them by just reading the code. They are events that Transport generates and can be handled here.

Add Protocol
----------------------

We now add the last bit to make our example complete, i.e. Protocol. We have subclassed Protocol to build separate Server and Client Protocols. The reason is that they both have different behavior. For example, the client has to send some data first while the server reads data first.

::

    import sys
    from twisted.python import log
    from twisted.internet import reactor
    from twisted.internet.protocol import ServerFactory, ClientFactory, Protocol


    class EchoServerProtocol(Protocol):
        def dataReceived(self, data):
            log.msg('Data received {}'.format(data))
            self.transport.write(data)

        def connectionMade(self):
            log.msg('Client connection from {}'.format(self.transport.getPeer()))

        def connectionLost(self, reason):
            log.msg('Lost connection because {}'.format(reason))


    class EchoClientProtocol(Protocol):
        def dataReceived(self, data):
            log.msg('Data received {}'.format(data))
            self.transport.loseConnection()

        def connectionMade(self):
            data = 'Hello, Server!'
            self.transport.write(data.encode())
            log.msg('Data sent {}'.format(data))

        def connectionLost(self, reason):
            log.msg('Lost connection because {}'.format(reason))


    class EchoServerFactory(ServerFactory):
        def buildProtocol(self, addr):
            return EchoServerProtocol()


    class EchoClientFactory(ClientFactory):
        def startedConnecting(self, connector):
            log.msg('Started to connect.')

        def buildProtocol(self, addr):
            log.msg('Connected.')
            return EchoClientProtocol()

        def clientConnectionLost(self, connector, reason):
            log.msg('Lost connection. Reason: {}'.format(reason))

        def clientConnectionFailed(self, connector, reason):
            log.msg('Lost failed. Reason: {}'.format(reason))


    def main():
        log.startLogging(sys.stdout)
        log.msg('Start your engines...')
        reactor.listenTCP(16000, EchoServerFactory())
        reactor.connectTCP('127.0.0.1', 16000, EchoClientFactory())
        reactor.run()


    if __name__ == '__main__':
        main()

Server Protocol
+++++++++++++++++++++++++++++

We're interested in three methods here, `connectionMade`, `dataReceived`, and `connectionLost`. We don't do anything in the connectionMade method because this server expects to received data first.

::

    class EchoServerProtocol(Protocol):
        def connectionMade(self):
            log.msg('Client connection from {}'.format(self.transport.getPeer()))

        def dataReceived(self, data):
            log.msg('Data received {}'.format(data))
            self.transport.write(data)

        def connectionLost(self, reason):
            log.msg('Lost connection because {}'.format(reason))


Client Protocol
+++++++++++++++++++++++++++++

We're interested in three methods here, `connectionMade`, `dataReceived`, and `connectionLost`. We send data in the connectionMade method because this is an echo client. We also expect the server to echo back what we sent and once we do get a reply it's handled in the `dataReceived` method.

::

    class EchoClientProtocol(Protocol):
        def connectionMade(self):
            data = 'Hello, Server!'
            self.transport.write(data.encode())
            log.msg('Data sent {}'.format(data))

        def dataReceived(self, data):
            log.msg('Data received {}'.format(data))
            self.transport.loseConnection()

        def connectionLost(self, reason):
            log.msg('Lost connection because {}'.format(reason))

Additional Resources
---------------------------------

* `Everything You Always Wanted to Know about Twisted <http://mumak.net/stuff/twisted-intro.html>`_
* `Writing Clients <http://twistedmatrix.com/documents/current/core/howto/clients.html>`_
* `Introduction to Deferreds <http://twistedmatrix.com/documents/current/core/howto/defer-intro.html>`_
* `Twisted - listen to multiple ports for multiple processes with one reactor <http://stackoverflow.com/questions/1411281/twisted-listen-to-multiple-ports-for-multiple-processes-with-one-reactor>`_
* `python twisted agent timeout <http://stackoverflow.com/questions/14984969/python-twisted-agent-timeout>`_