adding middleware docs

Author: Matthew Weppennaar

docs/index.rst

@@ -15,6 +15,7 @@ Welcome to vumi2's documentation!
    transports
    applications
    message_caches
+   middleware
 
 
 

docs/middleware.rst

@@ -0,0 +1,43 @@
+Middleware
+==========
+Middleware provides additional functionality that can be attached to any existing transport, 
+application or dispatcher worker. For example, middleware could log inbound and outbound messages, decode a unicoded message,
+store delivery reports in a database or modify a message.
+
+Attaching middleware to your worker is fairly straight forward. Just extend your YAML configuration file with lines like:Attaching middleware to your worker is fairly straight forward. Just extend your YAML configuration file with lines like:
+
+ .. sourcecode:: YAML
+    middlewares:
+    - class_path: vumi2.middlewares.unidecoder.Unidecoder
+        enable_for_connectors: ["mc_qa_expressway"]
+        outbound_enabled: true
+
+The middleware section contains a list of middleware items. 
+Each item contains a :py:data:`class_path` class_path which is the full Python path to the class implementing the middleware, 
+:py:data:`enable_for_connectors` enable_for_connectors this is the list of transports that the middleware will run on, 
+and :py:data:`{type}_enabled`  fields which describes which type of message the middleware is enabled on (inbound, outbound or event)
+
+Multiple layers of middleware may be specified as follows:
+
+.. sourcecode:: YAML
+    middlewares:
+    - class_path: vumi2.middlewares.unidecoder.Unidecoder
+        enable_for_connectors: ["mc_qa_expressway"]
+        outbound_enabled: true
+    - class_path: vumi2.middlewares.logging.LoggingMiddlewarer
+        enable_for_connectors: ["mc_qa_expressway"]
+        outbound_enabled: true
+        inbound_enabled: true
+        event_enabled: true
+
+You can think of the layers of middleware sitting on top of the underlying transport or application worker. 
+Messages being consumed by the worker enter from the top and are processed by the middleware in the order you 
+have defined them and eventually reach the worker at the bottom. Messages published by the worker start at the 
+bottom and travel up through the layers of middleware before finally exiting the middleware at the top.
+
+Adding your middleware:
+
+.. toctree::
+    :maxdepth: 2
+
+    middleware/building_your_own_middleware

docs/middleware/building_your_own_middleware.rst

@@ -0,0 +1,102 @@
+Implementing your own middleware
+==========
+
+A middleware class provides three handler functions, one for processing each of the three kinds of messages transports, applications and dispatchers typically send and receive (i.e. inbound user messages, outbound user messages, event messages) and three enabled functions for each of the types of messages
+
+Although transport and application middleware potentially both provide the same sets of handlers, the two make use of them in slightly different ways. Inbound messages and events are published by transports but consumed by applications while outbound messages are opposite.
+
+Middleware is required to have the same interface as the BaseMiddleware class which is described  below.
+
+BaseMiddleware
+-----------------
+
+``class vumi2.middlewares.BaseMiddleware(name, config)``
+
+Common middleware base class.
+
+This is a convenient definition of and set of common functionality for middleware classes. You need not subclass this and should not instantiate this directly.
+
+The ``__init__()`` method should take exactly the following options so that your class can be instantiated from configuration in a standard way:
+
+Parameters: 
+-----------------
+.. py:data:: config
+   :type: dict[str]
+    Dictionary of configuraiton items.
+
+If you are subclassing this class, you should not override ``__init__()``. Custom setup should be done in 
+``setup()`` instead. The config class can be overidden by replacing the ``config`` variable.
+
+:py:data:`setup()`
+
+Any custom setup may be done here.
+
+Return type:	Deferred or None
+Returns:	May return a deferred that is called when setup is complete.
+
+``teardown()``
+“Any custom teardown may be done here
+
+Return type:	Deferred or None
+Returns:	May return a Deferred that is called when teardown is complete
+
+For each of inbound, outbound, and event message types, basemiddleware class implements two methods:
+
+``{type}_enabled(self, connector_name)``: 
+Checks the configuration to determine if the middleware is enabled for this message type and connector.
+
+``handle_{type}(self, msg)``:
+Processes the message before passing it to the next handler.
+
+Example of a simple middleware implementation from ``vumi2.middlewares.logging:``
+
+.. code-block:: python
+    from logging import getLevelNamesMapping, getLogger
+
+    from attr import define
+
+    from vumi2.middlewares.base import BaseMiddleware, BaseMiddlewareConfig
+
+
+    @define
+    class LoggingMiddlewareConfig(BaseMiddlewareConfig):
+        log_level: str = "info"
+        logger_name: str = __name__
+
+
+    class LoggingMiddleware(BaseMiddleware):
+        config: LoggingMiddlewareConfig
+
+        async def setup(self):
+            self.log_level = getLevelNamesMapping()[self.config.log_level.upper()]
+            self.logger = getLogger(self.config.logger_name)
+
+        def _log_msg(self, direction, msg, connector_name):
+            self.logger.log(
+                self.log_level,
+                f"Processed {direction} message for {connector_name}: {msg}",
+            )
+            return msg
+
+        async def handle_inbound(self, message, connector_name):
+            return self._log_msg("inbound", message, connector_name)
+
+        async def handle_outbound(self, message, connector_name):
+            return self._log_msg("outbound", message, connector_name)
+
+        async def handle_event(self, event, connector_name):
+            return self._log_msg("event", event, connector_name)
+
+How your middleware is used inside Vumi: 
+-----------------
+
+While writing complex middleware, it may help to understand how a middleware class is used by Vumi transports and applications.
+
+When a transport or application is started a list of middleware to load is read from the configuration. 
+An instance of each piece of middleware is created and then ``setup()`` is called on each middleware object in 
+order within the ``setup()`` of the worker
+
+``middleware_{type}_handler function``  (e.g., middleware_outbound_handler) of BaseWorker of each message type. This function will:
+Filter the middleware list based on connector_name and the middleware's ``{type}_enabled`` method.
+Create a decorated handler function that sequentially applies each enabled middleware's ``handle_{type}`` method to the message.
+Return the decorated handler. This decorated handler is then used in setting up the connection