Debugging server side REST calls with mitmproxy in Meteor

Introduction

Isomorphic javascript offers a great flexibility for developper letting them choose where the code will get executed. On the clients or on the servers. While the Chrome DevTools have evolved at an impressive pace allowing to easily debug and see what is going on with your client's REST calls, the state of server side debugging may be a little bit deceptive in this field. Hence, when you need to put data fetching within your server, checking the flows of requests / responses could be a bit cumbersome if you solely rely on transaction logs or node-inspector. You could use technics like packet sniffing using Wireshark but setting a proper environment when using TLS with your REST call is a huge configuration process.

This is where mitmproxy comes to the rescue. Its name says it all: Man In The Middle PROXY. This incredible tool relies on the same technics as the infamous security attack, this time, for a greater purpose. For this article, I'm using mitmproxy as a local proxy server allowing me to trace all outgoing and incoming transactions from my server side Meteor app. Note that it is not the only use case that mitmproxy offers. Actually, this tool is incredibly agile allowing you to debug your mobile apps with great ease. Let us put our white hat on.

Installation on OSX

Using homebrew for installing mitmproxy is a one command line call:

brew install mitmproxy

Note that this steps actually installs 2 tools mitmproxy and mitmdump as well as a set of certificates in ~/.mitmproxy. We only use the first tool in this article but the second one could be handy when you need a lightweight display of your transaction flow and an easy way to save them in a replayable log. Note also that if you also intend to use mitmproxy for debugging other applications like native client ones, you can accept the certificate ~/.mitmproxy/mitmproxy-ca-cert.pemcertificate in OSX that mitmproxy has created just for you. Just a caveat here. When you set your proxy settings in OSX, be sure to remove the local filters that Apple gently put for you if you want to see your native apps talking to your local servers.

Here's a little screenshot of a debugging session using mitmproxy:

Not that isomorphic

For REST calls, Meteor comes with its own isomorphic library: HTTP. On the client, it acts as basic AJAX calls, while on the server, it relies on the request module. Like all isomorphic libraries, this allows you to put your code logic in your clients or in your server with a nice and uniform API. But this comes at a price, it only covers the features that makes sense for both the clients and the server.

A server is not a browser. Indeed, while having cookies on the browser make sense for retaining states in your application for each user, cookies on the server starts to be a weird concept. Are these cookies used for the clients that you will serve or should the server could be seen as a client to another server? The same goes for the proxy settings on a server. Why would you need a proxy on a production server? Therefore, what is available on the client side of your app may not be available on the server side. 

Unfortunately, you could have use cases where the REST API that you are using within your server may need cookies (which is a bit weird when speaking of REpresentational State Transfer which should rely on state-less operations ☺).  Additionally, for debugging our flows, we also need the proxy capabilities that are offered out of the box by our browsers. This is where comes the nice package http-more from dandv. It exposes some already baked in feature of the request module for your server side REST calls.

You install it by replacing Meteor's default package like so:

meteor remove http
meteor add dandv:http-more

Proxyfy your REST calls in your server

Now that our toolbox is setup, it is time to orient our server so that it takes advantages of mitmproxy. This is achieved via the freshly exposed options in the HTTP library.

For orienting your request to the proxy, play nice with your TLS layer and setting a cookie jar:

HTTP.get url,
  proxy: 'http://127.0.0.1:8080'
  strictSSL: false
  jar: true
, (e, r) ->
  console.log 'GET sent through the proxy :-)'

Now, you should see all your flows of your server on mitmproxy.

Some nice mitmproxy shortcuts and moves

mitmproxy's documentation is clear and very helpful. But before let you dive in, I'm sharing some of my most used shortcuts:

• C to clear the list of flows.
• ↑ and ↓ for selecting a flow.
• ↵ for seeing the details of an exchange and q to go back to the list.
• ⇥ in detail of a flow to switch between the request and the response.

When you need to save a mitmproxy session for comparing it with other ones, you can use the save feature:

mitmproxy -w FILENAME

Once saved, you can reread it and modify it to only spare the relevant parts of your flows:

mitmproxy -nr FILENAME

Where:

• n prevents to launch a proxy server.
• r read the previous saved file.

For the editing your flows, use the following shortcuts:

• d for deleting flows.
• e for editing a request or a response.
• w then a for saving your all your modifications of the flows for the session.
• q then y for quitting edition mode.

Another nice move is mitmproxy's capabilities to replay a session. For instance, you can replay the client part of your spared session or the server part just by specifying -c or -s respectively. Very handy for debugging without assaulting the REST service that your implementing or using:

mitmproxy -c FILENAME