A small HTTP debug server in Go

 August 31, 2018

Lately, I found myself to work on an application that was communicating via SOAP with a server. My goal was to understand how this application worked with the SOAP server to emulate its behavior. Even if I had access to the source code of the application, I thought it would have been easier, faster and more fun to do the work without actually reading the code. It’s important to note that actually, the application is fairly small and self-contained. Otherwise, I would have probably taken a different approach.

Since I was not very interested in the application itself, but more to the SOAP API, I decided to handle the whole situation as a reverse-engineering effort. One nice thing about this application, like many others, is that it’s possible to set the server URL with a command line configuration.

So, I decided to create a small server which would expose an HTTP service, accept a request, log the request, forward the request to the server, log the answer and finally forward the server’s response to the client.

I know this is also possible with proxies, but proxies are overkilling for this kind of things. Today having HTTPS traffic is the default, so a proxy has to have an SSL certificate for the real domain that is valid for the client. Also, you have to tweak system-level configurations, which was not a thing I was looking for doing. That’s why I took this approach.

This occasion was also an excellent way to play a little bit more with Go, a language that I’m enjoying lately.

My first approach was to log in the terminal requests and responses, but that became very messy very quickly, so I moved to log in files.

So, the first thing we need is an HTTP server that would forward all requests to a function.

func main() {
    http.HandleFunc("/", handler)
    if err := http.ListenAndServe(":8888", nil); err != nil {
        panic(err)
    }
}

func handler(w http.ResponseWriter, r *http.Request) {}

This simple main method would do precisely that. Now Go would create an HTTP server on port 8888 and use the handler function for any endpoint.

As you can imagine, right now the handler function is not doing anything (so the server will reply with an empty response to any request). We now need to populate the handler function.

The first thing that I want to do is designate a prefix for the two files (request and response) that every HTTP call will create.

To make it easier to find them afterword, I’m going to name the files with the time (with the format YYYYMMDD-HHmmSS), the HTTP method, and then the fact that is a request or response.

So first thing, let’s create a file prefix with the constant part between the two files (time and HTTP method)

filePrefix := fmt.Sprintf("%s_%s", time.Now().Format("20060102-150405"), r.Method)

Now that we have defined the file prefix, we need to dump the HTTP request. Thanks to the impressive Go standard library, it provides a simple way to do so in the httputil package. The httputil.DumpRequest method returns an []byte and an error. After having checked the error, we can proceed writing the byte sequence to a file.

dump, err := httputil.DumpRequest(r, true)
if err != nil {
    fmt.Println(err)
}
if err := ioutil.WriteFile(filePrefix+"_request.dump", dump, 0644); err != nil {
    fmt.Println(err)
}

Now that we saved the request the client created, we can create a new HTTP request to the real endpoint so that we obtain the answer to feed to the client. Our new HTTP request will be a clone of the one we received, with just a different URL. We are going to put in the realUrl variable the name of the endpoint we are masking, and the http.NewRequest will do the dirty work for us: After we create the request, we also fire it.

As you will notice, we are also going to copy the HTTP headers from the original request, since very often the server will rely (as in the SOAP case) on specific HTTP headers to decide how to handle the request.

nr, err := http.NewRequest(r.Method, realUrl+r.URL.String(), r.Body)
if err != nil {
    fmt.Println(err)
}
nr.Header = r.Header
response, err := http.DefaultClient.Do(nr)
if err != nil {
    fmt.Println(err)
}

Now that we hold the real server response, we can carry on saving it:

responseDump, err := httputil.DumpResponse(response, true)
if err != nil {
    fmt.Println(err)
}
if err := ioutil.WriteFile(filePrefix+"_response.dump", responseDump, 0644); err != nil {
    fmt.Println(err)
}

The only thing that we still need to do is reply to the client the real server response:

io.Copy(w, response.Body)

So, putting all together:

package main

import (
    "fmt"
    "io"
    "io/ioutil"
    "net/http"
    "net/http/httputil"
    "time"
)

var realUrl = "https://real-url.tld"

func main() {
    http.HandleFunc("/", handler)
    if err := http.ListenAndServe(":8888", nil); err != nil {
        panic(err)
    }
}

func handler(w http.ResponseWriter, r *http.Request) {
    filePrefix := fmt.Sprintf("%s_%s", time.Now().Format("20060102-150405"), r.Method)

    // Log request
    dump, err := httputil.DumpRequest(r, true)
    if err != nil {
        fmt.Println(err)
    }
    if err := ioutil.WriteFile(filePrefix+"_request.dump", dump, 0644); err != nil {
        fmt.Println(err)
    }

    // Redirect request
    nr, err := http.NewRequest(r.Method, realUrl+r.URL.String(), r.Body)
    if err != nil {
        fmt.Println(err)
    }
    nr.Header = r.Header
    response, err := http.DefaultClient.Do(nr)
    if err != nil {
        fmt.Println(err)
    }

    // Log answer
    responseDump, err := httputil.DumpResponse(response, true)
    if err != nil {
        fmt.Println(err)
    }
    if err := ioutil.WriteFile(filePrefix+"_response.dump", responseDump, 0644); err != nil {
        fmt.Println(err)
    }

    // Redirect answer
    io.Copy(w, response.Body)
}

All in all, in just 55 lines of code (with comments and empty lines in the count!), we manage to perform a kind-of man-in-the-middle attack and log all the passing traffic.

This small piece of code was a fun thing to do, and I hope it will also help some people trying to do similar things.