Generating API Documentation from Live Traffic

Hello everyone! Last week, I attended a special session at the company I work for. In short, I had to pair up with a colleague from another team to work on a Mini Project. It was a great experience, and I learned a lot about how full-stack developers work. On top of that, I explored some new tools and technologies. In this post, I’ll share my experience and how you can create API documentation efficiently.

Why API Documentation

API documentation is crucial for developers to understand how to use APIs effectively. It provides clear instructions on endpoints, request/response formats, and error handling. Good documentation can reduce the learning curve for new developers and improve collaboration between teams.

Common Challenges in API Development

When developing APIs, teams often face challenges such as Time-Consuming, Inconsistent Documentation, and Lack of Clarity. However, these challenges can be mitigated with the right tools and practices. There are several ways to write API documentation:

• Manual Documentation: Writing documentation by hand can be time-consuming and prone to errors.
• Plugins integration: Some frameworks offer plugins to generate documentation.
• Annotated Code: Using annotations or comments in the code to generate documentation can be effective but requires consistent coding practices. If the codebase is old or you lack permission to modify it, these approaches may not be feasible.

Introducing DocuRift

DocuRift is a tool that automatically generates and maintains REST API documentation by acting as a proxy between your clients and API server. It captures and analyzes real API traffic to create comprehensive, up-to-date documentation. It’s ideal for existing services that lack documentation.

Key Features

• Generates Swagger/OpenAPI specs directly.
• Exports a Postman Collection.

  sequenceDiagram
    participant Client
    participant DocuRift
    participant Backend
    participant Analyzer
    Client->>DocuRift: HTTP Request
    DocuRift->>Backend: Forward Request
    Backend->>DocuRift: HTTP Response
    DocuRift->>Analyzer: Process Traffic

Demo

In this section, I will demonstrate how to set up DocuRift and generate API documentation from live traffic data. first, you need to install DocuRift:

go install github.com/tienanr/docurift/cmd/docurift@latest

Then create a config file for DocuRift. Here’s a basic example of a config.yaml configuration file:

proxy:
    port: 9876
    backend-url: http://localhost:8080

analyzer:
    port: 9877
    max-examples: 20
    redacted-fields:
        - password
    storage:
        path: .
        frequency: 10

Next, you can set up a simple Go backend to test DocuRift. Here’s a basic example:

package main

import (
    "encoding/json"
    "net/http"
)

type User struct {
    ID   int    `json:"id"`
    Name string `json:"name"`
}

func getUser(w http.ResponseWriter, r *http.Request) {
    user := User{ID: 1, Name: "John Doe"}
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(user)
}

func createUser(w http.ResponseWriter, r *http.Request) {
    var user User
    json.NewDecoder(r.Body).Decode(&user)
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(user)
}

func main() {
    http.HandleFunc("/user", func(w http.ResponseWriter, r *http.Request) {
        switch r.Method {
        case http.MethodGet:
            getUser(w, r)
        case http.MethodPost:
            createUser(w, r)
        default:
            http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
        }
    })
    http.ListenAndServe(":8080", nil)
}

Now, you can run your Go server on port :8080:

go run main.go

Once your server is running, you can start DocuRift to capture the traffic:

docurift -config config.yaml

Now, you can make requests to your API using a tool like Postman or cURL.

curl -X GET http://localhost:9876/user

curl -X POST http://localhost:9876/user -H "Content-Type: application/json
" -d '{"id": 1, "name": "John Doe"}'

After making some requests, you can access the analyzer at http://localhost:9877 to view the captured traffic and generated documentation.

Here’s a screenshot of the DocuRift dashboard showing the captured traffic and generated documentation

Swagger UI

Additionally, you can export ready-to-use Postman collections via http://localhost:9877/api/postman.json.

Conclusion

Writing API documentation can be a daunting task, especially for existing services without proper documentation. There are several ways to solve this problem, DocuRift is one of tools that can help you generate API documentation from live traffic data. Perhaps you can try it out in your next project.