Back to BlogJAY KYE

Building Production-Grade Go API: Project Setup and Database Architecture

Jay Kye
11 min read

Setting up a production-ready Go/Gin API server with PostgreSQL, proper project structure, and development workflow. From zero to deployment-ready backend infrastructure.

GoGinPostgreSQLBackend APIProject StructureProduction

Building Production-Grade Go API: Project Setup and Database Architecture

After building REST APIs with Node.js and exploring Go fundamentals, I decided to tackle a more ambitious project: creating a production-grade Go API server with proper architecture, database migrations, and development workflow. This isn't just about making it work—it's about building it right from the start.

Project Goals and Philosophy

Unlike my previous Go REST API experiment, this project focuses on:

  • Production-ready architecture following Go best practices
  • Proper project structure for maintainability and scalability
  • Database migration system for version control
  • Development workflow with hot reloading and automation
  • Cloud-native design ready for deployment

The complete project is available on GitHub: my-go-next-todo

Project Structure: Following Go Standards

The Standard Go Project Layout

my-go-next-todo/
├── cmd/
│   └── server/
│       └── main.go              # Application entry point
├── internal/                    # Private packages (not importable)
│   ├── config/
│   │   └── config.go           # Configuration management
│   ├── database/
│   │   └── database.go         # Database connection management
│   └── server/
│       └── server.go           # HTTP server setup
├── migrations/                  # Database migrations
│   ├── 000001_create_users_table.up.sql
│   ├── 000001_create_users_table.down.sql
│   ├── 000002_create_todos_table.up.sql
│   └── 000002_create_todos_table.down.sql
├── pkg/                        # Public libraries (future expansion)
├── .env.example               # Environment template
├── air.toml                   # Hot reload configuration
├── Makefile                   # Development commands
├── go.mod                     # Go module definition
└── README.md                  # Project documentation

Why This Structure Matters

cmd/: Application entry points. Each subdirectory represents a different binary.

internal/: Private packages that cannot be imported by external projects. This enforces encapsulation and prevents internal APIs from being used elsewhere.

pkg/: Public libraries that can be imported by external applications (when needed).

migrations/: Database schema changes tracked in version control.

Development Environment Setup

Essential Tools Installation

# Hot reload for development
go install github.com/cosmtrek/air@latest

# Database migration tool
go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest

Project Initialization

# Initialize Go module
go mod init github.com/jayk0001/my-go-next-todo

# Create directory structure
mkdir -p cmd/server internal/{config,database,server} migrations pkg

Dependencies and Technology Stack

Core Dependencies

// go.mod
module github.com/jayk0001/my-go-next-todo

go 1.21

require (
    github.com/gin-gonic/gin v1.9.1           // HTTP framework
    github.com/jackc/pgx/v5 v5.4.3            // PostgreSQL driver
    github.com/joho/godotenv v1.4.0           // Environment loader
    github.com/golang-migrate/migrate/v4 v4.16.2  // Migration tool
)

Technology Choices

Gin Framework: Fast HTTP router with middleware support, similar to Express.js but with Go's performance benefits.

PGX Driver: Pure Go PostgreSQL driver with excellent performance and feature support.

Neon Database: Serverless PostgreSQL for development and production scalability.

golang-migrate: Industry-standard database migration tool with up/down migration support.

Database Design and Architecture

Users Table Schema

-- migrations/000001_create_users_table.up.sql
CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    email VARCHAR(255) UNIQUE NOT NULL,
    password_hash VARCHAR(255) NOT NULL,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

-- Add index for email lookups
CREATE INDEX idx_users_email ON users(email);

Todos Table Schema

-- migrations/000002_create_todos_table.up.sql
CREATE TABLE todos (
    id SERIAL PRIMARY KEY,
    user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    title VARCHAR(500) NOT NULL,
    description TEXT,
    completed BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

-- Indexes for efficient queries
CREATE INDEX idx_todos_user_id ON todos(user_id);
CREATE INDEX idx_todos_completed ON todos(completed);
CREATE INDEX idx_todos_created_at ON todos(created_at);

Migration Strategy

Each migration consists of two files:

  • .up.sql: Forward migration (create/modify)
  • .down.sql: Reverse migration (rollback changes)
-- migrations/000002_create_todos_table.down.sql
DROP INDEX IF EXISTS idx_todos_created_at;
DROP INDEX IF EXISTS idx_todos_completed;
DROP INDEX IF EXISTS idx_todos_user_id;
DROP TABLE IF EXISTS todos;

Core Implementation

Configuration Management

// internal/config/config.go
package config

import (
    "os"
    "strconv"
    
    "github.com/joho/godotenv"
)

type Config struct {
    DatabaseURL string
    Port        string
    JWTSecret   string
    Environment string
}

func Load() (*Config, error) {
    // Load .env file in development
    if os.Getenv("ENVIRONMENT") != "production" {
        godotenv.Load()
    }
    
    return &Config{
        DatabaseURL: getEnv("DATABASE_URL", ""),
        Port:        getEnv("PORT", "8080"),
        JWTSecret:   getEnv("JWT_SECRET", ""),
        Environment: getEnv("ENVIRONMENT", "development"),
    }, nil
}

func getEnv(key, defaultValue string) string {
    if value := os.Getenv(key); value != "" {
        return value
    }
    return defaultValue
}

Database Connection Management

// internal/database/database.go
package database

import (
    "context"
    "fmt"
    "time"
    
    "github.com/jackc/pgx/v5/pgxpool"
)

type DB struct {
    Pool *pgxpool.Pool
}

func New(databaseURL string) (*DB, error) {
    config, err := pgxpool.ParseConfig(databaseURL)
    if err != nil {
        return nil, fmt.Errorf("failed to parse database URL: %w", err)
    }
    
    // Configure connection pool
    config.MaxConns = 30
    config.MinConns = 5
    config.MaxConnLifetime = time.Hour
    config.MaxConnIdleTime = time.Minute * 30
    
    pool, err := pgxpool.NewWithConfig(context.Background(), config)
    if err != nil {
        return nil, fmt.Errorf("failed to create connection pool: %w", err)
    }
    
    // Test connection
    if err := pool.Ping(context.Background()); err != nil {
        return nil, fmt.Errorf("failed to ping database: %w", err)
    }
    
    return &DB{Pool: pool}, nil
}

func (db *DB) Health(ctx context.Context) error {
    return db.Pool.Ping(ctx)
}

func (db *DB) Close() {
    db.Pool.Close()
}

HTTP Server Setup

// internal/server/server.go
package server

import (
    "context"
    "fmt"
    "net/http"
    "time"
    
    "github.com/gin-gonic/gin"
    "your-project/internal/database"
)

type Server struct {
    router *gin.Engine
    db     *database.DB
}

func New(db *database.DB) *Server {
    // Set Gin mode based on environment
    if gin.Mode() == gin.ReleaseMode {
        gin.SetMode(gin.ReleaseMode)
    }
    
    router := gin.Default()
    
    server := &Server{
        router: router,
        db:     db,
    }
    
    server.setupRoutes()
    return server
}

func (s *Server) setupRoutes() {
    // Health check endpoints (Kubernetes compatible)
    s.router.GET("/health", s.healthHandler)
    s.router.GET("/health/ready", s.readinessHandler)
    s.router.GET("/health/live", s.livenessHandler)
    
    // API routes
    api := s.router.Group("/api/v1")
    {
        api.GET("/ping", s.pingHandler)
    }
}

func (s *Server) healthHandler(c *gin.Context) {
    ctx, cancel := context.WithTimeout(c.Request.Context(), 5*time.Second)
    defer cancel()
    
    dbStatus := "healthy"
    if err := s.db.Health(ctx); err != nil {
        dbStatus = "unhealthy"
    }
    
    c.JSON(http.StatusOK, gin.H{
        "status":    "healthy",
        "timestamp": time.Now().Format(time.RFC3339),
        "version":   "0.1.0",
        "services": gin.H{
            "database": dbStatus,
        },
    })
}

func (s *Server) readinessHandler(c *gin.Context) {
    ctx, cancel := context.WithTimeout(c.Request.Context(), 5*time.Second)
    defer cancel()
    
    if err := s.db.Health(ctx); err != nil {
        c.JSON(http.StatusServiceUnavailable, gin.H{
            "status": "not ready",
            "error":  "database connection failed",
        })
        return
    }
    
    c.JSON(http.StatusOK, gin.H{"status": "ready"})
}

func (s *Server) livenessHandler(c *gin.Context) {
    c.JSON(http.StatusOK, gin.H{"status": "alive"})
}

func (s *Server) pingHandler(c *gin.Context) {
    c.JSON(http.StatusOK, gin.H{"message": "pong"})
}

func (s *Server) Start(port string) error {
    return s.router.Run(":" + port)
}

Application Entry Point

// cmd/server/main.go
package main

import (
    "context"
    "log"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"
    
    "your-project/internal/config"
    "your-project/internal/database"
    "your-project/internal/server"
)

func main() {
    // Load configuration
    cfg, err := config.Load()
    if err != nil {
        log.Fatal("Failed to load config:", err)
    }
    
    // Connect to database
    db, err := database.New(cfg.DatabaseURL)
    if err != nil {
        log.Fatal("Failed to connect to database:", err)
    }
    defer db.Close()
    
    // Create server
    srv := server.New(db)
    
    // Setup graceful shutdown
    httpServer := &http.Server{
        Addr:    ":" + cfg.Port,
        Handler: srv,
    }
    
    // Start server in goroutine
    go func() {
        log.Printf("Server starting on port %s", cfg.Port)
        if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Fatal("Server failed to start:", err)
        }
    }()
    
    // Wait for interrupt signal
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
    <-quit
    
    log.Println("Server shutting down...")
    
    // Graceful shutdown with timeout
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    
    if err := httpServer.Shutdown(ctx); err != nil {
        log.Fatal("Server forced to shutdown:", err)
    }
    
    log.Println("Server exited")
}

Development Workflow Automation

Makefile for Common Tasks

# Makefile
.PHONY: help dev build test clean db-up db-down db-reset

# Default target
help:
	@echo "Available commands:"
	@echo "  dev      - Start development server with hot reload"
	@echo "  build    - Build production binary"
	@echo "  test     - Run tests"
	@echo "  db-up    - Run database migrations"
	@echo "  db-down  - Rollback database migrations"
	@echo "  db-reset - Reset database (down then up)"

# Development server with hot reload
dev:
	air

# Build production binary
build:
	CGO_ENABLED=0 GOOS=linux go build -o bin/server ./cmd/server

# Run tests
test:
	go test -v ./...

# Database migrations
db-up:
	migrate -path migrations -database "$(DATABASE_URL)" up

db-down:
	migrate -path migrations -database "$(DATABASE_URL)" down

db-reset: db-down db-up

# Clean build artifacts
clean:
	rm -rf bin/ tmp/

Air Hot Reload Configuration

# air.toml
root = "."
testdata_dir = "testdata"
tmp_dir = "tmp"

[build]
  args_bin = []
  bin = "./tmp/main"
  cmd = "go build -o ./tmp/main ./cmd/server"
  delay = 1000
  exclude_dir = ["assets", "tmp", "vendor", "testdata"]
  exclude_file = []
  exclude_regex = ["_test.go"]
  exclude_unchanged = false
  follow_symlink = false
  full_bin = ""
  include_dir = ["cmd", "internal"]
  include_ext = ["go", "tpl", "tmpl", "html"]
  kill_delay = "0s"
  log = "build-errors.log"
  send_interrupt = false
  stop_on_root = false

[color]
  app = ""
  build = "yellow"
  main = "magenta"
  runner = "green"
  watcher = "cyan"

[log]
  time = false

[misc]
  clean_on_exit = false

API Endpoints and Testing

Health Check Endpoints

The server provides comprehensive health check endpoints compatible with Kubernetes:

# Overall health status
curl http://localhost:8080/health
{
  "status": "healthy",
  "timestamp": "2025-10-25T15:24:27Z",
  "version": "0.1.0",
  "services": {
    "database": "healthy"
  }
}

# Readiness probe (ready to serve traffic)
curl http://localhost:8080/health/ready
{"status": "ready"}

# Liveness probe (application is running)
curl http://localhost:8080/health/live
{"status": "alive"}

# Basic connectivity test
curl http://localhost:8080/api/v1/ping
{"message": "pong"}

Environment Configuration

Environment Variables Setup

# .env.example
DATABASE_URL=postgresql://username:password@host:5432/database?sslmode=require
JWT_SECRET=your-super-secret-32-characters-minimum-key
PORT=8080
ENVIRONMENT=development

Neon Database Integration

For this project, I used Neon as the PostgreSQL provider:

# Neon connection string format
DATABASE_URL=postgresql://username:password@ep-example.us-east-2.aws.neon.tech/neondb?sslmode=require

Benefits of Neon:

  • Serverless: Automatic scaling and hibernation
  • Branching: Database branches for different environments
  • Built-in SSL: Secure connections by default

Running the Project

Step 1: Environment Setup

# Clone and setup
git clone https://github.com/jayk0001/my-go-next-todo
cd my-go-next-todo

# Copy environment template
cp .env.example .env

# Edit .env with your database credentials

Step 2: Database Migration

# Run migrations
make db-up

# Verify migration status
migrate -path migrations -database "$DATABASE_URL" version

Step 3: Start Development Server

# Start with hot reload
make dev

# Test the server
curl http://localhost:8080/health

Key Learning Points

Go Best Practices Applied

Standard Project Layout: Following the community-accepted structure improves maintainability and makes the project familiar to other Go developers.

Dependency Injection: All dependencies are injected through constructors, making the code testable and modular.

Context Propagation: Every function that might perform I/O accepts a context.Context for cancellation and timeout handling.

Error Handling: Explicit error handling at every level with proper error wrapping using fmt.Errorf.

Database Management Insights

Migration Versioning: Each migration has a version number and both up/down scripts, enabling safe rollbacks.

Connection Pooling: Proper pool configuration prevents connection exhaustion and improves performance.

Index Strategy: Strategic indexing on frequently queried columns improves performance without over-indexing.

Production Readiness Features

Health Checks: Multiple health check endpoints for different monitoring needs (Kubernetes, load balancers).

Graceful Shutdown: Proper signal handling ensures in-flight requests complete before shutdown.

Environment-based Configuration: Twelve-factor app compliance with environment variable configuration.

SSL/TLS Ready: Database connections use SSL by default for security.

Troubleshooting Common Issues

Migration Failures

# Check current migration version
migrate -path migrations -database "$DATABASE_URL" version

# Force to specific version if stuck
migrate -path migrations -database "$DATABASE_URL" force 1

# Then run migrations again
make db-up

Connection Pool Issues

# Check active connections
SELECT count(*) FROM pg_stat_activity WHERE datname = 'your_database';

# Adjust pool settings in database.go if needed
config.MaxConns = 10  // Reduce if hitting connection limits

Hot Reload Not Working

# Ensure air.toml includes correct directories
include_dir = ["cmd", "internal"]
include_ext = ["go"]

Performance Considerations

Database Optimization

  • Connection Pooling: Configured for cloud database latency
  • Index Strategy: Covering indexes for common query patterns
  • Query Timeout: All database operations have timeouts

Server Optimization

  • Gin Release Mode: Production builds use optimized Gin mode
  • Graceful Shutdown: Prevents connection drops during deployment
  • Health Check Caching: Avoid overwhelming database with health checks

Next Steps and Future Enhancements

This foundation sets up the infrastructure for a full-featured TODO application. The next phases will include:

  1. JWT Authentication System: User registration, login, and token management
  2. TODO CRUD Operations: Complete REST API for todo management
  3. Middleware Architecture: Request logging, rate limiting, and validation
  4. Error Handling Framework: Structured error responses and logging
  5. Testing Strategy: Unit tests, integration tests, and API testing
  6. Docker Containerization: Production deployment preparation

Conclusion

Building a production-grade Go API requires more than just making the code work. This project demonstrates:

Proper Architecture: Following Go conventions and best practices from the start prevents technical debt.

Development Workflow: Automation and hot reloading improve developer productivity significantly.

Database Management: Proper migration strategy and connection management are crucial for long-term maintainability.

Production Readiness: Health checks, graceful shutdown, and environment-based configuration prepare the application for real-world deployment.

The complete implementation is available on GitHub, and the next post will cover implementing JWT authentication and user management on top of this foundation.

Key Takeaway: Taking time to set up proper infrastructure pays dividends throughout the project lifecycle. The extra effort in project structure and tooling makes future development much more efficient and maintainable.

References:

This post documents the actual development process of building a production-ready Go API server. All code examples are from the working implementation available on GitHub.