Development Setup

This guide provides comprehensive instructions for setting up a development environment for nginx-torblocker, including building, testing, debugging, and contributing to the project.

Overview

The nginx-torblocker development environment includes building from source, setting up test environments, debugging tools, and development workflows for creating custom features or fixing bugs.

Prerequisites

System Requirements

Ubuntu/Debian:

# Essential build tools
sudo apt update
sudo apt install -y build-essential git wget curl

# Nginx development dependencies
sudo apt install -y libpcre3-dev libssl-dev zlib1g-dev

# Development tools
sudo apt install -y gdb valgrind strace
sudo apt install -y nginx-dev  # If using packaged Nginx

# Optional: Testing tools
sudo apt install -y python3-pytest python3-requests
sudo apt install -y docker.io docker-compose  # For containerized testing

CentOS/RHEL:

# Essential build tools
sudo yum groupinstall -y "Development Tools"
sudo yum install -y git wget curl

# Nginx development dependencies  
sudo yum install -y pcre-devel openssl-devel zlib-devel

# Development tools
sudo yum install -y gdb valgrind strace
sudo yum install -y nginx-devel  # If available

# EPEL for additional tools
sudo yum install -y epel-release
sudo yum install -y python3-pip
pip3 install pytest requests

macOS:

# Install Xcode command line tools
xcode-select --install

# Install Homebrew if not present
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Install dependencies
brew install pcre openssl wget

# Development tools
brew install gdb lldb  # Note: gdb requires additional setup on macOS

# Optional: Testing tools
brew install python3
pip3 install pytest requests

Environment Setup

Create development directory structure:

# Create development workspace
mkdir -p ~/nginx-dev
cd ~/nginx-dev

# Clone the nginx-torblocker repository
git clone https://github.com/yourusername/nginx-torblocker.git
cd nginx-torblocker

# Create development branch
git checkout -b feature/development-setup

# Set up development environment variables
export NGINX_DEV_ROOT=~/nginx-dev
export TORBLOCK_SRC=$PWD/src
export NGINX_SRC=$NGINX_DEV_ROOT/nginx-source

Development Workflow

1. Source Code Organization

Understanding the codebase structure:

src/
├── config                              # Module configuration file
├── ngx_http_torblocker_module.c        # Main module implementation
├── ngx_http_torblocker_module.h        # Header file with declarations
└── README.dev.md                       # Development notes

Development workflow files:
.development/                           # Private development scripts
├── build.sh                          # Build automation
├── test.sh                           # Testing scripts
├── debug.sh                          # Debug helpers
└── setup-env.sh                      # Environment setup

2. Setting Up Nginx Source

Download and prepare Nginx source for development:

#!/bin/bash
# setup-nginx-source.sh

NGINX_VERSION=${1:-1.26.0}
NGINX_SRC_DIR="$NGINX_DEV_ROOT/nginx-source"

echo "Setting up Nginx $NGINX_VERSION source for development..."

# Create source directory
mkdir -p "$NGINX_SRC_DIR"
cd "$NGINX_SRC_DIR"

# Download Nginx source
if [ ! -f "nginx-$NGINX_VERSION.tar.gz" ]; then
    echo "Downloading Nginx $NGINX_VERSION..."
    wget "https://nginx.org/download/nginx-$NGINX_VERSION.tar.gz"
fi

# Extract source
if [ ! -d "nginx-$NGINX_VERSION" ]; then
    echo "Extracting Nginx source..."
    tar -xzf "nginx-$NGINX_VERSION.tar.gz"
fi

# Create symlink for easier access
ln -sf "nginx-$NGINX_VERSION" current

echo "Nginx source ready at: $NGINX_SRC_DIR/current"

3. Build Configuration

Create development-specific build configuration:

#!/bin/bash
# .development/build.sh

set -e

NGINX_SRC="${NGINX_SRC:-$NGINX_DEV_ROOT/nginx-source/current}"
MODULE_SRC="${MODULE_SRC:-$PWD/src}"
BUILD_TYPE="${BUILD_TYPE:-debug}"

if [ ! -d "$NGINX_SRC" ]; then
    echo "Error: Nginx source not found at $NGINX_SRC"
    echo "Run setup-nginx-source.sh first"
    exit 1
fi

cd "$NGINX_SRC"

echo "Building nginx-torblocker module in $BUILD_TYPE mode..."

# Configure build based on type
case "$BUILD_TYPE" in
    "debug")
        CFLAGS="-g -O0 -DDEBUG -Wall -Wextra"
        DEBUG_OPTS="--with-debug"
        ;;
    "release")
        CFLAGS="-O2 -DNDEBUG"
        DEBUG_OPTS=""
        ;;
    "profile")
        CFLAGS="-g -O2 -pg -DPROFILE"
        DEBUG_OPTS="--with-debug"
        ;;
    *)
        echo "Unknown build type: $BUILD_TYPE"
        exit 1
        ;;
esac

# Configure Nginx with the module
./configure \
    --prefix=/usr/local/nginx-dev \
    --sbin-path=/usr/local/nginx-dev/sbin/nginx \
    --conf-path=/usr/local/nginx-dev/conf/nginx.conf \
    --pid-path=/usr/local/nginx-dev/logs/nginx.pid \
    --lock-path=/usr/local/nginx-dev/logs/nginx.lock \
    --error-log-path=/usr/local/nginx-dev/logs/error.log \
    --access-log-path=/usr/local/nginx-dev/logs/access.log \
    --with-http_ssl_module \
    --with-http_v2_module \
    --with-http_geoip_module \
    --add-dynamic-module="$MODULE_SRC" \
    $DEBUG_OPTS

# Build the module
make modules

# Copy module to development location
mkdir -p /usr/local/nginx-dev/modules
cp objs/ngx_http_torblocker_module.so /usr/local/nginx-dev/modules/

echo "Build complete! Module available at:"
echo "/usr/local/nginx-dev/modules/ngx_http_torblocker_module.so"

4. Development Configuration

Create development-specific Nginx configuration:

# .development/nginx-dev.conf

# Development configuration for nginx-torblocker
load_module modules/ngx_http_torblocker_module.so;

# Development-specific settings
daemon off;  # Run in foreground for debugging
master_process off;  # Single process for easier debugging
worker_processes 1;

# Enhanced error logging
error_log /usr/local/nginx-dev/logs/error.log debug;
pid /usr/local/nginx-dev/logs/nginx.pid;

events {
    worker_connections 1024;
    use epoll;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    
    # Development logging
    log_format dev '$remote_addr - $remote_user [$time_local] '
                  '"$request" $status $body_bytes_sent '
                  '"$http_referer" "$http_user_agent" '
                  'torblock_status="$torblock_status" '
                  'torblock_exit_node="$torblock_exit_node"';
    
    access_log /usr/local/nginx-dev/logs/access.log dev;
    
    # Torblock development settings
    torblock on;
    torblock_debug on;                 # Enable debug output
    torblock_cache_size 1m;           # Small cache for testing
    torblock_update_interval 60000;   # 1 minute for quick testing
    
    # Development server
    server {
        listen 8080;
        server_name localhost dev.local;
        
        # Debug information endpoint
        location /debug/torblock {
            access_log off;
            
            add_header Content-Type application/json always;
            return 200 '{
                "module_version": "$torblock_version",
                "client_ip": "$remote_addr",
                "tor_status": "$torblock_status",
                "exit_node": "$torblock_exit_node",
                "country": "$torblock_country",
                "asn": "$torblock_asn",
                "cache_status": "$torblock_cache_status",
                "last_update": "$torblock_last_update",
                "total_exits": "$torblock_total_exits"
            }';
        }
        
        # Test endpoints
        location /test/allow {
            torblock off;
            return 200 "Always allowed\n";
        }
        
        location /test/block {
            torblock on;
            torblock_action 403;
            return 200 "Should be blocked for Tor\n";
        }
        
        location /test/log {
            torblock on;
            torblock_action log;
            return 200 "Tor access logged\n";
        }
        
        # Main test area
        location / {
            root /usr/local/nginx-dev/html;
            index index.html;
            
            # Default torblock behavior
            torblock on;
        }
    }
}

Debugging Tools and Techniques

1. GDB Debugging

Set up GDB for debugging the module:

#!/bin/bash
# .development/debug.sh

NGINX_BINARY="/usr/local/nginx-dev/sbin/nginx"
CONFIG_FILE="/usr/local/nginx-dev/conf/nginx.conf"

echo "Starting Nginx debugging session..."

# Ensure nginx is not running
sudo pkill -f nginx-dev || true

# Start GDB session
gdb --args "$NGINX_BINARY" -c "$CONFIG_FILE" -g "daemon off;"

GDB commands for nginx-torblocker debugging:

# Set breakpoints in the module
break ngx_http_torblocker_handler
break ngx_http_torblocker_init_worker
break ngx_http_torblocker_update_exits

# Set watchpoints for variables
watch torblock_exit_list
watch torblock_cache_hits

# Useful commands during debugging
info locals
print *r
print ngx_cycle->log->level
bt  # backtrace

# Continue execution
continue

2. Logging and Tracing

Enhanced logging for development:

// Add to ngx_http_torblocker_module.c for debugging

#ifdef DEBUG
#define torblock_debug_log(level, log, fmt, ...) \
    ngx_log_error(level, log, 0, "TORBLOCK_DEBUG: " fmt, ##__VA_ARGS__)
#else
#define torblock_debug_log(level, log, fmt, ...)
#endif

// Example usage in handler function
static ngx_int_t
ngx_http_torblocker_handler(ngx_http_request_t *r)
{
    torblock_debug_log(NGX_LOG_DEBUG_HTTP, r->connection->log, 
                      "Processing request for %V", &r->uri);
    
    // Rest of handler implementation
}

3. Memory Debugging with Valgrind

Use Valgrind to detect memory issues:

#!/bin/bash
# .development/valgrind-test.sh

NGINX_BINARY="/usr/local/nginx-dev/sbin/nginx"
CONFIG_FILE="/usr/local/nginx-dev/conf/nginx.conf"

echo "Running Nginx with Valgrind..."

# Stop any running nginx instances
sudo pkill -f nginx-dev || true

# Run with Valgrind
valgrind \
    --tool=memcheck \
    --leak-check=full \
    --show-leak-kinds=all \
    --track-origins=yes \
    --verbose \
    --log-file=valgrind-output.log \
    "$NGINX_BINARY" -c "$CONFIG_FILE" -g "daemon off;"

4. Performance Profiling

Profile the module performance:

#!/bin/bash
# .development/profile.sh

echo "Building with profiling enabled..."
BUILD_TYPE=profile .development/build.sh

echo "Starting profiling run..."
# Run nginx with test load
.development/load-test.sh &
LOAD_PID=$!

# Let it run for specified time
sleep 60

# Stop load test
kill $LOAD_PID

# Generate profile report
gprof /usr/local/nginx-dev/sbin/nginx gmon.out > profile-report.txt

echo "Profile report generated: profile-report.txt"

Testing Framework

1. Unit Testing Setup

Create unit tests for module functions:

#!/usr/bin/env python3
# tests/test_torblock.py

import pytest
import requests
import json
import subprocess
import time

class TestTorblockModule:
    
    @classmethod
    def setup_class(cls):
        """Setup test environment"""
        cls.base_url = "http://localhost:8080"
        cls.start_nginx()
        time.sleep(1)  # Wait for nginx to start
    
    @classmethod
    def teardown_class(cls):
        """Cleanup test environment"""
        cls.stop_nginx()
    
    @classmethod
    def start_nginx(cls):
        """Start nginx for testing"""
        subprocess.run([
            "/usr/local/nginx-dev/sbin/nginx",
            "-c", "/usr/local/nginx-dev/conf/nginx.conf"
        ], check=True)
    
    @classmethod
    def stop_nginx(cls):
        """Stop nginx after testing"""
        subprocess.run(["pkill", "-f", "nginx-dev"], check=False)
    
    def test_module_loading(self):
        """Test that the module loads correctly"""
        response = requests.get(f"{self.base_url}/debug/torblock")
        assert response.status_code == 200
        
        data = response.json()
        assert "module_version" in data
        assert "tor_status" in data
    
    def test_tor_detection(self):
        """Test Tor IP detection"""
        # Test with known Tor exit node IP (mock)
        headers = {"X-Real-IP": "1.2.3.4"}  # Mock Tor IP
        response = requests.get(f"{self.base_url}/test/block", headers=headers)
        
        # Should be blocked if IP is in Tor list
        assert response.status_code in [200, 403]
    
    def test_allow_endpoint(self):
        """Test that allow endpoint works"""
        response = requests.get(f"{self.base_url}/test/allow")
        assert response.status_code == 200
        assert "Always allowed" in response.text
    
    def test_log_endpoint(self):
        """Test logging functionality"""
        response = requests.get(f"{self.base_url}/test/log")
        assert response.status_code == 200
        assert "Tor access logged" in response.text
    
    def test_configuration_variables(self):
        """Test that configuration variables are accessible"""
        response = requests.get(f"{self.base_url}/debug/torblock")
        data = response.json()
        
        required_vars = [
            "client_ip",
            "tor_status", 
            "cache_status",
            "total_exits"
        ]
        
        for var in required_vars:
            assert var in data

    def test_cache_functionality(self):
        """Test caching behavior"""
        # Make multiple requests to same endpoint
        responses = []
        for _ in range(5):
            response = requests.get(f"{self.base_url}/debug/torblock")
            responses.append(response.json())
        
        # Check cache hit/miss patterns
        cache_statuses = [r["cache_status"] for r in responses]
        assert len(set(cache_statuses)) >= 1  # Should have cache hits after first miss

if __name__ == "__main__":
    pytest.main([__file__, "-v"])

2. Integration Testing

Test integration with different Nginx configurations:

#!/bin/bash
# tests/integration-test.sh

set -e

echo "=== Integration Testing ==="

# Test different configurations
CONFIGS=(
    "basic"
    "advanced"
    "mixed-policies"
)

for config in "${CONFIGS[@]}"; do
    echo "Testing configuration: $config"
    
    # Copy configuration
    cp "tests/configs/$config.conf" "/usr/local/nginx-dev/conf/nginx.conf"
    
    # Restart nginx
    pkill -f nginx-dev || true
    /usr/local/nginx-dev/sbin/nginx -c /usr/local/nginx-dev/conf/nginx.conf
    
    # Run tests
    python3 tests/test_config_$config.py
    
    echo "✓ Configuration $config passed"
done

echo "=== All Integration Tests Passed ==="

3. Load Testing

Performance testing setup:

#!/usr/bin/env python3
# tests/load_test.py

import asyncio
import aiohttp
import time
import statistics

async def make_request(session, url, semaphore):
    """Make a single HTTP request"""
    async with semaphore:
        start_time = time.time()
        try:
            async with session.get(url) as response:
                await response.text()
                return time.time() - start_time, response.status
        except Exception as e:
            return time.time() - start_time, 0

async def load_test(url, concurrent_requests=50, total_requests=1000):
    """Run load test against the server"""
    
    semaphore = asyncio.Semaphore(concurrent_requests)
    
    async with aiohttp.ClientSession() as session:
        tasks = []
        
        for _ in range(total_requests):
            task = make_request(session, url, semaphore)
            tasks.append(task)
        
        results = await asyncio.gather(*tasks)
    
    # Analyze results
    response_times = [r[0] for r in results]
    status_codes = [r[1] for r in results]
    
    print(f"Load Test Results:")
    print(f"Total Requests: {total_requests}")
    print(f"Concurrent: {concurrent_requests}")
    print(f"Average Response Time: {statistics.mean(response_times):.3f}s")
    print(f"Median Response Time: {statistics.median(response_times):.3f}s")
    print(f"95th Percentile: {statistics.quantiles(response_times, n=20)[18]:.3f}s")
    print(f"Success Rate: {status_codes.count(200)/len(status_codes)*100:.1f}%")

if __name__ == "__main__":
    asyncio.run(load_test("http://localhost:8080/test/block"))

Development Best Practices

1. Code Style and Standards

Follow Nginx coding conventions:

// ngx_http_torblocker_module.c style guide

// Function naming: ngx_http_module_function_name
static ngx_int_t ngx_http_torblocker_handler(ngx_http_request_t *r);

// Variable naming: lower_case_with_underscores
static ngx_str_t torblock_exit_list_url;

// Constants: NGX_HTTP_TORBLOCK_CONSTANT_NAME
#define NGX_HTTP_TORBLOCK_DEFAULT_TIMEOUT 30000

// Error handling pattern
if (condition == NGX_ERROR) {
    ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                  "torblock: failed to process request");
    return NGX_HTTP_INTERNAL_SERVER_ERROR;
}

// Memory allocation pattern
void *ptr = ngx_palloc(r->pool, size);
if (ptr == NULL) {
    return NGX_ERROR;
}

2. Git Workflow for Development

Structured development workflow:

# Create feature branch
git checkout -b feature/new-torblock-feature

# Make incremental commits
git add src/ngx_http_torblocker_module.c
git commit -m "feat: add new Tor detection algorithm

- Implement enhanced exit node detection
- Add caching for improved performance
- Include unit tests for new functionality"

# Run tests before committing
.development/test.sh

# Push and create pull request
git push origin feature/new-torblock-feature

3. Documentation Guidelines

Maintain comprehensive documentation:

/**
 * Updates the Tor exit node list from remote source
 * 
 * @param cycle The nginx cycle object
 * @param url The URL to fetch the exit list from
 * @param timeout Timeout in milliseconds
 * @return NGX_OK on success, NGX_ERROR on failure
 * 
 * This function downloads the latest Tor exit node list
 * and updates the internal cache. It should be called
 * periodically based on the configured update interval.
 */
static ngx_int_t
ngx_http_torblock_update_exit_list(ngx_cycle_t *cycle, 
                                  ngx_str_t *url, 
                                  ngx_msec_t timeout)
{
    // Implementation
}

Contributing Guidelines

1. Before Contributing

Prerequisites for contributions:

# Fork and clone the repository
git clone https://github.com/yourusername/nginx-torblocker.git
cd nginx-torblocker

# Set up development environment
.development/setup-env.sh

# Run existing tests to ensure everything works
.development/test.sh

# Create feature branch
git checkout -b feature/your-contribution

2. Code Quality Checks

Automated quality assurance:

#!/bin/bash
# .development/quality-check.sh

echo "Running code quality checks..."

# Check code formatting
if command -v clang-format > /dev/null; then
    echo "Checking code formatting..."
    clang-format -i src/*.c src/*.h
fi

# Static analysis with cppcheck
if command -v cppcheck > /dev/null; then
    echo "Running static analysis..."
    cppcheck --enable=all --inconclusive src/
fi

# Build with warnings as errors
echo "Building with strict warnings..."
CFLAGS="-Wall -Wextra -Werror" .development/build.sh

# Run all tests
echo "Running test suite..."
.development/test.sh

echo "Quality checks complete!"

3. Pull Request Process

Standard process for contributions:

Feature Development
- Create descriptive branch name
- Implement feature with tests
- Update documentation
Testing
- Run complete test suite
- Add new tests for new functionality
- Verify performance impact
Documentation
- Update relevant wiki pages
- Add inline code documentation
- Update CHANGELOG.md
Submission
- Create pull request with detailed description
- Link relevant issues
- Request review from maintainers

Next Steps

After setting up the development environment:

Building from Source: Learn detailed compilation in Building from Source
Advanced Features: Explore Advanced Configuration for complex implementations
Testing: Review comprehensive testing in the test suite
Contributing: Follow the contribution guidelines in CONTRIBUTING.md

Uh oh!

Development Setup

Development Setup

Overview

Prerequisites

System Requirements

Environment Setup

Development Workflow

1. Source Code Organization

2. Setting Up Nginx Source

3. Build Configuration

4. Development Configuration

Debugging Tools and Techniques

1. GDB Debugging

2. Logging and Tracing

3. Memory Debugging with Valgrind

4. Performance Profiling

Testing Framework

1. Unit Testing Setup

2. Integration Testing

3. Load Testing

Development Best Practices

1. Code Style and Standards

2. Git Workflow for Development

3. Documentation Guidelines

Contributing Guidelines

1. Before Contributing

2. Code Quality Checks

3. Pull Request Process

Next Steps

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally