HTTP Random Testing Tool

A tool for simulating normal user web browsing behavior, which can be used to test network devices (such as NAT). To avoid putting pressure on the tested websites, please adjust the access interval to an appropriate time.

Language: English | 中文

Features

• Search Engine Access: Automatically generates random keywords and accesses search engines for searching
• Normal URL Access: Direct access to specified URL lists
• Navigation Website Access: Extracts URLs from navigation websites and accesses them
• Concurrent Control: Multi-threaded concurrent access support
• Multiple Access Modes: Random and round-robin modes for reading URL libraries
• Detailed Logging: Records access URLs, timing, success status, source identification, and more. Automatically cleans up old log files (keeps latest 10)
• Statistical Reports: Provides access statistics at the end of execution

Project Structure

HTTPRandTest/
├── main.py              # Main program entry point
├── lib/                 # Library files directory
│   ├── __init__.py     # Python package initialization
│   ├── http_client.py  # HTTP client module
│   ├── logger.py       # Logging module
│   └── url_scheduler.py # URL scheduler module
├── config.yaml         # Configuration file
├── requirements.txt    # Dependencies list
├── .gitignore         # Git ignore file
├── README.md          # Project documentation (English)
├── README_CN.md       # Project documentation (Chinese)
└── logs/              # Log files directory (auto-created)

Installation

pip install -r requirements.txt

Configuration

The configuration file config.yaml contains the following main sections:

1. Base Configuration

• interval: Interval between requests (seconds), supports decimal values
• enable_logging: Enable access logging, true/false
• concurrent_threads: Number of concurrent threads
• duration: Execution duration (seconds, 0 = unlimited)
• max_requests: Maximum requests per thread (0 = unlimited)
• access_mode: URL access mode, options: random (random access), round_robin (round-robin access)

2. HTTP Configuration

• timeout: Request timeout (seconds)
• user_agent: Browser user agent string
• headers: Custom HTTP request headers, including Accept, Accept-Language, Accept-Encoding, etc.

3. Search Engine Configuration

Search engine URL list with {keyword} placeholder support. Keywords are automatically generated as random alphanumeric strings (3-8 characters).

4. Normal URL Configuration

Direct access URL list containing various types of website links.

5. Navigation URL Configuration

Navigation website URL list. These URLs are visited to extract embedded links, which are then accessed.

6. Logging Configuration

• log_file: Log file path
• level: Log level, options: DEBUG, INFO, WARNING, ERROR
• format: Log format string
• console_output: Whether to output logs to console, true/false

Usage

Basic Usage

# Run with default configuration file
python main.py

# Specify configuration file
python main.py -c config.yaml

Command Line Parameters

Parameter Description

Parameter	Description	Default
-c, --config	Configuration file path	config.yaml
-t, --threads	Number of concurrent threads	Config file setting
-i, --interval	Request interval in seconds	Config file setting
-d, --duration	Execution duration in seconds	Config file setting
-n, --max-requests	Maximum number of requests	Config file setting
--no-log	Disable access logging	false
--search-only	Test search engines only	false
--normal-only	Test normal URLs only	false
--navigation-only	Test navigation URLs only	false
-v, --verbose	Show more detailed log content (DEBUG level)	false
-m, --mode	URL access mode (random/round_robin)	Config file setting

Usage Examples

# Set concurrent threads to 5, request interval to 2 seconds, duration to 60 seconds
python main.py -t 5 -i 2 -d 60

# Disable logging
python main.py --no-log

# Access search engines only
python main.py --search-only

# Access normal URLs only
python main.py --normal-only

# Access navigation websites only
python main.py --navigation-only

# Specify access mode, sequential access to URL library
python main.py -m round_robin

Logging

The program records the following information:

• Accessed URLs
• Access time
• Request success status
• HTTP status codes
• Response times
• Error messages (if any)

Log files use timestamp-based naming (e.g., http_test_20250925_095214.log) and are automatically cleaned up to keep only the latest 10 files.

Statistics

The program displays detailed statistics at the end of execution:

• Total requests
• Successful requests
• Failed requests
• Success rate
• Average response time
• Minimum/maximum response times
• Total execution time

Dependencies

• requests: HTTP request library
• PyYAML: YAML configuration file parsing
• typing-extensions: Type hint support (Python < 3.8)

Contributing

Issues and Pull Requests are welcome to improve this project.