vLLM Tools

This repository contains tools for interacting with vLLM (Very Large Language Model) servers.

Contents

• vllm_query_app.py: A Python script for sending queries to a vLLM server and receiving responses.

Setup

1. Ensure you have Python installed on your system.
2. Install the required dependencies:

   pip install requests
   

3. Set up your vLLM server according to your specific configuration.

Usage

Run the vLLM query application:

python vllm_query_app.py

Follow the prompts to enter queries and receive responses from the vLLM server.

Configuration

Modify the VLLM_API_BASE and API_KEY variables in vllm_query_app.py to match your vLLM server setup.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License