Structured_Prompt_Generation

AI Prompt Refinement & Execution Framework

This project implements a prompt-engineering analysis and transformation system that treats prompt construction as a learnable software engineering skill, rather than trial-and-error interaction with LLMs.

It operationalizes a Prompt Composition Framework, computes a Prompt Quality Score (PQS), and empirically compares naïve prompts vs structured prompts using a deterministic transformation pipeline.

---

1. Motivation

Large Language Models (LLMs) are widely used for tasks such as code generation, reasoning, and problem solving. However, the quality of the output is highly dependent on the quality of the input prompt.

Novice users typically write:

• vague prompts
• incomplete task descriptions
• missing constraints and validation criteria

This project addresses that gap by:

• formally defining prompt components,
• detecting which components a naïve prompt satisfies,
• transforming the prompt into a fully structured version,
• and quantifying prompt quality using a reproducible metric.

---

2. Core Concepts

2.1 Prompt Components (C1–C6)

Each prompt is analyzed against six locked components:

Component	Description
C1	Task specification
C2	Input / output specification
C3	Constraints & efficiency requirements
C4	Algorithmic & implementation guidance
C5	Edge-case & boundary handling
C6	Verification & validation instruction

---

2.2 Prompt Quality Score (PQS)

Prompt Quality Score measures prompt completeness, not output quality.

Definition:

PQS = (Number of present components) / 6

• Range: 0.0 – 1.0
• Naïve prompts typically score < 0.5
• Structured prompts always score 1.0 by construction

---

3. System Pipeline

Naïve Prompt
     ↓
Component Detection (C1–C6)
     ↓
Naïve PQS Computation
     ↓
Deterministic Prompt Transformation
     ↓
Structured Prompt (C1–C6 enforced)
     ↓
Structured PQS = 1.0
     ↓
LLM Execution

---

4. Architecture

Backend

• Python
• Gemini API (google-genai)
• Deterministic PQS computation
• Robust JSON extraction and repair

Frontend

• Streamlit

• Interactive visualization of:

  • component qualification
  • naïve vs structured PQS
  • structured prompt
  • final model output

---

5. Project Structure

Structured_Prompt_Generation/
│
├── app.py                # Streamlit frontend
├── prompt_engine.py      # Core prompt analysis & transformation engine
├── README.md             # Project documentation
├── .env                  # API keys (not committed)
├── .gitignore
└── venv/                 # Python virtual environment

---

6. Setup Instructions

6.1 Create Virtual Environment

python -m venv venv
source venv/bin/activate   # Linux / macOS
venv\Scripts\activate      # Windows

---

6.2 Install Dependencies

pip install streamlit python-dotenv google-genai

---

6.3 Configure Environment Variables

Create a .env file in the project root:

GEMINI_API_KEY=your_api_key_here

Add .env to .gitignore.

---

6.4 Run the Application

streamlit run app.py

Open the browser at:

http://localhost:8501

---

7. Usage

1. Enter a naïve prompt (student-like, informal, incomplete).

2. Click “Refine and Execute”.

3. The system displays:

   • which prompt components (C1–C6) are satisfied,
   • naïve prompt PQS,
   • the fully structured prompt,
   • structured PQS (1.0),
   • the final model-generated answer.

---

8. Robustness & Design Decisions

• No dataset dependency
• No fine-tuning
• Model treated as a black box
• JSON output sanitized and repaired before parsing
• PQS computed deterministically (model output is not trusted for scoring)

---

9. Academic Alignment

This implementation supports:

• empirical comparison of naïve vs structured prompts,
• reproducible prompt transformations,
• correlation studies between prompt quality and model performance.

It is suitable for:

• prompt-learning datasets,
• controlled experiments,
• research publications in AI-assisted software engineering.

---

10. Limitations

• Currently single-prompt, single-run
• Python-only focus
• No batch evaluation UI (can be added)

---

11. Future Extensions

• Batch evaluation for large datasets
• CSV export of PQS metrics
• Prompt complexity classification (Low / Medium / High)
• PQS vs correctness / efficiency correlation plots
• Multi-model comparison (Gemini Flash / Pro / Gemma)