PASTA: Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification

Ningzhi Tang @ SaNDwich Lab, University of Notre Dame

PASTA (Prompt-Assisted Software TrAnsformation) is a research prototype plugin for JetBrains IDEs (e.g., PyCharm, WebStorm) designed to investigate how developers use different prompting techniques for real-world code modification tasks. PASTA supports both summary-mediated and direct instruction prompting, providing an environment for empirical studies of LLM-assisted code editing.

PASTA is available on JetBrains Marketplace and is compatible with the entire family of JetBrains IDEs from 2024.2 to 2025.2. You can download it from: https://plugins.jetbrains.com/plugin/28121-pasta-ai or search for "PASTA AI" in the JetBrains Marketplace.

Features & Workflow

• Selection-Based Prompting: Select code in the editor to specify the target for modification.
• Summary-Mediated Prompting: Generate a concise, editable summary (1–3 sentences) of the selected code. Edit the summary to express the intended change, and view before/after summary differences to aid revision.
• Direct Instruction Prompting: Alternatively, write a free-form natural language command describing the desired modification.
• Commit & Review: Submit your prompt (summary or instruction) to the LLM. The plugin displays the resulting code changes in a diff view with line- and token-level highlights for inspection and selective acceptance.
• Context Inclusion: The full content of the current file is always included as context in LLM requests.

Implementation

• Built with the IntelliJ Platform Plugin SDK, compatible with all JetBrains IDEs.
• Uses Java Diff Utils for summary comparison and the JetBrains SDK's diff package for code changes.
• LLM integration via OpenAI GPT-4o chat completions API, with few-shot examples for consistency.

Running

For detailed running instructions, refer to the IntelliJ Platform Plugin SDK. Some environment setup and adjustments should be referred to or adjusted in build.gradle.kts.

Please note that you should set up your own OpenAI API key in your local OS environment.

Running in Development

1. Clone this repository and open the project in IntelliJ IDEA
2. Run the plugin by clicking Run > Run 'Run Plugin' button

Building for Production

1. Build the plugin ZIP file using Gradle: ./gradlew build
2. The plugin package will be generated in build/distributions/

Citation

PASTA was developed as part of our research on LLM-assisted code modification. The system enabled our paper: Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification (VL/HCC 2025).

If you use PASTA in your research, please cite:

@inproceedings{tang2025exploring,
  title={Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification},
  author={Tang, Ningzhi and Smith, Emory and Huang, Yu and McMillan, Collin and Li, Toby Jia-Jun},
  booktitle={2025 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)},
  year={2025},
  organization={IEEE}
}

For the full replication package of our paper, please refer to: ND-SaNDwichLAB/direct-vs-summary-study.

Contact

For questions or collaboration inquiries, please contact Ningzhi Tang at ntang@nd.edu or ningzhitang2001@gmail.com.

Acknowledgments

This research was supported in part by an AnalytiXIN Faculty Fellowship, an NVIDIA Academic Hardware Grant, a Google Cloud Research Credit Award, a Google Research Scholar Award, and NSF grants CCF-2211428, CCF-2315887, and CCF-2100035. Any opinions, findings, or recommendations expressed here are those of the authors and do not necessarily reflect the views of the sponsors.