Skip to content

Automative threat generation for LLM-agent based systems #1261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

marvel-code
Copy link

Summary:
These changes' goal is to provide automative threat generation for LLM-agent based systems.

Description for the changelog:

  • Add "Agent", "Tool" stencils
  • Add threat generation automation button with threat rules in json (ai generated)

Declaration:

  • appropriate unit tests have been created / modified
  • functional tests created / modified for changes in functionality
  • any use of AI has been declared in this pull request

@lreading
Copy link
Collaborator

Hey @marvel-code, this is awesome!

I think it's a bit unclear how the new graph button works. It took me playing around with it a bit (and reading the code) to figure out how it was supposed to work. Would it maybe make sense to add this button to the "Threats" pane at the bottom instead, and only show it when it would be able to generate threats? I'm open to other ideas/opinions as well!

There are also a couple of linting errors preventing the tests from running. 😃

Again, this is great and is really appreciated!

@marvel-code
Copy link
Author

Hey @lreading, I really appreciate getting such positive, detailed, and constructive feedback - it means a lot!

I think your idea about moving the button is excellent, it would definitely make it much more convenient. I did realize that the current implementation might be a bit rough around the edges; I mainly wanted to get some early feedback. Your comments were super helpful - I'll fix the issues you pointed out and come back with the improvements!

@jgadsden
Copy link
Collaborator

jgadsden commented Apr 28, 2025

Hello @marvel-code , an impressive amount of work
Could you give more context on your changes? by raising an issue or creating a discussion, because it is not clear why we need two new shapes in the threat model stencil and how the threats are generated
Screenshot 2025-04-28 at 21 41 53

There is an example threat model in the OWASP Gen Top 10 for LLM Application Security Project that could be used for the look and feel: https://genai.owasp.org/resource/owasp-top-10-for-llm-applications-2025/

@jgadsden jgadsden marked this pull request as draft April 30, 2025 06:16
@jgadsden
Copy link
Collaborator

Hello @marvel-code , marking this pull request as draft while there is work being done it 👍🏾

@jgadsden jgadsden removed this from the Version 2.5 milestone May 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants