Skip to content

novacLearnsCyber/gitScrapper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 

Repository files navigation

gitcheck

git scraper, good for checking vulnerable info and AI comments in public repos.

give it a github username, profile url, or repo link and it goes through every public repo, grabs the raw content of every file and flags stuff that looks sketchy - leaked secrets, hardcoded passwords, comments that read like they were written by chatgpt instead of an actual person.

setup

chmod +x install.sh
./install.sh
source ~/.bashrc   # or .zshrc if ur on zsh

then just run gitcheck from wherever.

vulnerable info checked (just regex)

  • generic_password
  • generic_secret
  • aws_access_key
  • aws_secret
  • private_key
  • slack_token
  • openai_key
  • jwt
  • db_connection_url

these are pattern matches not confirmed leaks, so check em before freaking out. test files and stuff like password: "changeme" will trigger false positives, thats normal.

how the ai comment detection works

theres no single "yep thats ai" check, its a bunch of small things stacked up into a score out of 100, once it passes a threshold the file gets flagged.

stuff it looks at:

  • comment frequency, how many comments vs actual lines of code. ai tends to comment way too much, explaining lines that dont need explaining
  • generic phrasing, comments starting with "Initialize", "Loop through", "Check if", "Return the" etc. people dont really talk to their code like that
  • emoji in comments or in print/log lines (✅ ❌ 🚀). shows up a lot in ai generated stuff, barely ever in real projects
  • full docstrings on tiny 2 line functions, big tell
  • comment length being suspiciously uniform, real comments are messy, some 2 words some a whole sentence, ai comments all kinda look the same length
  • perfect grammar/punctuation across every single comment in the file, real devs dont bother capitalizing half the time
  • verbose print statements, stuff like [1/5], [OK], "Successfully processed X", "Starting..." shows up a ton in ai written scripts

none of these mean much by themselves, one emoji or one clean docstring doesnt prove anything. its the combo + ratio across the whole file that pushes the score up. treat it as "worth checking" not "guilty"

notes

  • only checks the current state of files on the default branch, doesnt dig through commit history for stuff that got deleted later (yet)
  • skips empty repos and .git stuff automatically
  • uses the raw github content (same as clicking Raw on github) so its just requests under the hood, no browser needed to actually read the files

About

git scrapper great for checking vulnerable information and AI comments

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages