Skip to content

Latest commit

 

History

History
252 lines (189 loc) · 8.83 KB

checkpoint.md

File metadata and controls

252 lines (189 loc) · 8.83 KB

AI Directories Project — Checkpoint

Date: 2026-02-16 (updated)

Directory Database (directories.json)

Metric Count
Total entries 827
Active 489
Not found 162
Domain dead 64
Timeout 30
Cloudflare blocked 20
Error 32
Invalid URL 12
Facebook group 11
Domain parked 7

Auth Type Breakdown (all 827)

Auth Type Count
unknown 308
none (open submit) 296
google_only 142
email_password 36
facebook 22
google_and_email 14
other combos 9

Data Sources

  1. Original dataset — 388 entries
  2. sergiuchiriac/ai-directories — 209 parsed, 47 duplicates skipped, 162 new added
  3. Curated AI directory lists + SaaS aggregators + web directories — 50 new added (Feb 16)
  4. best-of-ai/ai-directories (714★) — 155 parsed, 28 duplicates skipped, 127 new added (Feb 16)
  5. FutureGenApps directory submission sites — 219 parsed, 15 dupes, 104 low-DA skipped, 100 new added (DA≥30, Feb 16)

Submission Plan (submission_plan.json)

Target: All active directories with auth_type = none (173) + auth_type = email_password (19) = 192 total

Discovery Status

Status Count
discovered (form fields found) 116
no_form_found 28
timeout 48

Best Candidates for Auto-Submit

45 directories have forms with >= 2 fields AND no captcha. These are the easiest to automate.

Copy Variations

30 unique title/description pairs stored in submission_plan.json, rotated across all 192 directories.

Credentials

Configure credentials in submission_plan.json under each entry's credentials block and in submit_directories.py PRODUCT dict. Fields:

Field Placeholder
Email YOUR_EMAIL
Name YOUR_NAME
Username YOUR_USERNAME
Password YOUR_PASSWORD

Search for YOUR_ placeholders and replace with your actual values before running.

Pipeline Scripts

Script Purpose
analyze_directories.py HTTP-level analysis (auth, captcha, pricing, dead domains). ThreadPoolExecutor.
cleanup_and_categorize.py Triage errors, build browser_check_list.json for Playwright pass.
browser_verify.py Async Playwright, 10 workers. --recheck-unknown for deep DOM pass on active unknowns.
discover_forms.py Async Playwright, 10 workers. Visits submission URLs, extracts form fields via DOM queries. Updates submission_plan.json.
submit_directories.py Async Playwright, 5 workers. Heuristic field mapping + file uploads. Reads submission_plan.json.

All scripts use .venv/bin/python and are idempotent.

Key Files

File Description
directories.json Master database — 827 entries with auth, captcha, status, etc.
submission_plan.json 318 submission targets with copy, form fields, form paths, status.
browser_check_list.json Intermediate — sites needing browser verification (generated by cleanup script).
checkpoint.md This file — project state snapshot.

Submission Results

Script: submit_directories.py — Async Playwright, 5 workers, heuristic field mapping, file uploads for logo/screenshot.

Status Count Description
submitted 71 Form filled + submit button clicked
timeout 40 From discovery phase (never got forms)
no_fields_matched 34 Page had forms but fields didn't match product data patterns
no_form_found 28 No forms found on the page at all
discovered 14 Forms found but not attempted (search-only / trivial)
submit_timeout 4 Timed out during submission attempt
filled_no_submit 1 Fields filled but no submit button found

Assets Used

File Purpose
site-image.png Product screenshot (viewport capture)
logo.png Product logo (nav element crop)

Captcha Directories — Manual Submission Required

Directories with captcha require manual browser interaction. Across two batches:

Batch 1 (5 sites): 4 submitted, 1 blocked (broken captcha). Batch 2 (10 sites): 5 submitted, 1 timeout, 4 skipped (not real directory forms).

Timeout Retry Results

Retried the 40 timeout entries with a fresh pass:

  • 4 submitted (forms worked on retry)
  • 5 skipped (paid-only, not product directories, or login-gated)
  • 3 captcha pending (form filled but captcha unsolved)
  • 28 handled total, 12 timeout entries remained

Updated Submission Plan Summary (318 entries)

Status Count
submitted 66
skipped 94
no_form_found 40
no_fields_matched 41
timeout 30
skipped_paid 17
discovered (forms found, not yet attempted) 9
cloudflare_blocked 6
submit_timeout 4
skipped_login_required 3
domain_parked 2
other (captcha, deferred, errors) 6

Manual Browser Submissions

Several directories were submitted manually via Playwright browser automation (Google login, form filling, etc.). These include SaaSHub, FutureTools.io, AItools.inc, AI Hunt (partial), and others.

Directories Tried but Not Viable (Manual)

Directory Reason
Best of AI Paid ($29)
AI Parabellum Paid ($99/$199)
TopTools.ai No submit form
AI-Tools Directory No submit form
StackShare Not a submission directory
EU-Startups Requires membership/login
Sidebar.io Design newsletter, not directory
Jasmine Directory Paid review fee
Alive Directory Requires reciprocal link + reCAPTCHA
SoftwareSuggest Rejects free email domains (requires business email)
Feedough Submit page 404
Desifounder Requires login
Smart Tools AI Backend API broken
AI Directory (aidirectory.org) Broken reCAPTCHA
Futurepedia Paid (free tier "sold out")
AIWizard Requires NFT purchase
AIToolHunt Empty/broken page
Dang.ai Backlink requirement + reCAPTCHA

GitHub PRs Created

Repo Stars PR Status
best-of-ai/best-of-ai 572 PR #22 Open
cporter202/ai-agent-tools 357 PR #2 Open
ArshdeepGrover/ai-tools-manager 29 PR #155 Open
aitoollist/awesome-ai-tool-list 18 PR #46 Open
DhanushNehru/Ultimate-AI-Resources 19 PR #6 Open

Cumulative Submission Totals

Category Count
Auto-submitted (script) ~101
Manually submitted (browser) ~18
Total submitted ~119
Skipped (various reasons) ~94
Paid directories identified ~17
Still pending 9 discovered + 30 timeout

Lessons Learned

Common Blockers

  • Paid directories: ~20% of AI directories require payment ($15-$299). Many don't disclose pricing until the submit page.
  • reCAPTCHA / Turnstile: ~5-10% have captchas; some are broken (invalid site keys).
  • Login-gated: Many require account creation before submission. Google OAuth is common.
  • Business email required: Some (SoftwareSuggest) reject free email domains (gmail, yahoo).
  • Reciprocal links: Older web directories require you to link back to them first.
  • NFT / crypto gates: A few require purchasing tokens/NFTs.
  • Backlink requirements: Some AI directories check if you link back before listing.
  • Newsletter-only forms: Many pages that look like submission forms are actually newsletter signups.

Automation Success Factors

  • Directories with simple HTML forms (no JS frameworks) have highest auto-submit success.
  • reCAPTCHA v3 (invisible) sometimes passes automatically; v2 (checkbox) never does.
  • Google Forms are reliably automatable.
  • Rich text editors (TinyMCE, Quill) need special handling.
  • Cloudinary/custom upload widgets often break automation.
  • Cross-origin Google OAuth popups need tab-switching logic.

Best ROI Directories (Free, No Captcha, Real Forms)

AI-specific directories with simple forms: FutureTools.io, SaaSHub, AItools.inc, ToolScout AI, Open Future AI, AI Respo, AINave, NoCodeList, NoCodeDevs.

Next Steps

  1. Verify high-confidence submissions — Check ~20 top submissions actually landed (not newsletter signups).
  2. Retry 30 timeout entries — Many are slow-loading general directories.
  3. Manual submit to no_fields_matched — 41 pages had forms but heuristic mapping didn't match.
  4. Handle 9 discovered but not submitted — Forms found but not yet attempted.
  5. Handle no_form_found — 40 entries with JS SPAs or non-standard UIs.
  6. Paid directories — Several require payment ($15-$299) for listing.
  7. Monitor GitHub PRs — Check merge status on the 5 PRs created.