Discover what people actually want by turning online communities into searchable knowledge.
An open source project
Created by: Michael Shehata.
gmIdeas is an AI powered research platform that crawls online communities, forums, and documentation to identify recurring problems, feature requests, frustrations, and unmet needs.
Instead of scraping a single webpage, gmIdeas aims to build a structured knowledge base from entire communities, allowing researchers to explore trends, validate ideas, and uncover opportunities through natural language.
The project was originally created to answer a simple question:
"What applications or products do people genuinely want that don't exist yet?"
Rather than manually reading hundreds of Reddit posts, GitHub issues, forum discussions, or blog comments, gmIdeas automates the process of discovering patterns and surfacing evidence-backed insights.
The long term goal is to build an AI research assistant capable of answering questions such as:
- What applications are people asking for?
- What problems are mentioned most frequently?
- Which software receives the most complaints?
- What features are users repeatedly requesting?
- What startup opportunities appear across multiple communities?
- How have opinions changed over time?
Instead of returning a simple summary, gmIdeas provides the supporting evidence behind every insight.
- Crawl entire communities instead of individual pages
- Discover new pages automatically
- Resume interrupted crawls
- Deduplicate repeated content
- Incrementally update existing datasets
- Natural language querying
- Topic clustering
- Sentiment analysis
- Trend detection
- Automatic summarisation
- Opportunity discovery
Examples include:
- Most requested application ideas
- Frequently mentioned frustrations
- Common feature requests
- Emerging trends
- Market gaps
- Popular alternatives
- Community sentiment over time
- "What applications do developers wish existed?"
- "What are the biggest complaints about Docker?"
- "Which AI tools are gaining popularity?"
- "Find recurring startup ideas from r/selfhosted."
- "What features are users requesting for Obsidian?"
- "Compare opinions on local LLMs over the last two years."
🚧 Early development.
The initial milestone focuses on building a reliable Reddit crawler capable of collecting posts and comments, indexing them into a searchable database, and enabling research over the collected data.
Future milestones will expand support to additional platforms such as GitHub, documentation sites, community forums, and other publicly available sources.
gmIdeas is not intended to be "just another web scraper."
Its objective is to become a research platform that helps founders, developers, researchers, and businesses understand what people actually need by transforming large volumes of community discussions into structured, evidence backed knowledge.