Vomit-Scrapper — DevNet Project README

Live demo: https://revom.vercel.app/

This README is written from my point of view about what we built as a team on the DevNet project and how we shipped the app to production (Vercel). It summarizes the work we did, the architecture, how to run the project locally, and what we plan to do next.

---

TL;DR

We built a web application that scrapes and extracts structured property information from public property pages (legal description, owner, addresses, property location, etc.), normalizes and parses the legal text, and presents results through a simple UI. I worked with the team on scraping, parsing, testing, and the deployment pipeline. The app is deployed at: https://revom.vercel.app/

---

What we built (team summary)

• A resilient scraper that pulls property pages and captures:
  • The long legal description text
  • Owner details
  • Property address/location lines
  • Any imagery (where available) for offline processing
• OCR + parser workflow to turn messy legal-description text/images into structured fields:
  • Extract parcel identifiers, acreage, beginning-of-line/POB, row/road/description fragments
  • Normalize direction, distance and common legal terms so downstream tools can use them
• A lightweight frontend to:
  • Search for a parcel or paste raw legal text
  • Display extracted fields and raw legal text
  • Export results as JSON/CSV
• Deployment and CI:
  • Continuous deploy to Vercel (site: revom.vercel.app)
  • Basic unit and integration checks for the parser and scraper

---

My role & contributions

From my perspective on the team I was primarily responsible for:

• Designing and implementing the scraping and parsing pipeline.
  • Built the page fetcher and resilient retry logic.
  • Integrated OCR for scanned/embedded images and combined OCR output with text extraction.
• Implementing the normalization rules for legal descriptions (abbreviations, direction/distance normalization).
• Creating end-to-end testcases for the parser to ensure stability across many property record formats.
• Helping with the Vercel deployment: environment configuration and simple CI checks.
• Coordinating reviews, triaging parser edge-cases, and writing documentation for team onboarding.

Team members contributed as follows (high-level):

• Frontend: UI pages, search, UX, and client-side display/export features.
• Backend: scraper orchestration, queueing and storage integration.
• QA & Data: collected example records, created test-cases and validated parsing results across counties.
• DevOps: set up Vercel deployment and environment variable management.

---

Tech stack (high-level)

• Frontend: Next.js (React) — deployed to Vercel
• Scraper: Node.js script (headless browsing or HTTP client)
• OCR: Tesseract.js (or external OCR service in certain flows)
• Parsing/Normalization: Node.js text-processing modules (regex and small grammar helpers)
• Storage/Exports: JSON/CSV; can be extended to DB (Postgres / Mongo) or object storage
• CI / Deploy: GitHub Actions (optional) + Vercel for frontend

Note: The repo contains implementation details and exact packages used — this README is a focused project-summary and how-to-run guide.

---

Run locally (quickstart)

1. Clone the repo:

   git clone https://github.com/NaeemChakera/Vomit-Scrapper.git
   cd Vomit-Scrapper
   

2. Install dependencies:

   npm install
   

3. Environment variables

   • Create a .env.local for Next.js and .env/.env.scraper for the scraper.
   • Typical variables:

     NEXT_PUBLIC_API_URL=http://localhost:3000/api
     SCRAPER_USER_AGENT="..."
     OCR_API_KEY=...         # if using 3rd-party OCR
     

   • (See repository docs or env.example if present.)

4. Run frontend:

   npm run dev
   

   • Open http://localhost:3000 to view the interface.

5. Run scraper (example):

   npm run scrape -- --target "https://example-county/property?id=12345"
   

   • Check ./output or configured storage for results.

6. Run parser tests:

   npm test
   

---

How we deployed to Vercel

• The frontend (Next.js) is connected to the GitHub repo and deploys automatically on commits to main.
• Add Vercel environment variables in the Project Settings on Vercel (matching .env keys).
• For scraper/back-end processes we either:
  • Run scheduled scrapers on a server/CI runner and push results to storage used by the Vercel frontend, or
  • Use serverless functions (subject to runtime limits), depending on the run-time needs.

The live site is here: https://revom.vercel.app/

---

Data, privacy, and ethics

• We only processed public property records available from county property portals.
• Where personal data (PII) appears in scraped owner fields, team agreed to:
  • Keep PII use to the minimum required for the product
  • Provide secure storage/export and remove sensitive fields on request
  • Respect the source site terms of service and robots rules

---

Tests & quality

• Unit tests for the parser (covers numerous edge cases in legal-description formatting).
• Integration tests for scraper (mocked) to ensure parsing pipeline accepts the expected inputs.
• We validated key real-world examples from multiple county pages to improve robustness.

---

Lessons learned

• Legal descriptions are very inconsistent — a small grammar/regex approach works for common cases, but we must handle many rare edge-cases gracefully.
• OCR quality varies; combining text extraction + OCR usually gives the best coverage.
• Building good seed test-cases from real records is essential to stabilize the parser.

---

Next steps / Roadmap

• Harden the parser with more test-cases and a small rule-engine or lightweight grammar.
• Add a configurable DB backend (Postgres/Mongo) for long-term storage and analytics.
• Build a scheduled scraping/orchestration system (queue + workers) to scale safely.
• Add better UI for correcting parser mistakes and re-training rules from user feedback.

---

Contact / Acknowledgements

If you want to discuss anything about the project, or have sample property pages to improve the parser, please open an issue or reach out to me via GitHub: @NaeemChakera.

Thanks to everyone on the DevNet team — we shipped revom.vercel.app together.