Skip to content

Create a public, curated immigrant-knowledge-base to improve retrieval quality for Finland guidance #3

Description

@brylie

We’re seeing low-relevance retrieval from the current corpus of official Finland sources in some common immigrant Q&A flows.
Even when official information exists, retrieval quality can degrade because of:

  • fragmented information architecture across agencies/sites
  • high information density in official pages
  • legal/administrative wording that is hard to parse
  • language mismatch (English queries vs Finnish/Swedish-heavy source material)
  • terminology mismatch between how users ask and how authorities phrase topics

Problem

Our agent can miss or under-rank the best official sources for practical immigrant questions.
This creates avoidable friction for users trying to understand real-world processes (residence, registration, work, healthcare, tax, family matters, etc.).

Proposal

Create a custom, public knowledge base of high-quality, article-style guides that:

  1. are written in plain language around real user intents/questions
  2. link to and cite official Finnish sources as the ground truth
  3. standardize terminology and synonyms (user language ↔ official language)
  4. are structured for strong retrieval and chunking performance
  5. can be ingested as a prioritized corpus for the assistant

This should improve retrieval relevance while also creating a standalone public resource that benefits users directly and increases visibility/credibility for the Finntegrate project.

Goals

  • Improve answer relevance and consistency for top immigrant question categories
  • Reduce “missed official page” failures in retrieval
  • Provide user-friendly explanations without replacing official authorities
  • Publish a useful public artifact that can attract and help more users

Non-goals

  • Replacing legal advice or official decision-making
  • Publishing uncited claims
  • Creating content that diverges from official guidance without clear labeling

Content principles

  • Grounded: Every actionable claim points to an official source
  • Readable: Plain language, short sections, examples, glossary
  • Transparent: “Last reviewed” date + source list per article
  • Multilingual-aware: Finnish terms, English paraphrases, common query variants
  • Safety-first: Clear disclaimers where legal interpretation is sensitive

Suggested article template

  • What this is
  • Who this applies to
  • Step-by-step
  • Documents you may need
  • Costs and timelines (if known)
  • Common mistakes
  • Official sources
  • Last reviewed / update cadence

Initial scope (MVP)

Create 15–25 priority articles for highest-volume intents, e.g.:

  • first registration steps after arrival
  • municipality of residence
  • personal identity code basics
  • Kela and healthcare access basics
  • tax card and first employment
  • opening a bank account
  • family reunification basics
  • student vs worker pathways (high-level comparison)
  • renewing permits and deadlines
  • where to get authoritative case-specific help

Implementation notes (for retrieval)

  • Store articles in structured Markdown with frontmatter (title, topics, official_sources, last_reviewed, language, aliases)
  • Include canonical question variants per article
  • Add cross-links between related processes
  • Prefer short, semantically focused sections for better embedding/chunk retrieval
  • Consider citation block format standardized for parser ingestion

Success metrics

  • Higher retrieval precision@k on benchmark immigrant queries
  • Lower rate of unanswered/low-confidence responses
  • Better user feedback on clarity and usefulness
  • Coverage score for top intents (e.g., % with at least one high-quality guide)

Risks & mitigations

  • Risk: Content drift from policy changes
    Mitigation: review cadence + “last reviewed” + source checks
  • Risk: Oversimplification of legal nuance
    Mitigation: explicit limits/disclaimers + deep links to official text
  • Risk: Editorial overhead
    Mitigation: start with MVP topics and a strict template

Acceptance criteria

  • Article template defined and documented
  • MVP topic list approved
  • First 15+ articles published with official citations
  • Ingestion/retrieval pipeline includes KB as prioritized source set
  • Baseline vs post-KB retrieval evaluation documented

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions