Merge pull request #9 from sharath-satish-msft/docs/public-repo-cleanup

docs: revise terminology, slim public docs, relocate CPDM-specific content

Author: avsrivastava

CHANGELOG.md

@@ -7,6 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Changed
+- Reworded public docs to refer to "data consumers" instead of "customers" to better describe the intended audience of this public repository.
+- Slimmed `docs/architecture.md` and `docs/faq.md` to keep only generic starter-kit guidance; removed deeper operator-specific content that lives in the internal platform repo.
+- Trimmed `docs/consumer-ingestion-guide.md` to focus on the manifest, index-file, and checkpointing contracts; removed references to internal platform pipeline names.
+
+### Added
+- `docs/extending-the-starter-kit.md` covering target-database substitution patterns (Fabric, SQL, Snowflake), extracted from the previous ingestion guide.
+
+### Removed
+- `docs/implementation-checklist.md` (internal operator checklist; relocated to the internal platform repo).
+
 ### Added
 - Initial open-source release setup
 - Microsoft OSS-compliant repository structure

CONTRIBUTING.md

@@ -64,14 +64,14 @@ See [.github/PULL_REQUEST_TEMPLATE.md](.github/PULL_REQUEST_TEMPLATE.md) for our
 
 ❌ Microsoft internal code or proprietary assets
 ❌ Secrets, credentials, or sensitive configuration
-❌ Tenant-specific or customer-identifying information
+❌ Tenant-specific or identifying information
 ❌ Code designed for Microsoft internal environments
 ❌ Changes requiring Microsoft internal dependencies
 
 ## Important Notes
 
 - **No SLA or Production Support**: This is a community project, not a Microsoft service. Contributions and responses are provided on a best-effort basis.
-- **Reference Architecture**: This is a starter kit and reference implementation. Production deployments require customer validation and additional controls.
+- **Reference Architecture**: This is a starter kit and reference implementation. Production deployments require additional validation and controls by the deploying organization.
 - **Your Responsibility**: When you deploy resources, you are responsible for their security, compliance, and operational management.
 
 ## Licensing

README.md

@@ -1,16 +1,15 @@
 # CDM Starter Kit
 
-A customer-focused Azure infrastructure starter kit for deploying CDM Starter Kit utility resources using Bicep and Azure CreateUIDefinition.
+An Azure infrastructure starter kit for deploying CDM Starter Kit utility resources using Bicep and Azure CreateUIDefinition.
 
 ## What This Repository Is
 
-This repository provides **infrastructure-as-code templates and deployment automation** for customers who want to establish foundational CDM Starter Kit capabilities in their Azure environments. It is **not** a Microsoft service, does not deploy into Microsoft tenants, and contains no first-party production code or secrets.
+This repository provides **infrastructure-as-code templates and deployment automation** for data consumers who want to establish foundational CDM Starter Kit capabilities in their own Azure environments. It is **not** a Microsoft service, does not deploy into Microsoft tenants, and contains no first-party production code or secrets.
 
 ## Who This Is For
 
-- Azure customers and partners
-- Organizations establishing CDM Starter Kit practices
-- Teams needing structured infrastructure foundations for data analytics and cost optimization
+- Data consumers establishing CDM Starter Kit practices in their own Azure subscriptions
+- Teams needing a structured infrastructure foundation for data analytics workloads
 
 ## What It Deploys
 
@@ -25,27 +24,27 @@ This starter kit provisions the following infrastructure components:
 
 ### Important Usage Constraints
 
-This starter kit is designed for **customer-owned Azure subscriptions only**:
+This starter kit is designed for **consumer-owned Azure subscriptions only**:
 
 - ✅ Deploy into your own Azure subscriptions
 - ✅ Verify subscription quotas, target-region service availability, and selected SKUs before deployment
-- ❌ **Do not** deploy into Microsoft internal or production tenants
 - ❌ This repository **does not contain secrets, API keys, certificates, or tenant-specific credentials**
 - ❌ **Never** add secrets, connection strings, or subscription IDs to this repository
 
 ### Implementation Details
 
 - **Infrastructure-as-Code**: Uses Azure Bicep for declarative, version-controlled infrastructure
 - **UI-Driven Deployment**: Includes Azure CreateUIDefinition forms for guided Azure Portal deployments
-- **Customer-Owned Identity**: All RBAC and access is configured by and for the deploying customer
+- **Consumer-Owned Identity**: All RBAC and access is configured by and for the deploying organization
 - **No Microsoft Runtime**: Does not provision Microsoft-managed services or long-running cloud infrastructure on our behalf
 
 ## Getting Started
 
 1. Review [CONTRIBUTING.md](./CONTRIBUTING.md) for contribution guidelines
 2. See [docs/architecture.md](./docs/architecture.md) for infrastructure overview
 3. Check [docs/faq.md](./docs/faq.md) for common questions
-4. Report security concerns privately via [SECURITY.md](./SECURITY.md)
+4. See [docs/extending-the-starter-kit.md](./docs/extending-the-starter-kit.md) for guidance on enhancing the kit
+5. Report security concerns privately via [SECURITY.md](./SECURITY.md)
 
 ## Code of Conduct
 
@@ -58,7 +57,7 @@ This project is licensed under the [MIT License](./LICENSE).
 ## Security & Transparency
 
 - **Secret Scanning**: This repository has Secret Scanning enabled to prevent accidental credential commits
-- **No Hardcoded Credentials**: All examples use parameterized deployment with customer-supplied values
+- **No Hardcoded Credentials**: All examples use parameterized deployment with consumer-supplied values
 - **Vulnerable Dependency Scanning**: Automated scanning is configured via Dependabot
 
 For security concerns, see [SECURITY.md](./SECURITY.md).
@@ -71,4 +70,4 @@ For security concerns, see [SECURITY.md](./SECURITY.md).
 
 ---
 
-**Not a Microsoft service.** Customer-deployed reference architecture for CDM Starter Kit infrastructure on Azure. Deploy and use at your own discretion.
+**Not a Microsoft service.** Consumer-deployed reference architecture for CDM Starter Kit infrastructure on Azure. Deploy and use at your own discretion.

SECURITY.md

@@ -30,7 +30,7 @@ We will acknowledge receipt of your report and work to assess and address the vu
 
 - ❌ **No secrets, API keys, or connection strings**
 - ❌ **No credentials, certificate files, or private keys**
-- ❌ **No Microsoft tenant IDs or customer subscription IDs**
+- ❌ **No tenant IDs or subscription IDs**
 - ❌ **No service principal credentials or managed identities**
 - ❌ **No hardcoded authentication tokens or SAS keys**
 
@@ -45,7 +45,7 @@ We will acknowledge receipt of your report and work to assess and address the vu
 
 This is a **reference architecture / starter kit**, not a production service.
 
-**Customers are responsible for:**
+**Data consumers are responsible for:**
 - ✅ Deploying into their own Azure subscriptions
 - ✅ Configuring appropriate RBAC and access controls
 - ✅ Implementing network security policies for their environments
@@ -54,10 +54,10 @@ This is a **reference architecture / starter kit**, not a production service.
 - ✅ Maintaining compliance with their own organizational policies
 
 **This repository is not responsible for:**
-- ❌ Customer execution environments or Azure tenants
-- ❌ Customer-supplied credentials or secrets
+- ❌ Consumer execution environments or Azure tenants
+- ❌ Consumer-supplied credentials or secrets
 - ❌ Ongoing operational security of deployed infrastructure
-- ❌ Compliance with customer-specific security policies
+- ❌ Compliance with consumer-specific security policies
 
 ## Dependency Management
 

SUPPORT.md

@@ -2,7 +2,7 @@
 
 ## Scope
 
-CDM Starter Kit is an open-source infrastructure starter kit for Azure customers and partners. This repository is **not** a Microsoft service and does not include SLA-backed production support.
+CDM Starter Kit is an open-source infrastructure starter kit for data consumers running workloads in their own Azure subscriptions. This repository is **not** a Microsoft service and does not include SLA-backed production support.
 
 ## How to Get Help
 
@@ -39,14 +39,14 @@ Follow `SECURITY.md` and report privately to the security contact listed there.
 
 ## What Is Out of Scope
 
-- Customer-specific production operations
+- Consumer-specific production operations
 - Tenant-specific deployment customization requests
 - Emergency/on-call support
 - Requests requiring internal Microsoft systems
 
 ## Responsible Use
 
-- Deploy only to customer-owned Azure subscriptions
+- Deploy only to Azure subscriptions you own
 - Do not commit secrets, credentials, tenant IDs, or subscription IDs
 - Validate changes with Bicep linting and PR checks before merge
 

docs/architecture.md

@@ -2,94 +2,55 @@
 
 ## High-Level Design
 
-This starter kit provides Infrastructure-as-Code templates to deploy a foundational CDM Starter Kit infrastructure stack on Azure. It is designed as a **customer-deployed reference architecture** on customer-owned Azure subscriptions.
+This starter kit provides Infrastructure-as-Code templates to deploy a foundational analytics stack on Azure. It is a **consumer-deployed reference architecture** intended to run in the data consumer's own Azure subscription.
 
 ## Core Principles
 
-### 1. Infrastructure-as-Code (Bicep)
+### Infrastructure-as-Code (Bicep)
 
 All infrastructure is declared using Azure Bicep:
 
-- **Version-controlled** - All configurations tracked in Git
-- **Repeatable** - Consistent, idempotent deployments
-- **Parameterized** - Customer-supplied values at deployment time
-- **Modular** - Components organized for reusability and maintainability
+- Version-controlled, repeatable, idempotent deployments
+- Parameterized so values are supplied at deploy time
+- Modular components for reuse and maintainability
 
-### 2. Customer-Owned Identity & Access
+### Consumer-Owned Identity & Access
 
-- **RBAC-First**: Role-based access control is the foundation
-- **No Service Accounts**: Customers manage all identities
-- **Least Privilege**: RBAC scaffolding supports principled access assignment
-- **Customer-Controlled**: All access decisions made by deploying organization
+- RBAC-first access model
+- All identities and role assignments are managed by the deploying organization
+- Least-privilege role mappings are surfaced as parameters, not embedded
 
-### 3. No Microsoft-Managed Runtime
+### No Microsoft-Managed Runtime
 
-This architecture does NOT provision:
+This architecture does **not** provision:
 
-- ❌ Microsoft-hosted services on behalf of the customer
-- ❌ First-party production infrastructure
-- ❌ Long-running Microsoft managed processes
-- ❌ Internal authentication or tenant-scoped components
+- Microsoft-hosted services on the consumer's behalf
+- First-party production infrastructure
+- Long-running Microsoft-managed processes
+- Internal authentication or tenant-scoped components
 
-Instead, it deploys **standard Azure customer-owned resources** that the organization operates directly.
+It deploys **standard Azure resources** that the data consumer operates directly.
 
 ## Infrastructure Components
 
-### 1. Azure Storage
+| Component | Purpose |
+| --- | --- |
+| Azure Storage | Foundation data storage layer |
+| Azure Data Explorer (Kusto/ADX) | Analytics and telemetry ingestion |
+| Azure Synapse Analytics | Data warehousing and processing |
+| RBAC scaffolding | Parameterized role assignments for users and service principals |
 
-**Purpose**: Foundation data storage layer
-
-- Supports data lakes, backups, and artifact repositories
-- RBAC configured by customer for multi-user access
-- Diagnostic data and logs stored securely
-- Network access restricted via SAS/RBAC policies
-
-**Customer Responsibility**: Configure lifecycle policies, network security, data retention
-
-### 2. Azure Data Explorer (Kusto/ADX)
-
-**Purpose**: High-performance analytics and telemetry ingestion
-
-- Time-series data analysis and exploration
-- Real-time dashboard and alerting capabilities
-- Data aggregation
-- Query-based insights and reporting
-
-**Customer Responsibility**: Query optimization, data retention policies, cluster sizing
-
-### 3. Azure Synapse Analytics
-
-**Purpose**: Data warehousing, ELT/ETL, and advanced analytics
-
-- Structured data warehouse for BI and reporting
-- SQL and Spark pools for processing
-- Integration with Power BI for visualization
-- Cost and resource optimization analysis
-
-**Customer Responsibility**: Workload tuning, compute scaling, metadata management
-
-### 4. RBAC Scaffolding
-
-**Purpose**: Principled access control foundation
-
-- Role assignments for service principals and users
-- Audit-ready access management
-
-**Customer Responsibility**: Assigning actual users/services, ongoing access reviews
+Configuration of lifecycle, network security, retention, scaling, and ongoing access reviews is the responsibility of the deploying organization.
 
 ## Deployment Flow
 
 ### Option 1: Azure Portal (CreateUIDefinition)
 
-1. Customer opens **Create** → Search "CDM Starter Kit"
-2. Guided form captures:
-   - Subscription and resource group
-   - Resource naming conventions
-   - Compliance/regulatory tags
-3. One-click deployment
-4. Resources created in customer's subscription
+1. Open **Create** in the Azure Portal and search for "CDM Starter Kit".
+2. The guided form captures subscription, resource group, naming conventions, and tags.
+3. Submit to deploy resources into your own subscription.
 
-### Option 2: Infrastructure-as-Code (Bicep CLI)
+### Option 2: Bicep CLI
 
 ```bash
 az deployment group create \
@@ -102,63 +63,17 @@ Both paths produce identical resource configurations.
 
 ## Security Boundaries
 
-### What Is Secure
-
-✅ Customer controls all identities (AAD users, service principals)
-✅ Customer controls RBAC and permissions
-✅ Data is encrypted at rest and in transit (Azure defaults)
-✅ Audit logs are captured and retained
-✅ Network access is configurable via NSGs
-
-### What Requires Customer Hardening
-
-⚠️ Network security policies (firewalls, private endpoints, NSGs)
-⚠️ Identity federation and conditional access policies
-⚠️ Data retention and compliance scoping
-⚠️ Backup and disaster recovery strategies
-⚠️ Monitoring and alerting configuration
-
-### What Is NOT Secured
-
-❌ This architecture does not include:
-- Multi-tenant isolation (customer-owned resources only)
-- Advanced threat detection (customer enablement required)
-- Compliance automation (customer responsibility)
-- Secrets/credentials management (customer's Azure Key Vault)
-
-## Operational Responsibility Model
-
-| Component | Deployed | Managed | Secured |
-|-----------|----------|---------|---------|
-| Azure Storage | Starter Kit | Customer | Customer |
-| Data Explorer | Starter Kit | Customer | Customer |
-| Synapse | Starter Kit | Customer | Customer |
-| RBAC Foundation | Starter Kit | Customer | Customer |
-| VNets / NSGs | Starter Kit | Customer | Customer |
-| Credentials | Customer | Customer | Customer |
-| Access Reviews | - | Customer | Customer |
-| Monitoring/Alerting | Starter Kit | Customer | Customer |
-
-**Summary**: This kit provides the **infrastructure template**. Customers are responsible for **deployment configuration, operational management, and security hardening**.
-
-## Scaling & Customization
-
-The starter kit is designed for easy customization:
-
-- **SKU Flexibility**: Bicep parameters support resource sizing
-- **Regional Deployment**: Works across Azure regions
-- **Tagging Strategy**: Customer-defined tag schemas (TBD)
-- **Module Reusability**: Components can be extended or replaced
-- **Integration**: Designed to connect with existing Azure infrastructure
+What the template provides out of the box:
 
-## Compliance & Governance
+- Encryption at rest and in transit (Azure defaults)
+- Audit-log capture on supported resources
+- Configurable network access via NSGs and resource firewalls
 
-- **Audit Logging**: All deployments and access logged to Activity Log
-- **Blueprint Support**: Can be packaged as Azure Blueprint
-- **Policy-Compatible**: Works with Azure Policy for governance
-- **Tagging**: Supports cost allocation and compliance tagging (TBD)
-- **Documentation**: Clear parameter descriptions for compliance review
+What you need to harden yourself:
 
----
+- Network policies (private endpoints, firewall rules, NSG tuning)
+- Identity assignments (AAD users, groups, service principals)
+- Data retention, lifecycle, and backup policies
+- Workload-specific compliance controls
 
-**Key Takeaway**: This is a **reference architecture starter kit**, not a production service. Customers deploy, operate, and secure the infrastructure. Microsoft provides the template and examples.
+See [extending-the-starter-kit.md](./extending-the-starter-kit.md) for patterns to adapt the kit to other target databases or runtimes.