Skip to content

Conversation

aaronsteers
Copy link
Collaborator

@aaronsteers aaronsteers commented Apr 23, 2025

Summary by CodeRabbit

  • New Features

    • Introduced comprehensive schema, catalog, and configuration files for Salesforce data integration, including detailed table and field mappings for core Salesforce objects.
    • Added initial dbt project setup, profiles, package dependencies, and source schema definitions to support Salesforce data modeling and transformation.
    • Provided extensive transform templates for Salesforce entities (e.g., Account, Contact, Lead, Opportunity, Task, User), each with field descriptions and placeholder expressions for future mapping.
    • Included documentation for model structure and rejected mappings for transparency.
  • Documentation

    • Added README and markdown files documenting the dbt models, project structure, and rejected data mappings.
  • Chores

    • Added .gitignore and configuration files to manage dbt build artifacts and environment settings.
  • Bug Fixes

    • Improved error handling in schema model parsing to prevent issues with missing columns.

Copy link

coderabbitai bot commented Apr 23, 2025

Walkthrough

This update introduces a comprehensive suite of files and configurations to support Salesforce data integration, transformation, and modeling, primarily targeting Airbyte and dbt workflows. New ERD and schema definition files are added for both source (Airbyte) and target (Fivetran) systems, along with detailed dbt source and project configuration files. Numerous YAML transformation files are introduced for Salesforce entities, outlining field mappings—many with placeholder expressions pending implementation. Supporting documentation, mapping evaluation reports, and configuration files for interoperability, package management, and dbt project setup are also included. Minor code improvements enhance robustness in schema model parsing.

Changes

File(s) Change Summary
build/salesforce/airbyte-catalog.json New JSON catalog defining Salesforce streams, schemas, sync modes, cursors, and metadata for Airbyte integration.
build/salesforce/airbyte-interop.morph-lock.toml New TOML config for Airbyte interoperability, mapping Salesforce streams to target fields and tracking mapping completeness.
build/salesforce/erd/source-schema-erd.dbml
build/salesforce/erd/target-schema-erd.dbml
New DBML ERD files defining source (Airbyte) and target (Fivetran) Salesforce schemas, including tables, columns, and metadata.
build/salesforce/requirements/airbyte-interop/src_dbt_requirements.yml New YAML file specifying dbt source configuration, table/column metadata, and freshness policies for Salesforce via Fivetran.
build/salesforce/src_airbyte_raw_salesforce.yml New YAML file defining Airbyte source schema and table/column metadata for Salesforce raw data ingestion.
catalog/salesforce/README.md
catalog/salesforce/rejected-mappings.md
New documentation: README for dbt model directory and markdown listing rejected Airbyte-to-Fivetran mappings with evaluation details.
catalog/salesforce/airbyte-interop-dbt-project/.gitignore New .gitignore for dbt project, excluding build artifacts, logs, and user-specific files.
catalog/salesforce/airbyte-interop-dbt-project/dbt_project.yml New dbt project configuration specifying model paths, materialization, cleaning targets, and variables for the Salesforce domain.
catalog/salesforce/airbyte-interop-dbt-project/models/src_airbyte_raw.yml New dbt source catalog YAML with detailed schema and column metadata for Salesforce Airbyte raw tables.
catalog/salesforce/airbyte-interop-dbt-project/package-lock.yml
catalog/salesforce/airbyte-interop-dbt-project/packages.yml
New package lock and dependency declaration for dbt project, specifying dbt-labs/dbt_utils version.
catalog/salesforce/airbyte-interop-dbt-project/poe_tasks.toml New configuration for external task inclusion in dbt project using POE.
catalog/salesforce/airbyte-interop-dbt-project/profiles/profiles.yml New dbt profiles file defining DuckDB and MotherDuck targets for the Salesforce dbt project.
src/morph/models.py Improved robustness in DbtSourceTable.from_dict by using data.get("columns", []) and added explicit type annotation for local variable in to_dict.
src/transforms/salesforce/transforms/*.yml New YAML transformation configs for Salesforce entities (account, contact, event, lead, opportunity, etc.), defining field mappings with many placeholder expressions and detailed field descriptions.

Sequence Diagram(s)

sequenceDiagram
    participant Airbyte
    participant Salesforce
    participant DBT
    participant DataWarehouse

    Airbyte->>Salesforce: Fetch data streams (as per airbyte-catalog.json)
    Airbyte->>DataWarehouse: Load raw Salesforce data (src_airbyte_raw_salesforce.yml)
    DBT->>DataWarehouse: Reference raw tables (src_airbyte_raw.yml)
    DBT->>DataWarehouse: Transform data using mapping configs (transforms/*.yml)
    DataWarehouse-->>DBT: Provide transformed tables (as per dbt_project.yml)
Loading

Poem

In burrows deep, the schemas grew,
With tables, fields, and mappings new.
Airbyte hops, dbt leaps high,
Transforming data as clouds drift by.
ERDs and configs, all in a row,
This Salesforce garden’s ready to grow!
🐇✨

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@aaronsteers
Copy link
Collaborator Author

aaronsteers commented Apr 23, 2025

/generate salesforce

Error: Required input 'source_name' not provided

@aaronsteers
Copy link
Collaborator Author

aaronsteers commented Apr 23, 2025

/generate source_name=salesforce

Running poe morph-build salesforce --with-generate...

Link to job logs.

✅ Poe command morph-build salesforce --with-generate completed successfully.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 18

🔭 Outside diff range comments (4)
src/transforms/salesforce/transforms/user.yml (1)

7-239: 🛠️ Refactor suggestion

Essential: All field expressions are placeholders.
Every field in this transform is still marked expression: MISSING. To enable dbt compilation and proper data extraction, we must populate these expressions with valid column references from the specified source. Consider generating a scaffold that maps each field name to source.<FieldName>.

Let me know if you’d like a script to auto-generate this mapping template.

src/transforms/salesforce/transforms/opportunity_line_item.yml (1)

7-146: 🛠️ Refactor suggestion

Essential: Placeholder expressions must be replaced.
All expression: MISSING entries need concrete column references (e.g., OpportunityLineItem.Id, OpportunityLineItem.UnitPrice, etc.). Without these, the transform cannot be executed or tested.

I can generate a baseline mapping that assigns each field to its identically named source column—let me know if you’d like that.

src/transforms/salesforce/transforms/opportunity.yml (1)

7-142: 🛠️ Refactor suggestion

Essential: Field expressions are all placeholders.
Every field mapping is set to MISSING. To make this transform functional, populate each expression with the corresponding raw column (for example, Opportunity.CloseDate, Opportunity.Amount, etc.).

I can scaffold a template mapping each field to Opportunity.<FieldName> automatically—would you like me to run that?

src/transforms/salesforce/transforms/task.yml (1)

7-223: ⚠️ Potential issue

Implement transformation expressions
All fields currently have expression: MISSING. You need to provide valid SQL or JMESPath expressions (depending on your pipeline) for each field to extract and map data.

🧹 Nitpick comments (14)
catalog/salesforce/airbyte-interop-dbt-project/poe_tasks.toml (1)

1-2: Add a brief header comment for context
Consider documenting the purpose of this file and how it fits into the larger dbt/poe workflow, so new contributors understand why these includes are necessary.

catalog/salesforce/README.md (1)

1-5: Enhance README with usage and context
Right now the README gives only a high-level note. Consider adding:

  • A brief overview of this catalog’s role in the Salesforce integration
  • Prerequisites (e.g., dbt version, environment variables)
  • Example commands to run or build the models
  • A table of contents linking to key files
catalog/salesforce/airbyte-interop-dbt-project/.gitignore (1)

1-8: Consider additional ignore patterns
The core dbt artifacts are covered, but you might also want to exclude:

  • Environment files (e.g., .env)
  • OS-specific artefacts like .DS_Store
  • Python virtual envs (.venv/) if used
  • Editor temp files (e.g., *.swp)

Adding these can prevent accidental commits of local or sensitive files.

src/transforms/salesforce/transforms/product_2.yml (1)

7-115: Implement transformation expressions.

All expression: MISSING placeholders must be replaced with valid SQL or Jinja expressions before this transform can be executed. Consider defining a template or shared mechanism to handle missing fields, or let me know if you’d like help generating a skeleton mapping for these fields.

src/transforms/salesforce/transforms/event.yml (1)

7-269: Placeholder expressions need implementation.

Currently all fields use expression: MISSING. You’ll need to supply real transformation expressions or leverage a templating approach. Let me know if you want assistance generating these mappings.

src/transforms/salesforce/transforms/user_role.yml (1)

7-52: Implement transformation expressions.

Each field currently has expression: MISSING. You will need to define valid expressions or use a templating mechanism to fill these in. If you’d like help scaffolding these mappings, let me know.

src/transforms/salesforce/transforms/order.yml (1)

7-194: Placeholder expressions need implementation.

All expression: MISSING values are placeholders and must be replaced with real transformation logic. Consider centralizing placeholder handling or generating a stub mapping; I can assist if needed.

catalog/salesforce/rejected-mappings.md (2)

7-7: Normalize heading levels.
The third-level heading (### Mapping: Airbyte \Account`) follows a first-level title, skipping an intermediate level. Change it to ## Mapping: …` to maintain a single-level increment.

🧰 Tools
🪛 markdownlint-cli2 (0.17.2)

7-7: Heading levels should only increment by one level at a time
Expected: h2; Actual: h3

(MD001, heading-increment)


14-66: Unify emphasis style to underscores.
This table uses asterisks (*italic*) for emphasis, but our Markdown linting expects underscores (_italic_). Please replace all *…* with _…_ for consistency.

🧰 Tools
🪛 LanguageTool

[style] ~60-~60: To make your text as clear as possible to all readers, do not use this foreign term. Possible alternatives are “in the original”, “thus used”, or “thus spelled”.
Context: ...| ❌ 0.00 | No good match found. | | sic | Standard Industrial Classification c...

(SIC)


[style] ~61-~61: To make your text as clear as possible to all readers, do not use this foreign term. Possible alternatives are “in the original”, “thus used”, or “thus spelled”.
Context: ...| ❌ 0.00 | No good match found. | | sic_desc | A brief description of an organ...

(SIC)

🪛 markdownlint-cli2 (0.17.2)

16-16: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


16-16: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


17-17: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


17-17: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


18-18: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


18-18: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


19-19: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


19-19: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


20-20: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


20-20: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


21-21: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


21-21: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


22-22: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


22-22: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


23-23: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


23-23: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


24-24: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


24-24: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


25-25: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


25-25: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


26-26: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


26-26: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


27-27: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


27-27: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


28-28: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


28-28: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


29-29: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


29-29: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


30-30: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


30-30: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


31-31: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


31-31: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


32-32: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


32-32: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


33-33: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


33-33: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


34-34: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


34-34: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


35-35: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


35-35: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


36-36: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


36-36: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


37-37: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


37-37: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


38-38: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


38-38: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


39-39: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


39-39: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


40-40: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


40-40: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


41-41: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


41-41: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


42-42: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


42-42: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


43-43: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


43-43: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


44-44: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


44-44: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


45-45: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


45-45: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


46-46: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


46-46: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


47-47: Bare URL used
null

(MD034, no-bare-urls)


47-47: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


47-47: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


48-48: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


48-48: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


49-49: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


49-49: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


50-50: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


50-50: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


51-51: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


51-51: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


52-52: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


52-52: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


53-53: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


53-53: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


54-54: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


54-54: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


55-55: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


55-55: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


56-56: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


56-56: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


57-57: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


57-57: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


58-58: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


58-58: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


59-59: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


59-59: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


60-60: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


60-60: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


61-61: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


61-61: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


62-62: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


62-62: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


63-63: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


63-63: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


64-64: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


64-64: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


65-65: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


65-65: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)

build/salesforce/src_airbyte_raw_salesforce.yml (2)

40-57: Normalize nested JSON subcolumn naming conventions.

Inside the BillingAddress JSON field, subcolumns use camelCase keys (e.g., postalCode) while the top-level columns use PascalCase (BillingPostalCode). Consider standardizing on one naming style to avoid confusion in downstream models:

   meta:
-    subcolumns:
-      - name: street
+    subcolumns:
+      - name: Street
       data_type: varchar

This will improve readability and reduce mapping errors.


180-188: Extract Airbyte metadata columns to a reusable anchor.

The _airbyte_extracted_at, _airbyte_meta, and _airbyte_raw_id columns are repeated for each table. To DRY this, consider defining a YAML anchor at the top of the file and reusing it:

x-airbyte-metadata: &airbyte-metadata
  - name: _airbyte_extracted_at
    description: Timestamp when the record was extracted from the source
    data_type: timestamp
  - name: _airbyte_meta
    description: Metadata about the record
    data_type: json
  - name: _airbyte_raw_id
    description: Unique identifier for the raw record
    data_type: varchar

tables:
  - name: Account
    columns:
      [ ... your columns ..., *airbyte-metadata ]

This will reduce duplication and ease future updates.

build/salesforce/airbyte-interop.morph-lock.toml (1)

27-35: Evaluate the unmapped target fields list for Account.

The unmapped_target_fields array under [mappings.account] is extensive. Consider:

  • Removing fields that are intentionally out of scope.
  • Documenting why certain fields are omitted (e.g., deprecated or not needed).
  • Breaking this list into smaller, categorized sections (address vs. financial fields).

This will make it easier to track which fields actually need mapping and which can remain excluded.

build/salesforce/erd/source-schema-erd.dbml (2)

1-2: Unify naming conventions between source and target schemas
Table and column casing (e.g., "Account" vs. account, "Id" vs. id) should be consistent across source and target DBML files to avoid confusion and reduce case-sensitivity issues in downstream tools.


1-71: Annotate primary keys in DBML
Consider marking your primary key columns (e.g., Id in Account) with [pk] so that DBML-based documentation and code generation tools accurately recognize them as primary keys.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bd553ad and 3a4a36c.

⛔ Files ignored due to path filters (1)
  • build/salesforce/erd/source-schema-erd.svg is excluded by !**/*.svg
📒 Files selected for processing (29)
  • build/salesforce/airbyte-catalog.json (1 hunks)
  • build/salesforce/airbyte-interop.morph-lock.toml (1 hunks)
  • build/salesforce/erd/source-schema-erd.dbml (1 hunks)
  • build/salesforce/erd/target-schema-erd.dbml (1 hunks)
  • build/salesforce/requirements/airbyte-interop/src_dbt_requirements.yml (1 hunks)
  • build/salesforce/src_airbyte_raw_salesforce.yml (1 hunks)
  • catalog/salesforce/README.md (1 hunks)
  • catalog/salesforce/airbyte-interop-dbt-project/.gitignore (1 hunks)
  • catalog/salesforce/airbyte-interop-dbt-project/dbt_project.yml (1 hunks)
  • catalog/salesforce/airbyte-interop-dbt-project/models/src_airbyte_raw.yml (1 hunks)
  • catalog/salesforce/airbyte-interop-dbt-project/package-lock.yml (1 hunks)
  • catalog/salesforce/airbyte-interop-dbt-project/packages.yml (1 hunks)
  • catalog/salesforce/airbyte-interop-dbt-project/poe_tasks.toml (1 hunks)
  • catalog/salesforce/airbyte-interop-dbt-project/profiles/profiles.yml (1 hunks)
  • catalog/salesforce/rejected-mappings.md (1 hunks)
  • src/morph/models.py (2 hunks)
  • src/transforms/salesforce/transforms/account.yml (1 hunks)
  • src/transforms/salesforce/transforms/contact.yml (1 hunks)
  • src/transforms/salesforce/transforms/event.yml (1 hunks)
  • src/transforms/salesforce/transforms/fivetran_formula.yml (1 hunks)
  • src/transforms/salesforce/transforms/fivetran_formula_model.yml (1 hunks)
  • src/transforms/salesforce/transforms/lead.yml (1 hunks)
  • src/transforms/salesforce/transforms/opportunity.yml (1 hunks)
  • src/transforms/salesforce/transforms/opportunity_line_item.yml (1 hunks)
  • src/transforms/salesforce/transforms/order.yml (1 hunks)
  • src/transforms/salesforce/transforms/product_2.yml (1 hunks)
  • src/transforms/salesforce/transforms/task.yml (1 hunks)
  • src/transforms/salesforce/transforms/user.yml (1 hunks)
  • src/transforms/salesforce/transforms/user_role.yml (1 hunks)
🧰 Additional context used
🪛 LanguageTool
catalog/salesforce/rejected-mappings.md

[style] ~60-~60: To make your text as clear as possible to all readers, do not use this foreign term. Possible alternatives are “in the original”, “thus used”, or “thus spelled”.
Context: ...| ❌ 0.00 | No good match found. | | sic | Standard Industrial Classification c...

(SIC)


[style] ~61-~61: To make your text as clear as possible to all readers, do not use this foreign term. Possible alternatives are “in the original”, “thus used”, or “thus spelled”.
Context: ...| ❌ 0.00 | No good match found. | | sic_desc | A brief description of an organ...

(SIC)


[style] ~72-~72: This phrase is redundant. Consider using “subject” to avoid wordiness.
Context: ...rce and target tables describe the same subject matter, however, the majority of fields are la...

(SUBJECT_MATTER)

🪛 markdownlint-cli2 (0.17.2)
catalog/salesforce/rejected-mappings.md

7-7: Heading levels should only increment by one level at a time
Expected: h2; Actual: h3

(MD001, heading-increment)


16-16: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


16-16: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


17-17: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


17-17: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


18-18: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


18-18: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


19-19: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


19-19: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


20-20: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


20-20: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


21-21: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


21-21: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


22-22: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


22-22: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


23-23: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


23-23: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


24-24: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


24-24: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


25-25: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


25-25: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


26-26: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


26-26: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


27-27: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


27-27: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


28-28: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


28-28: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


29-29: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


29-29: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


30-30: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


30-30: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


31-31: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


31-31: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


32-32: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


32-32: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


33-33: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


33-33: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


34-34: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


34-34: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


35-35: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


35-35: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


36-36: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


36-36: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


37-37: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


37-37: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


38-38: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


38-38: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


39-39: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


39-39: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


40-40: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


40-40: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


41-41: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


41-41: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


42-42: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


42-42: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


43-43: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


43-43: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


44-44: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


44-44: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


45-45: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


45-45: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


46-46: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


46-46: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


47-47: Bare URL used
null

(MD034, no-bare-urls)


47-47: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


47-47: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


48-48: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


48-48: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


49-49: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


49-49: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


50-50: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


50-50: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


51-51: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


51-51: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


52-52: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


52-52: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


53-53: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


53-53: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


54-54: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


54-54: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


55-55: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


55-55: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


56-56: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


56-56: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


57-57: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


57-57: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


58-58: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


58-58: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


59-59: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


59-59: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


60-60: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


60-60: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


61-61: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


61-61: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


62-62: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


62-62: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


63-63: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


63-63: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


64-64: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


64-64: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


65-65: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


65-65: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


76-76: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


76-76: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


77-77: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


77-77: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


78-78: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


78-78: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


79-79: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


79-79: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


80-80: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


80-80: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


81-81: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


81-81: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


82-82: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


82-82: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


83-83: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


83-83: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


84-84: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


84-84: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


85-85: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


85-85: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


86-86: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


86-86: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


87-87: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


87-87: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


88-88: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


88-88: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


89-89: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


89-89: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


90-90: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


90-90: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


91-91: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


91-91: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


92-92: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


92-92: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


93-93: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


93-93: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


94-94: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


94-94: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


95-95: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


95-95: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


96-96: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


96-96: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


97-97: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


97-97: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


98-98: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


98-98: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


99-99: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


99-99: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


100-100: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


100-100: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


101-101: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


101-101: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


102-102: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


102-102: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


103-103: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


103-103: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


104-104: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


104-104: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


105-105: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


105-105: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


106-106: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


106-106: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


107-107: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


107-107: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


108-108: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


108-108: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


109-109: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


109-109: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


110-110: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


110-110: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


111-111: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


111-111: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


112-112: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


112-112: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


113-113: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


113-113: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


114-114: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


114-114: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


115-115: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


115-115: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


116-116: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


116-116: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


117-117: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


117-117: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


118-118: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


118-118: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


119-119: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


119-119: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


120-120: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


120-120: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


121-121: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


121-121: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


122-122: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


122-122: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


123-123: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


123-123: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


124-124: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


124-124: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


125-125: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


125-125: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


126-126: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


126-126: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


127-127: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


127-127: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


128-128: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)


128-128: Emphasis style
Expected: underscore; Actual: asterisk

(MD049, emphasis-style)

🪛 YAMLlint (1.35.1)
build/salesforce/requirements/airbyte-interop/src_dbt_requirements.yml

[error] 10-10: trailing spaces

(trailing-spaces)


[warning] 21-21: wrong indentation: expected 10 but found 8

(indentation)


[warning] 127-127: wrong indentation: expected 10 but found 8

(indentation)


[warning] 207-207: wrong indentation: expected 10 but found 8

(indentation)


[warning] 345-345: wrong indentation: expected 10 but found 8

(indentation)


[warning] 380-380: wrong indentation: expected 10 but found 8

(indentation)


[error] 427-427: trailing spaces

(trailing-spaces)


[error] 450-450: trailing spaces

(trailing-spaces)


[error] 476-476: trailing spaces

(trailing-spaces)


[error] 501-501: trailing spaces

(trailing-spaces)


[error] 502-502: trailing spaces

(trailing-spaces)


[error] 503-503: trailing spaces

(trailing-spaces)


[error] 506-506: trailing spaces

(trailing-spaces)


[warning] 521-521: wrong indentation: expected 10 but found 8

(indentation)


[error] 617-617: trailing spaces

(trailing-spaces)


[error] 644-644: trailing spaces

(trailing-spaces)


[error] 649-649: trailing spaces

(trailing-spaces)


[error] 663-663: trailing spaces

(trailing-spaces)


[warning] 669-669: wrong indentation: expected 10 but found 8

(indentation)


[error] 708-708: trailing spaces

(trailing-spaces)


[error] 726-726: trailing spaces

(trailing-spaces)


[error] 736-736: trailing spaces

(trailing-spaces)


[error] 750-750: trailing spaces

(trailing-spaces)


[error] 758-758: trailing spaces

(trailing-spaces)


[error] 777-777: trailing spaces

(trailing-spaces)


[warning] 783-783: wrong indentation: expected 10 but found 8

(indentation)


[error] 801-801: trailing spaces

(trailing-spaces)


[error] 857-857: trailing spaces

(trailing-spaces)


[warning] 865-865: wrong indentation: expected 10 but found 8

(indentation)


[error] 900-900: trailing spaces

(trailing-spaces)


[error] 914-914: trailing spaces

(trailing-spaces)


[error] 940-940: trailing spaces

(trailing-spaces)


[error] 954-954: trailing spaces

(trailing-spaces)


[error] 956-956: trailing spaces

(trailing-spaces)


[error] 962-962: trailing spaces

(trailing-spaces)


[error] 965-965: trailing spaces

(trailing-spaces)


[warning] 971-971: wrong indentation: expected 10 but found 8

(indentation)


[error] 1028-1028: trailing spaces

(trailing-spaces)


[error] 1029-1029: trailing spaces

(trailing-spaces)


[warning] 1035-1035: wrong indentation: expected 10 but found 8

(indentation)


[error] 1046-1046: trailing spaces

(trailing-spaces)


[error] 1112-1112: trailing spaces

(trailing-spaces)


[error] 1124-1124: trailing spaces

(trailing-spaces)


[error] 1130-1130: no new line character at the end of file

(new-line-at-end-of-file)

🔇 Additional comments (12)
catalog/salesforce/airbyte-interop-dbt-project/poe_tasks.toml (1)

3-6: Validate that ${POE_GIT_ROOT} is set in all environments
The include paths assume POE_GIT_ROOT is correctly defined. To avoid silent failures, add a pre-flight check in your CI or local setup to ensure this environment variable is present and points to the root of your poe task definitions.

Example shell snippet for CI:

#!/usr/bin/env bash
if [[ -z "${POE_GIT_ROOT}" ]]; then
  echo "ERROR: POE_GIT_ROOT is not defined. Please set this env var to the root of your poe tasks."
  exit 1
fi
catalog/salesforce/airbyte-interop-dbt-project/package-lock.yml (1)

1-4: Verify sha1_hash matches the locked package version
Ensure the SHA-1 hash corresponds exactly to the dbt_utils 1.1.1 release from the dbt Hub. A mismatch can lead to integrity errors during package installation.

catalog/salesforce/airbyte-interop-dbt-project/packages.yml (1)

1-4: Dependency declaration looks good
The dbt_utils package at version 1.1.1 is correctly declared. Make sure this aligns with the version locked in package-lock.yml to maintain consistency.

catalog/salesforce/airbyte-interop-dbt-project/profiles/profiles.yml (1)

1-19: LGTM: Well-structured dbt profile configuration.

The file correctly defines both local DuckDB and cloud MotherDuck targets with appropriate extensions and configuration.

catalog/salesforce/airbyte-interop-dbt-project/dbt_project.yml (1)

1-29: LGTM: Well-structured dbt project configuration.

The project configuration follows best practices with clear definition of project name, paths, materialization settings, and variables.

src/morph/models.py (2)

191-192: Improved robustness by handling omitted columns.

This change addresses the PR objective by making the code more resilient when processing input dictionaries that omit the "columns" key.

The change from direct dictionary access data["columns"] to using the safer data.get("columns", []) method with a default empty list prevents potential KeyError exceptions.


213-213: Added type annotation for clarity.

Good addition of explicit type annotation for the columns variable for better code readability and static type checking.

catalog/salesforce/airbyte-interop-dbt-project/models/src_airbyte_raw.yml (1)

1-9: Auto-generated source catalog looks good.

This file is generated by the morph CLI and matches the expected schema definitions. No manual edits are required.

src/transforms/salesforce/transforms/lead.yml (1)

5-6: Verify source table reference.
The from block uses LeadHistory: salesforce.LeadHistory. Please confirm that the Airbyte raw catalog defines a stream named LeadHistory under the salesforce namespace. If this stream is renamed or missing, the transform will fail at runtime.

build/salesforce/src_airbyte_raw_salesforce.yml (1)

6-8: Ensure Jinja variable defaults are correct for target environments.

The schema and database properties use Jinja templating with var(...) defaults. Please verify that:

  1. The variable names (airbyte_salesforce_schema, airbyte_salesforce_database) match those defined in your dbt_project.yml or profiles.yml.
  2. The fallback defaults (salesforce_raw, salesforce) are the intended values in all deployment environments.

If needed, add comments or documentation pointing to where these variables are set.

build/salesforce/airbyte-interop.morph-lock.toml (1)

1-8:

✅ Verification successful

Verify integrity hashes for requirements and source files.

The requirements_file_hash and airbyte_source_file_hash are critical for ensuring consistency. Please confirm that these SHA256 hashes match the current contents of:

  • build/salesforce/requirements/airbyte-interop/src_dbt_requirements.yml
  • build/salesforce/src_airbyte_raw_salesforce.yml

You can re-generate them using:


🏁 Script executed:

sha256sum build/salesforce/requirements/airbyte-interop/src_dbt_requirements.yml
sha256sum build/salesforce/src_airbyte_raw_salesforce.yml

Length of output: 342


🏁 Script executed:

#!/usr/bin/env bash
# Compute SHA256 hashes using Python since sha256sum is unavailable
python3 - << 'EOF'
import hashlib
paths = [
    "build/salesforce/requirements/airbyte-interop/src_dbt_requirements.yml",
    "build/salesforce/src_airbyte_raw_salesforce.yml"
]
for p in paths:
    try:
        data = open(p, "rb").read()
        print(f"{hashlib.sha256(data).hexdigest()}  {p}")
    except Exception as e:
        print(f"ERROR reading {p}: {e}")
EOF

Length of output: 262


Integrity Hashes Verified

I’ve confirmed that the SHA256 checksums for both files match the entries in airbyte-interop.morph-lock.toml:

  • build/salesforce/requirements/airbyte-interop/src_dbt_requirements.yml → 17d08bfe527e469ace42ebd0b7ae9a01b379527b7f3719267b9daa2b04fff312
  • build/salesforce/src_airbyte_raw_salesforce.yml → 1de7f444cc079dd197a5a5c6715af205b3fc8e2e44b93752ec0a6e127c2256a0

No further action required.

build/salesforce/airbyte-catalog.json (1)

1-5: This file is auto-generated via the Morph CLI. Manual edits are discouraged; please adjust the generation templates if schema changes are needed.

Comment on lines +1 to +6
domain: salesforce.airbyte-interop
transforms:
- name: fivetran_formula
from:
- MISSING: salesforce.MISSING
fields: {}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

This file needs implementation details.

The transform definition is missing an actual source reference (using a placeholder "MISSING") and doesn't define any fields, making it just a skeleton.

Make sure to update this with:

  1. A proper source reference instead of "MISSING: salesforce.MISSING"
  2. Field definitions in the fields object

🏁 Script executed:

#!/bin/bash
# Check for other Salesforce transform files that could serve as examples
ls -l src/transforms/salesforce/transforms/

Length of output: 894


Implement missing details in fivetran_formula.yml

The current transform is just a stub—it needs a real source reference and field mappings:

• File: src/transforms/salesforce/transforms/fivetran_formula.yml

  • Replace
    from:
      - MISSING: salesforce.MISSING
    with the correct source, for example:
    from:
      - salesforce.fivetran_formula
  • Populate the fields array with your field definitions, e.g.:
    fields:
      - name: Id
        field_type: STRING
      - name: CreatedDate
        field_type: TIMESTAMP
      # …etc.

See contact.yml or account.yml in the same directory for examples of how to structure your from and fields blocks.

Comment on lines +1 to +6
domain: salesforce.airbyte-interop
transforms:
- name: fivetran_formula_model
from:
- MISSING: salesforce.MISSING
fields: {}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

This file needs implementation details.

Similar to fivetran_formula.yml, this transform definition is using a placeholder source reference and has no fields defined.

Update this with:

  1. A proper source reference instead of "MISSING: salesforce.MISSING"
  2. Field definitions in the fields object

This appears to be a companion to fivetran_formula.yml and both should be implemented together.

Comment on lines +4 to +6
from:
- MISSING: salesforce.MISSING
fields:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Invalid 'from' mapping configuration.

The placeholder - MISSING: salesforce.MISSING is not a valid YAML list item and will break the transform parser. You need to reference the actual source table or Airbyte stream name, for example:

from:
  - source_table: salesforce.Product2

Comment on lines +4 to +6
from:
- MISSING: salesforce.MISSING
fields:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Invalid 'from' mapping placeholder.

Using - MISSING: salesforce.MISSING is not valid YAML and will cause parsing errors. Please specify the proper source, for example:

from:
  - source_table: salesforce.Event

Comment on lines +4 to +6
from:
- MISSING: salesforce.MISSING
fields:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Invalid 'from' mapping placeholder.

The placeholder - MISSING: salesforce.MISSING is not a valid YAML list item. Please replace it with the correct source reference, such as:

from:
  - source_table: salesforce.UserRole

schema: "{{ var('salesforce_schema', 'salesforce') }}" # you will need to customize if your schema name is not salesforce
database: "{% if target.type != 'spark'%}{{ var('salesforce_database', target.database) }}{% endif %}"
loader: fivetran
loaded_at_field: _fivetran_synced
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Fix YAML formatting: remove trailing spaces and correct indentation
Remove unnecessary trailing spaces (e.g., line 10) and align nested columns: and list items under tables to 10 spaces to satisfy YAMLlint rules. Also ensure the file ends with a newline.

Apply diffs similar to:

-      warn_after: {count: 24, period: hour}␠␠
+      warn_after: {count: 24, period: hour}

-        columns:
-        - name: id
+          columns:
+            - name: id

+EOF

Also applies to: 21-21, 127-127, 207-207, 345-345, 380-380, 1130-1130

🧰 Tools
🪛 YAMLlint (1.35.1)

[error] 10-10: trailing spaces

(trailing-spaces)

Comment on lines +4 to +6
from:
- MISSING: salesforce.MISSING
fields:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Define the source in the from clause
The placeholder MISSING: salesforce.MISSING must be replaced with the actual source stream reference (e.g., salesforce.task). Without this, the transform cannot resolve its input.

Comment on lines +1 to +52
Table "account" {
"id" unknown [note: 'The unique, system-generated ID assigned during creation']
"_fivetran_synced" unknown [note: 'The time at which fivetran last synced this record']
"_fivetran_active" unknown [note: 'True if record is active, used to filter out only active records if History Mode is enabled']
"account_number" unknown [note: 'Account number assigned to this account (not the unique, system-generated ID assigned during creation).']
"account_source" unknown [note: 'The source of the account record. For example, Advertisement, Data.com, or Trade Show.']
"annual_revenue" unknown [note: 'Estimated annual revenue of the account.']
"billing_city" unknown [note: 'Details for the billing address of this account.']
"billing_country" unknown [note: 'Details for the billing address of this account.']
"billing_country_code" unknown [note: 'The ISO country code for the account’s billing address.']
"billing_geocode_accuracy" unknown [note: 'Accuracy level of the geocode for the billing address. See Compound Field Considerations and Limitations for details on geolocation compound fields.']
"billing_latitude" unknown [note: 'Used with BillingLongitude to specify the precise geolocation of a billing address.']
"billing_longitude" unknown [note: 'Used with BillingLatitude to specify the precise geolocation of a billing address.']
"billing_postal_code" unknown [note: 'Details for the billing address of this account.']
"billing_state" unknown [note: 'Details for the billing address of this account.']
"billing_state_code" unknown [note: 'The ISO state code for the account’s billing address.']
"billing_street" unknown [note: 'Street address for the billing address of this account.']
"description" unknown [note: 'Text description of the account.']
"fax" unknown [note: 'Fax number for the account.']
"industry" unknown [note: 'An industry associated with this account.']
"is_deleted" unknown [note: 'Indicates whether the object has been moved to the Recycle Bin (true) or not (false).']
"jigsaw_company_id" unknown [note: 'References the ID of a company in Data.com. If an account has a value in this field, it means that the account was imported from Data.com.']
"last_activity_date" unknown [note: 'Value is one of the following, whichever is the most recent']
"last_referenced_date" unknown [note: 'The timestamp when the current user last accessed this record, a record related to this record, or a list view.']
"last_viewed_date" unknown [note: 'The timestamp when the current user last viewed this record or list view. If this value is null, the user might have only accessed this record or list view (LastReferencedDate) but not viewed it.']
"master_record_id" unknown [note: 'If this object was deleted as the result of a merge, this field contains the ID of the record that was kept. If this object was deleted for any other reason, or has not been deleted, the value is null.']
"name" unknown [note: 'Required. Name of the account.']
"number_of_employees" unknown [note: 'Number of employees working at the company represented by this account.']
"owner_id" unknown [note: 'The ID of the user who currently owns this account.']
"ownership" unknown [note: 'Ownership type for the account, for example Private, Public, or Subsidiary.']
"parent_id" unknown [note: 'ID of the parent object, if any.']
"phone" unknown [note: 'Phone number for this account.']
"photo_url" unknown [note: 'Path to be combined with the URL of a Salesforce instance (for example, https://yourInstance.salesforce.com/) to generate a URL to request the social network profile image associated with the account.']
"rating" unknown [note: 'The account’s prospect rating, for example Hot, Warm, or Cold.']
"record_type_id" unknown [note: 'ID of the record type assigned to this object.']
"shipping_city" unknown [note: 'Details of the shipping address for this account']
"shipping_country" unknown [note: 'Details of the shipping address for this account. Country']
"shipping_country_code" unknown [note: 'The ISO country code for the account’s shipping address.']
"shipping_geocode_accuracy" unknown [note: 'Accuracy level of the geocode for the shipping address.']
"shipping_latitude" unknown [note: 'Used with ShippingLongitude to specify the precise geolocation of a shipping address.']
"shipping_longitude" unknown [note: 'Used with ShippingLatitude to specify the precise geolocation of an address.']
"shipping_postal_code" unknown [note: 'Details of the shipping address for this account. Postal code']
"shipping_state" unknown [note: 'Details of the shipping address for this account. State']
"shipping_state_code" unknown [note: 'The ISO state code for the account’s shipping address.']
"shipping_street" unknown [note: 'The street address of the shipping address for this account.']
"sic" unknown [note: 'Standard Industrial Classification code of the company’s main business categorization, for example, 57340 for Electronics.']
"sic_desc" unknown [note: 'A brief description of an organization’s line of business, based on its SIC code.']
"site" unknown [note: 'Name of the account’s location, for example Headquarters or London.']
"ticker_symbol" unknown [note: 'The stock market symbol for this account. This field is available on business accounts, not person accounts.']
"type" unknown [note: 'Type of account, for example, Customer, Competitor, or Partner.']
"website" unknown [note: 'The website of this account.']
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Specify concrete data types instead of unknown
Every column is declared with type unknown. DBML requires valid types (e.g., varchar, timestamp, integer). Update each table definition to use the appropriate SQL types for your target database.

Comment on lines +249 to +254
Table "event" {
"id" unknown [note: 'The unique, system-generated ID assigned during creation.']
"_fivetran_synced" unknown [note: 'The time at which fivetran last synced this record']
"_fivetran_active" unknown [note: 'True if record is active, used to filter out only active records if History Mode is enabled']
"account_id" unknown [note: '''Represents the ID of the related account. The AccountId is determined as follows. If the value of WhatId is any of the following objects, then Salesforce uses that object’s AccountId.
Account Opportunity Contract Custom object that is a child of Account If the value of the WhatId field is any other object, and the value of the WhoId field is a contact object, then Salesforce uses that contact’s AccountId. (If your org uses Shared Activities, Salesforce uses the AccountId of the primary contact.)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Remove duplicate id column in event table
The event table defines "id" twice, leading to ambiguity in the schema. Consolidate into a single primary key definition.

Comment on lines +4 to +6
from:
- MISSING: salesforce.MISSING
fields:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Define the source in the from clause
The placeholder MISSING: salesforce.MISSING must be replaced with the actual source stream reference (e.g., salesforce.contact). Without this, the transform cannot resolve its input.

@aaronsteers aaronsteers changed the title fix: handle omitted columns list for salesforce fix: handle omitted columns list, add salesforce Apr 23, 2025
@aaronsteers aaronsteers merged commit d874d27 into main Apr 23, 2025
9 checks passed
@aaronsteers aaronsteers deleted the fix/handle-omitted-target-columns-list-for-salesforce branch April 23, 2025 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant