Support atmos docs generate feature natively #934
Conversation
TODO
|
mergify
bot
added
conflict
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (5)
internal/exec/docs_generate.go (5)
104-104: 🛠️ Refactor suggestionConsider making 'README.yaml' filename configurable
Instead of hardcoding 'README.yaml', allow the filename to come from the configuration for greater flexibility.
117-117: 🛠️ Refactor suggestionUpdate error message for clarity
Change the error message to reflect that we're generating Terraform docs within atmos, not running an external command.
-return fmt.Errorf("failed to run terraform-docs: %w", err) +return fmt.Errorf("failed to generate terraform docs: %w", err)
156-169: 🛠️ Refactor suggestionReuse existing HTTP fetch methods
Use the existing methods (like
go-getter) for HTTP fetching to handle authentication (e.g.,GITHUB_TOKEN) and avoid duplicating code.
239-239: 🛠️ Refactor suggestionUse consistent HTTP fetch methods for templates
Consider using the same HTTP fetching mechanisms used elsewhere to handle authentication and maintain consistency.
276-295: 🛠️ Refactor suggestionLeverage existing deep merge functionality
Reuse atmos's native deep merging capabilities (possibly using
mergo) instead of implementing a customdeepMergefunction.
🧹 Nitpick comments (6)
cmd/docs_generate.go (1)
9-9: Fix typo in commentCorrect the typo: "genrates" should be "generates".
-// docsGenerateCmd genrates README.md +// docsGenerateCmd generates README.mdinternal/exec/template_utils.go (2)
226-234: Add function documentation.Consider adding a detailed function documentation comment that explains:
- The purpose of the function
- Parameter descriptions
- Return value descriptions
- Usage examples
+// ProcessTmplWithDatasourcesGomplate parses and executes Go templates with datasources using Gomplate. +// It processes the template specified by tmplName and tmplValue using the provided merged data. +// +// Parameters: +// - atmosConfig: The Atmos configuration +// - settingsSection: Settings for template processing +// - tmplName: Name of the template +// - tmplValue: The template content +// - mergedData: Data to be used in template processing +// - ignoreMissingTemplateValues: If true, missing values are replaced with empty strings +// +// Returns: +// - string: The processed template +// - error: Any error that occurred during processing func ProcessTmplWithDatasourcesGomplate(
246-259: Enhance error handling for temporary file operations.The temporary file operations could benefit from more robust cleanup and detailed error messages.
tmpfile, err := os.CreateTemp("", "gomplate-data-*.json") if err != nil { - return "", fmt.Errorf("failed to create temp data file for gomplate: %w", err) + return "", fmt.Errorf("failed to create temporary data file for gomplate (pattern: gomplate-data-*.json): %w", err) } tmpName := tmpfile.Name() -defer os.Remove(tmpName) +defer func() { + if err := os.Remove(tmpName); err != nil { + // Log the error but don't fail the operation + u.LogWarning(atmosConfig, fmt.Sprintf("failed to remove temporary file %s: %v", tmpName, err)) + } +}()pkg/schema/schema.go (2)
195-204: Add field documentation for TerraformDocsSettings.Consider adding documentation comments for each field to explain their purpose and valid values.
type TerraformDocsSettings struct { + // Enabled determines if Terraform documentation generation is active Enabled bool `yaml:"enabled,omitempty" json:"enabled,omitempty" mapstructure:"enabled,omitempty"` + // Format specifies the output format (e.g., "markdown", "asciidoc") Format string `yaml:"format,omitempty" json:"format,omitempty" mapstructure:"format,omitempty"` + // ShowProviders controls whether provider documentation is included ShowProviders bool `yaml:"show_providers,omitempty" json:"show_providers,omitempty" mapstructure:"show_providers,omitempty"` + // ShowInputs controls whether input variable documentation is included ShowInputs bool `yaml:"show_inputs,omitempty" json:"show_inputs,omitempty" mapstructure:"show_inputs,omitempty"` + // ShowOutputs controls whether output variable documentation is included ShowOutputs bool `yaml:"show_outputs,omitempty" json:"show_outputs,omitempty" mapstructure:"show_outputs,omitempty"` + // SortBy defines the sorting criteria for documentation elements (e.g., "name", "required") SortBy string `yaml:"sort_by,omitempty" json:"sort_by,omitempty" mapstructure:"sort_by,omitempty"` + // HideEmpty determines if empty sections should be omitted from the output HideEmpty bool `yaml:"hide_empty,omitempty" json:"hide_empty,omitempty" mapstructure:"hide_empty,omitempty"` + // IndentLevel specifies the number of spaces to use for indentation IndentLevel int `yaml:"indent_level,omitempty" json:"indent_level,omitempty" mapstructure:"indent_level,omitempty"` }
206-211: Enhance DocsGenerate struct with documentation and validation.Consider adding field documentation and validation tags to ensure proper usage.
type DocsGenerate struct { + // Input is a list of paths to input files (e.g., README.yaml) + // At least one input file must be specified Input []string `yaml:"input,omitempty" json:"input,omitempty" mapstructure:"input"` + // Template is a list of template files to be used for generation + // At least one template file must be specified Template []string `yaml:"template,omitempty" json:"template,omitempty" mapstructure:"template"` + // Output specifies the path where the generated documentation will be written Output string `yaml:"output,omitempty" json:"output,omitempty" mapstructure:"output"` + // Terraform contains settings specific to Terraform documentation generation Terraform TerraformDocsSettings `yaml:"terraform,omitempty" json:"terraform,omitempty" mapstructure:"terraform"` }atmos.yaml (1)
398-404: Consider making paths more configurable.The hardcoded paths could be made more flexible by supporting:
- Environment variable substitution
- Relative path resolution
- Multiple file patterns
input: - - "./README.yaml" + - "${ATMOS_README_INPUT:-./README.yaml}" + - "**/*.yaml" # Support recursive search template: - - "./README.md.gotmpl" + - "${ATMOS_README_TEMPLATE:-./README.md.gotmpl}" + - "https://raw.githubusercontent.com/org/repo/main/templates/*.gotmpl" output: "README.md"
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (7)
atmos.yaml(1 hunks)cmd/docs_generate.go(1 hunks)go.mod(8 hunks)internal/exec/docs_generate.go(1 hunks)internal/exec/file_utils.go(2 hunks)internal/exec/template_utils.go(8 hunks)pkg/schema/schema.go(2 hunks)
🧰 Additional context used
🪛 GitHub Check: Build (macos-latest, macos)
internal/exec/file_utils.go
[failure] 7-7:
url redeclared in this block
[failure] 7-7:
"net/url" imported and not used
[failure] 12-12:
filepath redeclared in this block
[failure] 9-9:
other declaration of filepath
[failure] 12-12:
"path/filepath" imported and not used
[failure] 13-13:
runtime redeclared in this block
[failure] 10-10:
other declaration of runtime
[failure] 13-13:
"runtime" imported and not used
[failure] 14-14:
strings redeclared in this block
🪛 GitHub Check: Build (windows-latest, windows)
internal/exec/file_utils.go
[failure] 7-7:
url redeclared in this block
[failure] 7-7:
"net/url" imported and not used
[failure] 12-12:
filepath redeclared in this block
[failure] 9-9:
other declaration of filepath
[failure] 12-12:
"path/filepath" imported and not used
[failure] 13-13:
runtime redeclared in this block
[failure] 10-10:
other declaration of runtime
[failure] 13-13:
"runtime" imported and not used
[failure] 14-14:
strings redeclared in this block
🪛 GitHub Check: Build (ubuntu-latest, linux)
internal/exec/file_utils.go
[failure] 7-7:
url redeclared in this block
[failure] 7-7:
"net/url" imported and not used
[failure] 12-12:
filepath redeclared in this block
[failure] 9-9:
other declaration of filepath
[failure] 12-12:
"path/filepath" imported and not used
[failure] 13-13:
runtime redeclared in this block
[failure] 10-10:
other declaration of runtime
[failure] 13-13:
"runtime" imported and not used
[failure] 14-14:
strings redeclared in this block
🪛 GitHub Actions: Tests
internal/exec/file_utils.go
[error] 7-7: url redeclared in this block
go.mod
[error] 53-69: Module dependencies are not properly ordered. Run 'go mod tidy' to fix the ordering and commit the changes.
🪛 yamllint (1.35.1)
atmos.yaml
[error] 413-413: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Summary
🔇 Additional comments (3)
cmd/docs_generate.go (1)
23-32: Enhance readability of command examplesWrap command examples in code fences for better formatting in the help output.
Examples: - Generate a README.md in the current path: + ``` atmos docs generate + ``` - Generate a README.md for the VPC component: + ``` atmos docs generate components/terraform/vpc + ``` - Generate all README.md (recursively searches for README.yaml to rebuild docs): + ``` atmos docs generate --all + ```atmos.yaml (2)
397-398: Address TODO comment regarding README.yaml location logic.The TODO comment indicates missing functionality for finding README.yaml files. Consider implementing this before merging.
Would you like me to help implement the logic for finding README.yaml files based on the repository owner/organization structure?
400-402: Address TODO comment regarding remote templates.The TODO comment suggests that remote template support is planned. Consider implementing this capability now.
Would you like me to help implement support for remote template URLs?
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
internal/exec/file_utils.go (2)
101-124: Strong implementation with clear documentation! 💪The function effectively handles path conversion for both Windows and Linux systems, with excellent documentation and examples.
Consider optimizing the Windows path handling.
The Windows path handling could be simplified using an unconditional
TrimPrefix:- if strings.HasPrefix(pathSlashed, "/") { - pathSlashed = strings.TrimPrefix(pathSlashed, "/") - } + pathSlashed = strings.TrimPrefix(pathSlashed, "/")🧰 Tools
🪛 golangci-lint (1.62.2)
112-112: S1017: should replace this
ifstatement with an unconditionalstrings.TrimPrefix(gosimple)
126-142: Consider making the drive letter check more explicit! 🎯The drive letter check on line 136 could be more explicit about its purpose:
- if strings.HasPrefix(u.Path, "/") && len(u.Path) > 2 && u.Path[2] == ':' { + // Check if path starts with a drive letter (e.g., "/C:/...") + if strings.HasPrefix(u.Path, "/") && len(u.Path) > 2 && + ((u.Path[1] >= 'A' && u.Path[1] <= 'Z') || (u.Path[1] >= 'a' && u.Path[1] <= 'z')) && + u.Path[2] == ':' {
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (2)
go.mod(9 hunks)internal/exec/file_utils.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- go.mod
🧰 Additional context used
🪛 golangci-lint (1.62.2)
internal/exec/file_utils.go
112-112: S1017: should replace this if statement with an unconditional strings.TrimPrefix
(gosimple)
⏰ Context from checks skipped due to timeout of 90000ms (2)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Summary
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (7)
tests/fixtures/scenarios/docs-generate/terraform/atmos.yaml (2)
32-32: Fix spacing after colon in template key.Line 32 has two spaces after the colon, which is inconsistent with YAML style. It should be a single space:
- template: "./README.md.gotmpl" + template: "./README.md.gotmpl"🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 32-32: too many spaces after colon
(colons)
44-44: Remove trailing whitespace.Line 44 ends with an extra space. Trim the trailing space so it reads:
- indent_level: 2 + indent_level: 2🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 44-44: trailing spaces
(trailing-spaces)
tests/fixtures/scenarios/docs-generate/input-variants/atmos.yaml (3)
30-30: Remove trailing whitespace in input list.Line 30 has extra spaces at the end. Trim it to:
- - "./README-local.yaml" + - "./README-local.yaml"🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 30-30: trailing spaces
(trailing-spaces)
36-36: Fix spacing after colon in template key.Line 36 contains two spaces after the colon. Update to a single space:
- template: "./README.md.gotmpl" + template: "./README.md.gotmpl"🧰 Tools
🪛 YAMLlint (1.35.1)
[warning] 36-36: too many spaces after colon
(colons)
39-39: Add newline at end of file.The file is missing a terminating newline. Add a blank line at the end to comply with POSIX standards.
tests/test-cases/docs-generate.yaml (2)
23-23: Trim trailing spaces.There's trailing whitespace after
"## Providers". Remove it:- - "## Providers" + - "## Providers"🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 23-23: trailing spaces
(trailing-spaces)
72-72: Ensure newline at end of file and remove trailing spaces.The YAML file should end with a single newline and no trailing spaces on the last line. Please add the newline and strip any trailing whitespace.
🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 72-72: no new line character at the end of file
(new-line-at-end-of-file)
[error] 72-72: trailing spaces
(trailing-spaces)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
internal/exec/docs_generate.go(1 hunks)internal/exec/docs_generate_test.go(1 hunks)internal/exec/error.go(1 hunks)pkg/config/default.go(1 hunks)pkg/schema/schema.go(4 hunks)tests/fixtures/scenarios/docs-generate/input-variants/README-local.yaml(1 hunks)tests/fixtures/scenarios/docs-generate/input-variants/README.md.gotmpl(1 hunks)tests/fixtures/scenarios/docs-generate/input-variants/atmos.yaml(1 hunks)tests/fixtures/scenarios/docs-generate/terraform/README.md.gotmpl(1 hunks)tests/fixtures/scenarios/docs-generate/terraform/README.yaml(1 hunks)tests/fixtures/scenarios/docs-generate/terraform/atmos.yaml(1 hunks)tests/test-cases/docs-generate.yaml(1 hunks)
✅ Files skipped from review due to trivial changes (4)
- tests/fixtures/scenarios/docs-generate/terraform/README.yaml
- tests/fixtures/scenarios/docs-generate/input-variants/README.md.gotmpl
- tests/fixtures/scenarios/docs-generate/input-variants/README-local.yaml
- tests/fixtures/scenarios/docs-generate/terraform/README.md.gotmpl
🚧 Files skipped from review as they are similar to previous changes (4)
- internal/exec/docs_generate_test.go
- pkg/schema/schema.go
- pkg/config/default.go
- internal/exec/docs_generate.go
🧰 Additional context used
🧠 Learnings (1)
tests/fixtures/scenarios/docs-generate/input-variants/atmos.yaml (1)
Learnt from: RoseSecurity
PR: cloudposse/atmos#797
File: pkg/list/atmos.yaml:213-214
Timestamp: 2024-11-25T17:17:15.703Z
Learning: The file `pkg/list/atmos.yaml` is primarily intended for testing purposes.
🪛 YAMLlint (1.35.1)
tests/fixtures/scenarios/docs-generate/input-variants/atmos.yaml
[error] 30-30: trailing spaces
(trailing-spaces)
[warning] 36-36: too many spaces after colon
(colons)
tests/fixtures/scenarios/docs-generate/terraform/atmos.yaml
[warning] 32-32: too many spaces after colon
(colons)
[error] 44-44: trailing spaces
(trailing-spaces)
tests/test-cases/docs-generate.yaml
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 72-72: no new line character at the end of file
(new-line-at-end-of-file)
[error] 72-72: trailing spaces
(trailing-spaces)
⏰ Context from checks skipped due to timeout of 90000ms (16)
- GitHub Check: [mock-macos] tests/fixtures/scenarios/complete
- GitHub Check: [mock-macos] examples/demo-vendoring
- GitHub Check: [mock-windows] tests/fixtures/scenarios/complete
- GitHub Check: [mock-windows] examples/demo-vendoring
- GitHub Check: [mock-windows] examples/demo-context
- GitHub Check: [mock-windows] examples/demo-component-versions
- GitHub Check: [mock-windows] examples/demo-atlantis
- GitHub Check: [mock-linux] tests/fixtures/scenarios/complete
- GitHub Check: [mock-linux] examples/demo-vendoring
- GitHub Check: [mock-linux] examples/demo-context
- GitHub Check: Acceptance Tests (macos-latest, macos)
- GitHub Check: Acceptance Tests (windows-latest, windows)
- GitHub Check: Acceptance Tests (ubuntu-latest, linux)
- GitHub Check: [localstack] demo-localstack
- GitHub Check: [lint] Dockerfile
- GitHub Check: [k3s] demo-helmfile
🔇 Additional comments (3)
internal/exec/error.go (3)
24-32: Properly defined error variables for the docs generate feature.The new error variables are clear, concise, and cover the key failure scenarios for the documentation generation workflow. These errors align well with the PR objective of supporting Atmos docs generation natively.
24-28: Good validation coverage for file operations.These error definitions properly handle file-related edge cases that could occur during document generation: empty directories, ambiguous multiple files, non-existent directories, empty paths, and path resolution failures.
30-32: Appropriate input validation errors.These error variables provide necessary validation for the command's configuration and input parameters. The error messages are specific enough to guide users through troubleshooting.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
website/docs/cli/commands/docs/docs-generate.mdx (4)
10-16: Add comma after “e.g.” in purpose note
Stylistically, “e.g.” should be followed by a comma. Please update the sentence accordingly.-Use this command to generate one of your documentation artifacts (e.g. a README) as defined by the **named** section under `docs.generate.<KEY>` in `atmos.yaml`. +Use this command to generate one of your documentation artifacts (e.g., a README) as defined by the **named** section under `docs.generate.<KEY>` in `atmos.yaml`.🧰 Tools
🪛 LanguageTool
[style] ~12-~12: A comma is missing here.
Context: ...te one of your documentation artifacts (e.g. a README) as defined by the named s...(EG_NO_COMMA)
20-46: Ensure the YAML code block is closed
The openingyaml fence at line 20 must have a matching closing fence to render correctly. Please verify there’s aafter therelease-notessnippet (around line 46).
67-68: Add comma after “By default” in Terraform docs note
For grammatical clarity, insert a comma after “By default” and adjust the list punctuation.-The resultant file will also have a corresponding section rendered. By default `terraform.format` is set to `markdown table`, and can also be `markdown`, `tfvars hcl`, and `tfvars json`. +The resultant file will also have a corresponding section rendered. By default, `terraform.format` is set to `markdown table`, and can also be `markdown`, `tfvars hcl`, or `tfvars json`.🧰 Tools
🪛 LanguageTool
[uncategorized] ~67-~67: Did you mean: “By default,”?
Context: ... have a corresponding section rendered. By defaultterraform.formatis set to `markdown ...(BY_DEFAULT_COMMA)
[grammar] ~67-~67: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...By defaultterraform.formatis set tomarkdown table, and can also bemarkdown, `tf...(MARKDOWN_NNP)
127-130: Consider adding concrete remote source examples
While you mention that remote files usego-getter, including a few example URLs or Git URIs (e.g.,git::https://github.com/...) would help users understand valid remote syntaxes.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
tests/fixtures/scenarios/docs-generate/.gitignore(1 hunks)website/docs/cli/commands/docs/docs-generate.mdx(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- tests/fixtures/scenarios/docs-generate/.gitignore
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/commands/docs/docs-generate.mdx
[style] ~12-~12: A comma is missing here.
Context: ...te one of your documentation artifacts (e.g. a README) as defined by the named s...
(EG_NO_COMMA)
[typographical] ~51-~51: Consider using a typographic close quote here.
Context: ... ```yaml {{- $data := (ds "config") -}} {{ $data.name | default "Project ...
(EN_QUOTES)
[typographical] ~53-~53: Consider using a typographic close quote here.
Context: ... {{ $data.name | default "Project Title" }} {{ $data.description | default "No ...
(EN_QUOTES)
[typographical] ~55-~55: Consider using a typographic close quote here.
Context: ...tion | default "No description provided." }} {{ if has $data "extra_info" }} Ext...
(EN_QUOTES)
[uncategorized] ~55-~55: Loose punctuation mark.
Context: ...on | default "No description provided." }} {{ if has $data "extra_info" }} Extra...
(UNLIKELY_OPENING_PUNCTUATION)
[grammar] ~57-~57: It seems that a pronoun is missing.
Context: ...fault "No description provided." }} {{ if has $data "extra_info" }} Extra info: {...
(IF_VB)
[typographical] ~57-~57: Consider using a typographic close quote here.
Context: ...ovided." }} {{ if has $data "extra_info" }} Extra info: {{ $data.extra_info }} {...
(EN_QUOTES)
[grammar] ~61-~61: It seems that a pronoun is missing.
Context: ...o: {{ $data.extra_info }} {{ end }} {{ if has $data "terraform_docs" }} ## Terraf...
(IF_VB)
[typographical] ~61-~61: Consider using a typographic close quote here.
Context: ... end }} {{ if has $data "terraform_docs" }} ## Terraform Docs {{ $data.terraform...
(EN_QUOTES)
[uncategorized] ~67-~67: Did you mean: “By default,”?
Context: ... have a corresponding section rendered. By default terraform.format is set to `markdown ...
(BY_DEFAULT_COMMA)
[grammar] ~67-~67: Did you mean the formatting language “Markdown” (= proper noun)?
Context: ...By default terraform.format is set to markdown table, and can also be markdown, `tf...
(MARKDOWN_NNP)
⏰ Context from checks skipped due to timeout of 90000ms (6)
- GitHub Check: Build (macos-latest, macos)
- GitHub Check: Build (windows-latest, windows)
- GitHub Check: Build (ubuntu-latest, linux)
- GitHub Check: Lint (golangci)
- GitHub Check: website-deploy-preview
- GitHub Check: Analyze (go)
🔇 Additional comments (5)
website/docs/cli/commands/docs/docs-generate.mdx (5)
1-7: Frontmatter is well-defined
The frontmatter fields (title, sidebar_label, description, id) align with the file’s purpose and path.
8-8: Importing Screengrab component
TheScreengrabimport is correctly included for later usage in the Usage section.
69-77: Dynamic Keys section is clear and actionable
The explanation of using dynamic top-level keys underdocs.generateand passing them as CLI arguments is concise and easy to follow.
81-86: Usage examples are concise and helpful
Including both a screengrab and shell snippet foratmos docs generate --helpandatmos docs generate readmegives users clear guidance.
89-98: Local Sources examples are comprehensive
The section covering absolute, relative, andbase-dir-relative paths provides all the necessary context for configuring local inputs.
|
💥 This pull request now has conflicts. Could you fix it @Listener430? 🙏 |
what
This PR introduces a new command to generate a README.md from one or more input sources defined at
docs.generate.readmesection ofatmos.yaml. The command combines all local or remote YAML files specified atsourceand generates README.md at the working directory. In case the template containsterraform_docskey, the resultant README.md will also have a corresponding section rendered. By defaultterraform.formatis set tomarkdown table, and can also bemarkdown,tfvars hcl, andtfvars json.This command also aliases to “atmos generate docs”.
Run with
atmos docs generate readme, e.g.Examples of outputs
markdown table (preview)
markdown table (source)
markdown (preview)
markdown (source)
tfvars hcl (source)
tfvars json (source)
Testing
Refer to
tests\test-cases\docs-generate.yamlwhy
Have an option to run a command natively from within atmos (not using builld-harness repo)
references
Summary by CodeRabbit
New Features
Documentation
Tests
Chores