-
Notifications
You must be signed in to change notification settings - Fork 60
Add Multi Org Support Section for GitHub Ocean #2885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
emekanwaoma
wants to merge
4
commits into
main
Choose a base branch
from
PORT-16573
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
4 changes: 2 additions & 2 deletions
4
...are-catalog/sync-data-to-catalog/git/github-ocean/_github_docker_parameters.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1 change: 1 addition & 0 deletions
1
.../sync-data-to-catalog/git/github-ocean/_github_exporter_supported_resources.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
39 changes: 39 additions & 0 deletions
39
...amples/example-organization/_github_exporter_example_organization_blueprint.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
<details> | ||
<summary><b>Organization blueprint (click to expand)</b></summary> | ||
|
||
```json showLineNumbers | ||
{ | ||
"identifier": "githubOrganization", | ||
"title": "GitHub Organization", | ||
"icon": "Github", | ||
"schema": { | ||
"properties": { | ||
"url": { | ||
"title": "Organization URL", | ||
"type": "string", | ||
"format": "url" | ||
}, | ||
"description": { | ||
"title": "Description", | ||
"type": "string" | ||
}, | ||
"repositories": { | ||
"title": "Repositories", | ||
"type": "number" | ||
}, | ||
"createdAt": { | ||
"title": "Created At", | ||
"type": "string", | ||
"format": "date-time" | ||
} | ||
}, | ||
"required": [] | ||
}, | ||
"mirrorProperties": {}, | ||
"calculationProperties": {}, | ||
"relations": {} | ||
} | ||
``` | ||
|
||
</details> | ||
|
39 changes: 39 additions & 0 deletions
39
.../example-organization/_github_exporter_example_organization_port_app_config.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
<details> | ||
|
||
<summary><b>Port port-app-config.yml (click to expand)</b></summary> | ||
|
||
```yaml showLineNumbers | ||
resources: | ||
- kind: organization | ||
selector: | ||
query: "true" # JQ boolean query. If evaluated to false - skip syncing the object. | ||
port: | ||
entity: | ||
mappings: | ||
identifier: .login | ||
title: .name | ||
blueprint: '"githubOrganization"' | ||
properties: | ||
url: .html_url | ||
description: .description | ||
repositories: .public_repos | ||
createdAt: .created_at | ||
- kind: repository | ||
selector: | ||
query: "true" | ||
port: | ||
entity: | ||
mappings: | ||
identifier: .name | ||
title: .name | ||
blueprint: '"githubRepository"' | ||
properties: | ||
readme: file://README.md | ||
url: .html_url | ||
defaultBranch: .default_branch | ||
relations: | ||
organization: .__organization.login | ||
``` | ||
|
||
</details> | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,6 +18,27 @@ Here's what you can do with the GitHub integration: | |
- Map and organize your desired GitHub resources and their metadata in Port (see supported resources below). | ||
- Watch for GitHub object changes (create/update/delete) in real-time, and automatically apply the changes to your software catalog. | ||
- Manage Port entities using GitOps. | ||
- **Sync data from multiple GitHub organizations** using a single integration instance. | ||
|
||
### Multi-organization support | ||
|
||
The GitHub integration supports syncing data from multiple GitHub organizations starting from **version 2.0.0-beta**. You can configure which organizations to sync using the `githubOrganizations` configuration parameter. | ||
|
||
:::info Authentication requirements | ||
- **Classic PAT**: Supports multiple organizations. Fine-grained PAT tokens do not support multi-organization authentication. | ||
- **GitHub App**: Only supports a **single organization** (minimum 1, maximum 1). You must specify exactly one organization in the `githubOrganizations` array. | ||
::: | ||
|
||
:::caution Performance impact | ||
Syncing multiple organizations will increase the number of API calls to GitHub and may slow down the integration. The more organizations you sync, the longer the resync time and the higher the API rate limit consumption. Consider syncing only the organizations you need. | ||
::: | ||
|
||
**Configuration options:** | ||
- **With classic PAT**: Specify a list of organizations: `githubOrganizations: ["org1", "org2", "org3"]` | ||
- **With classic PAT**: Leave empty to sync all organizations the PAT user is a member of: `githubOrganizations: []` | ||
|
||
- **With GitHub App**: Specify exactly one organization: `githubOrganizations: ["my-org"]` | ||
|
||
When using multi-organization support, the `organization` resource kind is automatically synced, allowing you to model your GitHub organizations in Port. | ||
|
||
### Supported resources | ||
|
||
|
@@ -136,10 +157,39 @@ resources: | |
|
||
Using Port's GitHub integration, you can automatically ingest GitHub resources into Port based on real-time events. | ||
|
||
The app allows you to ingest a variety of objects resources provided by the GitHub API, including repositories, pull requests, workflows and more. It also allows you to perform "extract, transform, load (ETL)" on data from the GitHub API into the desired software catalog data model. | ||
The app allows you to ingest a variety of objects resources provided by the GitHub API, including organizations, repositories, pull requests, workflows and more. It also allows you to perform "extract, transform, load (ETL)" on data from the GitHub API into the desired software catalog data model. | ||
|
||
The GitHub integration uses a YAML configuration file to describe the ETL process to load data into the developer portal. This approach provides a flexible and powerful way to model your Git data without being overly opinionated or complex. | ||
|
||
### Ingest organizations | ||
|
||
The GitHub integration can automatically sync organization-level data when using multi-organization support (available from **v2.0.0-beta**). | ||
|
||
Here's an example configuration for the `organization` kind: | ||
|
||
```yaml showLineNumbers | ||
resources: | ||
- kind: organization | ||
selector: | ||
query: 'true' | ||
port: | ||
entity: | ||
mappings: | ||
identifier: .login | ||
title: .name | ||
blueprint: '"githubOrganization"' | ||
properties: | ||
url: .html_url | ||
description: .description | ||
createdAt: .created_at | ||
blog: .blog | ||
location: .location | ||
``` | ||
|
||
:::tip Organization as parent entity | ||
Organizations can serve as parent entities for repositories, teams, and other GitHub resources, helping you model your organizational structure in Port. | ||
::: | ||
|
||
### Ingest files from your repositories | ||
|
||
Port allows you to fetch `JSON` and `YAML` files from your repositories, and create entities from them in your software catalog. | ||
|
@@ -149,6 +199,10 @@ For example, say you want to manage your `package.json` files in Port. One optio | |
|
||
The following configuration fetches all `package.json` files from "MyRepo" and "MyOtherRepo", and creates an entity for each of them, based on the `manifest` blueprint: | ||
|
||
:::warning Breaking change in v2.0.0-beta | ||
Starting from version 2.0.0-beta, the `file` kind requires an `organization` field to be specified. This is a breaking change from previous versions. | ||
::: | ||
|
||
```yaml showLineNumbers | ||
resources: | ||
- kind: file | ||
|
@@ -157,6 +211,7 @@ resources: | |
files: | ||
# Note that glob patterns are supported, so you can use wildcards to match multiple files | ||
- path: '**/package.json' | ||
organization: my-org # Organization name is required from v2.0.0-beta | ||
# The `repos` key can be used to filter the repositories and branch where files should be fetched | ||
repos: | ||
- name: MyRepo | ||
|
@@ -679,6 +734,7 @@ resources: | |
query: 'true' | ||
files: | ||
- path: '**/package.json' | ||
organization: my-org # Organization name is required from v2.0.0-beta | ||
# Note that in this case we are fetching from a specific repository | ||
repos: | ||
- name: MyRepo | ||
|
@@ -736,7 +792,11 @@ resources: | |
query: 'true' | ||
files: | ||
- path: values.yaml | ||
organization: my-org # Organization name is required from v2.0.0-beta | ||
skipParsing: true | ||
repos: | ||
- name: MyRepo | ||
branch: main | ||
port: | ||
entity: | ||
mappings: | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment about the admonition