Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bump version to 0.14.1-beta1 #18

Merged
merged 2 commits into from
Mar 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@
# the repo. Unless a later match takes precedence,
# @global-owner1 and @global-owner2 will be requested for
# review when someone opens a pull request.
* @jbristow @zprobst @ccloes @angelosantos4
* @jbristow @zprobst @ccloes @angelosantos4 @rreddy15
61 changes: 37 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,32 @@
# nodestream-plugin-github

# Overview
This plugin provides a way to scrape github data from the REST api and ingest
them as extractors in nodestream pipelines.

This plugin provides a way to scrape GitHub data from the REST api and ingest
them as extractors in nodestream pipelines.

# Setup Neo4j

1. Download and install Neo4j: https://neo4j.com/docs/desktop-manual/current/installation/download-installation/
1. Create and start database (version 5.7.0: https://neo4j.com/docs/desktop-manual/current/operations/create-dbms/
1. Install APOC: https://neo4j.com/docs/apoc/5/installation/
2. Create and start database (version 5.7.0: https://neo4j.com/docs/desktop-manual/current/operations/create-dbms/
3. Install APOC: https://neo4j.com/docs/apoc/5/installation/

# Create github credentials
1. Create and github access codes: https://docs.github.com/en/[email protected]/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app
NOTE: These values will be used in your `.env`
# Create GitHub credentials

1. Create and GitHub access
codes: https://docs.github.com/en/[email protected]/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app
NOTE: These values will be used in your `.env`

# Install and run the app
1. Install python3: https://www.python.org/downloads/
1. Install poetry: https://python-poetry.org/docs/#installation
1. Install nodestream: https://nodestream-proj.github.io/nodestream/0.5/docs/tutorial/
1. Generate a new nodestream project
1. Add `nodestream-github` to your project dependencies in your nodestream projects pyproject.toml file.
1. Install necessary dependencies: `poetry install`
1. In `nodestream.yaml` add the following:

1. Install python3: https://www.python.org/downloads/
2. Install poetry: https://python-poetry.org/docs/#installation
3. Install nodestream: https://nodestream-proj.github.io/nodestream/0.5/docs/tutorial/
4. Generate a new nodestream project
5. Add `nodestream-github` to your project dependencies in your nodestream projects pyproject.toml file.
6. Install necessary dependencies: `poetry install`
7. In `nodestream.yaml` add the following:

```yaml
plugins:
- name: github
Expand All @@ -34,25 +39,33 @@ plugins:
all_public: True
rate_limit_per_minute: 225
targets:
- my-db:
- my-db:
pipelines:
- name: github_repos
- name: github_teams
- name: github_repos
- name: github_teams
targets:
database: neo4j
uri: bolt://localhost:7687
username: neo4j
password: neo4j123
database: neo4j
uri: bolt://localhost:7687
username: neo4j
password: neo4j123
```
1. Set environment variables in your terminal session for: `GITHUB_ACCESS_TOKEN`.
1. Verify nodestream has loaded the pipelines: `poetry run nodestream show`
1. Use nodestream to run the pipelines: `poetry run nodestream run <pipeline-name> --target my-db`
2. Verify nodestream has loaded the pipelines: `poetry run nodestream show`
3. Use nodestream to run the pipelines: `poetry run nodestream run <pipeline-name> --target my-db`

# Using make

1. Install make (ie. `brew install make`)
1. Run `make run`
2. Run `make run`

# Contributing

When contributing, make sure to sign your commits. To find out more about how to do this, refer to
this [GitHub documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits).

# Authors

* Jon Bristow
* Zach Probst
* Rohith Reddy
40 changes: 20 additions & 20 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 7 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
[tool.poetry]
name = "nodestream-plugin-github"
version = "0.14.0-beta.1"
version = "0.14.1-beta.1"
description = ""
authors = ["Jon Bristow <[email protected]>"]
authors = [
"Jon Bristow <[email protected]>",
"Rohith Reddy <[email protected]>"
]
packages = [
{ include = "nodestream_github" }
]
Expand All @@ -16,8 +19,8 @@ tenacity = "^9.0.0"
httpx = ">=0.27,<0.28"

[tool.poetry.group.dev.dependencies]
ruff = "^0.9"
black = "^25.1"
ruff = "^0.11.0"
black = "^25.1.0"
isort = "^6.0.0"
pytest = "^8.3.4"
pytest-asyncio = "^0.25.3"
Expand Down
Loading