Skip to content

Commit f01cc24

Browse files
Claudelukekim
authored andcommitted
Fix github recipe: document required token scopes and cap comments at 75
Two datasets in this recipe cannot load with the token the README describes, and one param is silently clamped by the runtime. 1. spiceai.stargazers and apache.members select the user 'email' field (connector-github stargazers.rs / members.rs), which GitHub only returns for tokens carrying 'read:user' or 'user:email'. A token with just 'repo' — all the prerequisite implies — fails both datasets at load. Spell out the scopes the recipe actually needs. 2. github_max_comments_fetched was set to 100, above the connector's hard cap of 75 (MAX_COMMENTS_FETCHED in connector-github/src/lib.rs). The runtime clamps it and logs a warning, so the configured value never took effect. Set it to the real maximum. Also corrects a 'Setp 2.' typo.
1 parent 8b97e57 commit f01cc24

2 files changed

Lines changed: 10 additions & 2 deletions

File tree

github/README.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,14 @@ This recipe will use the [spiceai/spiceai](https://github.com/spiceai/spiceai) r
99
- Spice is installed (see the [Getting Started](https://docs.spiceai.org/getting-started) documentation).
1010
- GitHub personal access token, [Learn more](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic) how to create one.
1111

12+
Grant the token both of these scopes:
13+
14+
- `repo` — for the `files`, `issues`, `pulls`, and `commits` datasets.
15+
- `read:user` (or `user:email`) — the `stargazers` and `members` datasets select
16+
the user `email` field, which GitHub only returns for tokens with one of these
17+
scopes. Without it those two datasets fail to load with `Your token has not
18+
been granted the required scopes to execute this query`.
19+
1220
[![Watch the Spice.ai local GitHub connector demo](https://img.youtube.com/vi/mxwt0HEF1VQ/hqdefault.jpg)](https://www.youtube.com/embed/mxwt0HEF1VQ)
1321

1422
**Step 0.** Clone the repository if not already cloned.
@@ -24,7 +32,7 @@ cd cookbook/github
2432
GITHUB_TOKEN=<your_github_token>
2533
```
2634

27-
**Setp 2.** Run the Spice runtime with `spice run` from the directory with the `spicepod.yaml` file.
35+
**Step 2.** Run the Spice runtime with `spice run` from the directory with the `spicepod.yaml` file.
2836

2937
```bash
3038
spice run

github/spicepod.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ datasets:
2727
github_query_mode: search
2828
github_token: ${secrets:GITHUB_TOKEN}
2929
github_include_comments: all
30-
github_max_comments_fetched: 100
30+
github_max_comments_fetched: 75 # 75 is the maximum; higher values are clamped
3131
time_column: updated_at
3232
acceleration:
3333
enabled: true

0 commit comments

Comments
 (0)