feat: Implement github as a remote source

[secp192k1]

Remote source flow

1. When a github link is detected it converts the dialog text from Add to Next.

Picture Preview

2. After clicking next, it will fetch the latest release.

Picture Preview

Note

The only files allowed to be picked are .apk for downloaders and .rvp for patches

3. After selecting a version, wait for it to download.

Picture Preview

Logcat Preview

Note

The logcat above has a DownloaderEntity which is the external github downloader

Page of added external patches

[oSumAtrIX]

This could cause limitations in the future. If API adds features for manager that github cant satisfy on its own, manager would fail to properly handle the sources under a common interface

[oSumAtrIX]

Using GitHub as a source will exhaust the API and can lead to rate limits

[secp192k1]

Using GitHub as a source will exhaust the API and can lead to rate limits

How about a warning informing the user of this? Or do you have any proposals in mind?

[oSumAtrIX]

The only way to increase the limit is with an github api key. I wouldnt add github as a source at all. The API/json source is already a very opaque layer providing a transformation layer for manager. You can make that json point to github if you want to use github as a source effectively, which is also what we currently have with our api. This also offloads the api calls to the api/json instead of github. Without a transformation layer like API you lose an important control point in the infrastructure and strongly depend on the target platform in this case github.

[secp192k1]

Telling the user to provide an github api key to "unlock" this functionality effectively is a bad idea, since its too many steps and way too complex for the average user; and yes it would make the manager dependent on github if this functionality is being used, limiting the requests that can be made by the manager is way to mitigate this but its your call osum

[Ushie]

This could cause limitations in the future. If API adds features for manager that github cant satisfy on its own, manager would fail to properly handle the sources under a common interface

In that case revanced manager should offer an API for sources, like it has for downloaders, and GitHub/GitLab etc can be officially implemented while still allowing for future modularity

[Ushie]

Using GitHub as a source will exhaust the API and can lead to rate limits

You would have to make 60 requests an hour to hit the rate limit, it used to be an issue in ReVanced Manager alpha because everything depended on the GitHub API but it's only one request per patch bundle in this case here

[oSumAtrIX]

With every source you add, you drastically reach the limit

[Ushie]

You're unlikely to reach 60 GitHub sources, besides for users who are that deep into it, I'd say asking for a scoped GitHub PAT isn't an issue, when the GitHub plugin faces it's first ratelimit it can ask for a token then

[oSumAtrIX]

It doesn't need 60, for example 30 sources means refreshing once to hit the limit.

[Ushie]

Refreshes shouldn't be done carelessly by the manager, at most the user should be allowed to force refresh a single source, otherwise manager can just dictate all sources being refreshed once in an hour timespan

[oSumAtrIX]

This was just an example of a pitfall that was not previously considered before I named it. There's definitely many more like this that in sum speak against this and in favor of a proper solution with an API that provides a transformation and control layer for sourcing, preventing any unforseen issues as the two I just named and potentially others

[secp192k1]

@oSumAtrIX perhaps a switch in the settings page, Allow external sources, and just like other switches, explain to the user the (api) limitations and other warnings before turning on
Edit: obviously this will be disabled by default

[tristan23612]

How about

• limit max amount of external github source
• disable auto refresh, only manual one by one

when api key isn't provided?

[oSumAtrIX]

I am bad with explanation so here's chattys understanding of what im saying, but i checked this is what i mean:

[secp192k1]

Maybe a deploy script in the patches repo?

I dont think using the patches repo as a base for this would be the best, because then youd need both revanced patches and the github project patches the user would downloaded, when youd only want the latter

Or a standalone generateJson script that generates the file

I guess? Like a "handshake" file so to say, which holds the links to the rvp files?

[oSumAtrIX]

I dont think using the patches repo as a base for this would be the best, because then youd need both revanced patches and the github project patches the user would downloaded, when youd only want the latter

Can you explain what you mean, i dont think i understand you

[secp192k1]

I am bad with explanation so here's chattys understanding of what im saying, but i checked this is what i mean:

Then the idea of importing a json file that has targets for the rvp and asc files would fit then?
Not really sure if this is what youre suggesting so forgive me for that 😄

[secp192k1]

I dont think using the patches repo as a base for this would be the best, because then youd need both revanced patches and the github project patches the user would downloaded, when youd only want the latter

Can you explain what you mean, i dont think i understand you

You said "It probably could make sense to put this part into the patches repository" then how would an user import the patches into manager? My first idea would that the files/assets would reside in the releases of the dev's custom patches, then what

[oSumAtrIX]

ChatGPT reworded:

Yes, that would fit the direction I mean.

The idea is that Manager should have one canonical source model for patches: a JSON source descriptor.

That JSON describes the actual patch bundle and related metadata, for example the .rvp, signature, version, creation date, description, compatibility metadata, etc. Manager then only has to understand this one source format.

Everything else should be treated as a transport concern.

For example, the user could import:

https://example.com/patches.json
file:///storage/emulated/0/patches.json
data:application/json;base64,...
github://user/repo

But semantically, all of these should resolve to the same thing: a JSON source descriptor.

The protocol handler decides how to fetch or produce that JSON. For example:

• https://... fetches the JSON over HTTP.
• file://... reads the JSON from local storage.
• data://... inlines the JSON directly.
• github://user/repo could resolve GitHub releases into a JSON descriptor.

So Manager does not need separate source models like “GitHub source”, “local source”, “direct RVP source”, etc. It only needs one source model: JSON.

This also means the import textbox should not really mean “import any patch-related thing”. It should mean “import a URL that resolves to a patch source JSON”.

That keeps the resource layer clean:

Manager source model:
  JSON source descriptor

Transport layer:
  https://
  file://
  data://
  github://
  ...

So the important distinction is:

JSON = source format / semantic model
URL protocol handler = transport / resolution mechanism
RVP = downloadable artifact referenced by the JSON

That way GitHub can still be supported, but only as a transport/resolution backend, not as a separate source model inside Manager.

---

My original:

Then the idea of importing a json file that has targets for the rvp and asc files would fit then?

Right now manager has two models for the resource layer:

direct rvp file
the json file

right now the direct rvp file is currently being debated. Basically the "intended" way to add patches to manager would "always" be done over the json file, while the transport layer takes over the concern of where to get what.

E.g.

You could point to a local json file with file://some/path/your.json or https://some.path/your.json. This is simply done via URL protocol handlers. Similarly the URLs inside the JSON itself can benefit from the handlers.

data:// protocol handlers even allow inlining data in base64 inside the url.

You can even register a custom handler e.g. "github://name/repo"

In other words the "import patches textbox" of manager should "always" require an URL pointing to a JSON file. While the serializer of the input of type URL has registered various protocol handlers of our choice concerning the transport layer.

[oSumAtrIX]

You said "It probably could make sense to put this part into the patches repository" then how would an user import the patches into manager? My first idea would that the files/assets would reside in the releases of the dev's custom patches, then what

Well, for example when the release workflow runs, it also creates a JSON pointing to that releases RVP file.

[oSumAtrIX]

So one can add a source of patches with github://name/repo

[oSumAtrIX]

The protocol handler of github:// unwraps the right hand side into:

github.com/name/repo/releases/latest/assets/patches.json

[oSumAtrIX]

And now we have "full" support of what this repository aims, without ever touching the resource layer of manager, only the transport layer.

[oSumAtrIX]

Side note:

Since the input field accepts URLs, but URLs are not "user friendly" the import field can simlpy have a dropdown followed by a text field. The dropdown lists all available protocol handlers, while the input field concerns with the right hand side of the url.

Sample UI:

[= GitHub =] __user/repo__

[oSumAtrIX]

An even higher abstraction of the UI could be a Map of handler and compose layout, e.g.

mapof(githubProtocolHandler to { ...compose layout })

this way you could even expand the compose layout when you select a specific handler.

So when you for example select github, it will not just show 1 input field where you have to correctly format your string to a/b but it could show neatly displayed two input boxes one for name and repo.

[oSumAtrIX]

Oh and, protocol handlers can even support configurations, e.g.

github://name/repo?gh_token=...

or for example a private https url:

https://name:password@yourprivate/json

[Ushie]

We still haven't really proven a necessity to this transformative layer as there's no information being provided that isn't provided by any of the public release APIs or alternatively as part of the RVP manifest

[secp192k1]

You said "It probably could make sense to put this part into the patches repository" then how would an user import the patches into manager? My first idea would that the files/assets would reside in the releases of the dev's custom patches, then what

Well, for example when the release workflow runs, it also creates a JSON pointing to that releases RVP file.

Well since that JSON file would just point to the latest files, it wouldnt change, I mean it cant change because then it would be off sync with the target home where the custom dev's patches are at

[oSumAtrIX]

You can even have global configs for protocol handlers:

GitHubHandler(managerPref.gitHubHandler.userProvidedAccessToken).handle("github://name/repo")

[secp192k1]

You can even have global configs for protocol handlers:

GitHubHandler(managerPref.gitHubHandler.userProvidedAccessToken).handle("github://name/repo")

But this is using the github api... in fact, the user would also need to create a PAT

[oSumAtrIX]

Well since that JSON file would just point to the latest files, it wouldnt change, I mean it cant change because then it would be off sync with the target home where the custom dev's patches are at

The JSON itself would point to the specific releases asset. While the URL to the JSON itself, would use /latest. So when manager starts, it can query the URL, compare the version tag, and update the patches. But the individual json always shows to the respective releases rvp, so you can for example point to a specific releases JSON if you want to use old patches

[optimatrix0]

Suggestion: Consider a local manifest-based architecture for custom sources

Summary: I would like to suggest an alternative architecture that could simplify source management, reduce infrastructure requirements, improve scalability, and make the ecosystem independent of any specific Git hosting platform. This is only a suggestion for consideration.

First of all, thank you for all the work on this project. The ReVanced community has helped me a lot over the years, and this suggestion is simply my way of giving something back. If this approach doesn't fit the project's goals, please feel free to disregard it.

The idea

Instead of relying on a centralized API for retrieving repository metadata, the Manager could support a static manifest file hosted directly inside each custom source repository.

For example, each compatible source could expose a file such as:

"patcheslist.json"

This manifest would contain all the information required by the Manager, such as:

• source information;
• available ".rvp" packages;
• downloader ".apk" files;
• download URLs;
• versions;
• compatibility;
• file hashes for integrity verification;
• optional descriptions and changelogs.

The Manager would simply:

1. Download the manifest.
2. Parse it locally.
3. Compare installed and available versions.
4. Download only the required ".rvp" or ".apk" files.

The update process would therefore be:

Source URL → download manifest → local processing → version comparison → download required files

Why I think this could be beneficial

With this model, normal source operation would not require a dedicated API.

Checking for updates would only require downloading the manifest again, which is a very small static file that can be served directly by the hosting platform or a CDN.

Another advantage is that the source format becomes independent of GitHub.

The same repository structure could work on:

• GitHub;
• GitLab;
• Codeberg;
• Forgejo;
• Gitea;
• or even a regular web server hosting static files.

The only requirement would be following a documented repository template and exposing the manifest URL.

About compatibility

Supporting multiple Git hosting providers would naturally require some compatibility code inside the Manager (or alternatively allowing users to provide the direct manifest URL).

However, this is mostly a one-time implementation effort.

Once those providers are supported, the architecture remains simple, lightweight and independent of any centralized backend.

Where an API still makes sense

I do think an API can still provide a lot of value.

For example:

• source discovery;
• search;
• categorization;
• moderation;
• usage statistics;
• remote "Add Source" links;
• deep-link integration with the Manager.

Those are excellent use cases for an API.

The difference is that, after a source has been added, the Manager could communicate directly with that source instead of depending on an API for routine operations.

This would reduce:

• API rate-limit concerns;
• backend maintenance;
• infrastructure costs;
• bandwidth costs;
• dependence on continuous server funding;
• centralized points of failure.

It would also allow the Manager to perform metadata processing locally, making the architecture more resilient and portable.

Final thoughts

I don't believe the API approach is inherently wrong.

I simply think it may introduce additional complexity for a problem that could potentially be solved using a much simpler local architecture.

From my perspective, it feels similar to using a crane for a task that a well-equipped pickup truck could already handle.

Again, this is only an architectural suggestion intended to contribute to the discussion. Thank you for taking the time to read it, and thank you for all the work you put into this project.

[Ushie]

that's already the case

[oSumAtrIX]

Right now there's #3436 which decouples transport from resource. The resource is what you describe as Manifest. A small structure that contains metadata & URL to the actual rvp. Manager uses URL as the transport specification and will support some popular protocols such as file:// and https://. You can then point to such a resource (serialized as JSON). For example file://sdcard/theresource or https://some.com/theresource

The resources themselves contain a URL to the rvp or other future resources, which once again are handled through the same supported protocols.

The PR can easily supercede this one by abstracting the supported HTTP protocol handler into a github:// handler, albeit not necessary.

for example:

github://name/repo

It can also support optional queries such as

?release_tag=

In case you want to pin a specific release.

The handler would simply reuse the existing http handler and construct the internal URL from the input.

github://user/name?release_tag=latest can simply be constructed to a https://GitHub.com/ release url pointing to the resource as described above. The rest is identical.

[oSumAtrIX]

Some advancements with the PR will also be a mapping from protocol to UI, so when you for example use the file;// handler, the import UI would present a file select button, while for say GitHub it would provide fields for name and repo and optional queries.

The PR decouples transport and resource and as such locality merely depends on the supported protocols, while the resource always stays the same and consistent across the code regardless of resource locality.

[secp192k1]

this became ""offtopic"" and aparently not wanted... so fuck it ig
it was a simple idea, still have NO idea why this should be a protocol, thats just plain dumb

[oSumAtrIX]

To clarify the APIs role. The API decouples the resource from its locality. The resources usually origin from external places like GitHub and co, however there are concerns to why you wouldn't want to point outside your own domain, for example due to availability or privacy. As such, API proxies the actual locality and moves the resource back to your own domain you have full control over. That said, the API neither controls the resource, nor restricts managers ability to source resources elsewhere. It is merely a deployment decision from the resources perspective. Instead of advertising itself at an external location, it moves itself to another.

[oSumAtrIX]

To clarify why this should be transport. GitHub is a location. Not the resource. The resource is the same, only the locality changes of the resource. Therefore the concern of this PR should never ever breach locality and touch resources. For this reason #3436 reworks the architecture to allow handling locality separately from resource. In doing so, accomplishing what this PR does by touching transport and resource, is possible through merely concerning the transport layer, via a transport protocol handler, such as GitHub:// (which can simply be a thin wrapper of the https handler)

This explanation is identical to the one we had in our discord, in case you forgot

[oSumAtrIX]

That said, this PR is relevant for as long as we support the idea of a thin GitHub handler out of the http handler once #3436 is ready. What is contested in this PR however, is the implementation touching the resource layer, when it's supposed to not, invoking #3436's existence and making itself dependent on it. Once the architecture is fixed and locality can be concerned without having to worry about resource, this PR can build upon it and bring GitHub as a handler, by merely adding the handler next to the existing ones. The primary support for this PR would be nicer integration of GitHub as a location for the resource in form of a dedicated UI mapping to the handler (Dedicate inputs for username and repo for example), but since GitHub would merely be a thin wrapper of http, it's not strictly necessary to support GitHub through the http handler. So it's mostly just ui sugar effectively

[secp192k1]

yap yap yap who cares man, branch deleted anyways

Sorry to everyone who was looking forward to this...

[oSumAtrIX]

Who cares should be everyone who values code that can be maintained in the future. And proper architecture aids that. Everyone who looks forward to this can still do with #3436 because as previously explained it makes pathway for what this PR aims to accomplish. The difference is in how it's implemented. As previously explained even without dedicated support for GitHub, the https protocol handler provides 100% functionality to support GitHub as a source. The GitHub handler only effectively provides additional ui sugar. I don't understand this behavior, when you clearly said that you don't understand the necessity of this in your previous comment and I provided you with one, in addition clarifying how the goal of this PR is still relevant. This is a technical space and describing technical and relevant concerns as yap uncalled for.