Automatic provider additions && tests

[DoganM95]

Do you have the latest provider files?

• I have checked that the provider are the latest

Have you tested the provider files?

• I have tested the provider files

Can you create a PR for this config?

• I can create a PR for this

Provider details

Any (testing via expressVPN)

Anything else?

This is more a suggestion than a bug report.

Currently your repository is pretty manual, meaning you add e.g. ovpn files from expressVPN/any provider to the respective folder and push that. I see you have a test_provider.py script to test these first.

I see some automation potential here, you could write a script, which downloads all ovpn config files from each provider (curl in a shell script?) (daily/weekly), compares the list of downloaded ovpn files with the files of the repo and adds new / removes old ones (to keep a clean commit history).

To achive this as an automated task, you could create a job for each vpn provider in github actions, which does this and commits/pushes updates automatically, which would perhaps save you a lot of maintenance time.

Lately i created 10 docker containers to see 2 of them not working due to ovpn file not existing, being:

my_expressvpn_hong_kong_-_4_udp.ovpn
my_expressvpn_japan_-_tokyo_-_3_udp.ovpn

If you like the idea, i could create a pull request with automation for expressVPN, so you have a template and can use that for other providers and/or generify.

[pkishino]

Yes please, that would be great!!

[DoganM95]

Tought a bit about how to realize this, there are some stones to get past.
ExpressVPN for example has a couple caveats:

• You have to be logged in as a subscribed (paying) user, to be able to download manual config files (.ovpn)
• After 5 downloads, it will enter a cooldown of like 30 seconds or something, until the next download is possible

Some implementation options, staring with the obvious:

1. ❌Your repository saves Credentials (uname, pass) per vpn provider in Github Secrets. Problem here is, you probably don't have a subscription for every vpn provider, so users who are already subscribed would have to provide tehir credentials and trust your repository to never get hacked, which is not an option.
2. ❌This repository is extended to provide a Github Action, so a user who forks this repo can put his credentials in their forked Repository's Secrets and run the ovpn file downloader for their chosen vpn provider on a daily basis, update their ovpn folder on changes and automatically create a pull request. This would give keep their credentials completely secret. The problem here is, Github restricted Actions-Quota to 500mb && 2000minutes / month for free users, a bit more for pro users. This would eat up their quota very fast, due to the ovpn file donwloading cooldown, causing idle times while running the action, counting to quota, not an option.
3. ❓Like option 2, except the downloader script would not run on github actions, but on machines of people, who forked this repository and have an active vpn subscription. A batch ovpn downloader script is created per provider.
   I for example could run this expressvpn/downloader_script.py on my NAS server, which runs 24/7, ideally as a docker container (infinite loop, which sleeps until a user-specified time is reached, the downloading time). This forked script/container would need to know my forked repository's url, github creds/ssh key, vpn provider creds and the mentioned downloader-time. The script would pull the latest master from your repo, then download the ovpn files and create a automatic pull request on your repo, if there are changes. Your repo could then run an action to automatically merge pull requests which contain file changes only to an ovpn containing folder, checking file integrity first for security reasons (are really only ovpn files uploaded? so they match the typical ovpn schema? is the domain correct and not a fake-domain?). This would ultimately create a cluster of users, which update your repository's ovpn files and increase resilience with every user joining the "downloading cluster". I think this is an option, but a bit more complex.

Whether or not you are ready to spend time into such a complex approach to completely automate this repository and saving yourselves thousands of pull-request reviews is of course up to you.
My brain needs a bit of rest now :P

[pkishino]

Thanks for the details.
I converted this to a discussion for easier chatting..
Anyway, yeah, providers making the download use dependent is a hassle. I’m not sure how many of them do this, I know many who don’t.. is there a way to make a GitHub triggered by the user and await user input? Then the action could be generalised and a flag added for those repos that don’t allow unauthorised users to download?
I’ll thinking about a main action with sub actions per provider if possible. The overall gist would be that each provider has a download section, extract and then “test”?
i suspect mainly the download part will be different between them.
URL and possibly authentication required
So the main action would trigger e.g over a day or week like a cron job. Then it would go through each provider and run etc except those with some flag which require manual input.
For those perhaps we can get a user of each to be “responsible” to trigger when possible with their credentials?

[DoganM95]

is there a way to make a GitHub triggered by the user and await user input?

I assume you mean the action starts and awaits user input at some point to further proceed the action? Currently not possible, as far as i know

Then the action could be generalised and a flag added for those repos that don’t allow unauthorised users to download?

Sure, totally realizable

I’ll thinking about a main action with sub actions per provider if possible. The overall gist would be that each provider has a download section, extract and then “test”?

Good idea, adds to my "validation on pull request" point

So the main action would trigger e.g over a day or week like a cron job. Then it would go through each provider and run etc except
those with some flag which require manual input.

Yes, the actions would need to be timed daily/weekly/etc, so you don't run out of actions quota and best triggered all at once i guess

For those perhaps we can get a user of each to be “responsible” to trigger when possible with their credentials?

Putting this into only one user's hand may be "risky", as a user can cancel any time, burn his house, you name it :)
That's why i came up with this "cluster" approach to create resilience