Skip to content

Commit e2ad1ea

Browse files
Kim Neunertben-kaufmankdmukaimoneymanolis
authored
Bugfix: Very last fixes for Service swan (#1537)
* Feature: Voltoro trading first commit * fix create_order issue * fix url and better error-handling * basic balances tab * Add deposit vaultoro page * activating services * Service management * Improve settings page and default to it if token unset * Fix history tab * Fix trade error handling * Add withdraw * calling specter-cloud for creating vaultoro orders * Improve trade screen and fixes * sidebar fix * black * refactor Service integration * refactor to have Service Classes like manifests * maturity * dynamic initialisation of service-classes and blueprints * fix sidebar_services * migrated templates and static into vaultoro folder * refactor config to manifest * some minor things * swan initial * rename and fix test * adding ServiceApiKeyStorageUserAware * fix * store the access token * directory indirection to shield templates from each others blueprint * first attempts with automatic withdrawals * proper tab highlighting * Interim commit * Update service_apikey_storage.py * Update oauth2_success.jinja * Awaiting refresh_token support * Service logo display on Addresses * Associate addr with a Service * address-data component reorg Separates the presentation html from the data as much as possible. * Services data/icon added to tx History * Now hitting the updated Swan endpoint to save deposit addrs * Simplified injecting Services data into JS * Reducing js calls back to server in tx-data; templatizing utxo in/outs * renaming "reserving" to "associating" an Address with a Service. * rename `manifest.py` files to `service.py`. * rename "api_data" to "service_data" to make the storage a bit more generalized. `ServiceApiKeyStorage` is now `ServiceEncryptedStorage` to match. * Beginning of factoring out Swan api to its own `api.py` file; need to rectify with `swan_client.py`. * interim commit * Removed tx-table/row/data changes and address-table/row/data Kept only the bare minimum changes required to display the Services icon, plus optimizations. * interim commit * Interim commit * Cleanup commit * Update controller.py * Update services.md * Update services.md * deleted no longer used CustomElement * Adding services docs to mkdocs * Configuration for Services * more clever configuration * fix test * deleted swan_client * Changing the address abbreviation format to 7...7 and little big fix for not vertically aligned addresses in Firefox. * Better state management if Auth method changes * Cleanup, better user messaging; pulling Service methods out of controller and User * Redirect to services endpoint after setting up authentication. * PR cleanup, bug fixes, test suite updates * Fixed test case problem * First fix for delete API key button. * Service hooks; Option to fully remove Swan Integration; Logout clears plaintext_user_secret * Restoring bugfix from @moneymanolis * black * cleanup and black * Make service-decovery in AppImage work * import hashlib, maybe fix cypress * Update service_encrypted_storage.py * further bugfix on update * TODO: remove debugging in client.py before first release * testing env setup markdown * make service-list more resilient * tiny bit more logging in case of issues. * Swan api firewall fix * Restoring lost services-related code in wallets_api.py; bugfix on service_data mismatch * Bugfix on updated completed autowithdrawal addr labels * publish markdown on doc-page * Still awaiting final Swan prod tests * move ServiceManager outside Specter.__init__() * Update config.py * Applying Kim's prop patch * Disabling extension loading from cwd in prod * build-script adjustments for clarity and right order * More comprehensive input validation for the rate limit in the auth settings. * monkey patch rthooks for a successfull MacOS-build * fix build-ci.sh * Refactor service_manager and templates where they belong to * Swan - different fronend-links links for dev/prod * include templates and services in sdist * implement dynamic loading but from list in config * No dynamic cwd services in appimages * tidy up Co-authored-by: benk10 <ben.kaufman10@gmail.com> Co-authored-by: kdmukai <kdmukai@gmail.com> Co-authored-by: moneymanolis <moneymanolis@protonmail.com>
1 parent 352355e commit e2ad1ea

18 files changed

Lines changed: 200 additions & 45 deletions

MANIFEST.in

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
recursive-include src/cryptoadvance/specter/templates *
22
recursive-include src/cryptoadvance/specter/static *
3+
recursive-include src/cryptoadvance/specter/services/templates *
4+
recursive-include src/cryptoadvance/specter/services/static *
5+
recursive-include src/cryptoadvance/specter/services/*/templates *
6+
recursive-include src/cryptoadvance/specter/services/*/static *
37
recursive-include src/cryptoadvance/specter/translations/*/LC_MESSAGES *.mo
48
recursive-include src/cryptoadvance/specter/translations/*/LC_MESSAGES *.po
59
include requirements.txt

pyinstaller/hooks/hook-cryptoadvance.specter.services.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
from cryptoadvance.specter.services.service_manager import ServiceManager
1+
from cryptoadvance.specter.managers.service_manager import ServiceManager
22

33

44
# Collecting template and static files from the different services in src/cryptoadvance/specter/services

src/cryptoadvance/specter/config.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,12 @@ class BaseConfig(object):
143143
# THIS MIGHT BE A SECURITY_CRITICAL SETTING. DON'T SWITH TO TRUE IN PROD
144144
SERVICES_LOAD_FROM_CWD = False
145145

146+
# List of extensions (services) to potentially load
147+
EXTENSION_LIST = [
148+
"cryptoadvance.specter.services.swan.service",
149+
"cryptoadvance.specter.services.bitcoinreserve.service",
150+
]
151+
146152
# This is just a placeholder in order to be aware that you cannot set this
147153
# It'll be filled up with the fully qualified Classname the Config is derived from
148154
SPECTER_CONFIGURATION_CLASS_FULLNAME = None

src/cryptoadvance/specter/services/service_manager.py renamed to src/cryptoadvance/specter/managers/service_manager.py

Lines changed: 49 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,28 @@
11
import json
22
import logging
33
import os
4-
5-
from flask import current_app as app, url_for
6-
from flask.blueprints import Blueprint
74
from importlib import import_module
85
from inspect import isclass
96
from pathlib import Path
107
from pkgutil import iter_modules
118
from typing import Dict, List
12-
from cryptoadvance.specter.user import User
139

10+
from cryptoadvance.specter.config import ProductionConfig
1411
from cryptoadvance.specter.managers.singleton import ConfigurableSingletonException
15-
from ..util.reflection import get_subclasses_for_class, _get_module_from_class
12+
from cryptoadvance.specter.user import User
13+
from flask import current_app as app
14+
from flask import url_for
15+
from flask.blueprints import Blueprint
1616

17-
from .service import Service
18-
from .service_encrypted_storage import ServiceEncryptedStorageManager
17+
from ..services.service import Service
18+
from ..services.service_encrypted_storage import ServiceEncryptedStorageManager
19+
from ..util.reflection import (
20+
_get_module_from_class,
21+
get_classlist_of_type_clazz_from_modulelist,
22+
get_package_dir_for_subclasses_of,
23+
get_subclasses_for_clazz,
24+
get_subclasses_for_clazz_in_cwd,
25+
)
1926

2027
logger = logging.getLogger(__name__)
2128

@@ -30,9 +37,19 @@ def __init__(self, specter, devstatus_threshold):
3037
# Each Service class is stored here, keyed on its Service.id str
3138
self._services: Dict[str, Service] = {}
3239
logger.info("----> starting service discovery <----")
33-
for clazz in get_subclasses_for_class(
34-
Service, app.config.get("SERVICES_LOAD_FROM_CWD", False)
35-
):
40+
# How do we discover services? Two configs are relevant:
41+
# * SERVICES_LOAD_FROM_CWD (boolean, CWD is current working directory)
42+
# * EXTENSION_LIST (array of Fully Qualified module strings like ["cryptoadvance.specter.services.swan.service"])
43+
# Ensuring security (especially for the CWD) is NOT done here but
44+
# in the corresponding (Production)Config
45+
logger.debug(f"EXTENSION_LIST = {app.config.get('EXTENSION_LIST')}")
46+
class_list = get_classlist_of_type_clazz_from_modulelist(
47+
Service, app.config.get("EXTENSION_LIST", [])
48+
)
49+
if app.config.get("SERVICES_LOAD_FROM_CWD", False):
50+
class_list.extend(get_subclasses_for_clazz_in_cwd(Service))
51+
class_list = set(class_list) # remove duplicates (shouldn't happen but ...)
52+
for clazz in class_list:
3653
compare_map = {"alpha": 1, "beta": 2, "prod": 3}
3754
if compare_map[self.devstatus_threshold] <= compare_map[clazz.devstatus]:
3855
# First configure the service
@@ -84,9 +101,6 @@ def configure_service_for_module(cls, service_id):
84101
cls.import_config(clazz)
85102
return
86103

87-
logger.warning(
88-
f"Could not find a configuration for Service {module} ... trying parent-classes of main-config"
89-
)
90104
config_module = import_module(".".join(main_config_clazz_name.split(".")[0:-1]))
91105

92106
config_clazz = getattr(config_module, main_config_clazz_slug)
@@ -97,10 +111,13 @@ def configure_service_for_module(cls, service_id):
97111
cls.import_config(clazz)
98112
return
99113
config_candidate_class = config_candidate_class.__bases__[0]
114+
logger.warning(
115+
f"Could not find a configuration for Service {module}. Skipping configuration."
116+
)
100117

101118
@classmethod
102119
def import_config(cls, clazz):
103-
logger.info(f"Loading Service-specific configuration from {clazz}")
120+
logger.info(f" Loading Service-specific configuration from {clazz}")
104121
for key in dir(clazz):
105122
if key.isupper():
106123
if app.config.get(key):
@@ -166,19 +183,26 @@ def remove_all_services_from_user(self, user: User):
166183
@classmethod
167184
def get_service_x_dirs(cls, x):
168185
"""returns a list of package-directories which represents a specific service.
169-
This is primarily used by the pyinstaller packaging specter
186+
This is used by the pyinstaller packaging specter
170187
"""
171188
arr = [
172189
Path(Path(_get_module_from_class(clazz).__file__).parent, x)
173-
for clazz in get_subclasses_for_class(Service)
190+
for clazz in get_subclasses_for_clazz(Service)
174191
]
175192
arr = [path for path in arr if path.is_dir()]
176193
return [Path("..", *path.parts[-6:]) for path in arr]
177194

178195
@classmethod
179196
def get_service_packages(cls):
180-
"""returns a list of strings containing the service-classes. This is used for hiddenimports in pyinstaller"""
181-
arr = get_subclasses_for_class(Service)
197+
"""returns a list of strings containing the service-classes (+ controller/config-classes)
198+
This is used for hiddenimports in pyinstaller
199+
"""
200+
arr = get_subclasses_for_clazz(Service)
201+
arr.extend(
202+
get_classlist_of_type_clazz_from_modulelist(
203+
Service, ProductionConfig.EXTENSION_LIST
204+
)
205+
)
182206
arr = [clazz.__module__ for clazz in arr]
183207
# Controller-Packagages from the services are not imported via the service but via the baseclass
184208
# Therefore hiddenimport don't find them. We have to do it here.
@@ -201,4 +225,11 @@ def get_service_packages(cls):
201225
# RuntimeError: Working outside of application context.
202226
# shows that the package is existing
203227
arr.append(controller_package)
228+
config_arr = [".".join(package.split(".")[:-1]) + ".config" for package in arr]
229+
for config_package in config_arr:
230+
try:
231+
import_module(config_package)
232+
arr.append(config_package)
233+
except ModuleNotFoundError as e:
234+
pass
204235
return arr

src/cryptoadvance/specter/server.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
from cryptoadvance.specter.liquid.rpc import LiquidRPC
1313

1414
from cryptoadvance.specter.rpc import BitcoinRPC
15-
from cryptoadvance.specter.services.service_manager import ServiceManager
15+
from cryptoadvance.specter.managers.service_manager import ServiceManager
1616
from cryptoadvance.specter.util.reflection import get_template_static_folder
1717

1818
from .helpers import hwi_get_config

src/cryptoadvance/specter/services/service.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ def inject_stuff():
6767
controller_module = (
6868
f"cryptoadvance.specter.services.{self.id}.controller"
6969
)
70+
logger.info(f" Loading Controller {controller_module}")
7071
import_module(controller_module)
7172
app.register_blueprint(
7273
self.__class__.blueprint, url_prefix=f"/svc/{self.id}"

src/cryptoadvance/specter/services/swan/config.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ class BaseConfig:
1010
"BcetcVcmueWf5P3UPJnHhCBMQ49p38fhzYwM7t3DJGzsXSjm89dDR5URE46SY69j"
1111
)
1212
SWAN_API_URL = "https://dev-api.swanbitcoin.com"
13+
SWAN_FRONTEND_URL = "https://dev-app.swanbitcoin.com/signup"
1314

1415

1516
class ProductionConfig(BaseConfig):
@@ -18,3 +19,4 @@ class ProductionConfig(BaseConfig):
1819
"UcqMZw3D70#E*Zo1hnC8f8P^Ils^6wligXMB*vL1fX@DYm6zloDI#p9Eemk8!y9#"
1920
)
2021
SWAN_API_URL = "https://api.swanbitcoin.com"
22+
SWAN_FRONTEND_URL = "https://www.swanbitcoin.com/Specter/"

src/cryptoadvance/specter/services/swan/controller.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ def index():
5454
# User has already completed Swan integration; skip ahead
5555
return redirect(url_for(f"{SwanService.get_blueprint_name()}.withdrawals"))
5656
return render_template(
57-
"swan/index.jinja",
57+
"swan/index.jinja", swan_frontend_url=app.config["SWAN_FRONTEND_URL"]
5858
)
5959

6060

src/cryptoadvance/specter/services/swan/templates/swan/index.jinja

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@
7777
<img src="{{ url_for(service.id +'_endpoint' + '.static', filename=service.logo) }}" width="300"/>
7878
<div class="tagline">&ldquo;Swan is the best way to accumulate Bitcoin with automatic recurring buys and instant buys.&rdquo;</div>
7979
<div class="card center" style="width: auto; min-width: 90%; margin: 40px;">
80-
<a class="button_wide" href="https://www.swanbitcoin.com/Specter/" target="_swan">
80+
<a class="button_wide" href="{{ swan_frontend_url }}" target="_swan">
8181
<span class="big_option">
8282
<div class="big_option_text">
8383
Join Swan!

src/cryptoadvance/specter/specter.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@
3737
RpcError,
3838
get_default_datadir,
3939
)
40-
from .services.service_manager import ServiceManager
40+
from .managers.service_manager import ServiceManager
4141
from .services.service import devstatus_alpha, devstatus_beta, devstatus_prod
4242
from .specter_error import ExtProcTimeoutException, SpecterError
4343
from .tor_daemon import TorDaemonController

0 commit comments

Comments
 (0)