Merge pull request #369 from swirlai/patch-2.0.2

dnicodemus · web-flow · commit 9b0b3960cacb · 2023-06-05T14:42:20.000-04:00
Patch 2.0.2
diff --git a/README.md b/README.md
@@ -17,11 +17,11 @@ Using the updated Spyglass UI, knowledge workers can systematically review the b
 
 Built on the Python/Django stack, Swirl is intended for use by search managers, developers, data scientists and engineers who want to solve multi-silo search problems - including notification services - without moving, re-indexing or re-permissioning sensitive information.
 
-Learn more: [Documentation Wiki](https://github.com/swirlai/swirl-search/wiki)
+### Learn more: [Documentation Wiki](https://github.com/swirlai/swirl-search/wiki)
 
 <br/>
 
-# Try SWIRL Now
+# Try Swirl Now
 
 * Download [https://raw.githubusercontent.com/swirlai/swirl-search/main/docker-compose.yaml](https://raw.githubusercontent.com/swirlai/swirl-search/main/docker-compose.yaml)
 
@@ -32,14 +32,14 @@ curl https://raw.githubusercontent.com/swirlai/swirl-search/main/docker-compose.
 * From the console:
 
 ```
-docker compose up
+docker-compose pull && docker-compose up
 ```
 
 After a few minutes the following or similar should appear:
 
 ```
 ssdtest-app-1  | Command successful!
-ssdtest-app-1  | __S_W_I_R_L__2_._0______________________________________________________________
+ssdtest-app-1  | __S_W_I_R_L__2_._0_._2________________________________________________________
 ssdtest-app-1  |
 ssdtest-app-1  | Warning: logs directory does not exist, creating it
 ssdtest-app-1  | Start: rabbitmq -> rabbitmq-server ... Ok, pid: 53
@@ -53,17 +53,13 @@ ssdtest-app-1  |   577 ?        00:00:11 celery
 ssdtest-app-1  |   609 ?        00:00:06 celery
 ssdtest-app-1  |
 ssdtest-app-1  | Command successful!
-ssdtest-app-1  | 2023-01-21 13:16:11,070 INFO     Starting server at tcp:port=8000:interface=0.0.0.0
-ssdtest-app-1  | 2023-01-21 13:16:11,074 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
-ssdtest-app-1  | 2023-01-21 13:16:11,075 INFO     Configuring endpoint tcp:port=8000:interface=0.0.0.0
-ssdtest-app-1  | 2023-01-21 13:16:11,079 INFO     Listening on TCP address 0.0.0.0:8000
+ssdtest-app-1  | 2023-05-23 13:16:11,070 INFO     Starting server at tcp:port=8000:interface=0.0.0.0
+ssdtest-app-1  | 2023-05-23 13:16:11,074 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)
+ssdtest-app-1  | 2023-05-23 13:16:11,075 INFO     Configuring endpoint tcp:port=8000:interface=0.0.0.0
+ssdtest-app-1  | 2023-05-23 13:16:11,079 INFO     Listening on TCP address 0.0.0.0:8000
 ```
 
-* Open the following URL with a browser:
-
-```
-http://localhost:8000/spyglass/
-```
+* Open this URL with a browser: http://localhost:8000/spyglass/
 
 The search page will appear. Click `Log Out` at top, right. The Swirl login page will appear:
 
@@ -75,15 +71,15 @@ Enter username `admin` and password `password`. Then click Login.
 
 ![Swirl Metasearch 2.0 Results](https://raw.githubusercontent.com/wiki/swirlai/swirl-search/images/swirl_metasearch_results.png)
 
-:warning: The Docker version of SWIRL does *not* retain results or configuration when shut down!
+:warning: The Docker version of Swirl does *not* retain results or configuration when shut down!
 
-:key: Swirl includes three (3) Google Programmable Search Engines (PSEs), complete with shared credentials, to get you up and running with right away. These credentials are shared with the SWIRL Community.
+:key: Swirl includes three (3) Google Programmable Search Engines (PSEs), complete with shared credentials, to get you up and running with right away. These credentials are shared with the Swirl Community.
 
-:key: Using Swirl with Microsoft 365 requires installation and approval by an authorized company administrator. For more information please review the [M365 Guide](https://github.com/swirlai/swirl-search/wiki/5.-M365-Guide) or [contact us](mailto:hello@swirl.today) for more information.
+:key: Using Swirl with Microsoft 365 requires installation and approval by an authorized company administrator. For more information please review the [M365 Guide](https://github.com/swirlai/swirl-search/wiki/4.-M365-Guide) or [contact us](mailto:hello@swirl.today) for more information.
 
 <br/>
 
-Learn more: [Quick Start](https://github.com/swirlai/swirl-search/wiki/1.-Quick-Start)
+### Learn more: [Quick Start](https://github.com/swirlai/swirl-search/wiki/1.-Quick-Start)
 
 <br/>
 
@@ -93,7 +89,7 @@ Learn more: [Quick Start](https://github.com/swirlai/swirl-search/wiki/1.-Quick-
 
 <br/>
 
-# Download SWIRL
+# Download Swirl
 
 | Version                     | Date                        | Notes |
 | --------------------------- | --------------------------- | ----- |
@@ -103,53 +99,53 @@ Learn more: [Quick Start](https://github.com/swirlai/swirl-search/wiki/1.-Quick-
 
 # Documentation Wiki
 
-### [Home](https://github.com/swirlai/swirl-search/wiki) | [Quick Start](https://github.com/swirlai/swirl-search/wiki/1.-Quick-Start) | [User Guide](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide) | [Developer Guide](https://github.com/swirlai/swirl-search/wiki/3.-Developer-Guide) | [Admin Guide](https://github.com/swirlai/swirl-search/wiki/4.-Admin-Guide) | [M365 Guide](https://github.com/swirlai/swirl-search/wiki/5.-M365-Guide)
+### [Home](https://github.com/swirlai/swirl-search/wiki) | [Quick Start](https://github.com/swirlai/swirl-search/wiki/1.-Quick-Start) | [User Guide](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide) | [Admin Guide](https://github.com/swirlai/swirl-search/wiki/3.-Admin-Guide) | [M365 Guide](https://github.com/swirlai/swirl-search/wiki/4.-M365-Guide) | [Developer Guide](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide) | [Developer Reference](https://github.com/swirlai/swirl-search/wiki/6.-Developer-Reference)
 
 <br/>
 
 # Key Features
 
-* [Microsoft 365 integration and OAUTH2 support](https://github.com/swirlai/swirl-search/wiki/5.-M365-Guide)
+* [Microsoft 365 integration and OAUTH2 support](https://github.com/swirlai/swirl-search/wiki/4.-M365-Guide)
 
 * [SearchProvider configurations](https://github.com/swirlai/swirl-search/tree/main/SearchProviders) for all included Connectors. They can be [organized with the active, default and tags properties](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#organizing-searchproviders-with-active-default-and-tags).
 
 * [Adaptation of the query for each provider](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#search-syntax) such as rewriting `NOT term` to `-term`, removing NOTted terms from providers that don't support NOT, and passing down the AND, + and OR operators.
 
-* [Synchronous or asynchronous search federation](https://github.com/swirlai/swirl-search/wiki/3.-Developer-Guide#architecture) via [APIs](http://localhost:8000/swirl/swagger-ui/)
+* [Synchronous or asynchronous search federation](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#architecture) via [APIs](http://localhost:8000/swirl/swagger-ui/)
 
-* [Optional subscribe feature](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#subscribing-to-a-search) to continuously monitor any search for new results
+* [Optional subscribe feature](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#subscribe-to-a-search) to continuously monitor any search for new results
 
-* Pipelining of [Processor](https://github.com/swirlai/swirl-search/wiki/3.-Developer-Guide#processors) stages for real-time adaptation and transformation of queries, responses and results
+* Pipelining of [Processor](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#develop-new-processors) stages for real-time adaptation and transformation of queries, responses and results
 
-* [Results stored](https://github.com/swirlai/swirl-search/wiki/3.-Developer-Guide#result-object) in SQLite3 or PostgreSQL for post-processing, consumption and/or analytics
+* [Results stored](https://github.com/swirlai/swirl-search/wiki/6.-Developer-Reference#result-objects) in SQLite3 or PostgreSQL for post-processing, consumption and/or analytics
 
-* Built-in [Query Transformation](https://github.com/swirlai/swirl-search/wiki/3.-Developer-Guide#query-transformations) support, including re-writing and replacement
+* Built-in [Query Transformation](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#using-query-transformations) support, including re-writing and replacement
 
-* [Matching on word stems](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#relevancy) and [handling of stopword](https://github.com/swirlai/swirl-search/wiki/3.-Developer-Guide#stopwords-language) via NLTK
+* [Matching on word stems](https://github.com/swirlai/swirl-search/wiki/6.-Developer-Reference#cosinerelevancypostresultprocessor) and [handling of stopwords](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#configure-stopwords-language) via NLTK
 
-* [Duplicate detection](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#detecting-and-removing-duplicate-results) on field or by configurable Cosine Similarity threshold
+* [Duplicate detection](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#detect-and-remove-duplicate-results) on field or by configurable Cosine Similarity threshold
 
-* Re-ranking of unified results [using Cosine Vector Similarity](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#relevancy) based on [spaCy](https://spacy.io/)'s large language model and [NLTK](https://www.nltk.org/)
+* Re-ranking of unified results [using Cosine Vector Similarity](https://github.com/swirlai/swirl-search/wiki/6.-Developer-Reference#cosinerelevancypostresultprocessor) based on [spaCy](https://spacy.io/)'s large language model and [NLTK](https://www.nltk.org/)
 
-* [Result mixers](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#result-mixers) order results by relevancy, date or round-robin (stack) format, with optional filtering of just new items in subscribe mode
+* [Result mixers](https://github.com/swirlai/swirl-search/wiki/6.-Developer-Reference#mixers-1) order results by relevancy, date or round-robin (stack) format, with optional filtering of just new items in subscribe mode
 
 * Page through all results requested, re-run, re-score and update searches using URLs provided with each result set
 
 * [Sample data sets](https://github.com/swirlai/swirl-search/tree/main/Data) for use with SQLite3 and PostgreSQL
 
-* [Optional spell correction](https://github.com/swirlai/swirl-search/wiki/2.-User-Guide#spell-correction) using [TextBlob](https://textblob.readthedocs.io/en/dev/quickstart.html#spelling-correction)
+* [Optional spell correction](https://github.com/swirlai/swirl-search/wiki/5.-Developer-Guide#add-spelling-correction) using [TextBlob](https://textblob.readthedocs.io/en/dev/quickstart.html#spelling-correction)
 
-* [Optional search/result expiration service](https://github.com/swirlai/swirl-search/wiki/4.-Admin-Guide#search-expiration-service) to limit storage use
+* [Optional search/result expiration service](https://github.com/swirlai/swirl-search/wiki/3.-Admin-Guide#search-expiration-service) to limit storage use
 
 * Easily extensible [Connector](https://github.com/swirlai/swirl-search/tree/main/swirl/connectors) and [Mixer](https://github.com/swirlai/swirl-search/tree/main/swirl/mixers) objects
 
 <br/>
 
 # Support
 
-* [Join Swirl Metasearch on Slack!](https://join.slack.com/t/swirlmetasearch/shared_invite/zt-1qk7q02eo-kpqFAbiZJGOdqgYVvR1sfw)
+* [Join the Swirl Metasearch Community on Slack!](https://join.slack.com/t/swirlmetasearch/shared_invite/zt-1qk7q02eo-kpqFAbiZJGOdqgYVvR1sfw)
 
-* [Create an Issue](https://github.com/swirlai/swirl-search/issues) if something doesn't work, isn't clear, or should be documented
+* [Create a GitHub Issue](https://github.com/swirlai/swirl-search/issues) if something doesn't work, isn't clear, or should be documented
 
 * Email: [support@swirl.today](mailto:support@swirl.today) with issues, requests, questions, etc - we'd love to hear from you!
 
diff --git a/swirl/banner.py b/swirl/banner.py
@@ -10,9 +10,9 @@ class bcolors:
     ENDC = '\033[0m'
     BOLD = '\033[1m'
 
-SWIRL_VERSION = '2.0'
+SWIRL_VERSION = '2.0.2'
 
-SWIRL_BANNER_TEXT = "__S_W_I_R_L__2_._0_._1__________________________________________________________"
+SWIRL_BANNER_TEXT = "__S_W_I_R_L__2_._0_._2__________________________________________________________"
 SWIRL_BANNER = f'{bcolors.BOLD}{SWIRL_BANNER_TEXT}{bcolors.ENDC}'
 
 #############################################
diff --git a/swirl/connectors/requests.py b/swirl/connectors/requests.py
@@ -168,6 +168,12 @@ def execute_search(self, session=None):
                             }
                             # response = requests.get(page_query, headers=headers)
                             response = self.send_request(page_query, headers=headers, query=self.query_string_to_provider)
+                        elif self.provider.credentials.startswith('X-Api-Key='):
+                            headers = {
+                                "X-Api-Key": f"{self.provider.credentials.split('X-Api-Key=')[1]}"
+                            }
+                            logger.info(f"{self}: sending request with auth header X-Api-Key")
+                            response = self.send_request(page_query, headers=headers, query=self.query_string_to_provider)
                             # all others
                         else:
                             # response = requests.get(page_query)
