Merge branch 'release/3.0.0'

# Conflicts:
#	tntsearch.php

Author: rhukster

CHANGELOG.md

@@ -1,3 +1,24 @@
+# v3.0.0
+## 04/14/2018
+
+1. [](#new)
+    * Added new Grav Scheduler integration
+    * Added new Multi-Language Support
+1. [](#improved)
+    * Switched to latest TNTSearch version 2.0 (PHP 7.1+)
+    * Added a new `onFlexObjecSave()` event
+    * Simplified indexing logic
+    * Code cleanup
+    * Minor CSS improvements for search field
+    * Implemented a unified indexer process that always uses the CLI command for consistency
+    * Use Grav YAML handler
+1. [](#bugfix)
+    * Use custom search object in query [#63](https://github.com/trilbymedia/grav-plugin-tntsearch/issues/63)  
+    * Fixed issue with Ajax results escaping
+    * Fixed issues when updating search index  
+    * Set the db index file as a property of `GravTNTSearch` to allow for better overriding
+    * Put better type checking around `onTNTSearchIndex()` example that indexes `page.header.author`
+   
 # v2.0.4
 ## 09/21/2018
 

README.md

@@ -2,6 +2,8 @@
 
 The **TNTSearch** Plugin is for [Grav CMS](http://github.com/getgrav/grav). Powerful indexed-based full text search engine powered by the [TNTSearch library](https://github.com/teamtnt/tntsearch) that provides fast Ajax-based Grav content searches.  This plugin is highly flexible allowing indexes of arbitrary content data as well as custom Twig templates to provide the opportunity to index modular and other dynamic page types. TNTSearch provides CLI as well as Admin based administration and re-indexing, as well as a built-in Ajax-powered front-end search tool.
 
+> NOTE: TNTSearch version 3.0.0 now requires Grav 1.6.0 or newer to function as it makes use of new functionality not available in previous versions.
+
 ![](assets/tntsearch-ajax.gif)
 
 ## Installation
@@ -63,8 +65,10 @@ query_route: '/s'
 built_in_css: true
 built_in_js: true
 built_in_search_page: true
+enable_admin_page_events: true
 search_type: auto
 fuzzy: false
+phrases: true
 stemmer: default
 display_route: true
 display_hits: true
@@ -74,10 +78,15 @@ limit: 20
 min: 3
 snippet: 300
 index_page_by_default: true
+scheduled_index:
+    enabled: false
+    at: '0 */3 * * *'
+    logs: 'logs/tntsearch-index.out'
 filter:
   items:
-    taxonomy@:
-      category: [news]
+    - [email protected]
+powered_by: true
+search_object_type: Grav      
 ```
 
 The configuration options are as follows:
@@ -88,11 +97,13 @@ The configuration options are as follows:
 * `built_in_css` - enable or disable the built-in css styling
 * `built_in_js` - enable or disable the built-in javascript
 * `built_in_search_page` - enable or disable the built-in search page
+* `enable_admin_page_events` - enable or disable the page events which occur `on-save` to add/update/remove page in index
 * `search_type` - can be one of these types:
   * `basic` - standard string matching
   * `boolean` - supports `or` or `minus`. e.g. `foo -bar`
   * `auto` - automatically detects whether to use `basic` or `boolean`
 * `fuzzy` - matches if the words are 'close' but not necessarily exact matches
+* `phrases` - automatically handle phrases support
 * `stemmer` - can be one of these types:
   * `default` - no stemmer
   * `arabic` - Arabic language
@@ -109,7 +120,10 @@ The configuration options are as follows:
 * `min` - mininum amount of characters typed before performing search
 * `snippet` - amount of characters for previewing a result item
 * `index_page_by_default` - should all pages be indexed by default unless frontmatter overrides
+* `scheduled_index` - New scheduled index job.  Disabled by default, when enabled defaulted to run every 3 hours, and output results to `logs/tntsearch-index.out`
 * `filter` - a [Page Collections filter](https://learn.getgrav.org/content/collections#summary-of-collection-options) that lets you pick specific pages to index via a collection query
+* `powered_by` - Display the **powered-by TNTSearch** text
+* `search_object_type` - Allows custom classes to override the default **Grav Page** support. This allows completely custom searching capabilities for any data type.
 
 ## Usage
 
@@ -168,6 +182,20 @@ tntsearch:
     index: false
 ```
 
+#### Multi-Language Support
+
+With the new 3.0 version of TNTSearch, support has been added for multiple languages (Grav 1.6 required).  Internally, this means that rather that store the index as `user:://data/tntsearch/grav.index`, multiple indexes are created per language configured in Grav.  For example if you have set the supported languages to `['en', 'fr', 'de']`, then when you perform an index,  you will get three files: `en.index`, `fr.index`, and `de.index`.  When querying the appropriate **active language** determines which index is queried.  For example, performing the search on a page called `/fr/search` will result in the `fr.index` database to be used, and French results to be returned.  
+
+Note Indexing will take longer depending on the number of languages you support as TNTSearch has to index each page in each language.
+
+> NOTE: While accented characters is supported in this release, there is currently no support in the underlying TNTSearch library to match non-accented characters to accented ones, so exact matches are required.
+
+#### Scheduler Support
+
+One of the great new features of Grav 1.6 is the built in **Scheduler** that allows plugin-provided functionality to be run periodically.  TNTSearch is a great use-case for this capability as it allows an indexing job to be scheduled to be run every few hours without the need to manually keep things in sync. There are a few options that allow you to configure this capability.
+
+First note, that this scheduler functionality is disable by default, so you first have to enable the scheduler functionality in the TNTSearch plugin settings.  After that you can configure how often the indexing job should run.  The default is every 3 hours. Lastly, you can configure where any indexing output is logged to.
+
 #### Admin Page CrUD Events
 
 Once you have an index, TNTSearch makes use of admin events to **C**reate, **U**pdate, and **D**elete index entries when you edit pages.  If your index ever looks like it's out of sync, you can simply reindex your whole site.

assets/tntsearch.css

@@ -1,3 +1,8 @@
+.tntsearch-form .form-input {
+    height: 2.4rem;
+    padding-left: 1rem;
+}
+
 #tntsearch-wrapper {
     position: relative;
 }

blueprints.yaml

@@ -1,5 +1,6 @@
 name: TNT Search
-version: 2.0.4
+version: 3.0.0
+testing: false
 description: Powerful indexed-based full text search engine powered by TNTSearch
 icon: binoculars
 author:
@@ -12,7 +13,7 @@ docs: https://github.com/trilbymedia/grav-plugin-tntsearch/blob/develop/README.m
 license: MIT
 
 dependencies:
-    - { name: grav, version: '>=1.3.2' }
+    - { name: grav, version: '>=1.6.3' }
 
 form:
   validation: strict
@@ -30,13 +31,18 @@ form:
       validate:
         type: bool
 
+    index_title:
+      type: spacer
+      title: Indexer Settings
+
     index_status:
       type: indexstatus
       label: Search Index Status
 
-    built_in_css:
+    enable_admin_page_events:
       type: toggle
-      label: Built-in CSS
+      label: Enable Admin Page Events
+      help: Disable this if you are having problems with timeouts during page saving
       highlight: 1
       default: 1
       options:
@@ -45,9 +51,39 @@ form:
       validate:
         type: bool
 
-    built_in_js:
+    scheduled_index.enabled:
       type: toggle
-      label: Built-in Javascript
+      label: Enable Index Scheduled Job
+      help: Use the Grav Scheduler to kick off a background index job
+      highlight: 0
+      default: 0
+      options:
+        1: Enabled
+        0: Disabled
+      validate:
+        type: bool
+
+    scheduled_index.at:
+      type: cron
+      label: Scheduled Job Frequency
+      size: medium
+      help: Use 'cron' format
+      default: '0 */3 * * *'
+      placeholder: '0 */3 * * *'
+
+    scheduled_index.logs:
+      type: text
+      label: Scheduled Job Log File
+      placeholder: 'logs/tntsearch-index.out'
+      size: medium
+
+    ui_title:
+      type: spacer
+      title: UI Settings
+
+    built_in_css:
+      type: toggle
+      label: Built-in CSS
       highlight: 1
       default: 1
       options:
@@ -56,9 +92,9 @@ form:
       validate:
         type: bool
 
-    built_in_search_page:
+    built_in_js:
       type: toggle
-      label: Built-in Search Page
+      label: Built-in Javascript
       highlight: 1
       default: 1
       options:
@@ -67,10 +103,13 @@ form:
       validate:
         type: bool
 
-    enable_admin_page_events:
+    search_title:
+      type: spacer
+      title: Search Settings
+
+    built_in_search_page:
       type: toggle
-      label: Enable Admin Page Events
-      help: Disable this if you are having problems with timeouts during page saving
+      label: Built-in Search Page
       highlight: 1
       default: 1
       options:
@@ -79,6 +118,8 @@ form:
       validate:
         type: bool
 
+
+
     search_route:
       type: text
       size: medium
@@ -202,7 +243,7 @@ form:
     index_page_by_default:
       type: toggle
       label: Index Every Page
-      help: Index every page by default unless a page specifically declares `tntsearch: process: false`. Disabling this requires a `process: true` declartion to be added to each page that should be indexed.
+      help: 'Index every page by default unless a page specifically declares `tntsearch: process: false`. Disabling this requires a `process: true` declartion to be added to each page that should be indexed.'
       highlight: 1
       default: 1
       options:

classes/GravConnector.php

@@ -2,8 +2,8 @@
 namespace Grav\Plugin\TNTSearch;
 
 use Grav\Common\Grav;
+use Grav\Common\Yaml;
 use Grav\Common\Page\Page;
-use Symfony\Component\Yaml\Yaml;
 
 class GravConnector extends \PDO
 {
@@ -25,8 +25,7 @@ public function query($query)
         $config = Grav::instance()['config'];
         $filter = $config->get('plugins.tntsearch.filter');
         $default_process = $config->get('plugins.tntsearch.index_page_by_default');
-        $gtnt = new GravTNTSearch();
-
+        $gtnt = \Grav\Plugin\TNTSearchPlugin::getSearchObjectType();
 
 
         if ($filter && array_key_exists('items', $filter)) {
@@ -61,7 +60,8 @@ public function query($query)
             try {
                 $fields = $gtnt->indexPageData($page);
                 $results[] = (array) $fields;
-                echo("Added   $counter $route\n");
+                $display_route = isset($fields->display_route) ? $fields->display_route : $route;
+                echo("Added   $counter $display_route\n");
             } catch (\Exception $e) {
                 echo("Skipped $counter $route - {$e->getMessage()}\n");
                 continue;

classes/GravTNTSearch.php

@@ -2,10 +2,11 @@
 namespace Grav\Plugin\TNTSearch;
 
 use Grav\Common\Grav;
+use Grav\Common\Language\Language;
+use Grav\Common\Yaml;
 use Grav\Common\Page\Collection;
 use Grav\Common\Page\Page;
 use RocketTheme\Toolbox\Event\Event;
-use Symfony\Component\Yaml\Yaml;
 use TeamTNT\TNTSearch\Exceptions\IndexNotFoundException;
 use TeamTNT\TNTSearch\TNTSearch;
 
@@ -15,6 +16,7 @@ class GravTNTSearch
     protected $options;
     protected $bool_characters = ['-', '(', ')', 'or'];
     protected $index = 'grav.index';
+    protected $language;
 
     public function __construct($options = [])
     {
@@ -24,6 +26,15 @@ public function __construct($options = [])
         $snippet = Grav::instance()['config']->get('plugins.tntsearch.snippet', 300);
         $data_path = Grav::instance()['locator']->findResource('user://data', true) . '/tntsearch';
 
+        /** @var Language $language */
+        $language = Grav::instance()['language'];
+
+        if ($language->enabled()) {
+            $active = $language->getActive();
+            $default = $language->getDefault();
+            $this->language = $active ? $active : $default;
+            $this->index =  $this->language . '.index';
+        }
 
         if (!file_exists($data_path)) {
             mkdir($data_path);
@@ -44,6 +55,7 @@ public function __construct($options = [])
         $this->tnt->loadConfig([
             "storage"   => $data_path,
             "driver"    => 'sqlite',
+            'charset'	=> 'utf8'
         ]);
     }
 
@@ -131,7 +143,7 @@ public static function getCleanContent($page)
 
         if (isset($header->tntsearch['template'])) {
             $processed_page = $twig->processTemplate($header->tntsearch['template'] . '.html.twig', ['page' => $page]);
-            $content =$processed_page;
+            $content = $processed_page;
         } else {
             $content = $page->content();
         }
@@ -232,11 +244,17 @@ public function indexPageData($page)
             throw new \RuntimeException('redirect only...');
         }
 
+        $route = $page->route();
+
         $fields = new \stdClass();
-        $fields->id = $page->route();
+        $fields->id = $route;
         $fields->name = $page->title();
         $fields->content = $this->getCleanContent($page);
 
+        if ($this->language) {
+            $fields->display_route = '/' . $this->language . $route;
+        }
+
         Grav::instance()->fireEvent('onTNTSearchIndex', new Event(['page' => $page, 'fields' => $fields]));
 
         return $fields;