how to visit the plain index and rt index at same time #1297

l1t1 · 2023-07-29T08:11:06Z

l1t1
Jul 29, 2023

if didn't comment the data_dir line, searchd raise an error.
or it only show the plain index.

root@localhost:~/dy/ml-latest-small# cat confd
searchd {
    listen = 127.0.0.1:9312
    listen = 127.0.0.1:9306:mysql
    listen = 127.0.0.1:9308:http
    log = /var/log/manticore/searchd.log
    query_log = /var/log/manticore/query.log
    pid_file = /var/run/manticore/searchd.pid
#    data_dir = /var/lib/manticore
    query_log_format = sphinxql
}
source csv_band
{
    type = csvpipe
    csvpipe_command = cat bands.csv
    csvpipe_field = name
    csvpipe_attr_multi = genre_tags
}
table band {
  type   = plain
  source = csv_band
  path   = /var/lib/manticore/band
 }
root@localhost:~/dy/ml-latest-small# searchd -c confd
Manticore 6.0.4 b16b9cc97@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2023, Manticore Software LTD (https://manticoresearch.com)

[54:39.470] [13696] using config file '/root/dy/ml-latest-small/confd' (530 chars)...
[54:39.472] [13696] TCP fast open unavailable (can't read /proc/sys/net/ipv4/tcp_fastopen, look Server_settings/Searchd#Technical-details-about-Sphinx-API-protocol-and-TFO in manual)
starting daemon version '6.0.4 b16b9cc97@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)' ...
listening on 127.0.0.1:9312 for sphinx and http(s)
listening on 127.0.0.1:9306 for mysql
listening on 127.0.0.1:9308 for sphinx and http(s)
precaching table 'band'
precached 1 tables in 0.012 sec
root@localhost:~/dy/ml-latest-small# mysql -hlocalhost -P9306
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 6.0.4 b16b9cc97@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306) git branch HEAD (no branch)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> show tables;
+-------+-------+
| Index | Type  |
+-------+-------+
| band  | local |
+-------+-------+
1 row in set (0.002 sec)

Answered by tomatolog

Jul 29, 2023

you have rt indexes with data into /var/lib/manticore with additional folder for each index but for plain mode you put all indexes into folder

could you check or provide content of the data folder?

View full answer

l1t1 · 2023-07-29T09:19:23Z

l1t1
Jul 29, 2023
Author

try to create a rt table and failed.

MySQL [(none)]> create table t(a int,b text);

ERROR 1064 (42000): CREATE TABLE requires data_dir to be set in the config file

1 reply

tomatolog Jul 29, 2023
Collaborator

you can not use RT mode (data_dir \ create table statement) and plain indexes (source definition at the config ) at the same time as described at the manual

you need to define RT index at your config - you could check RT index definition at the manual - just select config tab at the examples there

l1t1 · 2023-07-29T12:45:07Z

l1t1
Jul 29, 2023
Author

I add following config as you told, the rt tables can be visit now, but no data found.


MySQL [(none)]> describe tbl;
+-------+--------+----------------+
| Field | Type   | Properties     |
+-------+--------+----------------+
| id    | bigint |                |
| title | text   | indexed stored |
| type  | uint   |                |
| price | float  | columnar       |
+-------+--------+----------------+
4 rows in set (0.002 sec)



table tbl
{
    type = rt
    path = /var/lib/manticore/tbl

    # when configuring fields via config, they are indexed (and not stored) by default
    rt_field = title
 

    # this option needs to be specified for the field to be stored
    stored_fields = title

    rt_attr_uint = type
    rt_attr_float = price
 
}

MySQL [(none)]> describe testrt;
+---------+--------+----------------+
| Field   | Type   | Properties     |
+---------+--------+----------------+
| id      | bigint |                |
| title   | text   | indexed stored |
| content | text   | indexed stored |
| gid     | uint   |                |
+---------+--------+----------------+
4 rows in set (0.001 sec)



table testrt
{
    type = rt
    path = /var/lib/manticore/testrt

    # when configuring fields via config, they are indexed (and not stored) by default
    rt_field = title
    rt_field = content

    # this option needs to be specified for the field to be stored
    stored_fields = title, content

    rt_attr_uint = gid
 
}

precaching table 'tbl'
precaching table 'testrt'
precaching table 'band'
precached 3 tables in 0.020 sec
root@localhost:~/dy/ml-latest-small# mysql -hlocalhost -P9306
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 6.0.4 b16b9cc97@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306) git branch HEAD (no branch)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> select * from tbl;
Empty set (0.003 sec)

MySQL [(none)]> describe tbl;
+-------+--------+----------------+
| Field | Type   | Properties     |
+-------+--------+----------------+
| id    | bigint |                |
| title | text   | indexed stored |
| type  | uint   |                |
| price | float  |                |
+-------+--------+----------------+
4 rows in set (0.002 sec)

0 replies

l1t1 · 2023-07-29T13:02:25Z

l1t1
Jul 29, 2023
Author

they have data in rt mode

root@localhost:~/dy/ml-latest-small# searchd
Manticore 6.0.4 b16b9cc97@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)
Copyright (c) 2001-2016, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)
Copyright (c) 2017-2023, Manticore Software LTD (https://manticoresearch.com)

[46:01.868] [1922] using config file '/etc/manticoresearch/manticore.conf' (305 chars)...
[46:01.872] [1922] TCP fast open unavailable (can't read /proc/sys/net/ipv4/tcp_fastopen, look Server_settings/Searchd#Technical-details-about-Sphinx-API-protocol-and-TFO in manual)
starting daemon version '6.0.4 b16b9cc97@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306)' ...
listening on 127.0.0.1:9312 for sphinx and http(s)
listening on 127.0.0.1:9306 for mysql
listening on 127.0.0.1:9308 for sphinx and http(s)
precaching table 'testrt'
precaching table 'tbl'
precached 2 tables in 0.018 sec
root@localhost:~/dy/ml-latest-small# mysql -hlocalhost -P9306
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 6.0.4 b16b9cc97@230314 (columnar 2.0.4 5a49bd7@230306) (secondary 2.0.4 5a49bd7@230306) git branch HEAD (no branch)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> select * from tbl;
+---------------------+------+----------+-------+
| id                  | type | price    | title |
+---------------------+------+----------+-------+
| 1515454872139333634 |    3 | 4.000000 | a     |
| 1515454872139333633 |    2 | 3.000000 | 1     |
+---------------------+------+----------+-------+
2 rows in set (0.003 sec)

MySQL [(none)]> select * from testrt;
+---------------------+------+--------------+----------------------+
| id                  | gid  | title        | content              |
+---------------------+------+--------------+----------------------+
| 1515452682192551937 |    1 | first record | 买新的Apple电脑 |
+---------------------+------+--------------+----------------------+
1 row in set (0.005 sec)

1 reply

tomatolog Jul 29, 2023
Collaborator

you have rt indexes with data into /var/lib/manticore with additional folder for each index but for plain mode you put all indexes into folder

could you check or provide content of the data folder?

Answer selected by l1t1

l1t1 · 2023-07-29T13:54:15Z

l1t1
Jul 29, 2023
Author

root@localhost:~/dy/ml-latest-small# ll /var/lib/manticore
total 0
drwxr-xr-x 7 root root   3488 Jul 29 12:46 ./
drwxr-xr-x 9 root root   3488 Jul 22 12:03 ../
drwx------ 2 root root   3488 Jul 29 11:47 band/
-rw-r--r-- 1 root root    112 Jul 29 11:40 band.spa
-rw-r--r-- 1 root root 131130 Jul 29 11:40 band.spb
-rw-r--r-- 1 root root     31 Jul 29 11:40 band.spd
-rw-r--r-- 1 root root     80 Jul 29 11:40 band.spds
-rw-r--r-- 1 root root      1 Jul 29 11:40 band.spe
-rw-r--r-- 1 root root    673 Jul 29 11:40 band.sph
-rw-r--r-- 1 root root    192 Jul 29 11:40 band.sphi
-rw-r--r-- 1 root root     88 Jul 29 11:40 band.spi
-rw-r--r-- 1 root root    306 Jul 29 11:40 band.spidx
-rw-r--r-- 1 root root      4 Jul 29 11:40 band.spm
-rw-r--r-- 1 root root      1 Jul 29 11:40 band.spp
-rw-r--r-- 1 root root     46 Jul 29 11:40 band.spt
drwx------ 2 root root   3488 Jul 29 12:46 binlog/
drwxr-xr-x 2 root root   3488 Jul 29 12:45 data/
-rw------- 1 root root    144 Jul 29 12:32 manticore.json
-rw------- 1 root root      0 Jul 29 12:46 state.sql
drwx------ 2 root root   3488 Jul 29 12:32 tbl/
-rw------- 1 root root      0 Jul 29 12:37 tbl.lock
-rw------- 1 root root    632 Jul 29 12:45 tbl.meta
-rw------- 1 root root     20 Jul 29 12:45 tbl.ram
drwx------ 2 root root   3488 Jul 29 12:32 testrt/
-rw------- 1 root root      0 Jul 29 12:37 testrt.lock
-rw------- 1 root root    603 Jul 29 12:45 testrt.meta
-rw------- 1 root root     28 Jul 29 12:45 testrt.ram

0 replies

l1t1 · 2023-07-29T13:56:52Z

l1t1
Jul 29, 2023
Author

root@localhost:~/dy/ml-latest-small# ll /var/lib/manticore/tbl
total 0
drwx------ 2 root root 3488 Jul 29 12:32 ./
drwxr-xr-x 7 root root 3488 Jul 29 12:46 ../
-rw------- 1 root root    0 Jul 29 12:46 tbl.lock
-rw------- 1 root root  628 Jul 29 12:32 tbl.meta
-rw------- 1 root root  224 Jul 29 12:32 tbl.ram
-rw------- 1 root root    3 Jul 29 12:32 tbl.settings
root@localhost:~/dy/ml-latest-small# ll /var/lib/manticore/testrt
total 0
drwx------ 2 root root 3488 Jul 29 12:32 ./
drwxr-xr-x 7 root root 3488 Jul 29 12:46 ../
-rw------- 1 root root    0 Jul 29 12:46 testrt.lock
-rw------- 1 root root  651 Jul 29 12:32 testrt.meta
-rw------- 1 root root  216 Jul 29 12:32 testrt.ram
-rw------- 1 root root    3 Jul 29 12:32 testrt.settings
root@localhost:~/dy/ml-latest-small# ll /var/lib/manticore/band
total 0
drwx------ 2 root root 3488 Jul 29 11:47 ./
drwxr-xr-x 7 root root 3488 Jul 29 12:46 ../

0 replies

l1t1 · 2023-07-29T14:01:46Z

l1t1
Jul 29, 2023
Author

after edit path = /var/lib/manticore/testrt
to path = /var/lib/manticore/testrt/testrt
the data can be fetched now
thanks

0 replies

l1t1 · 2023-07-29T14:10:27Z

l1t1
Jul 29, 2023
Author

after edit path = /var/lib/manticore/testrt
to path = /var/lib/manticore/testrt/testrt
the data of testrt can be fetched now
thanks

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

how to visit the plain index and rt index at same time #1297

{{title}}

Replies: 7 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

how to visit the plain index and rt index at same time #1297

l1t1 Jul 29, 2023

Replies: 7 comments · 2 replies

l1t1 Jul 29, 2023 Author

tomatolog Jul 29, 2023 Collaborator

l1t1 Jul 29, 2023 Author

l1t1 Jul 29, 2023 Author

tomatolog Jul 29, 2023 Collaborator

l1t1 Jul 29, 2023 Author

l1t1 Jul 29, 2023 Author

l1t1 Jul 29, 2023 Author

l1t1 Jul 29, 2023 Author

l1t1
Jul 29, 2023

Replies: 7 comments 2 replies

l1t1
Jul 29, 2023
Author

tomatolog Jul 29, 2023
Collaborator

l1t1
Jul 29, 2023
Author

l1t1
Jul 29, 2023
Author

tomatolog Jul 29, 2023
Collaborator

l1t1
Jul 29, 2023
Author

l1t1
Jul 29, 2023
Author

l1t1
Jul 29, 2023
Author

l1t1
Jul 29, 2023
Author