RediSearch response support?

[eaziz4]

When executing the following command, I would assume the response would be 1 element with the amount of records, and then the rest being the ids of the matching records. I am instead getting a lot of extra nodes that just contain metadata that I do not care about(as seen in the attached picture). How can I only return the elements that I care about(like running the same command on cli would give me) and not extra metadata?

const auto query = buildQuery(payload);
boost::redis::request searchReq;
searchReq.push("FT.SEARCH", "idx", query, "NOCONTENT", "LIMIT", "0", "10000");

boost::redis::generic_response response;
co_await conn->async_exec(searchReq, response, boost::asio::deferred);

My usecase would most likely want me to return all of my columns(20) for each key as well which gives me around 1,600,000 nodes in my response for 34,000 matching keys where so many of them are just metadata. In production, I may have to match over 200,000 keys which would be a very large amount of data in memory and seems like I must be doing something really wrong. This might just be my lack of knowledge around redis. Is there a way to serialize this data into a vector of custom objects?

[criatura2]

Hi, ft.search has a non trivial return type and can't be mapped easily into STL containers, see here https://redis.io/docs/latest/commands/ft.search.

What you can do is to understand that return type and write a function that converts it into a data structure that suits your use.

[eaziz4]

Thank you for the help!. I'm also seeing that this request takes around .45 seconds to execute for 34,000 keys. I'm not sure if its my redis stack configuration that is leading to it taking so long, something with how I'm executing the commands using this client library, or just the nature of my command. I am trying to switch my datasource from parquet files with 20 columns to redis hashes and searching using an index. I expected considerable speed improvements with this, but I'm seeing a >5x increase in time using redis. I am simply running my redis server using docker with this command docker run -d --name redis-stack -p 6379:6379 redis/redis-stack-server:latest

// const auto query = buildQuery(payload);
const std::string query = "*";
boost::redis::request searchReq;
searchReq.push("FT.SEARCH", "betsIdx", query, "RETURN", "21",
            "id", "devigId", "marketId", "book", "betName", "market", "selection", "odds", "line", "limit", "player", "isMainline", "isPlayerProp",
            "link", "scraped", "eventId", "homeTeam", "awayTeam", "sport", "league", "startTime",
            "LIMIT", "0", "200000");

boost::redis::generic_response response;
co_await conn->async_exec(searchReq, response, boost::asio::use_awaitable);

[anarthal]

As a random idea, you might use Wireshark with the resp3 protocol filter to see the timings of the frames and see where the slowness comes from.

[criatura2]

@eaziz4 How much time does it take to execute that command with redis-cli? Does the keys command also take that long? If yes you can try to use hscan with a cursor to see how exactly the time starts growing so much.