Does ParallelExecution omit PostBigRequest?

[leuchtdiode]

Hi,

I'm using the PostBigRequest plugin in my Solarium client which is working fine with my "miles long queries".

But it looks like in combination with ParallelExecution the plugin gets omitted?

The result after executing in parallel throws "Request URI too long".

Is it possible to combine these two plugins?

Thx & BR,
Alex

[thomascorthals]

Oh right, I forgot that ParallelExecution kind of hides the error if you don't go looking for it. My example does in fact encounter the same error as you did, @leuchtdiode. Sorry to get your hopes up too soon!

[leuchtdiode]

Yes I was excited too that the requests executed so fast until I iterated the results array and saw the error.

I looked at the execute method which uses curl_multi* without POST and not respecting PostBigRequest as it looks like.

Would it be possible to enhance it?

BR,
Alex

[thomascorthals]

I've opened #1015 to look into this.

[leuchtdiode]

Thanks.

[thomascorthals]

Hi Alex,

This has been fixed in Solarium 6.2.5.

You can test it by modifying the ParallelExecution example.

<?php

require_once(__DIR__.'/init.php');

htmlHeader();

// create a client instance and autoload the customize request plugin
$client = new Solarium\Client($adapter, $eventDispatcher, $config);
$parallel = $client->getPlugin('parallelexecution');
$client->getPlugin('postbigrequest');

// create two queries to execute
$queryInStock = $client->createSelect()->setQuery('inStock:true');
$queryLowPrice = $client->createSelect()->setQuery('price:[1 TO 30]');

// add a huge filterquery to create a very long query string
$fq = '-cat:'.str_repeat(implode('', range('a', 'z')), 1000);
$queryInStock->createFilterQuery('fq')->setQuery($fq);
$queryLowPrice->createFilterQuery('fq')->setQuery($fq);

// first execute the queries the normal way and time it
echo '<h1>Serial execution</h1>';
$start = microtime(true);
$resultInStock = $client->execute($queryInStock);
$resultLowPrice = $client->execute($queryLowPrice);
echo 'Execution time for normal "serial" execution of two queries: ' . round(microtime(true)-$start, 3) . ' s';
echo '<hr/>';
echo 'In stock: ' . $resultInStock->getNumFound() . '<br/>';
echo 'Low price: ' . $resultLowPrice->getNumFound() . '<br/>';

echo '<hr/>';

// now execute the two queries parallel and time it
echo '<h1>Parallel execution</h1>';
$start = microtime(true);
// keys for each query are important for fetching the results later!
$parallel->addQuery('instock', $queryInStock);
$parallel->addQuery('lowprice', $queryLowPrice);
$results = $parallel->execute();
echo 'Execution time for parallel execution of two queries: ' . round(microtime(true)-$start, 3) . ' s';
echo '<hr/>';
echo 'In stock: ' . $results['instock']->getNumFound() . '<br/>';
echo 'Low price: ' . $results['lowprice']->getNumFound() . '<br/>';

htmlFooter();

The order in which the plugins are registered doesn't matter.

Regards,

Thomas

[leuchtdiode]

Thank you very much, I will give it a try very soon.

BR,
Alex

[thomascorthals]

Probably something to do with #994 which was meant to fix a bug with single nested child documents that was an unfortunate side effect of another bugfix.

Can you share the code to reproduce this?

[leuchtdiode]

I can send a var_dump of the document or what would you need exactly?

[thomascorthals]

A var_dump and the code that creates/executes the request.

[thomascorthals]

And the Solr version could be relevant as well.

[thomascorthals]

The solr version is the newest: 6.2.5.

That's the Solarium version. 😉

Are you trying to index this with ParallelExecution? (I have to admit I didn't test with update queries.) If not, can you copy the question to a new discussion?

[leuchtdiode]

LOL right, sorry. Solr version is 8.6.1

Yes, I'm enabling PostBigRequest generally in when creating the client.