Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ability to query bulk operations across multiple bulk_uuid values (broken by 2.4 release) #29579

Open
wants to merge 13 commits into
base: 2.5-develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ public function retryBulk($bulkUuid, array $errorCodes)
$currentBatchSize = 0;
}
$currentBatchSize++;
$operationIds[] = $operation->getId();
$operationIds[] = $operation->getOperationKey();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not oberation ids anymore. Pls rename it in all places

Suggested change
$operationIds[] = $operation->getOperationKey();
$operationKeys[] = $operation->getOperationKey();

}
// remove operations from the last batch
if (!empty($operationIds)) {
Expand Down
17 changes: 17 additions & 0 deletions app/code/Magento/AsynchronousOperations/Model/Operation.php
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,23 @@ public function setErrorCode($errorCode)
return $this->setData(self::ERROR_CODE, $errorCode);
}

/**
* @inheritDoc
*/
public function getOperationKey()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing return type on prototype

{
return $this->getData(self::OPERATION_KEY);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK when data returning from DB - it might be a string. Could you convert it to int here?

}

/**
* @inheritDoc
*/
public function setOperationKey(int $operationKey)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing return type on prototype

{
$this->setData(self::OPERATION_KEY, $operationKey);
return $this;
}

/**
* Retrieve existing extension attributes object.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ public function createByTopic($topicName, $entityParams, $groupId, $operationId)
];
$data = [
'data' => [
OperationInterface::ID => $operationId,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget this class exists as well in the webapi async module, and needs to be changed there as well.

Magento/WebApiAsync/Model/OperationRepository.php:92

OperationInterface::OPERATION_KEY => $operationId,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should not work correctly.
OPERATION_KEY - its a DB key for "operation_key" column.

if you switch OPERATION_KEY to "id", it will stop working, cause $operationId here is a key of array, and in database "id" is an auto_increment value. And in case of multiple operations, will be duplicated for different bulk operations.

Implementing of OPERATION_KEY was done in purpose, for decouple operation keys from database increments and improve performance on saving big operations.

Another issue, it that not all Magento API functionality have a tests coverage, so we overslept broken another part of API

OperationInterface::BULK_ID => $groupId,
OperationInterface::TOPIC_NAME => $topicName,
OperationInterface::SERIALIZED_DATA => $this->jsonSerializer->serialize($serializedData),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ public function testRetryBulk()
$operation = $this->getMockForAbstractClass(OperationInterface::class);
$operationCollection->expects($this->once())->method('getItems')->willReturn([$operation]);
$connection->expects($this->once())->method('beginTransaction')->willReturnSelf();
$operation->expects($this->once())->method('getId')->willReturn($operationId);
$operation->expects($this->once())->method('getOperationKey')->willReturn($operationId);
$this->resourceConnection->expects($this->once())
->method('getTableName')->with($operationTable)->willReturn($operationTable);
$connection->expects($this->at(1))
Expand Down Expand Up @@ -265,7 +265,7 @@ public function testRetryBulkWithException()
$operation = $this->getMockForAbstractClass(OperationInterface::class);
$operationCollection->expects($this->once())->method('getItems')->willReturn([$operation]);
$connection->expects($this->once())->method('beginTransaction')->willReturnSelf();
$operation->expects($this->once())->method('getId')->willReturn($operationId);
$operation->expects($this->once())->method('getOperationKey')->willReturn($operationId);
$this->resourceConnection->expects($this->once())
->method('getTableName')->with($operationTable)->willReturn($operationTable);
$connection->expects($this->at(1))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -121,4 +121,52 @@ public function testGetList()
$this->assertEquals('bulk-uuid-searchable-6', $item['bulk_uuid']);
}
}

/**
* @magentoApiDataFixture Magento/AsynchronousOperations/_files/operation_searchable.php
*/
public function testGetMultipleBulkUuids()
{
$searchCriteria = [
'searchCriteria' => [
'filter_groups' => [
[
'filters' => [
[
'field' => 'status',
'value' => OperationInterface::STATUS_TYPE_COMPLETE,
'condition_type' => 'eq',
],
],
],
],
'current_page' => 1,
],
];

$serviceInfo = [
'rest' => [
'resourcePath' => self::RESOURCE_PATH . '?' . http_build_query($searchCriteria),
'httpMethod' => Request::HTTP_METHOD_GET,
],
'soap' => [
'service' => self::SERVICE_NAME,
'operation' => self::SERVICE_NAME . 'GetList',
],
];

$response = $this->_webApiCall($serviceInfo, $searchCriteria);

$this->assertArrayHasKey('search_criteria', $response);
$this->assertArrayHasKey('total_count', $response);
$this->assertArrayHasKey('items', $response);

$this->assertEquals($searchCriteria['searchCriteria'], $response['search_criteria']);
$this->assertEquals(2, $response['total_count']);
$this->assertCount(2, $response['items']);

foreach ($response['items'] as $item) {
$this->assertEquals('0', $item['operation_key']);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ public function testGetBulkStatus()
}
/** @var OperationInterface $operation */
$operation = array_shift($operations);
$operationId = $operation->getId();
$operationId = $operation->getOperationKey();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$operationId = $operation->getOperationKey();
$operationKey = $operation->getOperationKey();


$this->assertTrue($this->model->changeOperationStatus(
'bulk-uuid-5',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,15 @@
'result_message' => '',
'operation_key' => 5
],
[
'bulk_uuid' => 'bulk-uuid-searchable-6-1',
'topic_name' => 'topic-5',
'serialized_data' => json_encode(['entity_id' => 5]),
'status' => OperationInterface::STATUS_TYPE_COMPLETE,
'error_code' => null,
'result_message' => '',
'operation_key' => 0
],
];

$bulkQuery = "INSERT INTO {$bulkTable} (`uuid`, `user_id`, `description`, `operation_count`, `start_time`)"
Expand Down
20 changes: 19 additions & 1 deletion lib/internal/Magento/Framework/Bulk/OperationInterface.php
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,12 @@ interface OperationInterface extends \Magento\Framework\Api\ExtensibleDataInterf
/**#@+
* Constants for keys of data array. Identical to the name of the getter in snake case
*/
const ID = 'operation_key';
const ID = 'id';
const BULK_ID = 'bulk_uuid';
const TOPIC_NAME = 'topic_name';
const SERIALIZED_DATA = 'serialized_data';
const RESULT_SERIALIZED_DATA = 'result_serialized_data';
const OPERATION_KEY = 'operation_key';
const STATUS = 'status';
const RESULT_MESSAGE = 'result_message';
const ERROR_CODE = 'error_code';
Expand Down Expand Up @@ -172,4 +173,21 @@ public function getErrorCode();
* @since 103.0.0
*/
public function setErrorCode($errorCode);

/**
* Get operation key
*
* @return int
* @since 103.0.1
*/
public function getOperationKey();

/**
* Set operation key
*
* @param int $operationKey
* @return $this
* @since 103.0.1
*/
public function setOperationKey(int $operationKey);
}