[Transformations][GPU] Constant tensor deduplication pass #29052

dnkurek · 2025-02-18T13:36:13Z

Details:

Deduplicate constant tensors in order to reduce memory usage and improve cache usage

Tickets:

CVS-156968

src/common/transformations/src/transformations/common_optimizations/constants_reduce.cpp

itikhono · 2025-02-20T07:10:13Z

src/common/transformations/include/transformations/common_optimizations/constants_reduce.hpp

@@ -0,0 +1,22 @@
+// Copyright (C) 2024 Intel Corporation


Suggested change

// Copyright (C) 2024 Intel Corporation

// Copyright (C) 2025 Intel Corporation

itikhono · 2025-02-20T07:11:31Z

src/common/transformations/src/transformations/common_optimizations/constants_reduce.cpp

@@ -0,0 +1,110 @@
+// Copyright (C) 2024 Intel Corporation


Suggested change

// Copyright (C) 2024 Intel Corporation

// Copyright (C) 2025 Intel Corporation

itikhono · 2025-02-20T07:14:29Z

src/common/transformations/include/transformations/common_optimizations/constants_reduce.hpp

+namespace ov {
+namespace pass {
+
+class TRANSFORMATIONS_API ConstantsReduce : public ov::pass::GraphRewrite {


usually we use ov::pass::GraphRewrite as a container for matcher passes to execute them efficiently
it's better to use ModelPass instead of GraphRewrite if you do not plan to combine several matchers inside this transformation

also we can change it to MatherPass and match a constant with a condition (predicate)
e.g. predicate = const_node->get_byte_size() > 256

https://docs.openvino.ai/2025/documentation/openvino-extensibility/transformation-api/matcher-pass.html

itikhono · 2025-02-20T07:15:25Z

src/common/transformations/include/transformations/common_optimizations/constants_reduce.hpp

+
+class TRANSFORMATIONS_API ConstantsReduce : public ov::pass::GraphRewrite {
+public:
+    OPENVINO_GRAPH_REWRITE_RTTI("ConstantsReduce");


please use OPENVINO_MODEL_PASS_RTTI or OPENVINO_MATCHER_PASS_RTTI according to the comment above

itikhono · 2025-02-20T07:17:07Z

src/common/transformations/include/transformations/common_optimizations/constants_reduce.hpp

+#include "openvino/pass/graph_rewrite.hpp"
+
+namespace ov {
+namespace pass {


minor: we can use c++17 standart here:
namespace ov::pass {

itikhono · 2025-02-20T07:18:50Z

src/common/transformations/src/transformations/common_optimizations/constants_reduce.cpp

+
+    int copies = 0;
+
+    const std::vector<std::shared_ptr<ov::Node>> ops = m->get_ops();


Suggested change

const std::vector<std::shared_ptr<ov::Node>> ops = m->get_ops();

const auto& ops = m->get_ops();

itikhono · 2025-02-20T07:26:44Z

src/common/transformations/src/transformations/common_optimizations/constants_reduce.cpp

+        auto const_node = ov::as_type_ptr<op::v0::Constant>(op);
+
+        // Limit size of node reading to avoid reading large tensors
+        if (const_node->get_byte_size() > 256) continue;


it's better to define a macro variable to make 256 more visible and informative
e.g.
#define LARGE_TENSOR_BYTE_SIZE 256

src/common/transformations/src/transformations/common_optimizations/constants_reduce.cpp

itikhono · 2025-02-20T07:39:31Z

src/common/transformations/include/transformations/common_optimizations/constants_reduce.hpp

+class TRANSFORMATIONS_API ConstantsReduce : public ov::pass::GraphRewrite {
+public:
+    OPENVINO_GRAPH_REWRITE_RTTI("ConstantsReduce");
+    ConstantsReduce();


Suggested change

ConstantsReduce();

ConstantsReduce() = default;

itikhono · 2025-02-20T07:53:59Z

src/common/transformations/include/transformations/common_optimizations/constants_reduce.hpp

+namespace ov {
+namespace pass {
+
+class TRANSFORMATIONS_API ConstantsReduce : public ov::pass::GraphRewrite {


could you add new unit tests for this transformation?

itikhono · 2025-02-20T07:55:12Z

@sshlyapn @dnkurek do we plan to use this transformation for gpu only? does it have a gpu specific?
if so, should we move it from common to some gpu folder?

p-durandin · 2025-02-20T09:18:16Z

@sshlyapn @dnkurek do we plan to use this transformation for gpu only? does it have a gpu specific? if so, should we move it from common to some gpu folder?

It is better to make it general

github-actions · 2025-04-13T00:51:04Z

This PR was closed because it has been stalled for 2 week with no activity.

src/plugins/intel_gpu/src/plugin/ops/constant.cpp

yeonbok · 2025-04-22T20:05:16Z

Do you have any ticket which describes the background or impact of this change? If so, please add a link, and if not, could you please add description in this PR? It would be great for anyone who is interested in its effect or affected models.

praasz · 2025-04-23T07:02:52Z

src/common/transformations/src/transformations/common_optimizations/constants_reduce.cpp

+        auto lhs_node = ov::as_type_ptr<op::v0::Constant>(lhs);
+        auto rhs_node = ov::as_type_ptr<op::v0::Constant>(rhs);
+
+        auto lhs_type = lhs_node->get_output_element_type(0);


This part of code looks similar to this function

Consider expert these function to tensor_util.hpp (part of dev API ) and re-use it. The Constant node can provide tensor view

itikhono · 2025-04-23T11:17:19Z

src/common/transformations/src/transformations/common_optimizations/constants_reduce.cpp

+};
+
+bool ConstantsReduce::run_on_model(const std::shared_ptr<ov::Model>& m) {
+    RUN_ON_FUNCTION_SCOPE(ConstantsReduce);


minor: RUN_ON_FUNCTION_SCOPE -> RUN_ON_MODEL_SCOPE

dnkurek added 3 commits February 18, 2025 13:53

[GPU] Constants duplicate reduction

81a251c

[GPU] Add data computation hash

b0319c0

[GPU] Add hash collision safety and size limits

36561a8

dnkurek requested review from a team as code owners February 18, 2025 13:36

dnkurek requested review from itikhono and removed request for a team February 18, 2025 13:36

github-actions bot added category: GPU OpenVINO GPU plugin category: transformations OpenVINO Runtime library - Transformations labels Feb 18, 2025

dnkurek changed the title ~~[GPU][Draft] Constant tensor deduplication optimization pass~~ [Common][GPU][Draft] Constant tensor deduplication optimization pass Feb 18, 2025

dnkurek changed the title ~~[Common][GPU][Draft] Constant tensor deduplication optimization pass~~ [Common][GPU][Draft] Constant tensor deduplication pass Feb 18, 2025

dnkurek marked this pull request as draft February 18, 2025 13:42

dnkurek added 2 commits February 19, 2025 11:02

[GPU] Improve and fix

bb4e391

Improve

8ebcc72

dnkurek changed the title ~~[Common][GPU][Draft] Constant tensor deduplication pass~~ [TRANSFORMATIONS][GPU][Draft] Constant tensor deduplication pass Feb 19, 2025

dnkurek changed the title ~~[TRANSFORMATIONS][GPU][Draft] Constant tensor deduplication pass~~ [TRANSFORMATIONS][GPU] Constant tensor deduplication pass Feb 19, 2025

dnkurek marked this pull request as ready for review February 19, 2025 10:48

Merge branch 'master' into constants

3a6a3dd

dnkurek changed the title ~~[TRANSFORMATIONS][GPU] Constant tensor deduplication pass~~ [Transformations][GPU] Constant tensor deduplication pass Feb 19, 2025

sshlyapn reviewed Feb 19, 2025

View reviewed changes

Merge branch 'master' into constants

a9bb558

itikhono reviewed Feb 20, 2025

View reviewed changes

Merge branch 'master' into constants

2714bfc

itikhono reviewed Feb 20, 2025

View reviewed changes

github-actions bot added the Stale label Apr 6, 2025

github-actions bot closed this Apr 13, 2025

p-durandin reopened this Apr 15, 2025

github-actions bot removed the Stale label Apr 16, 2025

dnkurek added 11 commits April 22, 2025 11:58

Update commit

11dc772

Merge branch 'master' into constants

6ea44e8

Update transformations_pipeline.cpp

01fb8e6

Update constants_reduce.cpp

52a8e48

Merge branch 'master' into constants

b1e498b

Update constants_reduce.hpp

42ffbc4

Update constants_reduce.cpp

f14c431

Update constants_reduce.hpp

286250e

Update constants_reduce.cpp

68ea41b

Update constants_reduce.hpp

8ddc160

Update constants_reduce.cpp

37f289f

sshlyapn reviewed Apr 22, 2025

View reviewed changes

src/plugins/intel_gpu/src/plugin/ops/constant.cpp Outdated Show resolved Hide resolved

dnkurek added 5 commits April 22, 2025 12:47

Update constant.cpp

49ff00c

Update program_builder.hpp

53f6caa

Update constants_reduce.hpp

7294f31

Update constants_reduce.cpp

341a535

Update constants_reduce.cpp

a4b3b7e

sshlyapn approved these changes Apr 23, 2025

View reviewed changes

p-durandin approved these changes Apr 23, 2025

View reviewed changes

p-durandin added this pull request to the merge queue Apr 23, 2025

praasz reviewed Apr 23, 2025

View reviewed changes

Merged via the queue into openvinotoolkit:master with commit 7f95394 Apr 23, 2025
187 checks passed

itikhono reviewed Apr 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Transformations][GPU] Constant tensor deduplication pass #29052

[Transformations][GPU] Constant tensor deduplication pass #29052

dnkurek commented Feb 18, 2025 •

edited by p-durandin

Loading

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

dnkurek Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono Feb 20, 2025

itikhono commented Feb 20, 2025

p-durandin commented Feb 20, 2025

github-actions bot commented Apr 13, 2025

yeonbok commented Apr 22, 2025

praasz Apr 23, 2025

itikhono Apr 23, 2025

	// Copyright (C) 2024 Intel Corporation
	// Copyright (C) 2025 Intel Corporation


		int copies = 0;

		const std::vector<std::shared_ptr<ov::Node>> ops = m->get_ops();

	const std::vector<std::shared_ptr<ov::Node>> ops = m->get_ops();
	const auto& ops = m->get_ops();

[Transformations][GPU] Constant tensor deduplication pass #29052

[Transformations][GPU] Constant tensor deduplication pass #29052

Conversation

dnkurek commented Feb 18, 2025 • edited by p-durandin Loading

Details:

Tickets:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

itikhono commented Feb 20, 2025

p-durandin commented Feb 20, 2025

github-actions bot commented Apr 13, 2025

yeonbok commented Apr 22, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dnkurek commented Feb 18, 2025 •

edited by p-durandin

Loading