Skip to content

ATLAS-4988: BusinessMetadata with attribute of data type Array takes time to Delete. #490

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
UmeshPatil-1 wants to merge 3 commits into
base: master
Choose a base branch
from
+95 −20
Open

ATLAS-4988: BusinessMetadata with attribute of data type Array takes time to Delete. #490

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fbf8033
ATLAS_4988: Resolve slow deletion issue of Business Metadata with arr…
UmeshPatil-1 Dec 27, 2025
5bf649f
ATLAS-4988 : Replaced manual iteration with graph query, added error …
UmeshPatil-1 Dec 30, 2025
8478f2b
ATLAS-4988 : Removed entity-type filter from graph query,Ensured pare…
UmeshPatil-1 Jan 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 48 additions & 16 deletions ...main/java/org/apache/atlas/repository/store/graph/v2/AtlasBusinessMetadataDefStoreV2.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,12 @@
import org.apache.atlas.model.TypeCategory;
import org.apache.atlas.model.discovery.AtlasSearchResult;
import org.apache.atlas.model.discovery.SearchParameters;
import org.apache.atlas.model.instance.AtlasEntity;
import org.apache.atlas.model.typedef.AtlasBaseTypeDef;
import org.apache.atlas.model.typedef.AtlasBusinessMetadataDef;
import org.apache.atlas.model.typedef.AtlasStructDef;
import org.apache.atlas.repository.Constants;
import org.apache.atlas.repository.graphdb.AtlasGraph;
import org.apache.atlas.repository.graphdb.AtlasVertex;
import org.apache.atlas.type.AtlasBusinessMetadataType;
import org.apache.atlas.type.AtlasStructType;
Expand Down Expand Up @@ -58,6 +60,9 @@ public class AtlasBusinessMetadataDefStoreV2 extends AtlasAbstractDefStoreV2<Atl

private final EntityDiscoveryService entityDiscoveryService;

@Inject
private AtlasGraph graph;

@Inject
public AtlasBusinessMetadataDefStoreV2(AtlasTypeDefGraphStoreV2 typeDefStore, AtlasTypeRegistry typeRegistry, EntityDiscoveryService entityDiscoveryService) {
super(typeDefStore, typeRegistry);
Expand Down Expand Up @@ -363,44 +368,71 @@ private AtlasBusinessMetadataDef toBusinessMetadataDef(AtlasVertex vertex) throw
private void checkBusinessMetadataRef(String typeName) throws AtlasBaseException {
AtlasBusinessMetadataDef businessMetadataDef = typeRegistry.getBusinessMetadataDefByName(typeName);

if (businessMetadataDef != null) {
List<AtlasStructDef.AtlasAttributeDef> attributeDefs = businessMetadataDef.getAttributeDefs();
if (businessMetadataDef == null || CollectionUtils.isEmpty(businessMetadataDef.getAttributeDefs())) {
return;
}

for (AtlasStructDef.AtlasAttributeDef attributeDef : attributeDefs) {
String qualifiedName = AtlasStructType.AtlasAttribute.getQualifiedAttributeName(businessMetadataDef, attributeDef.getName());
String vertexPropertyName = AtlasStructType.AtlasAttribute.generateVertexPropertyName(businessMetadataDef, attributeDef, qualifiedName);
Set<String> applicableTypes = AtlasJson.fromJson(attributeDef.getOption(AtlasBusinessMetadataDef.ATTR_OPTION_APPLICABLE_ENTITY_TYPES), Set.class);
for (AtlasStructDef.AtlasAttributeDef attributeDef : businessMetadataDef.getAttributeDefs()) {
String applicableTypesStr = attributeDef.getOption(ATTR_OPTION_APPLICABLE_ENTITY_TYPES);
Set<String> applicableTypes = StringUtils.isBlank(applicableTypesStr) ? null : AtlasJson.fromJson(applicableTypesStr, Set.class);

if (CollectionUtils.isNotEmpty(applicableTypes) && isBusinessAttributePresent(vertexPropertyName, applicableTypes)) {
throw new AtlasBaseException(AtlasErrorCode.TYPE_HAS_REFERENCES, typeName);
}
if (CollectionUtils.isEmpty(applicableTypes)) {
continue;
}

String qualifiedName = AtlasStructType.AtlasAttribute.getQualifiedAttributeName(businessMetadataDef, attributeDef.getName());
String vertexPropertyName = AtlasStructType.AtlasAttribute.generateVertexPropertyName(businessMetadataDef, attributeDef, qualifiedName);

boolean isPresent;

if (Boolean.TRUE.equals(attributeDef.getIsIndexable())) {
isPresent = isBusinessAttributePresent(vertexPropertyName, applicableTypes);
} else {
isPresent = isBusinessAttributePresentInGraph(vertexPropertyName, applicableTypes);
}

if (isPresent) {
throw new AtlasBaseException(AtlasErrorCode.TYPE_HAS_REFERENCES, typeName);
}
}
}

private boolean isBusinessAttributePresent(String attrName, Set<String> applicableTypes) throws AtlasBaseException {
SearchParameters.FilterCriteria criteria = new SearchParameters.FilterCriteria();

criteria.setAttributeName(attrName);
criteria.setOperator(SearchParameters.Operator.NOT_EMPTY);
criteria.setOperator(SearchParameters.Operator.NOT_NULL);

SearchParameters.FilterCriteria entityFilters = new SearchParameters.FilterCriteria();

entityFilters.setCondition(SearchParameters.FilterCriteria.Condition.OR);
entityFilters.setCriterion(Collections.singletonList(criteria));

SearchParameters searchParameters = new SearchParameters();

searchParameters.setTypeName(String.join(SearchContext.TYPENAME_DELIMITER, applicableTypes));
searchParameters.setExcludeDeletedEntities(true);
searchParameters.setIncludeSubClassifications(false);
searchParameters.setEntityFilters(entityFilters);
searchParameters.setAttributes(Collections.singleton(attrName));
searchParameters.setOffset(0);
searchParameters.setAttributes(Collections.emptySet());
searchParameters.setLimit(1);

AtlasSearchResult atlasSearchResult = entityDiscoveryService.searchWithParameters(searchParameters);
return atlasSearchResult != null && CollectionUtils.isNotEmpty(atlasSearchResult.getEntities());
}

private boolean isBusinessAttributePresentInGraph(String vertexPropertyName, Set<String> applicableTypes) {
Copy link
Contributor

@sheetalshah1007 sheetalshah1007 Dec 29, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. Replace manual iteration with Graph Query for better performance:
The method isBusinessAttributePresentInGraph manually iterates over all vertices of each applicable type using graph.getVertices(Constants.ENTITY_TYPE_PROPERTY_KEY, typeName), then checks each vertex for the property and state. This can be slow for large graphs, as it loads and inspects every vertex in memory. This needs to be improved

2. Add error handling and logging:
No exception handling in the graph operations.
If the graph is unavailable or queries fail, it could lead to unhandled exceptions. Wrap the query in a try-catch and log warnings/errors.
Also, add optional performance logging to monitor query times in production.

3. Add JUnit tests for the new method covering:
-Non-indexable attributes with/without assignments.
-Inactive entities (should not count).

for (String typeName : applicableTypes) {

Iterable<AtlasVertex> vertices = graph.getVertices(Constants.ENTITY_TYPE_PROPERTY_KEY, typeName);

return CollectionUtils.isNotEmpty(atlasSearchResult.getEntities());
if (vertices != null) {
for (AtlasVertex vertex : vertices) {

if (vertex.getProperty(vertexPropertyName, Object.class) != null &&
AtlasGraphUtilsV2.getState(vertex) == AtlasEntity.Status.ACTIVE) {
return true;
}
}
}
}
return false;
}
}
Loading