Skip to content

Conversation

emekanwaoma
Copy link
Contributor

@emekanwaoma emekanwaoma commented Oct 3, 2025

User description

Description

What - Improved cache key generation for instance/class methods in Ocean's caching utilities

Why - The existing cache key generation only used function names and didn't properly handle instance/class methods. This caused inconsistent caching behavior where different instances of the same class would share cache keys inappropriately, or instance methods would have cache keys that included the self parameter which shouldn't affect caching.

How -

  • Updated hash_func to accept function objects instead of just names
  • Added sanitize_identifier function to create backend-safe cache keys
  • Implemented logic to filter out self parameter for instance/class methods
  • Enhanced cache keys to include module and qualname for better uniqueness
  • Added comprehensive test coverage for all method types

Type of change

Please leave one option from the following and delete the rest:

  • New feature (non-breaking change which adds functionality)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

  • Integration able to create all default resources from scratch
  • Resync finishes successfully
  • Resync able to create entities
  • Resync able to update entities
  • Resync able to detect and delete entities
  • Scheduled resync able to abort existing resync and start a new one
  • Tested with at least 2 integrations from scratch
  • Tested with Kafka and Polling event listeners
  • Tested deletion of entities that don't pass the selector

Integration testing checklist

  • Integration able to create all default resources from scratch
  • Completed a full resync from a freshly installed integration and it completed successfully
  • Resync able to create entities
  • Resync able to update entities
  • Resync able to detect and delete entities
  • Resync finishes successfully
  • If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
  • If resource kind is updated, run the integration with the example data and check if the expected result is achieved
  • If new resource kind is added or updated, validate that live-events for that resource are working as expected
  • Docs PR link here

Preflight checklist

  • Handled rate limiting
  • Handled pagination
  • Implemented the code in async
  • Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.

API Documentation

Provide links to the API documentation used for this integration.


PR Type

Enhancement


Description

  • Enhanced cache key generation for instance/class methods

  • Added comprehensive test coverage for all method types

  • Improved cache key uniqueness with module and qualname

  • Added backend-safe identifier sanitization


Diagram Walkthrough

flowchart LR
  A["hash_func"] --> B["sanitize_identifier"]
  A --> C["Filter self/cls params"]
  C --> D["Generate unique cache key"]
  B --> D
  D --> E["cache_iterator_result"]
  D --> F["cache_coroutine_result"]
Loading

File Walkthrough

Relevant files
Enhancement
cache.py
Enhanced cache key generation logic                                           

port_ocean/utils/cache.py

  • Enhanced hash_func to accept function objects instead of names
  • Added sanitize_identifier for backend-safe cache keys
  • Implemented logic to filter out self parameter for instance methods
  • Updated cache key format to include module and qualname
+45/-9   
Tests
test_cache.py
Comprehensive test coverage for method caching                     

port_ocean/tests/utils/test_cache.py

  • Added comprehensive tests for instance method caching
  • Added tests for class method and static method caching
  • Added tests for both iterator and coroutine decorators
  • Added edge case tests for regular functions with self parameter
+240/-0 
Documentation
CHANGELOG.md
Version 0.28.13 changelog entry                                                   

CHANGELOG.md

  • Added entry for version 0.28.13
  • Documented cache key generation improvements
  • Listed comprehensive test coverage addition
+8/-0     

@emekanwaoma emekanwaoma requested a review from a team as a code owner October 3, 2025 00:19
@github-actions github-actions bot added the size/L label Oct 3, 2025
Copy link
Contributor

qodo-merge-pro bot commented Oct 3, 2025

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
🟢
No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
🎫 No ticket provided
  • Create ticket/issue
Codebase Duplication Compliance
Codebase context is not defined

Follow the guide to enable codebase context checks.

Custom Compliance
No custom compliance provided

Follow the guide to enable custom compliance check.

  • Update
Compliance status legend 🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

Copy link
Contributor

qodo-merge-pro bot commented Oct 3, 2025

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
High-level
The method detection logic is fragile

The current method detection logic is unreliable and can cause bugs. It should
be replaced with a more robust check using Python's inspect module to correctly
identify bound methods and avoid incorrectly modifying arguments for cache key
generation.

Examples:

port_ocean/utils/cache.py [38-41]
    if args and hasattr(args[0], func.__name__):
        filtered_args = args[1:]
    else:
        filtered_args = args

Solution Walkthrough:

Before:

def hash_func(func: Callable, *args: Any, **kwargs: Any) -> str:
    """
    Generate a backend-safe cache key.
    ...
    """

    # Fragile check for instance/class methods
    if args and hasattr(args[0], func.__name__):
        filtered_args = args[1:]
    else:
        filtered_args = args

    arg_string = repr(filtered_args)
    # ... generate hash from args
    return f"{func_id}_{short_hash}"

After:

import inspect

def hash_func(func: Callable, *args: Any, **kwargs: Any) -> str:
    """
    Generate a backend-safe cache key.
    ...
    """
    filtered_args = args
    # Robustly check for bound methods (instance or class)
    if args:
        # func is the original unbound function from the class
        # args[0] is the instance (self) or class (cls)
        # getattr(args[0], func.__name__) is the bound method
        bound_method = getattr(args[0], func.__name__, None)
        if bound_method and (inspect.ismethod(bound_method) or inspect.isbuiltin(bound_method)):
            if getattr(bound_method, "__self__", None) is args[0]:
                filtered_args = args[1:]

    arg_string = repr(filtered_args)
    # ... generate hash from args
    return f"{func_id}_{short_hash}"
Suggestion importance[1-10]: 9

__

Why: This suggestion correctly identifies a critical flaw in the method detection logic (hasattr(args[0], func.__name__)) that can lead to incorrect cache key generation and cache poisoning, and proposes a robust solution using the inspect module.

High
  • Update

Copy link
Contributor

github-actions bot commented Oct 3, 2025

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/18209100207/artifacts/4170815102

Code Coverage Total Percentage: 87.22%

Copy link
Contributor

github-actions bot commented Oct 3, 2025

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/18209234394/artifacts/4170858846

Code Coverage Total Percentage: 87.22%

Copy link
Contributor

github-actions bot commented Oct 3, 2025

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/18209261665/artifacts/4170865791

Code Coverage Total Percentage: 87.22%

Copy link
Contributor

github-actions bot commented Oct 3, 2025

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/18209244689/artifacts/4170866123

Code Coverage Total Percentage: 87.22%

Copy link
Member

@mk-armah mk-armah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Oct 3, 2025

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/18220579215/artifacts/4174395741

Code Coverage Total Percentage: 87.22%

Copy link
Member

@matan84 matan84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/18466049968/artifacts/4254658519

Code Coverage Total Percentage: 87.84%

Copy link
Contributor

Code Coverage Artifact 📈: https://github.com/port-labs/ocean/actions/runs/18471179225/artifacts/4256516717

Code Coverage Total Percentage: 87.82%

@emekanwaoma emekanwaoma merged commit 7935968 into main Oct 13, 2025
24 checks passed
@emekanwaoma emekanwaoma deleted the PORT-16520 branch October 13, 2025 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants