Skip to content

Conversation

@abdullah-kasim
Copy link
Contributor

@abdullah-kasim abdullah-kasim commented Dec 3, 2025

Description

Part of VIP-2449. Requires #7 to be merged.

We're opting for post_ingestion_failed rather than post_ingestion_complete as I couldn't come up with a use-case to trigger on all completion.

Pre-review checklist

Please make sure the items below have been covered before requesting a review:

  • This change works and has been tested locally or in Codespaces (or has an appropriate fallback).
  • This change has relevant unit tests (if applicable).
  • This change has relevant documentation additions / updates (if applicable).
  • I've created a changelog description that aligns with the provided examples.

Pre-deploy checklist

  • VIP staff: Ensure any alerts added/updated conform to internal standards (see internal documentation).

Steps to Test

Click here for testing steps # Manual Testing: Ingestion Failure Action

Prerequisites

  • VIP dev-env running: vip dev-env start
  • Plugin activated on the dev environment
  • Logger output: /wp/log/debug.log
  • error_log() output: appears in terminal (not debug.log)
  • Enable debug logging by adding to vip-agentforce.php:
    add_filter( 'enable_wp_debug_mode_checks', '__return_true' );

Test 1: Action Does NOT Fire on Successful Ingestion

Goal: Verify the failure action does not fire when ingestion succeeds.

  1. Add to the bottom of vip-agentforce.php:
use Automattic\VIP\Salesforce\Agentforce\Ingestion\Ingestion_Failure;

// TEST: Enable ingestion - remove after testing
add_filter( 'vip_agentforce_should_ingest_post', '__return_true' );

// TEST: Listen to ingestion failure (should NOT fire)
add_action(
    'vip_agentforce_post_ingestion_failed',
    function ( Ingestion_Failure $failure ) {
        error_log( '=== UNEXPECTED FAILURE ===' );
        error_log( 'This should not appear on successful ingestion!' );
    }
);
  1. Create a published post:
vip dev-env shell -- wp post create --post_title="Success Test" --post_content="Test content" --post_status=publish
  1. Check terminal output.

Expected: No "UNEXPECTED FAILURE" message. Verify success in debug.log:

vip dev-env shell -- grep "Post ingested successfully" /wp/log/debug.log | tail -1

Test 2: Action Does NOT Fire on Filter Rejection (Skip)

Goal: Verify the failure action does not fire when ingestion is skipped.

  1. Replace the test filter in vip-agentforce.php with:
use Automattic\VIP\Salesforce\Agentforce\Ingestion\Ingestion_Failure;

// TEST: Reject all posts (skip, not failure)
add_filter( 'vip_agentforce_should_ingest_post', '__return_false' );

// TEST: Listen to ingestion failure (should NOT fire)
add_action(
    'vip_agentforce_post_ingestion_failed',
    function ( Ingestion_Failure $failure ) {
        error_log( '=== UNEXPECTED FAILURE ===' );
        error_log( 'Skip should NOT trigger failure action!' );
    }
);
  1. Create a published post:
vip dev-env shell -- wp post create --post_title="Skip Test" --post_status=publish
  1. Check terminal output.

Expected: No "UNEXPECTED FAILURE" message. Verify skip in debug.log:

vip dev-env shell -- grep "Post will not be ingested" /wp/log/debug.log | tail -1

Test 3: Action DOES Fire on Transform Failure

Goal: Verify the failure action fires when transform returns null.

  1. Replace the test filter in vip-agentforce.php with:
use Automattic\VIP\Salesforce\Agentforce\Ingestion\Ingestion_Failure;

// TEST: Allow ingestion but fail transform
add_filter( 'vip_agentforce_should_ingest_post', '__return_true' );
add_filter( 'vip_agentforce_transform_post', '__return_null', 5 ); // Priority 5 = before default

// TEST: Listen to ingestion failure
add_action(
    'vip_agentforce_post_ingestion_failed',
    function ( Ingestion_Failure $failure ) {
        error_log( '=== INGESTION FAILURE ===' );
        error_log( 'Failure code: ' . $failure->failure_code );
        error_log( 'Post ID: ' . $failure->post->ID );
        error_log( 'Is transform failure: ' . ( $failure->is_transform_failure() ? 'yes' : 'no' ) );
        error_log( 'Is API error: ' . ( $failure->is_api_error() ? 'yes' : 'no' ) );
        error_log( 'Error code: ' . $failure->error->get_error_code() );
        error_log( 'Error message: ' . $failure->error->get_error_message() );
        $data = $failure->error->get_qerror_data();
        error_log( 'Has backtrace: ' . ( isset( $data['backtrace'] ) ? 'yes' : 'no' ) );
    }
);
  1. Create a published post:
vip dev-env shell -- wp post create --post_title="Transform Fail Test" --post_status=publish
  1. Check terminal output.

Expected:

=== INGESTION FAILURE ===
Failure code: transform_failed
Post ID: <post_id>
Is transform failure: yes
Is API error: no
Error code: vip_agentforce_transform_failed
Error message: Post transformation returned null
Has backtrace: yes

Test 4: Action DOES Fire on API Failure

Goal: Verify the failure action fires when API returns failure.

  1. Replace the test filter in vip-agentforce.php with:
use Automattic\VIP\Salesforce\Agentforce\Ingestion\Ingestion;
use Automattic\VIP\Salesforce\Agentforce\Ingestion\Ingestion_Failure;
use Automattic\VIP\Salesforce\Agentforce\Ingestion\Ingestion_Post_Record;

// TEST: Allow ingestion with valid transform
add_filter( 'vip_agentforce_should_ingest_post', '__return_true' );

// TEST: Mock API failure by extending the class
class Test_Failing_Ingestion extends Ingestion {
    public static function send_to_api( Ingestion_Post_Record $record ): array {
        return [
            'success'       => false,
            'error_message' => 'Simulated API failure for manual testing',
            'timestamp'     => gmdate( 'c' ),
        ];
    }
}

// Replace the save_post hook with our failing version
remove_action( 'save_post', [ Ingestion::class, 'ingest_post' ], 10 );
add_action( 'save_post', [ Test_Failing_Ingestion::class, 'ingest_post' ], 10, 2 );

// TEST: Listen to ingestion failure
add_action(
    'vip_agentforce_post_ingestion_failed',
    function ( Ingestion_Failure $failure ) {
        error_log( '=== INGESTION FAILURE ===' );
        error_log( 'Failure code: ' . $failure->failure_code );
        error_log( 'Post ID: ' . $failure->post->ID );
        error_log( 'Is transform failure: ' . ( $failure->is_transform_failure() ? 'yes' : 'no' ) );
        error_log( 'Is API error: ' . ( $failure->is_api_error() ? 'yes' : 'no' ) );
        error_log( 'Error code: ' . $failure->error->get_error_code() );
        error_log( 'Error message: ' . $failure->error->get_error_message() );
        $data = $failure->error->get_error_data();
        error_log( 'Has response: ' . ( isset( $data['response'] ) ? 'yes' : 'no' ) );
        error_log( 'Has backtrace: ' . ( isset( $data['backtrace'] ) ? 'yes' : 'no' ) );
    }
);
  1. Create a published post:
vip dev-env shell -- wp post create --post_title="API Fail Test" --post_content="Test content" --post_status=publish
  1. Check terminal output.

Expected:

=== INGESTION FAILURE ===
Failure code: api_error
Post ID: <post_id>
Is transform failure: no
Is API error: yes
Error code: vip_agentforce_api_error
Error message: Simulated API failure for manual testing
Has response: yes
Has backtrace: yes

Failure Codes Reference

Code Constant is_transform_failure() is_api_error()
transform_failed Ingestion_Failure::CODE_TRANSFORM_FAILED true false
api_error Ingestion_Failure::CODE_API_ERROR false true

Note: Filter rejection (skip) does NOT trigger the failure action.


Cleanup

Remove all test filters from vip-agentforce.php after testing.

Copilot AI review requested due to automatic review settings December 3, 2025 01:30
Copilot finished reviewing on behalf of abdullah-kasim December 3, 2025 01:31
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a failure handling mechanism for post ingestion by adding a new vip_agentforce_post_ingestion_failed action hook. The hook allows consumers to catch and respond to ingestion failures, specifically distinguishing between actual failures (transform errors or API errors) and intentional skips (filter rejections).

Key Changes:

  • Added Ingestion_Failure data class to encapsulate failure information including failure code, post object, and WP_Error details
  • Implemented vip_agentforce_post_ingestion_failed action hook that fires only on actual failures (transform or API errors)
  • Enhanced error reporting with backtraces and detailed error data for debugging

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
vip-agentforce.php Added autoload for new Ingestion_Failure class
modules/ingestion/class-ingestion-failure.php New data class encapsulating failure information with helper methods for failure type detection
modules/ingestion/class-ingestion.php Added failure action firing, new send_to_api() method stub, and enhanced error handling with backtraces
tests/phpunit/test-ingestion.php Added comprehensive tests for failure action behavior covering skip vs. failure scenarios, backtrace inclusion, and to_array() method
tests/phpunit/test-ingestion-api-failure.php New test file for API failure scenarios using test double to simulate API errors
tests/phpunit/doubles/class-ingestion-with-failing-api.php Test double extending Ingestion class to simulate API failures for testing

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[
'post_id' => $post_id,
// phpcs:ignore WordPress.PHP.DevelopmentFunctions.error_log_debug_backtrace -- Intentional for error tracing.
'backtrace' => debug_backtrace( DEBUG_BACKTRACE_IGNORE_ARGS, 5 ),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abdullah-kasim What data does the backtrace gives us to better debug? The origin function that triggered the ingestion?

I wonder if we could also rely on some ingest post parameter to detect where it's coming from and rely on the backtrace only when there's an exception.

);

self::fire_ingestion_failure(
new Ingestion_Failure(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion (not required): Move the Ingestion_Failure object creation with WP Error under the fire_ingestion_failure to keep code cleaner, this way we could pass the failure code and post_id only, an fill the rest there.

]
);

self::fire_ingestion_failure(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abdullah-kasim One other thing I've been wondering (for both the ingestion failures) is that if there's a config error we might get DDoSed via ingestion failures.

I do wonder

  1. Should we instantiate as little as possible here, and let the action do the work of instantiating object as needed?
  2. Do we need some kind of protection for when this scenario happens (or SF is down)? I'm referring to having some protection -before- the action is triggered.

Regarding point 2, I know it could also make sense to do it as the action level, but I do wonder if it would be better/more performant, to catch this earlier.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also true for the logging part, we should be aware that this might raise too much logging data when an error happens at this level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants