Skip to content

Massive spike in CPU usage when running diff on a large object #183

Open
@User9684

Description

@User9684

Hi, I run a Discord bot that constantly checks differences in a relatively large object and sends messages accordingly, and I've noticed that every time diff is ran on the object, CPU usage spikes a ton. (See image)
image

In my case, the object has roughly 8,100 elements.

Activity

User9684

User9684 commented on Jul 28, 2022

@User9684
Author

Also, I have no idea how CPU spiked to 160% in that image.

cerebralkungfu

cerebralkungfu commented on Jan 16, 2024

@cerebralkungfu

I'd love to see a "relatively large object" similar to your case, that I can use for testing. I am working on a significant update (v2.x) which uses a different (more efficient) algorithm and would like to test.

Without the data to test your case, I would guess it is due to the stack. Not only is deep-diff recursive, I track cycles in the graph (recursion in your data), which can cause lots of tracking data on deeply nested objects. But of course, this is speculation.

The new version will give you the option to turn off checking for cycles, which should save some memory in your use case... but I'd only know for sure if I can test with similar data.

User9684

User9684 commented on Jan 19, 2024

@User9684
Author

I'd love to see a "relatively large object" similar to your case, that I can use for testing. I am working on a significant update (v2.x) which uses a different (more efficient) algorithm and would like to test.

Without the data to test your case, I would guess it is due to the stack. Not only is deep-diff recursive, I track cycles in the graph (recursion in your data), which can cause lots of tracking data on deeply nested objects. But of course, this is speculation.

The new version will give you the option to turn off checking for cycles, which should save some memory in your use case... but I'd only know for sure if I can test with similar data.

I was particularly monitoring changes to the Roblox Fast Flags list, which changes relatively often.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @cerebralkungfu@User9684

        Issue actions

          Massive spike in CPU usage when running diff on a large object · Issue #183 · flitbit/diff