Skip to content

Reduce TXD load memory before import#4897

Open
Zephkek wants to merge 3 commits intomultitheftauto:masterfrom
Zephkek:fix/4062-txd-raw-load-memory
Open

Reduce TXD load memory before import#4897
Zephkek wants to merge 3 commits intomultitheftauto:masterfrom
Zephkek:fix/4062-txd-raw-load-memory

Conversation

@Zephkek
Copy link
Copy Markdown
Contributor

@Zephkek Zephkek commented May 5, 2026

Summary

Delay retaining decoded TXD replacement textures until engineImportTXD actually applies them.

engineLoadTXD still validates file-path and raw-data TXDs, but it now releases the decoded RenderWare textures immediately after validation instead of keeping them on the TXD element before import.

Motivation

Fixes unnecessary memory usage reported in #4062 where engineLoadTXD could hold decoded TXD textures even before they were imported into any model.

For raw-data TXDs, the original byte buffer is still kept because clothes replacement stores pointers to that data. For normal model imports, decoded textures are loaded again on engineImportTXD and the existing raw-data cleanup path still frees the buffer afterward.

Test plan

  • engineLoadTXD(file) validates then releases decoded textures.
  • engineLoadTXD(rawData) validates, keeps raw bytes for clothes compatibility, and releases decoded textures.
  • engineImportTXD reloads textures through the existing file/raw import paths.
  • engineAddClothingTXD and clothes TXD ID imports still have valid m_FileData.

Checklist

  • Your code should follow the coding guidelines.
  • Smaller pull requests are easier to review. If your pull request is beefy, your pull request should be reviewable commit-by-commit.

Zephkek and others added 2 commits May 5, 2026 20:58
Co-authored-by: Cursor <cursoragent@cursor.com>
Copy link
Copy Markdown
Member

@qaisjp qaisjp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't fully understand the implications of the code changes themselves, I'll dive into the code when I get chance, but I have some feedback about some of the points in the description:

  • if engineLoadTXD is validating textures, but doesn't hold onto any resources, and then engineImportTXD reloads from disk; isn't there an opportunity here for the file to be rewritten on disk and to load something that hasn't been validated?
  • the test plan you've written ins useful, but not actually a test plan. Was there a resource you tested with?

@Zephkek
Copy link
Copy Markdown
Contributor Author

Zephkek commented May 6, 2026

@qaisjp I think the rewrite-on-disk window already exists in master, CClientDFF::LoadFromFile does no format validation at all, it just checks FileExists plus CheckFile and defers ReadDFF to replace time so the TXD path now just matches what DFF already does.

The reload from disk branch inside Import() (CClientTXD.cpp L110-L117) is also reachable when a TXD is used as clothes first then re-imported on a non clothes model since the clothes path clears m_ReplacementTextures at L84-88. CheckFile runs again on every import through GetFilenameToUse and if the bytes changed to garbage in between then ReadTXD just returns NULL and Import would return false so there's no new crash surface either. Either way I took care of it it's now slurped into m_FileData on load.

As for testing, I haven't gotten the chance to test yet as the fix seemed obvious enough just from the code logic, I'm waiting on FileEX to test it the plan was just what needed to be validated code wise. The real root cause of the issue isn't what this PR fixes, D3DPOOL_MANAGED keeps a CPU side shadow copy in addition to the vram copy so a 10mb DXTn TXD becomes 10MB shadow + 10MB VRAM that's what the issue reporter sees as "2x"

What im fixing here however is a 3x path, where in the buffer path there's a second duplication on top CClientTXD.cpp L177 keeps m_FileData (the raw 10 mb) and the decoded textures live until Import() frees one of them. So the buffer pathway memory peaks at raw bytes + D3D shadow + VRAM so 3x pre this PR, so the issue being fixed here is correct.

LoadFromFile now reads the file into m_FileData, validates from that
buffer, and discards the decoded textures. Import decodes the same
buffer instead of re-reading the path, so a rewrite-on-disk between
engineLoadTXD and engineImportTXD can't substitute unvalidated content.

The buffer is dropped after a non-clothes Import for both raw-data and
file paths; the clothes branch still falls back to a fresh disk read
when the buffer is gone, matching prior behavior.
@Zephkek
Copy link
Copy Markdown
Contributor Author

Zephkek commented May 6, 2026

ill be writing an enhancement PR that alters D3DPOOL_MANAGED behavior, this one will need a lot of testing but it'll bring massive improvements if proven to be good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants