Reduce TXD load memory before import#4897
Reduce TXD load memory before import#4897Zephkek wants to merge 3 commits intomultitheftauto:masterfrom
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
I don't fully understand the implications of the code changes themselves, I'll dive into the code when I get chance, but I have some feedback about some of the points in the description:
- if engineLoadTXD is validating textures, but doesn't hold onto any resources, and then engineImportTXD reloads from disk; isn't there an opportunity here for the file to be rewritten on disk and to load something that hasn't been validated?
- the test plan you've written ins useful, but not actually a test plan. Was there a resource you tested with?
|
@qaisjp I think the rewrite-on-disk window already exists in master, The reload from disk branch inside As for testing, I haven't gotten the chance to test yet as the fix seemed obvious enough just from the code logic, I'm waiting on FileEX to test it the plan was just what needed to be validated code wise. The real root cause of the issue isn't what this PR fixes, D3DPOOL_MANAGED keeps a CPU side shadow copy in addition to the vram copy so a 10mb DXTn TXD becomes 10MB shadow + 10MB VRAM that's what the issue reporter sees as "2x" What im fixing here however is a 3x path, where in the buffer path there's a second duplication on top CClientTXD.cpp L177 keeps m_FileData (the raw 10 mb) and the decoded textures live until Import() frees one of them. So the buffer pathway memory peaks at raw bytes + D3D shadow + VRAM so 3x pre this PR, so the issue being fixed here is correct. |
LoadFromFile now reads the file into m_FileData, validates from that buffer, and discards the decoded textures. Import decodes the same buffer instead of re-reading the path, so a rewrite-on-disk between engineLoadTXD and engineImportTXD can't substitute unvalidated content. The buffer is dropped after a non-clothes Import for both raw-data and file paths; the clothes branch still falls back to a fresh disk read when the buffer is gone, matching prior behavior.
|
ill be writing an enhancement PR that alters D3DPOOL_MANAGED behavior, this one will need a lot of testing but it'll bring massive improvements if proven to be good. |
Summary
Delay retaining decoded TXD replacement textures until
engineImportTXDactually applies them.engineLoadTXDstill validates file-path and raw-data TXDs, but it now releases the decoded RenderWare textures immediately after validation instead of keeping them on the TXD element before import.Motivation
Fixes unnecessary memory usage reported in #4062 where
engineLoadTXDcould hold decoded TXD textures even before they were imported into any model.For raw-data TXDs, the original byte buffer is still kept because clothes replacement stores pointers to that data. For normal model imports, decoded textures are loaded again on
engineImportTXDand the existing raw-data cleanup path still frees the buffer afterward.Test plan
engineLoadTXD(file)validates then releases decoded textures.engineLoadTXD(rawData)validates, keeps raw bytes for clothes compatibility, and releases decoded textures.engineImportTXDreloads textures through the existing file/raw import paths.engineAddClothingTXDand clothes TXD ID imports still have validm_FileData.Checklist