Convert engineLoadTXD textures to D3DPOOL_DEFAULT (#4062)

GTA SA's RenderWare loader (D3DResourceSystem::CreateTexture @ 0x730510
and the inline CreateCubeTexture branch in _rwD3D9NativeTextureRead @
0x4CD982) hardcodes D3DPOOL_MANAGED for every texture in a TXD. MANAGED
textures are mirrored 1:1 in system memory so D3D9 can auto-restore them
on a device reset, which is exactly what makes a 10 MB DXT TXD count as
~20 MB of working set in MTA's process.

After RW finishes decoding a script-loaded TXD in CRenderWareSA::ReadTXD
we now walk every raster, allocate a fresh IDirect3DTexture9 (or
IDirect3DCubeTexture9) in D3DPOOL_DEFAULT, copy each mip through a
transient SYSTEMMEM scratch + UpdateTexture, release the original
MANAGED texture, and swap the pointer in rasterExt->texture. The release
deliberately bypasses D3DResourceSystem::DestroyTexture so the
gD3DTextureBuffer cache (MANAGED-only, keyed by w/h/format/levels) never
sees DEFAULT-pool entries; the destroy intercept in
CRenderWareSA::DestroyTexture NULLs rasterExt->texture before
RwTextureDestroy so _rwD3D9RasterDestroy hits its existing early-out.

Conversion happens before ScriptAddedTxd so the shader-matching map
(m_D3DDataTexInfoMap) is keyed against the new IDirect3DTexture9
pointer the renderer will see when GTA later calls SetTexture.

Coverage:
  - Regular 2D rasters (RwRaster::type == 4, no cube flag, no palette,
    no D3DUSAGE_AUTOGENMIPMAP): converted via CreateTexture.
  - Cube-map rasters (cubeTextureFlags & 0x01): converted via
    CreateCubeTexture; both faces and sub-levels are explicitly locked
    on the SYSTEMMEM scratch so UpdateTexture copies the full mipchain
    of every face. EdgeLength == desc.Width == desc.Height (D3D9
    enforces square cube faces).
  - Palettised rasters (P8 + 1024-byte palette buffer) and rasters
    flagged with auto-mipmap generation are skipped: D3DUSAGE_AUTOGENMIPMAP
    is incompatible with D3DPOOL_SYSTEMMEM and palette-format DEFAULT
    textures aren't supported by modern drivers; both paths leave the
    raster MANAGED so behaviour matches today.
  - Unrecognised formats (anything not in our explicit byte-count table)
    stay MANAGED.
  - CreateTexture / CreateCubeTexture for the DEFAULT pool retry once
    after IDirect3DDevice9::EvictManagedResources on
    D3DERR_OUTOFVIDEOMEMORY, mirroring CDirect3DEvents9::CreateTexture.

Because DEFAULT-pool resources are auto-destroyed on a D3D9 cooperative
loss, CRenderWareSA gains OnDeviceLost / OnDeviceReset hooks called from
CGraphics::OnDeviceInvalidate / OnDeviceRestore around the existing
CRenderItemManager handling. CClientTXD implements a new
CRwReplacementOwner interface and re-decodes on Reset: file-path TXDs
re-read from disk, raw-data TXDs use the m_FileData buffer that
LoadFromBuffer already keeps for the clothes system.

Memory impact for a 10 MB TXD:
  - file path (the common case for vehicle/skin packs):
    ~10 MB sysmem dropped, VRAM usage unchanged. ~50% total saving.
  - raw-data path (rare; engineLoadTXD with a buffer):
    m_FileData was already kept by master for clothes compatibility, so
    the MANAGED shadow is replaced by that buffer at no net memory cost.

For texture-heavy servers (large vehicle / skin / map packs) this
typically reclaims hundreds of megabytes of sysmem with zero per-frame
cost (DEFAULT and MANAGED render identically once bound).

Author: Zephkek

Client/core/Graphics/CGraphics.cpp

@@ -11,6 +11,8 @@
 
 #include "StdInc.h"
 #include <game/CSettings.h>
+#include <game/CGame.h>
+#include <game/CRenderWare.h>
 #include <memory>
 #include "DXHook/CProxyDirect3DDevice9.h"
 #include "CTileBatcher.h"
@@ -1727,6 +1729,13 @@ void CGraphics::OnDeviceInvalidate(IDirect3DDevice9* pDevice)
     SAFE_RELEASE(m_pSavedFrontBufferData);
     SAFE_RELEASE(m_pTempBackBufferData);
 
+    // Release D3DPOOL_DEFAULT replacement textures owned by engineLoadTXD before
+    // IDirect3DDevice9::Reset is attempted; per the D3D9 contract, every DEFAULT-pool
+    // resource must be gone first. The game module rebuilds these in OnDeviceRestore.
+    if (CGame* pGame = g_pCore->GetGame())
+        if (CRenderWare* pRenderWare = pGame->GetRenderWare())
+            pRenderWare->OnDeviceLost();
+
     // Reset render zone tracking on device loss
     m_MTARenderZone = MTA_RZONE_NONE;
     m_iOutsideZoneCount = 0;
@@ -1757,6 +1766,13 @@ void CGraphics::OnDeviceRestore(IDirect3DDevice9* pDevice)
     m_pRenderItemManager->OnResetDevice();
     m_pScreenGrabber->OnResetDevice();
 
+    // Re-create the D3DPOOL_DEFAULT replacement textures we released in OnDeviceInvalidate.
+    // File-path TXDs re-read from disk; raw-data TXDs use the m_FileData buffer kept by
+    // CClientTXD since LoadFromBuffer.
+    if (CGame* pGame = g_pCore->GetGame())
+        if (CRenderWare* pRenderWare = pGame->GetRenderWare())
+            pRenderWare->OnDeviceReset();
+
     const uint uiViewportWidth = GetViewportWidth();
     const uint uiViewportHeight = GetViewportHeight();
     if (uiViewportWidth > 0 && uiViewportHeight > 0)

Client/game_sa/CRenderWareSA.PoolMemorySaver.cpp

@@ -239,6 +239,13 @@ RwTexDictionary* CRenderWareSA::ReadTXD(const SString& strFilename, const SStrin
     // close the stream
     RwStreamClose(streamTexture, NULL);
 
+    // Replacement TXDs are static for the lifetime of the script that loaded them, so the
+    // D3DPOOL_MANAGED system-memory shadow GTA's RW loader created is dead weight (issue
+    // #4062). Convert each raster to D3DPOOL_DEFAULT before ScriptAddedTxd runs so the
+    // texinfo shader-matching map is keyed against the new IDirect3DTexture9 pointer.
+    if (pTex)
+        ConvertScriptTxdToDefaultPool(pTex);
+
     ScriptAddedTxd(pTex);
 
     return pTex;
@@ -649,6 +656,13 @@ void CRenderWareSA::DestroyTexture(RwTexture* pTex)
     if (pTex)
     {
         ScriptRemovedTexture(pTex);
+
+        // If we previously converted this raster's D3D resource to D3DPOOL_DEFAULT,
+        // release it ourselves and clear rasterExt->texture so _rwD3D9RasterDestroy
+        // hits its NULL-pointer early-out and skips D3DResourceSystem::DestroyTexture
+        // (which caches MANAGED textures for reuse and would corrupt with a DEFAULT one).
+        ReleaseTrackedDefaultPoolTexture(pTex->raster);
+
         RwTextureDestroy(pTex);
     }
 }

Client/game_sa/CRenderWareSA.cpp

@@ -32,6 +32,14 @@ class CRenderWareSA : public CRenderWare
     bool ModelInfoTXDLoadTextures(SReplacementTextures* pReplacementTextures, const SString& strFilename, const SString& buffer, bool bFilteringEnabled);
     bool ModelInfoTXDAddTextures(SReplacementTextures* pReplacementTextures, ushort usModelId);
     void ModelInfoTXDRemoveTextures(SReplacementTextures* pReplacementTextures);
+
+    // Pool-conversion / device-reset support (see CRenderWareSA.PoolMemorySaver.cpp)
+    void OnDeviceLost() override;
+    void OnDeviceReset() override;
+    void RegisterReplacementOwner(CRwReplacementOwner* pOwner) override;
+    void UnregisterReplacementOwner(CRwReplacementOwner* pOwner) override;
+    void ConvertScriptTxdToDefaultPool(RwTexDictionary* pTxd);
+    bool ReleaseTrackedDefaultPoolTexture(RwRaster* pRaster);
     void ClothesAddReplacement(char* pFileData, size_t fileSize, ushort usFileId);
     void ClothesRemoveReplacement(char* pFileData);
     bool HasClothesReplacementChanged();
@@ -167,4 +175,20 @@ class CRenderWareSA : public CRenderWare
     bool                                m_bGTAVertexShadersEnabled;
     std::set<RwTexture*>                m_SpecialTextures;
     static int                          ms_iRenderingType;
+
+    // Replacement-texture pool memory saver state.
+    //
+    // m_DefaultPoolRasters tracks every RwRaster whose underlying IDirect3DTexture9
+    // we converted from D3DPOOL_MANAGED to D3DPOOL_DEFAULT. Lookups happen on the
+    // texture-destruction hot path so a vector with linear scan would be unwise; an
+    // unordered_set keyed by the (stable) raster pointer is the right tool.
+    std::unordered_set<RwRaster*> m_DefaultPoolRasters;
+    // m_ReplacementOwners is the list of CClientTXD-side objects we call back to
+    // re-decode after a device reset. CClientTXD subscribes itself in its ctor and
+    // unsubscribes in its dtor.
+    std::vector<CRwReplacementOwner*> m_ReplacementOwners;
+    // True between OnDeviceLost and OnDeviceReset. Protects against re-entrant
+    // raster destruction during Restore (we'd call Release on an already-NULL ptr,
+    // which is harmless but easier to reason about with a flag).
+    bool m_bDeviceLost = false;
 };

Client/game_sa/CRenderWareSA.h

@@ -22,6 +22,7 @@
 #include <map>
 #include <set>
 #include <string>
+#include <unordered_set>
 #include <vector>
 
 // Game includes

Client/game_sa/StdInc.h

@@ -15,10 +15,16 @@ CClientTXD::CClientTXD(class CClientManager* pManager, ElementID ID) : ClassInit
     // Init
     m_pManager = pManager;
     SetTypeName("txd");
+
+    // Subscribe so the renderware module can re-decode our replacement textures
+    // after a D3D9 device reset (DEFAULT-pool textures are auto-destroyed on Reset).
+    g_pGame->GetRenderWare()->RegisterReplacementOwner(this);
 }
 
 CClientTXD::~CClientTXD()
 {
+    g_pGame->GetRenderWare()->UnregisterReplacementOwner(this);
+
     // Remove us from all the models
     g_pGame->GetRenderWare()->ModelInfoTXDRemoveTextures(&m_ReplacementTextures);
 
@@ -124,12 +130,15 @@ bool CClientTXD::Import(unsigned short usModelID)
             }
         }
 
-        // If raw data and not used as clothes textures yet, then free raw data buffer to save RAM
-        if (m_bIsRawData && !m_bUsingFileDataForClothes)
-        {
-            // This means the texture can't be used for clothes now
-            SString().swap(m_FileData);
-        }
+        // The replacement textures live in D3DPOOL_DEFAULT after the conversion done by
+        // CRenderWareSA::ReadTXD (issue #4062). DEFAULT-pool resources are auto-destroyed
+        // on a D3D9 device reset, so we need the original TXD bytes to re-decode them in
+        // CRenderWareSA::OnDeviceReset:
+        //   - file-path TXDs re-read from disk via GetFilenameToUse,
+        //   - raw-data TXDs use the m_FileData buffer kept since LoadFromBuffer.
+        // We deliberately don't drop m_FileData here for raw-data TXDs (master used to)
+        // because the system-memory cost is the same as the MANAGED shadow we saved by
+        // converting the textures, and keeping it lets us recover from any device reset.
 
         // Have we got textures and haven't already imported into this model?
         if (g_pGame->GetRenderWare()->ModelInfoTXDAddTextures(&m_ReplacementTextures, usModelID))
@@ -263,3 +272,52 @@ bool CClientTXD::IsTXDData(const SString& strData)
 {
     return strData.length() > 32 && memcmp(strData, "\x16\x00\x00\x00", 4) == 0;
 }
+
+bool CClientTXD::RebuildReplacementsAfterDeviceReset()
+{
+    // CRenderWareSA::OnDeviceLost has already released our IDirect3DTexture9 instances
+    // (DEFAULT pool) and NULLed rasterExt->texture. The RwTextures themselves are still
+    // alive inside their respective GTA TXDs, but they now wrap rasters with no backing
+    // D3D resource, so we must tear them down before re-decoding fresh ones.
+
+    // Snapshot the set of models we were applied to so we can replay the imports.
+    std::vector<unsigned short> savedModelIds = m_ReplacementTextures.usedInModelIds;
+
+    // Walk the full perTxdList: removes our textures from each GTA TXD (originals come
+    // back), destroys the cloned copies, and cascades through DestroyTexture for the
+    // originals (which calls ReleaseTrackedDefaultPoolTexture, a no-op in this state
+    // because the D3D pointer was already cleared in OnDeviceLost).
+    g_pGame->GetRenderWare()->ModelInfoTXDRemoveTextures(&m_ReplacementTextures);
+    m_ReplacementTextures = SReplacementTextures();
+
+    // Buffer-path TXDs whose source bytes were freed cannot be rebuilt; the
+    // replacements stay gone for the rest of the session and the script must
+    // re-call engineLoadTXD to recover.
+    if (m_bIsRawData && m_FileData.empty())
+        return false;
+
+    bool bLoaded = false;
+    if (m_bIsRawData)
+    {
+        bLoaded = g_pGame->GetRenderWare()->ModelInfoTXDLoadTextures(&m_ReplacementTextures, NULL, m_FileData, m_bFilteringEnabled);
+    }
+    else
+    {
+        SString strUseFilename;
+        if (!GetFilenameToUse(strUseFilename))
+            return false;
+        bLoaded = g_pGame->GetRenderWare()->ModelInfoTXDLoadTextures(&m_ReplacementTextures, strUseFilename, SString(), m_bFilteringEnabled);
+    }
+
+    if (!bLoaded || m_ReplacementTextures.textures.empty())
+        return false;
+
+    // Re-apply to every model the script had previously imported this TXD onto.
+    for (unsigned short modelId : savedModelIds)
+    {
+        if (g_pGame->GetRenderWare()->ModelInfoTXDAddTextures(&m_ReplacementTextures, modelId))
+            Restream(modelId);
+    }
+
+    return true;
+}

Client/mods/deathmatch/logic/CClientTXD.cpp

@@ -14,7 +14,7 @@
 #include <game/CRenderWare.h>
 #include "CClientEntity.h"
 
-class CClientTXD final : public CClientEntity
+class CClientTXD final : public CClientEntity, public CRwReplacementOwner
 {
     DECLARE_CLASS(CClientTXD, CClientEntity)
 public:
@@ -32,6 +32,11 @@ class CClientTXD final : public CClientEntity
     static bool       IsImportableModel(unsigned short usModelID);
     static bool       IsTXDData(const SString& strData);
 
+    // CRwReplacementOwner: re-decode this TXD after a D3D9 device reset.
+    // Returns true if rebuild succeeded, false if the source bytes are unrecoverable
+    // (raw-data TXDs whose buffer was freed at Import).
+    bool RebuildReplacementsAfterDeviceReset() override;
+
 private:
     bool LoadFromFile(SString filePath);
     bool LoadFromBuffer(SString buffer);

Client/mods/deathmatch/logic/CClientTXD.h

@@ -23,6 +23,7 @@ class CColModel;
 struct RpAtomicContainer;
 struct RwFrame;
 struct RwMatrix;
+struct RwRaster;
 struct RwTexDictionary;
 struct RwTexture;
 struct RpClump;
@@ -45,6 +46,23 @@ struct SReplacementTextures
     std::vector<ushort>     usedInModelIds;
 };
 
+// Called by CRenderWare when the D3D device is reset and a replacement TXD's
+// D3DPOOL_DEFAULT textures need to be re-decoded from their original source.
+//
+// Only owners that successfully re-imported their TXD bytes (file path with the
+// source still on disk) can recover. Owners that cannot rebuild should return
+// false; their replacements are then permanently lost for this session and the
+// underlying GTA TXDs are reverted to the unreplaced state.
+class CRwReplacementOwner
+{
+public:
+    virtual ~CRwReplacementOwner() = default;
+
+    // Re-decode and re-apply this owner's replacement textures using whatever
+    // source it has (file on disk or kept buffer). Returns true on success.
+    virtual bool RebuildReplacementsAfterDeviceReset() = 0;
+};
+
 // Shader layers to render
 struct SShaderItemLayers
 {
@@ -116,6 +134,16 @@ class CRenderWare
     virtual bool     RightSizeTxd(const SString& strInTxdFilename, const SString& strOutTxdFilename, uint uiSizeLimit) = 0;
     virtual void     TxdForceUnload(ushort usTxdId, bool bDestroyTextures) = 0;
 
+    // Pool-conversion / device-reset hooks for engineLoadTXD replacement textures.
+    // OnDeviceLost is called from CGraphics during a D3D9 cooperative-level loss; it
+    // must release every D3DPOOL_DEFAULT replacement texture before IDirect3DDevice9::Reset.
+    // OnDeviceReset is called once Reset succeeds and asks each registered owner to
+    // re-decode its replacement textures.
+    virtual void OnDeviceLost() = 0;
+    virtual void OnDeviceReset() = 0;
+    virtual void RegisterReplacementOwner(CRwReplacementOwner* pOwner) = 0;
+    virtual void UnregisterReplacementOwner(CRwReplacementOwner* pOwner) = 0;
+
     virtual void CMatrixToRwMatrix(const CMatrix& mat, RwMatrix& rwOutMatrix) = 0;
     virtual void RwMatrixToCMatrix(const RwMatrix& rwMatrix, CMatrix& matOut) = 0;
     virtual void RwMatrixGetRotation(const RwMatrix& rwMatrix, CVector& vecOutRotation) = 0;