add disk limit in rocksdb (disk/mem ratio)

Summary:
Introducting a disk size limit (relative memory) ```db_rocksdb_disk_mem_capacity_ratio_limit``` and changing the RocksDB impl of ```getTotalCapacity```
* ratio is optional, if not provided we use full host (as today)
* ratio defined in server_config, defined per-tier. So possible to deploy to ```glean.query``` tiers only

If we go for this solution:
* No more load balancing work needed for a mix of T3/T10
* we can set a ratio (e.g., 30 to start with) which is higher than possible ratio for a T10, so effectively a T10 will use the actual full disk space, and a T3 will be capped at 30 * RAM_capacity
* will also allow to easily change cache size programmatically with a similar config value: ```db_rocks_cache_mem_capacity_ratio```

_______
**OSS compat**
see MemoryReader in D87988951

______
**How SM loadbalancing works**
We define a resource where formula is
  shard_size = counter / thresholdCounter
==>
  shard_size = glean.shard.[SHARDID.LSB].disk_used_bytes / glean.db.disk.capacity_bytes

defined here
https://www.internalfb.com/code/configerator/[3227e8542f281d93b3a2b529c501e76e92463167]/source/glean/server/glean.query.cinc?lines=201-207

We log to fb303 counters here, using the **getTotalCapacity**
so we'll report a limited capacity here if ratio is set

https://www.internalfb.com/code/fbsource/[6137dfc20518d1c567e635052c344e6fd14e9c81]/fbcode/glean/db/Glean/Database/Env.hs?lines=222%2C225-226

Reviewed By: iamirzhan

Differential Revision: D88269912

fbshipit-source-id: a7acccf33f74b0eecf59645e90aba3c82dd5b19e

Author: jjuliamolin

glean.cabal.in

@@ -416,6 +416,7 @@ library util
         Glean.Impl.ThriftService
         Glean.Impl.ConfigProvider
         Glean.Impl.TestConfigProvider
+        Glean.Impl.MemoryReader
         Glean.Util.Bisect
         Glean.Util.ConfigProvider
         Glean.Util.Disk

glean/db/Glean/Database/Storage/RocksDB.hs

@@ -47,6 +47,7 @@ import Glean.RTS.Types (Fid(..), invalidFid, Pid(..))
 import qualified Glean.ServerConfig.Types as ServerConfig
 import Glean.Types (Repo)
 import Glean.Util.Disk
+import Glean.Impl.MemoryReader
 import System.IO.Extra (withTempFile)
 
 newtype Cache = Cache (ForeignPtr Cache)
@@ -68,6 +69,11 @@ data RocksDB = RocksDB
   { rocksRoot :: FilePath
   , rocksCache :: Maybe Cache
   , rocksCacheIndexAndFilterBlocks :: Bool
+  , rocksMaxDiskSize :: Maybe Int
+    -- ^ virtual limit to report capped disk capacities. The limit is
+    -- not enforced. It's up to each io usage to check diskspace before writing.
+    -- We're using this to avoid serving too many dbs on query servers,
+    -- and smarter sharding.
   }
 
 newStorage :: FilePath -> ServerConfig.Config -> IO RocksDB
@@ -76,11 +82,16 @@ newStorage root ServerConfig.Config{..} = do
     then
       Just <$> newCache (fromIntegral config_db_rocksdb_cache_mb * 1024 * 1024)
     else return Nothing
+  mem_capacity <- totalMemCapacity
   return RocksDB
     { rocksRoot = root
     , rocksCache = cache
     , rocksCacheIndexAndFilterBlocks =
         config_db_rocksdb_cache_index_and_filter_blocks
+    , rocksMaxDiskSize = case mem_capacity of
+        Just mem -> (* mem) . fromIntegral <$>
+          config_db_rocksdb_disk_mem_capacity_ratio_limit
+        Nothing -> Nothing
     }
 
 newtype Container = Container (Ptr Container)
@@ -236,7 +247,11 @@ instance Storage RocksDB where
   getTotalCapacity rocksdb = do
     exists <- doesDirectoryExist (rocksRoot rocksdb)
     if exists
-      then Just <$> getDiskSize (rocksRoot rocksdb)
+      then do
+        fullDiskCapacity <- getDiskSize (rocksRoot rocksdb)
+        return $ Just $ case rocksMaxDiskSize rocksdb of
+          Just maxDiskSize -> min maxDiskSize fullDiskCapacity
+          Nothing -> fullDiskCapacity
       else return Nothing
 
   getUsedCapacity rocksdb = do
@@ -245,7 +260,12 @@ instance Storage RocksDB where
       then Just <$> getUsedDiskSpace (rocksRoot rocksdb)
       else return Nothing
 
-  getFreeCapacity = getFreeDiskSpace . rocksRoot
+  getFreeCapacity rocksdb = do
+    used <- getUsedCapacity rocksdb
+    total <- getTotalCapacity rocksdb
+    case (used,total) of
+      (Just used, Just total) -> return $ total - used
+      _ -> getFreeDiskSpace (rocksRoot rocksdb) -- not aware of disk limit
 
   withScratchRoot rocks f = f $ rocksRoot rocks </> ".scratch"