Merge pull request #23 from liulanzheng/main

update python api and doc for model

Author: yuchen0cc

README.md

@@ -12,7 +12,7 @@ Currently, the OSS connector is composed of two libraries: OSS Model Connector a
 
 - [OSS Model Connector](https://aliyun.github.io/oss-connector-for-ai-ml/#/modelconnector/introduction) focuses on AI inference scenarios, loading large model files from OSS into local AI inference frameworks.
 
-The core part of is OSS Connector for AI/ML is implemented in C++ using [PhotonLibOS](https://github.com/alibaba/PhotonLibOS). This repository only contains the code of Python.
+The core component of the OSS Connector for AI/ML is implemented in C++ using [PhotonLibOS](https://github.com/alibaba/PhotonLibOS) and is provided as dynamic link libraries within wheel packages. This repository only contains the code of Python.
 
 For details, please refer to [ossconnector.github.io](https://ossconnector.github.io/) or [aliyun.github.io/oss-connector-for-ai-ml](https://aliyun.github.io/oss-connector-for-ai-ml).
 

docs/README.md

@@ -8,7 +8,7 @@ Currently, the OSS connector is composed of two libraries: OSS Model Connector a
 
 - [OSS Model Connector](https://aliyun.github.io/oss-connector-for-ai-ml/#/modelconnector/introduction) focuses on AI inference scenarios, loading large model files from OSS into local AI inference frameworks.
 
-The core part of is OSS Connector for AI/ML is implemented in C++ using [PhotonLibOS](https://github.com/alibaba/PhotonLibOS). This repository only contains the code of Python.
+The core component of the OSS Connector for AI/ML is implemented in C++ using [PhotonLibOS](https://github.com/alibaba/PhotonLibOS) and is provided as dynamic link libraries within wheel packages. This repository only contains the code of Python.
 
 
 ## License

docs/modelconnector/framworks.md

@@ -56,9 +56,11 @@ total 136G
 -rw-r--r-- 1 root root 2.7M Sep 25 15:43 vocab.json
 ```
 
+Another common scenario is like the Stable Diffusion web UI, where a large number of models are stored in one or several folders, and there might be situations where models need to be switched during use.
+
 The OssModelConnector offers a method to directly pass in an OSS directory to the inference frameworks and read the models directly from OSS.
 
-Compared to downloading before loading to framworks, the OssModelConnector allows for simultaneous downloading and loading, achieving faster model deployment speeds.
+Compared to the FUSE-based mounting solution, OssModelConnector has a significant performance advantage. Compared to downloading before loading to framworks, the OssModelConnector allows for simultaneous downloading and loading, achieving faster model deployment speeds.
 
 ## Usage
 
@@ -123,4 +125,32 @@ llm = LLM(model=model_dir, trust_remote_code=True)
 connector.close()
 
 # do inference
-```
+```
+
+# Stable Diffusion web UI
+
+Edit launch.py to initalize and configure OssModelConnector.
+
+```python
+from modules import launch_utils
+
+import oss2
+from oss2.credentials import EnvironmentVariableCredentialsProvider
+from ossmodelconnector import OssModelConnector
+
+...
+
+def main():
+    ...
+
+
+if __name__ == "__main__":
+    connector = OssModelConnector(endpoint='oss-cn-beijing-internal.aliyuncs.com',
+                                  cred_provider=EnvironmentVariableCredentialsProvider(),
+                                  config_path='/etc/connector.json')
+    connector.prepare_directory('oss://ai-testset/Stable-diffusion/', '/root/stable-diffusion-webui/models/Stable-diffusion')
+
+    main()
+```
+
+Currently, prepare_directory() loads all models into memory, which can put pressure on memory and even cause crashes in scenarios with a large number of models. In the future, prepare_directory() will support lazy loading, downloading models only when switching to or open them, and it will include a garbage collection feature to release memory for unused models after a specified time.

docs/modelconnector/python_apis.md

@@ -22,8 +22,8 @@ Users can create an OssModelConnector in Python and call its provided methods to
 
     ```python
     connector = OssModelConnector(endpoint=ENDPOINT,
-                              cred_provider=EnvironmentVariableCredentialsProvider(),
-                              config_path='/tmp/config.json')
+                                  cred_provider=EnvironmentVariableCredentialsProvider(),
+                                  config_path='/tmp/config.json')
     ```
 
 - List objects

oss-model-connector/ossmodelconnector/oss_model_connector.py

@@ -107,24 +107,25 @@ def open(self, uri, binary = True):
         return self._connector.open(uri, True, True, binary)
 
     def _from_file_helper(self, filename, shared, nbytes):
-        file = self._connector.open(filename, True, True)
-        return UntypedStorageEx(file, nbytes)
-
-    def _connector_open(self, *args, **kwargs):
-        filename = args[0]
-        if isinstance(filename, pathlib.Path):
-            filename = str(filename)
-        open_mode = 'r' if len(args) == 1 else args[1]
         if self._hook_dir and filename.startswith(self._hook_dir):
+            file = self._connector.open(filename, True, True)
+            return UntypedStorageEx(file, nbytes)
+        else:
+            return self._origin_from_file(filename, shared, nbytes)
+
+    def _connector_open(self, file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None):
+        if isinstance(file, pathlib.Path):
+            file = str(file)
+        if self._hook_dir and file.startswith(self._hook_dir):
             binary = False
-            if open_mode == "rb":
+            if 'b' in mode:
                 binary = True
             try:
-                return self.open(filename, binary)
+                return self.open(file, binary)
             except:
-                return self._origin_open(*args, **kwargs)
+                return self._origin_open(file, mode, buffering, encoding, errors, newline, closefd, opener)
         else:
-            return self._origin_open(*args, **kwargs)
+            return self._origin_open(file, mode, buffering, encoding, errors, newline, closefd, opener)
 
     def prepare_directory(self, uri: str, dir: str, libc_hook: bool = False):
         """