Add HF_TRUST_REMOTE_CODE environment variable

[alvarobartt]

Description

As flagged by Changyu Zhu from Google, there was no way of setting trust_remote_code=True when loading a model from the Hugging Face Hub that requires remote code execution; whilst for Text Generation Inference (TGI) is indeed possible via the TRUST_REMOTE_CODE environment variable (as per https://huggingface.co/docs/text-generation-inference/en/basic_tutorials/safety).

This PR adds the HF_TRUST_REMOTE_CODE environment variable in order to be able to set it for transformers, sentence-transformers, and diffusers pipelines. Additionally, this PR also fixes the **kwargs propagation for both sentence-transformers and diffusers. Finally, this PR also updates the README.md accordingly as of the HF_TRUST_REMOTE_CODE addition, while also fixing some typos and aligning the formatting.

Note

As of the recent merges within #76 and #77, the version in setup.py has also been bumped to 0.4.1 to include all those changes and generate the wheel accordingly. So on, once this PR is merged, the version in the main branch should point to 0.4.2.dev0 instead.

[oOraph]

LGTM, question: under the hood, this trust_remote_code bypasses security flags like weights_only=True in pytorch and stuff right ? (allowing for arbitrary pickle loading)

Where do you intend to activate it ? In one of your hf endpoints using a custom env variable ? (just want to make sure we do not activate it by default there :) )

[alvarobartt]

LGTM, question: under the hood, this trust_remote_code bypasses security flags like weights_only=True in pytorch and stuff right ? (allowing for arbitrary pickle loading)
Where do you intend to activate it ? In one of your hf endpoints using a custom env variable ? (just want to make sure we do not activate it by default there :) )

It's indeed disabled by default, and explicitly enabled on the user's side if desired; to basically load models that live within the Hub repository instead of directly being supported / integrated within transformers i.e. custom modeling. And the use case for this is when users run a job in either Vertex AI or GKE using our custom DLCs, as the PyTorch Inference DLC is running huggingface_inference_toolkit under the hood, so that users can load models such as https://huggingface.co/LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct 🤗

P.S. Thanks for the review!

[alvarobartt]

To add more context to this PR, it's updated here huggingface/Google-Cloud-Containers#64