docs(python): Add IO plugins to Python API reference (#21028)

Co-authored-by: Henry Harbeck <[email protected]>

Author: henryharbeck

Diff for: docs/source/user-guide/plugins/index.md

@@ -1,6 +1,6 @@
 # Plugins
 
-Polars allows you to extend it's functionality with either Expression plugins or IO plugins.
+Polars allows you to extend its functionality with either Expression plugins or IO plugins.
 
 - [Expression plugins](./expr_plugins.md)
 - [IO plugins](./io_plugins.md)

Diff for: docs/source/user-guide/plugins/io_plugins.md

@@ -1,10 +1,10 @@
 # IO Plugins
 
-Besides [expression plugins](./index.md), we also support IO plugins. These allow you to register
-different file formats as sources to the Polars engines. Because sources can move data zero copy via
-Arrow FFI and sources can produce large chunks of data before returning, we've decided to interface
-to IO plugins via Python for now, as we don't think the short time the GIL is needed should lead to
-any contention.
+Besides [expression plugins](./expr_plugins.md), we also support IO plugins. These allow you to
+register different file formats as sources to the Polars engines. Because sources can move data zero
+copy via Arrow FFI and sources can produce large chunks of data before returning, we've decided to
+interface to IO plugins via Python for now, as we don't think the short time the GIL is needed
+should lead to any contention.
 
 E.g. an IO source can read their dataframe's in rust and only at the rendez-vous move the data
 zero-copy having only a short time the GIL is needed.

Diff for: py-polars/docs/source/reference/plugins.rst

@@ -3,13 +3,45 @@ Plugins
 =======
 .. currentmodule:: polars
 
-Plugins allow for extending Polars' functionality. See the
-`user guide <https://docs.pola.rs/user-guide/plugins/>`_ for more information
-and resources.
+Polars allows you to extend its functionality with either Expression plugins or IO plugins.
+See the `user guide <https://docs.pola.rs/user-guide/plugins/>`_ for more information and resources.
 
-Available plugin utility functions are:
+Expression plugins
+------------------
 
-.. automodule:: polars.plugins
-    :members:
-    :autosummary:
-    :autosummary-no-titles:
+Expression plugins are the preferred way to create user defined functions. They allow you to compile
+a Rust function and register that as an expression into the Polars library. The Polars engine will
+dynamically link your function at runtime and your expression will run almost as fast as native
+expressions. Note that this works without any interference of Python and thus no GIL contention.
+
+See the `expression plugins section of the user guide <https://docs.pola.rs/user-guide/plugins/expr_plugins/>`_
+for more information.
+
+.. autosummary::
+    :toctree: api/
+
+    plugins.register_plugin_function
+
+
+IO plugins
+------------------
+
+IO plugins allow you to register different file formats as sources to the Polars engines.
+
+See the `IO plugins section of the user guide <https://docs.pola.rs/user-guide/plugins/io_plugins/>`_
+for more information.
+
+.. note::
+
+    The ``io.plugins`` module is not imported by default in order to optimise import speed of
+    the primary ``polars`` module. Either import ``polars.io.plugins`` and *then* use that
+    namespace, or import ``register_io_source`` from the full module path, e.g.:
+
+    .. code-block:: python
+
+        from polars.io.plugins import register_io_source
+
+.. autosummary::
+    :toctree: api/
+
+    io.plugins.register_io_source

Diff for: py-polars/polars/io/plugins.py

@@ -26,6 +26,14 @@ def register_io_source(
     """
     Register your IO plugin and initialize a LazyFrame.
 
+    See the `user guide <https://docs.pola.rs/user-guide/plugins/io_plugins>`_
+    for more information about plugins.
+
+    .. warning::
+        This functionality is considered **unstable**. It may be changed
+        at any point without it being considered a breaking change.
+
+
     Parameters
     ----------
     callable
@@ -36,17 +44,21 @@ def register_io_source(
             predicate
                 Polars expression. The reader must filter
                 their rows accordingly.
-            n_rows:
+            n_rows
                 Materialize only n rows from the source.
                 The reader can stop when `n_rows` are read.
             batch_size
                 A hint of the ideal batch size the reader's
                 generator must produce.
+
         The function should return a DataFrame batch
         (an iterator over individual DataFrames).
     schema
         Schema that the reader will produce before projection pushdown.
 
+    Returns
+    -------
+    LazyFrame
     """
 
     def wrap(

Diff for: py-polars/polars/plugins.py

@@ -36,7 +36,7 @@ def register_plugin_function(
     """
     Register a plugin function.
 
-    See the `user guide <https://docs.pola.rs/user-guide/plugins/>`_
+    See the `user guide <https://docs.pola.rs/user-guide/plugins/expr_plugins>`_
     for more information about plugins.
 
     Parameters