You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/src/introduction/overview.md
+20-22Lines changed: 20 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,61 +12,59 @@ it both compiler and autodiff friendly.
12
12
13
13
Lux.jl takes a **Reactant-first approach** to deliver exceptional performance and seamless deployment capabilities:
14
14
15
-
***XLA Compilation** -- Lux models compile to highly optimized XLA code via [Reactant.jl](https://github.com/EnzymeAD/Reactant.jl), delivering significant speedups on CPU, GPU, and TPU.
15
+
-**XLA Compilation** -- Lux models compile to highly optimized XLA code via [Reactant.jl](https://github.com/EnzymeAD/Reactant.jl), delivering significant speedups on CPU, GPU, and TPU.
16
16
17
-
***Cross-Platform Performance** -- Run the same Lux model with optimal performance across different hardware backends (CPU, NVIDIA GPUs, AMD GPUs, TPUs) without code changes, simply by switching the Reactant backend.
17
+
-**Cross-Platform Performance** -- Run the same Lux model with optimal performance across different hardware backends (CPU, NVIDIA GPUs, AMD GPUs, TPUs) without code changes, simply by switching the Reactant backend.
18
18
19
-
***Production Deployment** -- Compiled models can be exported and deployed to production servers and edge devices by leveraging the rich TensorFlow ecosystem, making Lux suitable for real-world applications.
19
+
-**Production Deployment** -- Compiled models can be exported and deployed to production servers and edge devices by leveraging the rich TensorFlow ecosystem, making Lux suitable for real-world applications.
20
20
21
-
***Large Model Support** -- With Reactant compilation, Lux now excels at training very large models that were previously challenging, making it competitive with other frameworks for large-scale deep learning.
21
+
-**Large Model Support** -- With Reactant compilation, Lux now excels at training very large models that were previously challenging, making it competitive with other frameworks for large-scale deep learning.
22
22
23
23
## Design Principles
24
24
25
-
***Layers must be immutable** -- cannot store any parameter/state but rather store the
25
+
-**Layers must be immutable** -- cannot store any parameter/state but rather store the
26
26
information to construct them
27
-
***Layers are pure functions**
28
-
***Layers return a Tuple containing the result and the updated state**
29
-
***Given same inputs the outputs must be same** -- yes this must hold true even for
27
+
-**Layers are pure functions**
28
+
-**Layers return a Tuple containing the result and the updated state**
29
+
-**Given same inputs the outputs must be same** -- yes this must hold true even for
30
30
stochastic functions. Randomness must be controlled using `rng`s passed in the state.
31
-
***Easily extensible**
32
-
***Extensive Testing** -- All layers and features are tested across all supported AD
31
+
-**Easily extensible**
32
+
-**Extensive Testing** -- All layers and features are tested across all supported AD
33
33
backends across all supported hardware backends.
34
34
35
35
## Why use Lux over Flux?
36
36
37
-
***High-Performance XLA Compilation** -- Lux's Reactant-first approach enables XLA compilation for dramatic performance improvements across CPU, GPU, and TPU. Models compile to highly optimized code that eliminates Julia overhead and leverages hardware-specific optimizations.
37
+
-**High-Performance XLA Compilation** -- Lux's Reactant-first approach enables XLA compilation for dramatic performance improvements across CPU, GPU, and TPU. Models compile to highly optimized code that eliminates Julia overhead and leverages hardware-specific optimizations.
38
38
39
-
***Production-Ready Deployment** -- Deploy Lux models to production environments using the mature TensorFlow ecosystem. Compiled models can be exported and run on servers, edge devices, and mobile platforms.
39
+
-**Production-Ready Deployment** -- Deploy Lux models to production environments using the mature TensorFlow ecosystem. Compiled models can be exported and run on servers, edge devices, and mobile platforms.
40
40
41
-
***Neural Networks for SciML**: For SciML Applications (Neural ODEs, Deep Equilibrium
41
+
-**Neural Networks for SciML**: For SciML Applications (Neural ODEs, Deep Equilibrium
42
42
Models) solvers typically expect a monolithic parameter vector. Flux enables this via its
43
43
`destructure` mechanism, but `destructure` comes with various
44
44
[edge cases and limitations](https://fluxml.ai/Optimisers.jl/dev/api/#Optimisers.destructure). Lux
45
45
forces users to make an explicit distinction between state variables and parameter
46
46
variables to avoid these issues. Also, it comes battery-included for distributed training.
47
47
48
-
***Sensible display of Custom Layers** -- Ever wanted to see Pytorch like Network printouts
48
+
-**Sensible display of Custom Layers** -- Ever wanted to see Pytorch like Network printouts
49
49
or wondered how to extend the pretty printing of Flux's layers? Lux handles all of that
50
50
by default.
51
51
52
-
***Truly immutable models** - No *unexpected internal mutations* since all layers are
53
-
implemented as pure functions. All layers are also *deterministic* given the parameters
52
+
-**Truly immutable models** - No _unexpected internal mutations_ since all layers are
53
+
implemented as pure functions. All layers are also _deterministic_ given the parameters
54
54
and state: if a layer is supposed to be stochastic (say [`Lux.Dropout`](@ref)), the state
55
55
must contain a seed which is then updated after the function call.
56
56
57
-
***Easy Parameter Manipulation** -- By separating parameter data and layer structures,
57
+
-**Easy Parameter Manipulation** -- By separating parameter data and layer structures,
58
58
Lux makes implementing [`WeightNorm`](@ref), `SpectralNorm`, etc. downright trivial.
59
59
Without this separation, it is much harder to pass such parameters around without
60
60
mutations which AD systems don't like.
61
61
62
-
***Wider AD Support** -- Lux has extensive support for most
62
+
-**Wider AD Support** -- Lux has extensive support for most
63
63
[AD systems in julia](@ref autodiff-lux), while Flux is mostly tied to Zygote (with some
64
64
initial support for Enzyme).
65
65
66
-
***Optimized for All Model Sizes** -- Whether you're working with small prototypes or large production models, Lux delivers optimal performance. For the smallest networks where minimal overhead is critical, you can use [`ToSimpleChainsAdaptor`](@ref) to leverage SimpleChains.jl's specialized CPU optimizations.
66
+
-**Optimized for All Model Sizes** -- Whether you're working with small prototypes or large production models, Lux delivers optimal performance. For the smallest networks where minimal overhead is critical, you can use [`ToSimpleChainsAdaptor`](@ref) to leverage SimpleChains.jl's specialized CPU optimizations.
67
67
68
-
***Reliability** -- We have learned from the mistakes of the past with Flux and everything
68
+
-**Reliability** -- We have learned from the mistakes of the past with Flux and everything
69
69
in our core framework is extensively tested, along with downstream CI to ensure that
0 commit comments