L1 Gauss Seidel preconditioner #191

Abdelrahman912 · 2025-02-28T03:32:42Z

Rough ideas for the data structures that incorporate partitioned arrays with CPU example of Gauss Seidel with and without partitions 📚

codecov-commenter · 2025-02-28T03:41:37Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

src/solver/linear/smoother.jl

codecov · 2025-03-26T01:37:24Z

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

ext/CuThunderboltExt.jl

termi-official · 2025-04-09T21:58:15Z

~~Is it possible that you just use too many threads? Also, do you pin the threads for the benchmarks?~~ Performance is more fundamentally broken here. If you try the matrix "MathWorks/Kuu" you see that there is a massive performance diff.

src/solver/linear/preconditioners/Preconditioners.jl

src/solver/linear/preconditioners/l1_gauss_seidel.jl

Abdelrahman912 · 2025-04-10T01:52:44Z

Unsymmetric Stats

Unpreconditioned

SimpleStats
 niter: 981
 solved: true
 inconsistent: false
 residuals: [ 6.2e+01  6.2e+01  6.2e+01 ...  1.1e-06  1.0e-06  8.2e-07 ]
 Aresiduals: []
 κ₂(A): []
 timer: 719.44ms
 status: solution good enough given atol and rtol

Preconditioned

nparts = 8 (same as cores)
partsize = 414

SimpleStats
 niter: 158
 solved: true
 inconsistent: false
 residuals: [ 4.5e+00  4.3e+00  3.9e+00 ...  1.1e-07  8.8e-08  6.5e-08 ]
 Aresiduals: []
 κ₂(A): []
 timer: 186.18ms
 status: solution good enough given atol and rtol

Abdelrahman912 · 2025-04-10T06:06:29Z

Symmetric Problem:

bcsstk15 https://math.nist.gov/MatrixMarket/data/Harwell-Boeing/bcsstruc2/bcsstruc2.html

Unpreconditioned stats:

SimpleStats
 niter: 3067
 solved: true
 inconsistent: false
 residuals: [ 6.3e+01  5.9e+01  5.8e+01 ...  9.8e-07  9.6e-07  9.2e-07 ]
 Aresiduals: []
 κ₂(A): []
 timer: 10.24s
 status: solution good enough given atol and rtol

Preconditioned stats:

SimpleStats
 niter: 497
 solved: true
 inconsistent: false
 residuals: [ 2.4e+00  3.7e-04  2.3e-04 ...  5.4e-08  5.2e-08  5.0e-08 ]
 Aresiduals: []
 κ₂(A): []
 timer: 487.80ms
 status: solution good enough given atol and rtol

termi-official

Looks great already. Some last fine tuning and we are ready to go.

src/solver/linear/preconditioners/l1_gauss_seidel.jl

ext/cuda/cuda_preconditioner.jl

src/solver/linear/preconditioners/l1_gauss_seidel.jl

test/test_preconditioners.jl

ext/cuda/cuda_preconditioner.jl

src/solver/linear/preconditioners/l1_gauss_seidel.jl

ext/cuda/cuda_preconditioner.jl

…arning to installed CUDA version

ext/CuThunderboltExt.jl

Abdelrahman912 · 2025-04-18T13:16:05Z

ext/cuda/cuda_preconditioner.jl

+# SparseMatricesCSR.colvals(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.colVal
+# SparseMatricesCSR.getrowptr(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.rowPtr


these two lines should be added in JuliaGPU/CUDA.jl#2720, right ?

Actually I am not sure about this one. Probably not, because SparseMatricesCSR is not an interface package. I would discuss this in the linked PR.

termi-official · 2025-04-22T15:33:16Z

Project.toml

Can you add a compat entry for the new CUDA release?

termi-official · 2025-04-22T15:33:48Z

Project.toml


 [targets]
-test = ["Aqua", "DelimitedFiles", "JET", "Pkg", "StaticArrays", "Tensors", "Test"]
+test = ["Aqua", "DelimitedFiles", "JET", "Pkg", "StaticArrays", "Tensors", "Test","MatrixDepot","ThreadPinning"]


Do we need ThreadPinning for testing?

termi-official · 2025-04-22T15:34:43Z

ext/cuda/cuda_preconditioner.jl

+# Adapt.adapt(::CUDABackend, x::Vector) = x |> cu # not needed 
+# Adapt.adapt(::CUDABackend, x::CuVector) = x # not needed
+
+# TODO: remove this function if back compatibility is not needed


Backward compat with what?

termi-official · 2025-04-22T15:35:44Z

test/test_preconditioners.jl

I think we are missing a test for the ThreadedSparseCSR format here

termi-official · 2025-04-22T15:36:44Z

src/solver/linear/preconditioners/l1_gauss_seidel.jl

+@doc raw"""
+    L1GSPreconditioner{Partitioning, VectorType}
+
+The ℓ₁ Gauss–Seidel preconditioner is a robust and parallel-friendly preconditioner, particularly effective for sparse and ill-conditioned systems.


ill-conditioned systems

Where have you got that statement from?

termi-official · 2025-04-22T15:47:28Z

ext/CuThunderboltExt.jl

@@ -10,7 +15,7 @@ import CUDA:
 import Thunderbolt:
    UnPack.@unpack,
    SimpleMesh,
-    SparseMatrixCSR, SparseMatrixCSC,
+    SparseMatrixCSR, SparseMatrixCSC, AbstractSparseMatrix,


Can we import these types here from their respective packages instead of Thunderbolt? We might hit weird bugs otherwise.

termi-official · 2025-04-22T15:48:22Z

ext/cuda/cuda_preconditioner.jl

+# PIRACY ALERT: the following code is commented out to avoid piracy
+# SparseMatricesCSR.colvals(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.colVal
+# SparseMatricesCSR.getrowptr(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.rowPtr


Suggested change

# PIRACY ALERT: the following code is commented out to avoid piracy

# SparseMatricesCSR.colvals(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.colVal

# SparseMatricesCSR.getrowptr(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.rowPtr

termi-official · 2025-04-22T15:49:52Z

ext/cuda/cuda_utils.jl

Can we remove these here for now and add a comment to the tests that prior to 2720 we need to add these dispatches? I do not want to artificially postpone this PR here further due to missing optional stuff in the dependencies.

termi-official · 2025-04-22T15:50:26Z

ext/CuThunderboltExt.jl

+if __cuda_version__ >= __min_cuda_version__
+    include("cuda/cuda_preconditioner.jl")
+else
+    @warn("CuThunderboltExt.jl: CUDA.jl version is too old $__cuda_version__ <$__min_cuda_version__, skipping `cuda_preconditioner.jl`.")
+end


Suggested change

if __cuda_version__ >= __min_cuda_version__

include("cuda/cuda_preconditioner.jl")

else

@warn("CuThunderboltExt.jl: CUDA.jl version is too old $__cuda_version__ <$__min_cuda_version__, skipping `cuda_preconditioner.jl`.")

end

include("cuda/cuda_preconditioner.jl")

add a compat entry to the Project.toml.

termi-official · 2025-04-22T16:04:17Z

src/solver/linear/preconditioners/l1_gauss_seidel.jl

+    # y: preconditioned residual
+    @unpack partitioning,B,D = P
+    @unpack backend = partitioning
+    @. y = x / (B + D)


Can we precompute 1/(B+D)? This should give us a speedup, as multiplication is usually way faster than division.

Also, I think we are missing the Gauss-Seidel part on the local block?

init l1 smoother

2382a8f

minor fix

e8991ba

termi-official reviewed Feb 28, 2025

View reviewed changes

src/solver/linear/smoother.jl Outdated Show resolved Hide resolved

src/solver/linear/smoother.jl Outdated Show resolved Hide resolved

Abdelrahman912 and others added 10 commits March 23, 2025 21:50

init cuda prec setup

a127bc0

init working cuda

d216e34

fix partition limit indices

086ed78

add comment

99b1ab6

minor change

64a247c

minor adjustment for csc

343c99d

add cuda csr

ee66f73

check symmetry for csc

fbb3338

Merge branch 'main' into l1-gs-smoother

2b2dc17

rm unnecessary code

5e1203a

termi-official reviewed Mar 26, 2025

View reviewed changes

ext/CuThunderboltExt.jl Outdated Show resolved Hide resolved

Abdelrahman912 and others added 3 commits March 26, 2025 23:52

Merge branch 'main' into l1-gs-smoother

bc1cec3

add cpu version

c8cc291

Merge branch 'add-multi-threading-l1-prec' into l1-gs-smoother

457c110

Abdelrahman912 changed the title ~~L1 Gauss Seidel smoother~~ L1 Gauss Seidel preconditioner Mar 28, 2025

Abdelrahman912 and others added 10 commits March 29, 2025 01:54

Merge branch 'main' into l1-gs-smoother

7f17845

Merge branch 'main' into l1-gs-smoother

7c9b474

init ka, working but buggy.

42026cd

Merge branch 'main' into l1-gs-smoother

1b3ce6f

Merge branch 'ka-porting' into l1-gs-smoother

8a5675b

Merge branch 'main' into l1-gs-smoother

476928b

fix ka buggy code

8bcca25

add tests

77f7148

minor fix

aecbf67

update manifest

1aa8986

minor fix

1f56bad

termi-official reviewed Apr 9, 2025

View reviewed changes

Abdelrahman912 added 2 commits April 10, 2025 01:58

set nparts to be ncores

226f55a

precompute blocks

292aacc

Abdelrahman912 added 4 commits April 10, 2025 05:40

separate CPU GPU tests

36f5754

fix ci

33f1de7

minor fix

3b52869

add symmetric test

a056a67

termi-official reviewed Apr 10, 2025

View reviewed changes

Abdelrahman912 added 5 commits April 10, 2025 14:10

rm dead code

bf2cc96

comment out adapt

a84ab45

rm direct solver

0794dce

add doc string

2532179

add gpu test examples

8203cac

Abdelrahman912 requested a review from termi-official April 11, 2025 00:12

termi-official reviewed Apr 11, 2025

View reviewed changes

src/solver/linear/preconditioners/l1_gauss_seidel.jl Outdated Show resolved Hide resolved

Abdelrahman912 commented Apr 14, 2025

View reviewed changes

ext/cuda/cuda_preconditioner.jl Outdated Show resolved Hide resolved

Abdelrahman912 added 5 commits April 14, 2025 14:17

minor fix

7ef9e15

elementwise operations refinement

869d3db

add reference

9609d59

add block partitioning to doc string + some comments for (CSC/CSR)Format

8e62678

rm piratical code (only those which were merged into CUDA.jl) + add w…

4a6454c

…arning to installed CUDA version

Abdelrahman912 commented Apr 18, 2025

View reviewed changes

ext/CuThunderboltExt.jl Show resolved Hide resolved

Abdelrahman912 commented Apr 18, 2025

View reviewed changes

rm dead code

cce6547

termi-official requested changes Apr 22, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

L1 Gauss Seidel preconditioner #191

L1 Gauss Seidel preconditioner #191

Abdelrahman912 commented Feb 28, 2025 •

edited

Loading

codecov-commenter commented Feb 28, 2025

codecov bot commented Mar 26, 2025

termi-official commented Apr 9, 2025 •

edited

Loading

Abdelrahman912 commented Apr 10, 2025

Abdelrahman912 commented Apr 10, 2025 •

edited

Loading

termi-official left a comment

Abdelrahman912 Apr 18, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

termi-official Apr 22, 2025

		# SparseMatricesCSR.colvals(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.colVal
		# SparseMatricesCSR.getrowptr(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.rowPtr

	# PIRACY ALERT: the following code is commented out to avoid piracy
	# SparseMatricesCSR.colvals(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.colVal
	# SparseMatricesCSR.getrowptr(A::CUSPARSE.CuSparseDeviceMatrixCSR{Tv,Ti,1}) where {Tv,Ti} = A.rowPtr

L1 Gauss Seidel preconditioner #191

Are you sure you want to change the base?

L1 Gauss Seidel preconditioner #191

Conversation

Abdelrahman912 commented Feb 28, 2025 • edited Loading

codecov-commenter commented Feb 28, 2025

Welcome to Codecov 🎉

codecov bot commented Mar 26, 2025

Welcome to Codecov 🎉

termi-official commented Apr 9, 2025 • edited Loading

Abdelrahman912 commented Apr 10, 2025

Unsymmetric Stats

Unpreconditioned

Preconditioned

Abdelrahman912 commented Apr 10, 2025 • edited Loading

Symmetric Problem:

Unpreconditioned stats:

Preconditioned stats:

termi-official left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Abdelrahman912 commented Feb 28, 2025 •

edited

Loading

termi-official commented Apr 9, 2025 •

edited

Loading

Abdelrahman912 commented Apr 10, 2025 •

edited

Loading