the constructors which allow for caching need to be documented

[micjoswig]

The caching mechanism is used extensively for polynomial rings and, more generally, objects/types from the Hecke/Nemo universe.

I think it would be useful to have one extra page of documentation which just lists all constructors which support this feature.

The reason is that the behavior is sometimes unexpected to the uninitiated user, as it is not uniform across all types in OSCAR.

@fieker

[thofma]

We can try to cook something up (after certain deadlines have passed), but I am not sure what to address exactly. Is the existence of caching surprising or the non-existence? I think it would be helpful if there is a victim of caching that I could talk to.

[thofma]

Conclusion of the meeting: The request is to have a table of functions which support caching and some explanation.

[JohnAAbbott]

I agree with @fieker that caching is a convenience for interactive use, but any code we write as part of OSCAR should be independent of caching (as regards correctness, and hopefully also execution efficiency). A suggestion of @benlorenz to have a (developer only) global flag which disables all caching would facilitate verification that our code is caching-independent.
Might it be helpful to designate/document 3 categories of function:

• functions where caching will "always" be available
• functions where caching will "never" be available
• the rest

Is caching relevant only for constructors? Maple's option remember can be applied to (almost?) any function.

[fingolfin]

Perhaps to get started we can write code which iterates over all functions exported (?) by Oscar and check if they have a cache keyword argument (we kinda need this list for #3491, too). That may contain some false positives (and/or miss some functions) but at least it would be a start...

Also this list is susceptible to becoming outdated over time, so if we could generate it automatically (perhaps by combining the above trick with a list of additional inclusions/exclusion) that would be best...?

Now we just need "someone"(TM) to work on it. Perhaps @aaruni96 can look into it (next week, after ICMS deadline)

[aaruni96]

I've come up with a pipe of greps and seds from the Oscar.jl directory which produces 28 functions.

grep -d recurse "^export" src | sed "s/^.*export /\^/" | sed "s/, /\n/g" | sed "s/\(\^[0-9a-zA-Z_\!]*\).*/\1/"  > exports.txt
grep -n -d recurse -w "^\s*function" src | grep "cached::Bool" | sed "s/^.*function //" | sed "s/).*$/)/" | grep -f exports.txt -w

FreeMod(R::Ring, n::Int, name::VarName = :e; cached::Bool = false)
FreeMod(R::Ring, names::Vector{String}; cached::Bool=false)
FreeMod(R::Ring, names::Vector{Symbol}; cached::Bool=false)
FreeMod_dec(R::CRing_dec, n::Int, name::VarName = :e; cached::Bool = false)
FreeMod_dec(R::CRing_dec, d::Vector{GrpAbFinGenElem}, name::VarName = :e; cached::Bool = false)
kaehler_differentials(R::Union{MPolyRing, MPolyLocRing}; cached::Bool=true)
kaehler_differentials(R::MPolyQuoRing; cached::Bool=true)
kaehler_differentials(R::MPolyQuoLocRing; cached::Bool=true)
kaehler_differentials(R::Ring, p::Int; cached::Bool=true)
de_rham_complex(R::Ring; cached::Bool=true)
exterior_power(M::SubquoModule, p::Int; cached::Bool=true)
exterior_power(F::FreeMod, p::Int; cached::Bool=true)
koszul_complex(v::FreeModElem; cached::Bool=true)
koszul_complex(v::FreeModElem, M::ModuleFP; cached::Bool=true)
koszul_homology(v::FreeModElem, i::Int; cached::Bool=true)
koszul_homology(v::FreeModElem, M::ModuleFP, i::Int; cached::Bool=true)
polynomial_ring(R::TropicalSemiring, s::Symbol; cached::Bool = true)
polynomial_ring(R::AbstractAlgebra.Ring, v1::Pair{<:VarName, <:Any}, v...; cached::Bool = false, ordering::Symbol = :lex)
Hecke.number_field(::QQField, a::QQAbElem; cached::Bool = false)
Hecke.number_field(::QQField, a::AbstractVector{<: QQAbElem}; cached::Bool = false)
SLPolyRing(r::Ring, s::Vector{Symbol}; cached::Bool = false)
Hecke.number_field(::QQField, chi::GAPGroupClassFunction; cached::Bool = false)
slpoly_ring(R::AbstractAlgebra.Ring, n::Int; cached::Bool = false)
slpoly_ring(R::AbstractAlgebra.Ring, p::Pair{Symbol, <:AbstractVector{Int}}...; cached::Bool = false)
extension_field(f::ZZPolyRingElem, n::String = "_a"; cached::Bool = true, check::Bool = true)
extension_field(f::QQPolyRingElem, n::String = "_a"; cached::Bool = true, check::Bool = true)
extension_field(f::Generic.Poly{<:Generic.RationalFunctionFieldElem{T}}, n::String = "_a";  cached::Bool = true, check::Bool = true)
extension_field(f::Generic.Poly{nf_elem}, n::String = "_a";  cached::Bool = true, check::Bool = true)

[lgoettgens]

I've come up with a pipe of greps and seds from the Oscar.jl directory which produces 28 functions.

Of course there is even more in Hecke/Nemo/AA reexported here

[fieker]

On Thu, Apr 11, 2024 at 02:07:25AM -0700, Aaruni Kaushik wrote: I've come up with a pipe of greps and seds from the Oscar.jl directory which produces 28 functions. ```bash grep -d recurse "^export" src | sed "s/^.*export /\^/" | sed "s/, /\n/g" | sed "s/\(\^[0-9a-zA-Z_\!]*\).*/\1/" > exports.txt grep -n -d recurse -w "^\s*function" src | grep "cached::Bool" | sed "s/^.*function //" | sed "s/).*$/)/" | grep -f exports.txt -w ```

I'm afaraid that will also have to be run against Hecke.jl Nemo.jl AbstractAlgebra.jl and possible Singular.jl Polymake.jl

…

``` FreeMod(R::Ring, n::Int, name::VarName = :e; cached::Bool = false) FreeMod(R::Ring, names::Vector{String}; cached::Bool=false) FreeMod(R::Ring, names::Vector{Symbol}; cached::Bool=false) FreeMod_dec(R::CRing_dec, n::Int, name::VarName = :e; cached::Bool = false) FreeMod_dec(R::CRing_dec, d::Vector{GrpAbFinGenElem}, name::VarName = :e; cached::Bool = false) kaehler_differentials(R::Union{MPolyRing, MPolyLocRing}; cached::Bool=true) kaehler_differentials(R::MPolyQuoRing; cached::Bool=true) kaehler_differentials(R::MPolyQuoLocRing; cached::Bool=true) kaehler_differentials(R::Ring, p::Int; cached::Bool=true) de_rham_complex(R::Ring; cached::Bool=true) exterior_power(M::SubquoModule, p::Int; cached::Bool=true) exterior_power(F::FreeMod, p::Int; cached::Bool=true) koszul_complex(v::FreeModElem; cached::Bool=true) koszul_complex(v::FreeModElem, M::ModuleFP; cached::Bool=true) koszul_homology(v::FreeModElem, i::Int; cached::Bool=true) koszul_homology(v::FreeModElem, M::ModuleFP, i::Int; cached::Bool=true) polynomial_ring(R::TropicalSemiring, s::Symbol; cached::Bool = true) polynomial_ring(R::AbstractAlgebra.Ring, v1::Pair{<:VarName, <:Any}, v...; cached::Bool = false, ordering::Symbol = :lex) Hecke.number_field(::QQField, a::QQAbElem; cached::Bool = false) Hecke.number_field(::QQField, a::AbstractVector{<: QQAbElem}; cached::Bool = false) SLPolyRing(r::Ring, s::Vector{Symbol}; cached::Bool = false) Hecke.number_field(::QQField, chi::GAPGroupClassFunction; cached::Bool = false) slpoly_ring(R::AbstractAlgebra.Ring, n::Int; cached::Bool = false) slpoly_ring(R::AbstractAlgebra.Ring, p::Pair{Symbol, <:AbstractVector{Int}}...; cached::Bool = false) extension_field(f::ZZPolyRingElem, n::String = "_a"; cached::Bool = true, check::Bool = true) extension_field(f::QQPolyRingElem, n::String = "_a"; cached::Bool = true, check::Bool = true) extension_field(f::Generic.Poly{<:Generic.RationalFunctionFieldElem{T}}, n::String = "_a"; cached::Bool = true, check::Bool = true) extension_field(f::Generic.Poly{nf_elem}, n::String = "_a"; cached::Bool = true, check::Bool = true) ``` -- Reply to this email directly or view it on GitHub: #3450 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

[micjoswig]

I've come up with a pipe of greps and seds from the Oscar.jl directory which produces 28 functions.

grep -d recurse "^export" src | sed "s/^.*export /\^/" | sed "s/, /\n/g" | sed "s/\(\^[0-9a-zA-Z_\!]*\).*/\1/"  > exports.txt
grep -n -d recurse -w "^\s*function" src | grep "cached::Bool" | sed "s/^.*function //" | sed "s/).*$/)/" | grep -f exports.txt -w

Thanks, @aaruni96

Honestly, when I look at the output, the situation is worse than I had imagined. What worries me most is the (to my eyes) rather wild distribution of true/false defaults.

[aaruni96]

There's a lot more functions which use cache in the various sub projects. I've added them in #3599 instead of posting a wall of text directly in this discussion

[afkafkafk13]

I've come up with a pipe of greps and seds from the Oscar.jl directory which produces 28 functions.

grep -d recurse "^export" src | sed "s/^.*export /\^/" | sed "s/, /\n/g" | sed "s/\(\^[0-9a-zA-Z_\!]*\).*/\1/"  > exports.txt
grep -n -d recurse -w "^\s*function" src | grep "cached::Bool" | sed "s/^.*function //" | sed "s/).*$/)/" | grep -f exports.txt -w

Thanks, @aaruni96

Honestly, when I look at the output, the situation is worse than I had imagined. What worries me most is the (to my eyes) rather wild distribution of true/false defaults.

Actually, what worries me most are the functions, for which I have experienced that they do trigger caching deep down in (towers of) functions they use, but do not have a cached=... argument. They do not do the caching themselves, but it simply happens somewhere down the road (sometimes even only in corner cases....), and you do not stand a chance to see it from the outside. One example of such a thing was issue #3488 (solved in a brute force way in PR #3536).