Proposals for advanced topics

[behinger]

I received some proposals for advanced topics:

• deep learning frameworks in Julia -- where to start? what are potential performance gaps wrt. jax/pytorch? @ranocha have you worked with Flux.jl?
• an entrypoint to the SciML ecosystem @behinger I have a simple demo - but not extensive experience
• probabilistic programming in Julia (e.g. Turing?) - @behinger I have a simple demo - but practically no experience iwth Turing itself

[ranocha]

• I have worked a bit with Flux.jl some time ago.
• I am part of the SciML ecosystem - focussing on differential equations, less on the ML stuff

[smith-garrett]

One thing that would interest me is writing custom types that are safe for automatic differentiation. This seems to involve setting up parametric types for any fields in the struct, e.g.

struct MyType{T} where T:
    x::T
end

But I don't really understand the details of how this works or what it is doing. How would this be different than leaving out type annotation totally and letting the compiler figure it out, especially when an AD number type gets passed as a value of x?

[ranocha]

I can mention that on Friday if there is interest

[JamieJQuinn]

I'd be particularly interested in a basic introduction to macros.

[ranocha]

I'd be particularly interested in a basic introduction to macros.

Like using macros or writing macros? Anything particularly interesting for you?

[JamieJQuinn]

@ranocha writing macros. I'm interested in writing small domain specific languages but just understanding the basics + some good practices would be great!

[behinger]

@JamieJQuinn given that @pszufe just talked about Macros - is that enough for you?

[JamieJQuinn]

Aha, sure. I can't imagine it's very useful for many anyway!

[pszufe]

I just said a small bit on macros so people understand how it is possible to have DSLs in Julia.
@JamieJQuin for your DSL you might want first to try to define your own set of data types and overload the algebra system for that. No matter what DSL you create this is usually the first step anyway. In many cases doing that and providing 1-2 helper macros if enough. Writing macros is actually quite difficult and time consuming so my advice is - avoid it if you only can. It is basically much more difficult that it seems and most problems in the end are solvable via Julia's abstraction mechanism.

[pszufe]

One thing that would interest me is writing custom types that are safe for automatic differentiation. This seems to involve setting up parametric types for any fields in the struct, e.g.

struct MyType{T} where T:
    x::T
end

But I don't really understand the details of how this works or what it is doing. How would this be different than leaving out type annotation totally and letting the compiler figure it out, especially when an AD number type gets passed as a value of x?

@smith-garrett whenever you define a struct one thing you should never do is:

struct MyType
     x
end

or

struct MyType
     x::SomeAbstractType
end

Both are terrible as they do not allow the compiler to optimize the code.
Naturally this will not work in predictable manner with AD packages as such object can contain anything and there is no enough information at the compile time to provide some symbolic processing of code.

Since we need flexibility of type we have:

struct MyType{T <: SomeAbstractType}
   x::T
end

or simply MyType{T} which is an equivalent of MyType{T <: Any}.
Writing code in this way makes it possible to find out the type at the compile type and hence perform various processing on code.

Google also for @code_warntype macro to find more information on code stability.

[JamieJQuinn]

Another suggestion - some good practices on writing robust Julia would be nice.

Specifically:

• How to effectively use the type system to ensure functions + structures compose safely.
• I'm not quite sure what type stability is and wouldn't mind an introduction.
• Logging & error reporting practices.