Compilation time degrades x3 to x4 times when consuming project targets net7.0 or net8.0 (rather than net6.0)

[DunetsNM]

I have a project with a lot of complex Fleece codecs.

Codecs have never been too fast to compile (due to heavy usage of SRTP) but after I tried to upgrade <TargetFramework> value from net6.0 to net7.0 or net8.0 it got worse.

• The project used to take 35 to 45 seconds to compile, with net7.0 it deteriorated to circa 2:30, similar number for net8.0

• It doesn't look like it's related to some specific codecs, just sheer amount of them. The more I delete/comment out the faster compilation becomes (dropping to just 15 seconds when all codecs are removed - same number for any target, net6/net7/net8 i.e. slowness is 100% codec-related).

• There were many changed parts between net6 and net7/net8 versions of my projects (e.g. different versions of FSharp.Core and FSharpPlus, newer SDK) but apparently the single cause of compile time degradation is the TargetFramework value: if I change it back to net6.0 while keeping all other changes, compilation time improves / goes back to normal.

• I tried to fork Fleece, added net7.0 and net8.0 targets to it and consumed from my project. It did not help.

The main suspect is SRTP and inlining which is heavily used in codecs. Did something change in newer F# versions that made it so much slower to compile on average?

[gusty]

Hi @DunetsNM
That's really odd, I don't see why by simply switching to net7/8 compile time would increase.
Some questions:
Do you use SRTP in codecs ? Last time I saw your codebase most codecs were explicit except a few cases.
Would it be possible to create a small repro?

@vzarytovskii do you have any idea of what could slow down in net7/8 ?

[vzarytovskii]

Yes, static abstracts introduction (both compiler feature and them being in BCL) slows it down significantly. Especially in traits heavy code. It now needs to check so many more things if they satisfy traits.

[gusty]

I see, I knew about that, but I didn't know the slow down was so bad (about 3x).

@DunetsNM if codecs in your codebase don't heavily use SRTPs (implicit codecs), it could be Fleece internals. If so, I can create a branch hardcoding some internal stuff, and therefore removing static resolution in the library.
Last time I tried that, the improvement was really marginal, but maybe the situation is different now.

[vzarytovskii]

Please create issue over at compiler repo, with build instructions, I will extract profiles for us to see what exactly changed.

[gusty]

Thanks @vzarytovskii

@DunetsNM can you contact me by FSSF Slack so we discuss what's the best approach to follow?

[DunetsNM]

thanks for quick replies guys, I got sick today, will create a smaller repro when I get better / soon.

Do you use SRTP in codecs ? Last time I saw your codebase most codecs were explicit except a few cases.

Correct, nearly everything is explicit

[DunetsNM]

Extra compilation time is pretty obvious even in a simple flat list of types with codecs - although only x2, perhaps it gets worse if there's a lot of nesting?

Here's a dummy example:

slower_net7.zip

For better readability contents of the files:

Types.fs

module Example.Types

[<RequireQualifiedAccess>]
type DummyUnionType1 =
| CaseA of string
| CaseB of string
| CaseC of string

[<RequireQualifiedAccess>]
type DummyUnionType2 =
| CaseA of string
| CaseB of string
| CaseC of string

[<RequireQualifiedAccess>]
type DummyUnionType3 =
| CaseA of string
| CaseB of string
| CaseC of string

...

[<RequireQualifiedAccess>]
type DummyUnionType10 =
| CaseA of string
| CaseB of string
| CaseC of string

type DummyRecordType = {
    X1: DummyUnionType1
    X2: DummyUnionType2
    X3: DummyUnionType3
    X4: DummyUnionType4
    X5: DummyUnionType5
    X6: DummyUnionType6
    X7: DummyUnionType7
    X8: DummyUnionType8
    X9: DummyUnionType9
    X10: DummyUnionType10
}

open Fleece
open System.Reflection
open Microsoft.FSharp.Reflection
open FSharpPlus.Data
open FSharpPlus

let inline mergeUnionCases (fn: 'Union -> '``Codec/Decoder<PropertyList<'Encoding>, 'Union>``) : '``Codec/Decoder<PropertyList<'Encoding>, 'Union>`` =
    FSharpType.GetUnionCases (typeof<'Union>, BindingFlags.NonPublic ||| BindingFlags.Public)
    |> NonEmptySeq.ofArray
    |> NonEmptySeq.map (fun c -> FSharpValue.MakeUnion (c, Array.zeroCreate (Array.length (c.GetFields ())), BindingFlags.NonPublic ||| BindingFlags.Public) :?> 'Union)
    |> NonEmptySeq.map fn
    |> choice

type DummyUnionType1 with
    static member get_Codec () =
        function
        | CaseA _ -> CaseA <!> jreqWith Codecs.string "CaseA" (function (CaseA x) -> Some x | _ -> None)
        | CaseB _ -> CaseB <!> jreqWith Codecs.string "CaseB" (function (CaseB x) -> Some x | _ -> None)
        | CaseC _ -> CaseC <!> jreqWith Codecs.string "CaseC" (function (CaseC x) -> Some x | _ -> None)
        |> mergeUnionCases
        |> ofObjCodec

type DummyUnionType2 with
    static member get_Codec () =
        function
        | CaseA _ -> CaseA <!> jreqWith Codecs.string "CaseA" (function (CaseA x) -> Some x | _ -> None)
        | CaseB _ -> CaseB <!> jreqWith Codecs.string "CaseB" (function (CaseB x) -> Some x | _ -> None)
        | CaseC _ -> CaseC <!> jreqWith Codecs.string "CaseC" (function (CaseC x) -> Some x | _ -> None)
        |> mergeUnionCases
        |> ofObjCodec

...

type DummyUnionType10 with
    static member get_Codec () =
        function
        | CaseA _ -> CaseA <!> jreqWith Codecs.string "CaseA" (function (CaseA x) -> Some x | _ -> None)
        | CaseB _ -> CaseB <!> jreqWith Codecs.string "CaseB" (function (CaseB x) -> Some x | _ -> None)
        | CaseC _ -> CaseC <!> jreqWith Codecs.string "CaseC" (function (CaseC x) -> Some x | _ -> None)
        |> mergeUnionCases
        |> ofObjCodec

type DummyRecordType with
    static member get_Codec () = 
        codec {
            let! x1 = jreqWith (DummyUnionType1.get_Codec()) "X1" (fun x -> Some x.X1)
            and! x2 = jreqWith (DummyUnionType2.get_Codec()) "X2" (fun x -> Some x.X2)
            and! x3 = jreqWith (DummyUnionType3.get_Codec()) "X3" (fun x -> Some x.X3)
            and! x4 = jreqWith (DummyUnionType4.get_Codec()) "X4" (fun x -> Some x.X4)
            and! x5 = jreqWith (DummyUnionType5.get_Codec()) "X5" (fun x -> Some x.X5)
            and! x6 = jreqWith (DummyUnionType6.get_Codec()) "X6" (fun x -> Some x.X6)
            and! x7 = jreqWith (DummyUnionType7.get_Codec()) "X7" (fun x -> Some x.X7)
            and! x8 = jreqWith (DummyUnionType8.get_Codec()) "X8" (fun x -> Some x.X8)
            and! x9 = jreqWith (DummyUnionType9.get_Codec()) "X9" (fun x -> Some x.X9)
            and! x10 = jreqWith (DummyUnionType10.get_Codec()) "X10" (fun x -> Some x.X10)
            return {
                X1 = x1
                X2 = x2
                X3 = x3
                X4 = x4
                X5 = x5
                X6 = x6
                X7 = x7
                X8 = x8
                X9 = x9
                X10 = x10
            }
        }
        |> ofObjCodec

Net6Client.fsproj

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net6.0</TargetFramework>
    </PropertyGroup>
    <ItemGroup>
		<PackageReference Include="Fleece.SystemTextJson" Version="0.10.0" />
    </ItemGroup>
    <ItemGroup>
        <Compile Include="Types.fs" />
    </ItemGroup>
</Project>

Net7Cilent.fsproj (one line diff)

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net7.0</TargetFramework>
    </PropertyGroup>
    <ItemGroup>
		<PackageReference Include="Fleece.SystemTextJson" Version="0.10.0" />
    </ItemGroup>
    <ItemGroup>
        <Compile Include="Types.fs" />
    </ItemGroup>
</Project>

global.json (SDK 7.0.300 performs similar)

{
  "sdk": {
    "version": "8.0.300",
    "rollForward": "latestMinor"
  }
}

If you run

dotnet build Net6Client.fsproj
dotnet build Net7Client.fsproj

it takes ~3.5 sec for net6 vs ~7 sec for net7 on my machine, YMMV
This example uses only explicit codecs passing, but implicit codecs didn't make any difference here (perhaps because of too simple / flat hierarchy).

[DunetsNM]

Also if you try to target Fleece.fsproj itself to net7.0 you will see proportional slowdown in compilation time.

So, if any SRTP-heavy code is affected, is there anything that can be done about it / apart from pinning to net6 ?
If, in theory, we make a fork of Fleece that uses IWSAM as a main vehicle for compile-time codec resolution, will it make the library and client code any faster to compile?

I mean something like this:

type IHaveDefaultCodec<'ImplType
    when 'ImplType :> IHaveDefaultCodec<'ImplType>> =
    static abstract member Codec<'Encoding when 'Encoding :> Fleece.IEncoding and 'Encoding : (new: unit -> 'Encoding)> : unit -> Fleece.Codec<'Encoding, 'ImplType>
    
 
type OrderDirection =
| Ascending
| Descending
with
    static member get_Codec () =
        function
        | Ascending  -> konst Ascending <!> jreqWith Codecs.unit "Ascending" (function Ascending -> Some () | _ -> None)
        | Descending -> konst Descending <!> jreqWith Codecs.unit "Descending" (function Descending -> Some () | _ -> None)
        |> mergeUnionCases
        |> ofObjCodec

    interface IHaveDefaultCodec<OrderDirection> with
         static member Codec() = OrderDirection.get_Codec ()
    

I guess it depends on where the bottleneck is. Is it emitting of the inlined code (which can't be avoided in case of SRTP , but not required for IWSAM), or computationally-heavy type resolution.

[gusty]

@DunetsNM hope you feel better today.
Thanks for the repro, I will have a look in order to find out where's the bottleneck on the code side.

Regarding IWSAMs, there's already #132 have you tried it? And there's even a nuget for it.

[gusty]

I see now, in your last code snippet you mean using IWSAMs for implicit codecs, one important limitation with that approach is retrofitting primitive types. That's not possible with IWSAMs unless you control the source code.

[DunetsNM]

@gusty yes I've played with #132 before (didn't switch because our code still requires AdHocEncoding), just double check that compilation speed difference is about the same. But it uses IWSAM only for IEncoding which is relatively small part of Fleece.

Also, you don't really need my examples, the Fleece library itself is affected:

FleeceNet6.fsproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>net6.0</TargetFrameworks>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="Fleece.fs" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="FSharpPlus" Version="1.2.4" />
  </ItemGroup>
</Project>

FleeceNet7.fsproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFrameworks>net7.0</TargetFrameworks>
  </PropertyGroup>
  <ItemGroup>
    <Compile Include="Fleece.fs" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="FSharpPlus" Version="1.2.4" />
  </ItemGroup>
</Project>

Here dotnet build FleeceNet6.fsproj and dotnet build Fleece.Net7.fsproj take ~11 sec and ~25 sec respectively for me.

Let me try the same with FSharpPlus which Fleece uses a lot ...

[DunetsNM]

So for current FSharpPlus master build time

net6.0 1:46
net7.0 2:43
net8.0: 2:47

[DunetsNM]

@vzarytovskii I had no luck reproducing slowdown of SRTP / inline with dummy vanilla code, it's only Fleece and/or FSharpPlus projects where slowdown in build time is obvious. Dissecting it further to find the bottleneck is not an obvious task to me.
Should that still be reported to F# compiler repo, or it rather belongs to those projects (as they take usage of SRTP to its own extremes / not your typical F# compiler usage) ?

Just in case here's my dumb inline-heavy SRTP code that compiles in about the same time for net6, net7 and net8 (~7 seconds in all cases)

SrtpJunk.zip

Contents of the source file:

module Example

type Record =
    { Number: int }
    static member Zero() = { Number = 0 }

let inline fmap1 (f: ^a -> ^b) (a: ^a list) =
    printfn "fmap1"
    List.map f a

let inline fmap2 f a =
    printfn "fmap2"
    fmap1 f a

let inline fmap3 f a =
    printfn "fmap3"
    fmap2 f a
    
...

let inline fmap99 f a =
    printfn "fmap99"
    fmap98 f a

let inline fmap100 f a =
    printfn "fmap100"
    fmap99 f a

let main () =
    fmap1 (fun x -> System.Int32.Parse(x)) ["1";"2";"3";"4"] |> ignore
    fmap1 (fun x -> System.Int32.Parse(x)) ["1";"2";"3";"4"] |> ignore
    fmap2 (fun x -> x.ToString()) [1;2;3;4] |> ignore
    fmap2 (fun x -> System.Int32.Parse(x)) ["1";"2";"3";"4"] |> ignore
...
    fmap100 (fun x -> x.ToString()) [1;2;3;4] |> ignore
    fmap100 (fun x -> System.Int32.Parse(x)) ["1";"2";"3";"4"] |> ignore
    0

[vzarytovskii]

Sure, still report it, but with project examples.

[gusty]

@DunetsNM thanks for all the repros.
On my side I can confirm that definitely heavy use of SRTPs becomes almost 2x slower or more as from .net7

We can help @vzarytovskii to investigate and hopefully fix whatever is causing that bottleneck in the compiler but as I pointed at the beginning I am a bit surprised since I know your codebase and AFAIK it doesn't rely that much on heavy SRTP.

However after seeing your repro I found that there is a small detail that causes an unnecessary level of genericity, but good news, it's fixable.

The problem is this line:

open FSharpPlus

Well, not that line alone, but the issue is that codecs use the <!> operator, but F#+ does an autoopen of generic operatos when you open the namespace FSharpPlus.

This is something we're planning to change in v2 of the library, to avoid precisely situations like this. It is still not clear what would be the new namespace design (suggestions are welcome).

So, it's a bit tricky right now, since the code in the end routes to the same implementation but it takes longer to compile, so you don't notice that something breaks or so. But making sure FSharpPlus is not open at the codec definition should fix the long compilation issue.

If just a few functions are needed from FSharpPlus you can simply fully qualify it, for instance in your repro you can do:

let inline mergeUnionCases (fn: 'Union -> '``Codec/Decoder<PropertyList<'Encoding>, 'Union>``) : '``Codec/Decoder<PropertyList<'Encoding>, 'Union>`` =
    FSharpType.GetUnionCases (typeof<'Union>, BindingFlags.NonPublic ||| BindingFlags.Public)
    |> NonEmptySeq.ofArray
    |> NonEmptySeq.map (fun c -> FSharpValue.MakeUnion (c, Array.zeroCreate (Array.length (c.GetFields ())), BindingFlags.NonPublic ||| BindingFlags.Public) :?> 'Union)
    |> NonEmptySeq.map fn
    |> FSharpPlus.Operators.choice

... or create a small namespace in your project aliasing these functions.

Another alternative could be doing exactly the opposite, I mean, keep opening FSharpPlus but open later another namespace which shadows the <!> operator.

As said, we'll fix this in v2 of F#+ as it will be focusing more on less generic code by default. Most likely you'll have to explicitly open a "generic operators" namespace.