Make fortran toolchain optional and expose binary_prefix, include_prefix and sysroot

[JSGette]

We are using crosstool-ng to create toolchains. Unfortunately, their layout isn't fully compatible with this repository and we would really like to use gcc-toolchain and just inject our toolchains into AVAILABLE_GCC_VERSIONS. I already made your project work with crosstool-ng and could potentially file a PR myself. But before I do that I wanted to ask your opinion on that matter. As the title says I need:

• to make fortran bits optional as our toolchain produced by crosstool-ng doesn't contain those, hence, build reports missing inputs (filegroups are hardcoded)
• to expose binary_prefix, because in our case it's not just gcc, but x86_64_unknown-linux-gnu-gcc. And in your current bzlmod work it's not added to ATTRS_SHARED_WITH_MODULE_EXTENSION
• to expose include_prefix, because, again, in our case it is {ARCH}_unknown-linux-gnu and not x86_64-linux/
• make sysroot configurable, because it is again different.

The implementation would be something like:

• Adding a new attribute enable_fortran or something similar and then only inject those bits in defs.bzl if this attribute is set to True
• To treat fortran in BUILD file we could move it out into BUILD.bazel.tpl and use rctx.template to then substitute certain filegroups. This way if enable_fortran is set to True we will inject all relevant search patterns into filegroups, not otherwise.
• Add binary_prefix to ATTRS_SHARED_WITH_MODULE_EXTENSION. This way it becomes configurable
• Make include_prefix an attribute and also add to ATTRS_SHARED_WITH_MODULE_EXTENSION
• Instead of adding sysroot to c_builtin_includes and doing the same in extra_ldflags we also make it an attribute and pass that value accordingly, default value remains the same as it is in current implementation.
• Make AVAILABLE_GCC_VERSIONS configurable as well. The default value will come directly from gcc-toolchain for those who are fine to use one of the versions that are supported, but if someone (us) wants to use something custom, there should be a way to overwrite or add more versions

That's how my changes look like that I would use as basis:

diff --git a/toolchain/cc_toolchain_config.bzl b/toolchain/cc_toolchain_config.bzl
index 3ad569b..5e90d8f 100644
--- a/toolchain/cc_toolchain_config.bzl
+++ b/toolchain/cc_toolchain_config.bzl
@@ -29,7 +29,6 @@ load(
     "with_feature_set",
 )
 load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES")
-load("//toolchain/fortran:action_names.bzl", FORTRAN_ACTION_NAMES = "ACTION_NAMES")
 
 all_compile_actions = [
     ACTION_NAMES.c_compile,
@@ -77,7 +76,6 @@ all_link_actions = [
     ACTION_NAMES.cpp_link_executable,
     ACTION_NAMES.cpp_link_dynamic_library,
     ACTION_NAMES.cpp_link_nodeps_dynamic_library,
-    FORTRAN_ACTION_NAMES.fortran_link_executable,
 ]
 
 lto_index_actions = [
@@ -91,7 +89,6 @@ def _impl(ctx):
     tool_paths = ctx.attr.tool_paths
     extra_cflags = ctx.attr.extra_cflags
     extra_cxxflags = ctx.attr.extra_cxxflags
-    extra_fflags = ctx.attr.extra_fflags
     extra_ldflags = ctx.attr.extra_ldflags
 
     action_configs = []
@@ -153,7 +150,7 @@ def _impl(ctx):
         enabled = True,
         flag_sets = [
             flag_set(
-                actions = all_compile_actions + [FORTRAN_ACTION_NAMES.fortran_compile],
+                actions = all_compile_actions,
                 flag_groups = [
                     flag_group(
                         flags = [
@@ -172,57 +169,12 @@ def _impl(ctx):
         enabled = True,
     )
 
-    fortran_compile_flags_feature = feature(
-        name = "fortran_compile_flags",
-        enabled = True,
-    )
-
-    static_libgfortran_feature = feature(name = "static_libgfortran")
-
-    fortran_link_flags_feature = feature(
-        name = "fortran_link_flags",
-        enabled = True,
-        flag_sets = [
-            flag_set(
-                actions = [FORTRAN_ACTION_NAMES.fortran_link_executable],
-                flag_groups = [
-                    flag_group(
-                        flags = ["-static-libgfortran"],
-                    ),
-                ],
-                with_features = [
-                    with_feature_set(
-                        features = ["static_libgfortran"],
-                    ),
-                ],
-            ),
-        ],
-    )
-
-    action_configs.append(action_config(
-        action_name = FORTRAN_ACTION_NAMES.fortran_compile,
-        enabled = True,
-        tools = [tool(path = tool_paths.get("gfortran"))],
-    ))
-
-    action_configs.append(action_config(
-        action_name = FORTRAN_ACTION_NAMES.fortran_link_executable,
-        enabled = True,
-        tools = [tool(path = tool_paths.get("gfortran"))],
-    ))
-
-    action_configs.append(action_config(
-        action_name = FORTRAN_ACTION_NAMES.fortran_archive,
-        enabled = True,
-        tools = [tool(path = tool_paths.get("ar"))],
-    ))
-
     default_compile_flags_feature = feature(
         name = "default_compile_flags",
         enabled = True,
         flag_sets = [
             flag_set(
-                actions = all_cpp_compile_actions + [FORTRAN_ACTION_NAMES.fortran_compile],
+                actions = all_cpp_compile_actions,
                 flag_groups = [
                     flag_group(
                         flags = [
@@ -236,7 +188,7 @@ def _impl(ctx):
                 ],
             ),
             flag_set(
-                actions = all_cpp_compile_actions + [FORTRAN_ACTION_NAMES.fortran_compile],
+                actions = all_cpp_compile_actions,
                 flag_groups = [
                     flag_group(
                         flags = [
@@ -248,12 +200,12 @@ def _impl(ctx):
                 with_features = [with_feature_set(features = ["opt"])],
             ),
             flag_set(
-                actions = all_cpp_compile_actions + [FORTRAN_ACTION_NAMES.fortran_compile],
+                actions = all_cpp_compile_actions,
                 flag_groups = [flag_group(flags = ["-g"])],
                 with_features = [with_feature_set(features = ["dbg"])],
             ),
             flag_set(
-                actions = all_cpp_compile_actions + [FORTRAN_ACTION_NAMES.fortran_compile],
+                actions = all_cpp_compile_actions,
                 flag_groups = [
                     flag_group(
                         flags = [
@@ -405,7 +357,7 @@ def _impl(ctx):
         enabled = True,
         flag_sets = [
             flag_set(
-                actions = all_compile_actions + [FORTRAN_ACTION_NAMES.fortran_compile],
+                actions = all_compile_actions,
                 flag_groups = [
                     flag_group(
                         flags = ["%{user_compile_flags}"],
@@ -439,17 +391,6 @@ def _impl(ctx):
         ] if len(extra_cxxflags) > 0 else [],
     )
 
-    extra_fflags_feature = feature(
-        name = "extra_fflags",
-        enabled = True,
-        flag_sets = [
-            flag_set(
-                actions = [FORTRAN_ACTION_NAMES.fortran_compile],
-                flag_groups = [flag_group(flags = extra_fflags)],
-            ),
-        ] if len(extra_fflags) > 0 else [],
-    )
-
     extra_ldflags_feature = feature(
         name = "extra_ldflags",
         enabled = True,
@@ -476,7 +417,6 @@ def _impl(ctx):
                     ACTION_NAMES.cpp_module_codegen,
                     ACTION_NAMES.lto_backend,
                     ACTION_NAMES.clif_match,
-                    FORTRAN_ACTION_NAMES.fortran_compile,
                 ] + all_link_actions + lto_index_actions,
                 flag_groups = [
                     flag_group(
@@ -489,9 +429,6 @@ def _impl(ctx):
     )
 
     features = sanitizers_features + [
-        fortran_compile_flags_feature,
-        static_libgfortran_feature,
-        fortran_link_flags_feature,
         default_compile_flags_feature,
         include_paths_feature,
         library_search_directories_feature,
@@ -507,7 +444,6 @@ def _impl(ctx):
         redacted_dates_feature,
         extra_cflags_feature,
         extra_cxxflags_feature,
-        extra_fflags_feature,
         extra_ldflags_feature,
     ]
 
@@ -541,7 +477,6 @@ cc_toolchain_config = rule(
         "cxx_builtin_include_directories": attr.string_list(mandatory = True),
         "extra_cflags": attr.string_list(mandatory = True),
         "extra_cxxflags": attr.string_list(mandatory = True),
-        "extra_fflags": attr.string_list(mandatory = True),
         "extra_ldflags": attr.string_list(mandatory = True),
         "tool_paths": attr.string_dict(mandatory = True),
     },
@@ -557,7 +492,7 @@ def _sanitizer_feature(sanitizer):
         name = sanitizer.name,
         flag_sets = [
             flag_set(
-                actions = all_compile_actions + [FORTRAN_ACTION_NAMES.fortran_compile],
+                actions = all_compile_actions,
                 flag_groups = [
                     flag_group(
                         flags = sanitizer.cflags,
diff --git a/toolchain/defs.bzl b/toolchain/defs.bzl
index 00ef405..ecb796b 100644
--- a/toolchain/defs.bzl
+++ b/toolchain/defs.bzl
@@ -32,6 +32,7 @@ def _gcc_toolchain_impl(rctx):
     absolute_toolchain_root = str(rctx.path("."))
     execroot = paths.normalize(paths.join(absolute_toolchain_root, "..", ".."))
     toolchain_root = paths.relativize(absolute_toolchain_root, execroot)
+    rctx.symlink("{}/{}-unknown-linux-gnu/bin".format(toolchain_root, rctx.attr.target_arch), "{}/xbin".format(toolchain_root))
 
     def _format_flags(flags):
         return [
@@ -58,7 +59,7 @@ def _gcc_toolchain_impl(rctx):
     elif target_arch == ARCHS.armv7:
         include_prefix = "arm-linux-gnueabihf/"
     elif target_arch == ARCHS.x86_64:
-        include_prefix = "x86_64-linux/"
+        include_prefix = "x86_64-unknown-linux-gnu/"
 
     c_builtin_includes = [
         include.format(
@@ -71,7 +72,9 @@ def _gcc_toolchain_impl(rctx):
         ] + ([
             "%workspace%/{include_prefix}include",
         ] if target_arch != ARCHS.x86_64 else []) + [
-            "%workspace%/sysroot/usr/include",
+            #"%workspace%/sysroot/usr/include",
+        ] + [
+            "%workspace%/{include_prefix}sysroot/usr/include",
         ]
     ]
 
@@ -101,16 +104,6 @@ def _gcc_toolchain_impl(rctx):
             ]
         ])
 
-    f_builtin_includes = [
-        include.format(
-            gcc_version = rctx.attr.gcc_version,
-            include_prefix = include_prefix,
-        )
-        for include in [
-            "%workspace%/lib/gcc/{include_prefix}{gcc_version}/finclude",
-        ]
-    ]
-
     target_compatible_with = [
         v.format(target_arch = target_arch)
         for v in rctx.attr.target_compatible_with
@@ -124,7 +117,6 @@ def _gcc_toolchain_impl(rctx):
     builtin_include_directories = []
     builtin_include_directories.extend(c_builtin_includes)
     builtin_include_directories.extend(cxx_builtin_includes)
-    builtin_include_directories.extend(f_builtin_includes)
     builtin_include_directories.extend(rctx.attr.includes)
     builtin_include_directories.extend(rctx.attr.fincludes)
 
@@ -163,25 +155,6 @@ def _gcc_toolchain_impl(rctx):
     ])
     extra_cxxflags.extend(rctx.attr.extra_cxxflags)
 
-    extra_fflags = [
-        "-nostdinc",
-        "-B%workspace%/bin",
-        "-B%workspace%/xbin",
-    ]
-    extra_fflags.extend([
-        "-I{}".format(include)
-        for include in f_builtin_includes
-    ])
-    extra_fflags.extend([
-        "-I{}".format(include)
-        for include in c_builtin_includes
-    ])
-    extra_fflags.extend([
-        "-I{}".format(finclude)
-        for finclude in rctx.attr.fincludes
-    ])
-    extra_fflags.extend(rctx.attr.extra_fflags)
-
     extra_ldflags = [
         lib.format(
             include_prefix = include_prefix,
@@ -193,14 +166,14 @@ def _gcc_toolchain_impl(rctx):
             "-B%workspace%/{include_prefix}lib",
             "-B%workspace%/lib64",
             "-B%workspace%/{include_prefix}lib64",
-            "-B%workspace%/sysroot/lib",
-            "-B%workspace%/sysroot/usr/lib",
+            "-B%workspace%/{include_prefix}sysroot/lib",
+            "-B%workspace%/{include_prefix}sysroot/usr/lib",
             "-L%workspace%/lib",
             "-L%workspace%/{include_prefix}lib",
             "-L%workspace%/lib64",
             "-L%workspace%/{include_prefix}lib64",
-            "-L%workspace%/sysroot/lib",
-            "-L%workspace%/sysroot/usr/lib",
+            "-L%workspace%/{include_prefix}sysroot/lib",
+            "-L%workspace%/{include_prefix}sysroot/usr/lib",
         ]
     ]
     extra_ldflags.extend(rctx.attr.extra_ldflags)
@@ -218,65 +191,20 @@ def _gcc_toolchain_impl(rctx):
         # Flags
         extra_cflags = _format_flags(extra_cflags),
         extra_cxxflags = _format_flags(extra_cxxflags),
-        extra_fflags = _format_flags(extra_fflags),
         extra_ldflags = _format_flags(extra_ldflags),
     ))
 
 AVAILABLE_GCC_VERSIONS = {
-    "12.5.0": {
-        "aarch64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-12.5.0-aarch64.tar.xz",
-            "sha256": "7b0e25133a98d44b648a925ba11f64a3adc470e87668af80ce2c3af389ebe9be",
-        },
-        "armv7": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-12.5.0-armv7.tar.xz",
-            "sha256": "a0ef76c8cc517b3d76dd2f09b1a371975b2ff1082e2f9372ed79af01b9292934",
-        },
+    "11.4.0": {
         "x86_64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-12.5.0-x86_64.tar.xz",
-            "sha256": "51076e175839b434bb2dc0006c0096916df585e8c44666d35b0e3ce821d535db",
+            "url": "https://dummy.url/agent_toolchain_x86.tar.gz",
+            "sha256": "839bb0a4c584df6f1f1de289f4518af85e6fb2fcb324bbf1b84f8d863ea51cd7",
         },
     },
-    "13.4.0": {
+    "12.3.0": {
         "aarch64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-13.4.0-aarch64.tar.xz",
-            "sha256": "770cf6bf62bdf78763de526d3a9f5cae4c19f1a3aca0ef8f18b05f1a46d1ffaf",
-        },
-        "armv7": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-13.4.0-armv7.tar.xz",
-            "sha256": "1b2739b5003c5a3f0ab7c4cc7fb95cc99c0e933982512de7255c2bd9ced757ad",
-        },
-        "x86_64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-13.4.0-x86_64.tar.xz",
-            "sha256": "d96071c1b98499afd7b7b56ebd69ad414020edf66e982004acffe7df8aaf7e02",
-        },
-    },
-    "14.3.0": {
-        "aarch64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-14.3.0-aarch64.tar.xz",
-            "sha256": "74b1f0072769f8865b62897ab962f6fce174115dab2e6596765bb4e700ffe0d1",
-        },
-        "armv7": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-14.3.0-armv7.tar.xz",
-            "sha256": "0c20a130f424ce83dd4eb2a4ec8fbcd0c0ddc5f42f0b4660bcd0108cb8c0fb21",
-        },
-        "x86_64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-14.3.0-x86_64.tar.xz",
-            "sha256": "0b365e5da451f5c7adc594f967885d7181ff6d187d6089a4bcf36f954bf3ccf9",
-        },
-    },
-    "15.2.0": {
-        "aarch64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-15.2.0-aarch64.tar.xz",
-            "sha256": "e1ae45038d350b297bea4ac10f095a98e2218971a8a37b8ab95f3faad2ec69f8",
-        },
-        "armv7": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-15.2.0-armv7.tar.xz",
-            "sha256": "fda64b3ee1c3d7ddcb28378a1b131eadc5d3e3ff1cfab2aab71da7a3f899b601",
-        },
-        "x86_64": {
-            "url": "https://github.com/f0rmiga/gcc-builds/releases/download/18082025/gcc-toolchain-15.2.0-x86_64.tar.xz",
-            "sha256": "50dd28021365e7443853d5e77bc94ab1d1c947ad48fd91cbec44dbdfa61412c9",
+            "url": "https://dummy.url/agent_toolchain_aarch64.tar.gz",
+            "sha256": "dummy",
         },
     },
 }
@@ -296,10 +224,6 @@ _FEATURE_ATTRS = {
         doc = "Extra flags for compiling C++.",
         default = [],
     ),
-    "extra_fflags": attr.string_list(
-        doc = "Extra flags for compiling Fortran.",
-        default = [],
-    ),
     "extra_ldflags": attr.string_list(
         doc = "Extra flags for linking." +
               " %workspace% is rendered to the toolchain root path." +
@@ -365,7 +289,7 @@ gcc_toolchain = repository_rule(
 
 ATTRS_SHARED_WITH_MODULE_EXTENSION = {
     attr_name: _FEATURE_ATTRS[attr_name]
-    for attr_name in ["gcc_version", "gcc_versions", "extra_cflags", "extra_cxxflags", "extra_ldflags", "extra_fflags"]
+    for attr_name in ["gcc_version", "gcc_versions", "extra_cflags", "extra_cxxflags", "extra_ldflags", "binary_prefix"]
 }
 
 def _render_tool_paths(rctx, path_prefix, binary_prefix):
@@ -394,10 +318,6 @@ def _render_tool_paths(rctx, path_prefix, binary_prefix):
             path_prefix = path_prefix,
             binary_prefix = binary_prefix,
         ),
-        "gfortran": "{path_prefix}/bin/{binary_prefix}gfortran".format(
-            path_prefix = path_prefix,
-            binary_prefix = binary_prefix,
-        ),
         "ld": "{path_prefix}/bin/{binary_prefix}ld".format(
             path_prefix = path_prefix,
             binary_prefix = binary_prefix,
@@ -477,7 +397,6 @@ def gcc_declare_toolchain(
         binary_prefix = binary_prefix,
         extra_cflags = kwargs.pop("extra_cflags", []),
         extra_cxxflags = kwargs.pop("extra_cxxflags", []),
-        extra_fflags = kwargs.pop("extra_fflags", []),
         extra_ldflags = kwargs.pop("extra_ldflags", []),
         includes = kwargs.pop("includes", []),
         fincludes = kwargs.pop("fincludes", []),
@@ -509,27 +428,10 @@ ARCHS = struct(
 _TOOLCHAIN_BUILD_FILE_CONTENT = """\
 load("@rules_cc//cc:defs.bzl", "cc_toolchain")
 load("@{gcc_toolchain_workspace_name}//toolchain:cc_toolchain_config.bzl", "cc_toolchain_config")
-load("@{gcc_toolchain_workspace_name}//toolchain/fortran:defs.bzl", "fortran_toolchain")
 load("//:tool_paths.bzl", "tool_paths")
 
 package(default_visibility = ["//visibility:public"])
 
-toolchain(
-    name = "fortran_toolchain",
-    exec_compatible_with = [
-        "@platforms//os:linux",
-        "@platforms//cpu:x86_64",
-    ],
-    target_compatible_with = {target_compatible_with},
-    toolchain = ":_fortran_toolchain",
-    toolchain_type = "@{gcc_toolchain_workspace_name}//toolchain/fortran:toolchain_type",
-)
-
-fortran_toolchain(
-    name = "_fortran_toolchain",
-    cc_toolchain = ":_cc_toolchain",
-)
-
 toolchain(
     name = "cc_toolchain",
     exec_compatible_with = [
@@ -563,7 +465,6 @@ cc_toolchain_config(
     cxx_builtin_include_directories = {cxx_builtin_include_directories},
     extra_cflags = {extra_cflags},
     extra_cxxflags = {extra_cxxflags},
-    extra_fflags = {extra_fflags},
     extra_ldflags = {extra_ldflags},
     tool_paths = tool_paths,
 )
@@ -620,16 +521,14 @@ filegroup(
         "lib/gcc/{include_prefix}*/include/**",
         "lib/gcc/{include_prefix}*/include-fixed/**",
         "{include_prefix}include/**",
-        "sysroot/usr/include/**",
+        "{include_prefix}sysroot/usr/include/**",
+        "{include_prefix}sysroot/usr/include/linux/**",
 
         # C++ includes
         "{include_prefix}include/c++/*/**",
         "include/c++/*/**",
         "{include_prefix}include/c++/*/backward/**",
         "include/c++/*/backward/**",
-
-        # Fortran includes
-        "lib/gcc/{include_prefix}*/finclude/**",
     ], allow_empty=True),
     visibility = ["//visibility:public"],
 )
@@ -657,11 +556,9 @@ filegroup(
         "bin/{binary_prefix}cpp",
         "bin/{binary_prefix}g++",
         "bin/{binary_prefix}gcc",
-        "bin/{binary_prefix}gfortran",
         "xbin/cpp",
         "xbin/g++",
         "xbin/gcc",
-        "xbin/gfortran",
     ] + glob([
         "**/libexec/gcc/**/cc1plus",
         "**/libexec/gcc/**/cc1",
@@ -670,9 +567,6 @@ filegroup(
         "lib/libgmp.so*",
         "lib/libmpc.so*",
         "lib/libmpfr.so*",
-        # Fortran spec files.
-        "**/lib*/libgfortran.spec",
-        "**/lib*/libgomp.spec",
     ], allow_empty=True),
     visibility = ["//visibility:public"],
 )
diff --git a/toolchain/module_extensions.bzl b/toolchain/module_extensions.bzl
index 1eb37ab..b6b86d0 100644
--- a/toolchain/module_extensions.bzl
+++ b/toolchain/module_extensions.bzl
@@ -15,10 +15,10 @@ def _gcc_register_toolchain_module_extension(mctx):
                 target_arch = declare.target_arch,
                 gcc_version = declare.gcc_version,
                 gcc_versions = declare.gcc_versions,
+                binary_prefix = declare.binary_prefix,
                 extra_cflags = declare.extra_cflags,
                 extra_cxxflags = declare.extra_cxxflags,
                 extra_ldflags = declare.extra_ldflags,
-                extra_fflags = declare.extra_fflags,
             )
 
     # Since we know that for each gcc toolchain repository we'll generate the same files, we mark the rule as reproducible.

[f0rmiga]

I'm welcome to a PR with your proposal. Once you open the PR, we can iterate from there.

[rdesgroppes]

I'm welcome to a PR with your proposal. Once you open the PR, we can iterate from there.

I filed a first increment:

• feat: make Fortran support optional #218.