Apollo-Compiler Feature Request: Schema Type Merging

[samtipton]

Description

Type merging is a highly desirable feature in gateway systems or in highly modularized graphql projects that want to present a coherent public schema.

Type merging means collecting extension objects fields and directives back into the base type for all types in the AST for non-executable documents.

This is supported in library such as graphql-java and apollo-kotlin.

Please describe in as much detail the kind of feature would you like us to add!

Add details like:

• What does it do?

Merges all extension objects fields into the base type for all types in the AST. That is, takes the fields and directives from each extension and mutates them into the type the extension is extending

type User {
    id: ID!
    name: String!
}

extend type User {
    email: String
}

After merging we would get

type User {
    id: ID!
    name: String!
    email: String
}

• What are the inputs/outputs?
  document in, document out.

• An API sketch

    Document::to_merged(&document)

    // or perhaps
    Document::parser().parse_into_merged(&schema)
    Document::parser().parse_into_merged(&schema_builder)

• Prior art (are there any other tools/products that do this now?)
  graphql-java and apollo-kotlin.

Document#toMergedGQLDocument

• What is your use case for this?
  Stitching federated subgraph schemas together for public use and not wanting extensions defined through public api contract while allowing developers freedom to split up schema definitions files

[SimonSapin]

apollo-compiler provides two different levels of abstraction:

• It looks like you’re using an AST Document whose structure is deliberately close to that of the source file. If your file has type A {…} type B {…} extend type A {…} the AST contains three definitions in that order in a Vec.
• A Schema is more abstract, and type merging is one of the main things it does. The other is storing stuff in maps indexed by name, instead of Vecs where you’d need to linear-search.

With your example, Schema::parse(input, "schema.graphql")?.types["User"].fields.len() is 3 because types is a map of merged (aka "extended") types.

[samtipton]

Hmm, I tested this and my schema isn't merged.

[SimonSapin]

Sorry, you're gonna have to be more specific. Please provide a reproducible example of code you're running together with the input schema, what you expected to see, and what you see instead.

Taking a guess in the dark: when I say that that Schema is merged I mean the Rust API acts "as if" the schema was merged. But it still keep tracks of what components come from which extensions (if at all). And when serializing back to string in GraphQL syntax, those extensions are reconstructed.

If you want to serialize without extensions you'd need to find everything in the schema that is stored as a Component and set component.origin = ComponentOrigin::Definition. You'd need .make_mut() on intermediate containers to get &mut references. There isn't an easy way to find all components, but they are:

• Directives of schema
• Directives of type definitions
• Object type field definitions
• Interface type field definitions
• Input object type field / input value definitions
• Enum type value definitions

… but I'm curious why you'd need that in the first place.

[samtipton]

Sorry for the delay here. Here's the script I'm working with. I still need to try your suggestion above.

use apollo_compiler::{ast, Schema};
use apollo_compiler::schema::SchemaBuilder;
use clap::Parser as ClapParser;
use std::fs;
use std::path::{Path, PathBuf};

#[derive(ClapParser)]
#[command(author, version, about, long_about = None)]
struct Cli {
    /// Name of the service
    #[arg(short = 'n', long)]
    service_name: Option<String>,

    /// Path to directory or file containing *.graphqls file(s)
    #[arg(short = 'p', long, required = true)]
    schema_path: PathBuf,
}

/// cargo run -- -p path/to/schema/file/or/dir
fn main() -> Result<(), Box<dyn std::error::Error>> {
    let cli = Cli::parse();

    let schema_files = if cli.schema_path.is_dir() {
         collect_schema_files_from_directory(&cli.schema_path)?
    } else {
        vec![cli.schema_path.clone()]
    };

    if schema_files.is_empty() {
        println!("No *.graphqls files found at: {}", cli.schema_path.display());
    } else {
        println!("Found {} schema file(s)", schema_files.len());
    }

    let file_count = schema_files.len();
    if file_count == 0 {
        return Err("No schema files found".into());
    }


    // if file_count == 1 {
    //     println!("{}", Schema::parse(fs::read_to_string(&schema_files[0]).unwrap(), cli.schema_path.to_string_lossy().as_ref()).unwrap().to_string());
    //     return Ok(());
    // }

    // Create a schema builder to merge schemas
    let mut schema_builder = SchemaBuilder::new();

    // Read and merge all schema files
    for file_path in &schema_files {
        println!("Reading schema from: {}", file_path.display());
        let schema_content = fs::read_to_string(file_path)?;
        ast::Document::parser().parse_into_schema_builder(&schema_content, file_path.to_string_lossy().as_ref(), &mut schema_builder);
    }

    let schema_built = schema_builder.build();

    match schema_built {
        Ok(schema) => {
            let merged_schema = Schema::parse(schema.to_string(), Path::new("merged_schema.graphql"));
            println!("Successfully validated and merged {} schema files",
                     file_count);
            println!("{}", merged_schema.unwrap().to_string());
        },
        Err(errors) => {
            println!("Schema validation errors:");
            for error in errors.errors.iter() {
                use apollo_compiler::diagnostic::ToCliReport;
                let path = error
                    .error
                    .location()
                    .and_then(|loc| error.sources.get(&loc.file_id()))
                    .map(|source| source.path());

                if let Some(path) = path {
                    println!("{}: {}", path.display(), error);
                } else {
                    println!("{}", error);
                }
            }
            return Err("Schema validation failed".into());
        }
    }

    Ok(())
}
/// Collect all *.graphqls files from a directory
fn collect_schema_files_from_directory(dir: &Path) -> Result<Vec<PathBuf>, Box<dyn std::error::Error>> {

    // Otherwise collect all *.graphqls files
    let mut schema_files = Vec::new();

    for entry in fs::read_dir(dir)? {
        let entry = entry?;
        let path = entry.path();

        if path.is_file() {
            if let Some(extension) = path.extension() {
                if extension == "graphqls" {
                    schema_files.push(path);
                }
            }
        }
        else if path.is_dir() {
            let sub_files = collect_schema_files_from_directory(&path)?;
            schema_files.extend(sub_files);
        }
    }

    Ok(schema_files)
}

input schema:

directive @tag(name: String!) repeatable on SCHEMA | SCALAR | OBJECT | FIELD_DEFINITION | ARGUMENT_DEFINITION | INTERFACE | UNION | ENUM | ENUM_VALUE | INPUT_OBJECT | INPUT_FIELD_DEFINITION

scalar EnergyLevel
extend scalar EnergyLevel @tag(name: "measured")

interface Character {
    id: ID!
    name: String!
}

type Query {
    hero(id: ID!): Hero
}

extend type Query {
    villain(id: ID!): Villain
    searchCharacters(term: String!): [Character!]
    teamAssemble(memberIds: [ID!]!): [Hero!]
    threatLevel: EnergyLevel
}

type Hero implements Character {
    id: ID!
    name: String!
    power: String!
}

extend type Hero {
    powerLevel: EnergyLevel
    sidekick: String
    secretIdentity: String
    teamAffiliation: [String!]
}

type Villain implements Character {
    id: ID!
    name: String!
    weakness: String!
}

union AntiHero = Hero | Villain

extend union AntiHero = Sidekick | Vigilante

type Sidekick implements Character {
    id: ID!
    name: String!
    hero: Hero!
    specialSkill: String
}

type Vigilante implements Character {
    id: ID!
    name: String!
    operatesInCity: String
    wantsRevenge: Boolean
}

enum Element {
    AIR,
    FIRE,
    WATER,
    EARTH,
    LIGHT
}

extend enum Element {
    DARKNESS
    ELECTRICITY
    ICE
    METAL
}

input OriginStoryInput {
    fullName: String!
    weakness: Element
    homePlanet: String!
    nemesis: String!
}

extend input OriginStoryInput {
    superpowerOrigin: String
    tragicBackstory: Boolean
    trainingMontage: Boolean
}

extend type Villain {
    evilPlan: String
    hideoutLocation: String
    minions: Int
}

extend interface Character {
    origin: String
    firstAppearance: String
    allies: [String!]
}

and after type merging, I'd expect it to look like:

directive @tag(name: String!) repeatable on SCHEMA | SCALAR | OBJECT | FIELD_DEFINITION | ARGUMENT_DEFINITION | INTERFACE | UNION | ENUM | ENUM_VALUE | INPUT_OBJECT | INPUT_FIELD_DEFINITION

scalar EnergyLevel
extend scalar EnergyLevel @tag(name: "measured")

interface Character {
    id: ID!
    name: String!
    origin: String
    firstAppearance: String
    allies: [String!]
}

type Query {
    hero(id: ID!): Hero
    villain(id: ID!): Villain
    searchCharacters(term: String!): [Character!]
    teamAssemble(memberIds: [ID!]!): [Hero!]
    threatLevel: EnergyLevel
}

type Hero implements Character {
    id: ID!
    name: String!
    power: String!
    powerLevel: EnergyLevel
    sidekick: String
    secretIdentity: String
    teamAffiliation: [String!]
}

type Villain implements Character {
    id: ID!
    name: String!
    weakness: String!
    evilPlan: String
    hideoutLocation: String
    minions: Int
}

type Sidekick implements Character {
    id: ID!
    name: String!
    hero: Hero!
    specialSkill: String
}

type Vigilante implements Character {
    id: ID!
    name: String!
    operatesInCity: String
    wantsRevenge: Boolean
}

union AntiHero = Hero | Villain | Sidekick | Vigilante

enum Element {
    AIR,
    FIRE,
    WATER,
    EARTH,
    LIGHT
    DARKNESS
    ELECTRICITY
    ICE
    METAL
}

input OriginStoryInput {
    fullName: String!
    weakness: Element
    homePlanet: String!
    nemesis: String!
    superpowerOrigin: String
    tragicBackstory: Boolean
    trainingMontage: Boolean
}

… but I'm curious why you'd need that in the first place.

I'm building a tool that sits in a federation graphql subgraph publish pipeline that has the job of linting and validating the schema, in addition to tagging fields for public graphql contract variants.

As part of this process, I want to present the public supergraph schema without extended types to simplify comprehension while allowing developers to write their schemas in whatever manner they see fit to simplify development and maintenance.