solver should complain if it encounters conflicting names instead proceeding to an impossible, exhaustive search

[o1lo01ol1o]

I have a database of 58 tables that appears to make beam-migrate choke:

creationCommands' = Pg.simpleSchema
  (Pgm.backendActionProvider Pgm.migrationBackend)
  (Pgm.defaultMigratableDbSettings :: Pg.CheckedDatabaseSettings
      Pg.Postgres
      FoodDb
  )

main = print $ fmap (pgRenderSyntaxScript . fromPgCommand) <$> creationCommands'

doesn't seem to terminate.

I've read the manual section on the solver and I see that the it is supposed to terminate. If I compile with -fllvm and -O2 memory consumption is better, but it currently I'm at about 201gb of memory usage with -N6.

Is there a way of addressing this? I've commented out tables from the database and tried running the above: There's no reliable way (or relation that I can see) to cause the the blowup. In one case, 16 tables are fine, and one additional table cause non-termination.

[tathougies]

It'll terminate eventually, but it's incredibly inefficient yes.

Usually, in the event that it doesn't terminate quickly, the eventual result is going to be that it could not find any sequence of commands. The issue is that it doesn't yet understand that the ordering of commands doesn't quite matter (figuring out how to make that work without losing generality), so if you have 58 tables, then that's 58! things it's going to try, which is why it's taking so long, before deciding it can't do it.

Without sharing the schema I'm not sure I can pinpoint the issue. Would you be able to share an anonymized version? I would be happy to look at it over a private messaging channel if that's better for you.

[o1lo01ol1o]

With -fllvm it terminates without a solution after an hour or so (~201gb).

Would you be able to share an anonymized version? I would be happy to look at it over a private messaging channel if that's better for you.

I can anonymize it and drop it in a gist.

Before I do, are the predicates for a "valid" solution documented anywhere? Given that I'm able to write the the CRUD operations on this schema, I'm not totally sure where there could be a problem. But of course could be user errors in my implementation.

[o1lo01ol1o]

I've managed to get a two-table case where the solver fails for me. Is there some issue with how things are being named?

data Foo_BazBarT f = Foo_BazBar
  { bazBar_id :: C f Int64
  , bazBar_unparsed           :: C f (Maybe Text)
  , bazBar_first              :: C f (Maybe Text)
  , bazBar_middle             ::C f (Maybe Text)
  , bazBar_last               ::C f (Maybe Text)
  , bazBar_prefix             ::C f (Maybe Text)
  , bazBar_generationalSuffix ::C f (Maybe Text)
  , bazBar_salutationalSuffix ::C f (Maybe Text)
  , bazBar_maternal           ::C f (Maybe Text)
  }
    deriving stock (Generic)
    deriving anyclass (Beamable)

instance Table Foo_BazBarT where
    newtype PrimaryKey Foo_BazBarT f = Foo_BazBarId (C f Int64)
        deriving stock (Generic)
        deriving anyclass (Beamable)

    primaryKey = coerce . bazBar_id

type Foo_BazBar = Foo_BazBarT Identity
type Foo_BazBarId = PrimaryKey Foo_BazBarT Identity

deriving instance Show Foo_BazBar
deriving instance Show Foo_BazBarId

data Foo_BarQualifier = 
    Quux
    | Quoad
    | Quoadd
    deriving (Show, Eq, Generic, Read)

data Foo_BarT f = Foo_Bar
  { bar_id :: C f Int64
  , bar_qualifier       :: C f (Maybe Foo_BarQualifier )
  , bar_unparsed        :: C f (Maybe Text)
  , bar_searchIndicator :: C f (Maybe Int)
  , bar_BazBar      :: PrimaryKey Foo_BazBarT (Nullable f)
  }
    deriving stock (Generic)
    deriving anyclass (Beamable)

instance Table Foo_BarT where
    newtype PrimaryKey Foo_BarT f = Foo_BarId (C f Int64)
        deriving stock (Generic)
        deriving anyclass (Beamable)

    primaryKey = coerce . bar_id

type Foo_Bar = Foo_BarT Identity
type Foo_BarId = PrimaryKey Foo_BarT Identity

deriving instance Show (PrimaryKey Foo_BazBarT (Nullable Identity))
deriving instance Show Foo_Bar
deriving instance Show Foo_BarId


data FooDb f = FooDb {
    bar :: f (TableEntity Foo_BarT)
    ,  bazBar :: f (TableEntity Foo_BazBarT) 
    }deriving stock (Generic)
    deriving anyclass (Database be)

FooDb :: DatabaseSettings be FooDb
FooDb = defaultDbSettings

[o1lo01ol1o]

@tathougies Does anything in the above look suspicious to you? (Apart from the metasyntactic noise introduced by the renaming . . . )

[tathougies]

Are you running this on postgres, or something else? What's the SQL type for Foo_BarQualifier? That's probably where it's getting hung up.

[o1lo01ol1o]

Postgres:

instance HasSqlValueSyntax be String => HasSqlValueSyntax be Foo_BarQualifier where
    sqlValueSyntax = autoSqlValueSyntax
instance FromBackendRow Postgres Foo_BarQualifier where
    fromBackendRow = read . T.unpack <$> fromBackendRow

instance (BeamSqlBackend be, Pgm.BeamMigrateSqlBackend be ) 
    => Pgm.HasDefaultSqlDataType be Foo_BarQualifier where 
    defaultSqlDataType _ _ _ = varCharType Nothing Nothing 

[tathougies]

BTW, The predicates for a valid solution are mainly in http://hackage.haskell.org/package/beam-migrate-0.4.0.1/docs/Database-Beam-Migrate-Actions.html

Beam-postgres has a few more, but you don't seem to be doing anything postgres-specific here.

I will investigate.

[o1lo01ol1o]

I solved this issue by prefixing all the table names:

data FooDb f = FooDb {
      preBar :: f (TableEntity Foo_BarT)
    ,  preBazBar :: f (TableEntity Foo_BazBarT) 
    }deriving stock (Generic)
    deriving anyclass (Database be)

I suppose the solver was overwriting values of tables / tables and columns after applied some naming rule?

[tathougies]

I suppose the solver was overwriting values of tables / tables and columns after applied some naming rule?

Oh my... yes you're right, and I can't believe I didn't see it.

That being said, yes, the original database definition you gave with bar and bazBar mean that there are two tables named bar. This naming oddity is documented here: http://tathougies.github.io/beam/user-guide/models/#defaults

But I suppose it should complain earlier.