Issue #515: Add ColumnStats Schema for JSON parsing

[osopardo1]

Description

Fixes #515 .

In this PR, we are adding a way of building a columnStats Schema using the current column Transformers and the actual Schema of the Data. We want to:

• Ensure the fields are properly parsed from the JSON string.
• If a JSON string is specified, but the row is returned null, we assume the string is not following the correct syntax.

For that, I've added a QbeastColumnStats case class that contains the columnStatsSchema and the columnStatsRow. Also, a QbeastColumnStatsBuilder is needed to retrieve all the information given the parameters mentioned above.

case class QbeastColumnStats(columnStatsSchema: StructType, columnStatsRow: Row)


object QbeastColumnStatsBuilder {
  /**
   * Builds the QbeastColumnStats
   *
   * @param statsString
   *   the stats in a JSON string
   * @param columnTransformers
   *   the set of columnTransformers to build the Stats from
   * @param dataSchema
   *   the data schema to build the Stats from
   * @return
   */
  def build(
      statsString: String,
      columnTransformers: Seq[Transformer],
      dataSchema: StructType): QbeastColumnStats
}

Type of change

Bug fix.

Checklist:

Here is the list of things you should do before submitting this pull request:

• New feature / bug fix has been committed following the Contribution guide.
• Add logging to the code following the Contribution guide.
• Add comments to the code (make it easier for the community!).
• Change the documentation.
• Add tests.
• Your branch is updated to the main branch (dependent changes have been merged).

How Has This Been Tested? (Optional)

Testing different parsings on QbeastColumnStatsTestBuilder.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.50%. Comparing base (b2e2f85) to head (81ab4a2).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #522   +/-   ##
=======================================
  Coverage   88.50%   88.50%           
=======================================
  Files          21       21           
  Lines         774      774           
  Branches      115      115           
=======================================
  Hits          685      685           
  Misses         89       89           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.