Aggregation support for numeric fields in Scorch metadata

[ajroetker]

I'm wondering if the bleve team or others besides myself would also find it useful to have the scorch metadata store aggregation data for numeric fields on segment reads. This would enable the ability to use bleve end to end for tracking the clickthrough rates on certain searches. We could store documents and sample query events to get stored in bleve as well.

// Hook into segment introduction
// In index/scorch/introducer.go after creating SegmentSnapshot

func (s *Scorch) computeSegmentAggregationStats(ss *SegmentSnapshot) error {
    // For each numeric field, scan values and compute stats
    for _, field := range ss.segment.Fields() {
        if isNumericField(field) {
            stats := &NumericFieldStats{
                Sum:        0,
                Min:        math.MaxFloat64,
                Max:        -math.MaxFloat64,
                Count:      0,
                SumSquares: 0,
            }
            
            // Scan the segment's values
            dict, _ := ss.segment.Dictionary(field)
            // ... iterate and compute stats ...
            
            // Store in cachedMeta
            ss.cachedMeta.updateMeta("agg_stats_"+field, stats)
        }
    }
    return nil
}

[ajroetker]

I've decided to take a stab at adding a more general Aggregations to bleve with subaggregations and the incorporation of facetting into that so that bleve can support more of the functionality available to elasticsearch.