Question on sum(w)

[OlivierBondu]

While discussing GridIn with @swertz we ran into this log:

Sum of event weight: 17010

Job done in 3617.79s
15350 processed events, 39 selected.

(full log at /storage/data/cms/store/user/swertz/MuonEG/MuonEG_Run2015C-PromptReco-v1_2015-09-04/150922_085937/0000/log/cmsRun_1.log.tar.gz)

given this is data I am not sure I understand why sum(w) != nprocessed... from #22 I expect they should be the exact same, but since recently the met filter is run before the producer, could this be the reason ?

also, from the log it seems event enters second in the scheduling sequence, which could be a possible reason:

producers:
 -> Adding producer 'electrons' of type 'electrons'
 -> Adding producer 'event' of type 'event'
 -> Adding producer 'fat_jets' of type 'fat_jets'
 -> Adding producer 'hlt' of type 'hlt'
 -> Adding producer 'jets' of type 'jets'
 -> Adding producer 'met' of type 'met'
 -> Adding producer 'muons' of type 'muons'
 -> Adding producer 'nohf_met' of type 'met'
 -> Adding producer 'vertices' of type 'vertices'

[blinkseb]

It's normal for the sum of event weight to be different of the number of processed event.

The number of processed event is the number of event going though the analyze function. An event belonging to no category before the analyzers won't be counted as processed, but still enter the sum of event weight. In this case, it means that:

• 17010 events have been read from the dataset
• 15350 events have at least one event_is_in_category_pre_analyzers
• 39 events have at least one event_is_in_category_post_analyzers

[OlivierBondu]

ok, I see, in that case we need a fix: the sum(w) is supposed to be running on all events 'good for physics' (ie not rejected from filters throwing away crazy met or buggy mc events)....
otherwise the MC normalization will depend on how tight your preselection is

but in any I am still not sure I understand all the code then: I understood the sum(w) is taken care of by the event producer, this is run before the analyzers ?

[blinkseb]

Yes indeed, filters are ran before the producers and the event weight will be incorrect if some event are dropped. I need to find a correct way to handle that, since we also have some filters running directly inside CMSSW.

The sum is indeed computed by the event producer, which is run, like all producers, before the analyzers and categories. If you forget filters, you are assured that the event producer is called on every events.

[OlivierBondu]

ok, so one thing we may have to be careful of is that filters are really only filtering out 'buggy' events (in case some day we have a preselection filter...)