Skip to content

Add a multilabel classification example (e.g. for Anchors) #697

Open
@jklaise

Description

@jklaise

We had an interesting question regarding explaining multilabel classification models. As a recap, whilst regular classification has one target of any cardinality (i.e. number of classes), multilabel classification is defined as having n>1 targets each of which has cardinality 2 (i.e. either each target is present or absent). Two examples might be object detection in images where the number of targets is the number of different possible objects known to the model or text labelling where each piece of text is assigned multiple labels from a fixed pool of labels.

Whilst alibi doesn't have direct support for these kinds of problems, it's still possible to use somewhat effectively. For example we migh:

  1. Define n models by wrapping the output of the original model so that the wrapped models only output one label, then one can explain the wrapped models individually.
  2. (@RobertSamoilescu) Define a label mapping. For example, suppose that there are 3 possible labels which can be encoded with 3 bits: [0, 0, 0], [0, 0, 1], [0, 1, 0], ..., [1, 1, 1]. We can define the following mapping: [0, 0, 0] -> 0, [0, 0, 1] -> 1, ..., [1, 1, 1] -> 2^3 - 1. Then the model becomes a regular classificier to which alibi algorithms can be applied.
  3. (@RobertSamoilescu) Instead of defining 2^n labels (as in 2.), you can also define them directly in a 1 vs the rest manner. Suppose that the label for the instance to be explained is [1, 1, 0]. Then the mapping will be [1, 1, 0] -> 1, and anything else goes to 0. The difference between the 2. and 3. is that for 3. you need to define a new mapping based on the prediction for each instance to be explained. With 2. you can use the same mapping with any

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions