-
Notifications
You must be signed in to change notification settings - Fork 407
Individual feature importance interpret QA - part1 #2167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
66 commits
Select commit
Hold shift + click to select a range
635f3ec
add individual feature importance for QA
JarvisG495 99dbafe
make QA UI compatible with text classification
JarvisG495 ee60172
fix left textbox data
JarvisG495 05ef8a1
add isAllDataCohort (#2094)
RubyZ10 ba39022
retain selections (#2096)
RubyZ10 c7a04ad
Adding Class Name To OD Dropdown (#2098)
natalie-isak 8775519
Update erroranalysis-dashboard-regression-superconductor.ipynb (#2102)
tongyu-microsoft dc8aaf9
Port over logic of `load_adult_income_dataset` from `dice-ml` to unbl…
gaugup ff55ea2
refactor cohortlist (#2103)
RubyZ10 efc048e
Fix dimension mismatch error when dropped_features contains cat_featu…
tongyu-microsoft d80f8f0
Add more dataset utilities in `rai_test_utils` (#2101)
gaugup 36b48fe
release erroranalysis 0.4.4 (#2105)
imatiach-msft 4fe5a4d
update responsibleai and raiwidgets to erroranalysis 0.4.4 (#2107)
imatiach-msft 3d23ef0
enable confusion matrix for binary and multiclass text and vision RAI…
imatiach-msft e43a9d3
release raiwidgets and responsibleai 0.28.0 (#2109)
imatiach-msft f3150b1
Release `rai-test-utils` 0.3.0 (#2110)
gaugup 5259e3d
Update rai_test_utils in erroranalysis and responsibleai to 0.3.0 (#2…
gaugup ba40273
update econml to latest 0.14.1 release (#2112)
imatiach-msft 12ed667
causal fix (#2113)
vinuthakaranth 3609179
[Image Explorer] CanvasTools Image Loading support for Object Detecti…
Advitya17 13faa8c
Add columns names to `create_multiclass_classification_dataset` (#2114)
gaugup f0d9661
[Object Detection] [Image Explorer] Bounding Box Support with CanvasT…
Advitya17 ef6d835
add ResponsibleAI Text package as initial commit
imatiach-msft 84f2d5d
update responsibleai-text package to responsibleai v0.25.0
imatiach-msft d5bf611
add model-agnostic explanation support for wrapped models with simple…
imatiach-msft 56fd516
add build and release github actions to responsibleai-text package (#2)
imatiach-msft 80d96a7
Enable error analysis for QA (#5)
JarvisG495 4605905
validate github actions, fix linting and builds (#4)
imatiach-msft 517736b
added question type feature extractor (#6)
sreeja-g 42feb24
remove train data parameter from RAITextInsights (#7)
imatiach-msft c1d6cc5
release responsibleai-text v0.1.0 (#8)
imatiach-msft c272cd7
cache error analysis predictions for RAITextInsights to optimize perf…
imatiach-msft bc9af3d
update responsibleai-text package to responsibleai v0.27.0 (#11)
imatiach-msft f0106b8
release responsibleai-text v0.1.1 (#12)
imatiach-msft baff863
Add QA feature context overlap and improve question type regex (#14)
JarvisG495 0747559
fix task type for multilabel and multiclass classification when passe…
imatiach-msft 14fbc7f
release responsibleai-text 0.1.2 (#16)
imatiach-msft a1c7d8a
Python backend metrics for question answering scenario (#17)
mehektulsyan bf434a0
added-syntactic-complexity (#18)
sreeja-g da1eb51
add metadata support to RAITextInsights (#19)
imatiach-msft 9646005
fix feature metadata serialization logic in RAITextInsights which was…
imatiach-msft cefd38c
add text_column parameter to RAITextInsights to allow user to specify…
imatiach-msft 3bb8a9a
release responsibleai-text 0.1.3 (#21)
imatiach-msft b2c1285
fix responsibleai-text linting and move directory to
imatiach-msft 2f5e0b1
Add loading spinner for causal policy tab (#2121)
gaugup a98f084
Refactor- Add datasetCohorts to ModelAssessementContext && create dat…
RubyZ10 b69d23a
remove console log (#2125)
RubyZ10 bb7628d
Update msdata-rai-widgets-onelocbuild.yml for Azure Pipelines
RubyZ10 633d966
fix build failures due to lint errors for onelocbuild yml (#2137)
imatiach-msft 763a9e3
Skipping python 3.7 tests on MACOS (#2129)
gaugup 4289d1a
`rai_test_utils`: Convert all namespaces in `__all__` list as strings…
gaugup 3fdfbcb
User Flows / Architectures
Advitya17 9120122
[Object Detection] DFD Documentation (#2138)
natalie-isak afab075
[Object Detection] Explanation Documentation (#2136)
natalie-isak 7fca2d0
CanvasTools documentation
Advitya17 01a9197
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
vinuthakaranth 3b36bbe
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
vinuthakaranth a9bbe9d
isQA as prop
vinuthakaranth 586da40
update and refine
vinuthakaranth 9f2929a
Merge branch 'main' of https://github.com/microsoft/responsible-ai-to…
vinuthakaranth 9c2df87
remove state isQA
vinuthakaranth 9a9b1ac
clean
vinuthakaranth af9657e
Merge branch 'main' into vikarant/textUpdatesQA
vinuthakaranth 4cc41cb
Merge branch 'main' into vikarant/textUpdatesQA
vinuthakaranth 469bfe7
Merge branch 'main' into vikarant/textUpdatesQA
vinuthakaranth ee0353f
Merge branch 'main' into vikarant/textUpdatesQA
vinuthakaranth File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
52 changes: 52 additions & 0 deletions
52
.../src/lib/TextExplanationDashboard/Control/TextExplanationView/ITextExplanationViewSpec.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
// Copyright (c) Microsoft Corporation. | ||
// Licensed under the MIT License. | ||
|
||
import { IChoiceGroupOption, IStackTokens } from "@fluentui/react"; | ||
import { localization } from "@responsible-ai/localization"; | ||
|
||
import { QAExplanationType, RadioKeys } from "../../CommonUtils"; | ||
|
||
export interface ITextExplanationViewState { | ||
/* | ||
* Holds the state of the dashboard | ||
*/ | ||
maxK: number; | ||
topK: number; | ||
radio: string; | ||
// qaRadio?: string; | ||
importances: number[]; | ||
singleTokenImportances: number[]; | ||
selectedToken: number; | ||
tokenIndexes: number[]; | ||
text: string[]; | ||
} | ||
|
||
export const options: IChoiceGroupOption[] = [ | ||
/* | ||
* Creates the choices for the radio button | ||
*/ | ||
{ key: RadioKeys.All, text: localization.InterpretText.View.allButton }, | ||
{ key: RadioKeys.Pos, text: localization.InterpretText.View.posButton }, | ||
{ key: RadioKeys.Neg, text: localization.InterpretText.View.negButton } | ||
]; | ||
|
||
export const qaOptions: IChoiceGroupOption[] = [ | ||
/* | ||
* Creates the choices for the QA prediction radio button | ||
*/ | ||
{ | ||
key: QAExplanationType.Start, | ||
text: localization.InterpretText.View.startingPosition | ||
}, | ||
{ | ||
key: QAExplanationType.End, | ||
text: localization.InterpretText.View.endingPosition | ||
} | ||
]; | ||
|
||
export const componentStackTokens: IStackTokens = { | ||
childrenGap: "m", | ||
padding: "m" | ||
}; | ||
|
||
export const MaxImportantWords = 15; |
134 changes: 134 additions & 0 deletions
134
...et-text/src/lib/TextExplanationDashboard/Control/TextExplanationView/SidePanelOfChart.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
// Copyright (c) Microsoft Corporation. | ||
// Licensed under the MIT License. | ||
|
||
import { | ||
ChoiceGroup, | ||
IChoiceGroupOption, | ||
Label, | ||
Slider, | ||
Stack, | ||
Text | ||
} from "@fluentui/react"; | ||
import { WeightVectorOption, WeightVectors } from "@responsible-ai/core-ui"; | ||
import { ClassImportanceWeights } from "@responsible-ai/interpret"; | ||
import { localization } from "@responsible-ai/localization"; | ||
import React from "react"; | ||
|
||
import { QAExplanationType, Utils } from "../../CommonUtils"; | ||
import { IDatasetSummary } from "../../Interfaces/IExplanationDashboardProps"; | ||
import { BarChart } from "../BarChart/BarChart"; | ||
|
||
import { | ||
componentStackTokens, | ||
options, | ||
qaOptions | ||
} from "./ITextExplanationViewSpec"; | ||
import { textExplanationDashboardStyles } from "./TextExplanationView.styles"; | ||
|
||
export interface ISidePanelOfChartProps { | ||
text: string[]; | ||
importances: number[]; | ||
topK: number; | ||
radio: string; | ||
isQA?: boolean; | ||
dataSummary: IDatasetSummary; | ||
maxK: number; | ||
selectedToken: number; | ||
tokenIndexes: number[]; | ||
selectedWeightVector: WeightVectorOption; | ||
weightOptions: WeightVectorOption[]; | ||
weightLabels: any; | ||
changeRadioButton: ( | ||
_event?: React.FormEvent, | ||
item?: IChoiceGroupOption | ||
) => void; | ||
switchQAPrediction: ( | ||
_event?: React.FormEvent, | ||
_item?: IChoiceGroupOption | ||
) => void; | ||
setTopK: (newNumber: number) => void; | ||
onWeightVectorChange: (weightOption: WeightVectorOption) => void; | ||
onSelectedTokenChange: (newIndex: number) => void; | ||
} | ||
|
||
export class SidePanelOfChart extends React.PureComponent<ISidePanelOfChartProps> { | ||
public render(): React.ReactNode { | ||
const classNames = textExplanationDashboardStyles(); | ||
|
||
return ( | ||
<Stack tokens={componentStackTokens} horizontal> | ||
<Stack.Item grow disableShrink> | ||
<BarChart | ||
text={this.props.text} | ||
localExplanations={this.props.importances} | ||
topK={this.props.topK} | ||
radio={this.props.radio} | ||
/> | ||
</Stack.Item> | ||
<Stack.Item grow className={classNames.chartRight}> | ||
<Stack tokens={componentStackTokens}> | ||
{!this.props.isQA && ( // classification | ||
<Stack.Item> | ||
<Text variant={"xLarge"}> | ||
{localization.InterpretText.View.label + | ||
localization.InterpretText.View.colon + | ||
Utils.predictClass( | ||
this.props.dataSummary.classNames, | ||
this.props.dataSummary.prediction | ||
)} | ||
</Text> | ||
</Stack.Item> | ||
)} | ||
|
||
{this.props.isQA && ( // select starting/ending for QA | ||
<Stack.Item id="TextChoiceGroup"> | ||
<ChoiceGroup | ||
defaultSelectedKey={QAExplanationType.Start} | ||
options={qaOptions} | ||
onChange={this.props.switchQAPrediction} | ||
required | ||
/> | ||
</Stack.Item> | ||
)} | ||
|
||
<Stack.Item> | ||
<Label>{localization.InterpretText.View.importantWords}</Label> | ||
</Stack.Item> | ||
|
||
<Stack.Item id="TextTopKSlider"> | ||
<Slider | ||
min={1} | ||
max={this.props.maxK} | ||
step={1} | ||
value={this.props.topK} | ||
showValue | ||
onChange={this.props.setTopK} | ||
/> | ||
</Stack.Item> | ||
{!this.props.isQA && ( | ||
<Stack.Item> | ||
<ClassImportanceWeights | ||
onWeightChange={this.props.onWeightVectorChange} | ||
selectedWeightVector={this.props.selectedWeightVector} | ||
weightOptions={this.props.weightOptions} | ||
weightLabels={this.props.weightLabels} | ||
/> | ||
</Stack.Item> | ||
)} | ||
|
||
{this.props.selectedWeightVector !== WeightVectors.AbsAvg && ( | ||
<Stack.Item id="TextChoiceGroup"> | ||
<ChoiceGroup | ||
defaultSelectedKey="all" | ||
options={options} | ||
onChange={this.props.changeRadioButton} | ||
required | ||
/> | ||
</Stack.Item> | ||
)} | ||
</Stack> | ||
</Stack.Item> | ||
</Stack> | ||
); | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.