Skip to content

Prototypical probing / spatial embeddings / reranking in hoplite #41

@sdenton4

Description

@sdenton4

ProtoPNet has been shown to improve classification outcomes for pre-trained classifier heads; potentially we could use a similar trick here.

It is not without some difficulty, though. We are currently storing average embeddings, rather than 'spatial' embeddings, and much of the advantage of the protopnet head seems to derive from having access to the spatial embeddings.

Good support for this could then look like either:
a) Optionally insert finer-grained spatial embeddings,
b) Use coarse embeddings for first-round classification, then 'rerank' by re-embedding+evaluating on spatial embeddings.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions