Perceiver Implementation in PyTorch

This is a PyTorch implementation of the Perceiver architecture, based on the original paper: Perceiver: General Perception with Iterative Attention.

Model Architecture

This implementation is designed for experiments on the ImageNet dataset, as described in the paper. Results from these experiments and usage instructions will be updated here soon.

Usage

Todo...

Experiment

Todo...

References

• This code is partly based on lucidrains' perceiver-pytorch implementation, with some shared code.
• The Lamb optimizer in the include/ directory is borrowed from the implementation by cybertronai.
• The Transformer code in include/transformer is based on attention-is-all-you-need-pytorch.