Skip to content

Conversation

@szaman19
Copy link
Collaborator

@szaman19 szaman19 commented Mar 15, 2023

Adds distconv support for layer normalization

  • Add CI test
  • Implement layer_norm_distconv_adapter
  • Move layer norm kernels to shared .cuh file
  • Use existing layer norm kernels to implement distconv layer norm operator

@szaman19 szaman19 force-pushed the distconv-layer-norm branch from 69269a7 to a4f367d Compare March 17, 2023 19:56
@szaman19 szaman19 marked this pull request as ready for review March 18, 2023 08:31
@szaman19 szaman19 force-pushed the distconv-layer-norm branch from 0a7ffbd to d59e2db Compare April 20, 2023 07:43
@szaman19 szaman19 force-pushed the distconv-layer-norm branch from 5b133d0 to 13aff25 Compare May 24, 2023 02:58
@szaman19
Copy link
Collaborator Author

szaman19 commented Dec 4, 2023

Tagging @tbennun @benson31 @bvanessen

@tbennun
Copy link

tbennun commented Dec 4, 2023

Tagging @tbennun @benson31 @bvanessen

Please fix the conflicts :)

@szaman19 szaman19 force-pushed the distconv-layer-norm branch from 13aff25 to 01a361a Compare December 4, 2023 18:43
@szaman19 szaman19 force-pushed the distconv-layer-norm branch from ecac28c to 021d2a7 Compare June 25, 2024 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants