Code

feat/pipeop-transformer-layer#388

Code

Draft

cxzhang4 wants to merge 57 commits intomainmlr-org/mlr3torch:mainfrom

feat/pipeop-transformer-layermlr-org/mlr3torch:feat/pipeop-transformer-layerCopy head branch name to clipboard

Commits

Commits on Apr 15, 2025

• double quotes

  cxzhang4

  committed
  50bc8e9

  Copy full SHA for 50bc8e9

• style

  cxzhang4

  committed
  8aa4470

  Copy full SHA for 8aa4470

• copied in old attic code to test file, still need to try

  cxzhang4

  committed
  4b5fafe

  Copy full SHA for 4b5fafe

Commits on Apr 21, 2025

• idrk

  cxzhang4

  committed
  258ea42

  Copy full SHA for 258ea42

• Merge branch 'main' into feat/pipeop-transformer-layer

  cxzhang4

  committed
  a8a8787

  Copy full SHA for a8a8787

• changed d_token in test

  cxzhang4

  committed
  b81e9dd

  Copy full SHA for b81e9dd

• small cleanup

  cxzhang4

  committed
  e7288f8

  Copy full SHA for e7288f8

• cleanup

  cxzhang4

  committed
  da059e2

  Copy full SHA for da059e2

• factored out d_token

  cxzhang4

  committed
  866a7ec

  Copy full SHA for 866a7ec

• idk

  cxzhang4

  committed
  e6e67f6

  Copy full SHA for e6e67f6

Commits on Apr 22, 2025

• test passes for now

  cxzhang4

  committed
  9d37216

  Copy full SHA for 9d37216

• intermrediate docs

  cxzhang4

  committed
  31ef199

  Copy full SHA for 31ef199

• TODO: implement custom checks for parameters that are nn_modules or nn_module_generators

  cxzhang4

  committed
  ebcb3c0

  Copy full SHA for ebcb3c0

• more TODOs

  cxzhang4

  committed
  946dba0

  Copy full SHA for 946dba0

• docs

  cxzhang4

  committed
  00c91eb

  Copy full SHA for 00c91eb

• change title of nn_ft_transformer_layer module

  cxzhang4

  committed
  da21ff2

  Copy full SHA for da21ff2

• removed is_first_layer param

  cxzhang4

  committed
  7d65f09

  Copy full SHA for 7d65f09

• some comments

  cxzhang4

  committed
  3872ce0

  Copy full SHA for 3872ce0

Commits on Apr 23, 2025

• a comment

  cxzhang4

  committed
  ce4809b

  Copy full SHA for ce4809b

• added back is_first_layer param

  cxzhang4

  committed
  41724f0

  Copy full SHA for 41724f0

• added back comment on prenormalization condition

  cxzhang4

  committed
  1c2ee1e

  Copy full SHA for 1c2ee1e

Commits on Apr 24, 2025

• comment on parameters

  cxzhang4

  committed
  6b4c34a

  Copy full SHA for 6b4c34a

• Merge branch 'main' into feat/pipeop-transformer-layer

  sebffischer

  authored
  24645c4

  Copy full SHA for 24645c4

• some changes

  sebffischer

  committed
  8b671ee

  Copy full SHA for 8b671ee

• some notes

  sebffischer

  committed
  f812cb5

  Copy full SHA for f812cb5

Commits on Apr 25, 2025

• some more changes

  sebffischer

  committed
  14c98c5

  Copy full SHA for 14c98c5

• ...

  sebffischer

  committed
  8d5f641

  Copy full SHA for 8d5f641

• factored out last_layer_query_idx from layer

  cxzhang4

  committed
  c479d9c

  Copy full SHA for c479d9c

• query_idx should be -1L (last dim) for last transformer layer

  cxzhang4

  committed
  0c020cf

  Copy full SHA for 0c020cf

• deleted file with old name (Layer, not Block)

  cxzhang4

  committed
  0f17330

  Copy full SHA for 0f17330

• formatting

  cxzhang4

  committed
  916648d

  Copy full SHA for 916648d

• check_nn_module_generator

  cxzhang4

  committed
  40d5e44

  Copy full SHA for 40d5e44

• Merge branch 'main' into feat/pipeop-transformer-layer

  cxzhang4

  committed
  44f003b

  Copy full SHA for 44f003b

Commits on Apr 26, 2025

• got rid of some defaults

  cxzhang4

  committed
  726e8af

  Copy full SHA for 726e8af

Commits on Apr 27, 2025

• idk

  cxzhang4

  committed
  5b12f99

  Copy full SHA for 5b12f99

• reduce blocks in learner when there are multipleg

  cxzhang4

  committed
  c69af36

  Copy full SHA for c69af36

• delete TODO

  cxzhang4

  committed
  8f80d18

  Copy full SHA for 8f80d18

• fix test

  cxzhang4

  committed
  d0ec4fc

  Copy full SHA for d0ec4fc

• some comments

  cxzhang4

  committed
  f6a6326

  Copy full SHA for f6a6326

Commits on Apr 28, 2025

• small changes

  

  Zhang

  authored and

  Zhang

  committed
  a11b217

  Copy full SHA for a11b217

• Merge branch 'main' into feat/pipeop-transformer-layer

  

  Zhang

  authored and

  Zhang

  committed
  0416ead

  Copy full SHA for 0416ead

• added custom error messages

  cxzhang4

  committed
  3e609bd

  Copy full SHA for 3e609bd

Commits on Apr 29, 2025

• x_residual

  cxzhang4

  committed
  852d69c

  Copy full SHA for 852d69c

• set block dependent default vals

  cxzhang4

  committed
  ec4e8ab

  Copy full SHA for ec4e8ab

• some comments

  cxzhang4

  committed
  d5c3cc4

  Copy full SHA for d5c3cc4

• Merge branch 'main' into feat/pipeop-transformer-layer

  cxzhang4

  committed
  e73100a

  Copy full SHA for e73100a

Commits on May 1, 2025

• intermediate

  cxzhang4

  committed
  92ced45

  Copy full SHA for 92ced45

• Merge branch 'main' into feat/pipeop-transformer-layer

  cxzhang4

  committed
  7a23834

  Copy full SHA for 7a23834

• remove first_prenormalization

  cxzhang4

  committed
  31571ff

  Copy full SHA for 31571ff

• address TODOs

  cxzhang4

  committed
  b5172a6

  Copy full SHA for b5172a6

• looks ok 2 me, still has compression

  cxzhang4

  committed
  17d25cb

  Copy full SHA for 17d25cb

• removed kv compression

  cxzhang4

  committed
  d50e1e0

  Copy full SHA for d50e1e0

• update docs for learner

  cxzhang4

  committed
  c9ad1e9

  Copy full SHA for c9ad1e9

• added block-dependent defaults, removed required tags from learner params

  cxzhang4

  committed
  5bf9ada

  Copy full SHA for 5bf9ada

• added activation-dependent ffn_d_hidden

  cxzhang4

  committed
  7707922

  Copy full SHA for 7707922

• a comment

  cxzhang4

  committed
  b62f95c

  Copy full SHA for b62f95c

Commits on May 2, 2025

• man

  cxzhang4

  committed
  abb2094

  Copy full SHA for abb2094