-
Notifications
You must be signed in to change notification settings - Fork 415
Open
Description
I see the function LigerFusedLinearCrossEntropyLoss, I think bias and lin_weight are from the previous layer. If so, maybe def forward(self, _input, target, lin_weight, bias=None):
is easy to understand.
def forward(self, lin_weight, _input, target, bias=None):
loss, z_loss = LigerFusedLinearCrossEntropyFunction.apply(
_input,
lin_weight,
target,
bias,
self.ce_weight,
self.ignore_index,
self.lse_square_scale,
self.label_smoothing,
self.reduction,
self.softcap,
self.return_z_loss,
self.accum_dtype,
)
if not self.return_z_loss:
return loss
return loss, z_loss
Metadata
Metadata
Assignees
Labels
No labels