Skip to content

Fix main.c #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

Fix main.c #12

wants to merge 7 commits into from

Conversation

Numbers0689
Copy link

Changes:

  • Add gradient functions to build a computational graph for auto differentiation (GradFn_crossentropy).
  • Refactor GradFn_matmul to compute gradient correctly with transpose (Tensor_transpose) and return propagated gradient.
  • Refactor Tensor_add to keep track of real inputs instead of broadcasted ones, thus keeping the model biases connected to the graph.
  • Add Tensor_sum_reduce to sum gradients along broadcasted axis.
  • Improve Tensor_backward and add gradient clipping.
  • Add normalization and shuffling of dataset.
  • Add He initialization.

Next:

  • Refactor Tensor_set and Tensor_get.
  • Ensure proper gradient accumulation and propagation through testing.
  • Add testing and revise operators accordingly.
  • Add basic Tensor operations and neural network activations, functions.
  • Currently, loss is computed through forward pass and gradients are updated with backprop and sgd, but losses fluctuate, indicating other issues. As well as the memory related crashes when layers are increased.

@Numbers0689
Copy link
Author

Numbers0689 commented Mar 17, 2025

n_samples: 150
n_train_samples: 120
n_test_samples: 30
==> epoch: 0
==> epoch: 1
==> epoch: 2
==> epoch: 3
==> epoch: 4
==> epoch: 5
==> epoch: 6
==> epoch: 7
==> epoch: 8
==> epoch: 9
test 120, y_true: 2, y_pred: 2
test 121, y_true: 0, y_pred: 0
test 122, y_true: 1, y_pred: 1
test 123, y_true: 2, y_pred: 2
test 124, y_true: 0, y_pred: 0
test 125, y_true: 2, y_pred: 1
test 126, y_true: 1, y_pred: 1
test 127, y_true: 1, y_pred: 1
test 128, y_true: 0, y_pred: 0
test 129, y_true: 0, y_pred: 0
test 130, y_true: 0, y_pred: 0
test 131, y_true: 0, y_pred: 0
test 132, y_true: 1, y_pred: 1
test 133, y_true: 0, y_pred: 0
test 134, y_true: 2, y_pred: 1
test 135, y_true: 2, y_pred: 0
test 136, y_true: 1, y_pred: 1
test 137, y_true: 0, y_pred: 0
test 138, y_true: 2, y_pred: 2
test 139, y_true: 0, y_pred: 0
test 140, y_true: 1, y_pred: 0
test 141, y_true: 0, y_pred: 0
test 142, y_true: 0, y_pred: 1
test 143, y_true: 1, y_pred: 1
test 144, y_true: 1, y_pred: 1
test 145, y_true: 2, y_pred: 2
test 146, y_true: 2, y_pred: 1
test 147, y_true: 1, y_pred: 1
test 148, y_true: 1, y_pred: 1
test 149, y_true: 2, y_pred: 2
accuracy: 0.8000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant