Skip to content
This repository was archived by the owner on Oct 11, 2023. It is now read-only.
This repository was archived by the owner on Oct 11, 2023. It is now read-only.

Learning rate for NN training  #26

Description

@pamio

in tensorflow or torch, I have observed that the learning rate is usually one number like 0.001 or 0.01 etc. But in this tutorial I find that the learning rate for 16 classes is something like this

cntk_lr_per_image = [0.01] * 10 + [0.001] * 5 + [0.0001] which is array of 16 items which is equal to the number of classes. So if I have 80 classes to train should this array cntk_lr_per_image contain 80 learning rate ?

Something like this cntk_lr_per_image = [0.01] * 40 + [0.001] * 30 + [0.0001]*10 ?
I understand what learning rate is and how it impacts the overall learning of the model and there's no straight forward way to choose a learning rate but this tutorial uses an array of learning rates which is what confuses me.

Can you help me with understanding how learning rate us used in this example ?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions