Detect convergence rate so that the controlling program can decide if we are stagnating. I am uncertain if we can be temporarily stuck in a local minima that we would eventually get out of, so this feature may also double as a tool for exploring that idea.
I.e. if the learning rate is less than the "width" of a local minima, would that eliminate any hope of getting out of it? We could possibly test this by forcing ourselves into a known local minima and then adjusting the learning rate to see if we bounce out of it.
Detect convergence rate so that the controlling program can decide if we are stagnating. I am uncertain if we can be temporarily stuck in a local minima that we would eventually get out of, so this feature may also double as a tool for exploring that idea.
I.e. if the learning rate is less than the "width" of a local minima, would that eliminate any hope of getting out of it? We could possibly test this by forcing ourselves into a known local minima and then adjusting the learning rate to see if we bounce out of it.