Code from BasicOptimizer.scala:75 executed in 995.63 seconds (11.425 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1161232099348800
Reset training subject: 1161240588512100
Adding measurement 4519f676 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.106054234780139;dx=-2.3922615198358638E-5
New Minimum: 5.106054234780139 > 5.10603158279082
WOLFE (weak): th(2.154434690031884)=5.10603158279082; dx=-2.3922352241333678E-5 evalInputDelta=2.2651989318944743E-5
New Minimum: 5.10603158279082 > 5.106009074882121
WOLFE (weak): th(4.308869380063768)=5.106009074882121; dx=-2.3922329272833513E-5 evalInputDelta=4.5159898017566036E-5
New Minimum: 5.106009074882121 > 5.105919186523064
WOLFE (weak): th(12.926608140191302)=5.105919186523064; dx=-2.3917480612057928E-5 evalInputDelta=1.350482570749989E-4
New Minimum: 5.105919186523064 > 5.105514751509801
WOLFE (weak): th(51.70643256076521)=5.105514751509801; dx=-2.3910511377867714E-5 evalInputDelta=5.394832703373709E-4
New Minimum: 5.105514751509801 > 5.103359543003735
WOLFE (weak): th(258.53216280382605)=5.103359543003735; dx=-2.3896915844584333E-5 evalInputDelta=0.0026946917764041345
New Minimum: 5.103359543003735 > 5.089950502081792
WOLFE (weak): th(1551.1929768229563)=5.089950502081792; dx=-2.3710212336039144E-5 evalInputDelta=0.01610373269834664
New Minimum: 5.089950502081792 > 4.996284468871887
WOLFE (weak): th(10858.350837760694)=4.996284468871887; dx=-2.246819923910524E-5 evalInputDelta=0.1097697659082515
New Minimum: 4.996284468871887 > 4.403053996634565
END: th(86866.80670208555)=4.403053996634565; dx=-1.432130324874397E-5 evalInputDelta=0.7030002381455738
Fitness changed from 5.106054234780139 to 4.403053996634565
Iteration 1 complete. Error: 4.403053996634565 Total: 99.4980; Orientation: 0.0188; Line Search: 74.5362
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3913f206 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.403053996634565;dx=-1.071464210974175E-5
New Minimum: 4.403
...skipping 23964 bytes...
cfd4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.567e+03, gradient 2.380e-04, dot -0.626; [34d38efd-324c-475a-83ad-0b1101dd52c2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.05456303738357926, 0.03676839028468562, 0.0190163434976299, 0.003388233359841286
LBFGS Accumulation History: 3 points
Removed measurement 402f8592 to history. Total: 4
th(0)=0.003388233359841286;dx=-5.664512142019268E-8
Armijo: th(1.3927950032274302E7)=0.32189319706463404; dx=5.706987391322795E-8 evalInputDelta=-0.31850496370479275
Armijo: th(6963975.016137151)=0.05415540491534743; dx=2.3879318957422338E-8 evalInputDelta=-0.05076717155550614
New Minimum: 0.003388233359841286 > -0.01701326958886587
END: th(2321325.0053790505)=-0.01701326958886587; dx=-2.0780496739929003E-8 evalInputDelta=0.020401502948707156
Fitness changed from 0.003388233359841286 to -0.01701326958886587
Iteration 29 complete. Error: -0.01701326958886587 Total: 40.8031; Orientation: 0.2177; Line Search: 32.5620
Adding measurement 1e3f86d5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.811e+03, gradient 2.049e-04, dot -0.691; [34d38efd-324c-475a-83ad-0b1101dd52c2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.03676839028468562, 0.0190163434976299, 0.003388233359841286, -0.01701326958886587
LBFGS Accumulation History: 3 points
Removed measurement fbe70d8 to history. Total: 4
th(0)=-0.01701326958886587;dx=-4.199765862906783E-8
New Minimum: -0.01701326958886587 > -0.018873544676522955
END: th(5001143.118427075)=-0.018873544676522955; dx=-4.618625290337834E-9 evalInputDelta=0.0018602750876570845
Fitness changed from -0.01701326958886587 to -0.018873544676522955
Iteration 30 complete. Error: -0.018873544676522955 Total: 30.0393; Orientation: 0.2067; Line Search: 19.7380
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: -0.018873544676522955 (> -Infinity) after 995.618s (< 3600.000s)
-0.018873544676522955