Code from BasicOptimizer.scala:75 executed in 1098.46 seconds (7.071 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1106797555165600
Reset training subject: 1106820377483499
Adding measurement 7b44bfb8 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=0.39682289958000183;dx=-6.274261046275891E-8
New Minimum: 0.39682289958000183 > 0.39682281017303467
WOLFE (weak): th(2.154434690031884)=0.39682281017303467; dx=-6.271902272993876E-8 evalInputDelta=8.940696716308594E-8
New Minimum: 0.39682281017303467 > 0.3968224823474884
WOLFE (weak): th(4.308869380063768)=0.3968224823474884; dx=-6.270592558376162E-8 evalInputDelta=4.172325134277344E-7
New Minimum: 0.3968224823474884 > 0.3968212604522705
WOLFE (weak): th(12.926608140191302)=0.3968212604522705; dx=-6.27245909540444E-8 evalInputDelta=1.6391277313232422E-6
New Minimum: 0.3968212604522705 > 0.3968162536621094
WOLFE (weak): th(51.70643256076521)=0.3968162536621094; dx=-6.271168251294244E-8 evalInputDelta=6.645917892456055E-6
New Minimum: 0.3968162536621094 > 0.3967900574207306
WOLFE (weak): th(258.53216280382605)=0.3967900574207306; dx=-6.270852119169194E-8 evalInputDelta=3.2842159271240234E-5
New Minimum: 0.3967900574207306 > 0.39662599563598633
WOLFE (weak): th(1551.1929768229563)=0.39662599563598633; dx=-6.268487934442925E-8 evalInputDelta=1.9690394401550293E-4
New Minimum: 0.39662599563598633 > 0.3954445421695709
WOLFE (weak): th(10858.350837760694)=0.3954445421695709; dx=-6.267044786009986E-8 evalInputDelta=0.0013783574104309082
New Minimum: 0.3954445421695709 > 0.38582032918930054
WOLFE (weak): th(86866.80670208555)=0.38582032918930054; dx=-6.228433509554512E-8 evalInputDelta=0.011002570390701294
New Minimum: 0.38582032918930054 > 0.3008058965206146
WOLFE (weak): th(781801.26031877)=0.3008058965206146; dx=-5.7608198383622425E-8 evalInputDelta=0.09601700305938721
New Minimum: 0.3008058965206146 > -0.2248360812664032
END: th(7818012.6031877)=-0.2248360812664032; dx=-2.1844843874038757E-8 evalInputDelta=0.621658980846405
Fitness changed from 0.39682289958000183 to -0.224836
...skipping 6418 bytes...
Orientation magnitude: 1.888e+03, gradient 3.501e-05, dot -0.683; [3b6ec25f-4047-427c-a292-d0900913e5d2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.7482684552669525, -1.8184625506401062, -1.8792795836925507, -1.9138798415660858
LBFGS Accumulation History: 3 points
Removed measurement 4a9869a8 to history. Total: 4
th(0)=-1.9138798415660858;dx=-1.224904374920104E-9
Armijo: th(7.560000000000003E7)=-1.8341145515441895; dx=1.5917586745404092E-9 evalInputDelta=-0.07976529002189636
New Minimum: -1.9138798415660858 > -1.9356207847595215
WOLF (strong): th(3.7800000000000015E7)=-1.9356207847595215; dx=4.731007872626945E-10 evalInputDelta=0.02174094319343567
New Minimum: -1.9356207847595215 > -1.9373097121715546
END: th(1.2600000000000006E7)=-1.9373097121715546; dx=-6.648442090303685E-10 evalInputDelta=0.02342987060546875
Fitness changed from -1.9138798415660858 to -1.9373097121715546
Iteration 9 complete. Error: -1.9373097121715546 Total: 113.3797; Orientation: 0.5929; Line Search: 90.1833
Adding measurement 473847fb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.970e+03, gradient 2.366e-05, dot -0.914; [3b6ec25f-4047-427c-a292-d0900913e5d2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8184625506401062, -1.8792795836925507, -1.9138798415660858, -1.9373097121715546
LBFGS Accumulation History: 3 points
Removed measurement 5835e24a to history. Total: 4
th(0)=-1.9373097121715546;dx=-5.586884056786176E-10
New Minimum: -1.9373097121715546 > -1.96040678024292
END: th(2.7145877094401747E7)=-1.96040678024292; dx=-3.8992722802366377E-10 evalInputDelta=0.023097068071365356
Fitness changed from -1.9373097121715546 to -1.96040678024292
Iteration 10 complete. Error: -1.96040678024292 Total: 70.5361; Orientation: 0.5967; Line Search: 47.5878
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -1.96040678024292 (> -Infinity) after 1098.462s (< 1800.000s)
-1.96040678024292