Code from BasicOptimizer.scala:75 executed in 3622.28 seconds (47.174 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1337030401114800
Reset training subject: 1337074000703300
Adding measurement 58c36104 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=6.883337376639247;dx=-4.9086824542370144E-6
New Minimum: 6.883337376639247 > 6.883331434801221
WOLFE (weak): th(2.154434690031884)=6.883331434801221; dx=-4.877739032932398E-6 evalInputDelta=5.941838026046753E-6
New Minimum: 6.883331434801221 > 6.883325330913067
WOLFE (weak): th(4.308869380063768)=6.883325330913067; dx=-4.877501914243836E-6 evalInputDelta=1.2045726180076599E-5
New Minimum: 6.883325330913067 > 6.883302783593535
WOLFE (weak): th(12.926608140191302)=6.883302783593535; dx=-4.877255934505355E-6 evalInputDelta=3.4593045711517334E-5
New Minimum: 6.883302783593535 > 6.88319842889905
WOLFE (weak): th(51.70643256076521)=6.88319842889905; dx=-4.8715603565858205E-6 evalInputDelta=1.389477401971817E-4
New Minimum: 6.88319842889905 > 6.882642045617104
WOLFE (weak): th(258.53216280382605)=6.882642045617104; dx=-4.869073731822072E-6 evalInputDelta=6.95331022143364E-4
New Minimum: 6.882642045617104 > 6.879176141694188
WOLFE (weak): th(1551.1929768229563)=6.879176141694188; dx=-4.842230389705609E-6 evalInputDelta=0.004161234945058823
New Minimum: 6.879176141694188 > 6.8547482173889875
WOLFE (weak): th(10858.350837760694)=6.8547482173889875; dx=-4.646654461490087E-6 evalInputDelta=0.0285891592502594
New Minimum: 6.8547482173889875 > 6.689243447035551
END: th(86866.80670208555)=6.689243447035551; dx=-3.3786892272627292E-6 evalInputDelta=0.19409392960369587
Fitness changed from 6.883337376639247 to 6.689243447035551
Iteration 1 complete. Error: 6.689243447035551 Total: 561.2924; Orientation: 0.0851; Line Search: 421.2146
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4023d23f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=6.689243447035551;dx=-3.1487652200876885E-6
New Minimum: 6.6
...skipping 13970 bytes...
elta=0.04063594713807106
New Minimum: 3.1279969587922096 > 3.0371288657188416
END: th(451265.5810456872)=3.0371288657188416; dx=-2.0628926804361433E-7 evalInputDelta=0.13150404021143913
Fitness changed from 3.1686329059302807 to 3.0371288657188416
Iteration 17 complete. Error: 3.0371288657188416 Total: 230.9783; Orientation: 2.4738; Line Search: 182.7900
Adding measurement 736f2fb8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.326e+03, gradient 5.518e-04, dot -0.671; [4c9c185a-1ad9-4587-832d-894e9ab5a3a2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.3917146660387516, 3.2694899328052998, 3.1686329059302807, 3.0371288657188416
LBFGS Accumulation History: 3 points
Removed measurement 3ab70d34 to history. Total: 4
th(0)=3.0371288657188416;dx=-3.044803928293884E-7
New Minimum: 3.0371288657188416 > 2.9388365671038628
END: th(972222.2222222231)=2.9388365671038628; dx=-1.2888400318087337E-7 evalInputDelta=0.09829229861497879
Fitness changed from 3.0371288657188416 to 2.9388365671038628
Iteration 18 complete. Error: 2.9388365671038628 Total: 140.7757; Orientation: 2.3743; Line Search: 92.7488
Adding measurement 64921450 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.358e+03, gradient 4.832e-04, dot -0.764; [4c9c185a-1ad9-4587-832d-894e9ab5a3a2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.2694899328052998, 3.1686329059302807, 3.0371288657188416, 2.9388365671038628
LBFGS Accumulation History: 3 points
Removed measurement 5216532a to history. Total: 4
th(0)=2.9388365671038628;dx=-2.3348061960216932E-7
New Minimum: 2.9388365671038628 > 2.835347130894661
END: th(2094589.2819754444)=2.835347130894661; dx=-2.7320206364056017E-8 evalInputDelta=0.10348943620920181
Fitness changed from 2.9388365671038628 to 2.835347130894661
Iteration 19 complete. Error: 2.835347130894661 Total: 138.9204; Orientation: 2.4064; Line Search: 90.5198
Final threshold in iteration 19: 2.835347130894661 (> -Infinity) after 3622.281s (< 3600.000s)
2.835347130894661