BasicOptimizer.scala:89 executed in 129.27 seconds (1.545 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 30770849253700
Reset training subject: 30773443193600
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -79.65631000224427 > -79.65898625178028
th(0)=-79.65898625178028;dx=-4.985124899371507E-6
New Minimum: -79.65898625178028 > -79.65899484312678
END: th(2.154434690031884)=-79.65899484312678; dx=-3.5723352711593187E-6 evalInputDelta=8.591346499997599E-6
Fitness changed from -79.65631000224427 to -79.65899484312678
Iteration 1 complete. Error: -79.65899484312678 Total: 12.8585; Orientation: 0.0385; Line Search: 5.0897
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 19192755 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-79.65899484312678;dx=-3.546155014488998E-6
New Minimum: -79.65899484312678 > -79.659012025817
WOLFE (weak): th(4.641588833612779)=-79.659012025817; dx=-3.545046959963854E-6 evalInputDelta=1.7182690228878528E-5
New Minimum: -79.659012025817 > -79.65903350417457
WOLFE (weak): th(9.283177667225559)=-79.65903350417457; dx=-3.543252199862572E-6 evalInputDelta=3.866104779604029E-5
New Minimum: -79.65903350417457 > -79.65910653054699
WOLFE (weak): th(27.849533001676676)=-79.65910653054699; dx=-3.5401977636324347E-6 evalInputDelta=1.1168742021538947E-4
New Minimum: -79.65910653054699 > -79.65945018316548
WOLFE (weak): th(111.3981320067067)=-79.65945018316548; dx=-3.5373348118139866E-6 evalInputDelta=4.5534003869818207E-4
New Minimum: -79.65945018316548 > -79.66126292619231
WOLFE (weak): th(556.9906600335335)=-79.66126292619231; dx=-3.5372578962340663E-6 evalInputDelta=0.0022680830655303907
New Minimum: -79.66126292619231 > -79.67260237301872
WOLFE (weak): th(3341.9439602012008)=-79.67260237301872; dx=-3.5358392455619725E-6 evalInputDelta=0.01360752989194225
New Minimum: -79.67260237301872 > -79.75419021571388
WOLFE (weak): th(23393.607721408407)=-79.75419021571388; dx=-3.532325521428496E-6 evalInputDelta=0.09519537258709931
New Minimum: -79.75419021571388 > -80.41419839286294
WOLFE (weak): th(187148.86177126726)=-80.41419839286294; dx=-3.467233012150806E-6 evalInputDelta=0.7552035497361658
New Minimum: -80.41419839286294 > -85.84632158695239
END: th(1684339.7559414054)=-85.84632158695239; dx=-2.813367875338312E-6 evalInputDelta=6.187326743825608
Fitness changed from -79.65899484312678 to -85.84632158695239
Iteration 2 complete. Error: -85.84632158695239 Total: 29.2201; Orientation: 0.0555; Line Search: 25.9219
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7f2839a0 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-85.84632158695239;dx=-2.8024790321125364E-6
New Minimum: -85.84632158695239 > -94.91174267797348
END: th(3628800.0000000005)=-94.91174267797348; dx=-1.682849017831986E-6 evalInputDelta=9.065421091021094
Fitness changed from -85.84632158695239 to -94.91174267797348
Iteration 3 complete. Error: -94.91174267797348 Total: 8.1045; Orientation: 0.0585; Line Search: 5.1529
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 79069d7a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-94.91174267797348;dx=-1.7469349518937678E-6
New Minimum: -94.91174267797348 > -106.1411983172955
END: th(7818012.603187701)=-106.1411983172955; dx=-8.253451522859675E-7 evalInputDelta=11.229455639322026
Fitness changed from -94.91174267797348 to -106.1411983172955
Iteration 4 complete. Error: -106.1411983172955 Total: 8.2930; Orientation: 0.0629; Line Search: 5.0887
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 365e40de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.049e+04, gradient 1.225e-03, dot -0.629; [0dae150c-d4b1-45b8-8c20-8ec63f9624f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -106.1411983172955, -94.91174267797348, -85.84632158695239, -79.65899484312678
LBFGS Accumulation History: 3 points
Removed measurement 365e40de to history. Total: 3
th(0)=-106.1411983172955;dx=-1.4999144727851672E-6
New Minimum: -106.1411983172955 > -112.23814150920434
WOLF (strong): th(1.6843397559414055E7)=-112.23814150920434; dx=1.8587156312933432E-7 evalInputDelta=6.096943191908835
New Minimum: -112.23814150920434 > -112.79346210164465
END: th(8421698.779707028)=-112.79346210164465; dx=-1.7878967274555253E-7 evalInputDelta=6.652263784349145
Fitness changed from -106.1411983172955 to -112.79346210164465
Iteration 5 complete. Error: -112.79346210164465 Total: 11.4456; Orientation: 1.0695; Line Search: 7.5309
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 335765a9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.067e+04, gradient 1.422e-03, dot -0.479; [0dae150c-d4b1-45b8-8c20-8ec63f9624f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -112.79346210164465, -94.91174267797348, -85.84632158695239, -79.65899484312678
LBFGS Accumulation History: 3 points
Removed measurement 335765a9 to history. Total: 3
th(0)=-112.79346210164465;dx=-2.0202292941649126E-6
Armijo: th(1.8144000000000004E7)=-108.3155214459358; dx=7.945053311811711E-7 evalInputDelta=-4.477940655708849
New Minimum: -112.79346210164465 > -116.8810847638482
WOLF (strong): th(9072000.000000002)=-116.8810847638482; dx=6.459984227940174E-7 evalInputDelta=4.087622662203543
New Minimum: -116.8810847638482 > -117.80710905443374
END: th(3024000.0000000005)=-117.80710905443374; dx=-8.400309425947876E-7 evalInputDelta=5.013646952789088
Fitness changed from -112.79346210164465 to -117.80710905443374
Iteration 6 complete. Error: -117.80710905443374 Total: 14.1969; Orientation: 1.0501; Line Search: 10.0461
Adding measurement 27dc0626 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.350e+04, gradient 9.538e-04, dot -0.715; [0dae150c-d4b1-45b8-8c20-8ec63f9624f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -117.80710905443374, -94.91174267797348, -85.84632158695239, -79.65899484312678
LBFGS Accumulation History: 3 points
Removed measurement 27dc0626 to history. Total: 3
th(0)=-117.80710905443374;dx=-9.097794898273595E-7
New Minimum: -117.80710905443374 > -122.68028377095173
END: th(6515010.502656418)=-122.68028377095173; dx=-4.6467742024380595E-7 evalInputDelta=4.873174716517994
Fitness changed from -117.80710905443374 to -122.68028377095173
Iteration 7 complete. Error: -122.68028377095173 Total: 8.9766; Orientation: 1.0579; Line Search: 5.4103
Adding measurement 19e66dc5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.679e+04, gradient 9.914e-04, dot -0.638; [0dae150c-d4b1-45b8-8c20-8ec63f9624f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -122.68028377095173, -94.91174267797348, -85.84632158695239, -79.65899484312678
LBFGS Accumulation History: 3 points
Removed measurement 19e66dc5 to history. Total: 3
th(0)=-122.68028377095173;dx=-9.81125485163369E-7
New Minimum: -122.68028377095173 > -126.59494695874407
WOLF (strong): th(1.4036164632845048E7)=-126.59494695874407; dx=7.147341901588399E-8 evalInputDelta=3.9146631877923426
END: th(7018082.316422524)=-126.58674572270992; dx=-1.6069202915941686E-7 evalInputDelta=3.906461951758189
Fitness changed from -122.68028377095173 to -126.59494695874407
Iteration 8 complete. Error: -126.59494695874407 Total: 11.0935; Orientation: 1.0012; Line Search: 7.6693
Adding measurement 2b9feccd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.479e+04, gradient 1.443e-03, dot -0.350; [0dae150c-d4b1-45b8-8c20-8ec63f9624f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -126.59494695874407, -94.91174267797348, -85.84632158695239, -79.65899484312678
LBFGS Accumulation History: 3 points
Removed measurement 2b9feccd to history. Total: 3
th(0)=-126.59494695874407;dx=-2.080311654104496E-6
Armijo: th(1.5120000000000006E7)=-123.57889900943374; dx=1.2592379489992212E-6 evalInputDelta=-3.016047949310334
New Minimum: -126.59494695874407 > -133.23636934263308
WOLF (strong): th(7560000.000000003)=-133.23636934263308; dx=5.372505605622639E-7 evalInputDelta=6.641422383889008
END: th(2520000.000000001)=-131.506785766714; dx=-1.3060046669509466E-6 evalInputDelta=4.9118388079699145
Fitness changed from -126.59494695874407 to -133.23636934263308
Iteration 9 complete. Error: -133.23636934263308 Total: 13.5932; Orientation: 1.0230; Line Search: 10.1531
Adding measurement 7a0c8c6a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.384e+04, gradient 1.274e-03, dot -0.388; [0dae150c-d4b1-45b8-8c20-8ec63f9624f6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -133.23636934263308, -94.91174267797348, -85.84632158695239, -79.65899484312678
LBFGS Accumulation History: 3 points
Removed measurement 7a0c8c6a to history. Total: 3
th(0)=-133.23636934263308;dx=-1.621343419223666E-6
New Minimum: -133.23636934263308 > -137.87119810735686
WOLF (strong): th(5429175.418880349)=-137.87119810735686; dx=1.3605537644489153E-7 evalInputDelta=4.634828764723778
END: th(2714587.7094401745)=-137.01011288769848; dx=-7.725789705530768E-7 evalInputDelta=3.7737435450654004
Fitness changed from -133.23636934263308 to -137.87119810735686
Iteration 10 complete. Error: -137.87119810735686 Total: 11.0486; Orientation: 1.0241; Line Search: 7.4999
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -137.87119810735686 (> -Infinity) after 129.269s (< 5400.000s)
Returns
-137.87119810735686