BasicOptimizer.scala:89 executed in 282.70 seconds (3.180 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 645807514164000
Reset training subject: 645813442164500
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -37.02403623353592 > -37.06193342258302
th(0)=-37.06193342258302;dx=-1.8601174013785141E-6
New Minimum: -37.06193342258302 > -37.06193546044886
END: th(2.154434690031884)=-37.06193546044886; dx=-1.6357078844524283E-6 evalInputDelta=2.0378658405206806E-6
Fitness changed from -37.02403623353592 to -37.06193546044886
Iteration 1 complete. Error: -37.06193546044886 Total: 29.2481; Orientation: 0.1027; Line Search: 11.7002
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 53a384f to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-37.06193546044886;dx=-1.6381981192697435E-6
New Minimum: -37.06193546044886 > -37.0619436119111
WOLFE (weak): th(4.641588833612779)=-37.0619436119111; dx=-1.6379968253341198E-6 evalInputDelta=8.1514622394252E-6
New Minimum: -37.0619436119111 > -37.06195380123638
WOLFE (weak): th(9.283177667225559)=-37.06195380123638; dx=-1.6378056099191831E-6 evalInputDelta=1.8340787519832702E-5
New Minimum: -37.06195380123638 > -37.06198844492137
WOLFE (weak): th(27.849533001676676)=-37.06198844492137; dx=-1.63649929908141E-6 evalInputDelta=5.298447251078642E-5
New Minimum: -37.06198844492137 > -37.06213924646701
WOLFE (weak): th(111.3981320067067)=-37.06213924646701; dx=-1.6364973388462503E-6 evalInputDelta=2.0378601815451702E-4
New Minimum: -37.06213924646701 > -37.06294826590492
WOLFE (weak): th(556.9906600335335)=-37.06294826590492; dx=-1.636412944572804E-6 evalInputDelta=0.0010128054560638589
New Minimum: -37.06294826590492 > -37.06799557783619
WOLFE (weak): th(3341.9439602012008)=-37.06799557783619; dx=-1.6360783180967121E-6 evalInputDelta=0.00606011738733514
New Minimum: -37.06799557783619 > -37.10430496377776
WOLFE (weak): th(23393.607721408407)=-37.10430496377776; dx=-1.6336155520011274E-6 evalInputDelta=0.04236950332889933
New Minimum: -37.10430496377776 > -37.3987356782246
WOLFE (weak): th(187148.86177126726)=-37.3987356782246; dx=-1.6119531686637413E-6 evalInputDelta=0.336800217775739
New Minimum: -37.3987356782246 > -39.89214432358629
END: th(1684339.7559414054)=-39.89214432358629; dx=-1.3872005747118878E-6 evalInputDelta=2.830208863137429
Fitness changed from -37.06193546044886 to -39.89214432358629
Iteration 2 complete. Error: -39.89214432358629 Total: 65.9717; Orientation: 0.1267; Line Search: 59.0227
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1658b63b to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-39.89214432358629;dx=-1.3138681500922797E-6
New Minimum: -39.89214432358629 > -44.22011229424467
END: th(3628800.0000000005)=-44.22011229424467; dx=-8.775268993439495E-7 evalInputDelta=4.327967970658385
Fitness changed from -39.89214432358629 to -44.22011229424467
Iteration 3 complete. Error: -44.22011229424467 Total: 18.3294; Orientation: 0.1322; Line Search: 11.4408
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 61139da0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-44.22011229424467;dx=-8.413035610318689E-7
New Minimum: -44.22011229424467 > -49.859539701578335
END: th(7818012.603187701)=-49.859539701578335; dx=-4.876714005007742E-7 evalInputDelta=5.639427407333663
Fitness changed from -44.22011229424467 to -49.859539701578335
Iteration 4 complete. Error: -49.859539701578335 Total: 18.5563; Orientation: 0.1284; Line Search: 11.5372
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 59edf544 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.331e+04, gradient 8.095e-04, dot -0.741; [760d7519-3a1f-4223-b529-28f8fbdd335c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -49.859539701578335, -44.22011229424467, -39.89214432358629, -37.06193546044886
LBFGS Accumulation History: 3 points
Removed measurement 59edf544 to history. Total: 3
th(0)=-49.859539701578335;dx=-6.550123221161446E-7
New Minimum: -49.859539701578335 > -56.300192036354574
END: th(1.6843397559414055E7)=-56.300192036354574; dx=-1.781023797235091E-7 evalInputDelta=6.440652334776239
Fitness changed from -49.859539701578335 to -56.300192036354574
Iteration 5 complete. Error: -56.300192036354574 Total: 20.9455; Orientation: 2.5374; Line Search: 11.3141
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1e7f468b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.631e+04, gradient 9.594e-04, dot -0.489; [760d7519-3a1f-4223-b529-28f8fbdd335c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -56.300192036354574, -44.22011229424467, -39.89214432358629, -37.06193546044886
LBFGS Accumulation History: 3 points
Removed measurement 1e7f468b to history. Total: 3
th(0)=-56.300192036354574;dx=-9.200324899975043E-7
Armijo: th(3.628800000000001E7)=-56.04580639062409; dx=1.511891248649308E-7 evalInputDelta=-0.25438564573048694
New Minimum: -56.300192036354574 > -59.9728143933726
WOLF (strong): th(1.8144000000000004E7)=-59.9728143933726; dx=1.8410632576549081E-7 evalInputDelta=3.672622357018028
New Minimum: -59.9728143933726 > -60.56590073421034
END: th(6048000.000000001)=-60.56590073421034; dx=-3.262789240219224E-7 evalInputDelta=4.265708697855764
Fitness changed from -56.300192036354574 to -60.56590073421034
Iteration 6 complete. Error: -60.56590073421034 Total: 32.0085; Orientation: 2.4398; Line Search: 22.5891
Adding measurement e13e620 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.242e+04, gradient 6.996e-04, dot -0.693; [760d7519-3a1f-4223-b529-28f8fbdd335c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -60.56590073421034, -44.22011229424467, -39.89214432358629, -37.06193546044886
LBFGS Accumulation History: 3 points
Removed measurement e13e620 to history. Total: 3
th(0)=-60.56590073421034;dx=-4.889727683865551E-7
New Minimum: -60.56590073421034 > -64.97311459160494
END: th(1.3030021005312836E7)=-64.97311459160494; dx=-1.8233215236443332E-7 evalInputDelta=4.407213857394602
Fitness changed from -60.56590073421034 to -64.97311459160494
Iteration 7 complete. Error: -64.97311459160494 Total: 19.8977; Orientation: 2.3614; Line Search: 11.6465
Adding measurement 9241ba7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.368e+04, gradient 8.033e-04, dot -0.570; [760d7519-3a1f-4223-b529-28f8fbdd335c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -64.97311459160494, -44.22011229424467, -39.89214432358629, -37.06193546044886
LBFGS Accumulation History: 3 points
Removed measurement 9241ba7 to history. Total: 3
th(0)=-64.97311459160494;dx=-6.44633774551522E-7
New Minimum: -64.97311459160494 > -66.69033417701397
WOLF (strong): th(2.8072329265690096E7)=-66.69033417701397; dx=1.052116983145416E-7 evalInputDelta=1.7172195854090262
New Minimum: -66.69033417701397 > -68.33884625555163
WOLF (strong): th(1.4036164632845048E7)=-68.33884625555163; dx=4.643639867733619E-8 evalInputDelta=3.36573166394669
END: th(4678721.544281683)=-67.49566475022753; dx=-3.2968888834264815E-7 evalInputDelta=2.522550158622593
Fitness changed from -64.97311459160494 to -68.33884625555163
Iteration 8 complete. Error: -68.33884625555163 Total: 30.9099; Orientation: 2.3909; Line Search: 23.0376
Adding measurement 4bd18e3d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.545e+04, gradient 9.250e-04, dot -0.447; [760d7519-3a1f-4223-b529-28f8fbdd335c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -68.33884625555163, -44.22011229424467, -39.89214432358629, -37.06193546044886
LBFGS Accumulation History: 3 points
Removed measurement 4bd18e3d to history. Total: 3
th(0)=-68.33884625555163;dx=-8.550793257333606E-7
New Minimum: -68.33884625555163 > -73.09089723033179
WOLF (strong): th(1.0080000000000004E7)=-73.09089723033179; dx=2.7955392805056165E-8 evalInputDelta=4.752050974780161
END: th(5040000.000000002)=-72.04440138205298; dx=-4.477904463098525E-7 evalInputDelta=3.705555126501352
Fitness changed from -68.33884625555163 to -73.09089723033179
Iteration 9 complete. Error: -73.09089723033179 Total: 26.0019; Orientation: 2.4166; Line Search: 17.7271
Adding measurement 5d9dc90b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.247e+04, gradient 7.417e-04, dot -0.556; [760d7519-3a1f-4223-b529-28f8fbdd335c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -73.09089723033179, -44.22011229424467, -39.89214432358629, -37.06193546044886
LBFGS Accumulation History: 3 points
Removed measurement 5d9dc90b to history. Total: 3
th(0)=-73.09089723033179;dx=-5.491917703347155E-7
New Minimum: -73.09089723033179 > -76.27561349787726
END: th(1.0858350837760698E7)=-76.27561349787726; dx=-2.977130168040264E-8 evalInputDelta=3.1847162675454683
Fitness changed from -73.09089723033179 to -76.27561349787726
Iteration 10 complete. Error: -76.27561349787726 Total: 19.8862; Orientation: 2.3738; Line Search: 12.0117
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -76.27561349787726 (> -Infinity) after 282.705s (< 5400.000s)
Returns
-76.27561349787726