BasicOptimizer.scala:89 executed in 69.65 seconds (0.824 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 138399797433800
Reset training subject: 138401121014800
Adding measurement 50faa9d2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-11.768513480720427;dx=-0.03287096192609992
New Minimum: -11.768513480720427 > -11.827307729567318
WOLFE (weak): th(2.154434690031884)=-11.827307729567318; dx=-0.03255121096262363 evalInputDelta=0.05879424884689044
New Minimum: -11.827307729567318 > -11.885997939655061
WOLFE (weak): th(4.308869380063768)=-11.885997939655061; dx=-0.032701624411889084 evalInputDelta=0.11748445893463355
New Minimum: -11.885997939655061 > -12.118568208933361
WOLFE (weak): th(12.926608140191302)=-12.118568208933361; dx=-0.03250748170362881 evalInputDelta=0.3500547282129336
New Minimum: -12.118568208933361 > -13.151346556903047
WOLFE (weak): th(51.70643256076521)=-13.151346556903047; dx=-0.032226940055946666 evalInputDelta=1.382833076182619
New Minimum: -13.151346556903047 > -18.594750334173202
WOLFE (weak): th(258.53216280382605)=-18.594750334173202; dx=-0.03260264577437857 evalInputDelta=6.826236853452775
New Minimum: -18.594750334173202 > -39.663493908422105
END: th(1551.1929768229563)=-39.663493908422105; dx=-0.013407856957325003 evalInputDelta=27.894980427701675
Fitness changed from -11.768513480720427 to -39.663493908422105
Iteration 1 complete. Error: -39.663493908422105 Total: 11.6733; Orientation: 0.1068; Line Search: 8.1805
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7db047f9 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-39.663493908422105;dx=-0.05115522510126027
New Minimum: -39.663493908422105 > -52.659126868740664
END: th(3341.943960201201)=-52.659126868740664; dx=-0.0033538975271458302 evalInputDelta=12.99563296031856
Fitness changed from -39.663493908422105 to -52.659126868740664
Iteration 2 complete. Error: -52.659126868740664 Total: 4.1163; Orientation: 0.0152; Line Search: 2.3759
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 33f7900e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-52.659126868740664;dx=-0.07757560663655833
New Minimum: -52.659126868740664 > -58.37673562061131
END: th(7200.000000000001)=-58.37673562061131; dx=-3.5866789053662926E-4 evalInputDelta=5.7176087518706495
Fitness changed from -52.659126868740664 to -58.37673562061131
Iteration 3 complete. Error: -58.37673562061131 Total: 2.9018; Orientation: 0.0179; Line Search: 1.9037
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 206102e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.556e+03, gradient 1.943e-01, dot -0.163; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -58.37673562061131, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 206102e3 to history. Total: 3
th(0)=-58.37673562061131;dx=-0.03710589936149861
New Minimum: -58.37673562061131 > -102.50238183208232
WOLF (strong): th(15511.929768229566)=-102.50238183208232; dx=0.0015645895870338789 evalInputDelta=44.12564621147101
New Minimum: -102.50238183208232 > -136.89910603935658
END: th(7755.964884114783)=-136.89910603935658; dx=-0.0011433149512685926 evalInputDelta=78.52237041874527
Fitness changed from -58.37673562061131 to -136.89910603935658
Iteration 4 complete. Error: -136.89910603935658 Total: 5.0325; Orientation: 0.2834; Line Search: 3.2080
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 18a3661c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.136e+03, gradient 1.792e-01, dot -0.156; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -136.89910603935658, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 18a3661c to history. Total: 3
th(0)=-136.89910603935658;dx=-0.03188188477511012
New Minimum: -136.89910603935658 > -188.9731710455217
WOLF (strong): th(16709.71980100601)=-188.9731710455217; dx=0.003658214583791329 evalInputDelta=52.074065006165114
New Minimum: -188.9731710455217 > -236.04697791170722
END: th(8354.859900503005)=-236.04697791170722; dx=-0.003823790704908552 evalInputDelta=99.14787187235063
Fitness changed from -136.89910603935658 to -236.04697791170722
Iteration 5 complete. Error: -236.04697791170722 Total: 5.6375; Orientation: 0.5059; Line Search: 3.5556
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 36653a8d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.090e+03, gradient 1.823e-01, dot -0.170; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -236.04697791170722, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 36653a8d to history. Total: 3
th(0)=-236.04697791170722;dx=-0.03303367029218289
New Minimum: -236.04697791170722 > -257.6890234614195
WOLF (strong): th(18000.000000000007)=-257.6890234614195; dx=0.003339958653941692 evalInputDelta=21.642045549712265
New Minimum: -257.6890234614195 > -329.18796872000047
END: th(9000.000000000004)=-329.18796872000047; dx=-0.0023867947205437495 evalInputDelta=93.14099080829325
Fitness changed from -236.04697791170722 to -329.18796872000047
Iteration 6 complete. Error: -329.18796872000047 Total: 4.8310; Orientation: 0.3537; Line Search: 3.5515
Adding measurement 7ea66cad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.237e+03, gradient 2.021e-01, dot -0.162; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -329.18796872000047, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 7ea66cad to history. Total: 3
th(0)=-329.18796872000047;dx=-0.040525412392025784
Armijo: th(19389.912210286962)=-272.16213285925727; dx=0.004154153050731397 evalInputDelta=-57.0258358607432
New Minimum: -329.18796872000047 > -407.2382786833772
WOLF (strong): th(9694.956105143481)=-407.2382786833772; dx=0.0013638740540564444 evalInputDelta=78.05030996337672
New Minimum: -407.2382786833772 > -415.9183739028609
END: th(3231.652035047827)=-415.9183739028609; dx=-0.018744546700756797 evalInputDelta=86.73040518286041
Fitness changed from -329.18796872000047 to -415.9183739028609
Iteration 7 complete. Error: -415.9183739028609 Total: 4.7968; Orientation: 0.2798; Line Search: 3.6493
Adding measurement 1247437d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.919e+03, gradient 1.618e-01, dot -0.200; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -415.9183739028609, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 1247437d to history. Total: 3
th(0)=-415.9183739028609;dx=-0.02598951504538119
New Minimum: -415.9183739028609 > -490.7665082551987
END: th(6962.383250419172)=-490.7665082551987; dx=-0.009163792228272556 evalInputDelta=74.84813435233781
Fitness changed from -415.9183739028609 to -490.7665082551987
Iteration 8 complete. Error: -490.7665082551987 Total: 3.0885; Orientation: 0.2742; Line Search: 1.9950
Adding measurement 4204cc2e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.386e+03, gradient 1.769e-01, dot -0.210; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -490.7665082551987, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 4204cc2e to history. Total: 3
th(0)=-490.7665082551987;dx=-0.0311396011203215
Armijo: th(15000.00000000001)=-473.3751650382601; dx=3.0647830964083457E-4 evalInputDelta=-17.391343216938594
New Minimum: -490.7665082551987 > -538.0174544101706
END: th(7500.000000000005)=-538.0174544101706; dx=-0.002601418809098319 evalInputDelta=47.25094615497187
Fitness changed from -490.7665082551987 to -538.0174544101706
Iteration 9 complete. Error: -538.0174544101706 Total: 3.7490; Orientation: 0.2653; Line Search: 2.6049
Adding measurement 694ec4d7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.720e+03, gradient 1.858e-01, dot -0.224; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -538.0174544101706, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 694ec4d7 to history. Total: 3
th(0)=-538.0174544101706;dx=-0.034176696690467176
Armijo: th(16158.260175239138)=-446.6810418240291; dx=0.007256466798684767 evalInputDelta=-91.33641258614148
New Minimum: -538.0174544101706 > -588.9697572880971
WOLF (strong): th(8079.130087619569)=-588.9697572880971; dx=0.0022147749591922856 evalInputDelta=50.95230287792651
New Minimum: -588.9697572880971 > -603.3367892396418
END: th(2693.0433625398564)=-603.3367892396418; dx=-0.018853825539998023 evalInputDelta=65.31933482947125
Fitness changed from -538.0174544101706 to -603.3367892396418
Iteration 10 complete. Error: -603.3367892396418 Total: 5.0393; Orientation: 0.2679; Line Search: 3.9624
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement b9a796 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.722e+03, gradient 1.605e-01, dot -0.277; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -603.3367892396418, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement b9a796 to history. Total: 3
th(0)=-603.3367892396418;dx=-0.025604915059354923
New Minimum: -603.3367892396418 > -649.3188738978715
END: th(5801.986042015978)=-649.3188738978715; dx=-0.006423873762950687 evalInputDelta=45.982084658229724
Fitness changed from -603.3367892396418 to -649.3188738978715
Iteration 11 complete. Error: -649.3188738978715 Total: 3.0621; Orientation: 0.2985; Line Search: 1.7437
Adding measurement 3669422e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.069e+04, gradient 1.862e-01, dot -0.242; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -649.3188738978715, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 3669422e to history. Total: 3
th(0)=-649.3188738978715;dx=-0.034457749337857356
Armijo: th(12500.00000000001)=-610.2800586615951; dx=0.0027093901027018742 evalInputDelta=-39.03881523627638
New Minimum: -649.3188738978715 > -680.163261651201
END: th(6250.000000000005)=-680.163261651201; dx=-4.13286171321114E-4 evalInputDelta=30.844387753329443
Fitness changed from -649.3188738978715 to -680.163261651201
Iteration 12 complete. Error: -680.163261651201 Total: 4.1155; Orientation: 0.2846; Line Search: 2.9942
Adding measurement 56ab1dda to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.528e+04, gradient 2.044e-01, dot -0.237; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -680.163261651201, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 56ab1dda to history. Total: 3
th(0)=-680.163261651201;dx=-0.04142694636522629
Armijo: th(13465.216812699284)=-564.5157770160192; dx=0.007132913376181788 evalInputDelta=-115.64748463518174
New Minimum: -680.163261651201 > -707.0979422965393
WOLF (strong): th(6732.608406349642)=-707.0979422965393; dx=0.007406088960757116 evalInputDelta=26.93468064533829
New Minimum: -707.0979422965393 > -745.1096815234654
END: th(2244.2028021165474)=-745.1096815234654; dx=-0.020338280942116108 evalInputDelta=64.94641987226441
Fitness changed from -680.163261651201 to -745.1096815234654
Iteration 13 complete. Error: -745.1096815234654 Total: 4.6528; Orientation: 0.2630; Line Search: 3.5578
Adding measurement 3f9bc9bd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.310e+04, gradient 1.658e-01, dot -0.299; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -745.1096815234654, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement 3f9bc9bd to history. Total: 3
th(0)=-745.1096815234654;dx=-0.02712080008092791
New Minimum: -745.1096815234654 > -782.5692461373626
END: th(4834.988368346649)=-782.5692461373626; dx=-0.006408484801572902 evalInputDelta=37.459564613897214
Fitness changed from -745.1096815234654 to -782.5692461373626
Iteration 14 complete. Error: -782.5692461373626 Total: 2.9522; Orientation: 0.3718; Line Search: 1.7623
Adding measurement dd9784f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.007e+04, gradient 1.815e-01, dot -0.279; [0e97582d-ff5b-468e-84df-bde4887429b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -782.5692461373626, -52.659126868740664, -39.663493908422105, -11.768513480720427
LBFGS Accumulation History: 3 points
Removed measurement dd9784f to history. Total: 3
th(0)=-782.5692461373626;dx=-0.03274156315496296
Armijo: th(10416.666666666675)=-752.7164555793902; dx=0.001279347098527296 evalInputDelta=-29.852790557972412
New Minimum: -782.5692461373626 > -810.5197021047669
END: th(5208.333333333338)=-810.5197021047669; dx=-0.0024321333218626064 evalInputDelta=27.95045596740431
Fitness changed from -782.5692461373626 to -810.5197021047669
Iteration 15 complete. Error: -810.5197021047669 Total: 3.8901; Orientation: 0.2561; Line Search: 2.7489
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -810.5197021047669 (> -Infinity) after 69.646s (< 5400.000s)
Returns
-810.5197021047669