BasicOptimizer.scala:89 executed in 8.19 seconds (0.506 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 147164391950900
Reset training subject: 147164522837900
Adding measurement 2b73e2d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.34125632585224847;dx=-1.4649734244817526E-5
New Minimum: -0.34125632585224847 > -0.3412771238598927
WOLFE (weak): th(2.154434690031884)=-0.3412771238598927; dx=-1.469521482954609E-5 evalInputDelta=2.079800764420714E-5
New Minimum: -0.3412771238598927 > -0.3412980395321099
WOLFE (weak): th(4.308869380063768)=-0.3412980395321099; dx=-1.4420036833828487E-5 evalInputDelta=4.17136798614437E-5
New Minimum: -0.3412980395321099 > -0.34138134968397593
WOLFE (weak): th(12.926608140191302)=-0.34138134968397593; dx=-1.4705737787065339E-5 evalInputDelta=1.2502383172746656E-4
New Minimum: -0.34138134968397593 > -0.3417557904385156
WOLFE (weak): th(51.70643256076521)=-0.3417557904385156; dx=-1.4633097791693214E-5 evalInputDelta=4.99464586267151E-4
New Minimum: -0.3417557904385156 > -0.3437577275819712
WOLFE (weak): th(258.53216280382605)=-0.3437577275819712; dx=-1.4720996365964384E-5 evalInputDelta=0.002501401729722741
New Minimum: -0.3437577275819712 > -0.35650602845081003
WOLFE (weak): th(1551.1929768229563)=-0.35650602845081003; dx=-1.4490406927195558E-5 evalInputDelta=0.015249702598561565
New Minimum: -0.35650602845081003 > -0.459351611653774
WOLFE (weak): th(10858.350837760694)=-0.459351611653774; dx=-1.4544251856440553E-5 evalInputDelta=0.1180952858015255
New Minimum: -0.459351611653774 > -0.6734242043185547
END: th(86866.80670208555)=-0.6734242043185547; dx=-2.2067727039618522E-7 evalInputDelta=0.3321678784663062
Fitness changed from -0.34125632585224847 to -0.6734242043185547
Iteration 1 complete. Error: -0.6734242043185547 Total: 1.3742; Orientation: 0.0018; Line Search: 1.0225
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 53b7d818 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.6734242043185547;dx=-2.176412157277299E-5
New Minimum: -0.6734242043185547 > -0.8265330579199642
END: th(187148.86177126726)=-0.8265330579199642; dx=-4.885407188426377E-7 evalInputDelta=0.15310885360140958
Fitness changed from -0.6734242043185547 to -0.8265330579199642
Iteration 2 complete. Error: -0.8265330579199642 Total: 0.3596; Orientation: 0.0015; Line Search: 0.2354
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3710a99b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.8265330579199642;dx=-9.174950078965838E-6
New Minimum: -0.8265330579199642 > -1.8230401815304045
END: th(403200.00000000006)=-1.8230401815304045; dx=-7.815350432002373E-7 evalInputDelta=0.9965071236104402
Fitness changed from -0.8265330579199642 to -1.8230401815304045
Iteration 3 complete. Error: -1.8230401815304045 Total: 0.3595; Orientation: 0.0015; Line Search: 0.2346
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6eaa7222 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.028e+03, gradient 3.461e-03, dot -0.316; [bf4c97ab-3efc-4abb-9d73-009ee3fe463a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8230401815304045, -0.8265330579199642, -0.6734242043185547, -0.34125632585224847
LBFGS Accumulation History: 3 points
Removed measurement 6eaa7222 to history. Total: 3
th(0)=-1.8230401815304045;dx=-1.1780443180527479E-5
New Minimum: -1.8230401815304045 > -4.527843802263235
END: th(868668.0670208557)=-4.527843802263235; dx=-1.0264351910154652E-6 evalInputDelta=2.7048036207328305
Fitness changed from -1.8230401815304045 to -4.527843802263235
Iteration 4 complete. Error: -4.527843802263235 Total: 0.3814; Orientation: 0.0275; Line Search: 0.2299
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1a34b02d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.013e+03, gradient 3.573e-03, dot -0.287; [bf4c97ab-3efc-4abb-9d73-009ee3fe463a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.527843802263235, -0.8265330579199642, -0.6734242043185547, -0.34125632585224847
LBFGS Accumulation History: 3 points
Removed measurement 1a34b02d to history. Total: 3
th(0)=-4.527843802263235;dx=-1.2503751682786625E-5
New Minimum: -4.527843802263235 > -6.102508544930288
WOLF (strong): th(1871488.617712673)=-6.102508544930288; dx=2.716266374251089E-7 evalInputDelta=1.5746647426670526
New Minimum: -6.102508544930288 > -8.16677563320553
END: th(935744.3088563365)=-8.16677563320553; dx=-2.9655818289125773E-7 evalInputDelta=3.638931830942295
Fitness changed from -4.527843802263235 to -8.16677563320553
Iteration 5 complete. Error: -8.16677563320553 Total: 0.4900; Orientation: 0.0316; Line Search: 0.3372
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 32c0bc36 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.788e+04, gradient 3.817e-03, dot -0.285; [bf4c97ab-3efc-4abb-9d73-009ee3fe463a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.16677563320553, -0.8265330579199642, -0.6734242043185547, -0.34125632585224847
LBFGS Accumulation History: 3 points
Removed measurement 32c0bc36 to history. Total: 3
th(0)=-8.16677563320553;dx=-1.4374440367500157E-5
Armijo: th(2016000.0000000007)=-7.726533491496349; dx=1.2827409691672342E-6 evalInputDelta=-0.44024214170918086
New Minimum: -8.16677563320553 > -12.444558523838138
WOLF (strong): th(1008000.0000000003)=-12.444558523838138; dx=9.701484640848603E-7 evalInputDelta=4.277782890632608
New Minimum: -12.444558523838138 > -12.902638868647129
END: th(336000.0000000001)=-12.902638868647129; dx=-5.258493388174324E-6 evalInputDelta=4.735863235441599
Fitness changed from -8.16677563320553 to -12.902638868647129
Iteration 6 complete. Error: -12.902638868647129 Total: 0.6070; Orientation: 0.0280; Line Search: 0.4572
Adding measurement 303184ca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.441e+05, gradient 4.155e-03, dot -0.339; [bf4c97ab-3efc-4abb-9d73-009ee3fe463a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.902638868647129, -0.8265330579199642, -0.6734242043185547, -0.34125632585224847
LBFGS Accumulation History: 3 points
Removed measurement 303184ca to history. Total: 3
th(0)=-12.902638868647129;dx=-1.703082637941272E-5
New Minimum: -12.902638868647129 > -17.083981159750863
WOLF (strong): th(723890.0558507132)=-17.083981159750863; dx=1.3514010860798558E-7 evalInputDelta=4.181342291103734
New Minimum: -17.083981159750863 > -17.51969703982796
END: th(361945.0279253566)=-17.51969703982796; dx=-5.266822073039016E-6 evalInputDelta=4.617058171180831
Fitness changed from -12.902638868647129 to -17.51969703982796
Iteration 7 complete. Error: -17.51969703982796 Total: 0.4868; Orientation: 0.0269; Line Search: 0.3450
Adding measurement 5d271b16 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.384e+04, gradient 4.911e-03, dot 0.430; [bf4c97ab-3efc-4abb-9d73-009ee3fe463a = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-17.51969703982796;dx=-28.994072100400686
Armijo: th(779786.9240469472)=-11.237771820558617; dx=7.336360187009473E-6 evalInputDelta=-6.281925219269343
Armijo: th(389893.4620234736)=-11.237771820558617; dx=1.4669386922269301E-5 evalInputDelta=-6.281925219269343
Armijo: th(129964.48734115786)=-11.237879611457064; dx=4.4153242194814235E-5 evalInputDelta=-6.281817428370896
Armijo: th(32491.121835289465)=-11.238016462137567; dx=1.740169546695818E-4 evalInputDelta=-6.281680577690393
Armijo: th(6498.224367057893)=-11.241683862338856; dx=8.730055046235546E-4 evalInputDelta=-6.278013177489104
Armijo: th(1083.0373945096487)=-11.218942216354302; dx=0.005269281834170314 evalInputDelta=-6.300754823473659
Armijo: th(154.71962778709266)=-11.268539552476154; dx=0.03637061121296477 evalInputDelta=-6.251157487351806
Armijo: th(19.339953473386583)=-11.483868224166164; dx=0.282641922997739 evalInputDelta=-6.035828815661796
Armijo: th(2.148883719265176)=-13.151607001351303; dx=2.147806637857625 evalInputDelta=-4.368090038476657
New Minimum: -17.51969703982796 > -19.28713433153657
END: th(0.2148883719265176)=-19.28713433153657; dx=-7.188103562752822 evalInputDelta=1.7674372917086103
Fitness changed from -17.51969703982796 to -19.28713433153657
Iteration 8 complete. Error: -19.28713433153657 Total: 1.3946; Orientation: 0.0261; Line Search: 1.2543
Adding measurement 3289f2a5 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 7.675e+03, gradient 4.711e-03, dot 0.454; [bf4c97ab-3efc-4abb-9d73-009ee3fe463a = 1.000/1.000e+00]
th(0)=-19.28713433153657;dx=-16.441472566071315
Armijo: th(0.46296296296296313)=-18.65547848300673; dx=-0.4158583734006893 evalInputDelta=-0.6316558485298422
Armijo: th(0.23148148148148157)=-19.06639538798642; dx=-0.8293365988372662 evalInputDelta=-0.220738943550149
Armijo: th(0.07716049382716052)=-19.238167118595218; dx=-1.1397608493845945 evalInputDelta=-0.048967212941352756
Armijo: th(0.01929012345679013)=-19.27717711240319; dx=-1.2697729406684957 evalInputDelta=-0.00995721913337988
Armijo: th(0.0038580246913580258)=-19.285285179334913; dx=-1.2779756671142528 evalInputDelta=-0.001849152201657489
Armijo: th(6.430041152263377E-4)=-19.286830154180386; dx=-1.3500415269250967 evalInputDelta=-3.041773561847094E-4
Armijo: th(9.185773074661967E-5)=-19.287089382553212; dx=-1.2846736642645515 evalInputDelta=-4.494898335849484E-5
Armijo: th(1.1482216343327458E-5)=-19.2871294056257; dx=-1.3496793944076164 evalInputDelta=-4.925910872088934E-6
Armijo: th(1.2758018159252732E-6)=-19.28713433153657; dx=-1.333560537478629 evalInputDelta=0.0
Armijo: th(1.2758018159252732E-7)=-19.28713433153657; dx=-1.3219169253587655 evalInputDelta=0.0
Armijo: th(1.1598198326593393E-8)=-19.28713433153657; dx=-1.2882460561552858 evalInputDelta=0.0
Armijo: th(9.665165272161161E-10)=-19.28713433153657; dx=-1.2730709046303468 evalInputDelta=0.0
MIN ALPHA (7.434742517047047E-11): th(0.0)=-19.28713433153657
Fitness changed from -19.28713433153657 to -19.28713433153657
Static Iteration Total: 2.2898; Orientation: 0.0367; Line Search: 1.6272
Iteration 9 failed. Error: -19.28713433153657
Previous Error: 0.0 -> -19.28713433153657
Retrying iteration 9
Reset training subject: 147172134850600
Adding measurement 6daa095f to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-19.28713433153657;dx=-2.1897772960720973E-5
END: th(2.1544346900318838E-10)=-19.28713433153657; dx=-1.5110490024671127E-5 evalInputDelta=0.0
Fitness changed from -19.28713433153657 to -19.28713433153657
Static Iteration Total: 0.4482; Orientation: 0.0015; Line Search: 0.2251
Iteration 10 failed. Error: -19.28713433153657
Previous Error: 0.0 -> -19.28713433153657
Optimization terminated 10
Final threshold in iteration 10: -19.28713433153657 (> -Infinity) after 8.190s (< 5400.000s)
Returns
-19.28713433153657