BasicOptimizer.scala:89 executed in 2321.82 seconds (9.712 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 19600663508516
Reset training subject: 19649371624454
Adding measurement 3d3e0258 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.3254101872444153;dx=-5.111979886071151E-7
New Minimum: -1.3254101872444153 > -1.3254111409187317
WOLFE (weak): th(2.154434690031884)=-1.3254111409187317; dx=-4.745538763183775E-7 evalInputDelta=9.5367431640625E-7
New Minimum: -1.3254111409187317 > -1.3254124224185944
END: th(4.308869380063768)=-1.3254124224185944; dx=-4.250953784580121E-7 evalInputDelta=2.2351741790771484E-6
Fitness changed from -1.3254101872444153 to -1.3254124224185944
Iteration 1 complete. Error: -1.3254124224185944 Total: 292.3168; Orientation: 0.0864; Line Search: 145.5447
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 34652204 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.3254124224185944;dx=-4.153938321353411E-7
New Minimum: -1.3254124224185944 > -1.3254169523715973
WOLFE (weak): th(9.283177667225559)=-1.3254169523715973; dx=-4.242771639254504E-7 evalInputDelta=4.5299530029296875E-6
New Minimum: -1.3254169523715973 > -1.3254214227199554
WOLFE (weak): th(18.566355334451117)=-1.3254214227199554; dx=-4.2422584678305083E-7 evalInputDelta=9.000301361083984E-6
New Minimum: -1.3254214227199554 > -1.3254394829273224
WOLFE (weak): th(55.69906600335335)=-1.3254394829273224; dx=-4.2411235595199327E-7 evalInputDelta=2.7060508728027344E-5
New Minimum: -1.3254394829273224 > -1.3255198001861572
WOLFE (weak): th(222.7962640134134)=-1.3255198001861572; dx=-4.2617790989072E-7 evalInputDelta=1.0737776756286621E-4
New Minimum: -1.3255198001861572 > -1.3259493708610535
WOLFE (weak): th(1113.981320067067)=-1.3259493708610535; dx=-4.239973856887634E-7 evalInputDelta=5.369484424591064E-4
New Minimum: -1.3259493708610535 > -1.3286321461200714
WOLFE (weak): th(6683.8879204024015)=-1.3286321461200714; dx=-4.233122374910537E-7 evalInputDelta=0.0032197237014770508
New Minimum: -1.3286321461200714 > -1.3478819131851196
WOLFE (weak): th(46787.215442816814)=-1.3478819131851196; dx=-4.2105717794609606E-7 evalInputDelta=0.02246949076652527
New Minimum: -1.3478819131851196 > -1.5002734661102295
WOLFE (weak): th(374297.7235425345)=-1.5002734661102295; dx=-3.982616130981318E-7 evalInputDelta=0.17486104369163513
New Minimum: -1.5002734661102295 > -2.4757286608219147
END: th(3368679.511882811)=-2.4757286608219147; dx=-1.9192145618971249E-7 evalInputDelta=1.1503162384033203
Fitness changed from -1.3254124224185944 to -2.4757286608219147
Iteration 2 complete. Error: -2.4757286608219147 Total: 541.1854; Orientation: 0.0914; Line Search: 490.7812
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3bdb062b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.4757286608219147;dx=-1.7758556280772266E-7
New Minimum: -2.4757286608219147 > -3.563203454017639
END: th(7257600.000000001)=-3.563203454017639; dx=-8.889058966856021E-8 evalInputDelta=1.0874747931957245
Fitness changed from -2.4757286608219147 to -3.563203454017639
Iteration 3 complete. Error: -3.563203454017639 Total: 148.0815; Orientation: 0.0879; Line Search: 98.0673
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 20584a21 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.569e+03, gradient 3.315e-04, dot -0.541; [99321f5c-3d09-497c-aedd-d4e060235a7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.563203454017639, -2.4757286608219147, -1.3254124224185944, -1.3254101872444153
LBFGS Accumulation History: 3 points
Removed measurement 20584a21 to history. Total: 3
th(0)=-3.563203454017639;dx=-9.959679244003421E-8
New Minimum: -3.563203454017639 > -4.710757836699486
END: th(1.5636025206375401E7)=-4.710757836699486; dx=-3.667668593243921E-8 evalInputDelta=1.1475543826818466
Fitness changed from -3.563203454017639 to -4.710757836699486
Iteration 4 complete. Error: -4.710757836699486 Total: 150.1490; Orientation: 2.2296; Line Search: 98.3617
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5b2c7228 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.510e+03, gradient 2.679e-04, dot -0.497; [99321f5c-3d09-497c-aedd-d4e060235a7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.710757836699486, -2.4757286608219147, -1.3254124224185944, -1.3254101872444153
LBFGS Accumulation History: 3 points
Removed measurement 5b2c7228 to history. Total: 3
th(0)=-4.710757836699486;dx=-6.791678966858713E-8
New Minimum: -4.710757836699486 > -5.511871352791786
END: th(3.368679511882811E7)=-5.511871352791786; dx=-4.439867363858144E-9 evalInputDelta=0.8011135160923004
Fitness changed from -4.710757836699486 to -5.511871352791786
Iteration 5 complete. Error: -5.511871352791786 Total: 150.7359; Orientation: 2.3105; Line Search: 98.8007
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 8f6cbea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.462e+03, gradient 2.457e-04, dot -0.239; [99321f5c-3d09-497c-aedd-d4e060235a7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.511871352791786, -2.4757286608219147, -1.3254124224185944, -1.3254101872444153
LBFGS Accumulation History: 3 points
Removed measurement 8f6cbea to history. Total: 3
th(0)=-5.511871352791786;dx=-6.568703738557411E-8
New Minimum: -5.511871352791786 > -5.596253365278244
WOLF (strong): th(7.257600000000001E7)=-5.596253365278244; dx=9.766306769513803E-9 evalInputDelta=0.08438201248645782
New Minimum: -5.596253365278244 > -5.953165948390961
WOLF (strong): th(3.628800000000001E7)=-5.953165948390961; dx=1.8041755883287805E-9 evalInputDelta=0.4412945955991745
END: th(1.2096000000000002E7)=-5.919124081730843; dx=-5.053892183680132E-9 evalInputDelta=0.4072527289390564
Fitness changed from -5.511871352791786 to -5.953165948390961
Iteration 6 complete. Error: -5.953165948390961 Total: 246.7643; Orientation: 2.2141; Line Search: 194.4159
Adding measurement 52dde92b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.864e+03, gradient 3.025e-04, dot -0.117; [99321f5c-3d09-497c-aedd-d4e060235a7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.953165948390961, -2.4757286608219147, -1.3254124224185944, -1.3254101872444153
LBFGS Accumulation History: 3 points
Removed measurement 52dde92b to history. Total: 3
th(0)=-5.953165948390961;dx=-9.147955060458068E-8
New Minimum: -5.953165948390961 > -6.196313679218292
WOLF (strong): th(2.606004201062567E7)=-6.196313679218292; dx=1.2064219743623044E-8 evalInputDelta=0.24314773082733154
New Minimum: -6.196313679218292 > -6.315649017691612
WOLF (strong): th(1.3030021005312836E7)=-6.315649017691612; dx=1.1971477227950891E-9 evalInputDelta=0.36248306930065155
END: th(4343340.335104278)=-6.2365018129348755; dx=-2.906414400425428E-8 evalInputDelta=0.2833358645439148
Fitness changed from -5.953165948390961 to -6.315649017691612
Iteration 7 complete. Error: -6.315649017691612 Total: 247.5462; Orientation: 2.2043; Line Search: 197.1119
Adding measurement 11ff9f77 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.039e+03, gradient 1.764e-04, dot -0.203; [99321f5c-3d09-497c-aedd-d4e060235a7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.315649017691612, -2.4757286608219147, -1.3254124224185944, -1.3254101872444153
LBFGS Accumulation History: 3 points
Removed measurement 11ff9f77 to history. Total: 3
th(0)=-6.315649017691612;dx=-3.088661142947974E-8
New Minimum: -6.315649017691612 > -6.4976866990327835
END: th(9357443.088563364)=-6.4976866990327835; dx=-4.058178311853192E-9 evalInputDelta=0.18203768134117126
Fitness changed from -6.315649017691612 to -6.4976866990327835
Iteration 8 complete. Error: -6.4976866990327835 Total: 148.7152; Orientation: 2.2883; Line Search: 98.2641
Adding measurement 485e7015 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.756e+03, gradient 1.302e-04, dot -0.349; [99321f5c-3d09-497c-aedd-d4e060235a7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.4976866990327835, -2.4757286608219147, -1.3254124224185944, -1.3254101872444153
LBFGS Accumulation History: 3 points
Removed measurement 485e7015 to history. Total: 3
th(0)=-6.4976866990327835;dx=-1.685118040974193E-8
New Minimum: -6.4976866990327835 > -6.64922671020031
END: th(2.0160000000000004E7)=-6.64922671020031; dx=-2.1283098649432076E-9 evalInputDelta=0.15154001116752625
Fitness changed from -6.4976866990327835 to -6.64922671020031
Iteration 9 complete. Error: -6.64922671020031 Total: 147.9792; Orientation: 2.2818; Line Search: 96.8619
Adding measurement 411c451c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.107e+03, gradient 1.434e-04, dot -0.326; [99321f5c-3d09-497c-aedd-d4e060235a7c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.64922671020031, -2.4757286608219147, -1.3254124224185944, -1.3254101872444153
LBFGS Accumulation History: 3 points
Removed measurement 411c451c to history. Total: 3
th(0)=-6.64922671020031;dx=-2.14486143442469E-8
Armijo: th(4.3433403351042785E7)=-6.560642093420029; dx=2.5298406800440455E-9 evalInputDelta=-0.08858461678028107
New Minimum: -6.64922671020031 > -6.704861491918564
WOLF (strong): th(2.1716701675521392E7)=-6.704861491918564; dx=7.5356965992783E-9 evalInputDelta=0.05563478171825409
New Minimum: -6.704861491918564 > -6.760739415884018
END: th(7238900.5585071305)=-6.760739415884018; dx=-6.437465072964215E-9 evalInputDelta=0.11151270568370819
Fitness changed from -6.64922671020031 to -6.760739415884018
Iteration 10 complete. Error: -6.760739415884018 Total: 247.1903; Orientation: 2.2576; Line Search: 195.8507
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -6.760739415884018 (> -Infinity) after 2321.822s (< 5400.000s)
Returns
-6.760739415884018