BasicOptimizer.scala:89 executed in 205.99 seconds (1.641 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 4909080356671
Reset training subject: 4913532464207
Adding measurement 73e00683 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.1918361932039261;dx=-5.603095210300676E-7
New Minimum: -0.1918361932039261 > -0.19183631241321564
WOLFE (weak): th(2.154434690031884)=-0.19183631241321564; dx=-5.20016366009445E-7 evalInputDelta=1.1920928955078125E-7
New Minimum: -0.19183631241321564 > -0.19183681905269623
WOLFE (weak): th(4.308869380063768)=-0.19183681905269623; dx=-5.200161537425692E-7 evalInputDelta=6.258487701416016E-7
New Minimum: -0.19183681905269623 > -0.19183950126171112
WOLFE (weak): th(12.926608140191302)=-0.19183950126171112; dx=-5.200137514639148E-7 evalInputDelta=3.3080577850341797E-6
New Minimum: -0.19183950126171112 > -0.19184976816177368
WOLFE (weak): th(51.70643256076521)=-0.19184976816177368; dx=-5.200070917791984E-7 evalInputDelta=1.3574957847595215E-5
New Minimum: -0.19184976816177368 > -0.1919035166501999
WOLFE (weak): th(258.53216280382605)=-0.1919035166501999; dx=-5.19960826392722E-7 evalInputDelta=6.732344627380371E-5
New Minimum: -0.1919035166501999 > -0.19223956763744354
WOLFE (weak): th(1551.1929768229563)=-0.19223956763744354; dx=-5.197779159666063E-7 evalInputDelta=4.0337443351745605E-4
New Minimum: -0.19223956763744354 > -0.19465571641921997
WOLFE (weak): th(10858.350837760694)=-0.19465571641921997; dx=-5.186678185413323E-7 evalInputDelta=0.0028195232152938843
New Minimum: -0.19465571641921997 > -0.21417365968227386
WOLFE (weak): th(86866.80670208555)=-0.21417365968227386; dx=-5.08348117653768E-7 evalInputDelta=0.02233746647834778
New Minimum: -0.21417365968227386 > -0.3734990805387497
END: th(781801.26031877)=-0.3734990805387497; dx=-4.0685754912705217E-7 evalInputDelta=0.1816628873348236
Fitness changed from -0.1918361932039261 to -0.3734990805387497
Iteration 1 complete. Error: -0.3734990805387497 Total: 56.5919; Orientation: 0.0429; Line Search: 43.6697
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4195574c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.3734990805387497;dx=-3.6070085142445537E-7
New Minimum: -0.3734990805387497 > -0.6251216530799866
END: th(1684339.7559414052)=-0.6251216530799866; dx=-2.397307132348568E-7 evalInputDelta=0.2516225725412369
Fitness changed from -0.3734990805387497 to -0.6251216530799866
Iteration 2 complete. Error: -0.6251216530799866 Total: 12.7056; Orientation: 0.0441; Line Search: 8.5686
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1677bc85 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.6251216530799866;dx=-2.1538494926337681E-7
New Minimum: -0.6251216530799866 > -0.9219798445701599
END: th(3628800.0)=-0.9219798445701599; dx=-1.206056611039973E-7 evalInputDelta=0.29685819149017334
Fitness changed from -0.6251216530799866 to -0.9219798445701599
Iteration 3 complete. Error: -0.9219798445701599 Total: 13.0561; Orientation: 0.0442; Line Search: 8.5438
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1dfd105f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.288e+03, gradient 3.609e-04, dot -0.825; [c4c2b6ce-572d-4cb0-977b-ed85a5377c35 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.9219798445701599, -0.6251216530799866, -0.3734990805387497, -0.1918361932039261
LBFGS Accumulation History: 3 points
Removed measurement 1dfd105f to history. Total: 3
th(0)=-0.9219798445701599;dx=-1.3024108441666673E-7
New Minimum: -0.9219798445701599 > -1.2249771654605865
END: th(7818012.6031877)=-1.2249771654605865; dx=-4.3573421990639126E-8 evalInputDelta=0.30299732089042664
Fitness changed from -0.9219798445701599 to -1.2249771654605865
Iteration 4 complete. Error: -1.2249771654605865 Total: 13.4226; Orientation: 1.0413; Line Search: 8.2354
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3af5967d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.745e+03, gradient 3.222e-04, dot -0.601; [c4c2b6ce-572d-4cb0-977b-ed85a5377c35 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.2249771654605865, -0.6251216530799866, -0.3734990805387497, -0.1918361932039261
LBFGS Accumulation History: 3 points
Removed measurement 3af5967d to history. Total: 3
th(0)=-1.2249771654605865;dx=-1.0378526303664336E-7
New Minimum: -1.2249771654605865 > -1.3801011741161346
WOLF (strong): th(1.684339755941405E7)=-1.3801011741161346; dx=1.4090216910163605E-8 evalInputDelta=0.1551240086555481
New Minimum: -1.3801011741161346 > -1.4064642637968063
END: th(8421698.779707026)=-1.4064642637968063; dx=-4.995309626314535E-9 evalInputDelta=0.1814870983362198
Fitness changed from -1.2249771654605865 to -1.4064642637968063
Iteration 5 complete. Error: -1.4064642637968063 Total: 18.5672; Orientation: 0.9952; Line Search: 13.0896
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1b8e08ff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.834e+03, gradient 3.077e-04, dot -0.500; [c4c2b6ce-572d-4cb0-977b-ed85a5377c35 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.4064642637968063, -0.6251216530799866, -0.3734990805387497, -0.1918361932039261
LBFGS Accumulation History: 3 points
Removed measurement 1b8e08ff to history. Total: 3
th(0)=-1.4064642637968063;dx=-9.467732467822297E-8
New Minimum: -1.4064642637968063 > -1.4143634289503098
WOLF (strong): th(1.8144E7)=-1.4143634289503098; dx=3.5017366316618665E-8 evalInputDelta=0.007899165153503418
New Minimum: -1.4143634289503098 > -1.546528398990631
WOLF (strong): th(9072000.0)=-1.546528398990631; dx=1.4828484688220408E-8 evalInputDelta=0.14006413519382477
END: th(3024000.0)=-1.509943351149559; dx=-4.402886924855028E-8 evalInputDelta=0.10347908735275269
Fitness changed from -1.4064642637968063 to -1.546528398990631
Iteration 6 complete. Error: -1.546528398990631 Total: 23.9237; Orientation: 0.9795; Line Search: 18.4832
Adding measurement 2e726bf5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.583e+03, gradient 2.905e-04, dot -0.437; [c4c2b6ce-572d-4cb0-977b-ed85a5377c35 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.546528398990631, -0.6251216530799866, -0.3734990805387497, -0.1918361932039261
LBFGS Accumulation History: 3 points
Removed measurement 2e726bf5 to history. Total: 3
th(0)=-1.546528398990631;dx=-8.440021931711724E-8
New Minimum: -1.546528398990631 > -1.6708109229803085
END: th(6515010.502656417)=-1.6708109229803085; dx=-4.9655053515980186E-9 evalInputDelta=0.12428252398967743
Fitness changed from -1.546528398990631 to -1.6708109229803085
Iteration 7 complete. Error: -1.6708109229803085 Total: 13.3351; Orientation: 1.0097; Line Search: 8.3033
Adding measurement 7102c763 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.903e+03, gradient 1.937e-04, dot -0.550; [c4c2b6ce-572d-4cb0-977b-ed85a5377c35 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.6708109229803085, -0.6251216530799866, -0.3734990805387497, -0.1918361932039261
LBFGS Accumulation History: 3 points
Removed measurement 7102c763 to history. Total: 3
th(0)=-1.6708109229803085;dx=-3.75314894914963E-8
New Minimum: -1.6708109229803085 > -1.756554290652275
WOLF (strong): th(1.4036164632845046E7)=-1.756554290652275; dx=8.958565758788899E-9 evalInputDelta=0.08574336767196655
END: th(7018082.316422523)=-1.7539434731006622; dx=-1.148620623954546E-8 evalInputDelta=0.0831325501203537
Fitness changed from -1.6708109229803085 to -1.756554290652275
Iteration 8 complete. Error: -1.756554290652275 Total: 17.8467; Orientation: 0.9899; Line Search: 12.4642
Adding measurement 1585a3d5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.460e+03, gradient 2.679e-04, dot -0.359; [c4c2b6ce-572d-4cb0-977b-ed85a5377c35 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.756554290652275, -0.6251216530799866, -0.3734990805387497, -0.1918361932039261
LBFGS Accumulation History: 3 points
Removed measurement 1585a3d5 to history. Total: 3
th(0)=-1.756554290652275;dx=-7.179291461974922E-8
Armijo: th(1.5120000000000004E7)=-1.7330758571624756; dx=3.740259667241481E-8 evalInputDelta=-0.0234784334897995
New Minimum: -1.756554290652275 > -1.841438576579094
WOLF (strong): th(7560000.000000002)=-1.841438576579094; dx=1.2323873048816998E-8 evalInputDelta=0.08488428592681885
END: th(2520000.0000000005)=-1.8216068893671036; dx=-3.267886507401299E-8 evalInputDelta=0.06505259871482849
Fitness changed from -1.756554290652275 to -1.841438576579094
Iteration 9 complete. Error: -1.841438576579094 Total: 22.7847; Orientation: 1.0353; Line Search: 16.8115
Adding measurement 527cd034 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.695e+03, gradient 1.853e-04, dot -0.433; [c4c2b6ce-572d-4cb0-977b-ed85a5377c35 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.841438576579094, -0.6251216530799866, -0.3734990805387497, -0.1918361932039261
LBFGS Accumulation History: 3 points
Removed measurement 527cd034 to history. Total: 3
th(0)=-1.841438576579094;dx=-3.433392493654147E-8
New Minimum: -1.841438576579094 > -1.900129497051239
END: th(5429175.418880348)=-1.900129497051239; dx=-1.029408719437128E-8 evalInputDelta=0.05869092047214508
Fitness changed from -1.841438576579094 to -1.900129497051239
Iteration 10 complete. Error: -1.900129497051239 Total: 13.6819; Orientation: 0.9884; Line Search: 8.7842
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -1.900129497051239 (> -Infinity) after 205.987s (< 5400.000s)
Returns
-1.900129497051239