BasicOptimizer.scala:89 executed in 299.85 seconds (4.389 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 659975106925900
Reset training subject: 659981231673900
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -34.62202265695267 > -34.66923368990335
th(0)=-34.66923368990335;dx=-1.9140170611204487E-6
New Minimum: -34.66923368990335 > -34.66923695766724
END: th(2.154434690031884)=-34.66923695766724; dx=-1.72091531050065E-6 evalInputDelta=3.2677638870381998E-6
Fitness changed from -34.62202265695267 to -34.66923695766724
Iteration 1 complete. Error: -34.66923695766724 Total: 30.8832; Orientation: 0.0953; Line Search: 12.3196
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement df53d48 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-34.66923695766724;dx=-1.7253519919833015E-6
New Minimum: -34.66923695766724 > -34.66924676095706
WOLFE (weak): th(4.641588833612779)=-34.66924676095706; dx=-1.7248195164258016E-6 evalInputDelta=9.803289820808914E-6
New Minimum: -34.66924676095706 > -34.66925656424411
WOLFE (weak): th(9.283177667225559)=-34.66925656424411; dx=-1.724369912258915E-6 evalInputDelta=1.9606576870501158E-5
New Minimum: -34.66925656424411 > -34.66929142035338
WOLFE (weak): th(27.849533001676676)=-34.66929142035338; dx=-1.7228340493563129E-6 evalInputDelta=5.4462686144063355E-5
New Minimum: -34.66929142035338 > -34.669452629402926
WOLFE (weak): th(111.3981320067067)=-34.669452629402926; dx=-1.7221414585265293E-6 evalInputDelta=2.1567173568826092E-4
New Minimum: -34.669452629402926 > -34.67030767838377
WOLFE (weak): th(556.9906600335335)=-34.67030767838377; dx=-1.7227524366950323E-6 evalInputDelta=0.0010707207165339128
New Minimum: -34.67030767838377 > -34.675658607932434
WOLFE (weak): th(3341.9439602012008)=-34.675658607932434; dx=-1.7209760423923762E-6 evalInputDelta=0.006421650265195922
New Minimum: -34.675658607932434 > -34.714129840039405
WOLFE (weak): th(23393.607721408407)=-34.714129840039405; dx=-1.7199167126542672E-6 evalInputDelta=0.044892882372167264
New Minimum: -34.714129840039405 > -35.02622236988909
WOLFE (weak): th(187148.86177126726)=-35.02622236988909; dx=-1.6990300410281693E-6 evalInputDelta=0.356985412221853
New Minimum: -35.02622236988909 > -37.66601038084862
END: th(1684339.7559414054)=-37.66601038084862; dx=-1.456513651465291E-6 evalInputDelta=2.9967734231813807
Fitness changed from -34.66923695766724 to -37.66601038084862
Iteration 2 complete. Error: -37.66601038084862 Total: 68.7933; Orientation: 0.1350; Line Search: 61.3626
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement d981137 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-37.66601038084862;dx=-1.3676697644554332E-6
New Minimum: -37.66601038084862 > -42.18998832382227
END: th(3628800.0000000005)=-42.18998832382227; dx=-9.011772655779947E-7 evalInputDelta=4.5239779429736515
Fitness changed from -37.66601038084862 to -42.18998832382227
Iteration 3 complete. Error: -42.18998832382227 Total: 20.7422; Orientation: 0.1494; Line Search: 13.4130
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7453d0b2 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-42.18998832382227;dx=-8.673517559140779E-7
New Minimum: -42.18998832382227 > -47.94885028422188
END: th(7818012.603187701)=-47.94885028422188; dx=-4.884008405865709E-7 evalInputDelta=5.7588619603996065
Fitness changed from -42.18998832382227 to -47.94885028422188
Iteration 4 complete. Error: -47.94885028422188 Total: 19.3583; Orientation: 0.1393; Line Search: 12.2343
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7f51b95 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.333e+04, gradient 8.309e-04, dot -0.731; [345419b7-5ff6-4282-96e7-375459fd911b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -47.94885028422188, -42.18998832382227, -37.66601038084862, -34.66923695766724
LBFGS Accumulation History: 3 points
Removed measurement 7f51b95 to history. Total: 3
th(0)=-47.94885028422188;dx=-6.89893147280949E-7
New Minimum: -47.94885028422188 > -54.30502834715755
END: th(1.6843397559414055E7)=-54.30502834715755; dx=-1.620529090858183E-7 evalInputDelta=6.356178062935676
Fitness changed from -47.94885028422188 to -54.30502834715755
Iteration 5 complete. Error: -54.30502834715755 Total: 22.6432; Orientation: 2.6129; Line Search: 12.7044
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 35fbe262 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.666e+04, gradient 9.985e-04, dot -0.483; [345419b7-5ff6-4282-96e7-375459fd911b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -54.30502834715755, -42.18998832382227, -37.66601038084862, -34.66923695766724
LBFGS Accumulation History: 3 points
Removed measurement 35fbe262 to history. Total: 3
th(0)=-54.30502834715755;dx=-9.970306434740045E-7
Armijo: th(3.628800000000001E7)=-52.8037715136526; dx=1.749013146934429E-7 evalInputDelta=-1.5012568335049536
New Minimum: -54.30502834715755 > -57.67064666904325
WOLF (strong): th(1.8144000000000004E7)=-57.67064666904325; dx=2.4298615154880164E-7 evalInputDelta=3.365618321885698
New Minimum: -57.67064666904325 > -58.88666345896532
END: th(6048000.000000001)=-58.88666345896532; dx=-3.3418700062727266E-7 evalInputDelta=4.5816351118077705
Fitness changed from -54.30502834715755 to -58.88666345896532
Iteration 6 complete. Error: -58.88666345896532 Total: 34.5819; Orientation: 2.6137; Line Search: 24.7380
Adding measurement 4574ed2e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.312e+04, gradient 7.238e-04, dot -0.680; [345419b7-5ff6-4282-96e7-375459fd911b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -58.88666345896532, -42.18998832382227, -37.66601038084862, -34.66923695766724
LBFGS Accumulation History: 3 points
Removed measurement 4574ed2e to history. Total: 3
th(0)=-58.88666345896532;dx=-5.231017836693965E-7
New Minimum: -58.88666345896532 > -63.2789737933354
END: th(1.3030021005312836E7)=-63.2789737933354; dx=-1.564212433048019E-7 evalInputDelta=4.39231033437008
Fitness changed from -58.88666345896532 to -63.2789737933354
Iteration 7 complete. Error: -63.2789737933354 Total: 21.1061; Orientation: 2.5352; Line Search: 12.5273
Adding measurement 60f4a3d5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.322e+04, gradient 8.470e-04, dot -0.547; [345419b7-5ff6-4282-96e7-375459fd911b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.2789737933354, -42.18998832382227, -37.66601038084862, -34.66923695766724
LBFGS Accumulation History: 3 points
Removed measurement 60f4a3d5 to history. Total: 3
th(0)=-63.2789737933354;dx=-7.16157019116834E-7
New Minimum: -63.2789737933354 > -63.91971983835177
WOLF (strong): th(2.8072329265690096E7)=-63.91971983835177; dx=1.417503812381181E-7 evalInputDelta=0.640746045016364
New Minimum: -63.91971983835177 > -66.44576257782572
WOLF (strong): th(1.4036164632845048E7)=-66.44576257782572; dx=1.0671437928386512E-7 evalInputDelta=3.166788784490315
END: th(4678721.544281683)=-66.05444196334962; dx=-3.382258863103794E-7 evalInputDelta=2.7754681700142143
Fitness changed from -63.2789737933354 to -66.44576257782572
Iteration 8 complete. Error: -66.44576257782572 Total: 32.5959; Orientation: 2.5992; Line Search: 24.3101
Adding measurement 16ef4b28 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.405e+04, gradient 9.735e-04, dot -0.430; [345419b7-5ff6-4282-96e7-375459fd911b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -66.44576257782572, -42.18998832382227, -37.66601038084862, -34.66923695766724
LBFGS Accumulation History: 3 points
Removed measurement 16ef4b28 to history. Total: 3
th(0)=-66.44576257782572;dx=-9.467562487744434E-7
New Minimum: -66.44576257782572 > -71.67668967669933
WOLF (strong): th(1.0080000000000004E7)=-71.67668967669933; dx=5.8307076879606464E-8 evalInputDelta=5.230927098873607
END: th(5040000.000000002)=-70.5986937727459; dx=-4.932783334042021E-7 evalInputDelta=4.1529311949201855
Fitness changed from -66.44576257782572 to -71.67668967669933
Iteration 9 complete. Error: -71.67668967669933 Total: 26.2580; Orientation: 2.4934; Line Search: 17.9117
Adding measurement 41467b91 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.255e+04, gradient 7.618e-04, dot -0.550; [345419b7-5ff6-4282-96e7-375459fd911b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -71.67668967669933, -42.18998832382227, -37.66601038084862, -34.66923695766724
LBFGS Accumulation History: 3 points
Removed measurement 41467b91 to history. Total: 3
th(0)=-71.67668967669933;dx=-5.794987694336367E-7
New Minimum: -71.67668967669933 > -74.9378775164868
END: th(1.0858350837760698E7)=-74.9378775164868; dx=-9.318733726695403E-9 evalInputDelta=3.2611878397874676
Fitness changed from -71.67668967669933 to -74.9378775164868
Iteration 10 complete. Error: -74.9378775164868 Total: 21.6066; Orientation: 2.5909; Line Search: 13.1017
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -74.9378775164868 (> -Infinity) after 299.846s (< 5400.000s)
Returns
-74.9378775164868