Code from BasicOptimizer.scala:75 executed in 907.62 seconds (7.029 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1068247323657100
Reset training subject: 1068255419669900
Adding measurement 62f11ebb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=9.970960260702412;dx=-1.0460779867814766E-5
New Minimum: 9.970960260702412 > 9.970940669291013
WOLFE (weak): th(2.154434690031884)=9.970940669291013; dx=-1.0461100110799394E-5 evalInputDelta=1.9591411399844105E-5
New Minimum: 9.970940669291013 > 9.970920735152903
WOLFE (weak): th(4.308869380063768)=9.970920735152903; dx=-1.0460949595080421E-5 evalInputDelta=3.9525549508923064E-5
New Minimum: 9.970920735152903 > 9.970841730565153
WOLFE (weak): th(12.926608140191302)=9.970841730565153; dx=-1.0460903089476956E-5 evalInputDelta=1.1853013725904304E-4
New Minimum: 9.970841730565153 > 9.970484946252693
WOLFE (weak): th(51.70643256076521)=9.970484946252693; dx=-1.0459529815719938E-5 evalInputDelta=4.753144497193773E-4
New Minimum: 9.970484946252693 > 9.968460602835126
END: th(258.53216280382605)=9.968460602835126; dx=-7.613773957207791E-6 evalInputDelta=0.0024996578672862313
Fitness changed from 9.970960260702412 to 9.968460602835126
Iteration 1 complete. Error: 9.968460602835126 Total: 68.8278; Orientation: 0.0323; Line Search: 45.3657
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4733f6f5 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=9.968460602835126;dx=-0.022499856538739354
New Minimum: 9.968460602835126 > 9.95998341740648
END: th(556.9906600335335)=9.95998341740648; dx=-1.744146147684616E-5 evalInputDelta=0.00847718542864584
Fitness changed from 9.968460602835126 to 9.95998341740648
Iteration 2 complete. Error: 9.95998341740648 Total: 22.7265; Orientation: 0.0149; Line Search: 15.0354
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 62cba181 to history. Total: 2
LBFGS Accumulation History: 3 points
th(
...skipping 23733 bytes...
1.6823
Adding measurement 5db948c9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.578e+03, gradient 2.468e-04, dot -0.617; [3ea302bf-0481-49e5-b7a7-07d1a864aa16 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.5143288037057592, 0.4281123710003105, 0.3903870924888113, 0.3694820419034217
LBFGS Accumulation History: 3 points
Removed measurement 5c48b72c to history. Total: 4
th(0)=0.3694820419034217;dx=-6.093233431094468E-8
Armijo: th(1.1192102704506138E7)=0.7736266678796149; dx=4.8693784920978675E-8 evalInputDelta=-0.40414462597619316
Armijo: th(5596051.352253069)=0.40120682351024506; dx=1.4294495848552497E-8 evalInputDelta=-0.03172478160682335
New Minimum: 0.3694820419034217 > 0.32620332681597697
END: th(1865350.4507510231)=0.32620332681597697; dx=-3.1628861105637215E-8 evalInputDelta=0.04327871508744474
Fitness changed from 0.3694820419034217 to 0.32620332681597697
Iteration 29 complete. Error: 0.32620332681597697 Total: 36.3105; Orientation: 0.3829; Line Search: 28.8198
Adding measurement 33d7765a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.476e+03, gradient 1.903e-04, dot -0.808; [3ea302bf-0481-49e5-b7a7-07d1a864aa16 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.4281123710003105, 0.3903870924888113, 0.3694820419034217, 0.32620332681597697
LBFGS Accumulation History: 3 points
Removed measurement 101bdd1c to history. Total: 4
th(0)=0.32620332681597697;dx=-3.620688594439602E-8
New Minimum: 0.32620332681597697 > 0.29501868218939054
END: th(4018775.720164615)=0.29501868218939054; dx=-2.4449718466668693E-8 evalInputDelta=0.031184644626586433
Fitness changed from 0.32620332681597697 to 0.29501868218939054
Iteration 30 complete. Error: 0.29501868218939054 Total: 21.9030; Orientation: 0.3929; Line Search: 14.5006
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 0.29501868218939054 (> -Infinity) after 907.608s (< 3600.000s)
0.29501868218939054