Code from BasicOptimizer.scala:75 executed in 1595.93 seconds (8.807 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 963084594339400
Reset training subject: 963097292099100
Adding measurement 56a05324 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=13.138007754248354;dx=-8.628188008537466E-6
New Minimum: 13.138007754248354 > 13.137991068508525
WOLFE (weak): th(2.154434690031884)=13.137991068508525; dx=-8.628091716575408E-6 evalInputDelta=1.6685739829824797E-5
New Minimum: 13.137991068508525 > 13.137974479681105
WOLFE (weak): th(4.308869380063768)=13.137974479681105; dx=-8.628007761456103E-6 evalInputDelta=3.32745672491086E-5
New Minimum: 13.137974479681105 > 13.137907301082267
WOLFE (weak): th(12.926608140191302)=13.137907301082267; dx=-8.627365980927956E-6 evalInputDelta=1.0045316608753296E-4
New Minimum: 13.137907301082267 > 13.137607085138407
WOLFE (weak): th(51.70643256076521)=13.137607085138407; dx=-8.625735692081785E-6 evalInputDelta=4.006691099469606E-4
New Minimum: 13.137607085138407 > 13.13600321682517
WOLFE (weak): th(258.53216280382605)=13.13600321682517; dx=-8.619232825649888E-6 evalInputDelta=0.002004537423184516
New Minimum: 13.13600321682517 > 13.126009947942713
WOLFE (weak): th(1551.1929768229563)=13.126009947942713; dx=-8.576444935543757E-6 evalInputDelta=0.011997806305641845
New Minimum: 13.126009947942713 > 13.05528095008854
WOLFE (weak): th(10858.350837760694)=13.05528095008854; dx=-8.328082794731954E-6 evalInputDelta=0.08272680415981526
New Minimum: 13.05528095008854 > 12.541456953981875
END: th(86866.80670208555)=12.541456953981875; dx=-6.634334677059639E-6 evalInputDelta=0.5965508002664794
Fitness changed from 13.138007754248354 to 12.541456953981875
Iteration 1 complete. Error: 12.541456953981875 Total: 158.8875; Orientation: 0.0169; Line Search: 119.7048
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 615439f7 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=12.541456953981875;dx=-6.053185298454921E-6
...skipping 23679 bytes...
7337163018264 to 2.0889982312773117
Iteration 28 complete. Error: 2.0889982312773117 Total: 66.8273; Orientation: 0.4274; Line Search: 52.7026
Adding measurement 374b6e33 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.948e+03, gradient 1.770e-04, dot -0.860; [13a01037-ede3-4c79-b03c-4ab4c2b4bd83 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.2535332144064504, 2.203432677287686, 2.17337163018264, 2.0889982312773117
LBFGS Accumulation History: 3 points
Removed measurement 2037f12b to history. Total: 4
th(0)=2.0889982312773117;dx=-3.1336630451430675E-8
New Minimum: 2.0889982312773117 > 2.0327296064670026
END: th(7834471.893154297)=2.0327296064670026; dx=-1.4185574435614116E-8 evalInputDelta=0.05626862481030903
Fitness changed from 2.0889982312773117 to 2.0327296064670026
Iteration 29 complete. Error: 2.0327296064670026 Total: 37.7351; Orientation: 0.4364; Line Search: 24.4123
Adding measurement 4beae1e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.702e+03, gradient 1.798e-04, dot -0.719; [13a01037-ede3-4c79-b03c-4ab4c2b4bd83 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.203432677287686, 2.17337163018264, 2.0889982312773117, 2.0327296064670026
LBFGS Accumulation History: 3 points
Removed measurement 40f9f60d to history. Total: 4
th(0)=2.0327296064670026;dx=-3.232142858835314E-8
Armijo: th(1.6878858024691384E7)=2.1493346738204093; dx=-2.4366128478374935E-9 evalInputDelta=-0.1166050673534067
New Minimum: 2.0327296064670026 > 1.995744421912557
END: th(8439429.012345692)=1.995744421912557; dx=-1.0964206901777793E-8 evalInputDelta=0.03698518455444555
Fitness changed from 2.0327296064670026 to 1.995744421912557
Iteration 30 complete. Error: 1.995744421912557 Total: 51.0036; Orientation: 0.4433; Line Search: 38.0086
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 1.995744421912557 (> -Infinity) after 1595.922s (< 3600.000s)
1.995744421912557