Code from BasicOptimizer.scala:75 executed in 103.15 seconds (1.837 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 736337910625000
Reset training subject: 736340704197300
Adding measurement 2dbc408c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=15.558271911926568;dx=-4.8546428754459446E-5
New Minimum: 15.558271911926568 > 15.558061758987606
WOLFE (weak): th(2.154434690031884)=15.558061758987606; dx=-4.850441810157548E-5 evalInputDelta=2.1015293896198273E-4
New Minimum: 15.558061758987606 > 15.55785267520696
WOLFE (weak): th(4.308869380063768)=15.55785267520696; dx=-4.847604276229347E-5 evalInputDelta=4.192367196083069E-4
New Minimum: 15.55785267520696 > 15.557012541219592
WOLFE (weak): th(12.926608140191302)=15.557012541219592; dx=-4.858665094049397E-5 evalInputDelta=0.00125937070697546
New Minimum: 15.557012541219592 > 15.55323429685086
WOLFE (weak): th(51.70643256076521)=15.55323429685086; dx=-4.853057900880425E-5 evalInputDelta=0.005037615075707436
New Minimum: 15.55323429685086 > 15.533136305399239
WOLFE (weak): th(258.53216280382605)=15.533136305399239; dx=-4.8378031161468496E-5 evalInputDelta=0.02513560652732849
New Minimum: 15.533136305399239 > 15.408833269029856
WOLFE (weak): th(1551.1929768229563)=15.408833269029856; dx=-4.7401307697496696E-5 evalInputDelta=0.14943864289671183
New Minimum: 15.408833269029856 > 14.602461121976376
END: th(10858.350837760694)=14.602461121976376; dx=-3.732091158562037E-5 evalInputDelta=0.955810789950192
Fitness changed from 15.558271911926568 to 14.602461121976376
Iteration 1 complete. Error: 14.602461121976376 Total: 28.9301; Orientation: 0.0128; Line Search: 20.7375
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 40113163 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=14.602461121976376;dx=-3.122466587687914E-5
New Minimum: 14.602461121976376 > 13.32138699106872
END: th(23393.607721408407)=13.32138699106872; dx=-2.131072984579399E-5 evalInputDelta=1.2810741309076548
Fitness chang
...skipping 4976 bytes...
rement 236eccd1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.785e+03, gradient 6.573e-04, dot -0.797; [f55464f3-5773-444c-934e-feba7d43ffcb = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.73303599562496, 7.029914582148194, 5.302216589450836, 3.7295089289546013
LBFGS Accumulation History: 3 points
Removed measurement 35c4697c to history. Total: 4
th(0)=3.7295089289546013;dx=-4.319836989464462E-7
New Minimum: 3.7295089289546013 > 2.7622984424233437
WOLF (strong): th(5040000.000000002)=2.7622984424233437; dx=4.060555673601296E-8 evalInputDelta=0.9672104865312576
New Minimum: 2.7622984424233437 > 2.722991093993187
END: th(2520000.000000001)=2.722991093993187; dx=-5.3873877005499355E-8 evalInputDelta=1.0065178349614143
Fitness changed from 3.7295089289546013 to 2.722991093993187
Iteration 9 complete. Error: 2.722991093993187 Total: 10.4435; Orientation: 0.0597; Line Search: 8.0078
Adding measurement 5099c59b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.320e+03, gradient 3.984e-04, dot -0.847; [f55464f3-5773-444c-934e-feba7d43ffcb = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.029914582148194, 5.302216589450836, 3.7295089289546013, 2.722991093993187
LBFGS Accumulation History: 3 points
Removed measurement 3b7c80c6 to history. Total: 4
th(0)=2.722991093993187;dx=-1.5847510197299317E-7
New Minimum: 2.722991093993187 > 2.194137666374445
WOLF (strong): th(5429175.418880349)=2.194137666374445; dx=1.498435067121503E-8 evalInputDelta=0.528853427618742
END: th(2714587.7094401745)=2.2021443508565426; dx=-3.7060719098742935E-8 evalInputDelta=0.5208467431366444
Fitness changed from 2.722991093993187 to 2.194137666374445
Iteration 10 complete. Error: 2.194137666374445 Total: 10.2350; Orientation: 0.0564; Line Search: 7.6793
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: 2.194137666374445 (> -Infinity) after 103.145s (< 1800.000s)
2.194137666374445