Code from BasicOptimizer.scala:75 executed in 199.91 seconds (5.829 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1376514069123800
Reset training subject: 1376517214767900
Adding measurement 4373f66f to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=20.74947824701667;dx=-8.251486148227868E-4
New Minimum: 20.74947824701667 > 20.747456992045045
WOLFE (weak): th(2.154434690031884)=20.747456992045045; dx=-8.254011788815002E-4 evalInputDelta=0.0020212549716234207
New Minimum: 20.747456992045045 > 20.745435405522585
WOLFE (weak): th(4.308869380063768)=20.745435405522585; dx=-8.251478652775145E-4 evalInputDelta=0.0040428414940834045
New Minimum: 20.745435405522585 > 20.737347904592752
WOLFE (weak): th(12.926608140191302)=20.737347904592752; dx=-8.243652311354543E-4 evalInputDelta=0.012130342423915863
New Minimum: 20.737347904592752 > 20.701173081994057
WOLFE (weak): th(51.70643256076521)=20.701173081994057; dx=-8.182566880824223E-4 evalInputDelta=0.04830516502261162
New Minimum: 20.701173081994057 > 20.5116171464324
WOLFE (weak): th(258.53216280382605)=20.5116171464324; dx=-7.823312264564669E-4 evalInputDelta=0.23786110058426857
New Minimum: 20.5116171464324 > 19.42634424380958
END: th(1551.1929768229563)=19.42634424380958; dx=-7.345504733059555E-4 evalInputDelta=1.3231340032070875
Fitness changed from 20.74947824701667 to 19.42634424380958
Iteration 1 complete. Error: 19.42634424380958 Total: 25.4337; Orientation: 0.0138; Line Search: 17.4135
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 158e9f6e to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=19.42634424380958;dx=-8.734551085030415E-4
New Minimum: 19.42634424380958 > 17.18819651566446
END: th(3341.943960201201)=17.18819651566446; dx=-5.086939266553249E-4 evalInputDelta=2.2381477281451225
Fitness changed from 19.42634424380958 to 17.18819651566446
Iteration 2 complete. Error: 17.18819651566446 Total: 7.3799; Orientation: 0.0045; Line Search: 4.9798
<a id="p-5"></a>Iteration 2
<
...skipping 14290 bytes...
89-a679-bbff6a77ed02 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.8452953100204468, 1.7465395033359528, 1.62954780459404, 1.5443890988826752
LBFGS Accumulation History: 3 points
Removed measurement 35bfa1bb to history. Total: 4
th(0)=1.5443890988826752;dx=-7.937250044305555E-7
New Minimum: 1.5443890988826752 > 1.5324094593524933
WOLF (strong): th(299227.0402822064)=1.5324094593524933; dx=3.4553310102703604E-7 evalInputDelta=0.011979639530181885
New Minimum: 1.5324094593524933 > 1.4928697645664215
END: th(149613.5201411032)=1.4928697645664215; dx=-4.743298010271897E-8 evalInputDelta=0.05151933431625366
Fitness changed from 1.5443890988826752 to 1.4928697645664215
Iteration 19 complete. Error: 1.4928697645664215 Total: 10.0573; Orientation: 0.0874; Line Search: 7.5933
Adding measurement 1cbc1dde to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.980e+02, gradient 8.961e-04, dot -0.623; [c1455e39-be9f-4989-a679-bbff6a77ed02 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.7465395033359528, 1.62954780459404, 1.5443890988826752, 1.4928697645664215
LBFGS Accumulation History: 3 points
Removed measurement 64e7d698 to history. Total: 4
th(0)=1.4928697645664215;dx=-8.030091683830143E-7
Armijo: th(322332.55788977665)=1.496561974287033; dx=3.746235874346516E-7 evalInputDelta=-0.0036922097206115723
New Minimum: 1.4928697645664215 > 1.4462280422449112
WOLF (strong): th(161166.27894488833)=1.4462280422449112; dx=5.5900664959580654E-8 evalInputDelta=0.046641722321510315
END: th(53722.092981629445)=1.4601231813430786; dx=-4.465214127497023E-7 evalInputDelta=0.032746583223342896
Fitness changed from 1.4928697645664215 to 1.4462280422449112
Iteration 20 complete. Error: 1.4462280422449112 Total: 11.9946; Orientation: 0.0893; Line Search: 9.5213
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: 1.4462280422449112 (> -Infinity) after 199.890s (< 3600.000s)
1.4462280422449112