Code from BasicOptimizer.scala:75 executed in 2674.71 seconds (16.862 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1177568147114400
Reset training subject: 1177588863719400
Adding measurement 59711ea7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=0.5652889975093471;dx=-2.8413858425692405E-7
New Minimum: 0.5652889975093471 > 0.5652887719884974
WOLFE (weak): th(2.154434690031884)=0.5652887719884974; dx=-2.8135131398661615E-7 evalInputDelta=2.255208497725647E-7
New Minimum: 0.5652887719884974 > 0.5652885376236678
WOLFE (weak): th(4.308869380063768)=0.5652885376236678; dx=-2.813510837500295E-7 evalInputDelta=4.5988567931587454E-7
New Minimum: 0.5652885376236678 > 0.5652878002905538
WOLFE (weak): th(12.926608140191302)=0.5652878002905538; dx=-2.813509561843728E-7 evalInputDelta=1.1972187933251277E-6
New Minimum: 0.5652878002905538 > 0.5652840958503056
WOLFE (weak): th(51.70643256076521)=0.5652840958503056; dx=-2.813487216859614E-7 evalInputDelta=4.901659041545159E-6
New Minimum: 0.5652840958503056 > 0.5652648160884856
WOLFE (weak): th(258.53216280382605)=0.5652648160884856; dx=-2.813372391875957E-7 evalInputDelta=2.4181420861579817E-5
New Minimum: 0.5652648160884856 > 0.5651442963674824
WOLFE (weak): th(1551.1929768229563)=0.5651442963674824; dx=-2.8131879017254346E-7 evalInputDelta=1.447011418647115E-4
New Minimum: 0.5651442963674824 > 0.5642771202415247
WOLFE (weak): th(10858.350837760694)=0.5642771202415247; dx=-2.808658850738812E-7 evalInputDelta=0.0010118772678224808
New Minimum: 0.5642771202415247 > 0.5572466352258096
WOLFE (weak): th(86866.80670208555)=0.5572466352258096; dx=-2.776702072966983E-7 evalInputDelta=0.008042362283537496
New Minimum: 0.5572466352258096 > 0.4975272119324641
END: th(781801.26031877)=0.4975272119324641; dx=-2.479404962501235E-7 evalInputDelta=0.06776178557688306
Fitness changed from 0.5652889975093471 to 0.4975272119324641
Iteration 1 complete. Error: 0.4975272119324641 Total: 269.4803; Orientation: 0.0185; Line Search: 206.7005
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>=0.02524153750846936;dx=-2.023404895606878E-8
New Minimum: 0.02524153750846936 > 0.024960317598886395
END: th(2937926.9599328605)=0.024960317598886395; dx=-1.7537229806871516E-8 evalInputDelta=2.812199095829637E-4
Fitness changed from 0.02524153750846936 to 0.024960317598886395
Iteration 29 complete. Error: 0.024960317598886395 Total: 62.7225; Orientation: 0.5577; Line Search: 41.3877
Adding measurement 5e5e680b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.134e+04, gradient 1.411e-04, dot -0.936; [0159bff2-c9da-4f6d-8859-689b164ea9ed = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0264100685060642, 0.025976365360864673, 0.02524153750846936, 0.024960317598886395
LBFGS Accumulation History: 3 points
Removed measurement 57c06659 to history. Total: 4
th(0)=0.024960317598886395;dx=-1.9909301454079672E-8
Armijo: th(6329571.759259267)=0.0295020192743497; dx=-1.599451604341206E-8 evalInputDelta=-0.0045417016754633055
Armijo: th(3164785.8796296334)=0.02503506685847634; dx=-1.7102692763589007E-8 evalInputDelta=-7.474925958994572E-5
New Minimum: 0.024960317598886395 > 0.024494446498282252
WOLFE (weak): th(1054928.626543211)=0.024494446498282252; dx=-1.795413138008303E-8 evalInputDelta=4.658711006041427E-4
END: th(2109857.253086422)=0.024519910497572517; dx=-1.7510499910035168E-8 evalInputDelta=4.404071013138777E-4
Fitness changed from 0.024960317598886395 to 0.024494446498282252
Iteration 30 complete. Error: 0.024494446498282252 Total: 124.5558; Orientation: 0.6911; Line Search: 103.1121
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 0.024494446498282252 (> -Infinity) after 2674.700s (< 3600.000s)
0.024494446498282252