Code from BasicOptimizer.scala:75 executed in 998.44 seconds (11.149 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1227329658953100
Reset training subject: 1227338474186200
Adding measurement 5d3b58ca to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.278280761595494;dx=-3.160081750715619E-5
New Minimum: 5.278280761595494 > 5.278251177281294
WOLFE (weak): th(2.154434690031884)=5.278251177281294; dx=-3.160049443416506E-5 evalInputDelta=2.9584314200192807E-5
New Minimum: 5.278251177281294 > 5.2782216180281685
WOLFE (weak): th(4.308869380063768)=5.2782216180281685; dx=-3.15999516071352E-5 evalInputDelta=5.914356732539261E-5
New Minimum: 5.2782216180281685 > 5.278103110806319
WOLFE (weak): th(12.926608140191302)=5.278103110806319; dx=-3.1597825533485175E-5 evalInputDelta=1.7765078917442167E-4
New Minimum: 5.278103110806319 > 5.2775693626987445
WOLFE (weak): th(51.70643256076521)=5.2775693626987445; dx=-3.159094643286033E-5 evalInputDelta=7.113988967493157E-4
New Minimum: 5.2775693626987445 > 5.27472487547453
WOLFE (weak): th(258.53216280382605)=5.27472487547453; dx=-3.1560935718814826E-5 evalInputDelta=0.0035558861209636916
New Minimum: 5.27472487547453 > 5.257025197909413
WOLFE (weak): th(1551.1929768229563)=5.257025197909413; dx=-3.1310942067232055E-5 evalInputDelta=0.021255563686080947
New Minimum: 5.257025197909413 > 5.133690093901552
WOLFE (weak): th(10858.350837760694)=5.133690093901552; dx=-2.9501356174544866E-5 evalInputDelta=0.1445906676939419
New Minimum: 5.133690093901552 > 4.374270154711843
END: th(86866.80670208555)=4.374270154711843; dx=-1.7938000395879398E-5 evalInputDelta=0.9040106068836504
Fitness changed from 5.278280761595494 to 4.374270154711843
Iteration 1 complete. Error: 4.374270154711843 Total: 99.6698; Orientation: 0.0199; Line Search: 74.1494
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7b2ccba5 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.374270154711843;dx=-1.3484042145559574E-5
New Minimum:
...skipping 24082 bytes...
8 complete. Error: -0.2312541535116694 Total: 25.1933; Orientation: 0.2209; Line Search: 16.7062
Adding measurement 291a4791 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.332e+03, gradient 2.051e-04, dot -0.891; [7e1dbd95-bfbf-4ae6-ac8d-2b5e3c32cfc9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.14691962104697798, -0.1903741254569101, -0.21107279239219462, -0.2312541535116694
LBFGS Accumulation History: 3 points
Removed measurement 5a090f62 to history. Total: 4
th(0)=-0.2312541535116694;dx=-4.2072882607621365E-8
New Minimum: -0.2312541535116694 > -0.25283875896093844
END: th(3917235.9465771476)=-0.25283875896093844; dx=-2.0313805380057532E-8 evalInputDelta=0.021584605449269045
Fitness changed from -0.2312541535116694 to -0.25283875896093844
Iteration 29 complete. Error: -0.25283875896093844 Total: 25.2131; Orientation: 0.2117; Line Search: 16.6951
Adding measurement 233f52f8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.472e+03, gradient 2.416e-04, dot -0.658; [7e1dbd95-bfbf-4ae6-ac8d-2b5e3c32cfc9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.1903741254569101, -0.21107279239219462, -0.2312541535116694, -0.25283875896093844
LBFGS Accumulation History: 3 points
Removed measurement 3b35798 to history. Total: 4
th(0)=-0.25283875896093844;dx=-5.83726861022012E-8
Armijo: th(8439429.01234569)=-0.21412131414864047; dx=-8.103347276553031E-9 evalInputDelta=-0.03871744481229797
New Minimum: -0.25283875896093844 > -0.25368191057462447
END: th(4219714.506172845)=-0.25368191057462447; dx=-6.711121263125982E-9 evalInputDelta=8.431516136860329E-4
Fitness changed from -0.25283875896093844 to -0.25368191057462447
Iteration 30 complete. Error: -0.25368191057462447 Total: 33.5509; Orientation: 0.2117; Line Search: 24.8927
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: -0.25368191057462447 (> -Infinity) after 998.423s (< 3600.000s)
-0.25368191057462447