Code from BasicOptimizer.scala:75 executed in 1300.04 seconds (9.022 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 901599892756100
Reset training subject: 901612821934900
Adding measurement 6397fa6e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=10.841300427913666;dx=-9.559682816756858E-4
New Minimum: 10.841300427913666 > 10.83953407406807
WOLFE (weak): th(2.154434690031884)=10.83953407406807; dx=-9.551968372543552E-4 evalInputDelta=0.0017663538455963135
New Minimum: 10.83953407406807 > 10.83776867389679
WOLFE (weak): th(4.308869380063768)=10.83776867389679; dx=-9.535506951234319E-4 evalInputDelta=0.0035317540168762207
New Minimum: 10.83776867389679 > 10.83069184422493
WOLFE (weak): th(12.926608140191302)=10.83069184422493; dx=-9.539882086652062E-4 evalInputDelta=0.010608583688735962
New Minimum: 10.83069184422493 > 10.798319652676582
WOLFE (weak): th(51.70643256076521)=10.798319652676582; dx=-9.738224802809444E-4 evalInputDelta=0.042980775237083435
New Minimum: 10.798319652676582 > 10.626576744019985
WOLFE (weak): th(258.53216280382605)=10.626576744019985; dx=-9.470615864680614E-4 evalInputDelta=0.21472368389368057
New Minimum: 10.626576744019985 > 9.577464260160923
END: th(1551.1929768229563)=9.577464260160923; dx=-8.130788982189282E-4 evalInputDelta=1.2638361677527428
Fitness changed from 10.841300427913666 to 9.577464260160923
Iteration 1 complete. Error: 9.577464260160923 Total: 130.2556; Orientation: 0.0143; Line Search: 91.7707
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5ce4a581 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=9.577464260160923;dx=-8.926896666854302E-4
New Minimum: 9.577464260160923 > 7.021501384675503
END: th(3341.943960201201)=7.021501384675503; dx=-4.103471180481046E-4 evalInputDelta=2.5559628754854202
Fitness changed from 9.577464260160923 to 7.021501384675503
Iteration 2 complete. Error: 7.021501384675503 Total: 38.8905; Orientation: 0.0144; Line Search: 26.0206
<a id="p-5"></a>Iteration 2
...skipping 18857 bytes...
98.0256761265337)=3.633262552320957; dx=-6.8745549382139665E-6 evalInputDelta=0.016971759498119354
Fitness changed from 3.6502343118190765 to 3.633262552320957
Iteration 23 complete. Error: 3.633262552320957 Total: 65.0691; Orientation: 0.4197; Line Search: 51.7760
Adding measurement 4230d4f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.601e+02, gradient 3.159e-03, dot -0.580; [d28fd555-e617-4d77-a0f9-3a2dd17ee1fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.680019985884428, 3.6716290563344955, 3.6502343118190765, 3.633262552320957
LBFGS Accumulation History: 3 points
Removed measurement cbec4b5 to history. Total: 4
th(0)=3.633262552320957;dx=-9.976460139229825E-6
New Minimum: 3.633262552320957 > 3.6176944747567177
END: th(6028.163580246921)=3.6176944747567177; dx=-4.449684097796106E-6 evalInputDelta=0.015568077564239502
Fitness changed from 3.633262552320957 to 3.6176944747567177
Iteration 24 complete. Error: 3.6176944747567177 Total: 39.3212; Orientation: 0.4162; Line Search: 26.0641
Adding measurement eb28f39 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.511e+01, gradient 2.655e-03, dot -0.793; [d28fd555-e617-4d77-a0f9-3a2dd17ee1fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.6716290563344955, 3.6502343118190765, 3.633262552320957, 3.6176944747567177
LBFGS Accumulation History: 3 points
Removed measurement 478cea4 to history. Total: 4
th(0)=3.6176944747567177;dx=-7.048914762603532E-6
New Minimum: 3.6176944747567177 > 3.56249987334013
END: th(12987.284734470766)=3.56249987334013; dx=-1.4328739347780866E-6 evalInputDelta=0.05519460141658783
Fitness changed from 3.6176944747567177 to 3.56249987334013
Iteration 25 complete. Error: 3.56249987334013 Total: 42.4143; Orientation: 0.4597; Line Search: 28.6909
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Final threshold in iteration 26: 3.56249987334013 (> -Infinity) after 1300.040s (< 3600.000s)
3.56249987334013