Code from BasicOptimizer.scala:75 executed in 2783.55 seconds (26.680 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 847586446009800
Reset training subject: 847612504216800
Adding measurement 60b5e80d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=6.10224312543869;dx=-3.402419206068681E-7
New Minimum: 6.10224312543869 > 6.1022422313690186
WOLFE (weak): th(2.154434690031884)=6.1022422313690186; dx=-3.402412976587085E-7 evalInputDelta=8.940696716308594E-7
New Minimum: 6.1022422313690186 > 6.102241516113281
WOLFE (weak): th(4.308869380063768)=6.102241516113281; dx=-3.402410408996948E-7 evalInputDelta=1.6093254089355469E-6
New Minimum: 6.102241516113281 > 6.102238595485687
WOLFE (weak): th(12.926608140191302)=6.102238595485687; dx=-3.4024072266120933E-7 evalInputDelta=4.5299530029296875E-6
New Minimum: 6.102238595485687 > 6.102225303649902
WOLFE (weak): th(51.70643256076521)=6.102225303649902; dx=-3.4023905583208853E-7 evalInputDelta=1.7821788787841797E-5
New Minimum: 6.102225303649902 > 6.1021535992622375
WOLFE (weak): th(258.53216280382605)=6.1021535992622375; dx=-3.4023564505742895E-7 evalInputDelta=8.952617645263672E-5
New Minimum: 6.1021535992622375 > 6.10170578956604
WOLFE (weak): th(1551.1929768229563)=6.10170578956604; dx=-3.4023267545404993E-7 evalInputDelta=5.373358726501465E-4
New Minimum: 6.10170578956604 > 6.098482608795166
WOLFE (weak): th(10858.350837760694)=6.098482608795166; dx=-3.401538902088676E-7 evalInputDelta=0.00376051664352417
New Minimum: 6.098482608795166 > 6.072173476219177
WOLFE (weak): th(86866.80670208555)=6.072173476219177; dx=-3.3913954480493344E-7 evalInputDelta=0.03006964921951294
New Minimum: 6.072173476219177 > 5.834013104438782
WOLFE (weak): th(781801.26031877)=5.834013104438782; dx=-3.278986954298825E-7 evalInputDelta=0.26823002099990845
New Minimum: 5.834013104438782 > 3.9820876121520996
END: th(7818012.6031877)=3.9820876121520996; dx=-1.797426841955141E-7 evalInputDelta=2.1201555132865906
Fitness changed from 6.10224312543869 to 3.9820876121520996
Iteration 1 complete. Error: 3.98
...skipping 15166 bytes...
4.803348321422479E-10 evalInputDelta=0.015053221955895424
Fitness changed from 0.5145101677626371 to 0.4994569458067417
Iteration 18 complete. Error: 0.4994569458067417 Total: 232.1854; Orientation: 1.9158; Line Search: 191.4057
Adding measurement 37b01ce2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.315e+04, gradient 2.190e-05, dot -0.823; [b970a378-bbaf-426c-823a-db9036f51d57 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.6096949987113476, 0.5712915118783712, 0.5145101677626371, 0.4994569458067417
LBFGS Accumulation History: 3 points
Removed measurement 1068176 to history. Total: 4
th(0)=0.4994569458067417;dx=-4.795073448024754E-10
New Minimum: 0.4994569458067417 > 0.48706447146832943
END: th(3.1418839229631666E7)=0.48706447146832943; dx=-4.205660518177912E-10 evalInputDelta=0.012392474338412285
Fitness changed from 0.4994569458067417 to 0.48706447146832943
Iteration 19 complete. Error: 0.48706447146832943 Total: 118.2099; Orientation: 1.9356; Line Search: 77.5681
Adding measurement 214b342f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.877e+04, gradient 2.055e-05, dot -0.950; [b970a378-bbaf-426c-823a-db9036f51d57 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.5712915118783712, 0.5145101677626371, 0.4994569458067417, 0.48706447146832943
LBFGS Accumulation History: 3 points
Removed measurement 2aa5bd48 to history. Total: 4
th(0)=0.48706447146832943;dx=-4.224977197501431E-10
New Minimum: 0.48706447146832943 > 0.4641258306801319
END: th(6.76898371568531E7)=0.4641258306801319; dx=-3.5430195868459656E-10 evalInputDelta=0.022938640788197517
Fitness changed from 0.48706447146832943 to 0.4641258306801319
Iteration 20 complete. Error: 0.4641258306801319 Total: 116.1970; Orientation: 1.8505; Line Search: 76.6530
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: 0.4641258306801319 (> -Infinity) after 2783.553s (< 3600.000s)
0.4641258306801319