Code from BasicOptimizer.scala:75 executed in 3666.57 seconds (33.941 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1055007821679099
Reset training subject: 1055060978333000
Adding measurement 3a62755a to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.5683215335011482;dx=-6.898395595379401E-8
New Minimum: -0.5683215335011482 > -0.5683216527104378
WOLFE (weak): th(2.154434690031884)=-0.5683216527104378; dx=-6.868942222570603E-8 evalInputDelta=1.1920928955078125E-7
New Minimum: -0.5683216527104378 > -0.5683217495679855
WOLFE (weak): th(4.308869380063768)=-0.5683217495679855; dx=-6.868290086155453E-8 evalInputDelta=2.1606683731079102E-7
New Minimum: -0.5683217495679855 > -0.5683225318789482
WOLFE (weak): th(12.926608140191302)=-0.5683225318789482; dx=-6.869875965114347E-8 evalInputDelta=9.98377799987793E-7
New Minimum: -0.5683225318789482 > -0.5683254823088646
WOLFE (weak): th(51.70643256076521)=-0.5683254823088646; dx=-6.869517941212851E-8 evalInputDelta=3.948807716369629E-6
New Minimum: -0.5683254823088646 > -0.5683406814932823
WOLFE (weak): th(258.53216280382605)=-0.5683406814932823; dx=-6.869536440283367E-8 evalInputDelta=1.9147992134094238E-5
New Minimum: -0.5683406814932823 > -0.568435937166214
WOLFE (weak): th(1551.1929768229563)=-0.568435937166214; dx=-6.868323108580613E-8 evalInputDelta=1.1440366506576538E-4
New Minimum: -0.568435937166214 > -0.5691213086247444
WOLFE (weak): th(10858.350837760694)=-0.5691213086247444; dx=-6.865038024304426E-8 evalInputDelta=7.997751235961914E-4
New Minimum: -0.5691213086247444 > -0.5747068151831627
WOLFE (weak): th(86866.80670208555)=-0.5747068151831627; dx=-6.842490803412719E-8 evalInputDelta=0.006385281682014465
New Minimum: -0.5747068151831627 > -0.6247304603457451
WOLFE (weak): th(781801.26031877)=-0.6247304603457451; dx=-6.614717658159607E-8 evalInputDelta=0.05640892684459686
New Minimum: -0.6247304603457451 > -1.0178711712360382
END: th(7818012.6031877)=-1.0178711712360382; dx=-4.330336318489431E-8 evalInputDelta=0.44954963773489
Fitness changed from -0.5683215335011482
...skipping 11819 bytes...
11E-9 evalInputDelta=0.012568458914756775
Fitness changed from -2.3889106810092926 to -2.4014791399240494
Iteration 14 complete. Error: -2.4014791399240494 Total: 257.7414; Orientation: 1.9950; Line Search: 204.7517
Adding measurement 62990ea3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.094e+03, gradient 3.639e-05, dot -0.949; [9ec609af-62d5-4d73-a894-8d1d84d58d26 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.368079423904419, -2.376838818192482, -2.3889106810092926, -2.4014791399240494
LBFGS Accumulation History: 3 points
Removed measurement 10f03ab4 to history. Total: 4
th(0)=-2.4014791399240494;dx=-1.3227536329390267E-9
New Minimum: -2.4014791399240494 > -2.4094203487038612
END: th(1.750000000000001E7)=-2.4094203487038612; dx=-1.145413326139436E-9 evalInputDelta=0.007941208779811859
Fitness changed from -2.4014791399240494 to -2.4094203487038612
Iteration 15 complete. Error: -2.4094203487038612 Total: 159.5152; Orientation: 2.0340; Line Search: 105.5265
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 32c903fd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.006e+03, gradient 3.468e-05, dot -0.962; [9ec609af-62d5-4d73-a894-8d1d84d58d26 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.376838818192482, -2.3889106810092926, -2.4014791399240494, -2.4094203487038612
LBFGS Accumulation History: 3 points
Removed measurement 17d4ac58 to history. Total: 4
th(0)=-2.4094203487038612;dx=-1.2012540239555312E-9
New Minimum: -2.4094203487038612 > -2.4219851419329643
END: th(3.770260707555799E7)=-2.4219851419329643; dx=-9.757765335158654E-10 evalInputDelta=0.012564793229103088
Fitness changed from -2.4094203487038612 to -2.4219851419329643
Iteration 16 complete. Error: -2.4219851419329643 Total: 157.4009; Orientation: 2.0201; Line Search: 101.8335
Final threshold in iteration 16: -2.4219851419329643 (> -Infinity) after 3666.565s (< 3600.000s)
-2.4219851419329643