BasicOptimizer.scala:89 executed in 229.59 seconds (2.610 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1645367347992
Reset training subject: 1650431055557
Adding measurement 3e4a575d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.068217724561691;dx=-2.334375246923506E-6
New Minimum: -15.068217724561691 > -15.068223237991333
END: th(2.154434690031884)=-15.068223237991333; dx=-2.0942353573159768E-6 evalInputDelta=5.513429641723633E-6
Fitness changed from -15.068217724561691 to -15.068223237991333
Iteration 1 complete. Error: -15.068223237991333 Total: 25.8702; Orientation: 0.0270; Line Search: 10.1323
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6b37c7f3 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-15.068223237991333;dx=-2.085812551192422E-6
New Minimum: -15.068223237991333 > -15.068235516548157
WOLFE (weak): th(4.641588833612779)=-15.068235516548157; dx=-2.084801268392039E-6 evalInputDelta=1.2278556823730469E-5
New Minimum: -15.068235516548157 > -15.068246215581894
WOLFE (weak): th(9.283177667225559)=-15.068246215581894; dx=-2.0833792908078063E-6 evalInputDelta=2.2977590560913086E-5
New Minimum: -15.068246215581894 > -15.068292945623398
WOLFE (weak): th(27.849533001676676)=-15.068292945623398; dx=-2.0832264937005387E-6 evalInputDelta=6.970763206481934E-5
New Minimum: -15.068292945623398 > -15.068502247333527
WOLFE (weak): th(111.3981320067067)=-15.068502247333527; dx=-2.08282582580703E-6 evalInputDelta=2.790093421936035E-4
New Minimum: -15.068502247333527 > -15.06961476802826
WOLFE (weak): th(556.9906600335335)=-15.06961476802826; dx=-2.081136213334531E-6 evalInputDelta=0.0013915300369262695
New Minimum: -15.06961476802826 > -15.076558828353882
WOLFE (weak): th(3341.9439602012008)=-15.076558828353882; dx=-2.075509300907518E-6 evalInputDelta=0.008335590362548828
New Minimum: -15.076558828353882 > -15.126008003950119
WOLFE (weak): th(23393.607721408407)=-15.126008003950119; dx=-2.036327287229469E-6 evalInputDelta=0.05778476595878601
New Minimum: -15.126008003950119 > -15.487190812826157
END: th(187148.86177126726)=-15.487190812826157; dx=-1.6142658867872787E-6 evalInputDelta=0.4189675748348236
Fitness changed from -15.068223237991333 to -15.487190812826157
Iteration 2 complete. Error: -15.487190812826157 Total: 51.2144; Orientation: 0.0267; Line Search: 46.1760
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 746424c0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-15.487190812826157;dx=-1.4712604270522813E-6
New Minimum: -15.487190812826157 > -16.038701325654984
END: th(403200.00000000006)=-16.038701325654984; dx=-8.40509406786133E-7 evalInputDelta=0.5515105128288269
Fitness changed from -15.487190812826157 to -16.038701325654984
Iteration 3 complete. Error: -16.038701325654984 Total: 15.4224; Orientation: 0.0272; Line Search: 10.0212
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 17e73707 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.833e+02, gradient 9.354e-04, dot -0.707; [482f17e2-ff61-40cb-a539-f8c1d5b9b556 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.038701325654984, -15.487190812826157, -15.068223237991333, -15.068217724561691
LBFGS Accumulation History: 3 points
Removed measurement 17e73707 to history. Total: 3
th(0)=-16.038701325654984;dx=-8.740017349907532E-7
New Minimum: -16.038701325654984 > -16.664022117853165
END: th(868668.0670208557)=-16.664022117853165; dx=-4.473880909311387E-7 evalInputDelta=0.6253207921981812
Fitness changed from -16.038701325654984 to -16.664022117853165
Iteration 4 complete. Error: -16.664022117853165 Total: 16.2683; Orientation: 0.5996; Line Search: 10.4274
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6b5f2f23 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.859e+02, gradient 7.825e-04, dot -0.668; [482f17e2-ff61-40cb-a539-f8c1d5b9b556 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.664022117853165, -15.487190812826157, -15.068223237991333, -15.068217724561691
LBFGS Accumulation History: 3 points
Removed measurement 6b5f2f23 to history. Total: 3
th(0)=-16.664022117853165;dx=-6.116339058642452E-7
New Minimum: -16.664022117853165 > -17.325235426425934
END: th(1871488.617712673)=-17.325235426425934; dx=-1.4989273243469814E-7 evalInputDelta=0.6612133085727692
Fitness changed from -16.664022117853165 to -17.325235426425934
Iteration 5 complete. Error: -17.325235426425934 Total: 15.8186; Orientation: 0.6263; Line Search: 9.9769
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement a57e1a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.777e+02, gradient 7.691e-04, dot -0.396; [482f17e2-ff61-40cb-a539-f8c1d5b9b556 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.325235426425934, -15.487190812826157, -15.068223237991333, -15.068217724561691
LBFGS Accumulation History: 3 points
Removed measurement a57e1a5 to history. Total: 3
th(0)=-17.325235426425934;dx=-5.90925928421967E-7
New Minimum: -17.325235426425934 > -17.56645315885544
WOLF (strong): th(4032000.0000000014)=-17.56645315885544; dx=9.656240926296311E-8 evalInputDelta=0.2412177324295044
New Minimum: -17.56645315885544 > -17.785201132297516
WOLF (strong): th(2016000.0000000007)=-17.785201132297516; dx=2.449840077869816E-8 evalInputDelta=0.45996570587158203
END: th(672000.0000000002)=-17.667231261730194; dx=-2.6489602171898117E-7 evalInputDelta=0.34199583530426025
Fitness changed from -17.325235426425934 to -17.785201132297516
Iteration 6 complete. Error: -17.785201132297516 Total: 26.3464; Orientation: 0.5959; Line Search: 20.4554
Adding measurement 1b512eb0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.015e+03, gradient 7.773e-04, dot -0.303; [482f17e2-ff61-40cb-a539-f8c1d5b9b556 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.785201132297516, -15.487190812826157, -15.068223237991333, -15.068217724561691
LBFGS Accumulation History: 3 points
Removed measurement 1b512eb0 to history. Total: 3
th(0)=-17.785201132297516;dx=-6.038542488004301E-7
New Minimum: -17.785201132297516 > -18.259767293930054
END: th(1447780.1117014263)=-18.259767293930054; dx=-4.3296510462702696E-8 evalInputDelta=0.47456616163253784
Fitness changed from -17.785201132297516 to -18.259767293930054
Iteration 7 complete. Error: -18.259767293930054 Total: 15.9976; Orientation: 0.5816; Line Search: 10.5125
Adding measurement 223a055 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.404e+03, gradient 5.362e-04, dot -0.496; [482f17e2-ff61-40cb-a539-f8c1d5b9b556 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.259767293930054, -15.487190812826157, -15.068223237991333, -15.068217724561691
LBFGS Accumulation History: 3 points
Removed measurement 223a055 to history. Total: 3
th(0)=-18.259767293930054;dx=-2.8724601755089015E-7
New Minimum: -18.259767293930054 > -18.596412122249603
WOLF (strong): th(3119147.6961877886)=-18.596412122249603; dx=6.252690531074858E-8 evalInputDelta=0.33664482831954956
END: th(1559573.8480938943)=-18.592549443244934; dx=-8.895948746133364E-8 evalInputDelta=0.33278214931488037
Fitness changed from -18.259767293930054 to -18.596412122249603
Iteration 8 complete. Error: -18.596412122249603 Total: 20.8110; Orientation: 0.6017; Line Search: 14.9454
Adding measurement 24582d25 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.311e+03, gradient 7.738e-04, dot -0.248; [482f17e2-ff61-40cb-a539-f8c1d5b9b556 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.596412122249603, -15.487190812826157, -15.068223237991333, -15.068217724561691
LBFGS Accumulation History: 3 points
Removed measurement 24582d25 to history. Total: 3
th(0)=-18.596412122249603;dx=-5.98181821582141E-7
Armijo: th(3360000.0000000014)=-18.33558475971222; dx=3.421033912883162E-7 evalInputDelta=-0.26082736253738403
New Minimum: -18.596412122249603 > -18.9216246008873
WOLF (strong): th(1680000.0000000007)=-18.9216246008873; dx=1.218749896205301E-7 evalInputDelta=0.3252124786376953
END: th(560000.0000000002)=-18.87222111225128; dx=-2.4090814273216256E-7 evalInputDelta=0.27580899000167847
Fitness changed from -18.596412122249603 to -18.9216246008873
Iteration 9 complete. Error: -18.9216246008873 Total: 25.4939; Orientation: 0.5944; Line Search: 20.1593
Adding measurement 4e0829ac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.586e+03, gradient 5.619e-04, dot -0.349; [482f17e2-ff61-40cb-a539-f8c1d5b9b556 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.9216246008873, -15.487190812826157, -15.068223237991333, -15.068217724561691
LBFGS Accumulation History: 3 points
Removed measurement 4e0829ac to history. Total: 3
th(0)=-18.9216246008873;dx=-3.1541480593548913E-7
New Minimum: -18.9216246008873 > -19.174247562885284
END: th(1206483.4264178555)=-19.174247562885284; dx=-7.250871976784559E-8 evalInputDelta=0.25262296199798584
Fitness changed from -18.9216246008873 to -19.174247562885284
Iteration 10 complete. Error: -19.174247562885284 Total: 16.0785; Orientation: 0.5899; Line Search: 10.4502
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -19.174247562885284 (> -Infinity) after 229.585s (< 5400.000s)
Returns
-19.174247562885284