BasicOptimizer.scala:89 executed in 136.00 seconds (1.613 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 70647687643100
Reset training subject: 70650403363700
Adding measurement 58d48748 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-13.55486881142071;dx=-8.548953112971057E-7
New Minimum: -13.55486881142071 > -13.554870715807727
WOLFE (weak): th(2.154434690031884)=-13.554870715807727; dx=-8.000155246339249E-7 evalInputDelta=1.9043870178592215E-6
New Minimum: -13.554870715807727 > -13.554873096291123
WOLFE (weak): th(4.308869380063768)=-13.554873096291123; dx=-7.999857109693526E-7 evalInputDelta=4.284870414039688E-6
New Minimum: -13.554873096291123 > -13.554880713835184
WOLFE (weak): th(12.926608140191302)=-13.554880713835184; dx=-7.999394969223592E-7 evalInputDelta=1.190241447446283E-5
New Minimum: -13.554880713835184 > -13.55491499273048
WOLFE (weak): th(51.70643256076521)=-13.55491499273048; dx=-8.000221546973385E-7 evalInputDelta=4.618130977007695E-5
New Minimum: -13.55491499273048 > -13.555097335952594
WOLFE (weak): th(258.53216280382605)=-13.555097335952594; dx=-7.998237005845253E-7 evalInputDelta=2.2852453188448862E-4
New Minimum: -13.555097335952594 > -13.556240376875312
WOLFE (weak): th(1551.1929768229563)=-13.556240376875312; dx=-7.996727675379864E-7 evalInputDelta=0.0013715654546029299
New Minimum: -13.556240376875312 > -13.564455438969903
WOLFE (weak): th(10858.350837760694)=-13.564455438969903; dx=-7.981987809165332E-7 evalInputDelta=0.009586627549193594
New Minimum: -13.564455438969903 > -13.630946966719312
WOLFE (weak): th(86866.80670208555)=-13.630946966719312; dx=-7.850851466959803E-7 evalInputDelta=0.07607815529860318
New Minimum: -13.630946966719312 > -14.174229673226101
END: th(781801.26031877)=-14.174229673226101; dx=-6.337884714282298E-7 evalInputDelta=0.6193608618053918
Fitness changed from -13.55486881142071 to -14.174229673226101
Iteration 1 complete. Error: -14.174229673226101 Total: 34.2756; Orientation: 0.0652; Line Search: 26.3234
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 39fb0560 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-14.174229673226101;dx=-7.1487570909258E-7
New Minimum: -14.174229673226101 > -15.198203256825177
END: th(1684339.7559414052)=-15.198203256825177; dx=-4.5094430423165883E-7 evalInputDelta=1.0239735835990764
Fitness changed from -14.174229673226101 to -15.198203256825177
Iteration 2 complete. Error: -15.198203256825177 Total: 8.4732; Orientation: 0.0630; Line Search: 5.2181
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1a756808 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-15.198203256825177;dx=-6.475264182784937E-7
New Minimum: -15.198203256825177 > -16.660735228613152
END: th(3628800.0)=-16.660735228613152; dx=-2.435634199432608E-7 evalInputDelta=1.4625319717879748
Fitness changed from -15.198203256825177 to -16.660735228613152
Iteration 3 complete. Error: -16.660735228613152 Total: 8.4708; Orientation: 0.0622; Line Search: 5.5029
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7df4f55c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.149e+04, gradient 7.896e-04, dot -0.616; [1f8e6519-7bb2-4546-888f-5ddd04cc1917 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.660735228613152, -15.198203256825177, -14.174229673226101, -13.55486881142071
LBFGS Accumulation History: 3 points
Removed measurement 7df4f55c to history. Total: 3
th(0)=-16.660735228613152;dx=-6.230210573124679E-7
New Minimum: -16.660735228613152 > -18.169795670368924
END: th(7818012.6031877)=-18.169795670368924; dx=-5.654623344211138E-8 evalInputDelta=1.5090604417557714
Fitness changed from -16.660735228613152 to -18.169795670368924
Iteration 4 complete. Error: -18.169795670368924 Total: 9.3463; Orientation: 1.1245; Line Search: 5.3248
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 508cb9bc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.076e+03, gradient 7.824e-04, dot -0.410; [1f8e6519-7bb2-4546-888f-5ddd04cc1917 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.169795670368924, -15.198203256825177, -14.174229673226101, -13.55486881142071
LBFGS Accumulation History: 3 points
Removed measurement 508cb9bc to history. Total: 3
th(0)=-18.169795670368924;dx=-6.116612286558065E-7
New Minimum: -18.169795670368924 > -19.296286464804318
WOLF (strong): th(1.684339755941405E7)=-19.296286464804318; dx=5.328197757582739E-8 evalInputDelta=1.1264907944353943
New Minimum: -19.296286464804318 > -19.930517944789162
WOLF (strong): th(8421698.779707026)=-19.930517944789162; dx=5.96443166184258E-8 evalInputDelta=1.7607222744202389
END: th(2807232.9265690087)=-19.69023733590161; dx=-2.852870542654457E-7 evalInputDelta=1.5204416655326867
Fitness changed from -18.169795670368924 to -19.930517944789162
Iteration 5 complete. Error: -19.930517944789162 Total: 14.7541; Orientation: 1.1110; Line Search: 10.6186
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6685429d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.992e+03, gradient 7.962e-04, dot -0.336; [1f8e6519-7bb2-4546-888f-5ddd04cc1917 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.930517944789162, -15.198203256825177, -14.174229673226101, -13.55486881142071
LBFGS Accumulation History: 3 points
Removed measurement 6685429d to history. Total: 3
th(0)=-19.930517944789162;dx=-6.333759778433652E-7
New Minimum: -19.930517944789162 > -22.092607408020108
WOLF (strong): th(6048000.0)=-22.092607408020108; dx=5.523379415287563E-8 evalInputDelta=2.162089463230945
END: th(3024000.0)=-21.69883711036266; dx=-3.358524666429053E-7 evalInputDelta=1.768319165573498
Fitness changed from -19.930517944789162 to -22.092607408020108
Iteration 6 complete. Error: -22.092607408020108 Total: 12.0141; Orientation: 1.1450; Line Search: 7.8293
Adding measurement 363433bd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.451e+04, gradient 7.765e-04, dot -0.369; [1f8e6519-7bb2-4546-888f-5ddd04cc1917 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.092607408020108, -15.198203256825177, -14.174229673226101, -13.55486881142071
LBFGS Accumulation History: 3 points
Removed measurement 363433bd to history. Total: 3
th(0)=-22.092607408020108;dx=-6.026739679787846E-7
New Minimum: -22.092607408020108 > -22.846891057914277
WOLF (strong): th(6515010.502656417)=-22.846891057914277; dx=2.098906551203339E-7 evalInputDelta=0.7542836498941696
New Minimum: -22.846891057914277 > -23.385442006512257
END: th(3257505.2513282085)=-23.385442006512257; dx=-1.7108883536316933E-8 evalInputDelta=1.2928345984921492
Fitness changed from -22.092607408020108 to -23.385442006512257
Iteration 7 complete. Error: -23.385442006512257 Total: 11.3483; Orientation: 1.0694; Line Search: 7.6696
Adding measurement 5f4df319 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.069e+04, gradient 6.970e-04, dot -0.453; [1f8e6519-7bb2-4546-888f-5ddd04cc1917 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.385442006512257, -15.198203256825177, -14.174229673226101, -13.55486881142071
LBFGS Accumulation History: 3 points
Removed measurement 5f4df319 to history. Total: 3
th(0)=-23.385442006512257;dx=-4.85288077303068E-7
Armijo: th(7018082.316422523)=-23.3747676734644; dx=1.1129342990599857E-7 evalInputDelta=-0.01067433304785581
New Minimum: -23.385442006512257 > -23.88957783784381
WOLF (strong): th(3509041.1582112615)=-23.88957783784381; dx=1.1808527269814238E-7 evalInputDelta=0.5041358313315527
New Minimum: -23.88957783784381 > -23.889627542773773
END: th(1169680.3860704205)=-23.889627542773773; dx=-2.4805115279404357E-7 evalInputDelta=0.5041855362615166
Fitness changed from -23.385442006512257 to -23.889627542773773
Iteration 8 complete. Error: -23.889627542773773 Total: 13.8393; Orientation: 1.0318; Line Search: 10.2318
Adding measurement 23b34ad4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.335e+04, gradient 5.316e-04, dot -0.606; [1f8e6519-7bb2-4546-888f-5ddd04cc1917 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.889627542773773, -15.198203256825177, -14.174229673226101, -13.55486881142071
LBFGS Accumulation History: 3 points
Removed measurement 23b34ad4 to history. Total: 3
th(0)=-23.889627542773773;dx=-2.824032715612209E-7
New Minimum: -23.889627542773773 > -24.374385545260143
END: th(2520000.0000000005)=-24.374385545260143; dx=-9.02982000818788E-8 evalInputDelta=0.4847580024863696
Fitness changed from -23.889627542773773 to -24.374385545260143
Iteration 9 complete. Error: -24.374385545260143 Total: 8.9224; Orientation: 1.0319; Line Search: 5.1666
Adding measurement 560f209c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.483e+04, gradient 6.092e-04, dot -0.528; [1f8e6519-7bb2-4546-888f-5ddd04cc1917 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.374385545260143, -15.198203256825177, -14.174229673226101, -13.55486881142071
LBFGS Accumulation History: 3 points
Removed measurement 560f209c to history. Total: 3
th(0)=-24.374385545260143;dx=-3.7097384202007585E-7
New Minimum: -24.374385545260143 > -24.629828772403254
WOLF (strong): th(5429175.418880348)=-24.629828772403254; dx=1.9620270316280067E-8 evalInputDelta=0.2554432271431111
New Minimum: -24.629828772403254 > -24.724304820765788
WOLF (strong): th(2714587.709440174)=-24.724304820765788; dx=3.139462957178686E-8 evalInputDelta=0.34991927550564483
END: th(904862.5698133913)=-24.66719965430005; dx=-1.8625909067491883E-7 evalInputDelta=0.2928141090399059
Fitness changed from -24.374385545260143 to -24.724304820765788
Iteration 10 complete. Error: -24.724304820765788 Total: 14.0891; Orientation: 1.0608; Line Search: 10.4358
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -24.724304820765788 (> -Infinity) after 136.002s (< 5400.000s)
Returns
-24.724304820765788