BasicOptimizer.scala:89 executed in 72.67 seconds (1.183 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140989294921400
Reset training subject: 140990236918400
Adding measurement 557b6839 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-6.120674605679582;dx=-3.9890244628621086E-4
New Minimum: -6.120674605679582 > -6.121545340997168
WOLFE (weak): th(2.154434690031884)=-6.121545340997168; dx=-3.990815830567214E-4 evalInputDelta=8.707353175854138E-4
New Minimum: -6.121545340997168 > -6.122414291519855
WOLFE (weak): th(4.308869380063768)=-6.122414291519855; dx=-3.9876242805698644E-4 evalInputDelta=0.0017396858402731752
New Minimum: -6.122414291519855 > -6.125891351038026
WOLFE (weak): th(12.926608140191302)=-6.125891351038026; dx=-3.993892759958642E-4 evalInputDelta=0.005216745358444186
New Minimum: -6.125891351038026 > -6.141555773582132
WOLFE (weak): th(51.70643256076521)=-6.141555773582132; dx=-4.0024383030943383E-4 evalInputDelta=0.02088116790254979
New Minimum: -6.141555773582132 > -6.225236001683856
WOLFE (weak): th(258.53216280382605)=-6.225236001683856; dx=-3.994697856711133E-4 evalInputDelta=0.10456139600427417
New Minimum: -6.225236001683856 > -6.742231369913775
WOLFE (weak): th(1551.1929768229563)=-6.742231369913775; dx=-3.935459204348483E-4 evalInputDelta=0.6215567642341924
New Minimum: -6.742231369913775 > -9.761893676228235
END: th(10858.350837760694)=-9.761893676228235; dx=-2.522706559827003E-4 evalInputDelta=3.6412190705486527
Fitness changed from -6.120674605679582 to -9.761893676228235
Iteration 1 complete. Error: -9.761893676228235 Total: 10.3099; Orientation: 0.0171; Line Search: 7.6106
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 706fde09 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-9.761893676228235;dx=-4.30808246384421E-4
New Minimum: -9.761893676228235 > -14.223059086899822
END: th(23393.607721408407)=-14.223059086899822; dx=-9.88561474890213E-5 evalInputDelta=4.461165410671587
Fitness changed from -9.761893676228235 to -14.223059086899822
Iteration 2 complete. Error: -14.223059086899822 Total: 2.6063; Orientation: 0.0145; Line Search: 1.6784
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3e8b44c4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-14.223059086899822;dx=-5.345637028028971E-4
New Minimum: -14.223059086899822 > -18.31225502301223
END: th(50400.00000000001)=-18.31225502301223; dx=-3.577071756424704E-5 evalInputDelta=4.089195936112409
Fitness changed from -14.223059086899822 to -18.31225502301223
Iteration 3 complete. Error: -18.31225502301223 Total: 2.6197; Orientation: 0.0146; Line Search: 1.6663
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7301b2c6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.470e+03, gradient 2.023e-02, dot -0.261; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.31225502301223, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 7301b2c6 to history. Total: 3
th(0)=-18.31225502301223;dx=-4.078628705961371E-4
New Minimum: -18.31225502301223 > -25.710481675320242
END: th(108583.50837760697)=-25.710481675320242; dx=-1.8118414189585128E-5 evalInputDelta=7.398226652308011
Fitness changed from -18.31225502301223 to -25.710481675320242
Iteration 4 complete. Error: -25.710481675320242 Total: 3.1748; Orientation: 0.2835; Line Search: 1.9527
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3158f2fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.506e+03, gradient 1.774e-02, dot -0.229; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.710481675320242, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 3158f2fe to history. Total: 3
th(0)=-25.710481675320242;dx=-3.135010971195325E-4
New Minimum: -25.710481675320242 > -37.62748563172261
END: th(233936.07721408413)=-37.62748563172261; dx=-5.714866175789744E-6 evalInputDelta=11.917003956402365
Fitness changed from -25.710481675320242 to -37.62748563172261
Iteration 5 complete. Error: -37.62748563172261 Total: 2.9040; Orientation: 0.2772; Line Search: 1.6801
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 11a256f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.846e+03, gradient 1.716e-02, dot -0.191; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.62748563172261, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 11a256f9 to history. Total: 3
th(0)=-37.62748563172261;dx=-2.9389314865918234E-4
New Minimum: -37.62748563172261 > -52.03229260433557
WOLF (strong): th(504000.0000000002)=-52.03229260433557; dx=4.626292179887448E-6 evalInputDelta=14.40480697261296
New Minimum: -52.03229260433557 > -60.02416309372177
END: th(252000.0000000001)=-60.02416309372177; dx=-6.4698224213342614E-6 evalInputDelta=22.39667746199916
Fitness changed from -37.62748563172261 to -60.02416309372177
Iteration 6 complete. Error: -60.02416309372177 Total: 3.7578; Orientation: 0.2895; Line Search: 2.5535
Adding measurement 1843c650 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.411e+03, gradient 1.776e-02, dot -0.227; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -60.02416309372177, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 1843c650 to history. Total: 3
th(0)=-60.02416309372177;dx=-3.1441859502528437E-4
New Minimum: -60.02416309372177 > -73.19152239177639
WOLF (strong): th(542917.5418880349)=-73.19152239177639; dx=1.964051354525026E-5 evalInputDelta=13.16735929805462
New Minimum: -73.19152239177639 > -87.32325752067284
WOLF (strong): th(271458.77094401745)=-87.32325752067284; dx=2.859715504601286E-6 evalInputDelta=27.299094426951072
END: th(90486.25698133915)=-83.4746141478146; dx=-1.3784689333888253E-4 evalInputDelta=23.450451054092838
Fitness changed from -60.02416309372177 to -87.32325752067284
Iteration 7 complete. Error: -87.32325752067284 Total: 4.8181; Orientation: 0.2770; Line Search: 3.7446
Adding measurement 28c71b5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.343e+04, gradient 1.835e-02, dot -0.274; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -87.32325752067284, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 28c71b5 to history. Total: 3
th(0)=-87.32325752067284;dx=-3.3644845463711906E-4
New Minimum: -87.32325752067284 > -116.72367888838731
END: th(194946.7310117368)=-116.72367888838731; dx=-2.38888924847196E-5 evalInputDelta=29.400421367714472
Fitness changed from -87.32325752067284 to -116.72367888838731
Iteration 8 complete. Error: -116.72367888838731 Total: 2.8931; Orientation: 0.2948; Line Search: 1.7616
Adding measurement 5e6cd5e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.905e+04, gradient 1.678e-02, dot -0.369; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -116.72367888838731, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 5e6cd5e0 to history. Total: 3
th(0)=-116.72367888838731;dx=-2.795794212999607E-4
New Minimum: -116.72367888838731 > -127.51777537338556
WOLF (strong): th(420000.0000000002)=-127.51777537338556; dx=2.683299583089186E-5 evalInputDelta=10.79409648499825
New Minimum: -127.51777537338556 > -140.4705344512099
END: th(210000.0000000001)=-140.4705344512099; dx=-1.603633287421103E-5 evalInputDelta=23.746855562822574
Fitness changed from -116.72367888838731 to -140.4705344512099
Iteration 9 complete. Error: -140.4705344512099 Total: 3.6708; Orientation: 0.2770; Line Search: 2.5682
Adding measurement 6e575144 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.660e+05, gradient 1.711e-02, dot -0.393; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -140.4705344512099, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 6e575144 to history. Total: 3
th(0)=-140.4705344512099;dx=-2.919860658798747E-4
New Minimum: -140.4705344512099 > -140.4849798650067
WOLF (strong): th(452431.2849066958)=-140.4849798650067; dx=2.8103886965241093E-5 evalInputDelta=0.01444541379680686
New Minimum: -140.4849798650067 > -158.15346265999742
WOLF (strong): th(226215.6424533479)=-158.15346265999742; dx=9.985439125319654E-7 evalInputDelta=17.68292820878753
END: th(75405.21415111596)=-156.44156582862723; dx=-1.2502356190868078E-4 evalInputDelta=15.971031377417347
Fitness changed from -140.4705344512099 to -158.15346265999742
Iteration 10 complete. Error: -158.15346265999742 Total: 4.7265; Orientation: 0.2860; Line Search: 3.6129
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 78a7f26 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.239e+06, gradient 1.761e-02, dot -0.376; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -158.15346265999742, -14.223059086899822, -9.761893676228235, -6.120674605679582
LBFGS Accumulation History: 3 points
Removed measurement 78a7f26 to history. Total: 3
th(0)=-158.15346265999742;dx=-3.083812342065599E-4
New Minimum: -158.15346265999742 > -179.82554039592023
END: th(162455.60917644732)=-179.82554039592023; dx=-3.047270081390197E-5 evalInputDelta=21.672077735922812
Fitness changed from -158.15346265999742 to -179.82554039592023
Iteration 11 complete. Error: -179.82554039592023 Total: 2.9613; Orientation: 0.2775; Line Search: 1.7613
Adding measurement 47858c16 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.132e+05, gradient 1.594e-02, dot 0.470; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-179.82554039592023;dx=-849.7030438322231
Armijo: th(350000.0000000002)=-126.23075080180459; dx=2.6273914310321516E-4 evalInputDelta=-53.59478959411564
Armijo: th(175000.0000000001)=-126.23045692071813; dx=5.249113673237148E-4 evalInputDelta=-53.5950834752021
Armijo: th(58333.333333333365)=-126.23076111340923; dx=0.001577372725388408 evalInputDelta=-53.594779282511
Armijo: th(14583.333333333341)=-126.23149839095808; dx=0.006306497494758987 evalInputDelta=-53.59404200496215
Armijo: th(2916.6666666666683)=-126.22677045949469; dx=0.03155667480407229 evalInputDelta=-53.59876993642554
Armijo: th(486.11111111111137)=-126.26179007268296; dx=0.18950161008108285 evalInputDelta=-53.56375032323727
Armijo: th(69.44444444444449)=-126.92280925456242; dx=1.3282968812154157 evalInputDelta=-52.902731141357805
Armijo: th(8.68055555555556)=-131.41403666523627; dx=10.38192386240808 evalInputDelta=-48.41150373068396
Armijo: th(0.9645061728395068)=-177.69150325544257; dx=59.03715431509035 evalInputDelta=-2.134037140477659
New Minimum: -179.82554039592023 > -218.8532313251944
END: th(0.09645061728395068)=-218.8532313251944; dx=-269.2427586313561 evalInputDelta=39.02769092927417
Fitness changed from -179.82554039592023 to -218.8532313251944
Iteration 12 complete. Error: -218.8532313251944 Total: 11.1413; Orientation: 0.2689; Line Search: 10.0842
Adding measurement 2dbf8f19 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.381e+04, gradient 1.739e-02, dot 0.255; [88c4184f-3d37-43e6-8876-4b8f15e54493 = 1.000/1.000e+00]
th(0)=-218.8532313251944;dx=-105.99137071871905
Armijo: th(0.20779655575153214)=-207.20392204584073; dx=45.157931700132224 evalInputDelta=-11.649309279353673
Armijo: th(0.10389827787576607)=-213.48162708503196; dx=41.38419039411342 evalInputDelta=-5.371604240162441
Armijo: th(0.03463275929192202)=-217.18903482568257; dx=37.200852025645744 evalInputDelta=-1.6641964995118315
Armijo: th(0.008658189822980505)=-218.4485304511376; dx=35.56130091096289 evalInputDelta=-0.40470087405680033
Armijo: th(0.001731637964596101)=-218.77282940167132; dx=35.27822378450712 evalInputDelta=-0.08040192352308395
Armijo: th(2.886063274326835E-4)=-218.83986081293187; dx=35.159229574949904 evalInputDelta=-0.013370512262525835
Armijo: th(4.1229475347526214E-5)=-218.85131620407688; dx=35.12095051289212 evalInputDelta=-0.0019151211175199023
Armijo: th(5.153684418440777E-6)=-218.85299342286157; dx=35.09585770380737 evalInputDelta=-2.3790233282738882E-4
Armijo: th(5.726316020489752E-7)=-218.85319563986096; dx=35.11584814957292 evalInputDelta=-3.568533344378011E-5
Armijo: th(5.726316020489752E-8)=-218.8532313251944; dx=35.125506633436935 evalInputDelta=0.0
Armijo: th(5.205741836808866E-9)=-218.8532313251944; dx=35.102735067640864 evalInputDelta=0.0
Armijo: th(4.3381181973407215E-10)=-218.8532313251944; dx=35.092405099578244 evalInputDelta=0.0
MIN ALPHA (3.337013997954401E-11): th(0.0)=-218.8532313251944
Fitness changed from -218.8532313251944 to -218.8532313251944
Static Iteration Total: 13.5907; Orientation: 0.3687; Line Search: 12.4006
Iteration 13 failed. Error: -218.8532313251944
Previous Error: 0.0 -> -218.8532313251944
Retrying iteration 13
Reset training subject: 141058469328800
Adding measurement 2c5ca359 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-218.8532313251944;dx=-3.0221055489896285E-4
END: th(2.1544346900318838E-10)=-218.8532313251944; dx=-2.212444140141078E-4 evalInputDelta=0.0
Fitness changed from -218.8532313251944 to -218.8532313251944
Static Iteration Total: 3.4933; Orientation: 0.0140; Line Search: 1.6978
Iteration 14 failed. Error: -218.8532313251944
Previous Error: 0.0 -> -218.8532313251944
Optimization terminated 14
Final threshold in iteration 14: -218.8532313251944 (> -Infinity) after 72.668s (< 5400.000s)
Returns
-218.8532313251944