BasicOptimizer.scala:89 executed in 68.90 seconds (0.767 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 145278151081400
Reset training subject: 145279054514600
Adding measurement 28152d64 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-95.44078931085103;dx=-0.006818646902751274
New Minimum: -95.44078931085103 > -95.45502197370668
WOLFE (weak): th(2.154434690031884)=-95.45502197370668; dx=-0.006644339604101334 evalInputDelta=0.014232662855647504
New Minimum: -95.45502197370668 > -95.46923876312815
WOLFE (weak): th(4.308869380063768)=-95.46923876312815; dx=-0.00663935412714187 evalInputDelta=0.028449452277115483
New Minimum: -95.46923876312815 > -95.52612599278388
WOLFE (weak): th(12.926608140191302)=-95.52612599278388; dx=-0.006634532726073226 evalInputDelta=0.08533668193284427
New Minimum: -95.52612599278388 > -95.7817415357808
WOLFE (weak): th(51.70643256076521)=-95.7817415357808; dx=-0.006634220549343005 evalInputDelta=0.3409522249297652
New Minimum: -95.7817415357808 > -97.15134166280646
WOLFE (weak): th(258.53216280382605)=-97.15134166280646; dx=-0.006639854613779273 evalInputDelta=1.7105523519554282
New Minimum: -97.15134166280646 > -105.62656659986412
WOLFE (weak): th(1551.1929768229563)=-105.62656659986412; dx=-0.006529084226644261 evalInputDelta=10.185777289013089
New Minimum: -105.62656659986412 > -154.4294976931261
END: th(10858.350837760694)=-154.4294976931261; dx=-0.0042428344513914425 evalInputDelta=58.98870838227508
Fitness changed from -95.44078931085103 to -154.4294976931261
Iteration 1 complete. Error: -154.4294976931261 Total: 9.9109; Orientation: 0.0156; Line Search: 7.1660
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 591bf32a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-154.4294976931261;dx=-0.005470637229908115
New Minimum: -154.4294976931261 > -243.4272943477545
END: th(23393.607721408407)=-243.4272943477545; dx=-0.0027678315016499977 evalInputDelta=88.99779665462839
Fitness changed from -154.4294976931261 to -243.4272943477545
Iteration 2 complete. Error: -243.4272943477545 Total: 2.6589; Orientation: 0.0145; Line Search: 1.7252
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 241ef7e0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-243.4272943477545;dx=-0.005594833320103835
New Minimum: -243.4272943477545 > -349.0756977789987
END: th(50400.00000000001)=-349.0756977789987; dx=-0.0014636140855252475 evalInputDelta=105.64840343124419
Fitness changed from -243.4272943477545 to -349.0756977789987
Iteration 3 complete. Error: -349.0756977789987 Total: 2.7359; Orientation: 0.0157; Line Search: 1.7739
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6754bfee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.842e+04, gradient 8.487e-02, dot -0.353; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -349.0756977789987, -243.4272943477545, -154.4294976931261, -95.44078931085103
LBFGS Accumulation History: 3 points
Removed measurement 6754bfee to history. Total: 3
th(0)=-349.0756977789987;dx=-0.007164447172145832
New Minimum: -349.0756977789987 > -455.90900188883217
END: th(108583.50837760697)=-455.90900188883217; dx=-3.4473736874575573E-4 evalInputDelta=106.83330410983348
Fitness changed from -349.0756977789987 to -455.90900188883217
Iteration 4 complete. Error: -455.90900188883217 Total: 3.0982; Orientation: 0.2994; Line Search: 1.8570
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 239cbacf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.962e+04, gradient 9.853e-02, dot -0.233; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -455.90900188883217, -243.4272943477545, -154.4294976931261, -95.44078931085103
LBFGS Accumulation History: 3 points
Removed measurement 239cbacf to history. Total: 3
th(0)=-455.90900188883217;dx=-0.00968111642420829
New Minimum: -455.90900188883217 > -502.2167380404006
WOLF (strong): th(233936.07721408413)=-502.2167380404006; dx=4.675319328383491E-4 evalInputDelta=46.30773615156846
New Minimum: -502.2167380404006 > -654.3562944967679
WOLF (strong): th(116968.03860704206)=-654.3562944967679; dx=2.4045884009469477E-4 evalInputDelta=198.4472926079357
New Minimum: -654.3562944967679 > -686.3761497164427
END: th(38989.346202347355)=-686.3761497164427; dx=-0.0024423548793918873 evalInputDelta=230.46714782761052
Fitness changed from -455.90900188883217 to -686.3761497164427
Iteration 5 complete. Error: -686.3761497164427 Total: 4.9255; Orientation: 0.2716; Line Search: 3.4539
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 360a455e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.957e+04, gradient 7.555e-02, dot -0.291; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -686.3761497164427, -243.4272943477545, -154.4294976931261, -95.44078931085103
LBFGS Accumulation History: 3 points
Removed measurement 360a455e to history. Total: 3
th(0)=-686.3761497164427;dx=-0.0056682917830709455
New Minimum: -686.3761497164427 > -879.447617484733
END: th(84000.00000000003)=-879.447617484733; dx=-9.05792232495702E-4 evalInputDelta=193.0714677682903
Fitness changed from -686.3761497164427 to -879.447617484733
Iteration 6 complete. Error: -879.447617484733 Total: 2.9387; Orientation: 0.2772; Line Search: 1.7518
Adding measurement 4750bba5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.594e+04, gradient 8.462e-02, dot -0.295; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -879.447617484733, -243.4272943477545, -154.4294976931261, -95.44078931085103
LBFGS Accumulation History: 3 points
Removed measurement 4750bba5 to history. Total: 3
th(0)=-879.447617484733;dx=-0.007112427293383589
New Minimum: -879.447617484733 > -889.2473432013729
WOLF (strong): th(180972.5139626783)=-889.2473432013729; dx=5.515869444921707E-4 evalInputDelta=9.799725716639955
New Minimum: -889.2473432013729 > -1016.7052412837852
END: th(90486.25698133915)=-1016.7052412837852; dx=-6.285356809001873E-5 evalInputDelta=137.25762379905223
Fitness changed from -879.447617484733 to -1016.7052412837852
Iteration 7 complete. Error: -1016.7052412837852 Total: 4.0520; Orientation: 0.3021; Line Search: 2.9039
Adding measurement 5bba859a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.007e+05, gradient 8.576e-02, dot -0.277; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1016.7052412837852, -243.4272943477545, -154.4294976931261, -95.44078931085103
LBFGS Accumulation History: 3 points
Removed measurement 5bba859a to history. Total: 3
th(0)=-1016.7052412837852;dx=-0.007311648982385087
Armijo: th(194946.7310117368)=-968.8920351833415; dx=0.0012896249144930829 evalInputDelta=-47.81320610044372
New Minimum: -1016.7052412837852 > -1202.5115732730842
WOLF (strong): th(97473.3655058684)=-1202.5115732730842; dx=2.2299700280438225E-4 evalInputDelta=185.806331989299
END: th(32491.121835289465)=-1177.2055967112751; dx=-0.0029231677299571476 evalInputDelta=160.50035542748992
Fitness changed from -1016.7052412837852 to -1202.5115732730842
Iteration 8 complete. Error: -1202.5115732730842 Total: 4.6842; Orientation: 0.2990; Line Search: 3.5329
Adding measurement 7da5d642 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.391e+05, gradient 8.819e-02, dot -0.283; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1202.5115732730842, -243.4272943477545, -154.4294976931261, -95.44078931085103
LBFGS Accumulation History: 3 points
Removed measurement 7da5d642 to history. Total: 3
th(0)=-1202.5115732730842;dx=-0.007744303862034475
New Minimum: -1202.5115732730842 > -1394.9133164771636
END: th(70000.00000000003)=-1394.9133164771636; dx=-5.637643039726591E-4 evalInputDelta=192.40174320407937
Fitness changed from -1202.5115732730842 to -1394.9133164771636
Iteration 9 complete. Error: -1394.9133164771636 Total: 2.8430; Orientation: 0.2892; Line Search: 1.7473
Adding measurement 37fba7f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.202e+07, gradient 7.707e-02, dot -0.330; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1394.9133164771636, -243.4272943477545, -154.4294976931261, -95.44078931085103
LBFGS Accumulation History: 3 points
Removed measurement 37fba7f to history. Total: 3
th(0)=-1394.9133164771636;dx=-0.005846748554457068
New Minimum: -1394.9133164771636 > -1447.4058197391148
WOLF (strong): th(150810.42830223194)=-1447.4058197391148; dx=7.896829464163844E-4 evalInputDelta=52.49250326195124
New Minimum: -1447.4058197391148 > -1546.032123999201
END: th(75405.21415111597)=-1546.032123999201; dx=-7.802368505570583E-4 evalInputDelta=151.1188075220375
Fitness changed from -1394.9133164771636 to -1546.032123999201
Iteration 10 complete. Error: -1546.032123999201 Total: 3.8857; Orientation: 0.2785; Line Search: 2.7900
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 770bbcb0 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.497e+04, gradient 7.634e-02, dot 0.466; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1546.032123999201;dx=-1583.635800503971
Armijo: th(162455.60917644735)=-775.8317259801972; dx=0.0025815539290797882 evalInputDelta=-770.2003980190038
Armijo: th(81227.80458822368)=-775.8733121721613; dx=0.005188811540456428 evalInputDelta=-770.1588118270398
Armijo: th(27075.934862741226)=-776.171335470573; dx=0.01556970357011023 evalInputDelta=-769.8607885286281
Armijo: th(6768.983715685306)=-776.8209598480693; dx=0.06214617477469185 evalInputDelta=-769.2111641511318
Armijo: th(1353.7967431370612)=-777.9937983655003; dx=0.3106503691680348 evalInputDelta=-768.0383256337008
Armijo: th(225.63279052284352)=-781.6214882986084; dx=1.8835597124472434 evalInputDelta=-764.4106357005927
Armijo: th(32.233255788977644)=-810.569384451775; dx=12.868972671577598 evalInputDelta=-735.462739547426
Armijo: th(4.0291569736222055)=-1003.9049078118007; dx=93.92812888155044 evalInputDelta=-542.1272161874003
New Minimum: -1546.032123999201 > -1626.9941520949708
END: th(0.44768410818024507)=-1626.9941520949708; dx=-22.25050224305415 evalInputDelta=80.96202809576971
Fitness changed from -1546.032123999201 to -1626.9941520949708
Iteration 11 complete. Error: -1626.9941520949708 Total: 10.0752; Orientation: 0.2716; Line Search: 8.8687
Adding measurement 6c3c9849 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.386e+04, gradient 8.662e-02, dot 0.167; [ce490236-f916-4afe-9fb2-b8cbb0464c11 = 1.000/1.000e+00]
th(0)=-1626.9941520949708;dx=-345.47279839740776
Armijo: th(0.9645061728395066)=-1424.4497676300039; dx=90.62152120591537 evalInputDelta=-202.54438446496692
Armijo: th(0.4822530864197533)=-1526.6811245081242; dx=67.18487051512986 evalInputDelta=-100.3130275868466
Armijo: th(0.16075102880658443)=-1596.516934586666; dx=21.93262162885858 evalInputDelta=-30.477217508304875
Armijo: th(0.04018775720164611)=-1619.914145698685; dx=-4.335839191647567 evalInputDelta=-7.080006396285853
Armijo: th(0.008037551440329221)=-1625.6077742791604; dx=-16.18262810126266 evalInputDelta=-1.3863778158104196
Armijo: th(0.0013395919067215368)=-1626.7646345995063; dx=-17.660832525357186 evalInputDelta=-0.22951749546450628
Armijo: th(1.9137027238879097E-4)=-1626.9614617739603; dx=-18.16479808160612 evalInputDelta=-0.03269032101047742
Armijo: th(2.392128404859887E-5)=-1626.9900722961509; dx=-19.05832813495723 evalInputDelta=-0.004079798819930147
Armijo: th(2.657920449844319E-6)=-1626.9936356653059; dx=-18.52048741159735 evalInputDelta=-5.164296649127209E-4
Armijo: th(2.657920449844319E-7)=-1626.9941004520115; dx=-19.35864482637872 evalInputDelta=-5.164295930626395E-5
Armijo: th(2.41629131804029E-8)=-1626.9941004520115; dx=-19.801880637339828 evalInputDelta=-5.164295930626395E-5
Armijo: th(2.0135760983669085E-9)=-1626.9941520949708; dx=-18.448232590398618 evalInputDelta=0.0
END: th(1.548904691051468E-10)=-1626.9941520949708; dx=-18.789813535174886 evalInputDelta=0.0
Fitness changed from -1626.9941520949708 to -1626.9941520949708
Static Iteration Total: 13.7146; Orientation: 0.5596; Line Search: 12.3437
Iteration 12 failed. Error: -1626.9941520949708
Previous Error: 0.0 -> -1626.9941520949708
Retrying iteration 12
Reset training subject: 145343673819200
Adding measurement 98dfd29 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-1626.9941520949708;dx=-0.0074433422553302365
END: th(3.3370139979544E-10)=-1626.9941520949708; dx=-0.00439019294273754 evalInputDelta=0.0
Fitness changed from -1626.9941520949708 to -1626.9941520949708
Static Iteration Total: 3.3762; Orientation: 0.0149; Line Search: 1.6904
Iteration 13 failed. Error: -1626.9941520949708
Previous Error: 0.0 -> -1626.9941520949708
Optimization terminated 13
Final threshold in iteration 13: -1626.9941520949708 (> -Infinity) after 68.899s (< 5400.000s)
Returns
-1626.9941520949708