BasicOptimizer.scala:89 executed in 61.29 seconds (1.457 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 159022197772200
Reset training subject: 159023780404100
Adding measurement 18cc38cd to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-23.83004422648582;dx=-4.3301047877431474E-4
New Minimum: -23.83004422648582 > -23.831323531677672
WOLFE (weak): th(2.154434690031884)=-23.831323531677672; dx=-4.3087702373677126E-4 evalInputDelta=0.0012793051918507103
New Minimum: -23.831323531677672 > -23.832601995732873
WOLFE (weak): th(4.308869380063768)=-23.832601995732873; dx=-4.280189690765269E-4 evalInputDelta=0.0025577692470513114
New Minimum: -23.832601995732873 > -23.837711304790563
WOLFE (weak): th(12.926608140191302)=-23.837711304790563; dx=-4.306150066560869E-4 evalInputDelta=0.007667078304741892
New Minimum: -23.837711304790563 > -23.8607031587776
WOLFE (weak): th(51.70643256076521)=-23.8607031587776; dx=-4.315529444906655E-4 evalInputDelta=0.03065893229177874
New Minimum: -23.8607031587776 > -23.983416738910638
WOLFE (weak): th(258.53216280382605)=-23.983416738910638; dx=-4.3098534465424345E-4 evalInputDelta=0.15337251242481642
New Minimum: -23.983416738910638 > -24.74837042701459
WOLFE (weak): th(1551.1929768229563)=-24.74837042701459; dx=-4.2649852942081067E-4 evalInputDelta=0.9183262005287673
New Minimum: -24.74837042701459 > -30.133493711540094
WOLFE (weak): th(10858.350837760694)=-30.133493711540094; dx=-4.011899654869219E-4 evalInputDelta=6.303449485054273
New Minimum: -30.133493711540094 > -52.75927129613913
END: th(86866.80670208555)=-52.75927129613913; dx=-9.997428031130259E-5 evalInputDelta=28.92922706965331
Fitness changed from -23.83004422648582 to -52.75927129613913
Iteration 1 complete. Error: -52.75927129613913 Total: 10.9042; Orientation: 0.0145; Line Search: 7.6975
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 353f1487 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-52.75927129613913;dx=-3.420963007470899E-4
New Minimum: -52.75927129613913 > -80.57137340014417
END: th(187148.86177126726)=-80.57137340014417; dx=-3.30169364332594E-5 evalInputDelta=27.812102104005035
Fitness changed from -52.75927129613913 to -80.57137340014417
Iteration 2 complete. Error: -80.57137340014417 Total: 2.6228; Orientation: 0.0148; Line Search: 1.6670
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement fd8514 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-80.57137340014417;dx=-4.5234435393262E-4
New Minimum: -80.57137340014417 > -95.46048578923376
WOLF (strong): th(403200.00000000006)=-95.46048578923376; dx=1.9823775478568133E-5 evalInputDelta=14.88911238908959
New Minimum: -95.46048578923376 > -104.58412954021499
END: th(201600.00000000003)=-104.58412954021499; dx=-2.6962368612549886E-6 evalInputDelta=24.01275614007082
Fitness changed from -80.57137340014417 to -104.58412954021499
Iteration 3 complete. Error: -104.58412954021499 Total: 3.4880; Orientation: 0.0142; Line Search: 2.5799
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7735cde6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.030e+03, gradient 2.125e-02, dot -0.358; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -104.58412954021499, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 7735cde6 to history. Total: 3
th(0)=-104.58412954021499;dx=-4.468857608445736E-4
Armijo: th(434334.03351042786)=-103.46332713541345; dx=4.077146851373182E-5 evalInputDelta=-1.1208024048015375
New Minimum: -104.58412954021499 > -127.37558091180823
WOLF (strong): th(217167.01675521393)=-127.37558091180823; dx=2.181804442525569E-5 evalInputDelta=22.79145137159324
New Minimum: -127.37558091180823 > -128.84941650765003
END: th(72389.00558507131)=-128.84941650765003; dx=-1.4328182287825055E-4 evalInputDelta=24.265286967435046
Fitness changed from -104.58412954021499 to -128.84941650765003
Iteration 4 complete. Error: -128.84941650765003 Total: 4.7331; Orientation: 0.2742; Line Search: 3.5405
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 24b5bb1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.379e+03, gradient 1.695e-02, dot -0.399; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -128.84941650765003, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 24b5bb1 to history. Total: 3
th(0)=-128.84941650765003;dx=-2.842501137949538E-4
New Minimum: -128.84941650765003 > -151.0309864951542
END: th(155957.38480938942)=-151.0309864951542; dx=-7.289599713980193E-5 evalInputDelta=22.18156998750416
Fitness changed from -128.84941650765003 to -151.0309864951542
Iteration 5 complete. Error: -151.0309864951542 Total: 2.8155; Orientation: 0.2804; Line Search: 1.6586
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 57563fad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.654e+04, gradient 1.845e-02, dot -0.421; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -151.0309864951542, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 57563fad to history. Total: 3
th(0)=-151.0309864951542;dx=-3.3888800260115147E-4
New Minimum: -151.0309864951542 > -161.57019333978002
END: th(336000.0000000001)=-161.57019333978002; dx=-4.906936585296113E-6 evalInputDelta=10.539206844625824
Fitness changed from -151.0309864951542 to -161.57019333978002
Iteration 6 complete. Error: -161.57019333978002 Total: 2.9381; Orientation: 0.2840; Line Search: 1.7162
Adding measurement 363fca5f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.678e+04, gradient 2.148e-02, dot -0.287; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -161.57019333978002, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 363fca5f to history. Total: 3
th(0)=-161.57019333978002;dx=-4.585182414013044E-4
Armijo: th(723890.0558507132)=-126.77569462253477; dx=5.893115417374537E-5 evalInputDelta=-34.79449871724525
New Minimum: -161.57019333978002 > -182.63052555691021
WOLF (strong): th(361945.0279253566)=-182.63052555691021; dx=6.0244531158777E-5 evalInputDelta=21.060332217130195
New Minimum: -182.63052555691021 > -199.97351542747734
END: th(120648.34264178552)=-199.97351542747734; dx=-1.2251199993593994E-4 evalInputDelta=38.40332208769732
Fitness changed from -161.57019333978002 to -199.97351542747734
Iteration 7 complete. Error: -199.97351542747734 Total: 4.6434; Orientation: 0.2761; Line Search: 3.5927
Adding measurement 20caffd8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.169e+04, gradient 1.715e-02, dot -0.387; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -199.97351542747734, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 20caffd8 to history. Total: 3
th(0)=-199.97351542747734;dx=-2.9057983474500766E-4
New Minimum: -199.97351542747734 > -223.28098293299126
END: th(259928.9746823157)=-223.28098293299126; dx=-1.9806401394462596E-5 evalInputDelta=23.307467505513927
Fitness changed from -199.97351542747734 to -223.28098293299126
Iteration 8 complete. Error: -223.28098293299126 Total: 2.7782; Orientation: 0.2750; Line Search: 1.6666
Adding measurement 5151dee9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.596e+04, gradient 2.233e-02, dot -0.351; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -223.28098293299126, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 5151dee9 to history. Total: 3
th(0)=-223.28098293299126;dx=-4.96457768466816E-4
Armijo: th(560000.0000000002)=-194.67811837490524; dx=5.11600448156665E-5 evalInputDelta=-28.602864558086026
New Minimum: -223.28098293299126 > -230.80416796032387
WOLF (strong): th(280000.0000000001)=-230.80416796032387; dx=4.8062077754134865E-5 evalInputDelta=7.523185027332602
New Minimum: -230.80416796032387 > -247.65934674028108
END: th(93333.33333333337)=-247.65934674028108; dx=-7.718517463375723E-5 evalInputDelta=24.378363807289816
Fitness changed from -223.28098293299126 to -247.65934674028108
Iteration 9 complete. Error: -247.65934674028108 Total: 4.4536; Orientation: 0.2751; Line Search: 3.3981
Adding measurement 53a2d612 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.617e+04, gradient 1.596e-02, dot -0.452; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -247.65934674028108, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 53a2d612 to history. Total: 3
th(0)=-247.65934674028108;dx=-2.513307766076736E-4
New Minimum: -247.65934674028108 > -267.4960551708153
END: th(201080.57106964258)=-267.4960551708153; dx=-6.0239362846137336E-5 evalInputDelta=19.836708430534202
Fitness changed from -247.65934674028108 to -267.4960551708153
Iteration 10 complete. Error: -267.4960551708153 Total: 2.8424; Orientation: 0.2726; Line Search: 1.7253
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7004a924 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.160e+05, gradient 1.795e-02, dot -0.455; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -267.4960551708153, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 7004a924 to history. Total: 3
th(0)=-267.4960551708153;dx=-3.1855711850204896E-4
Armijo: th(433214.9578038596)=-261.8501048564419; dx=1.896572477775621E-5 evalInputDelta=-5.645950314373408
New Minimum: -267.4960551708153 > -279.97624715125136
END: th(216607.4789019298)=-279.97624715125136; dx=-5.321234685488081E-6 evalInputDelta=12.480191980436075
Fitness changed from -267.4960551708153 to -279.97624715125136
Iteration 11 complete. Error: -279.97624715125136 Total: 3.7168; Orientation: 0.2855; Line Search: 2.5283
Adding measurement 148e3d11 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.048e+05, gradient 1.927e-02, dot -0.378; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -279.97624715125136, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 148e3d11 to history. Total: 3
th(0)=-279.97624715125136;dx=-3.7011954177521787E-4
Armijo: th(466666.666666667)=-243.77133248308584; dx=1.1248361878171746E-4 evalInputDelta=-36.204914668165515
New Minimum: -279.97624715125136 > -296.30427465177524
WOLF (strong): th(233333.3333333335)=-296.30427465177524; dx=4.467759640493696E-5 evalInputDelta=16.328027500523888
New Minimum: -296.30427465177524 > -299.281652307345
END: th(77777.77777777782)=-299.281652307345; dx=-1.551068150083688E-4 evalInputDelta=19.305405156093627
Fitness changed from -279.97624715125136 to -299.281652307345
Iteration 12 complete. Error: -299.281652307345 Total: 4.8785; Orientation: 0.3344; Line Search: 3.6516
Adding measurement 40e67aca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.557e+05, gradient 1.537e-02, dot -0.496; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -299.281652307345, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 40e67aca to history. Total: 3
th(0)=-299.281652307345;dx=-2.3472018501510486E-4
New Minimum: -299.281652307345 > -316.4785629989081
END: th(167567.1425580355)=-316.4785629989081; dx=-9.114420689065174E-5 evalInputDelta=17.19691069156312
Fitness changed from -299.281652307345 to -316.4785629989081
Iteration 13 complete. Error: -316.4785629989081 Total: 2.8081; Orientation: 0.2920; Line Search: 1.7106
Adding measurement 6423963 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.327e+08, gradient 1.522e-02, dot -0.533; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -316.4785629989081, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 6423963 to history. Total: 3
th(0)=-316.4785629989081;dx=-2.291541786156542E-4
New Minimum: -316.4785629989081 > -331.8072589351861
END: th(361012.4648365497)=-331.8072589351861; dx=-4.1831251380583274E-5 evalInputDelta=15.328695936277995
Fitness changed from -316.4785629989081 to -331.8072589351861
Iteration 14 complete. Error: -331.8072589351861 Total: 2.8549; Orientation: 0.2734; Line Search: 1.7467
Adding measurement 3ff8489c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.002e+08, gradient 1.929e-02, dot -0.392; [88dffded-5dea-48b5-b45c-66f96340086d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -331.8072589351861, -80.57137340014417, -52.75927129613913, -23.83004422648582
LBFGS Accumulation History: 3 points
Removed measurement 3ff8489c to history. Total: 3
th(0)=-331.8072589351861;dx=-3.668893869843536E-4
Armijo: th(777777.7777777783)=-239.77172850483328; dx=7.323190852019929E-5 evalInputDelta=-92.03553043035282
Armijo: th(388888.88888888917)=-315.9990872214335; dx=1.0304918242582792E-4 evalInputDelta=-15.808171713752586
New Minimum: -331.8072589351861 > -353.1767939138755
END: th(129629.62962962972)=-353.1767939138755; dx=-5.8971079808508965E-5 evalInputDelta=21.369534978689387
Fitness changed from -331.8072589351861 to -353.1767939138755
Iteration 15 complete. Error: -353.1767939138755 Total: 4.7104; Orientation: 0.2721; Line Search: 3.6649
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -353.1767939138755 (> -Infinity) after 61.284s (< 5400.000s)
Returns
-353.1767939138755