BasicOptimizer.scala:89 executed in 65.93 seconds (2.647 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 222877221532000
Reset training subject: 222878058010500
Adding measurement ae99f3 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-22.03494712243341;dx=-0.003070466642841591
New Minimum: -22.03494712243341 > -22.039157495449725
WOLFE (weak): th(2.154434690031884)=-22.039157495449725; dx=-0.0030693476608128285 evalInputDelta=0.00421037301631344
New Minimum: -22.039157495449725 > -22.043371173083145
WOLFE (weak): th(4.308869380063768)=-22.043371173083145; dx=-0.003068204394490133 evalInputDelta=0.008424050649733061
New Minimum: -22.043371173083145 > -22.060197124201753
WOLFE (weak): th(12.926608140191302)=-22.060197124201753; dx=-0.0030646650791538157 evalInputDelta=0.02525000176834169
New Minimum: -22.060197124201753 > -22.135601789488888
WOLFE (weak): th(51.70643256076521)=-22.135601789488888; dx=-0.003054430251432682 evalInputDelta=0.10065466705547621
New Minimum: -22.135601789488888 > -22.538014999391354
WOLFE (weak): th(258.53216280382605)=-22.538014999391354; dx=-0.0030489240159998123 evalInputDelta=0.5030678769579424
New Minimum: -22.538014999391354 > -25.050186350009444
WOLFE (weak): th(1551.1929768229563)=-25.050186350009444; dx=-0.002943873802213582 evalInputDelta=3.015239227576032
New Minimum: -25.050186350009444 > -38.75381771029057
END: th(10858.350837760694)=-38.75381771029057; dx=-0.0017763435961973975 evalInputDelta=16.718870587857158
Fitness changed from -22.03494712243341 to -38.75381771029057
Iteration 1 complete. Error: -38.75381771029057 Total: 10.3218; Orientation: 0.0220; Line Search: 7.9491
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 42042bdb to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-38.75381771029057;dx=-0.003647937565934574
New Minimum: -38.75381771029057 > -61.76202097571447
END: th(23393.607721408407)=-61.76202097571447; dx=-9.821222414103646E-4 evalInputDelta=23.0082032654239
Fitness changed from -38.75381771029057 to -61.76202097571447
Iteration 2 complete. Error: -61.76202097571447 Total: 2.2776; Orientation: 0.0265; Line Search: 1.5314
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1cc6a41e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-61.76202097571447;dx=-0.003227453182433038
New Minimum: -61.76202097571447 > -74.70826336177676
END: th(50400.00000000001)=-74.70826336177676; dx=-4.753431639873949E-4 evalInputDelta=12.946242386062288
Fitness changed from -61.76202097571447 to -74.70826336177676
Iteration 3 complete. Error: -74.70826336177676 Total: 2.4631; Orientation: 0.0354; Line Search: 1.6382
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7eb20d3d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.086e+03, gradient 5.331e-02, dot -0.233; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.70826336177676, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 7eb20d3d to history. Total: 3
th(0)=-74.70826336177676;dx=-0.0028337677156490735
New Minimum: -74.70826336177676 > -79.31295558425302
END: th(108583.50837760697)=-79.31295558425302; dx=-1.9353626059412962E-4 evalInputDelta=4.60469222247626
Fitness changed from -74.70826336177676 to -79.31295558425302
Iteration 4 complete. Error: -79.31295558425302 Total: 2.9030; Orientation: 0.5606; Line Search: 1.5411
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6a2f4f69 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.220e+03, gradient 4.944e-02, dot -0.174; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.31295558425302, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 6a2f4f69 to history. Total: 3
th(0)=-79.31295558425302;dx=-0.0024357542974154257
Armijo: th(233936.07721408413)=-58.48959344723947; dx=-8.709340542796976E-6 evalInputDelta=-20.823362137013547
New Minimum: -79.31295558425302 > -110.50122673928904
END: th(116968.03860704206)=-110.50122673928904; dx=-2.95940766983192E-4 evalInputDelta=31.18827115503602
Fitness changed from -79.31295558425302 to -110.50122673928904
Iteration 5 complete. Error: -110.50122673928904 Total: 3.8158; Orientation: 0.5620; Line Search: 2.4311
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 934d654 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.584e+03, gradient 4.728e-02, dot -0.172; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -110.50122673928904, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 934d654 to history. Total: 3
th(0)=-110.50122673928904;dx=-0.0022307640147779004
Armijo: th(252000.0000000001)=-81.56130385905334; dx=-1.172240793948432E-5 evalInputDelta=-28.9399228802357
New Minimum: -110.50122673928904 > -147.2713401931438
END: th(126000.00000000004)=-147.2713401931438; dx=-3.4985585869852867E-4 evalInputDelta=36.77011345385476
Fitness changed from -110.50122673928904 to -147.2713401931438
Iteration 6 complete. Error: -147.2713401931438 Total: 4.7870; Orientation: 0.4949; Line Search: 3.4912
Adding measurement 75b71ffd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.480e+04, gradient 4.822e-02, dot -0.185; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -147.2713401931438, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 75b71ffd to history. Total: 3
th(0)=-147.2713401931438;dx=-0.002323215970995295
Armijo: th(271458.77094401745)=-100.95105218551376; dx=1.2629028486820708E-5 evalInputDelta=-46.32028800763004
New Minimum: -147.2713401931438 > -184.92687880363687
END: th(135729.38547200873)=-184.92687880363687; dx=-3.8145845804157916E-4 evalInputDelta=37.65553861049307
Fitness changed from -147.2713401931438 to -184.92687880363687
Iteration 7 complete. Error: -184.92687880363687 Total: 3.7730; Orientation: 0.5635; Line Search: 2.4444
Adding measurement 755204c9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.896e+04, gradient 4.903e-02, dot -0.189; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -184.92687880363687, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 755204c9 to history. Total: 3
th(0)=-184.92687880363687;dx=-0.002397544367805091
Armijo: th(292420.0965176052)=-121.53874004131674; dx=2.2092538909781043E-5 evalInputDelta=-63.388138762320125
New Minimum: -184.92687880363687 > -217.2209018531475
END: th(146210.0482588026)=-217.2209018531475; dx=-3.537704662586341E-4 evalInputDelta=32.29402304951063
Fitness changed from -184.92687880363687 to -217.2209018531475
Iteration 8 complete. Error: -217.2209018531475 Total: 3.2790; Orientation: 0.4719; Line Search: 2.1780
Adding measurement 6d74050d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.032e+04, gradient 4.870e-02, dot -0.197; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -217.2209018531475, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 6d74050d to history. Total: 3
th(0)=-217.2209018531475;dx=-0.002368345832355769
Armijo: th(315000.0000000001)=-137.2614901938279; dx=4.2155245112772126E-5 evalInputDelta=-79.95941165931958
New Minimum: -217.2209018531475 > -248.46004334802763
END: th(157500.00000000006)=-248.46004334802763; dx=-3.440613705135452E-4 evalInputDelta=31.23914149488013
Fitness changed from -217.2209018531475 to -248.46004334802763
Iteration 9 complete. Error: -248.46004334802763 Total: 3.6581; Orientation: 0.5949; Line Search: 2.3508
Adding measurement 3f05c97d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.933e+05, gradient 4.880e-02, dot -0.192; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -248.46004334802763, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 3f05c97d to history. Total: 3
th(0)=-248.46004334802763;dx=-0.002374764439035573
Armijo: th(339323.4636800218)=-143.04573159042408; dx=5.584755450308035E-5 evalInputDelta=-105.41431175760354
New Minimum: -248.46004334802763 > -270.604523299153
END: th(169661.7318400109)=-270.604523299153; dx=-3.2299911870564186E-4 evalInputDelta=22.14447995112539
Fitness changed from -248.46004334802763 to -270.604523299153
Iteration 10 complete. Error: -270.604523299153 Total: 3.9314; Orientation: 0.5044; Line Search: 2.7019
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 2ddb595b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.292e+05, gradient 4.936e-02, dot -0.182; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -270.604523299153, -61.76202097571447, -38.75381771029057, -22.03494712243341
LBFGS Accumulation History: 3 points
Removed measurement 2ddb595b to history. Total: 3
th(0)=-270.604523299153;dx=-0.002435069090087178
Armijo: th(365525.1206470065)=-147.40416394633675; dx=7.051773792183274E-5 evalInputDelta=-123.20035935281626
New Minimum: -270.604523299153 > -288.3358187697979
END: th(182762.56032350325)=-288.3358187697979; dx=-2.922735123546248E-4 evalInputDelta=17.73129547064491
Fitness changed from -270.604523299153 to -288.3358187697979
Iteration 11 complete. Error: -288.3358187697979 Total: 3.6235; Orientation: 0.5081; Line Search: 2.3940
Adding measurement 35c08057 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.843e+04, gradient 5.081e-02, dot 0.210; [74b6141e-fb14-4f14-bcfb-eff4d00d5098 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-288.3358187697979;dx=-516.7162358264434
Armijo: th(393750.0000000002)=566.5430631888645; dx=0.001095660409465419 evalInputDelta=-854.8788819586624
Armijo: th(196875.0000000001)=566.1896012447122; dx=0.0021908318541508376 evalInputDelta=-854.5254200145101
Armijo: th(65625.00000000003)=564.9388532070026; dx=0.006561846893456062 evalInputDelta=-853.2746719768005
Armijo: th(16406.250000000007)=561.7089478516602; dx=0.026070629044848463 evalInputDelta=-850.0447666214582
Armijo: th(3281.2500000000014)=555.008192466011; dx=0.12873213885143941 evalInputDelta=-843.3440112358089
Armijo: th(546.8750000000002)=541.4878540995068; dx=0.7541235601079703 evalInputDelta=-829.8236728693047
Armijo: th(78.12500000000003)=508.72498797802024; dx=5.006234681158672 evalInputDelta=-797.0608067478181
Armijo: th(9.765625000000004)=380.15609937245483; dx=32.10563514907447 evalInputDelta=-668.4919181422528
Armijo: th(1.0850694444444449)=-23.510043962549048; dx=104.56074187108368 evalInputDelta=-264.8257748072489
Armijo: th(0.10850694444444449)=-274.72180323079084; dx=-92.71152904708653 evalInputDelta=-13.614015539007084
Armijo: th(0.009864267676767681)=-288.00333325183016; dx=-213.2036805170236 evalInputDelta=-0.3324855179677684
Armijo: th(8.220223063973068E-4)=-288.31815943852405; dx=-228.44570712612722 evalInputDelta=-0.017659331273875978
Armijo: th(6.323248510748514E-5)=-288.3345218895886; dx=-230.79754458025633 evalInputDelta=-0.0012968802093382692
Armijo: th(4.516606079106081E-6)=-288.33570740812326; dx=-230.8339099653063 evalInputDelta=-1.1136167466929692E-4
Armijo: th(3.011070719404054E-7)=-288.33581125853004; dx=-230.32492412591589 evalInputDelta=-7.511267881454842E-6
Armijo: th(1.8819191996275338E-8)=-288.3358187697979; dx=-230.7206078006319 evalInputDelta=0.0
Armijo: th(1.1070112938985492E-9)=-288.3358187697979; dx=-230.8906039257962 evalInputDelta=0.0
MIN ALPHA (6.150062743880829E-11): th(0.0)=-288.3358187697979
Fitness changed from -288.3358187697979 to -288.3358187697979
Static Iteration Total: 17.3233; Orientation: 0.5343; Line Search: 16.0612
Iteration 12 failed. Error: -288.3358187697979
Previous Error: 0.0 -> -288.3358187697979
Retrying iteration 12
Reset training subject: 222939378315700
Adding measurement 5842fe98 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-288.3358187697979;dx=-0.002580373034122098
END: th(2.1544346900318838E-10)=-288.3358187697979; dx=-0.0019296446100295689 evalInputDelta=0.0
Fitness changed from -288.3358187697979 to -288.3358187697979
Static Iteration Total: 3.7702; Orientation: 0.0259; Line Search: 1.4880
Iteration 13 failed. Error: -288.3358187697979
Previous Error: 0.0 -> -288.3358187697979
Optimization terminated 13
Final threshold in iteration 13: -288.3358187697979 (> -Infinity) after 65.926s (< 5400.000s)
Returns
-288.3358187697979