BasicOptimizer.scala:89 executed in 16.58 seconds (0.889 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 142145641374600
Reset training subject: 142145771406400
Adding measurement 491b72da to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-22.743821866728307;dx=-0.16655077985874028
New Minimum: -22.743821866728307 > -23.26237568604277
WOLFE (weak): th(2.154434690031884)=-23.26237568604277; dx=-0.16894520811389413 evalInputDelta=0.5185538193144623
New Minimum: -23.26237568604277 > -23.784552887424383
WOLFE (weak): th(4.308869380063768)=-23.784552887424383; dx=-0.16556290917714664 evalInputDelta=1.0407310206960751
New Minimum: -23.784552887424383 > -25.901827408233913
WOLFE (weak): th(12.926608140191302)=-25.901827408233913; dx=-0.16690608357913478 evalInputDelta=3.158005541505606
New Minimum: -25.901827408233913 > -35.6336937719897
WOLFE (weak): th(51.70643256076521)=-35.6336937719897; dx=-0.15965199935649502 evalInputDelta=12.88987190526139
New Minimum: -35.6336937719897 > -72.80146778010386
END: th(258.53216280382605)=-72.80146778010386; dx=-0.09098028903410162 evalInputDelta=50.05764591337555
Fitness changed from -22.743821866728307 to -72.80146778010386
Iteration 1 complete. Error: -72.80146778010386 Total: 1.0271; Orientation: 0.0016; Line Search: 0.6753
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 700f51d1 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-72.80146778010386;dx=-0.17967028192156984
New Minimum: -72.80146778010386 > -135.91700995000588
END: th(556.9906600335335)=-135.91700995000588; dx=-0.023540099718552816 evalInputDelta=63.115542169902014
Fitness changed from -72.80146778010386 to -135.91700995000588
Iteration 2 complete. Error: -135.91700995000588 Total: 0.3508; Orientation: 0.0015; Line Search: 0.2281
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3470ce2 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-135.91700995000588;dx=-0.2195499654728556
New Minimum: -135.91700995000588 > -171.36684540226852
WOLF (strong): th(1200.0)=-171.36684540226852; dx=0.004193203700641683 evalInputDelta=35.44983545226265
New Minimum: -171.36684540226852 > -204.3194400779084
END: th(600.0)=-204.3194400779084; dx=-0.030228159514896276 evalInputDelta=68.40243012790253
Fitness changed from -135.91700995000588 to -204.3194400779084
Iteration 3 complete. Error: -204.3194400779084 Total: 0.4642; Orientation: 0.0013; Line Search: 0.3428
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 8fd358c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.402e+03, gradient 6.066e-01, dot -0.326; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -204.3194400779084, -135.91700995000588, -72.80146778010386, -22.743821866728307
LBFGS Accumulation History: 3 points
Removed measurement 8fd358c to history. Total: 3
th(0)=-204.3194400779084;dx=-0.35660303537553273
New Minimum: -204.3194400779084 > -208.42177601860607
WOLF (strong): th(1292.6608140191304)=-208.42177601860607; dx=0.018932319665031217 evalInputDelta=4.102335940697657
New Minimum: -208.42177601860607 > -287.4734090963893
WOLF (strong): th(646.3304070095652)=-287.4734090963893; dx=0.007930883664105335 evalInputDelta=83.15396901848086
END: th(215.4434690031884)=-270.5617420294303; dx=-0.12217703854669483 evalInputDelta=66.24230195152191
Fitness changed from -204.3194400779084 to -287.4734090963893
Iteration 4 complete. Error: -287.4734090963893 Total: 0.6247; Orientation: 0.0309; Line Search: 0.4727
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1d236bce to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.438e+03, gradient 6.378e-01, dot -0.230; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -287.4734090963893, -135.91700995000588, -72.80146778010386, -22.743821866728307
LBFGS Accumulation History: 3 points
Removed measurement 1d236bce to history. Total: 3
th(0)=-287.4734090963893;dx=-0.3981302474487002
New Minimum: -287.4734090963893 > -359.68136940909244
END: th(464.15888336127796)=-359.68136940909244; dx=-0.03361702065416064 evalInputDelta=72.20796031270316
Fitness changed from -287.4734090963893 to -359.68136940909244
Iteration 5 complete. Error: -359.68136940909244 Total: 0.3792; Orientation: 0.0279; Line Search: 0.2263
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 376350eb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.547e+03, gradient 6.941e-01, dot -0.224; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -359.68136940909244, -135.91700995000588, -72.80146778010386, -22.743821866728307
LBFGS Accumulation History: 3 points
Removed measurement 376350eb to history. Total: 3
th(0)=-359.68136940909244;dx=-0.4681046832789112
Armijo: th(1000.0000000000002)=-270.3464501801346; dx=0.04167039722773725 evalInputDelta=-89.33491922895786
New Minimum: -359.68136940909244 > -387.3209465352991
WOLF (strong): th(500.0000000000001)=-387.3209465352991; dx=0.078835537415729 evalInputDelta=27.639577126206632
New Minimum: -387.3209465352991 > -430.481852114581
END: th(166.6666666666667)=-430.481852114581; dx=-0.17938983201692887 evalInputDelta=70.80048270548855
Fitness changed from -359.68136940909244 to -430.481852114581
Iteration 6 complete. Error: -430.481852114581 Total: 0.6014; Orientation: 0.0282; Line Search: 0.4544
Adding measurement d4ceabc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.770e+03, gradient 5.384e-01, dot -0.346; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -430.481852114581, -135.91700995000588, -72.80146778010386, -22.743821866728307
LBFGS Accumulation History: 3 points
Removed measurement d4ceabc to history. Total: 3
th(0)=-430.481852114581;dx=-0.28102492356907127
New Minimum: -430.481852114581 > -495.97722351938705
END: th(359.0724483386474)=-495.97722351938705; dx=-0.10345798218413381 evalInputDelta=65.49537140480606
Fitness changed from -430.481852114581 to -495.97722351938705
Iteration 7 complete. Error: -495.97722351938705 Total: 0.3647; Orientation: 0.0269; Line Search: 0.2239
Adding measurement 1bf0e727 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.809e+04, gradient 5.831e-01, dot -0.348; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -495.97722351938705, -135.91700995000588, -72.80146778010386, -22.743821866728307
LBFGS Accumulation History: 3 points
Removed measurement 1bf0e727 to history. Total: 3
th(0)=-495.97722351938705;dx=-0.3348857313697633
New Minimum: -495.97722351938705 > -547.3907139785987
END: th(773.5981389354635)=-547.3907139785987; dx=-0.03375230281683268 evalInputDelta=51.41349045921163
Fitness changed from -495.97722351938705 to -547.3907139785987
Iteration 8 complete. Error: -547.3907139785987 Total: 0.3616; Orientation: 0.0279; Line Search: 0.2254
Adding measurement 6e9a0e72 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.926e+04, gradient 7.252e-01, dot -0.285; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -547.3907139785987, -135.91700995000588, -72.80146778010386, -22.743821866728307
LBFGS Accumulation History: 3 points
Removed measurement 6e9a0e72 to history. Total: 3
th(0)=-547.3907139785987;dx=-0.5172539403962151
New Minimum: -547.3907139785987 > -588.8685920899501
WOLF (strong): th(1666.6666666666674)=-588.8685920899501; dx=0.004436402634353621 evalInputDelta=41.4778781113514
New Minimum: -588.8685920899501 > -662.5103772772167
WOLF (strong): th(833.3333333333337)=-662.5103772772167; dx=0.010706819096641217 evalInputDelta=115.11966329861798
New Minimum: -662.5103772772167 > -689.6090604465693
END: th(277.7777777777779)=-689.609060
...skipping 6101 bytes...
E-5
New Minimum: -689.6090831043339 > -689.6091057620977
WOLFE (weak): th(1.4104082335664332E-6)=-689.6091057620977; dx=-0.26262014365045716 evalInputDelta=2.2657763793176855E-5
Armijo: th(1.802188298445998E-6)=-689.6090831043339; dx=-0.2629447679835527 evalInputDelta=0.0
Armijo: th(1.6062982660062156E-6)=-689.6090831043339; dx=-0.26244734824659754 evalInputDelta=0.0
WOLFE (weak): th(1.5083532497863244E-6)=-689.6091057620977; dx=-0.2618289286283149 evalInputDelta=2.2657763793176855E-5
WOLFE (weak): th(1.55732575789627E-6)=-689.6091057620977; dx=-0.2628069070440716 evalInputDelta=2.2657763793176855E-5
Armijo: th(1.5818120119512429E-6)=-689.6090831043339; dx=-0.2627713371774196 evalInputDelta=0.0
WOLFE (weak): th(1.5695688849237563E-6)=-689.6091057620977; dx=-0.2631896748316152 evalInputDelta=2.2657763793176855E-5
WOLFE (weak): th(1.5756904484374996E-6)=-689.6091057620977; dx=-0.26041654115645796 evalInputDelta=2.2657763793176855E-5
Armijo: th(1.5787512301943711E-6)=-689.6090831043339; dx=-0.2621943458437158 evalInputDelta=0.0
WOLFE (weak): th(1.5772208393159355E-6)=-689.6091057620977; dx=-0.2612199238218361 evalInputDelta=2.2657763793176855E-5
WOLFE (weak): th(1.5779860347551533E-6)=-689.6091057620977; dx=-0.26296813944010466 evalInputDelta=2.2657763793176855E-5
Armijo: th(1.578368632474762E-6)=-689.6090831043339; dx=-0.26271677688819434 evalInputDelta=0.0
Armijo: th(1.5781773336149576E-6)=-689.6090831043339; dx=-0.26180446570760824 evalInputDelta=0.0
WOLFE (weak): th(1.5780816841850553E-6)=-689.6091057620977; dx=-0.26313160911673417 evalInputDelta=2.2657763793176855E-5
WOLFE (weak): th(1.5781295089000065E-6)=-689.6091057620977; dx=-0.263359692539627 evalInputDelta=2.2657763793176855E-5
Armijo: th(1.578153421257482E-6)=-689.6090831043339; dx=-0.2617431521159865 evalInputDelta=0.0
WOLFE (weak): th(1.5781414650787441E-6)=-689.6091057620977; dx=-0.261708636413602 evalInputDelta=2.2657763793176855E-5
mu ~= nu (1.5781414650787441E-6): th(1.4104082335664332E-6)=-689.6091057620977
Fitness changed from -689.6090831043339 to -689.6091057620977
Iteration 11 complete. Error: -689.6091057620977 Total: 4.1605; Orientation: 0.0594; Line Search: 3.9821
Adding measurement 48305c40 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.625e+02, gradient 5.184e-01, dot 0.806; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
th(0)=-689.6091057620977;dx=-356.3743879148551
New Minimum: -689.6091057620977 > -689.6103745956843
WOLFE (weak): th(3.4000155975465037E-6)=-689.6103745956843; dx=-353.6804630391629 evalInputDelta=0.0012688335866641864
New Minimum: -689.6103745956843 > -689.6115074808424
WOLFE (weak): th(6.8000311950930075E-6)=-689.6115074808424; dx=-354.4715010839051 evalInputDelta=0.0024017187447498145
New Minimum: -689.6115074808424 > -689.6162202631258
WOLFE (weak): th(2.0400093585279023E-5)=-689.6162202631258; dx=-353.36298307794266 evalInputDelta=0.0071145010281270515
New Minimum: -689.6162202631258 > -689.637608639494
WOLFE (weak): th(8.160037434111609E-5)=-689.637608639494; dx=-354.8235347897225 evalInputDelta=0.028502877396363147
New Minimum: -689.637608639494 > -689.7520614322221
WOLFE (weak): th(4.0800187170558045E-4)=-689.7520614322221; dx=-353.8563105007521 evalInputDelta=0.14295567012447918
New Minimum: -689.7520614322221 > -690.4663958151765
WOLFE (weak): th(0.002448011230233483)=-690.4663958151765; dx=-353.425055461842 evalInputDelta=0.8572900530788274
New Minimum: -690.4663958151765 > -695.5506811153303
WOLFE (weak): th(0.01713607861163438)=-695.5506811153303; dx=-350.81163415044557 evalInputDelta=5.941575353232679
New Minimum: -695.5506811153303 > -733.3204023481142
WOLFE (weak): th(0.13708862889307505)=-733.3204023481142; dx=-311.26934655902477 evalInputDelta=43.71129658601649
New Minimum: -733.3204023481142 > -849.5158915523593
END: th(1.2337976600376754)=-849.5158915523593; dx=-135.18670705841993 evalInputDelta=159.90678579026167
Fitness changed from -689.6091057620977 to -849.5158915523593
Iteration 12 complete. Error: -849.5158915523593 Total: 1.2758; Orientation: 0.0277; Line Search: 1.1405
Adding measurement 5269544 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.622e+05, gradient 7.927e-01, dot -0.332; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -849.5158915523593, -689.6091057620977, -135.91700995000588, -72.80146778010386, -22.743821866728307
Rejected: LBFGS Orientation magnitude: 2.065e+05, gradient 7.927e-01, dot -0.362; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -849.5158915523593, -689.6091057620977, -135.91700995000588, -72.80146778010386
LBFGS Accumulation History: 3 points
Removed measurement 5269544 to history. Total: 4
Removed measurement 48305c40 to history. Total: 3
th(0)=-849.5158915523593;dx=-0.6193451626627207
New Minimum: -849.5158915523593 > -852.1150523843596
WOLFE (weak): th(2.658136479265333)=-852.1150523843596; dx=-0.6024527876210852 evalInputDelta=2.5991608320002797
New Minimum: -852.1150523843596 > -854.7107332308399
WOLFE (weak): th(5.316272958530666)=-854.7107332308399; dx=-0.605696763547696 evalInputDelta=5.194841678480543
New Minimum: -854.7107332308399 > -865.0459525366268
WOLFE (weak): th(15.948818875591998)=-865.0459525366268; dx=-0.5900204766432495 evalInputDelta=15.530060984267493
New Minimum: -865.0459525366268 > -909.1314192678636
WOLFE (weak): th(63.79527550236799)=-909.1314192678636; dx=-0.5291935177633029 evalInputDelta=59.6155277155043
New Minimum: -909.1314192678636 > -1071.3972185888854
END: th(318.97637751184)=-1071.3972185888854; dx=-0.24302187299130568 evalInputDelta=221.88132703652605
Fitness changed from -849.5158915523593 to -1071.3972185888854
Iteration 13 complete. Error: -1071.3972185888854 Total: 0.8861; Orientation: 0.0624; Line Search: 0.7132
Adding measurement bb3b03f to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.575e+03, gradient 5.607e-01, dot 0.532; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
th(0)=-1071.3972185888854;dx=-1357.340194606541
Armijo: th(687.2137730122141)=-686.6443994164665; dx=1.7113869597282658 evalInputDelta=-384.75281917241887
Armijo: th(343.60688650610706)=-693.3771069915706; dx=3.439919934818329 evalInputDelta=-378.0201115973148
Armijo: th(114.53562883536902)=-717.5828131651426; dx=10.091993552191525 evalInputDelta=-353.81440542374276
Armijo: th(28.633907208842256)=-752.5007059797352; dx=39.357813525745534 evalInputDelta=-318.89651260915014
New Minimum: -1071.3972185888854 > -1083.3083586864823
END: th(5.726781441768451)=-1083.3083586864823; dx=163.18136299033017 evalInputDelta=11.911140097596899
Fitness changed from -1071.3972185888854 to -1083.3083586864823
Iteration 14 complete. Error: -1083.3083586864823 Total: 0.8702; Orientation: 0.0298; Line Search: 0.7272
Adding measurement 3a2811c4 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.618e+03, gradient 5.392e-01, dot -0.050; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1083.3083586864823, -1071.3972185888854, -135.91700995000588, -72.80146778010386, -22.743821866728307
Rejected: LBFGS Orientation magnitude: 4.880e+03, gradient 5.392e-01, dot -0.076; [80f38921-2809-4833-b6b8-048a6c9ad5a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1083.3083586864823, -1071.3972185888854, -135.91700995000588, -72.80146778010386
LBFGS Accumulation History: 3 points
Removed measurement 3a2811c4 to history. Total: 4
Removed measurement bb3b03f to history. Total: 3
th(0)=-1083.3083586864823;dx=-0.29050152054303874
New Minimum: -1083.3083586864823 > -1086.442761952971
END: th(12.337976600376757)=-1086.442761952971; dx=-0.15525237265399455 evalInputDelta=3.134403266488789
Fitness changed from -1083.3083586864823 to -1086.442761952971
Iteration 15 complete. Error: -1086.442761952971 Total: 0.4304; Orientation: 0.0646; Line Search: 0.2522
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1086.442761952971 (> -Infinity) after 16.578s (< 5400.000s)
Returns
-1086.442761952971