BasicOptimizer.scala:89 executed in 64.36 seconds (1.269 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 213349687864100
Reset training subject: 213350306723400
Adding measurement 383cd77c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.1418840423081904;dx=-8.754322428183917E-6
New Minimum: -2.1418840423081904 > -2.141895481358792
WOLFE (weak): th(2.154434690031884)=-2.141895481358792; dx=-8.742177918325013E-6 evalInputDelta=1.1439050601413925E-5
New Minimum: -2.141895481358792 > -2.1419070311830866
WOLFE (weak): th(4.308869380063768)=-2.1419070311830866; dx=-8.73653710350412E-6 evalInputDelta=2.2988874896157085E-5
New Minimum: -2.1419070311830866 > -2.141953005488011
WOLFE (weak): th(12.926608140191302)=-2.141953005488011; dx=-8.754762047095902E-6 evalInputDelta=6.89631798205248E-5
New Minimum: -2.141953005488011 > -2.1421597217322166
WOLFE (weak): th(51.70643256076521)=-2.1421597217322166; dx=-8.731001140669534E-6 evalInputDelta=2.756794240261762E-4
New Minimum: -2.1421597217322166 > -2.1432610016901674
WOLFE (weak): th(258.53216280382605)=-2.1432610016901674; dx=-8.733444451782858E-6 evalInputDelta=0.0013769593819770165
New Minimum: -2.1432610016901674 > -2.1501238958508972
WOLFE (weak): th(1551.1929768229563)=-2.1501238958508972; dx=-8.623350357195121E-6 evalInputDelta=0.008239853542706843
New Minimum: -2.1501238958508972 > -2.1980240330880996
WOLFE (weak): th(10858.350837760694)=-2.1980240330880996; dx=-8.270658083495906E-6 evalInputDelta=0.056139990779909166
New Minimum: -2.1980240330880996 > -2.384374765139791
END: th(86866.80670208555)=-2.384374765139791; dx=-4.2127675205694675E-6 evalInputDelta=0.24249072283160045
Fitness changed from -2.1418840423081904 to -2.384374765139791
Iteration 1 complete. Error: -2.384374765139791 Total: 7.5783; Orientation: 0.0165; Line Search: 5.8110
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 38d78a07 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.384374765139791;dx=-9.19550855081275E-6
Armijo: th(187148.86177126726)=-2.0571873365364106; dx=-9.544630332885362E-7 evalInputDelta=-0.32718742860338024
New Minimum: -2.384374765139791 > -2.440887053641481
END: th(93574.43088563363)=-2.440887053641481; dx=-3.2102310924850393E-6 evalInputDelta=0.05651228850169021
Fitness changed from -2.384374765139791 to -2.440887053641481
Iteration 2 complete. Error: -2.440887053641481 Total: 2.4952; Orientation: 0.0165; Line Search: 1.8793
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 42b9b847 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.440887053641481;dx=-8.873963140214623E-6
Armijo: th(201600.00000000003)=-1.9760132503044803; dx=-1.3434236907615265E-6 evalInputDelta=-0.4648738033370008
New Minimum: -2.440887053641481 > -2.47031727042582
END: th(100800.00000000001)=-2.47031727042582; dx=-3.608760800043956E-6 evalInputDelta=0.02943021678433899
Fitness changed from -2.440887053641481 to -2.47031727042582
Iteration 3 complete. Error: -2.47031727042582 Total: 2.4905; Orientation: 0.0164; Line Search: 1.8695
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3340501 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.363e+03, gradient 2.955e-03, dot -0.424; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.47031727042582, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 3340501 to history. Total: 3
th(0)=-2.47031727042582;dx=-8.713973848240063E-6
Armijo: th(217167.01675521393)=-1.8545949878783716; dx=-1.5031398803465236E-6 evalInputDelta=-0.6157222825474484
Armijo: th(108583.50837760697)=-2.44664514165183; dx=-3.8125623037009887E-6 evalInputDelta=-0.023672128773990053
New Minimum: -2.47031727042582 > -2.55630875614358
END: th(36194.502792535655)=-2.55630875614358; dx=-6.881863574627834E-6 evalInputDelta=0.08599148571776016
Fitness changed from -2.47031727042582 to -2.55630875614358
Iteration 4 complete. Error: -2.55630875614358 Total: 3.3599; Orientation: 0.3156; Line Search: 2.4455
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1456519e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.036e+03, gradient 2.727e-03, dot -0.459; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.55630875614358, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 1456519e to history. Total: 3
th(0)=-2.55630875614358;dx=-7.404581394484534E-6
New Minimum: -2.55630875614358 > -2.5964282471171813
END: th(77978.69240469471)=-2.5964282471171813; dx=-4.78862700808755E-6 evalInputDelta=0.040119490973601124
Fitness changed from -2.55630875614358 to -2.5964282471171813
Iteration 5 complete. Error: -2.5964282471171813 Total: 2.3458; Orientation: 0.3091; Line Search: 1.4735
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 28936392 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.970e+03, gradient 2.766e-03, dot -0.432; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.5964282471171813, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 28936392 to history. Total: 3
th(0)=-2.5964282471171813;dx=-7.628436314215625E-6
Armijo: th(168000.00000000006)=-2.351112856936135; dx=-3.26567834621652E-6 evalInputDelta=-0.2453153901810463
New Minimum: -2.5964282471171813 > -2.6246912417675667
END: th(84000.00000000003)=-2.6246912417675667; dx=-4.788248531223356E-6 evalInputDelta=0.028262994650385398
Fitness changed from -2.5964282471171813 to -2.6246912417675667
Iteration 6 complete. Error: -2.6246912417675667 Total: 2.7705; Orientation: 0.2968; Line Search: 1.8506
Adding measurement 66934a44 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.581e+03, gradient 2.786e-03, dot -0.420; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.6246912417675667, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 66934a44 to history. Total: 3
th(0)=-2.6246912417675667;dx=-7.716684566732787E-6
Armijo: th(180972.5139626783)=-2.269096222430023; dx=-3.1607374881298626E-6 evalInputDelta=-0.35559501933754367
Armijo: th(90486.25698133915)=-2.616979531663656; dx=-4.779028835400511E-6 evalInputDelta=-0.007711710103910807
New Minimum: -2.6246912417675667 > -2.6750825647310092
END: th(30162.08566044638)=-2.6750825647310092; dx=-6.648251895624221E-6 evalInputDelta=0.05039132296344251
Fitness changed from -2.6246912417675667 to -2.6750825647310092
Iteration 7 complete. Error: -2.6750825647310092 Total: 3.2521; Orientation: 0.3073; Line Search: 2.3918
Adding measurement 5ee546e4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.843e+03, gradient 2.665e-03, dot -0.438; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.6750825647310092, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 5ee546e4 to history. Total: 3
th(0)=-2.6750825647310092;dx=-7.069609485609745E-6
New Minimum: -2.6750825647310092 > -2.699120734712536
END: th(64982.24367057892)=-2.699120734712536; dx=-5.257179232502373E-6 evalInputDelta=0.024038169981526636
Fitness changed from -2.6750825647310092 to -2.699120734712536
Iteration 8 complete. Error: -2.699120734712536 Total: 2.1280; Orientation: 0.3060; Line Search: 1.2464
Adding measurement 75e32608 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.996e+03, gradient 2.612e-03, dot -0.422; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element
...skipping 1442 bytes...
2920883097; dx=-6.21572320228519E-6 evalInputDelta=0.025321793052613728
END: th(50270.142767410645)=-2.7424546947906725; dx=-5.666723806597359E-6 evalInputDelta=0.023583566960189373
Fitness changed from -2.718871127830483 to -2.744192920883097
Iteration 10 complete. Error: -2.744192920883097 Total: 4.0055; Orientation: 0.2924; Line Search: 3.1430
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7f6cd769 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.148e+04, gradient 2.561e-03, dot -0.410; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.744192920883097, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 7f6cd769 to history. Total: 3
th(0)=-2.744192920883097;dx=-6.545196333060418E-6
Armijo: th(108303.7394509649)=-2.6514065019658366; dx=-4.43435656359006E-6 evalInputDelta=-0.09278641891726025
New Minimum: -2.744192920883097 > -2.751968145132575
END: th(54151.86972548245)=-2.751968145132575; dx=-5.422949484901688E-6 evalInputDelta=0.007775224249478185
Fitness changed from -2.744192920883097 to -2.751968145132575
Iteration 11 complete. Error: -2.751968145132575 Total: 2.9677; Orientation: 0.5077; Line Search: 1.8547
Adding measurement a9fa77 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.363e+04, gradient 2.545e-03, dot -0.408; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.751968145132575, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement a9fa77 to history. Total: 3
th(0)=-2.751968145132575;dx=-6.4615844674239745E-6
Armijo: th(116666.66666666674)=-2.62890644536051; dx=-4.173509829292531E-6 evalInputDelta=-0.12306169977206505
Armijo: th(58333.33333333337)=-2.7480069456488163; dx=-5.0799277936581105E-6 evalInputDelta=-0.003961199483758726
New Minimum: -2.751968145132575 > -2.7645897174944203
WOLFE (weak): th(19444.444444444456)=-2.7645897174944203; dx=-5.902111190404039E-6 evalInputDelta=0.012621572361845246
END: th(38888.88888888891)=-2.763143713349139; dx=-5.476887640753723E-6 evalInputDelta=0.01117556821656418
Fitness changed from -2.751968145132575 to -2.7645897174944203
Iteration 12 complete. Error: -2.7645897174944203 Total: 4.1512; Orientation: 0.3041; Line Search: 3.3093
Adding measurement 1441aa8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.340e+04, gradient 2.442e-03, dot -0.413; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.7645897174944203, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 1441aa8 to history. Total: 3
th(0)=-2.7645897174944203;dx=-5.948488959785121E-6
Armijo: th(83783.57127901775)=-2.718897858777029; dx=-4.586438743265561E-6 evalInputDelta=-0.04569185871739112
New Minimum: -2.7645897174944203 > -2.7680333344871837
END: th(41891.785639508875)=-2.7680333344871837; dx=-5.210708080442874E-6 evalInputDelta=0.0034436169927634097
Fitness changed from -2.7645897174944203 to -2.7680333344871837
Iteration 13 complete. Error: -2.7680333344871837 Total: 2.6640; Orientation: 0.2927; Line Search: 1.7969
Adding measurement 13132e66 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.607e+04, gradient 2.402e-03, dot -0.417; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.7680333344871837, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 13132e66 to history. Total: 3
th(0)=-2.7680333344871837;dx=-5.753541722693E-6
Armijo: th(90253.11620913743)=-2.6970176797357786; dx=-4.399554476876124E-6 evalInputDelta=-0.07101565475140514
Armijo: th(45126.558104568714)=-2.760794087153558; dx=-4.957640112333212E-6 evalInputDelta=-0.007239247333625798
New Minimum: -2.7680333344871837 > -2.772317872940958
WOLFE (weak): th(15042.186034856239)=-2.772317872940958; dx=-5.468379280199957E-6 evalInputDelta=0.004284538453774189
WOLFE (weak): th(30084.372069712477)=-2.7696324011568327; dx=-5.197939527334819E-6 evalInputDelta=0.0015990666696490408
Armijo: th(37605.4650871406)=-2.76596929227402; dx=-5.0636592490449954E-6 evalInputDelta=-0.002064042213163475
Armijo: th(33844.918578426536)=-2.76799996014043; dx=-5.131211901185464E-6 evalInputDelta=-3.337434675376727E-5
END: th(31964.645324069505)=-2.768859289753528; dx=-5.151662653181567E-6 evalInputDelta=8.259552663441028E-4
Fitness changed from -2.7680333344871837 to -2.772317872940958
Iteration 14 complete. Error: -2.772317872940958 Total: 5.8494; Orientation: 0.3043; Line Search: 4.9545
Adding measurement 2df2d95 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.664e+04, gradient 2.369e-03, dot -0.420; [4c03ea91-8640-41ef-8d5c-d585fb2eef74 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.772317872940958, -2.440887053641481, -2.384374765139791, -2.1418840423081904
LBFGS Accumulation History: 3 points
Removed measurement 2df2d95 to history. Total: 3
th(0)=-2.772317872940958;dx=-5.596779016345533E-6
Armijo: th(68865.74074074079)=-2.742416210252512; dx=-4.679884269497182E-6 evalInputDelta=-0.029901662688446073
Armijo: th(34432.870370370394)=-2.7722322302223015; dx=-5.0537392143981594E-6 evalInputDelta=-8.564271865640904E-5
New Minimum: -2.772317872940958 > -2.7755171127500873
WOLFE (weak): th(11477.623456790132)=-2.7755171127500873; dx=-5.371298993680478E-6 evalInputDelta=0.003199239809129395
New Minimum: -2.7755171127500873 > -2.775551664279522
WOLFE (weak): th(22955.246913580264)=-2.775551664279522; dx=-5.203218909747113E-6 evalInputDelta=0.0032337913385642736
WOLFE (weak): th(28694.05864197533)=-2.7743035083475203; dx=-5.120386241547687E-6 evalInputDelta=0.0019856354065623805
WOLFE (weak): th(31563.464506172862)=-2.773362521745983; dx=-5.08408205326781E-6 evalInputDelta=0.0010446488050250835
WOLFE (weak): th(32998.16743827163)=-2.7728169073422384; dx=-5.067684972194859E-6 evalInputDelta=4.990344012805181E-4
WOLFE (weak): th(33715.51890432101)=-2.7725303488573303; dx=-5.065833531685468E-6 evalInputDelta=2.1247591637241214E-4
WOLFE (weak): th(34074.1946373457)=-2.7723830326691123; dx=-5.057246922110864E-6 evalInputDelta=6.515972815446247E-5
Armijo: th(34253.53250385805)=-2.772307729008846; dx=-5.049394904643601E-6 evalInputDelta=-1.0143932112072207E-5
WOLFE (weak): th(34163.863570601876)=-2.7723457716144813; dx=-5.041707343015878E-6 evalInputDelta=2.7898673523374384E-5
WOLFE (weak): th(34208.69803722996)=-2.7723267511930034; dx=-5.057480309035702E-6 evalInputDelta=8.878252045541046E-6
Armijo: th(34231.115270544)=-2.7723179069300947; dx=-5.054779074422997E-6 evalInputDelta=3.3989136838386E-8
WOLFE (weak): th(34219.90665388698)=-2.7723227114374955; dx=-5.064726391659953E-6 evalInputDelta=4.838496537651338E-6
WOLFE (weak): th(34225.51096221549)=-2.772319833746585; dx=-5.051051026431245E-6 evalInputDelta=1.960805627199136E-6
WOLFE (weak): th(34228.31311637975)=-2.7723188727892643; dx=-5.057644892988986E-6 evalInputDelta=9.998483063711205E-7
Armijo: th(34229.71419346188)=-2.77231800998985; dx=-5.056135036581022E-6 evalInputDelta=1.3704889223475902E-7
WOLFE (weak): th(34229.01365492081)=-2.7723189168400326; dx=-5.058223208146231E-6 evalInputDelta=1.0438990747374532E-6
WOLFE (weak): th(34229.36392419135)=-2.77231855899446; dx=-5.057932464636693E-6 evalInputDelta=6.860535020081215E-7
WOLFE (weak): th(34229.53905882662)=-2.77231838251888; dx=-5.0551236364583285E-6 evalInputDelta=5.095779220809504E-7
mu ~= nu (34229.53905882662): th(22955.246913580264)=-2.775551664279522
Fitness changed from -2.772317872940958 to -2.775551664279522
Iteration 15 complete. Error: -2.775551664279522 Total: 15.3542; Orientation: 0.3264; Line Search: 14.3725
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2.775551664279522 (> -Infinity) after 64.357s (< 5400.000s)
Returns
-2.775551664279522