BasicOptimizer.scala:89 executed in 275.12 seconds (3.116 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 7284727173073
Reset training subject: 7287282653453
Adding measurement 3215b77e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.8543418272912944;dx=-4.860181402236144E-4
New Minimum: -1.8543418272912944 > -1.8543857987889474
WOLFE (weak): th(2.154434690031884)=-1.8543857987889474; dx=-4.884704450199737E-4 evalInputDelta=4.39714976530059E-5
New Minimum: -1.8543857987889474 > -1.854433690535079
WOLFE (weak): th(4.308869380063768)=-1.854433690535079; dx=-4.8753350666646904E-4 evalInputDelta=9.186324378451793E-5
New Minimum: -1.854433690535079 > -1.8546024910422916
WOLFE (weak): th(12.926608140191302)=-1.8546024910422916; dx=-4.889472852525638E-4 evalInputDelta=2.606637509972032E-4
New Minimum: -1.8546024910422916 > -1.8553683926395874
WOLFE (weak): th(51.70643256076521)=-1.8553683926395874; dx=-4.879353264048204E-4 evalInputDelta=0.0010265653482930404
New Minimum: -1.8553683926395874 > -1.858851767898541
WOLFE (weak): th(258.53216280382605)=-1.858851767898541; dx=-4.848224762288329E-4 evalInputDelta=0.004509940607246632
New Minimum: -1.858851767898541 > -1.8864775003870449
WOLFE (weak): th(1551.1929768229563)=-1.8864775003870449; dx=-4.503266626325925E-4 evalInputDelta=0.032135673095750494
New Minimum: -1.8864775003870449 > -2.1095622791935162
END: th(10858.350837760694)=-2.1095622791935162; dx=-3.304610676530334E-4 evalInputDelta=0.25522045190222187
Fitness changed from -1.8543418272912944 to -2.1095622791935162
Iteration 1 complete. Error: -2.1095622791935162 Total: 17.1291; Orientation: 0.0174; Line Search: 10.9159
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 321efaa2 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.1095622791935162;dx=-5.223466581233125E-4
New Minimum: -2.1095622791935162 > -3.144833757528018
END: th(23393.607721408407)=-3.144833757528018; dx=-2.2020638066030338E-4 evalInputDelta=1.0352714783345016
Fitness changed from -2.1095622791935162 to -3.144833757528018
Iteration 2 complete. Error: -3.144833757528018 Total: 3.8993; Orientation: 0.0194; Line Search: 2.4259
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 137c7d55 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.144833757528018;dx=-8.639157073193405E-4
New Minimum: -3.144833757528018 > -4.431341921951865
END: th(50400.00000000001)=-4.431341921951865; dx=-9.422021034121698E-5 evalInputDelta=1.286508164423847
Fitness changed from -3.144833757528018 to -4.431341921951865
Iteration 3 complete. Error: -4.431341921951865 Total: 3.8930; Orientation: 0.0175; Line Search: 2.4181
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 60d6cf2b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.598e+03, gradient 2.548e-02, dot -0.157; [bfb95c68-ab56-40db-b521-496d6b2ff37c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.431341921951865, -3.144833757528018, -2.1095622791935162, -1.8543418272912944
LBFGS Accumulation History: 3 points
Removed measurement 60d6cf2b to history. Total: 3
th(0)=-4.431341921951865;dx=-6.368983963308784E-4
New Minimum: -4.431341921951865 > -7.580299301478801
END: th(108583.50837760697)=-7.580299301478801; dx=-3.114361217356227E-5 evalInputDelta=3.1489573795269363
Fitness changed from -4.431341921951865 to -7.580299301478801
Iteration 4 complete. Error: -7.580299301478801 Total: 4.3268; Orientation: 0.4755; Line Search: 2.4620
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 54084be1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.360e+03, gradient 2.749e-02, dot -0.116; [bfb95c68-ab56-40db-b521-496d6b2ff37c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.580299301478801, -3.144833757528018, -2.1095622791935162, -1.8543418272912944
LBFGS Accumulation History: 3 points
Removed measurement 54084be1 to history. Total: 3
th(0)=-7.580299301478801;dx=-7.430764126551376E-4
New Minimum: -7.580299301478801 > -8.431679759797412
END: th(233936.07721408413)=-8.431679759797412; dx=-7.483938964062143E-6 evalInputDelta=0.8513804583186104
Fitness changed from -7.580299301478801 to -8.431679759797412
Iteration 5 complete. Error: -8.431679759797412 Total: 4.2854; Orientation: 0.5034; Line Search: 2.3688
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 68eed4fd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.002e+02, gradient 1.483e-02, dot -0.118; [bfb95c68-ab56-40db-b521-496d6b2ff37c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.431679759797412, -3.144833757528018, -2.1095622791935162, -1.8543418272912944
LBFGS Accumulation History: 3 points
Removed measurement 68eed4fd to history. Total: 3
th(0)=-8.431679759797412;dx=-2.1556122871915537E-4
New Minimum: -8.431679759797412 > -9.558012264571909
END: th(504000.0000000002)=-9.558012264571909; dx=-4.1804804908398435E-6 evalInputDelta=1.1263325047744974
Fitness changed from -8.431679759797412 to -9.558012264571909
Iteration 6 complete. Error: -9.558012264571909 Total: 4.8256; Orientation: 0.4882; Line Search: 2.4649
Adding measurement 721f5c9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.758e+02, gradient 1.096e-02, dot -0.101; [bfb95c68-ab56-40db-b521-496d6b2ff37c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.558012264571909, -3.144833757528018, -2.1095622791935162, -1.8543418272912944
LBFGS Accumulation History: 3 points
Removed measurement 721f5c9 to history. Total: 3
th(0)=-9.558012264571909;dx=-1.1738195710648783E-4
New Minimum: -9.558012264571909 > -10.20135716403858
END: th(1085835.0837760698)=-10.20135716403858; dx=-1.3785822945624364E-6 evalInputDelta=0.6433448994666708
Fitness changed from -9.558012264571909 to -10.20135716403858
Iteration 7 complete. Error: -10.20135716403858 Total: 4.2257; Orientation: 0.4912; Line Search: 2.5507
Adding measurement 7c9e1a23 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.148e+03, gradient 8.494e-03, dot -0.063; [bfb95c68-ab56-40db-b521-496d6b2ff37c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.20135716403858, -3.144833757528018, -2.1095622791935162, -1.8543418272912944
LBFGS Accumulation History: 3 points
Removed measurement 7c9e1a23 to history. Total: 3
th(0)=-10.20135716403858;dx=-7.075839492555079E-5
Armijo: th(2339360.7721408415)=-9.440304775167318; dx=-3.752122481195264E-7 evalInputDelta=-0.7610523888712617
Armijo: th(1169680.3860704207)=-9.909978026703495; dx=-3.1446948748200024E-6 evalInputDelta=-0.2913791373350847
Armijo: th(389893.4620234736)=-9.661940592256856; dx=-2.1329703460329574E-5 evalInputDelta=-0.5394165717817234
New Minimum: -10.20135716403858 > -10.243840297373145
END: th(97473.3655058684)=-10.243840297373145; dx=-5.234584777502096E-5 evalInputDelta=0.04248313333456544
Fitness changed from -10.20135716403858 to -10.243840297373145
Iteration 8 complete. Error: -10.243840297373145 Total: 7.8430; Orientation: 0.4917; Line Search: 6.1865
Adding measurement 3fb81421 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.579e+03, gradient 8.378e-03, dot -0.073; [bfb95c68-ab56-40db-b521-496d6b2ff37c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.243840297373145, -3.144833757528018, -2.1095622791935162, -1.8543418272912944
LBFGS Accumulation History: 3 points
Removed measurement 3fb81421 to history. Total: 3
th(0)=-10.243840297373145;dx=-6.890585125797164E-5
New Minimum: -10.243840297373145 > -10.50001380556198
END: th(210000.0000000001)=-10.50001380556198; dx=-3.66116797397041E-5 evalInputDelta=0.25617350818883544
Fitness changed from -10.243840297373145 to -10.50001380556198
Iteration 9 complete. Error: -10.50001380556198 Total: 4.0383; Orientation: 0.4960; Line Search: 2.3937
Adding measurement 2d9d0eb5 to history. Total: 3
Rejected: LBFGS Orien
...skipping 15081 bytes...
): th(3.753414659588324E-5)=-12.591917903336846; dx=-5.6360133640211265E-5 evalInputDelta=0.0
WOLFE (weak): th(3.793344602775434E-5)=-12.591917903336846; dx=-5.616477962701434E-5 evalInputDelta=0.0
WOLFE (weak): th(3.8133095743689895E-5)=-12.591917903336846; dx=-5.6369055057489514E-5 evalInputDelta=0.0
Armijo: th(3.823292060165767E-5)=-12.591917297440686; dx=-5.610835430350465E-5 evalInputDelta=-6.058961599109125E-7
WOLFE (weak): th(3.8183008172673784E-5)=-12.591917903336846; dx=-5.629345369395475E-5 evalInputDelta=0.0
Armijo: th(3.820796438716573E-5)=-12.591916691544496; dx=-5.58649377981679E-5 evalInputDelta=-1.2117923500198913E-6
WOLFE (weak): th(3.8195486279919756E-5)=-12.591917903336846; dx=-5.606947655745872E-5 evalInputDelta=0.0
WOLFE (weak): th(3.820172533354274E-5)=-12.591917903336846; dx=-5.624770868826148E-5 evalInputDelta=0.0
WOLFE (weak): th(3.820484486035423E-5)=-12.591917903336846; dx=-5.586404376655755E-5 evalInputDelta=0.0
WOLFE (weak): th(3.820640462375998E-5)=-12.591917903336846; dx=-5.5316461296180105E-5 evalInputDelta=0.0
New Minimum: -12.591917903336846 > -12.591918509232977
WOLFE (weak): th(3.820718450546285E-5)=-12.591918509232977; dx=-5.593528187156345E-5 evalInputDelta=6.058961314892031E-7
WOLFE (weak): th(3.820757444631429E-5)=-12.591918509232977; dx=-5.611590717347941E-5 evalInputDelta=6.058961314892031E-7
Armijo: th(3.8207769416740006E-5)=-12.591916691544496; dx=-5.6133158451783174E-5 evalInputDelta=-1.2117923500198913E-6
mu ~= nu (3.820757444631429E-5): th(3.820718450546285E-5)=-12.591918509232977
Fitness changed from -12.591917903336846 to -12.591918509232977
Iteration 16 complete. Error: -12.591918509232977 Total: 27.7140; Orientation: 0.0190; Line Search: 25.3391
Adding measurement 6062bb82 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-12.591918509232977;dx=-5.8371917723052365E-5
Armijo: th(8.231593383463961E-5)=-12.591916085648277; dx=-5.638191327097784E-5 evalInputDelta=-2.4235847000397825E-6
Armijo: th(4.1157966917319804E-5)=-12.591917297440686; dx=-5.641362865452584E-5 evalInputDelta=-1.2117922914001156E-6
Armijo: th(1.3719322305773267E-5)=-12.591917297440686; dx=-5.6832491306348714E-5 evalInputDelta=-1.2117922914001156E-6
Armijo: th(3.429830576443317E-6)=-12.591917297440686; dx=-5.660051051383894E-5 evalInputDelta=-1.2117922914001156E-6
Armijo: th(6.859661152886634E-7)=-12.591916085648277; dx=-5.6614122840772436E-5 evalInputDelta=-2.4235847000397825E-6
Armijo: th(1.143276858814439E-7)=-12.591917297440686; dx=-5.6114006453935624E-5 evalInputDelta=-1.2117922914001156E-6
Armijo: th(1.6332526554491986E-8)=-12.591917297440686; dx=-5.6554259303104936E-5 evalInputDelta=-1.2117922914001156E-6
Armijo: th(2.0415658193114983E-9)=-12.591917903336846; dx=-5.664907998231819E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(2.2684064659016647E-10)=-12.591918509232977; dx=-5.650375001274139E-5 evalInputDelta=0.0
Armijo: th(1.1342032329508324E-9)=-12.591916691544496; dx=-5.587359107195186E-5 evalInputDelta=-1.8176884815090943E-6
Armijo: th(6.805219397704995E-10)=-12.591917903336846; dx=-5.65275981619982E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(4.5368129318033294E-10)=-12.591918509232977; dx=-5.680520402048888E-5 evalInputDelta=0.0
Armijo: th(5.671016164754162E-10)=-12.591917903336846; dx=-5.624812791009916E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(5.103914548278746E-10)=-12.591918509232977; dx=-5.656223831943503E-5 evalInputDelta=0.0
Armijo: th(5.387465356516454E-10)=-12.591917903336846; dx=-5.641641717217141E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.2456899523976E-10)=-12.591917903336846; dx=-5.675324903950916E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(5.174802250338172E-10)=-12.591918509232977; dx=-5.651983094161737E-5 evalInputDelta=0.0
WOLFE (weak): th(5.210246101367886E-10)=-12.591918509232977; dx=-5.664948352296873E-5 evalInputDelta=0.0
Armijo: th(5.227968026882742E-10)=-12.591917903336846; dx=-5.6109493996476233E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.219107064125314E-10)=-12.591917903336846; dx=-5.620579197253455E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.2146765827466E-10)=-12.591917903336846; dx=-5.6699897601302585E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.212461342057243E-10)=-12.591917903336846; dx=-5.6372953182805995E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.211353721712564E-10)=-12.591917903336846; dx=-5.642220876692188E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(5.210799911540225E-10)=-12.591918509232977; dx=-5.655689817694404E-5 evalInputDelta=0.0
Armijo: th(5.211076816626395E-10)=-12.591917903336846; dx=-5.677272272745787E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.21093836408331E-10)=-12.591917903336846; dx=-5.643116190342946E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(5.210869137811768E-10)=-12.591918509232977; dx=-5.619065479302654E-5 evalInputDelta=0.0
Armijo: th(5.210903750947539E-10)=-12.591917903336846; dx=-5.67815193499694E-5 evalInputDelta=-6.058961314892031E-7
mu ~= nu (5.210869137811768E-10): th(0.0)=-12.591918509232977
Fitness changed from -12.591918509232977 to -12.591918509232977
Static Iteration Total: 36.9876; Orientation: 0.0179; Line Search: 35.8199
Iteration 17 failed. Error: -12.591918509232977
Previous Error: 0.0 -> -12.591918509232977
Retrying iteration 17
Reset training subject: 7531236998840
Adding measurement 1626f1 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-12.591918509232977;dx=-5.8445462687092314E-5
Armijo: th(1.1226514521588422E-9)=-12.591916691544496; dx=-5.628381034025052E-5 evalInputDelta=-1.8176884815090943E-6
Armijo: th(5.613257260794211E-10)=-12.591917903336846; dx=-5.706112148418474E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(1.8710857535980705E-10)=-12.591918509232977; dx=-5.7019549886041694E-5 evalInputDelta=0.0
WOLFE (weak): th(3.742171507196141E-10)=-12.591918509232977; dx=-5.6718164606142113E-5 evalInputDelta=0.0
WOLFE (weak): th(4.677714383995176E-10)=-12.591918509232977; dx=-5.664289750425384E-5 evalInputDelta=0.0
WOLFE (weak): th(5.145485822394694E-10)=-12.591918509232977; dx=-5.6545792707451236E-5 evalInputDelta=0.0
Armijo: th(5.379371541594453E-10)=-12.591917903336846; dx=-5.698628330191143E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.262428681994573E-10)=-12.591917903336846; dx=-5.6468123352177975E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(5.203957252194634E-10)=-12.591918509232977; dx=-5.6724821861341265E-5 evalInputDelta=0.0
Armijo: th(5.233192967094604E-10)=-12.591917903336846; dx=-5.684854868787608E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.21857510964462E-10)=-12.591917903336846; dx=-5.643900511712922E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.211266180919626E-10)=-12.591917903336846; dx=-5.721151379318164E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(5.207611716557131E-10)=-12.591918509232977; dx=-5.6823920149040954E-5 evalInputDelta=0.0
WOLFE (weak): th(5.209438948738378E-10)=-12.591918509232977; dx=-5.699727293602515E-5 evalInputDelta=0.0
WOLFE (weak): th(5.210352564829003E-10)=-12.591918509232977; dx=-5.7081975563073146E-5 evalInputDelta=0.0
WOLFE (weak): th(5.210809372874314E-10)=-12.591918509232977; dx=-5.687136914421528E-5 evalInputDelta=0.0
Armijo: th(5.21103777689697E-10)=-12.591917903336846; dx=-5.709831642089509E-5 evalInputDelta=-6.058961314892031E-7
Armijo: th(5.210923574885642E-10)=-12.591917903336846; dx=-5.642951143853936E-5 evalInputDelta=-6.058961314892031E-7
WOLFE (weak): th(5.210866473879978E-10)=-12.591918509232977; dx=-5.7098540780877606E-5 evalInputDelta=0.0
Armijo: th(5.210895024382811E-10)=-12.591917903336846; dx=-5.716250855200781E-5 evalInputDelta=-6.058961314892031E-7
mu ~= nu (5.210866473879978E-10): th(0.0)=-12.591918509232977
Fitness changed from -12.591918509232977 to -12.591918509232977
Static Iteration Total: 28.6098; Orientation: 0.0201; Line Search: 26.3455
Iteration 18 failed. Error: -12.591918509232977
Previous Error: 0.0 -> -12.591918509232977
Optimization terminated 18
Final threshold in iteration 18: -12.591918509232977 (> -Infinity) after 275.120s (< 3600.000s)
Returns
-12.591918509232977