BasicOptimizer.scala:89 executed in 67.56 seconds (0.885 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 155656763113700
Reset training subject: 155657928479600
Adding measurement 227d4a06 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-24.08668069234772;dx=-8.507314887460151E-4
New Minimum: -24.08668069234772 > -24.088325480860654
WOLFE (weak): th(2.154434690031884)=-24.088325480860654; dx=-8.358416701893571E-4 evalInputDelta=0.0016447885129338147
New Minimum: -24.088325480860654 > -24.089973948266486
WOLFE (weak): th(4.308869380063768)=-24.089973948266486; dx=-8.348740283400561E-4 evalInputDelta=0.0032932559187663912
New Minimum: -24.089973948266486 > -24.096561383939182
WOLFE (weak): th(12.926608140191302)=-24.096561383939182; dx=-8.361888054295389E-4 evalInputDelta=0.00988069159146221
New Minimum: -24.096561383939182 > -24.126222334068284
WOLFE (weak): th(51.70643256076521)=-24.126222334068284; dx=-8.367663074119706E-4 evalInputDelta=0.03954164172056451
New Minimum: -24.126222334068284 > -24.284129411330465
WOLFE (weak): th(258.53216280382605)=-24.284129411330465; dx=-8.218822754402454E-4 evalInputDelta=0.19744871898274496
New Minimum: -24.284129411330465 > -25.270365918277843
WOLFE (weak): th(1551.1929768229563)=-25.270365918277843; dx=-8.344613899671055E-4 evalInputDelta=1.1836852259301232
New Minimum: -25.270365918277843 > -31.92915425469899
END: th(10858.350837760694)=-31.92915425469899; dx=-7.2056447615291E-4 evalInputDelta=7.84247356235127
Fitness changed from -24.08668069234772 to -31.92915425469899
Iteration 1 complete. Error: -31.92915425469899 Total: 12.6391; Orientation: 0.0204; Line Search: 9.3655
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3b796b25 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-31.92915425469899;dx=-7.952204215265215E-4
New Minimum: -31.92915425469899 > -45.96957459758301
END: th(23393.607721408407)=-45.96957459758301; dx=-5.590808644674774E-4 evalInputDelta=14.040420342884023
Fitness changed from -31.92915425469899 to -45.96957459758301
Iteration 2 complete. Error: -45.96957459758301 Total: 2.7991; Orientation: 0.0196; Line Search: 1.8041
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 77134f31 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-45.96957459758301;dx=-7.338816218926519E-4
New Minimum: -45.96957459758301 > -68.40060108453332
END: th(50400.00000000001)=-68.40060108453332; dx=-3.903115818119974E-4 evalInputDelta=22.43102648695031
Fitness changed from -45.96957459758301 to -68.40060108453332
Iteration 3 complete. Error: -68.40060108453332 Total: 2.7385; Orientation: 0.0157; Line Search: 1.7620
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6a1bcfb7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.654e+04, gradient 2.592e-02, dot -0.498; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -68.40060108453332, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 6a1bcfb7 to history. Total: 3
th(0)=-68.40060108453332;dx=-6.616958769118749E-4
New Minimum: -68.40060108453332 > -101.43632415025593
END: th(108583.50837760697)=-101.43632415025593; dx=-2.3623993237703245E-4 evalInputDelta=33.03572306572261
Fitness changed from -68.40060108453332 to -101.43632415025593
Iteration 4 complete. Error: -101.43632415025593 Total: 3.2566; Orientation: 0.3159; Line Search: 1.9600
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3702f5ae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.735e+04, gradient 2.943e-02, dot -0.284; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -101.43632415025593, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 3702f5ae to history. Total: 3
th(0)=-101.43632415025593;dx=-8.57917690457869E-4
New Minimum: -101.43632415025593 > -125.05519310257515
END: th(233936.07721408413)=-125.05519310257515; dx=-8.125514827772917E-5 evalInputDelta=23.618868952319218
Fitness changed from -101.43632415025593 to -125.05519310257515
Iteration 5 complete. Error: -125.05519310257515 Total: 2.9700; Orientation: 0.2870; Line Search: 1.7264
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 11f07958 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.821e+04, gradient 3.379e-02, dot -0.190; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -125.05519310257515, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 11f07958 to history. Total: 3
th(0)=-125.05519310257515;dx=-0.00113301261297033
Armijo: th(504000.0000000002)=-118.75098826387718; dx=2.453787140344662E-5 evalInputDelta=-6.30420483869797
New Minimum: -125.05519310257515 > -161.30489624987723
END: th(252000.0000000001)=-161.30489624987723; dx=-2.7526875668600294E-5 evalInputDelta=36.249703147302085
Fitness changed from -125.05519310257515 to -161.30489624987723
Iteration 6 complete. Error: -161.30489624987723 Total: 3.8149; Orientation: 0.2812; Line Search: 2.5954
Adding measurement 3997410e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.646e+04, gradient 3.173e-02, dot -0.150; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -161.30489624987723, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 3997410e to history. Total: 3
th(0)=-161.30489624987723;dx=-0.0010049489308482677
Armijo: th(542917.5418880349)=-148.08900977179113; dx=6.0300304146621844E-5 evalInputDelta=-13.215886478086105
New Minimum: -161.30489624987723 > -205.31634016741657
WOLF (strong): th(271458.77094401745)=-205.31634016741657; dx=1.0615146258877121E-5 evalInputDelta=44.01144391753934
New Minimum: -205.31634016741657 > -211.96413039657267
END: th(90486.25698133915)=-211.96413039657267; dx=-3.815103277519665E-4 evalInputDelta=50.65923414669544
Fitness changed from -161.30489624987723 to -211.96413039657267
Iteration 7 complete. Error: -211.96413039657267 Total: 4.8218; Orientation: 0.3034; Line Search: 3.6366
Adding measurement 3ada2da to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.811e+04, gradient 2.509e-02, dot -0.244; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -211.96413039657267, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 3ada2da to history. Total: 3
th(0)=-211.96413039657267;dx=-6.276904185589425E-4
New Minimum: -211.96413039657267 > -253.98946808883437
END: th(194946.7310117368)=-253.98946808883437; dx=-2.032087156957374E-4 evalInputDelta=42.0253376922617
Fitness changed from -211.96413039657267 to -253.98946808883437
Iteration 8 complete. Error: -253.98946808883437 Total: 2.9995; Orientation: 0.3171; Line Search: 1.8244
Adding measurement 591117cb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.241e+04, gradient 2.718e-02, dot -0.260; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -253.98946808883437, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 591117cb to history. Total: 3
th(0)=-253.98946808883437;dx=-7.325043656703266E-4
New Minimum: -253.98946808883437 > -256.8887998191637
END: th(420000.0000000002)=-256.8887998191637; dx=-1.1042218575600282E-5 evalInputDelta=2.8993317303293225
Fitness changed from -253.98946808883437 to -256.8887998191637
Iteration 9 complete. Error: -256.8887998191637 Total: 2.8675; Orientation: 0.3046; Line Search: 1.7633
Adding measurement 7b6b9afc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.531e+04, gradient 3.406e-02, dot -0.167; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -256.8887998191637, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 7b6b9afc to history. Total: 3
th(0)=-256.8887998191637;dx=-0.001154909188373063
Armijo: th(904862.5698133915)=-154.60002385538175; dx=8.687726845697344E-5 evalInputDelta=-102.28877596378194
New Minimum: -256.8887998191637 > -265.23603143202126
WOLF (strong): th(452431.2849066958)=-265.23603143202126; dx=1.0212448631188818E-4 evalInputDelta=8.34723161285757
New Minimum: -265.23603143202126 > -326.7539812928146
END: th(150810.4283022319)=-326.7539812928146; dx=-1.814820117571533E-4 evalInputDelta=69.86518147365092
Fitness changed from -256.8887998191637 to -326.7539812928146
Iteration 10 complete. Error: -326.7539812928146 Total: 4.8587; Orientation: 0.3179; Line Search: 3.6671
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6a845921 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.574e+05, gradient 2.656e-02, dot -0.248; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -326.7539812928146, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 6a845921 to history. Total: 3
th(0)=-326.7539812928146;dx=-6.996706988781215E-4
New Minimum: -326.7539812928146 > -343.5859990730816
END: th(324911.21835289465)=-343.5859990730816; dx=-2.4818206900128715E-8 evalInputDelta=16.832017780266995
Fitness changed from -326.7539812928146 to -343.5859990730816
Iteration 11 complete. Error: -343.5859990730816 Total: 2.9702; Orientation: 0.2928; Line Search: 1.7631
Adding measurement 50c6d55a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.284e+04, gradient 3.394e-02, dot 0.271; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-343.5859990730816;dx=-484.2458733413356
Armijo: th(700000.0000000003)=-118.00259910022675; dx=1.3497497824079455E-4 evalInputDelta=-225.58339997285486
Armijo: th(350000.0000000002)=-118.00066727041947; dx=2.6981739330462655E-4 evalInputDelta=-225.58533180266215
Armijo: th(116666.66666666673)=-118.02049921891555; dx=8.090032904297269E-4 evalInputDelta=-225.56549985416606
Armijo: th(29166.666666666682)=-118.06143541193495; dx=0.003232569247433649 evalInputDelta=-225.52456366114666
Armijo: th(5833.333333333337)=-118.01751275251166; dx=0.01618803110289881 evalInputDelta=-225.56848632056995
Armijo: th(972.2222222222227)=-118.20732752794699; dx=0.09691878424836961 evalInputDelta=-225.37867154513464
Armijo: th(138.88888888888897)=-120.49757701514233; dx=0.6788890050276817 evalInputDelta=-223.08842205793928
Armijo: th(17.36111111111112)=-134.8538601029658; dx=5.509880982450939 evalInputDelta=-208.7321389701158
Armijo: th(1.9290123456790136)=-216.1592580098483; dx=43.257961102285606 evalInputDelta=-127.42674106323332
New Minimum: -343.5859990730816 > -356.4624539850666
END: th(0.19290123456790137)=-356.4624539850666; dx=-84.28728397617192 evalInputDelta=12.876454911984979
Fitness changed from -343.5859990730816 to -356.4624539850666
Iteration 12 complete. Error: -356.4624539850666 Total: 11.3100; Orientation: 0.3047; Line Search: 10.2192
Adding measurement 36f72e0d to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.142e+05, gradient 3.093e-02, dot -0.106; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -356.4624539850666, -343.5859990730816, -45.96957459758301, -31.92915425469899, -24.08668069234772
Rejected: LBFGS Orientation magnitude: 3.628e+05, gradient 3.093e-02, dot -0.122; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -356.4624539850666, -343.5859990730816, -45.96957459758301, -31.92915425469899
LBFGS Accumulation History: 3 points
Removed measurement 36f72e0d to history. Total: 4
Removed measurement 50c6d55a to history. Total: 3
th(0)=-356.4624539850666;dx=-9.535389080657181E-4
New Minimum: -356.4624539850666 > -356.46278193555133
END: th(0.41559311150306427)=-356.46278193555133; dx=-7.43293502174837E-4 evalInputDelta=3.279504847455428E-4
Fitness changed from -356.4624539850666 to -356.46278193555133
Iteration 13 complete. Error: -356.46278193555133 Total: 3.4684; Orientation: 0.6380; Line Search: 1.9802
Adding measurement 72192524 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.004e+04, gradient 3.093e-02, dot -0.124; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -356.46278193555133, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 72192524 to history. Total: 3
th(0)=-356.46278193555133;dx=-9.548401484650714E-4
New Minimum: -356.46278193555133 > -356.46343783561565
END: th(0.8953682163604904)=-356.46343783561565; dx=-7.439882379393446E-4 evalInputDelta=6.559000643164836E-4
Fitness changed from -356.46278193555133 to -356.46343783561565
Iteration 14 complete. Error: -356.46343783561565 Total: 3.0460; Orientation: 0.2700; Line Search: 1.9718
Adding measurement 6653ce73 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.992e+04, gradient 3.093e-02, dot -0.124; [6485d287-8db1-4213-bbe1-64afdf3ff971 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -356.46343783561565, -45.96957459758301, -31.92915425469899, -24.08668069234772
LBFGS Accumulation History: 3 points
Removed measurement 6653ce73 to history. Total: 3
th(0)=-356.46343783561565;dx=-9.509433518912564E-4
New Minimum: -356.46343783561565 > -356.4648808115091
END: th(1.9290123456790138)=-356.4648808115091; dx=-7.422220316901499E-4 evalInputDelta=0.0014429758934397796
Fitness changed from -356.46343783561565 to -356.4648808115091
Iteration 15 complete. Error: -356.4648808115091 Total: 2.8979; Orientation: 0.2885; Line Search: 1.7132
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -356.4648808115091 (> -Infinity) after 67.558s (< 5400.000s)
Returns
-356.4648808115091