BasicOptimizer.scala:89 executed in 47.58 seconds (0.855 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 217614346165300
Reset training subject: 217614971581000
Adding measurement 3a4b2440 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-45.84697426686745;dx=-0.008695539960850138
New Minimum: -45.84697426686745 > -45.85844779682379
WOLFE (weak): th(2.154434690031884)=-45.85844779682379; dx=-0.008691358210971264 evalInputDelta=0.011473529956340656
New Minimum: -45.85844779682379 > -45.8698977757814
WOLFE (weak): th(4.308869380063768)=-45.8698977757814; dx=-0.008689751093710577 evalInputDelta=0.022923508913955004
New Minimum: -45.8698977757814 > -45.91576495913588
WOLFE (weak): th(12.926608140191302)=-45.91576495913588; dx=-0.008684993769308775 evalInputDelta=0.06879069226842915
New Minimum: -45.91576495913588 > -46.12195572250428
WOLFE (weak): th(51.70643256076521)=-46.12195572250428; dx=-0.00870170375178242 evalInputDelta=0.2749814556368335
New Minimum: -46.12195572250428 > -47.21719902239241
WOLFE (weak): th(258.53216280382605)=-47.21719902239241; dx=-0.008612633826402896 evalInputDelta=1.3702247555249585
New Minimum: -47.21719902239241 > -53.394287267767375
END: th(1551.1929768229563)=-53.394287267767375; dx=-0.007462998264543883 evalInputDelta=7.547313000899926
Fitness changed from -45.84697426686745 to -53.394287267767375
Iteration 1 complete. Error: -53.394287267767375 Total: 6.0159; Orientation: 0.0158; Line Search: 4.2255
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 414862b7 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-53.394287267767375;dx=-0.008590582538607509
New Minimum: -53.394287267767375 > -67.39231036931604
END: th(3341.943960201201)=-67.39231036931604; dx=-0.006084998960873842 evalInputDelta=13.998023101548661
Fitness changed from -53.394287267767375 to -67.39231036931604
Iteration 2 complete. Error: -67.39231036931604 Total: 1.7652; Orientation: 0.0160; Line Search: 1.1967
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6a30143a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-67.39231036931604;dx=-0.008496288774118888
New Minimum: -67.39231036931604 > -87.36389876663422
END: th(7200.000000000001)=-87.36389876663422; dx=-0.00396011586681348 evalInputDelta=19.971588397318186
Fitness changed from -67.39231036931604 to -87.36389876663422
Iteration 3 complete. Error: -87.36389876663422 Total: 2.0343; Orientation: 0.0160; Line Search: 1.4259
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 68d9d003 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.220e+03, gradient 9.087e-02, dot -0.396; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -87.36389876663422, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 68d9d003 to history. Total: 3
th(0)=-87.36389876663422;dx=-0.008256971198938824
New Minimum: -87.36389876663422 > -103.61482019210291
END: th(15511.929768229566)=-103.61482019210291; dx=-0.001982055623856203 evalInputDelta=16.25092142546869
Fitness changed from -87.36389876663422 to -103.61482019210291
Iteration 4 complete. Error: -103.61482019210291 Total: 2.0503; Orientation: 0.2994; Line Search: 1.1896
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1b3be87c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.052e+03, gradient 8.415e-02, dot -0.235; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -103.61482019210291, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 1b3be87c to history. Total: 3
th(0)=-103.61482019210291;dx=-0.00708128758431718
New Minimum: -103.61482019210291 > -107.68230010000221
END: th(33419.43960201202)=-107.68230010000221; dx=-0.0010020948540069349 evalInputDelta=4.067479907899298
Fitness changed from -103.61482019210291 to -107.68230010000221
Iteration 5 complete. Error: -107.68230010000221 Total: 2.1364; Orientation: 0.3843; Line Search: 1.1906
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 39b726f4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.765e+03, gradient 8.740e-02, dot -0.167; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.68230010000221, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 39b726f4 to history. Total: 3
th(0)=-107.68230010000221;dx=-0.007636905146403503
Armijo: th(72000.00000000003)=-79.89008846370919; dx=-2.962953994438135E-4 evalInputDelta=-27.79221163629302
New Minimum: -107.68230010000221 > -125.41514933568374
END: th(36000.000000000015)=-125.41514933568374; dx=-0.0012996421596201838 evalInputDelta=17.732849235681527
Fitness changed from -107.68230010000221 to -125.41514933568374
Iteration 6 complete. Error: -125.41514933568374 Total: 2.7046; Orientation: 0.3007; Line Search: 1.8118
Adding measurement 4a34f164 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.624e+03, gradient 8.186e-02, dot -0.169; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -125.41514933568374, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 4a34f164 to history. Total: 3
th(0)=-125.41514933568374;dx=-0.006701404979952573
Armijo: th(77559.64884114785)=-99.32583734722189; dx=-3.652753724067196E-4 evalInputDelta=-26.089311988461844
New Minimum: -125.41514933568374 > -151.6482757976336
END: th(38779.824420573925)=-151.6482757976336; dx=-0.0015431212615440951 evalInputDelta=26.233126461949865
Fitness changed from -125.41514933568374 to -151.6482757976336
Iteration 7 complete. Error: -151.6482757976336 Total: 2.7149; Orientation: 0.2916; Line Search: 1.8515
Adding measurement 66eaeef0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.725e+03, gradient 8.073e-02, dot -0.168; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -151.6482757976336, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 66eaeef0 to history. Total: 3
th(0)=-151.6482757976336;dx=-0.006515135853049785
Armijo: th(83548.59900503006)=-109.58255357621438; dx=-3.464011243649725E-4 evalInputDelta=-42.06572222141922
New Minimum: -151.6482757976336 > -174.39627832586766
END: th(41774.29950251503)=-174.39627832586766; dx=-0.001481819589190102 evalInputDelta=22.748002528234053
Fitness changed from -151.6482757976336 to -174.39627832586766
Iteration 8 complete. Error: -174.39627832586766 Total: 3.0245; Orientation: 0.2914; Line Search: 2.1581
Adding measurement 793c6090 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.190e+04, gradient 8.049e-02, dot -0.172; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -174.39627832586766, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 793c6090 to history. Total: 3
th(0)=-174.39627832586766;dx=-0.006476492048795666
Armijo: th(90000.00000000004)=-125.02218205210599; dx=-3.258004449071012E-4 evalInputDelta=-49.374096273761666
New Minimum: -174.39627832586766 > -200.4353703902399
END: th(45000.00000000002)=-200.4353703902399; dx=-0.0014916915717089882 evalInputDelta=26.039092064372255
Fitness changed from -174.39627832586766 to -200.4353703902399
Iteration 9 complete. Error: -200.4353703902399 Total: 2.6547; Orientation: 0.2803; Line Search: 1.8341
Adding measurement 320166b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.786e+04, gradient 8.073e-02, dot -0.170; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -200.4353703902399, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 320166b2 to history. Total: 3
th(0)=-200.4353703902399;dx=-0.006514100067086041
Armijo: th(96949.56105143482)=-142.36408821728892; dx=-3.201580542549926E-4 evalInputDelta=-58.07128217295099
New Minimum: -200.4353703902399 > -223.07882357190377
END: th(48474.78052571741)=-223.07882357190377; dx=-0.0014168043237389166 evalInputDelta=22.64345318166386
Fitness changed from -200.4353703902399 to -223.07882357190377
Iteration 10 complete. Error: -223.07882357190377 Total: 2.7343; Orientation: 0.3047; Line Search: 1.8978
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 3f05d69c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.038e+04, gradient 7.818e-02, dot -0.180; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -223.07882357190377, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 3f05d69c to history. Total: 3
th(0)=-223.07882357190377;dx=-0.006110377275024271
Armijo: th(104435.74875628759)=-170.08069372537622; dx=-3.5189412658543414E-4 evalInputDelta=-52.99812984652755
New Minimum: -223.07882357190377 > -252.77258378861904
END: th(52217.874378143795)=-252.77258378861904; dx=-0.001473757134301868 evalInputDelta=29.693760216715276
Fitness changed from -223.07882357190377 to -252.77258378861904
Iteration 11 complete. Error: -252.77258378861904 Total: 2.7110; Orientation: 0.2927; Line Search: 1.8062
Adding measurement 5aff7330 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.484e+04, gradient 7.804e-02, dot -0.182; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -252.77258378861904, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 5aff7330 to history. Total: 3
th(0)=-252.77258378861904;dx=-0.006084434613705487
Armijo: th(112500.00000000007)=-182.27853644428347; dx=-2.4932304713138785E-4 evalInputDelta=-70.49404734433557
New Minimum: -252.77258378861904 > -276.8500290870017
END: th(56250.00000000004)=-276.8500290870017; dx=-0.0013563639272211525 evalInputDelta=24.077445298382656
Fitness changed from -252.77258378861904 to -276.8500290870017
Iteration 12 complete. Error: -276.8500290870017 Total: 3.2111; Orientation: 0.5202; Line Search: 2.1582
Adding measurement 44481268 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.209e+05, gradient 7.769e-02, dot -0.181; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -276.8500290870017, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 44481268 to history. Total: 3
th(0)=-276.8500290870017;dx=-0.006034117291859367
Armijo: th(121186.95131429355)=-199.3484268525293; dx=-2.83990358779681E-4 evalInputDelta=-77.50160223447241
New Minimum: -276.8500290870017 > -300.1137437077735
END: th(60593.47565714677)=-300.1137437077735; dx=-0.0013371049219322922 evalInputDelta=23.263714620771793
Fitness changed from -276.8500290870017 to -300.1137437077735
Iteration 13 complete. Error: -300.1137437077735 Total: 2.6610; Orientation: 0.2895; Line Search: 1.8099
Adding measurement 5ee5920d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.816e+05, gradient 7.785e-02, dot -0.182; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -300.1137437077735, -67.39231036931604, -53.394287267767375, -45.84697426686745
LBFGS Accumulation History: 3 points
Removed measurement 5ee5920d to history. Total: 3
th(0)=-300.1137437077735;dx=-0.006059472830420955
Armijo: th(130544.68594535951)=-203.73716930936803; dx=-1.7802106365182252E-4 evalInputDelta=-96.37657439840547
New Minimum: -300.1137437077735 > -318.8549091552026
END: th(65272.342972679755)=-318.8549091552026; dx=-0.001230980734396709 evalInputDelta=18.741165447429125
Fitness changed from -300.1137437077735 to -318.8549091552026
Iteration 14 complete. Error: -318.8549091552026 Total: 2.6633; Orientation: 0.2981; Line Search: 1.7990
Adding measurement 6f4be919 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.577e+04, gradient 7.803e-02, dot 0.195; [4a9eaa71-f85d-440f-8b46-39424747e768 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-318.8549091552026;dx=-1153.7226673987352
Armijo: th(140625.00000000012)=624.3393995015522; dx=0.0020917821702853027 evalInputDelta=-943.1943086567549
Armijo: th(70312.50000000006)=624.0001932388881; dx=0.004182695747833393 evalInputDelta=-942.8551023940906
Armijo: th(23437.50000000002)=622.7893028857812; dx=0.012553614058253528 evalInputDelta=-941.6442120409838
Armijo: th(5859.375000000005)=618.4529953934917; dx=0.05019862380648711 evalInputDelta=-937.3079045486943
Armijo: th(1171.875000000001)=607.478356558229; dx=0.2487869014625513 evalInputDelta=-926.3332657134317
Armijo: th(195.31250000000014)=590.0793321564838; dx=1.4646856949001554 evalInputDelta=-908.9342413116865
Armijo: th(27.901785714285733)=546.4529408449994; dx=9.61100278396458 evalInputDelta=-865.307850000202
Armijo: th(3.4877232142857166)=356.22620455126514; dx=52.25921674559012 evalInputDelta=-675.0811137064677
Armijo: th(0.38752480158730185)=-122.44312240607331; dx=-5.222651747150202 evalInputDelta=-196.4117867491293
Armijo: th(0.03875248015873019)=-315.2384800698381; dx=-527.65889888524 evalInputDelta=-3.61642908536453
New Minimum: -318.8549091552026 > -319.1739758203381
END: th(0.003522952741702744)=-319.1739758203381; dx=-776.1877945043747 evalInputDelta=0.31906666513549453
Fitness changed from -318.8549091552026 to -319.1739758203381
Iteration 15 complete. Error: -319.1739758203381 Total: 8.4777; Orientation: 0.2934; Line Search: 7.6226
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -319.1739758203381 (> -Infinity) after 47.581s (< 5400.000s)
Returns
-319.1739758203381