BasicOptimizer.scala:89 executed in 59.61 seconds (1.396 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 211277494930600
Reset training subject: 211278290159400
Adding measurement 4aeb3cb4 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=6.716665660324837;dx=-0.0028446904332683227
New Minimum: 6.716665660324837 > 6.7122798470782525
WOLFE (weak): th(2.154434690031884)=6.7122798470782525; dx=-0.002859658842599897 evalInputDelta=0.004385813246584824
New Minimum: 6.7122798470782525 > 6.707894344561994
WOLFE (weak): th(4.308869380063768)=6.707894344561994; dx=-0.0028639607186065254 evalInputDelta=0.008771315762843734
New Minimum: 6.707894344561994 > 6.690335293805577
WOLFE (weak): th(12.926608140191302)=6.690335293805577; dx=-0.002847364361703085 evalInputDelta=0.026330366519260018
New Minimum: 6.690335293805577 > 6.610963230979324
WOLFE (weak): th(51.70643256076521)=6.610963230979324; dx=-0.0028632299885454206 evalInputDelta=0.10570242934551377
New Minimum: 6.610963230979324 > 6.183235478498502
WOLFE (weak): th(258.53216280382605)=6.183235478498502; dx=-0.0029165031749573377 evalInputDelta=0.5334301818263354
New Minimum: 6.183235478498502 > 3.5078787686746065
WOLFE (weak): th(1551.1929768229563)=3.5078787686746065; dx=-0.002885197701777171 evalInputDelta=3.208786891650231
New Minimum: 3.5078787686746065 > -6.7229525631646965
END: th(10858.350837760694)=-6.7229525631646965; dx=-0.0014120550090570066 evalInputDelta=13.439618223489534
Fitness changed from 6.716665660324837 to -6.7229525631646965
Iteration 1 complete. Error: -6.7229525631646965 Total: 8.3954; Orientation: 0.0157; Line Search: 6.1580
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 113543ce to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-6.7229525631646965;dx=-0.0029493486799350554
New Minimum: -6.7229525631646965 > -23.62234329327167
END: th(23393.607721408407)=-23.62234329327167; dx=-8.020139742871174E-4 evalInputDelta=16.899390730106973
Fitness changed from -6.7229525631646965 to -23.62234329327167
Iteration 2 complete. Error: -23.62234329327167 Total: 2.2021; Orientation: 0.0158; Line Search: 1.4852
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 65d283bc to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-23.62234329327167;dx=-0.003466521769066989
New Minimum: -23.62234329327167 > -28.09258988530403
END: th(50400.00000000001)=-28.09258988530403; dx=-2.343918440550287E-4 evalInputDelta=4.47024659203236
Fitness changed from -23.62234329327167 to -28.09258988530403
Iteration 3 complete. Error: -28.09258988530403 Total: 2.2902; Orientation: 0.0158; Line Search: 1.5281
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4c77a6f2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.423e+03, gradient 6.149e-02, dot -0.194; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.09258988530403, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 4c77a6f2 to history. Total: 3
th(0)=-28.09258988530403;dx=-0.003757763715568953
Armijo: th(108583.50837760697)=-16.58098178169464; dx=-4.7611729389888394E-5 evalInputDelta=-11.51160810360939
New Minimum: -28.09258988530403 > -41.02858801356733
END: th(54291.75418880348)=-41.02858801356733; dx=-3.1344521077445635E-4 evalInputDelta=12.9359981282633
Fitness changed from -28.09258988530403 to -41.02858801356733
Iteration 4 complete. Error: -41.02858801356733 Total: 3.5649; Orientation: 0.4895; Line Search: 2.3257
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3ac8032e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.106e+03, gradient 5.910e-02, dot -0.182; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.02858801356733, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 3ac8032e to history. Total: 3
th(0)=-41.02858801356733;dx=-0.0034833540126080007
Armijo: th(116968.03860704206)=-32.632483793339524; dx=-8.32826588118725E-5 evalInputDelta=-8.396104220227805
New Minimum: -41.02858801356733 > -63.02291391045608
END: th(58484.01930352103)=-63.02291391045608; dx=-4.7403380765900175E-4 evalInputDelta=21.994325896888753
Fitness changed from -41.02858801356733 to -63.02291391045608
Iteration 5 complete. Error: -63.02291391045608 Total: 3.3161; Orientation: 0.2971; Line Search: 2.2467
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 23d3d8b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.841e+03, gradient 5.833e-02, dot -0.177; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.02291391045608, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 23d3d8b to history. Total: 3
th(0)=-63.02291391045608;dx=-0.0033860404185229857
Armijo: th(126000.00000000004)=-55.65247207530034; dx=-1.355120152853305E-4 evalInputDelta=-7.370441835155745
New Minimum: -63.02291391045608 > -89.72958228134483
END: th(63000.00000000002)=-89.72958228134483; dx=-5.098231459913219E-4 evalInputDelta=26.70666837088875
Fitness changed from -63.02291391045608 to -89.72958228134483
Iteration 6 complete. Error: -89.72958228134483 Total: 3.5813; Orientation: 0.3084; Line Search: 2.5127
Adding measurement 17484af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.582e+03, gradient 6.090e-02, dot -0.172; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -89.72958228134483, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 17484af to history. Total: 3
th(0)=-89.72958228134483;dx=-0.003703518432352187
Armijo: th(135729.38547200873)=-72.26623111436787; dx=-7.830327018480912E-5 evalInputDelta=-17.463351166976963
New Minimum: -89.72958228134483 > -117.47450732024271
END: th(67864.69273600436)=-117.47450732024271; dx=-4.8712904382077485E-4 evalInputDelta=27.744925038897875
Fitness changed from -89.72958228134483 to -117.47450732024271
Iteration 7 complete. Error: -117.47450732024271 Total: 3.2699; Orientation: 0.2802; Line Search: 2.3102
Adding measurement 443308d3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.243e+04, gradient 6.440e-02, dot -0.162; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -117.47450732024271, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 443308d3 to history. Total: 3
th(0)=-117.47450732024271;dx=-0.004140747679687297
Armijo: th(146210.0482588026)=-92.10293815171273; dx=-1.861644537613854E-5 evalInputDelta=-25.371569168529973
New Minimum: -117.47450732024271 > -146.31847227366504
END: th(73105.0241294013)=-146.31847227366504; dx=-4.728709712488692E-4 evalInputDelta=28.84396495342233
Fitness changed from -117.47450732024271 to -146.31847227366504
Iteration 8 complete. Error: -146.31847227366504 Total: 3.5436; Orientation: 0.3071; Line Search: 2.5363
Adding measurement 540921e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.013e+04, gradient 6.541e-02, dot -0.159; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -146.31847227366504, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 540921e3 to history. Total: 3
th(0)=-146.31847227366504;dx=-0.004269437769465225
Armijo: th(157500.00000000006)=-112.61415029868945; dx=-1.914772198077561E-5 evalInputDelta=-33.704321974975585
New Minimum: -146.31847227366504 > -178.78467920772792
END: th(78750.00000000003)=-178.78467920772792; dx=-5.013174147725457E-4 evalInputDelta=32.46620693406288
Fitness changed from -146.31847227366504 to -178.78467920772792
Iteration 9 complete. Error: -178.78467920772792 Total: 3.2432; Orientation: 0.2778; Line Search: 2.2478
Adding measurement dd181c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.708e+04, gradient 6.672e-02, dot -0.163; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -178.78467920772792, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement dd181c4 to history. Total: 3
th(0)=-178.78467920772792;dx=-0.004446930802394614
Armijo: th(169661.7318400109)=-138.04719375387492; dx=6.85955976801781E-6 evalInputDelta=-40.737485453853
New Minimum: -178.78467920772792 > -212.98779659203322
END: th(84830.86592000545)=-212.98779659203322; dx=-4.730950311508123E-4 evalInputDelta=34.2031173843053
Fitness changed from -178.78467920772792 to -212.98779659203322
Iteration 10 complete. Error: -212.98779659203322 Total: 3.1768; Orientation: 0.2904; Line Search: 2.2129
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5912ed64 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.029e+04, gradient 6.875e-02, dot -0.157; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -212.98779659203322, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 5912ed64 to history. Total: 3
th(0)=-212.98779659203322;dx=-0.004719394870721434
Armijo: th(182762.56032350325)=-161.5725040957257; dx=2.55401110771465E-5 evalInputDelta=-51.41529249630753
New Minimum: -212.98779659203322 > -246.94697086903295
END: th(91381.28016175162)=-246.94697086903295; dx=-4.992504496139083E-4 evalInputDelta=33.95917427699973
Fitness changed from -212.98779659203322 to -246.94697086903295
Iteration 11 complete. Error: -246.94697086903295 Total: 3.5023; Orientation: 0.2855; Line Search: 2.2917
Adding measurement 55407b41 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.065e+05, gradient 6.858e-02, dot -0.162; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -246.94697086903295, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 55407b41 to history. Total: 3
th(0)=-246.94697086903295;dx=-0.004698062405888337
Armijo: th(196875.0000000001)=-177.19471360951886; dx=1.2017538308812636E-4 evalInputDelta=-69.7522572595141
New Minimum: -246.94697086903295 > -280.52994402681196
END: th(98437.50000000004)=-280.52994402681196; dx=-4.5396207645807113E-4 evalInputDelta=33.58297315777901
Fitness changed from -246.94697086903295 to -280.52994402681196
Iteration 12 complete. Error: -280.52994402681196 Total: 3.1962; Orientation: 0.2876; Line Search: 2.2243
Adding measurement 35192cfb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.812e+07, gradient 7.066e-02, dot -0.151; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -280.52994402681196, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 35192cfb to history. Total: 3
th(0)=-280.52994402681196;dx=-0.004987226453786209
Armijo: th(212077.16480001365)=-196.1921556999908; dx=1.625129657345033E-4 evalInputDelta=-84.33778832682117
New Minimum: -280.52994402681196 > -312.0942579366943
END: th(106038.58240000682)=-312.0942579366943; dx=-3.9393757609978855E-4 evalInputDelta=31.564313909882344
Fitness changed from -280.52994402681196 to -312.0942579366943
Iteration 13 complete. Error: -312.0942579366943 Total: 3.4135; Orientation: 0.2731; Line Search: 2.2205
Adding measurement 4874471d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.612e+05, gradient 7.181e-02, dot -0.126; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -312.0942579366943, -23.62234329327167, -6.7229525631646965, 6.716665660324837
LBFGS Accumulation History: 3 points
Removed measurement 4874471d to history. Total: 3
th(0)=-312.0942579366943;dx=-0.005147251635721843
Armijo: th(228453.20040437908)=-206.31165536037787; dx=1.7582794797279549E-4 evalInputDelta=-105.78260257631644
New Minimum: -312.0942579366943 > -334.8367680055895
END: th(114226.60020218954)=-334.8367680055895; dx=-3.5378703033250993E-4 evalInputDelta=22.74251006889517
Fitness changed from -312.0942579366943 to -334.8367680055895
Iteration 14 complete. Error: -334.8367680055895 Total: 3.2621; Orientation: 0.3121; Line Search: 2.2971
Adding measurement 313769a5 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.324e+04, gradient 7.333e-02, dot 0.195; [4fbe6e26-469f-43f2-a565-4357223666a4 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-334.8367680055895;dx=-474.5796942423734
Armijo: th(246093.75000000015)=117.82651371704685; dx=0.0011438571099824913 evalInputDelta=-452.6632817226363
Armijo: th(123046.87500000007)=117.82673675102009; dx=0.002292772117203337 evalInputDelta=-452.66350475660954
Armijo: th(41015.62500000002)=117.8520139294387; dx=0.0068728421245263065 evalInputDelta=-452.68878193502815
Armijo: th(10253.906250000005)=117.77244054839178; dx=0.02747086151597952 evalInputDelta=-452.60920855398126
Armijo: th(2050.781250000001)=117.40765097625362; dx=0.1374282255419354 evalInputDelta=-452.24441898184307
Armijo: th(341.79687500000017)=116.02855970733114; dx=0.8215909026569468 evalInputDelta=-450.8653277129206
Armijo: th(48.82812500000002)=105.32634693435668; dx=5.672342784810898 evalInputDelta=-440.1631149399461
Armijo: th(6.103515625000003)=-10.766976241228292; dx=33.018393716050994 evalInputDelta=-324.0697917643612
Armijo: th(0.6781684027777781)=-321.90745371478596; dx=-49.77517330531549 evalInputDelta=-12.929314290803518
New Minimum: -334.8367680055895 > -342.8318318283698
END: th(0.06781684027777782)=-342.8318318283698; dx=-250.78443524650726 evalInputDelta=7.99506382278031
Fitness changed from -334.8367680055895 to -342.8318318283698
Iteration 15 complete. Error: -342.8318318283698 Total: 9.6316; Orientation: 0.2702; Line Search: 8.4501
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -342.8318318283698 (> -Infinity) after 59.613s (< 5400.000s)
Returns
-342.8318318283698