BasicOptimizer.scala:89 executed in 52.65 seconds (1.045 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 215124137508300
Reset training subject: 215124754879400
Adding measurement 50086158 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-110.91321477144868;dx=-0.09677221050218976
New Minimum: -110.91321477144868 > -111.08605330812694
WOLFE (weak): th(2.154434690031884)=-111.08605330812694; dx=-0.09725572714691005 evalInputDelta=0.1728385366782561
New Minimum: -111.08605330812694 > -111.26047456306924
WOLFE (weak): th(4.308869380063768)=-111.26047456306924; dx=-0.09699857049520459 evalInputDelta=0.34725979162055864
New Minimum: -111.26047456306924 > -111.96804654680136
WOLFE (weak): th(12.926608140191302)=-111.96804654680136; dx=-0.09742952051012677 evalInputDelta=1.0548317753526817
New Minimum: -111.96804654680136 > -115.31642465764841
WOLFE (weak): th(51.70643256076521)=-115.31642465764841; dx=-0.09631565150648423 evalInputDelta=4.403209886199733
New Minimum: -115.31642465764841 > -131.0361593074633
END: th(258.53216280382605)=-131.0361593074633; dx=-0.054273429642338764 evalInputDelta=20.122944536014614
Fitness changed from -110.91321477144868 to -131.0361593074633
Iteration 1 complete. Error: -131.0361593074633 Total: 5.3340; Orientation: 0.0156; Line Search: 3.6193
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 15c4709a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-131.0361593074633;dx=-0.08824282690120738
New Minimum: -131.0361593074633 > -155.29900278284055
END: th(556.9906600335335)=-155.29900278284055; dx=-0.02303027208404522 evalInputDelta=24.262843475377252
Fitness changed from -131.0361593074633 to -155.29900278284055
Iteration 2 complete. Error: -155.29900278284055 Total: 1.7810; Orientation: 0.0167; Line Search: 1.1954
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 61aa36eb to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-155.29900278284055;dx=-0.08685285932885067
New Minimum: -155.29900278284055 > -172.9793106001597
END: th(1200.0)=-172.9793106001597; dx=-0.014155534655043545 evalInputDelta=17.680307817319147
Fitness changed from -155.29900278284055 to -172.9793106001597
Iteration 3 complete. Error: -172.9793106001597 Total: 2.1129; Orientation: 0.0161; Line Search: 1.5142
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6e3a3c9d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.303e+02, gradient 2.991e-01, dot -0.307; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -172.9793106001597, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 6e3a3c9d to history. Total: 3
th(0)=-172.9793106001597;dx=-0.08942396045481518
New Minimum: -172.9793106001597 > -198.10202895518236
END: th(2585.3216280382608)=-198.10202895518236; dx=-0.01280817538788712 evalInputDelta=25.12271835502267
Fitness changed from -172.9793106001597 to -198.10202895518236
Iteration 4 complete. Error: -198.10202895518236 Total: 2.0290; Orientation: 0.2882; Line Search: 1.1793
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2b1fb25d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.187e+03, gradient 2.268e-01, dot -0.315; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -198.10202895518236, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 2b1fb25d to history. Total: 3
th(0)=-198.10202895518236;dx=-0.0512429709228287
New Minimum: -198.10202895518236 > -246.41504868731343
END: th(5569.9066003353355)=-246.41504868731343; dx=-0.009215207153452674 evalInputDelta=48.31301973213107
Fitness changed from -198.10202895518236 to -246.41504868731343
Iteration 5 complete. Error: -246.41504868731343 Total: 2.0877; Orientation: 0.2885; Line Search: 1.2130
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3f764f30 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.357e+03, gradient 2.350e-01, dot -0.210; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -246.41504868731343, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 3f764f30 to history. Total: 3
th(0)=-246.41504868731343;dx=-0.05509248724142127
Armijo: th(12000.000000000002)=-237.80717863248137; dx=-0.0033305528435878884 evalInputDelta=-8.607870054832063
New Minimum: -246.41504868731343 > -283.5899941153945
END: th(6000.000000000001)=-283.5899941153945; dx=-0.01040114726208921 evalInputDelta=37.17494542808109
Fitness changed from -246.41504868731343 to -283.5899941153945
Iteration 6 complete. Error: -283.5899941153945 Total: 2.6196; Orientation: 0.2892; Line Search: 1.7692
Adding measurement 6a2f2e90 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.956e+03, gradient 2.256e-01, dot -0.188; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -283.5899941153945, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 6a2f2e90 to history. Total: 3
th(0)=-283.5899941153945;dx=-0.05078021955885201
New Minimum: -283.5899941153945 > -287.39273369580786
END: th(12926.608140191305)=-287.39273369580786; dx=-0.004417626867615666 evalInputDelta=3.802739580413345
Fitness changed from -283.5899941153945 to -287.39273369580786
Iteration 7 complete. Error: -287.39273369580786 Total: 2.0727; Orientation: 0.2849; Line Search: 1.2234
Adding measurement 5df5c0f4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.790e+03, gradient 2.165e-01, dot -0.129; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -287.39273369580786, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 5df5c0f4 to history. Total: 3
th(0)=-287.39273369580786;dx=-0.04680243600354313
Armijo: th(27849.533001676682)=-249.23521108321694; dx=-0.0021791896019676153 evalInputDelta=-38.157522612590924
New Minimum: -287.39273369580786 > -348.0080051780726
END: th(13924.766500838341)=-348.0080051780726; dx=-0.00781773419523872 evalInputDelta=60.615271482264745
Fitness changed from -287.39273369580786 to -348.0080051780726
Iteration 8 complete. Error: -348.0080051780726 Total: 2.9150; Orientation: 0.2891; Line Search: 2.0628
Adding measurement 79f9dbeb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.990e+03, gradient 2.141e-01, dot -0.136; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -348.0080051780726, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 79f9dbeb to history. Total: 3
th(0)=-348.0080051780726;dx=-0.04542117874601228
Armijo: th(30000.00000000001)=-313.1538826712775; dx=-0.002554373965949152 evalInputDelta=-34.854122506795136
New Minimum: -348.0080051780726 > -415.28641353250265
END: th(15000.000000000005)=-415.28641353250265; dx=-0.008516324351862226 evalInputDelta=67.27840835443004
Fitness changed from -348.0080051780726 to -415.28641353250265
Iteration 9 complete. Error: -415.28641353250265 Total: 2.6298; Orientation: 0.2832; Line Search: 1.8077
Adding measurement 317f4001 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.472e+04, gradient 2.165e-01, dot -0.147; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -415.28641353250265, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 317f4001 to history. Total: 3
th(0)=-415.28641353250265;dx=-0.046789424158389784
Armijo: th(32316.52035047827)=-388.21974585856253; dx=-0.0022484641942844942 evalInputDelta=-27.066667673940117
New Minimum: -415.28641353250265 > -518.9513486128664
END: th(16158.260175239135)=-518.9513486128664; dx=-0.00959024859139267 evalInputDelta=103.6649350803637
Fitness changed from -415.28641353250265 to -518.9513486128664
Iteration 10 complete. Error: -518.9513486128664 Total: 2.8998; Orientation: 0.5210; Line Search: 1.8066
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 71698953 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.112e+04, gradient 2.173e-01, dot -0.155; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -518.9513486128664, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 71698953 to history. Total: 3
th(0)=-518.9513486128664;dx=-0.047132696283342185
Armijo: th(34811.916252095856)=-446.86983720533766; dx=-0.002289226840777691 evalInputDelta=-72.0815114075287
New Minimum: -518.9513486128664 > -612.481942473398
END: th(17405.958126047928)=-612.481942473398; dx=-0.009673453086719663 evalInputDelta=93.53059386053167
Fitness changed from -518.9513486128664 to -612.481942473398
Iteration 11 complete. Error: -612.481942473398 Total: 2.7197; Orientation: 0.3014; Line Search: 1.8283
Adding measurement 3dd3936d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.591e+05, gradient 2.188e-01, dot -0.163; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -612.481942473398, -155.29900278284055, -131.0361593074633, -110.91321477144868
LBFGS Accumulation History: 3 points
Removed measurement 3dd3936d to history. Total: 3
th(0)=-612.481942473398;dx=-0.04782277648093353
Armijo: th(37500.000000000015)=-516.281919738788; dx=-0.0022540028370135394 evalInputDelta=-96.20002273461
New Minimum: -612.481942473398 > -706.426760048162
END: th(18750.000000000007)=-706.426760048162; dx=-0.009819932488481497 evalInputDelta=93.94481757476399
Fitness changed from -612.481942473398 to -706.426760048162
Iteration 12 complete. Error: -706.426760048162 Total: 2.9775; Orientation: 0.3088; Line Search: 2.1332
Adding measurement 32b1c265 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.111e+04, gradient 2.117e-01, dot 0.184; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-706.426760048162;dx=-1601.7073131580162
Armijo: th(40395.65043809784)=498.5370034150432; dx=0.017025216681282155 evalInputDelta=-1204.9637634632052
Armijo: th(20197.82521904892)=495.6961217606106; dx=0.03397933498224001 evalInputDelta=-1202.1228818087725
Armijo: th(6732.608406349639)=488.07866150025075; dx=0.10109147218411782 evalInputDelta=-1194.5054215484129
Armijo: th(1683.1521015874098)=476.03615590475226; dx=0.3983886674247679 evalInputDelta=-1182.4629159529143
Armijo: th(336.63042031748193)=442.19228097414407; dx=1.8913496888505879 evalInputDelta=-1148.6190410223062
Armijo: th(56.10507005291365)=341.58779379030636; dx=9.797667580153814 evalInputDelta=-1048.0145538384684
Armijo: th(8.015010007559093)=124.74658280597578; dx=46.47178927006961 evalInputDelta=-831.1733428541378
Armijo: th(1.0018762509448866)=-371.068881876879; dx=60.52398029731911 evalInputDelta=-335.357878171283
New Minimum: -706.426760048162 > -709.3852332294127
END: th(0.11131958343832074)=-709.3852332294127; dx=-635.6199201287368 evalInputDelta=2.9584731812507243
Fitness changed from -706.426760048162 to -709.3852332294127
Iteration 13 complete. Error: -709.3852332294127 Total: 6.9327; Orientation: 0.2878; Line Search: 6.0667
Adding measurement 34d543e3 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.637e+04, gradient 2.056e-01, dot 0.137; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
th(0)=-709.3852332294127;dx=-743.346194644362
Armijo: th(0.23983077223941698)=-650.0350974634229; dx=-173.06622385289504 evalInputDelta=-59.350135765989876
Armijo: th(0.11991538611970849)=-683.7063051182465; dx=-265.524890921801 evalInputDelta=-25.678928111166215
Armijo: th(0.03997179537323616)=-702.356003124701; dx=-339.2424679511523 evalInputDelta=-7.029230104711701
Armijo: th(0.00999294884330904)=-707.8097935570269; dx=-376.76541296034713 evalInputDelta=-1.575439672385869
Armijo: th(0.001998589768661808)=-709.0781648978118; dx=-384.16615131057154 evalInputDelta=-0.30706833160093083
Armijo: th(3.33098294776968E-4)=-709.3336869555592; dx=-388.1052590221347 evalInputDelta=-0.051546273853546154
Armijo: th(4.7585470682424005E-5)=-709.3778849167699; dx=-389.24672209631996 evalInputDelta=-0.007348312642875499
Armijo: th(5.948183835303001E-6)=-709.3843188633292; dx=-387.22999888495735 evalInputDelta=-9.143660835206902E-4
Armijo: th(6.609093150336667E-7)=-709.3852180005287; dx=-390.048829132904 evalInputDelta=-1.5228884080897842E-5
Armijo: th(6.609093150336667E-8)=-709.3852308452269; dx=-388.3506485069946 evalInputDelta=-2.384185791015625E-6
New Minimum: -709.3852332294127 > -709.3853172125602
END: th(6.0082665003060615E-9)=-709.3853172125602; dx=-387.7798268194613 evalInputDelta=8.398314741953072E-5
Fitness changed from -709.3852332294127 to -709.3853172125602
Iteration 14 complete. Error: -709.3853172125602 Total: 8.5823; Orientation: 0.3695; Line Search: 7.6591
Adding measurement 1d565dbf to history. Total: 5
Accepted: LBFGS Orientation magnitude: 3.477e+04, gradient 2.056e-01, dot 0.101; [5af02bbe-4298-4f81-9919-78760bfe2ae8 = 1.000/1.000e+00]
th(0)=-709.3853172125602;dx=-722.4774017174327
Armijo: th(1.2944417775215842E-8)=-709.3853172125602; dx=-240.40231614715307 evalInputDelta=0.0
Armijo: th(6.472208887607921E-9)=-709.3853172125602; dx=-241.20932159681058 evalInputDelta=0.0
Armijo: th(2.1574029625359736E-9)=-709.3852799893592; dx=-241.64624105549058 evalInputDelta=-3.7223200934022316E-5
Armijo: th(5.393507406339934E-10)=-709.3853172125602; dx=-240.1790660234225 evalInputDelta=0.0
New Minimum: -709.3853172125602 > -709.3853219809317
END: th(1.0787014812679868E-10)=-709.3853219809317; dx=-240.9841474387552 evalInputDelta=4.76837158203125E-6
Fitness changed from -709.3853172125602 to -709.3853219809317
Iteration 15 complete. Error: -709.3853219809317 Total: 4.9331; Orientation: 0.4362; Line Search: 3.9489
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -709.3853219809317 (> -Infinity) after 52.649s (< 5400.000s)
Returns
-709.3853219809317