BasicOptimizer.scala:89 executed in 9.93 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 141615313617900
Reset training subject: 141615464459900
Adding measurement 22f8870d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.7054832604961508;dx=-1.2518895092884196E-5
New Minimum: -0.7054832604961508 > -0.7055241696405371
WOLFE (weak): th(2.154434690031884)=-0.7055241696405371; dx=-1.2476747896714036E-5 evalInputDelta=4.090914438625859E-5
New Minimum: -0.7055241696405371 > -0.7055651397343219
WOLFE (weak): th(4.308869380063768)=-0.7055651397343219; dx=-1.2904738189534719E-5 evalInputDelta=8.187923817104537E-5
New Minimum: -0.7055651397343219 > -0.7057294078178561
WOLFE (weak): th(12.926608140191302)=-0.7057294078178561; dx=-1.2630665420707153E-5 evalInputDelta=2.461473217052479E-4
New Minimum: -0.7057294078178561 > -0.7064653431296662
WOLFE (weak): th(51.70643256076521)=-0.7064653431296662; dx=-1.2119323101036337E-5 evalInputDelta=9.82082633515402E-4
New Minimum: -0.7064653431296662 > -0.7104145608065984
WOLFE (weak): th(258.53216280382605)=-0.7104145608065984; dx=-1.2534435716099748E-5 evalInputDelta=0.004931300310447528
New Minimum: -0.7104145608065984 > -0.7349729909154705
WOLFE (weak): th(1551.1929768229563)=-0.7349729909154705; dx=-1.2371179711902151E-5 evalInputDelta=0.029489730419319682
New Minimum: -0.7349729909154705 > -0.9108166762576755
WOLFE (weak): th(10858.350837760694)=-0.9108166762576755; dx=-1.2561985094869147E-5 evalInputDelta=0.20533341576152464
New Minimum: -0.9108166762576755 > -1.4087567726922208
END: th(86866.80670208555)=-1.4087567726922208; dx=-2.890811000349887E-6 evalInputDelta=0.7032735121960699
Fitness changed from -0.7054832604961508 to -1.4087567726922208
Iteration 1 complete. Error: -1.4087567726922208 Total: 1.4180; Orientation: 0.0017; Line Search: 1.0443
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 28c46fc2 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.4087567726922208;dx=-2.3853731858887454E-5
New Minimum: -1.4087567726922208 > -2.261035636538084
END: th(187148.86177126726)=-2.261035636538084; dx=-2.9466876351444253E-6 evalInputDelta=0.8522788638458634
Fitness changed from -1.4087567726922208 to -2.261035636538084
Iteration 2 complete. Error: -2.261035636538084 Total: 0.3732; Orientation: 0.0016; Line Search: 0.2396
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 406f1aac to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.261035636538084;dx=-1.7676945846241995E-5
New Minimum: -2.261035636538084 > -2.930607663645638
END: th(403200.00000000006)=-2.930607663645638; dx=-1.0655463788619186E-6 evalInputDelta=0.669572027107554
Fitness changed from -2.261035636538084 to -2.930607663645638
Iteration 3 complete. Error: -2.930607663645638 Total: 0.3780; Orientation: 0.0016; Line Search: 0.2490
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 719b3d5f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.598e+03, gradient 4.386e-03, dot -0.202; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.930607663645638, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
LBFGS Accumulation History: 3 points
Removed measurement 719b3d5f to history. Total: 3
th(0)=-2.930607663645638;dx=-1.6816697475158557E-5
New Minimum: -2.930607663645638 > -3.2188491042496756
WOLF (strong): th(868668.0670208557)=-3.2188491042496756; dx=1.1966971881551649E-8 evalInputDelta=0.28824144060403745
New Minimum: -3.2188491042496756 > -3.9380305983808364
END: th(434334.03351042786)=-3.9380305983808364; dx=-1.4183055337055116E-6 evalInputDelta=1.0074229347351982
Fitness changed from -2.930607663645638 to -3.9380305983808364
Iteration 4 complete. Error: -3.9380305983808364 Total: 0.5304; Orientation: 0.0363; Line Search: 0.3675
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5624eabc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.265e+03, gradient 4.376e-03, dot -0.234; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.9380305983808364, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
LBFGS Accumulation History: 3 points
Removed measurement 5624eabc to history. Total: 3
th(0)=-3.9380305983808364;dx=-1.776509880522968E-5
New Minimum: -3.9380305983808364 > -8.293353798403947
WOLF (strong): th(935744.3088563365)=-8.293353798403947; dx=9.258419120891038E-7 evalInputDelta=4.355323200023111
New Minimum: -8.293353798403947 > -8.582338000603443
END: th(467872.15442816826)=-8.582338000603443; dx=-2.2965009363845038E-6 evalInputDelta=4.644307402222607
Fitness changed from -3.9380305983808364 to -8.582338000603443
Iteration 5 complete. Error: -8.582338000603443 Total: 0.4935; Orientation: 0.0288; Line Search: 0.3438
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 255b4612 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.203e+03, gradient 5.170e-03, dot -0.221; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.582338000603443, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
LBFGS Accumulation History: 3 points
Removed measurement 255b4612 to history. Total: 3
th(0)=-8.582338000603443;dx=-2.6080773251934184E-5
New Minimum: -8.582338000603443 > -9.507102193339346
WOLF (strong): th(1008000.0000000003)=-9.507102193339346; dx=7.032593640709538E-7 evalInputDelta=0.9247641927359034
New Minimum: -9.507102193339346 > -12.245902315896231
END: th(504000.0000000002)=-12.245902315896231; dx=-1.7719632344170813E-6 evalInputDelta=3.6635643152927884
Fitness changed from -8.582338000603443 to -12.245902315896231
Iteration 6 complete. Error: -12.245902315896231 Total: 0.4926; Orientation: 0.0293; Line Search: 0.3416
Adding measurement d935008 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.135e+03, gradient 5.187e-03, dot -0.210; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.245902315896231, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
LBFGS Accumulation History: 3 points
Removed measurement d935008 to history. Total: 3
th(0)=-12.245902315896231;dx=-2.554452688815142E-5
New Minimum: -12.245902315896231 > -12.682212093484981
WOLF (strong): th(1085835.0837760698)=-12.682212093484981; dx=2.564606364650344E-6 evalInputDelta=0.43630977758875034
New Minimum: -12.682212093484981 > -16.516884171039763
END: th(542917.5418880349)=-16.516884171039763; dx=-9.135970178958515E-8 evalInputDelta=4.270981855143532
Fitness changed from -12.245902315896231 to -16.516884171039763
Iteration 7 complete. Error: -16.516884171039763 Total: 0.4939; Orientation: 0.0292; Line Search: 0.3514
Adding measurement 19382402 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.161e+04, gradient 5.442e-03, dot -0.217; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.516884171039763, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
LBFGS Accumulation History: 3 points
Removed measurement 19382402 to history. Total: 3
th(0)=-16.516884171039763;dx=-2.7626689252754056E-5
Armijo: th(1169680.3860704207)=-14.140530908314473; dx=2.4287259057821904E-6 evalInputDelta=-2.3763532627252903
New Minimum: -16.516884171039763 > -19.657014858194835
WOLF (strong): th(584840.1930352104)=-19.657014858194835; dx=9.167151761446083E-7 evalInputDelta=3.1401306871550716
New Minimum: -19.657014858194835 > -20.256890127834065
END: th(194946.7310117368)=-20.256890127834065; dx=-8.036254629909125E-6 evalInputDelta=3.7400059567943025
Fitness changed from -16.516884171039763 to -20.256890127834065
Iteration 8 complete. Error: -20.256890127834065 Total: 0.6007; Orientation: 0.0280; Line Search: 0.4589
Adding measurement 5e9257a1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.422e+05, gradient 4.653e-03, dot -0.314; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.256890127834065, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
LBFGS Accumulation History: 3 points
Removed measurement 5e9257a1 to history. Total: 3
th(0)=-20.256890127834065;dx=-2.0375395721485603E-5
New Minimum: -20.256890127834065 > -23.674893004246467
END: th(420000.0000000002)=-23.674893004246467; dx=-3.7797276406911017E-6 evalInputDelta=3.4180028764124017
Fitness changed from -20.256890127834065 to -23.674893004246467
Iteration 9 complete. Error: -23.674893004246467 Total: 0.4211; Orientation: 0.0345; Line Search: 0.2673
Adding measurement 7fc3f37 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.880e+05, gradient 4.941e-03, dot -0.290; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.674893004246467, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
LBFGS Accumulation History: 3 points
Removed measurement 7fc3f37 to history. Total: 3
th(0)=-23.674893004246467;dx=-2.284855995056614E-5
Armijo: th(904862.5698133915)=-22.143869104068727; dx=1.2705102886858494E-6 evalInputDelta=-1.5310239001777397
New Minimum: -23.674893004246467 > -24.761296828737592
END: th(452431.2849066958)=-24.761296828737592; dx=-1.2448855961012136E-6 evalInputDelta=1.086403824491125
Fitness changed from -23.674893004246467 to -24.761296828737592
Iteration 10 complete. Error: -24.761296828737592 Total: 0.5273; Orientation: 0.0320; Line Search: 0.3737
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 63f67db to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.650e+04, gradient 4.900e-03, dot 0.380; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-24.761296828737592;dx=-31.047752097598185
Armijo: th(974733.6550586841)=-22.08261482116956; dx=9.012139106118437E-6 evalInputDelta=-2.6786820075680318
Armijo: th(487366.82752934203)=-22.08261482116956; dx=1.8274810774016648E-5 evalInputDelta=-2.6786820075680318
Armijo: th(162455.60917644735)=-22.082543345794882; dx=5.423545296179261E-5 evalInputDelta=-2.67875348294271
Armijo: th(40613.90229411184)=-22.08284374798642; dx=2.1819827308974885E-4 evalInputDelta=-2.678453080751172
Armijo: th(8122.780458822368)=-22.088414950036565; dx=0.001093818013589245 evalInputDelta=-2.672881878701027
Armijo: th(1353.7967431370614)=-22.084853229963386; dx=0.006671149421815665 evalInputDelta=-2.676443598774206
Armijo: th(193.39953473386592)=-22.052984503458262; dx=0.04495582203569072 evalInputDelta=-2.70831232527933
Armijo: th(24.17494184173324)=-22.474884071565207; dx=0.34830009451260185 evalInputDelta=-2.2864127571723856
Armijo: th(2.686104649081471)=-24.642547932651514; dx=2.140804369169895 evalInputDelta=-0.1187488960860783
New Minimum: -24.761296828737592 > -27.270359558937706
END: th(0.2686104649081471)=-27.270359558937706; dx=-2.3260435668923143 evalInputDelta=2.509062730200114
Fitness changed from -24.761296828737592 to -27.270359558937706
Iteration 11 complete. Error: -27.270359558937706 Total: 1.4311; Orientation: 0.0279; Line Search: 1.2770
Adding measurement 72b83822 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.362e+04, gradient 4.868e-03, dot 0.288; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
th(0)=-27.270359558937706;dx=-19.004135406011738
Armijo: th(0.5787037037037042)=-26.96709322756615; dx=0.7849364344110913 evalInputDelta=-0.3032663313715567
New Minimum: -27.270359558937706 > -27.35734018806031
END: th(0.2893518518518521)=-27.35734018806031; dx=-0.2386360296730365 evalInputDelta=0.0869806291226034
Fitness changed from -27.270359558937706 to -27.35734018806031
Iteration 12 complete. Error: -27.35734018806031 Total: 0.5043; Orientation: 0.0348; Line Search: 0.3557
Adding measurement 1b250427 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 1.202e+04, gradient 4.956e-03, dot 0.234; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
th(0)=-27.35734018806031;dx=-15.11011132032726
Armijo: th(0.6233896672545965)=-26.52353963268974; dx=0.3278998762989377 evalInputDelta=-0.8338005553705692
Armijo: th(0.3116948336272983)=-27.00887356665586; dx=0.033103827243200545 evalInputDelta=-0.34846662140444806
Armijo: th(0.1038982778757661)=-27.27536850316575; dx=-0.049943317596130354 evalInputDelta=-0.0819716848945582
Armijo: th(0.025974569468941524)=-27.338498624803; dx=-0.29167562182218976 evalInputDelta=-0.01884156325730757
Armijo: th(0.0051949138937883044)=-27.353830648838983; dx=-0.5313252046226835 evalInputDelta=-0.003509539221326463
Armijo: th(8.658189822980507E-4)=-27.356749861042594; dx=-0.4632962866551135 evalInputDelta=-5.903270177149977E-4
Armijo: th(1.2368842604257868E-4)=-27.35725490080817; dx=-0.4958331797793076 evalInputDelta=-8.528725214063115E-5
Armijo: th(1.5461053255322335E-5)=-27.3573226289419; dx=-0.4612101950335006 evalInputDelta=-1.755911840817248E-5
Armijo: th(1.7178948061469262E-6)=-27.35734018806031; dx=-0.41863173260018804 evalInputDelta=0.0
New Minimum: -27.35734018806031 > -27.357341024208523
END: th(1.7178948061469263E-7)=-27.357341024208523; dx=-0.3324115313830952 evalInputDelta=8.361482137786425E-7
Fitness changed from -27.35734018806031 to -27.357341024208523
Iteration 13 complete. Error: -27.357341024208523 Total: 1.3980; Orientation: 0.0405; Line Search: 1.2459
Adding measurement 174083c2 to history. Total: 6
Accepted: LBFGS Orientation magnitude: 1.198e+04, gradient 5.029e-03, dot 0.260; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
th(0)=-27.357341024208523;dx=-16.597934789794927
New Minimum: -27.357341024208523 > -27.357341860356712
END: th(3.7010921641885365E-7)=-27.357341860356712; dx=-0.6484726226982754 evalInputDelta=8.361481889096467E-7
Fitness changed from -27.357341024208523 to -27.357341860356712
Iteration 14 complete. Error: -27.357341860356712 Total: 0.3817; Orientation: 0.0465; Line Search: 0.2267
Adding measurement 3aba12aa to history. Total: 7
Rejected: LBFGS Orientation magnitude: 1.130e+06, gradient 4.920e-03, dot -0.191; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.357341860356712, -27.357341024208523, -27.35734018806031, -27.270359558937706, -24.761296828737592, -2.261035636538084, -1.4087567726922208, -0.7054832604961508
Rejected: LBFGS Orientation magnitude: 4.754e+06, gradient 4.920e-03, dot -0.244; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.357341860356712, -27.357341024208523, -27.35734018806031, -27.270359558937706, -24.761296828737592, -2.261035636538084, -1.4087567726922208
Accepted: LBFGS Orientation magnitude: 1.464e+06, gradient 4.920e-03, dot 0.246; [1223ff5c-5423-4915-b484-16eb7acd1062 = 1.000/1.000e+00]
Overwriting history with 6 points
th(0)=-27.357341860356712;dx=-1819.197013514207
New Minimum: -27.357341860356712 > -27.357434672646846
END: th(7.973761349532963E-7)=-27.357434672646846; dx=-123.05772167029656 evalInputDelta=9.281229013424763E-5
Fitness changed from -27.357341860356712 to -27.357434672646846
Iteration 15 complete. Error: -27.357434672646846 Total: 0.4772; Orientation: 0.1412; Line Search: 0.2271
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -27.357434672646846 (> -Infinity) after 9.932s (< 5400.000s)
Returns
-27.357434672646846