BasicOptimizer.scala:89 executed in 55.55 seconds (1.691 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 210034282096800
Reset training subject: 210035227161400
Adding measurement 5005f205 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=8.41146616457186;dx=-7.15043912599929E-5
New Minimum: 8.41146616457186 > 8.41135591470411
WOLFE (weak): th(2.154434690031884)=8.41135591470411; dx=-7.146610770582966E-5 evalInputDelta=1.1024986774899048E-4
New Minimum: 8.41135591470411 > 8.41124400708569
WOLFE (weak): th(4.308869380063768)=8.41124400708569; dx=-7.147137338176212E-5 evalInputDelta=2.2215748616893904E-4
New Minimum: 8.41124400708569 > 8.410798312126307
WOLFE (weak): th(12.926608140191302)=8.410798312126307; dx=-7.146622914311178E-5 evalInputDelta=6.678524455523416E-4
New Minimum: 8.410798312126307 > 8.408795870832401
WOLFE (weak): th(51.70643256076521)=8.408795870832401; dx=-7.144923176348523E-5 evalInputDelta=0.0026702937394578186
New Minimum: 8.408795870832401 > 8.398107995154628
WOLFE (weak): th(258.53216280382605)=8.398107995154628; dx=-7.146933713413542E-5 evalInputDelta=0.01335816941723067
New Minimum: 8.398107995154628 > 8.331069771872517
WOLFE (weak): th(1551.1929768229563)=8.331069771872517; dx=-7.160587200439326E-5 evalInputDelta=0.08039639269934185
New Minimum: 8.331069771872517 > 7.838345932737878
WOLFE (weak): th(10858.350837760694)=7.838345932737878; dx=-7.252582435540267E-5 evalInputDelta=0.5731202318339808
New Minimum: 7.838345932737878 > 4.513356573177945
END: th(86866.80670208555)=4.513356573177945; dx=-4.492899161507637E-5 evalInputDelta=3.898109591393914
Fitness changed from 8.41146616457186 to 4.513356573177945
Iteration 1 complete. Error: 4.513356573177945 Total: 9.3449; Orientation: 0.0159; Line Search: 6.9566
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 53da2bf to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.513356573177945;dx=-7.097000612526614E-5
New Minimum: 4.513356573177945 > -0.4837433792831316
END: th(187148.86177126726)=-0.4837433792831316; dx=-2.362324900444614E-5 evalInputDelta=4.997099952461077
Fitness changed from 4.513356573177945 to -0.4837433792831316
Iteration 2 complete. Error: -0.4837433792831316 Total: 2.6314; Orientation: 0.0210; Line Search: 1.7325
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 634c7b1f to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.4837433792831316;dx=-6.277810436449044E-5
New Minimum: -0.4837433792831316 > -3.509409719540937
END: th(403200.00000000006)=-3.509409719540937; dx=-8.201948878233889E-6 evalInputDelta=3.0256663402578052
Fitness changed from -0.4837433792831316 to -3.509409719540937
Iteration 3 complete. Error: -3.509409719540937 Total: 2.1926; Orientation: 0.0159; Line Search: 1.4787
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 238c6321 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.610e+03, gradient 9.210e-03, dot -0.193; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.509409719540937, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 238c6321 to history. Total: 3
th(0)=-3.509409719540937;dx=-8.47534542766578E-5
Armijo: th(868668.0670208557)=3.743071066821061; dx=4.996725228947498E-6 evalInputDelta=-7.252480786361998
New Minimum: -3.509409719540937 > -6.10141419312361
END: th(434334.03351042786)=-6.10141419312361; dx=-6.187672951840785E-6 evalInputDelta=2.592004473582673
Fitness changed from -3.509409719540937 to -6.10141419312361
Iteration 4 complete. Error: -6.10141419312361 Total: 3.6792; Orientation: 0.3063; Line Search: 2.6052
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 245fba75 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.457e+03, gradient 9.140e-03, dot -0.180; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.10141419312361, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 245fba75 to history. Total: 3
th(0)=-6.10141419312361;dx=-8.347481317441186E-5
Armijo: th(935744.3088563365)=3.0516318491463643; dx=5.599298145131574E-6 evalInputDelta=-9.153046042269974
New Minimum: -6.10141419312361 > -9.074756108528181
END: th(467872.15442816826)=-9.074756108528181; dx=-7.884881171329618E-6 evalInputDelta=2.9733419154045713
Fitness changed from -6.10141419312361 to -9.074756108528181
Iteration 5 complete. Error: -9.074756108528181 Total: 3.5930; Orientation: 0.3190; Line Search: 2.5514
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 26eb35ef to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.218e+03, gradient 9.168e-03, dot -0.161; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.074756108528181, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 26eb35ef to history. Total: 3
th(0)=-9.074756108528181;dx=-8.400000115072266E-5
Armijo: th(1008000.0000000003)=2.181460047219513; dx=6.3944849670222885E-6 evalInputDelta=-11.256216155747694
New Minimum: -9.074756108528181 > -12.314430059948663
END: th(504000.0000000002)=-12.314430059948663; dx=-8.764875627106856E-6 evalInputDelta=3.2396739514204818
Fitness changed from -9.074756108528181 to -12.314430059948663
Iteration 6 complete. Error: -12.314430059948663 Total: 3.3618; Orientation: 0.2962; Line Search: 2.3116
Adding measurement 31aafab2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.914e+03, gradient 9.175e-03, dot -0.146; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.314430059948663, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 31aafab2 to history. Total: 3
th(0)=-12.314430059948663;dx=-8.411758991911888E-5
Armijo: th(1085835.0837760698)=2.493360047652363; dx=7.4765954543651715E-6 evalInputDelta=-14.807790107601026
New Minimum: -12.314430059948663 > -14.69857828444637
END: th(542917.5418880349)=-14.69857828444637; dx=-8.387406782215828E-6 evalInputDelta=2.3841482244977072
Fitness changed from -12.314430059948663 to -14.69857828444637
Iteration 7 complete. Error: -14.69857828444637 Total: 3.7759; Orientation: 0.2813; Line Search: 2.2231
Adding measurement 212f1171 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.042e+03, gradient 9.208e-03, dot -0.135; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.69857828444637, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 212f1171 to history. Total: 3
th(0)=-14.69857828444637;dx=-8.470140768242342E-5
Armijo: th(1169680.3860704207)=2.423516244314989; dx=8.700315920943876E-6 evalInputDelta=-17.12209452876136
New Minimum: -14.69857828444637 > -16.999663625599773
END: th(584840.1930352104)=-16.999663625599773; dx=-7.950947441607482E-6 evalInputDelta=2.301085341153403
Fitness changed from -14.69857828444637 to -16.999663625599773
Iteration 8 complete. Error: -16.999663625599773 Total: 3.2369; Orientation: 0.3003; Line Search: 2.2573
Adding measurement 1bf8fb25 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.357e+03, gradient 9.176e-03, dot -0.124; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.999663625599773, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 1bf8fb25 to history. Total: 3
th(0)=-16.999663625599773;dx=-8.415693055444822E-5
Armijo: th(1260000.0000000005)=3.192382512961643; dx=9.468444448750608E-6 evalInputDelta=-20.192046138561416
New Minimum: -16.999663625599773 > -18.89620064309765
END: th(630000.0000000002)=-18.89620064309765; dx=-7.2026515730134035E-6 evalInputDelta=1.8965370174978773
Fitness changed from -16.999663625599773 to -18.89620064309765
Iteration 9 complete. Error: -18.89620064309765 Total: 3.6306; Orientation: 0.3302; Line Search: 2.5638
Adding measurement 2341021b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.283e+03, gradient 9.149e-03, dot -0.111; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.89620064309765, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 2341021b to history. Total: 3
th(0)=-18.89620064309765;dx=-8.364844865920634E-5
Armijo: th(1357293.8547200873)=4.174802190520328; dx=1.0164931376789714E-5 evalInputDelta=-23.071002833617978
New Minimum: -18.89620064309765 > -19.99717974301472
END: th(678646.9273600436)=-19.99717974301472; dx=-6.3979207873190116E-6 evalInputDelta=1.100979099917069
Fitness changed from -18.89620064309765 to -19.99717974301472
Iteration 10 complete. Error: -19.99717974301472 Total: 3.2964; Orientation: 0.2835; Line Search: 2.2783
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 2ad62135 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.941e+03, gradient 9.180e-03, dot -0.100; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.99717974301472, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 2ad62135 to history. Total: 3
th(0)=-19.99717974301472;dx=-8.422682172746364E-5
Armijo: th(1462100.482588026)=5.200325238802215; dx=1.0807948206208299E-5 evalInputDelta=-25.197504981816934
New Minimum: -19.99717974301472 > -21.27424394120613
END: th(731050.241294013)=-21.27424394120613; dx=-6.060805176048759E-6 evalInputDelta=1.277064198191411
Fitness changed from -19.99717974301472 to -21.27424394120613
Iteration 11 complete. Error: -21.27424394120613 Total: 3.5099; Orientation: 0.3133; Line Search: 2.4309
Adding measurement 32a3730a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.922e+03, gradient 9.165e-03, dot -0.092; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.27424394120613, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 32a3730a to history. Total: 3
th(0)=-21.27424394120613;dx=-8.393480925964259E-5
Armijo: th(1575000.0000000007)=6.964219242191319; dx=1.185478324166408E-5 evalInputDelta=-28.23846318339745
New Minimum: -21.27424394120613 > -22.079793071007813
END: th(787500.0000000003)=-22.079793071007813; dx=-5.135994304593793E-6 evalInputDelta=0.8055491298016833
Fitness changed from -21.27424394120613 to -22.079793071007813
Iteration 12 complete. Error: -22.079793071007813 Total: 3.1716; Orientation: 0.2839; Line Search: 2.2257
Adding measurement 66c11682 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.874e+03, gradient 9.236e-03, dot -0.085; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.079793071007813, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 66c11682 to history. Total: 3
th(0)=-22.079793071007813;dx=-8.524689075506588E-5
Armijo: th(1696617.3184001092)=10.071251945446917; dx=1.2295377471567685E-5 evalInputDelta=-32.15104501645473
Armijo: th(848308.6592000546)=-22.0709490785825; dx=-3.6527434801929397E-6 evalInputDelta=-0.008843992425312308
New Minimum: -22.079793071007813 > -30.644250501693627
END: th(282769.55306668486)=-30.644250501693627; dx=-4.431014986469937E-5 evalInputDelta=8.564457430685813
Fitness changed from -22.079793071007813 to -30.644250501693627
Iteration 13 complete. Error: -30.644250501693627 Total: 4.1270; Orientation: 0.3075; Line Search: 3.1182
Adding measurement 6f84916c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.929e+04, gradient 7.618e-03, dot -0.097; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.644250501693627, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 6f84916c to history. Total: 3
th(0)=-30.644250501693627;dx=-5.796594081105986E-5
New Minimum: -30.644250501693627 > -34.54826719253225
END: th(609208.5344116775)=-34.54826719253225; dx=-1.877262659499122E-5 evalInputDelta=3.904016690838624
Fitness changed from -30.644250501693627 to -34.54826719253225
Iteration 14 complete. Error: -34.54826719253225 Total: 2.5941; Orientation: 0.2873; Line Search: 1.5973
Adding measurement 481b9384 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.101e+04, gradient 7.854e-03, dot -0.101; [08825750-db2f-44ed-9b1d-1ab1710ba1fe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.54826719253225, -0.4837433792831316, 4.513356573177945, 8.41146616457186
LBFGS Accumulation History: 3 points
Removed measurement 481b9384 to history. Total: 3
th(0)=-34.54826719253225;dx=-6.164445353373541E-5
Armijo: th(1312500.0000000007)=-18.909788597422697; dx=4.034738732075408E-6 evalInputDelta=-15.638478595109554
New Minimum: -34.54826719253225 > -36.64833999982841
END: th(656250.0000000003)=-36.64833999982841; dx=-1.2515300203757521E-5 evalInputDelta=2.100072807296158
Fitness changed from -34.54826719253225 to -36.64833999982841
Iteration 15 complete. Error: -36.64833999982841 Total: 3.3851; Orientation: 0.2959; Line Search: 2.3435
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -36.64833999982841 (> -Infinity) after 55.553s (< 5400.000s)
Returns
-36.64833999982841