BasicOptimizer.scala:89 executed in 61.91 seconds (1.349 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 209489792166700
Reset training subject: 209490547911700
Adding measurement 168298ad to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=7.928164436680879;dx=-5.975181856622386E-4
New Minimum: 7.928164436680879 > 7.927165885091384
WOLFE (weak): th(2.154434690031884)=7.927165885091384; dx=-5.711338042543334E-4 evalInputDelta=9.985515894950225E-4
New Minimum: 7.927165885091384 > 7.926168220759784
WOLFE (weak): th(4.308869380063768)=7.926168220759784; dx=-6.027967865551658E-4 evalInputDelta=0.0019962159210944463
New Minimum: 7.926168220759784 > 7.92217078645615
WOLFE (weak): th(12.926608140191302)=7.92217078645615; dx=-5.847777191967481E-4 evalInputDelta=0.00599365022472842
New Minimum: 7.92217078645615 > 7.904075601940665
WOLFE (weak): th(51.70643256076521)=7.904075601940665; dx=-5.971875026506136E-4 evalInputDelta=0.024088834740213727
New Minimum: 7.904075601940665 > 7.807228744697131
WOLFE (weak): th(258.53216280382605)=7.807228744697131; dx=-5.885388945348582E-4 evalInputDelta=0.12093569198374787
New Minimum: 7.807228744697131 > 7.167840683557493
WOLFE (weak): th(1551.1929768229563)=7.167840683557493; dx=-6.677514848401028E-4 evalInputDelta=0.7603237531233855
New Minimum: 7.167840683557493 > 3.103408186146538
END: th(10858.350837760694)=3.103408186146538; dx=-4.682762900251855E-4 evalInputDelta=4.824756250534341
Fitness changed from 7.928164436680879 to 3.103408186146538
Iteration 1 complete. Error: 3.103408186146538 Total: 8.4039; Orientation: 0.0162; Line Search: 6.2673
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 16c6c8bf to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=3.103408186146538;dx=-0.00140508096852947
New Minimum: 3.103408186146538 > -9.04335401297583
END: th(23393.607721408407)=-9.04335401297583; dx=-3.9864523888095385E-4 evalInputDelta=12.146762199122367
Fitness changed from 3.103408186146538 to -9.04335401297583
Iteration 2 complete. Error: -9.04335401297583 Total: 2.2496; Orientation: 0.0162; Line Search: 1.5415
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 788ee018 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-9.04335401297583;dx=-0.0016237317089737228
New Minimum: -9.04335401297583 > -29.12534762784035
END: th(50400.00000000001)=-29.12534762784035; dx=-2.7310384875653317E-4 evalInputDelta=20.08199361486452
Fitness changed from -9.04335401297583 to -29.12534762784035
Iteration 3 complete. Error: -29.12534762784035 Total: 2.4760; Orientation: 0.0167; Line Search: 1.7591
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4246fec7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.172e+04, gradient 3.958e-02, dot -0.138; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.12534762784035, -9.04335401297583, 3.103408186146538, 7.928164436680879
LBFGS Accumulation History: 3 points
Removed measurement 4246fec7 to history. Total: 3
th(0)=-29.12534762784035;dx=-0.0015391179069455252
New Minimum: -29.12534762784035 > -58.518441379247236
END: th(108583.50837760697)=-58.518441379247236; dx=-1.0094773764181996E-4 evalInputDelta=29.393093751406887
Fitness changed from -29.12534762784035 to -58.518441379247236
Iteration 4 complete. Error: -58.518441379247236 Total: 2.5652; Orientation: 0.2989; Line Search: 1.5247
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6054d815 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.266e+04, gradient 3.948e-02, dot -0.121; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -58.518441379247236, -9.04335401297583, 3.103408186146538, 7.928164436680879
LBFGS Accumulation History: 3 points
Removed measurement 6054d815 to history. Total: 3
th(0)=-58.518441379247236;dx=-0.0015401519254906665
New Minimum: -58.518441379247236 > -78.00100339855521
WOLF (strong): th(233936.07721408413)=-78.00100339855521; dx=2.430445553197058E-5 evalInputDelta=19.482562019307977
New Minimum: -78.00100339855521 > -105.18756673051033
END: th(116968.03860704206)=-105.18756673051033; dx=-1.2810926567569754E-4 evalInputDelta=46.66912535126309
Fitness changed from -58.518441379247236 to -105.18756673051033
Iteration 5 complete. Error: -105.18756673051033 Total: 3.2932; Orientation: 0.2942; Line Search: 2.2898
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 23e3df48 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.190e+04, gradient 4.308e-02, dot -0.091; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -105.18756673051033, -9.04335401297583, 3.103408186146538, 7.928164436680879
LBFGS Accumulation History: 3 points
Removed measurement 23e3df48 to history. Total: 3
th(0)=-105.18756673051033;dx=-0.0018359241379936947
Armijo: th(252000.0000000001)=-102.05696073236064; dx=5.1672471251275554E-5 evalInputDelta=-3.13060599814969
New Minimum: -105.18756673051033 > -154.09573272481472
END: th(126000.00000000004)=-154.09573272481472; dx=-1.38582358471281E-4 evalInputDelta=48.9081659943044
Fitness changed from -105.18756673051033 to -154.09573272481472
Iteration 6 complete. Error: -154.09573272481472 Total: 3.4589; Orientation: 0.2830; Line Search: 2.2952
Adding measurement 5c023a4f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.489e+04, gradient 4.514e-02, dot -0.098; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -154.09573272481472, -9.04335401297583, 3.103408186146538, 7.928164436680879
LBFGS Accumulation History: 3 points
Removed measurement 5c023a4f to history. Total: 3
th(0)=-154.09573272481472;dx=-0.0020187987459782094
Armijo: th(271458.77094401745)=-128.021241467835; dx=6.330420606109365E-5 evalInputDelta=-26.074491256979712
New Minimum: -154.09573272481472 > -204.24600968661093
END: th(135729.38547200873)=-204.24600968661093; dx=-1.40036845911115E-4 evalInputDelta=50.15027696179621
Fitness changed from -154.09573272481472 to -204.24600968661093
Iteration 7 complete. Error: -204.24600968661093 Total: 3.5204; Orientation: 0.3203; Line Search: 2.5318
Adding measurement 7bd32175 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.626e+04, gradient 4.785e-02, dot -0.073; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -204.24600968661093, -9.04335401297583, 3.103408186146538, 7.928164436680879
LBFGS Accumulation History: 3 points
Removed measurement 7bd32175 to history. Total: 3
th(0)=-204.24600968661093;dx=-0.0022812920215348476
Armijo: th(292420.0965176052)=-149.48678715422102; dx=1.1942949446433512E-4 evalInputDelta=-54.75922253238991
New Minimum: -204.24600968661093 > -251.78159437003518
END: th(146210.0482588026)=-251.78159437003518; dx=-9.063279971088104E-5 evalInputDelta=47.53558468342425
Fitness changed from -204.24600968661093 to -251.78159437003518
Iteration 8 complete. Error: -251.78159437003518 Total: 3.4516; Orientation: 0.2958; Line Search: 2.4679
Adding measurement 5ea7d52e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.017e+05, gradient 4.985e-02, dot -0.098; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -251.78159437003518, -9.04335401297583, 3.103408186146538, 7.928164436680879
LBFGS Accumulation History: 3 points
Removed measurement 5ea7d52e to history. Total: 3
th(0)=-251.78159437003518;dx=-0.0024801613695398135
Armijo: th(315000.0000000001)=-184.91562894602592; dx=1.4261218095474476E-4 evalInputDelta=-66.86596542400926
New Minimum: -251.78159437003518 > -294.0610252177284
END: th(157500.00000000006)=-294.0610252177284; dx=-8.893744419252638E-5 evalInputDelta=42.27943084769322
Fitness changed from -251.78159437003518 to -294.0610252177284
Iteration 9 complete. Error: -294.0610252177284 Total: 3.2415; Orientation: 0.2986; Line Search: 2.2668
Adding measurement 373568a2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.833e+05, gradient 4.961e-02, dot -0.095; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -294.0610252177284, -9.04335401297583, 3.103408186146538, 7.928164436680879
LBFGS Accumulation History: 3 points
Removed measurement 373568a2 to history. Total: 3
th(0)=-294.0610252177284;dx=-0.002447566452884052
Armijo: th(339323.4636800218)=-216.8747480653409; dx=1.7863624391564314E-4 evalInputDelta=-77.1862771523875
New Minimum: -294.0610252177284 > -351.3174430160132
END: th(169661.7318400109)=-351.3174430160132; dx=-1.268395990557299E-4 evalInputDelta=57.256417798284815
Fitness changed from -294.0610252177284 to -351.3174430160132
Iteration 10 complete. Error: -351.3174430160132 Total: 3.7349; Orientation: 0.4978; Line Search: 2.4938
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement c85ed5d to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.285e+05, gradient 5.052e-02, dot 0.129; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-351.3174430160132;dx=-850.3102399178182
Armijo: th(365525.1206470065)=174.09448982602052; dx=0.0014866724271925657 evalInputDelta=-525.4119328420337
Armijo: th(182762.56032350325)=174.09423350029408; dx=0.0029717076032309735 evalInputDelta=-525.4116765163074
Armijo: th(60920.85344116775)=174.0843950784262; dx=0.008937792622677656 evalInputDelta=-525.4018380944394
Armijo: th(15230.213360291937)=174.08597303683501; dx=0.03568838518173072 evalInputDelta=-525.4034160528482
Armijo: th(3046.0426720583873)=173.9035796881411; dx=0.17835421101887006 evalInputDelta=-525.2210227041543
Armijo: th(507.6737786763979)=173.00691264444393; dx=1.0686129501588437 evalInputDelta=-524.3243556604572
Armijo: th(72.5248255251997)=166.07894008457822; dx=7.3581602022842 evalInputDelta=-517.3963831005915
Armijo: th(9.065603190649963)=68.20028307365064; dx=41.804586980818904 evalInputDelta=-419.5177260896638
Armijo: th(1.0072892434055514)=-151.90745729808316; dx=105.07929419851197 evalInputDelta=-199.40998571793006
Armijo: th(0.10072892434055514)=-323.9763505463646; dx=170.7815406754595 evalInputDelta=-27.34109246964863
New Minimum: -351.3174430160132 > -354.19729954552406
END: th(0.009157174940050467)=-354.19729954552406; dx=-236.97155903508968 evalInputDelta=2.879856529510846
Fitness changed from -351.3174430160132 to -354.19729954552406
Iteration 11 complete. Error: -354.19729954552406 Total: 10.3273; Orientation: 0.2906; Line Search: 9.3012
Adding measurement 7fb08811 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.371e+05, gradient 5.018e-02, dot -0.120; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -354.19729954552406, -351.3174430160132, -9.04335401297583, 3.103408186146538, 7.928164436680879
Rejected: LBFGS Orientation magnitude: 6.753e+05, gradient 5.018e-02, dot -0.161; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -354.19729954552406, -351.3174430160132, -9.04335401297583, 3.103408186146538
LBFGS Accumulation History: 3 points
Removed measurement 7fb08811 to history. Total: 4
Removed measurement c85ed5d to history. Total: 3
th(0)=-354.19729954552406;dx=-0.002508671884905401
New Minimum: -354.19729954552406 > -354.19732858316013
END: th(0.019728535353535363)=-354.19732858316013; dx=-0.0020209250492833784 evalInputDelta=2.9037636068096617E-5
Fitness changed from -354.19729954552406 to -354.19732858316013
Iteration 12 complete. Error: -354.19732858316013 Total: 3.0851; Orientation: 0.9017; Line Search: 1.5041
Adding measurement 48fdab3e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.439e+05, gradient 5.021e-02, dot 0.116; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
th(0)=-354.19732858316013;dx=-844.7326983084351
Armijo: th(0.04250384094917702)=-346.2086636099584; dx=70.6863961949229 evalInputDelta=-7.988664973201708
Armijo: th(0.02125192047458851)=-353.4419072388318; dx=-37.81355122515653 evalInputDelta=-0.7554213443283402
New Minimum: -354.19732858316013 > -354.94021963553934
END: th(0.007083973491529504)=-354.94021963553934; dx=-180.5817848002697 evalInputDelta=0.7428910523792069
Fitness changed from -354.19732858316013 to -354.94021963553934
Iteration 13 complete. Error: -354.94021963553934 Total: 4.2453; Orientation: 0.2786; Line Search: 3.0112
Adding measurement 282949a1 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 7.934e+05, gradient 4.999e-02, dot -0.106; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -354.94021963553934, -354.19732858316013, -9.04335401297583, 3.103408186146538, 7.928164436680879
Rejected: LBFGS Orientation magnitude: 2.106e+06, gradient 4.999e-02, dot -0.162; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
Orientation rejected. Popping history element from -354.94021963553934, -354.19732858316013, -9.04335401297583, 3.103408186146538
LBFGS Accumulation History: 3 points
Removed measurement 282949a1 to history. Total: 4
Removed measurement 48fdab3e to history. Total: 3
th(0)=-354.94021963553934;dx=-0.0024875929477236056
New Minimum: -354.94021963553934 > -354.94024852281876
END: th(0.01526195823341745)=-354.94024852281876; dx=-0.0019763745987282826 evalInputDelta=2.8887279427181056E-5
Fitness changed from -354.94021963553934 to -354.94024852281876
Iteration 14 complete. Error: -354.94024852281876 Total: 2.8588; Orientation: 0.6511; Line Search: 1.5089
Adding measurement 736eb680 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.530e+05, gradient 5.003e-02, dot 0.109; [e74a959f-999d-43c6-9c57-01e478656fae = 1.000/1.000e+00]
th(0)=-354.94024852281876;dx=-833.894478056132
Armijo: th(0.03288089225589228)=-347.79010648516; dx=29.950238977823318 evalInputDelta=-7.150142037658782
Armijo: th(0.01644044612794614)=-353.37009154329485; dx=-68.3823986321143 evalInputDelta=-1.5701569795239152
Armijo: th(0.0054801487093153805)=-354.92552822822324; dx=-175.66392125988494 evalInputDelta=-0.014720294595520045
New Minimum: -354.94024852281876 > -354.9757937046434
END: th(0.0013700371773288451)=-354.9757937046434; dx=-226.50911687523794 evalInputDelta=0.03554518182465927
Fitness changed from -354.94024852281876 to -354.9757937046434
Iteration 15 complete. Error: -354.9757937046434 Total: 4.9775; Orientation: 0.2772; Line Search: 3.6794
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -354.9757937046434 (> -Infinity) after 61.911s (< 5400.000s)
Returns
-354.9757937046434