BasicOptimizer.scala:89 executed in 66.62 seconds (0.735 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 138263681046899
Reset training subject: 138264599182700
Adding measurement 693fc743 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-32.810551127618865;dx=-0.003414851848756915
New Minimum: -32.810551127618865 > -32.81741582541748
WOLFE (weak): th(2.154434690031884)=-32.81741582541748; dx=-0.0032946016479742323 evalInputDelta=0.006864697798611985
New Minimum: -32.81741582541748 > -32.82428221258935
WOLFE (weak): th(4.308869380063768)=-32.82428221258935; dx=-0.0032962475043187504 evalInputDelta=0.013731084970487473
New Minimum: -32.82428221258935 > -32.85175995021071
WOLFE (weak): th(12.926608140191302)=-32.85175995021071; dx=-0.0032918700103407063 evalInputDelta=0.041208822591848104
New Minimum: -32.85175995021071 > -32.975396276638456
WOLFE (weak): th(51.70643256076521)=-32.975396276638456; dx=-0.003290924448822974 evalInputDelta=0.16484514901959102
New Minimum: -32.975396276638456 > -33.63647663118688
WOLFE (weak): th(258.53216280382605)=-33.63647663118688; dx=-0.0033114818712292274 evalInputDelta=0.8259255035680155
New Minimum: -33.63647663118688 > -37.95648529999359
WOLFE (weak): th(1551.1929768229563)=-37.95648529999359; dx=-0.0034649158078062896 evalInputDelta=5.145934172374723
New Minimum: -37.95648529999359 > -68.42440301711342
END: th(10858.350837760694)=-68.42440301711342; dx=-0.002620631058468828 evalInputDelta=35.613851889494555
Fitness changed from -32.810551127618865 to -68.42440301711342
Iteration 1 complete. Error: -68.42440301711342 Total: 10.0850; Orientation: 0.0162; Line Search: 7.4370
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6d9a17a4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-68.42440301711342;dx=-0.004231494268246993
New Minimum: -68.42440301711342 > -128.59718262271915
END: th(23393.607721408407)=-128.59718262271915; dx=-0.0016038849004893201 evalInputDelta=60.172779605605726
Fitness changed from -68.42440301711342 to -128.59718262271915
Iteration 2 complete. Error: -128.59718262271915 Total: 2.9605; Orientation: 0.0174; Line Search: 1.7977
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 66b00ffe to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-128.59718262271915;dx=-0.00402919333454445
New Minimum: -128.59718262271915 > -212.96929066797017
END: th(50400.00000000001)=-212.96929066797017; dx=-8.606534976729014E-4 evalInputDelta=84.37210804525103
Fitness changed from -128.59718262271915 to -212.96929066797017
Iteration 3 complete. Error: -212.96929066797017 Total: 2.7887; Orientation: 0.0157; Line Search: 1.7882
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement ed099fb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.554e+04, gradient 7.372e-02, dot -0.255; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -212.96929066797017, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement ed099fb to history. Total: 3
th(0)=-212.96929066797017;dx=-0.00541296584028679
New Minimum: -212.96929066797017 > -264.28657789974426
END: th(108583.50837760697)=-264.28657789974426; dx=-6.410899238122036E-5 evalInputDelta=51.31728723177409
Fitness changed from -212.96929066797017 to -264.28657789974426
Iteration 4 complete. Error: -264.28657789974426 Total: 3.0084; Orientation: 0.2861; Line Search: 1.7384
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 21b97023 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.752e+04, gradient 7.348e-02, dot -0.161; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -264.28657789974426, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 21b97023 to history. Total: 3
th(0)=-264.28657789974426;dx=-0.0053819119058110595
New Minimum: -264.28657789974426 > -314.10862348190165
WOLF (strong): th(233936.07721408413)=-314.10862348190165; dx=1.432276658222185E-4 evalInputDelta=49.822045582157386
New Minimum: -314.10862348190165 > -394.69322762756696
END: th(116968.03860704206)=-394.69322762756696; dx=-1.4811419101971887E-4 evalInputDelta=130.4066497278227
Fitness changed from -264.28657789974426 to -394.69322762756696
Iteration 5 complete. Error: -394.69322762756696 Total: 4.0667; Orientation: 0.2784; Line Search: 2.6523
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7e5dcf8f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.148e+04, gradient 6.454e-02, dot -0.151; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -394.69322762756696, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 7e5dcf8f to history. Total: 3
th(0)=-394.69322762756696;dx=-0.0041558921319135925
New Minimum: -394.69322762756696 > -434.87261477282715
WOLF (strong): th(252000.0000000001)=-434.87261477282715; dx=2.7034881639784907E-4 evalInputDelta=40.17938714526019
New Minimum: -434.87261477282715 > -552.3682848401385
END: th(126000.00000000004)=-552.3682848401385; dx=-2.9085268351935806E-4 evalInputDelta=157.67505721257152
Fitness changed from -394.69322762756696 to -552.3682848401385
Iteration 6 complete. Error: -552.3682848401385 Total: 4.0520; Orientation: 0.2984; Line Search: 2.8071
Adding measurement 615e410e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.175e+04, gradient 6.394e-02, dot -0.199; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -552.3682848401385, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 615e410e to history. Total: 3
th(0)=-552.3682848401385;dx=-0.004071778435766323
Armijo: th(271458.77094401745)=-513.6068400459712; dx=4.0434345077855244E-4 evalInputDelta=-38.76144479416723
New Minimum: -552.3682848401385 > -667.1525340703039
END: th(135729.38547200873)=-667.1525340703039; dx=-4.169184112577154E-5 evalInputDelta=114.78424923016541
Fitness changed from -552.3682848401385 to -667.1525340703039
Iteration 7 complete. Error: -667.1525340703039 Total: 4.0536; Orientation: 0.2794; Line Search: 2.9209
Adding measurement 40d9ef44 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.360e+04, gradient 6.472e-02, dot -0.205; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -667.1525340703039, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 40d9ef44 to history. Total: 3
th(0)=-667.1525340703039;dx=-0.004160015096572571
Armijo: th(292420.0965176052)=-574.8965192625407; dx=5.559029944157003E-4 evalInputDelta=-92.25601480776322
New Minimum: -667.1525340703039 > -768.3973743898947
WOLF (strong): th(146210.0482588026)=-768.3973743898947; dx=2.4105903940772379E-4 evalInputDelta=101.24484031959082
New Minimum: -768.3973743898947 > -787.0620763241953
END: th(48736.6827529342)=-787.0620763241953; dx=-0.0015237291495814708 evalInputDelta=119.90954225389146
Fitness changed from -667.1525340703039 to -787.0620763241953
Iteration 8 complete. Error: -787.0620763241953 Total: 5.2958; Orientation: 0.4001; Line Search: 3.7636
Adding measurement 4fc55fda to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.668e+05, gradient 5.333e-02, dot -0.283; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -787.0620763241953, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 4fc55fda to history. Total: 3
th(0)=-787.0620763241953;dx=-0.002832685422216209
New Minimum: -787.0620763241953 > -885.7563722772198
END: th(105000.00000000004)=-885.7563722772198; dx=-8.18948987330814E-4 evalInputDelta=98.6942959530245
Fitness changed from -787.0620763241953 to -885.7563722772198
Iteration 9 complete. Error: -885.7563722772198 Total: 3.4729; Orientation: 0.2727; Line Search: 2.1415
Adding measurement 342241ba to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.470e+04, gradient 5.528e-02, dot 0.468; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-885.7563722772198;dx=-890.2435178805647
Armijo: th(226215.6424533479)=-467.0008584600926; dx=0.001221713106677833 evalInputDelta=-418.7555138171272
Armijo: th(113107.82122667394)=-467.00604567560373; dx=0.0024527980368786416 evalInputDelta=-418.7503266016161
Armijo: th(37702.60707555798)=-467.00275622854866; dx=0.007364808202320939 evalInputDelta=-418.7536160486712
Armijo: th(9425.651768889495)=-467.1073183810093; dx=0.02934972565986758 evalInputDelta=-418.64905389621055
Armijo: th(1885.130353777899)=-467.56601047040573; dx=0.1479967441263284 evalInputDelta=-418.1903618068141
Armijo: th(314.1883922963165)=-469.18153688290755; dx=0.8849306623742725 evalInputDelta=-416.5748353943123
Armijo: th(44.88405604233093)=-488.48416764181707; dx=6.219173626934312 evalInputDelta=-397.2722046354028
Armijo: th(5.610507005291367)=-611.3820970747338; dx=42.24380132875149 evalInputDelta=-274.37427520248605
New Minimum: -885.7563722772198 > -932.2648351844048
END: th(0.6233896672545963)=-932.2648351844048; dx=-52.778522396482316 evalInputDelta=46.508462907184935
Fitness changed from -885.7563722772198 to -932.2648351844048
Iteration 10 complete. Error: -932.2648351844048 Total: 11.7487; Orientation: 0.2775; Line Search: 9.0257
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 45cf81c9 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.445e+05, gradient 5.783e-02, dot -0.075; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -932.2648351844048, -885.7563722772198, -128.59718262271915, -68.42440301711342, -32.810551127618865
Rejected: LBFGS Orientation magnitude: 6.921e+05, gradient 5.783e-02, dot -0.202; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -932.2648351844048, -885.7563722772198, -128.59718262271915, -68.42440301711342
LBFGS Accumulation History: 3 points
Removed measurement 45cf81c9 to history. Total: 4
Removed measurement 342241ba to history. Total: 3
th(0)=-932.2648351844048;dx=-0.00331524833070393
New Minimum: -932.2648351844048 > -932.2680462768457
END: th(1.3430523245407355)=-932.2680462768457; dx=-0.002059537853335606 evalInputDelta=0.003211092440892571
Fitness changed from -932.2648351844048 to -932.2680462768457
Iteration 11 complete. Error: -932.2680462768457 Total: 3.3335; Orientation: 0.6097; Line Search: 1.7535
Adding measurement 7d7f8d79 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.736e+04, gradient 5.773e-02, dot -0.142; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -932.2680462768457, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 7d7f8d79 to history. Total: 3
th(0)=-932.2680462768457;dx=-0.0033187629730742378
New Minimum: -932.2680462768457 > -932.274919104198
END: th(2.8935185185185204)=-932.274919104198; dx=-0.0020635495980540638 evalInputDelta=0.006872827352367494
Fitness changed from -932.2680462768457 to -932.274919104198
Iteration 12 complete. Error: -932.274919104198 Total: 2.8957; Orientation: 0.2801; Line Search: 1.8049
Adding measurement 7163e365 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.701e+04, gradient 5.770e-02, dot -0.142; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -932.274919104198, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 7163e365 to history. Total: 3
th(0)=-932.274919104198;dx=-0.0033090205906455006
New Minimum: -932.274919104198 > -932.2899039455478
END: th(6.233896672545964)=-932.2899039455478; dx=-0.0020480533409657897 evalInputDelta=0.014984841349814815
Fitness changed from -932.274919104198 to -932.2899039455478
Iteration 13 complete. Error: -932.2899039455478 Total: 2.9918; Orientation: 0.2803; Line Search: 1.8784
Adding measurement 4b779225 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.748e+04, gradient 5.775e-02, dot -0.143; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -932.2899039455478, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 4b779225 to history. Total: 3
th(0)=-932.2899039455478;dx=-0.0033201134749506873
New Minimum: -932.2899039455478 > -932.3220135088588
END: th(13.430523245407356)=-932.3220135088588; dx=-0.0020575091391443587 evalInputDelta=0.032109563310996236
Fitness changed from -932.2899039455478 to -932.3220135088588
Iteration 14 complete. Error: -932.3220135088588 Total: 2.8810; Orientation: 0.2790; Line Search: 1.7406
Adding measurement 5a63f0a8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.734e+04, gradient 5.775e-02, dot -0.142; [c820847f-01b3-40de-8a9a-14cf26f54571 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -932.3220135088588, -128.59718262271915, -68.42440301711342, -32.810551127618865
LBFGS Accumulation History: 3 points
Removed measurement 5a63f0a8 to history. Total: 3
th(0)=-932.3220135088588;dx=-0.0033275093095903345
New Minimum: -932.3220135088588 > -932.3913551263073
END: th(28.935185185185208)=-932.3913551263073; dx=-0.002058899261536225 evalInputDelta=0.06934161744845824
Fitness changed from -932.3220135088588 to -932.3913551263073
Iteration 15 complete. Error: -932.3913551263073 Total: 2.8643; Orientation: 0.2763; Line Search: 1.7719
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -932.3913551263073 (> -Infinity) after 66.619s (< 5400.000s)
Returns
-932.3913551263073