BasicOptimizer.scala:89 executed in 9.69 seconds (0.617 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 138002989609499
Reset training subject: 138003285393400
Adding measurement 36d582cb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.5922944712438987;dx=-2.899000714205826E-5
New Minimum: -0.5922944712438987 > -0.5923771654624913
WOLFE (weak): th(2.154434690031884)=-0.5923771654624913; dx=-2.84096706080419E-5 evalInputDelta=8.269421859263826E-5
New Minimum: -0.5923771654624913 > -0.5924588505008763
WOLFE (weak): th(4.308869380063768)=-0.5924588505008763; dx=-2.846096739889238E-5 evalInputDelta=1.6437925697765365E-4
New Minimum: -0.5924588505008763 > -0.5927873453665016
WOLFE (weak): th(12.926608140191302)=-0.5927873453665016; dx=-2.9028293579960168E-5 evalInputDelta=4.928741226029043E-4
New Minimum: -0.5927873453665016 > -0.594268867822005
WOLFE (weak): th(51.70643256076521)=-0.594268867822005; dx=-2.84664876421977E-5 evalInputDelta=0.0019743965781063855
New Minimum: -0.594268867822005 > -0.6022658072582306
WOLFE (weak): th(258.53216280382605)=-0.6022658072582306; dx=-2.8862778995800865E-5 evalInputDelta=0.009971336014331977
New Minimum: -0.6022658072582306 > -0.6562738694420123
WOLFE (weak): th(1551.1929768229563)=-0.6562738694420123; dx=-3.056457093921492E-5 evalInputDelta=0.0639793981981136
New Minimum: -0.6562738694420123 > -1.1369698723271136
WOLFE (weak): th(10858.350837760694)=-1.1369698723271136; dx=-3.3005111508890913E-5 evalInputDelta=0.5446754010832149
New Minimum: -1.1369698723271136 > -2.7027998743696604
END: th(86866.80670208555)=-2.7027998743696604; dx=-6.8240465498612805E-6 evalInputDelta=2.110505403125762
Fitness changed from -0.5922944712438987 to -2.7027998743696604
Iteration 1 complete. Error: -2.7027998743696604 Total: 1.9017; Orientation: 0.0218; Line Search: 1.2750
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7a8b7e11 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.7027998743696604;dx=-7.035014703885164E-5
New Minimum: -2.7027998743696604 > -3.7779885267052387
WOLF (strong): th(187148.86177126726)=-3.7779885267052387; dx=1.7189961100877878E-6 evalInputDelta=1.0751886523355783
New Minimum: -3.7779885267052387 > -4.679030579844833
END: th(93574.43088563363)=-4.679030579844833; dx=-3.6668108445580365E-6 evalInputDelta=1.9762307054751722
Fitness changed from -2.7027998743696604 to -4.679030579844833
Iteration 2 complete. Error: -4.679030579844833 Total: 0.5714; Orientation: 0.0021; Line Search: 0.4024
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 45295cfd to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.679030579844833;dx=-5.6541599239314676E-5
New Minimum: -4.679030579844833 > -5.648915031119158
WOLF (strong): th(201600.00000000003)=-5.648915031119158; dx=4.216647388116254E-6 evalInputDelta=0.969884451274325
New Minimum: -5.648915031119158 > -7.060832724558783
END: th(100800.00000000001)=-7.060832724558783; dx=-5.853663650615915E-6 evalInputDelta=2.3818021447139506
Fitness changed from -4.679030579844833 to -7.060832724558783
Iteration 3 complete. Error: -7.060832724558783 Total: 0.5211; Orientation: 0.0019; Line Search: 0.3789
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement f7b7720 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.161e+03, gradient 8.138e-03, dot -0.299; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.060832724558783, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement f7b7720 to history. Total: 3
th(0)=-7.060832724558783;dx=-5.638544461320385E-5
Armijo: th(217167.01675521393)=-6.613155632438159; dx=4.986336398124916E-6 evalInputDelta=-0.44767709212062456
New Minimum: -7.060832724558783 > -8.216514723140868
WOLF (strong): th(108583.50837760697)=-8.216514723140868; dx=3.145736190862673E-7 evalInputDelta=1.155681998582085
New Minimum: -8.216514723140868 > -8.807352992302091
END: th(36194.502792535655)=-8.807352992302091; dx=-1.8224476240480198E-5 evalInputDelta=1.7465202677433078
Fitness changed from -7.060832724558783 to -8.807352992302091
Iteration 4 complete. Error: -8.807352992302091 Total: 0.9538; Orientation: 0.0597; Line Search: 0.4797
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement ae85aad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.450e+03, gradient 6.483e-03, dot -0.351; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.807352992302091, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement ae85aad to history. Total: 3
th(0)=-8.807352992302091;dx=-3.577802890012653E-5
New Minimum: -8.807352992302091 > -9.857582529999586
END: th(77978.69240469471)=-9.857582529999586; dx=-1.0494110783676454E-5 evalInputDelta=1.0502295376974953
Fitness changed from -8.807352992302091 to -9.857582529999586
Iteration 5 complete. Error: -9.857582529999586 Total: 0.3925; Orientation: 0.0337; Line Search: 0.2295
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5984742f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.987e+03, gradient 7.279e-03, dot -0.368; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.857582529999586, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 5984742f to history. Total: 3
th(0)=-9.857582529999586;dx=-4.673222964267658E-5
Armijo: th(168000.00000000006)=-9.691594496278181; dx=1.7132929423001556E-6 evalInputDelta=-0.1659880337214048
New Minimum: -9.857582529999586 > -11.148275797801118
END: th(84000.00000000003)=-11.148275797801118; dx=-3.644415567392658E-6 evalInputDelta=1.2906932678015313
Fitness changed from -9.857582529999586 to -11.148275797801118
Iteration 6 complete. Error: -11.148275797801118 Total: 0.4851; Orientation: 0.0298; Line Search: 0.3341
Adding measurement 4b1ef6a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.536e+03, gradient 6.971e-03, dot -0.323; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.148275797801118, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 4b1ef6a to history. Total: 3
th(0)=-11.148275797801118;dx=-4.426532046717301E-5
Armijo: th(180972.5139626783)=-9.796327591987007; dx=7.201732373919774E-6 evalInputDelta=-1.3519482058141108
New Minimum: -11.148275797801118 > -12.229942824556105
END: th(90486.25698133915)=-12.229942824556105; dx=-3.0234966407724293E-6 evalInputDelta=1.081667026754987
Fitness changed from -11.148275797801118 to -12.229942824556105
Iteration 7 complete. Error: -12.229942824556105 Total: 0.4834; Orientation: 0.0304; Line Search: 0.3432
Adding measurement 74f1e112 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.022e+03, gradient 6.814e-03, dot -0.385; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.229942824556105, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 74f1e112 to history. Total: 3
th(0)=-12.229942824556105;dx=-4.138659835995984E-5
Armijo: th(194946.7310117368)=-11.919396179594846; dx=6.616661754577523E-6 evalInputDelta=-0.31054664496125817
New Minimum: -12.229942824556105 > -13.706611424009237
WOLF (strong): th(97473.3655058684)=-13.706611424009237; dx=7.044082912296488E-7 evalInputDelta=1.4766685994531326
END: th(32491.121835289465)=-13.44194021849788; dx=-2.2423416121572718E-5 evalInputDelta=1.2119973939417754
Fitness changed from -12.229942824556105 to -13.706611424009237
Iteration 8 complete. Error: -13.706611424009237 Total: 0.5897; Orientation: 0.0280; Line Search: 0.4529
Adding measurement 5f3705e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.857e+03, gradient 7.085e-03, dot -0.337; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.706611424009237, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 5f3705e7 to history. Total: 3
th(0)=-13.706611424009237;dx=-4.464287219346831E-5
New Minimum: -13.706611424009237 > -14.812646421318155
END: th(70000.00000000003)=-14.812646421318155; dx=-2.293995984292323E-6 evalInputDelta=1.1060349973089174
Fitness changed from -13.706611424009237 to -14.812646421318155
Iteration 9 complete. Error: -14.812646421318155 Total: 0.3688; Orientation: 0.0312; Line Search: 0.2307
Adding measurement 24a29d8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.130e+03, gradient 5.976e-03, dot -0.407; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.812646421318155, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 24a29d8 to history. Total: 3
th(0)=-14.812646421318155;dx=-3.341287043043377E-5
New Minimum: -14.812646421318155 > -14.845527519581927
WOLF (strong): th(150810.42830223194)=-14.845527519581927; dx=2.140870071800127E-6 evalInputDelta=0.032881098263771946
New Minimum: -14.845527519581927 > -15.674484028339881
END: th(75405.21415111597)=-15.674484028339881; dx=-4.094165639979341E-6 evalInputDelta=0.8618376070217266
Fitness changed from -14.812646421318155 to -15.674484028339881
Iteration 10 complete. Error: -15.674484028339881 Total: 0.4812; Orientation: 0.0291; Line Search: 0.3411
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 2464c125 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.115e+03, gradient 5.413e-03, dot -0.441; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.674484028339881, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 2464c125 to history. Total: 3
th(0)=-15.674484028339881;dx=-2.681317076400427E-5
New Minimum: -15.674484028339881 > -16.16588012911447
WOLF (strong): th(162455.60917644735)=-16.16588012911447; dx=1.8628251072610374E-6 evalInputDelta=0.4913961007745886
New Minimum: -16.16588012911447 > -16.77074120368994
END: th(81227.80458822368)=-16.77074120368994; dx=-5.118536592743433E-6 evalInputDelta=1.0962571753500594
Fitness changed from -15.674484028339881 to -16.77074120368994
Iteration 11 complete. Error: -16.77074120368994 Total: 0.5111; Orientation: 0.0331; Line Search: 0.3468
Adding measurement 8814bb5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.300e+04, gradient 5.536e-03, dot -0.468; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.77074120368994, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 8814bb5 to history. Total: 3
th(0)=-16.77074120368994;dx=-2.745680334497409E-5
Armijo: th(175000.00000000012)=-16.396824843288805; dx=2.116233708176657E-6 evalInputDelta=-0.37391636040113596
New Minimum: -16.77074120368994 > -17.424511738180314
END: th(87500.00000000006)=-17.424511738180314; dx=-4.604582537774922E-6 evalInputDelta=0.6537705344903735
Fitness changed from -16.77074120368994 to -17.424511738180314
Iteration 12 complete. Error: -17.424511738180314 Total: 0.8836; Orientation: 0.0321; Line Search: 0.7295
Adding measurement 6c54bc3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.076e+04, gradient 5.153e-03, dot -0.463; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.424511738180314, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 6c54bc3 to history. Total: 3
th(0)=-17.424511738180314;dx=-2.4184973250300878E-5
New Minimum: -17.424511738180314 > -17.840499888899032
WOLF (strong): th(188513.03537778996)=-17.840499888899032; dx=2.0226116908657902E-6 evalInputDelta=0.415988150718718
New Minimum: -17.840499888899032 > -18.468052557479858
END: th(94256.51768889498)=-18.468052557479858; dx=-5.33979692784236E-6 evalInputDelta=1.043540819299544
Fitness changed from -17.424511738180314 to -18.468052557479858
Iteration 13 complete. Error: -18.468052557479858 Total: 0.4699; Orientation: 0.0274; Line Search: 0.3364
Adding measurement 525dd5af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.852e+04, gradient 5.103e-03, dot -0.504; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.468052557479858, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement 525dd5af to history. Total: 3
th(0)=-18.468052557479858;dx=-2.4163507584675496E-5
Armijo: th(203069.51147055923)=-18.00711635961167; dx=1.0381642235661644E-6 evalInputDelta=-0.4609361978681896
New Minimum: -18.468052557479858 > -18.94529285949242
END: th(101534.75573527961)=-18.94529285949242; dx=-2.3885164602739915E-6 evalInputDelta=0.47724030201256085
Fitness changed from -18.468052557479858 to -18.94529285949242
Iteration 14 complete. Error: -18.94529285949242 Total: 0.4763; Orientation: 0.0282; Line Search: 0.3384
Adding measurement d031623 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.746e+04, gradient 5.852e-03, dot -0.431; [610fa4fa-ff2c-473f-a3ea-c3e4853ad36e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.94529285949242, -4.679030579844833, -2.7027998743696604, -0.5922944712438987
LBFGS Accumulation History: 3 points
Removed measurement d031623 to history. Total: 3
th(0)=-18.94529285949242;dx=-3.1497865077240683E-5
Armijo: th(218750.00000000017)=-18.641916662895767; dx=4.629036150945515E-6 evalInputDelta=-0.30337619659665194
New Minimum: -18.94529285949242 > -19.646080703111544
WOLF (strong): th(109375.00000000009)=-19.646080703111544; dx=6.65193964172844E-7 evalInputDelta=0.7007878436191248
New Minimum: -19.646080703111544 > -19.84951596899901
END: th(36458.333333333365)=-19.84951596899901; dx=-1.2510489927724977E-5 evalInputDelta=0.9042231095065922
Fitness changed from -18.94529285949242 to -19.84951596899901
Iteration 15 complete. Error: -19.84951596899901 Total: 0.5713; Orientation: 0.0270; Line Search: 0.4376
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -19.84951596899901 (> -Infinity) after 9.674s (< 5400.000s)
Returns
-19.84951596899901