BasicOptimizer.scala:89 executed in 60.63 seconds (1.434 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 209889011717700
Reset training subject: 209889831474100
Adding measurement 633a71bb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=11.428498029708862;dx=-1.4908972206170826E-6
New Minimum: 11.428498029708862 > 11.428494453430176
WOLFE (weak): th(2.154434690031884)=11.428494453430176; dx=-1.4909748831643793E-6 evalInputDelta=3.5762786865234375E-6
New Minimum: 11.428494453430176 > 11.428492069244385
WOLFE (weak): th(4.308869380063768)=11.428492069244385; dx=-1.4908740956647668E-6 evalInputDelta=5.9604644775390625E-6
New Minimum: 11.428492069244385 > 11.42848014831543
WOLFE (weak): th(12.926608140191302)=11.42848014831543; dx=-1.4910176616749732E-6 evalInputDelta=1.7881393432617188E-5
New Minimum: 11.42848014831543 > 11.428422927856445
WOLFE (weak): th(51.70643256076521)=11.428422927856445; dx=-1.4908973564649746E-6 evalInputDelta=7.510185241699219E-5
New Minimum: 11.428422927856445 > 11.428124904632568
WOLFE (weak): th(258.53216280382605)=11.428124904632568; dx=-1.4901138158833444E-6 evalInputDelta=3.731250762939453E-4
New Minimum: 11.428124904632568 > 11.426259279251099
WOLFE (weak): th(1551.1929768229563)=11.426259279251099; dx=-1.490591075438744E-6 evalInputDelta=0.002238750457763672
New Minimum: 11.426259279251099 > 11.412838697433472
WOLFE (weak): th(10858.350837760694)=11.412838697433472; dx=-1.4886176712930254E-6 evalInputDelta=0.015659332275390625
New Minimum: 11.412838697433472 > 11.304113864898682
WOLFE (weak): th(86866.80670208555)=11.304113864898682; dx=-1.47281770607479E-6 evalInputDelta=0.12438416481018066
New Minimum: 11.304113864898682 > 10.395740270614624
END: th(781801.26031877)=10.395740270614624; dx=-1.287550237611198E-6 evalInputDelta=1.0327577590942383
Fitness changed from 11.428498029708862 to 10.395740270614624
Iteration 1 complete. Error: 10.395740270614624 Total: 10.0218; Orientation: 0.0155; Line Search: 7.7097
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 31f8f124 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=10.395740270614624;dx=-1.2579832300010173E-6
New Minimum: 10.395740270614624 > 0.3830705656824378
WOLFE (weak): th(1684339.7559414052)=0.3830705656824378; dx=-5.707987616509958E-6 evalInputDelta=10.012669704932186
New Minimum: 0.3830705656824378 > -13.517498499797107
WOLFE (weak): th(3368679.5118828104)=-13.517498499797107; dx=-3.851707244698074E-6 evalInputDelta=23.91323877041173
New Minimum: -13.517498499797107 > -14.463640361853887
END: th(1.0106038535648432E7)=-14.463640361853887; dx=-6.073728019123104E-7 evalInputDelta=24.85938063246851
Fitness changed from 10.395740270614624 to -14.463640361853887
Iteration 2 complete. Error: -14.463640361853887 Total: 3.9728; Orientation: 0.0161; Line Search: 3.2426
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3915a883 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-14.463640361853887;dx=-0.017491067382372716
Armijo: th(2.17728E7)=168.06475536648088; dx=8.941141119614036E-6 evalInputDelta=-182.52839572833477
Armijo: th(1.08864E7)=86.33546910056052; dx=9.176109407726648E-6 evalInputDelta=-100.7991094624144
Armijo: th(3628800.0)=9.491116753555431; dx=2.5732731617417497E-6 evalInputDelta=-23.95475711540932
New Minimum: -14.463640361853887 > -54.93643742846017
END: th(907200.0)=-54.93643742846017; dx=-4.422338029083319E-5 evalInputDelta=40.472797066606276
Fitness changed from -14.463640361853887 to -54.93643742846017
Iteration 3 complete. Error: -54.93643742846017 Total: 4.5844; Orientation: 0.0161; Line Search: 3.7878
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 244d166 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.524e+07, gradient 1.129e-01, dot -0.005; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -54.93643742846017, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 244d166 to history. Total: 3
th(0)=-54.93643742846017;dx=-0.012683214637799154
Armijo: th(1954503.150796925)=-0.05029318541515693; dx=-2.4107001365401903E-6 evalInputDelta=-54.88614424304501
Armijo: th(977251.5753984625)=-46.30823153087829; dx=-3.398246238982856E-5 evalInputDelta=-8.628205897581879
New Minimum: -54.93643742846017 > -172.65252977126326
END: th(325750.5251328208)=-172.65252977126326; dx=-1.8099644687184392E-4 evalInputDelta=117.7160923428031
Fitness changed from -54.93643742846017 to -172.65252977126326
Iteration 4 complete. Error: -172.65252977126326 Total: 4.2141; Orientation: 0.2951; Line Search: 3.2197
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement c9f832b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.579e+07, gradient 1.200e-01, dot -0.013; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -172.65252977126326, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement c9f832b to history. Total: 3
th(0)=-172.65252977126326;dx=-0.014352936349005604
New Minimum: -172.65252977126326 > -196.08869551111093
END: th(701808.2316422522)=-196.08869551111093; dx=-3.629933593903858E-5 evalInputDelta=23.436165739847667
Fitness changed from -172.65252977126326 to -196.08869551111093
Iteration 5 complete. Error: -196.08869551111093 Total: 2.5564; Orientation: 0.2908; Line Search: 1.5567
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 46d744c1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.782e+07, gradient 1.135e-01, dot -0.009; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -196.08869551111093, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 46d744c1 to history. Total: 3
th(0)=-196.08869551111093;dx=-0.012837956299802581
Armijo: th(1512000.0)=-68.26774677759013; dx=4.953634850800454E-5 evalInputDelta=-127.8209487335208
New Minimum: -196.08869551111093 > -286.7450960066356
END: th(756000.0)=-286.7450960066356; dx=-3.6784987892930964E-6 evalInputDelta=90.65640049552468
Fitness changed from -196.08869551111093 to -286.7450960066356
Iteration 6 complete. Error: -286.7450960066356 Total: 3.3353; Orientation: 0.2958; Line Search: 2.2779
Adding measurement 63b8e20e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.733e+07, gradient 1.220e-01, dot -0.011; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -286.7450960066356, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 63b8e20e to history. Total: 3
th(0)=-286.7450960066356;dx=-0.014769409749336632
Armijo: th(1628752.6256641042)=-63.4150312357319; dx=2.232885520766878E-5 evalInputDelta=-223.3300647709037
Armijo: th(814376.3128320521)=-245.0815053112924; dx=8.874036134736665E-5 evalInputDelta=-41.6635906953432
New Minimum: -286.7450960066356 > -634.6750883759158
END: th(271458.7709440174)=-634.6750883759158; dx=-5.746179078405907E-4 evalInputDelta=347.9299923692802
Fitness changed from -286.7450960066356 to -634.6750883759158
Iteration 7 complete. Error: -634.6750883759158 Total: 4.1224; Orientation: 0.2771; Line Search: 2.9106
Adding measurement b6e6491 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.273e+08, gradient 1.409e-01, dot -0.007; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -634.6750883759158, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement b6e6491 to history. Total: 3
th(0)=-634.6750883759158;dx=-0.019815761636011432
Armijo: th(584840.1930352102)=-439.3990942050792; dx=3.7312911157497803E-4 evalInputDelta=-195.2759941708366
New Minimum: -634.6750883759158 > -898.0269385043771
END: th(292420.0965176051)=-898.0269385043771; dx=-1.6503199422624197E-4 evalInputDelta=263.35185012846125
Fitness changed from -634.6750883759158 to -898.0269385043771
Iteration 8 complete. Error: -898.0269385043771 Total: 3.5282; Orientation: 0.2897; Line Search: 2.4879
Adding measurement 57ebcc75 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.610e+08, gradient 1.628e-01, dot -0.005; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -898.0269385043771, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 57ebcc75 to history. Total: 3
th(0)=-898.0269385043771;dx=-0.026464000523059437
Armijo: th(630000.0000000001)=-349.32913278965765; dx=4.982942301845225E-4 evalInputDelta=-548.6978057147194
New Minimum: -898.0269385043771 > -937.2705253843683
WOLF (strong): th(315000.00000000006)=-937.2705253843683; dx=4.0031805117984724E-4 evalInputDelta=39.2435868799912
New Minimum: -937.2705253843683 > -1515.5989262729586
END: th(105000.00000000001)=-1515.5989262729586; dx=-0.004537150999529666 evalInputDelta=617.5719877685815
Fitness changed from -898.0269385043771 to -1515.5989262729586
Iteration 9 complete. Error: -1515.5989262729586 Total: 3.9880; Orientation: 0.2894; Line Search: 3.0271
Adding measurement 7f752524 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.429e+09, gradient 1.637e-01, dot -0.007; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1515.5989262729586, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 7f752524 to history. Total: 3
th(0)=-1515.5989262729586;dx=-0.02675961846420711
New Minimum: -1515.5989262729586 > -1540.8283854376882
END: th(226215.64245334783)=-1540.8283854376882; dx=-1.313721365620747E-4 evalInputDelta=25.229459164729633
Fitness changed from -1515.5989262729586 to -1540.8283854376882
Iteration 10 complete. Error: -1540.8283854376882 Total: 2.6275; Orientation: 0.3069; Line Search: 1.5831
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 15b26370 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.595e+09, gradient 1.962e-01, dot -0.001; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1540.8283854376882, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 15b26370 to history. Total: 3
th(0)=-1540.8283854376882;dx=-0.03842851986193306
Armijo: th(487366.82752934186)=-504.9793888743525; dx=0.0012174994656779803 evalInputDelta=-1035.8489965633357
Armijo: th(243683.41376467093)=-1345.040424560612; dx=0.001173967973898655 evalInputDelta=-195.7879608770761
New Minimum: -1540.8283854376882 > -2138.3451757168723
END: th(81227.80458822365)=-2138.3451757168723; dx=-0.005893204636353046 evalInputDelta=597.5167902791841
Fitness changed from -1540.8283854376882 to -2138.3451757168723
Iteration 11 complete. Error: -2138.3451757168723 Total: 4.2309; Orientation: 0.3139; Line Search: 2.9183
Adding measurement 766bc8f5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.119e+09, gradient 1.797e-01, dot -0.003; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2138.3451757168723, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 766bc8f5 to history. Total: 3
th(0)=-2138.3451757168723;dx=-0.03222773753753536
Armijo: th(175000.00000000006)=-2083.5000761169513; dx=-5.915606576682786E-4 evalInputDelta=-54.845099599921014
New Minimum: -2138.3451757168723 > -2527.2953314823026
END: th(87500.00000000003)=-2527.2953314823026; dx=-0.005585047828156659 evalInputDelta=388.9501557654303
Fitness changed from -2138.3451757168723 to -2527.2953314823026
Iteration 12 complete. Error: -2527.2953314823026 Total: 3.4867; Orientation: 0.3066; Line Search: 2.4478
Adding measurement 704ec01f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.395e+08, gradient 1.871e-01, dot -0.004; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2527.2953314823026, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 704ec01f to history. Total: 3
th(0)=-2527.2953314823026;dx=-0.03498393674634648
Armijo: th(188513.0353777899)=-2107.313623864152; dx=3.293994696892728E-4 evalInputDelta=-419.9817076181507
New Minimum: -2527.2953314823026 > -2729.06488000443
END: th(94256.51768889495)=-2729.06488000443; dx=-0.004350696410786329 evalInputDelta=201.76954852212748
Fitness changed from -2527.2953314823026 to -2729.06488000443
Iteration 13 complete. Error: -2729.06488000443 Total: 3.1937; Orientation: 0.2799; Line Search: 2.2465
Adding measurement 4612af20 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.845e+08, gradient 2.032e-01, dot -0.002; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2729.06488000443, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 4612af20 to history. Total: 3
th(0)=-2729.06488000443;dx=-0.04123502887066102
Armijo: th(203069.51147055917)=-2008.445476464235; dx=0.0014939553233411732 evalInputDelta=-720.619403540195
New Minimum: -2729.06488000443 > -2834.24350529958
END: th(101534.75573527958)=-2834.24350529958; dx=-0.002701341739448271 evalInputDelta=105.17862529515014
Fitness changed from -2729.06488000443 to -2834.24350529958
Iteration 14 complete. Error: -2834.24350529958 Total: 3.5389; Orientation: 0.2896; Line Search: 2.5820
Adding measurement 3e6c57b6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.470e+07, gradient 2.172e-01, dot -0.001; [05f27024-5f34-412a-ac9b-48a5351ca236 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2834.24350529958, -14.463640361853887, 10.395740270614624, 11.428498029708862
LBFGS Accumulation History: 3 points
Removed measurement 3e6c57b6 to history. Total: 3
th(0)=-2834.24350529958;dx=-0.047161715108113775
Armijo: th(218750.00000000012)=-1864.6476191881204; dx=0.0026108020763729624 evalInputDelta=-969.5958861114598
New Minimum: -2834.24350529958 > -2864.2132292865595
END: th(109375.00000000006)=-2864.2132292865595; dx=-7.891524019002698E-4 evalInputDelta=29.96972398697926
Fitness changed from -2834.24350529958 to -2864.2132292865595
Iteration 15 complete. Error: -2864.2132292865595 Total: 3.1996; Orientation: 0.2811; Line Search: 2.2546
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2864.2132292865595 (> -Infinity) after 60.626s (< 5400.000s)
Returns
-2864.2132292865595