BasicOptimizer.scala:89 executed in 148.61 seconds (1.515 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 11590978930723
Reset training subject: 11592147803926
Adding measurement 1cf43531 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.013301416143348245;dx=-3.4411370648785897E-6
New Minimum: -0.013301416143348245 > -0.013309149110734676
WOLFE (weak): th(2.154434690031884)=-0.013309149110734676; dx=-3.439729678575919E-6 evalInputDelta=7.732967386431255E-6
New Minimum: -0.013309149110734676 > -0.013316975934113386
WOLFE (weak): th(4.308869380063768)=-0.013316975934113386; dx=-3.850004673703048E-6 evalInputDelta=1.5559790765140927E-5
New Minimum: -0.013316975934113386 > -0.01334854258015551
WOLFE (weak): th(12.926608140191302)=-0.01334854258015551; dx=-3.783347410226221E-6 evalInputDelta=4.712643680726547E-5
New Minimum: -0.01334854258015551 > -0.01349016167287786
WOLFE (weak): th(51.70643256076521)=-0.01349016167287786; dx=-3.576576995010428E-6 evalInputDelta=1.8874552952961518E-4
New Minimum: -0.01349016167287786 > -0.014240115696564632
WOLFE (weak): th(258.53216280382605)=-0.014240115696564632; dx=-3.647887871484301E-6 evalInputDelta=9.386995532163871E-4
New Minimum: -0.014240115696564632 > -0.01892871541395805
WOLFE (weak): th(1551.1929768229563)=-0.01892871541395805; dx=-3.5035043828744828E-6 evalInputDelta=0.005627299270609806
New Minimum: -0.01892871541395805 > -0.0472977666887374
END: th(10858.350837760694)=-0.0472977666887374; dx=-2.686484924818429E-6 evalInputDelta=0.03399635054538915
Fitness changed from -0.013301416143348245 to -0.0472977666887374
Iteration 1 complete. Error: -0.0472977666887374 Total: 13.1469; Orientation: 0.0170; Line Search: 9.6501
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4d4e8d76 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.0472977666887374;dx=-2.8541937121822702E-6
New Minimum: -0.0472977666887374 > -0.12241098930659
END: th(23393.607721408407)=-0.12241098930659; dx=-2.32361733498718E-6 evalInputDelta=0.07511322261785261
Fitness changed from -0.0472977666887374 to -0.12241098930659
Iteration 2 complete. Error: -0.12241098930659 Total: 3.7043; Orientation: 0.0177; Line Search: 2.3771
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 724a05b4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.12241098930659;dx=-3.992148498486877E-6
New Minimum: -0.12241098930659 > -0.3522311633409138
WOLFE (weak): th(50400.00000000001)=-0.3522311633409138; dx=-4.107730855671691E-6 evalInputDelta=0.2298201740343238
New Minimum: -0.3522311633409138 > -0.498573234516188
END: th(100800.00000000001)=-0.498573234516188; dx=-1.9477976587747035E-6 evalInputDelta=0.376162245209598
Fitness changed from -0.12241098930659 to -0.498573234516188
Iteration 3 complete. Error: -0.498573234516188 Total: 5.5281; Orientation: 0.1034; Line Search: 4.1176
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 158f2880 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.285e+03, gradient 5.397e-03, dot -0.223; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.498573234516188, -0.12241098930659, -0.0472977666887374, -0.013301416143348245
LBFGS Accumulation History: 3 points
Removed measurement 158f2880 to history. Total: 3
th(0)=-0.498573234516188;dx=-2.7160301427404017E-5
New Minimum: -0.498573234516188 > -0.7104203995040715
END: th(217167.01675521393)=-0.7104203995040715; dx=-1.425346069715058E-7 evalInputDelta=0.21184716498788353
Fitness changed from -0.498573234516188 to -0.7104203995040715
Iteration 4 complete. Error: -0.7104203995040715 Total: 4.1838; Orientation: 0.4505; Line Search: 2.4240
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6a05d9bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.279e+03, gradient 1.914e-02, dot -0.027; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.7104203995040715, -0.12241098930659, -0.0472977666887374, -0.013301416143348245
LBFGS Accumulation History: 3 points
Removed measurement 6a05d9bb to history. Total: 3
th(0)=-0.7104203995040715;dx=-3.5812222415084486E-4
New Minimum: -0.7104203995040715 > -1.9916502583464937
WOLF (strong): th(467872.15442816826)=-1.9916502583464937; dx=3.96868241591487E-6 evalInputDelta=1.2812298588424222
New Minimum: -1.9916502583464937 > -2.9967837577351157
WOLF (strong): th(233936.07721408413)=-2.9967837577351157; dx=6.3538930507029495E-6 evalInputDelta=2.286363358231044
New Minimum: -2.9967837577351157 > -3.761752214609715
END: th(77978.69240469471)=-3.761752214609715; dx=-6.356339586627767E-6 evalInputDelta=3.051331815105643
Fitness changed from -0.7104203995040715 to -3.761752214609715
Iteration 5 complete. Error: -3.761752214609715 Total: 6.6635; Orientation: 0.4982; Line Search: 4.8474
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6059f6c9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.482e+04, gradient 2.550e-02, dot -0.022; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.761752214609715, -0.12241098930659, -0.0472977666887374, -0.013301416143348245
LBFGS Accumulation History: 3 points
Removed measurement 6059f6c9 to history. Total: 3
th(0)=-3.761752214609715;dx=-6.279048656418226E-4
Armijo: th(168000.00000000006)=-2.023121930704272; dx=3.022888788842568E-6 evalInputDelta=-1.7386302839054428
New Minimum: -3.761752214609715 > -4.29363802967917
WOLF (strong): th(84000.00000000003)=-4.29363802967917; dx=2.297684562050423E-5 evalInputDelta=0.531885815069455
New Minimum: -4.29363802967917 > -8.584477863730042
END: th(28000.00000000001)=-8.584477863730042; dx=-1.614997593223592E-5 evalInputDelta=4.822725649120327
Fitness changed from -3.761752214609715 to -8.584477863730042
Iteration 6 complete. Error: -8.584477863730042 Total: 6.5987; Orientation: 0.4665; Line Search: 4.8256
Adding measurement 7722fa0f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.252e+04, gradient 3.142e-02, dot -0.020; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.584477863730042, -0.12241098930659, -0.0472977666887374, -0.013301416143348245
LBFGS Accumulation History: 3 points
Removed measurement 7722fa0f to history. Total: 3
th(0)=-8.584477863730042;dx=-9.711877918034191E-4
New Minimum: -8.584477863730042 > -10.262640625225998
WOLF (strong): th(60324.17132089277)=-10.262640625225998; dx=6.103860944760188E-6 evalInputDelta=1.6781627614959564
New Minimum: -10.262640625225998 > -12.922026147846358
END: th(30162.085660446384)=-12.922026147846358; dx=-1.350360239301028E-6 evalInputDelta=4.337548284116316
Fitness changed from -8.584477863730042 to -12.922026147846358
Iteration 7 complete. Error: -12.922026147846358 Total: 5.2221; Orientation: 0.4490; Line Search: 3.6363
Adding measurement 56f45f51 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.251e+03, gradient 3.199e-02, dot 0.112; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-12.922026147846358;dx=-7.778619285572191
Armijo: th(64982.24367057894)=-2.0814654814373066; dx=6.1023323685069E-6 evalInputDelta=-10.84056066640905
Armijo: th(32491.12183528947)=-2.4185210600151525; dx=-3.527874831434037E-6 evalInputDelta=-10.503505087831206
Armijo: th(10830.37394509649)=-2.276456630656461; dx=3.634568012655882E-5 evalInputDelta=-10.645569517189898
Armijo: th(2707.5934862741224)=-2.659562681861896; dx=-1.7842227778411306E-4 evalInputDelta=-10.262463465984462
Armijo: th(541.5186972548245)=-2.8370806279092267; dx=-0.0018949312163953958 evalInputDelta=-10.084945519937131
Armijo: th(90.25311620913742)=-2.956846445575076; dx=-0.0083689218470884 evalInputDelta=-9.965179702271282
Armijo: th(12.8933023155
...skipping 2571 bytes...
asurement 51859f5 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.915e+03, gradient 2.612e-02, dot 0.079; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
th(0)=-26.366606475079777;dx=-13.684847962502202
Armijo: th(37702.607075557986)=-2.039172290170157; dx=9.30936086850724E-6 evalInputDelta=-24.32743418490962
Armijo: th(18851.303537778993)=-2.004774355474857; dx=3.1173030418273655E-5 evalInputDelta=-24.36183211960492
Armijo: th(6283.767845926331)=-2.1594048239915957; dx=1.5052223368884453E-4 evalInputDelta=-24.20720165108818
Armijo: th(1570.9419614815827)=-2.5702738193743637; dx=-5.318064057399121E-4 evalInputDelta=-23.796332655705413
Armijo: th(314.1883922963165)=-2.381179800525706; dx=0.0011264423660723093 evalInputDelta=-23.98542667455407
Armijo: th(52.364732049386085)=-4.155804373578607; dx=0.010010269994833478 evalInputDelta=-22.21080210150117
Armijo: th(7.480676007055155)=-11.333337297625876; dx=0.3043319532771611 evalInputDelta=-15.033269177453901
New Minimum: -26.366606475079777 > -26.518061402870426
END: th(0.9350845008818943)=-26.518061402870426; dx=-1.159553055342704 evalInputDelta=0.15145492779064895
Fitness changed from -26.366606475079777 to -26.518061402870426
Iteration 10 complete. Error: -26.518061402870426 Total: 12.6096; Orientation: 0.4469; Line Search: 10.9956
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement a76c35 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.139e+03, gradient 2.574e-02, dot -0.012; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.518061402870426, -26.366606475079777, -0.12241098930659, -0.0472977666887374, -0.013301416143348245
Accepted: LBFGS Orientation magnitude: 3.155e+03, gradient 2.574e-02, dot 0.004; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-26.518061402870426;dx=-0.4121525836052322
Armijo: th(2.0145784868111027)=-19.718198257683103; dx=1.7897975655127119 evalInputDelta=-6.799863145187324
Armijo: th(1.0072892434055514)=-22.981433838416855; dx=2.127364725304096 evalInputDelta=-3.536627564453571
Armijo: th(0.33576308113518377)=-25.302756612422687; dx=2.501322650107708 evalInputDelta=-1.215304790447739
Armijo: th(0.08394077028379594)=-26.211641896272685; dx=2.579190236225455 evalInputDelta=-0.30641950659774153
Armijo: th(0.016788154056759187)=-26.456793344290613; dx=2.714018675642773 evalInputDelta=-0.0612680585798131
Armijo: th(0.002798025676126531)=-26.50785394559094; dx=2.5553806241305703 evalInputDelta=-0.0102074572794848
Armijo: th(3.997179537323616E-4)=-26.516610176158924; dx=2.6036130645977984 evalInputDelta=-0.0014512267115023292
Armijo: th(4.99647442165452E-5)=-26.517891080613506; dx=2.5938980649387195 evalInputDelta=-1.7032225692048542E-4
Armijo: th(5.5516382462828E-6)=-26.51803838641315; dx=2.53995576602765 evalInputDelta=-2.3016457276980873E-5
Armijo: th(5.5516382462828E-7)=-26.518057950403108; dx=2.6453240627380223 evalInputDelta=-3.4524673182545484E-6
Armijo: th(5.046943860257091E-8)=-26.51805679958057; dx=2.688163615301555 evalInputDelta=-4.603289855964476E-6
New Minimum: -26.518061402870426 > -26.518062553692765
WOLF (strong): th(4.205786550214243E-9)=-26.518062553692765; dx=2.636354375093098 evalInputDelta=1.150822338757962E-6
Armijo: th(3.235220423241725E-10)=-26.518059101225596; dx=2.587349560918661 evalInputDelta=-2.301644830282612E-6
MIN ALPHA (2.3108717308869464E-11): th(4.205786550214243E-9)=-26.518062553692765
Fitness changed from -26.518061402870426 to -26.518062553692765
Iteration 11 complete. Error: -26.518062553692765 Total: 20.9398; Orientation: 1.0049; Line Search: 18.6452
Adding measurement 59ed7006 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.283e+06, gradient 2.559e-02, dot -0.121; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.518062553692765, -26.518061402870426, -26.366606475079777, -0.12241098930659, -0.0472977666887374
Accepted: LBFGS Orientation magnitude: 5.039e+06, gradient 2.559e-02, dot 0.095; [fee70f2d-eef3-4cc6-bca9-911ed5e5d74a = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-26.518062553692765;dx=-12355.777087861286
Armijo: th(2.1544346900318838E-10)=-26.518061402870426; dx=-11281.34655477396 evalInputDelta=-1.150822338757962E-6
Armijo: th(1.0772173450159419E-10)=-26.518061402870426; dx=-11152.997461248078 evalInputDelta=-1.150822338757962E-6
MIN ALPHA (3.590724483386473E-11): th(0.0)=-26.518062553692765
Fitness changed from -26.518062553692765 to -26.518062553692765
Static Iteration Total: 6.9865; Orientation: 1.0020; Line Search: 4.8323
Iteration 12 failed. Error: -26.518062553692765
Previous Error: 0.0 -> -26.518062553692765
Retrying iteration 12
Reset training subject: 11701533675166
Adding measurement 461990f5 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-26.518062553692765;dx=-6.566522771736496E-4
WOLFE (weak): th(2.1544346900318838E-10)=-26.518062553692765; dx=-6.323985577349038E-4 evalInputDelta=0.0
WOLFE (weak): th(4.3088693800637676E-10)=-26.518062553692765; dx=-6.266538611885855E-4 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191302E-9)=-26.518062553692765; dx=-6.268960451191704E-4 evalInputDelta=0.0
WOLFE (weak): th(5.170643256076521E-9)=-26.518062553692765; dx=-6.268493776929825E-4 evalInputDelta=0.0
WOLFE (weak): th(2.5853216280382606E-8)=-26.518062553692765; dx=-6.315511453287507E-4 evalInputDelta=0.0
WOLFE (weak): th(1.5511929768229565E-7)=-26.518062553692765; dx=-6.32289691586416E-4 evalInputDelta=0.0
WOLFE (weak): th(1.0858350837760695E-6)=-26.518062553692765; dx=-6.19125965524841E-4 evalInputDelta=0.0
Armijo: th(8.686680670208556E-6)=-26.518062553692765; dx=-6.326745200930519E-4 evalInputDelta=0.0
Armijo: th(4.886257876992313E-6)=-26.518062553692765; dx=-6.260925693918182E-4 evalInputDelta=0.0
Armijo: th(2.986046480384191E-6)=-26.518062553692765; dx=-6.274964059105241E-4 evalInputDelta=0.0
WOLFE (weak): th(2.0359407820801304E-6)=-26.518062553692765; dx=-6.272882438437539E-4 evalInputDelta=0.0
WOLFE (weak): th(2.5109936312321608E-6)=-26.518062553692765; dx=-6.38000641890158E-4 evalInputDelta=0.0
Armijo: th(2.7485200558081758E-6)=-26.518062553692765; dx=-6.249145605052972E-4 evalInputDelta=0.0
WOLFE (weak): th(2.629756843520168E-6)=-26.518062553692765; dx=-6.325851609907843E-4 evalInputDelta=0.0
WOLFE (weak): th(2.689138449664172E-6)=-26.518062553692765; dx=-6.328980563732943E-4 evalInputDelta=0.0
Armijo: th(2.718829252736174E-6)=-26.518062553692765; dx=-6.324744235590804E-4 evalInputDelta=0.0
WOLFE (weak): th(2.703983851200173E-6)=-26.518062553692765; dx=-6.218566694664806E-4 evalInputDelta=0.0
Armijo: th(2.7114065519681736E-6)=-26.518062553692765; dx=-6.325590028940708E-4 evalInputDelta=0.0
Armijo: th(2.7076952015841734E-6)=-26.518062553692765; dx=-6.315391794087407E-4 evalInputDelta=0.0
Armijo: th(2.7058395263921733E-6)=-26.518062553692765; dx=-6.370391617959985E-4 evalInputDelta=0.0
WOLFE (weak): th(2.704911688796173E-6)=-26.518062553692765; dx=-6.364319388056798E-4 evalInputDelta=0.0
Armijo: th(2.705375607594173E-6)=-26.518062553692765; dx=-6.352136323890326E-4 evalInputDelta=0.0
WOLFE (weak): th(2.705143648195173E-6)=-26.518062553692765; dx=-6.292176608562352E-4 evalInputDelta=0.0
Armijo: th(2.705259627894673E-6)=-26.518062553692765; dx=-6.194527349859372E-4 evalInputDelta=0.0
Armijo: th(2.705201638044923E-6)=-26.518062553692765; dx=-6.334652868396899E-4 evalInputDelta=0.0
Armijo: th(2.705172643120048E-6)=-26.518062553692765; dx=-6.355076835510914E-4 evalInputDelta=0.0
WOLFE (weak): th(2.7051581456576106E-6)=-26.518062553692765; dx=-6.312650977283543E-4 evalInputDelta=0.0
mu ~= nu (2.7051581456576106E-6): th(0.0)=-26.518062553692765
Fitness changed from -26.518062553692765 to -26.518062553692765
Static Iteration Total: 38.0500; Orientation: 0.0166; Line Search: 35.7275
Iteration 13 failed. Error: -26.518062553692765
Previous Error: 0.0 -> -26.518062553692765
Optimization terminated 13
Final threshold in iteration 13: -26.518062553692765 (> -Infinity) after 148.605s (< 3600.000s)
Returns
-26.518062553692765