BasicOptimizer.scala:89 executed in 64.80 seconds (2.289 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 156980130415000
Reset training subject: 156981088779900
Adding measurement 48544b2c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-42.473922088938146;dx=-0.014995141785342048
New Minimum: -42.473922088938146 > -42.519051623672915
WOLFE (weak): th(2.154434690031884)=-42.519051623672915; dx=-0.014809356209555764 evalInputDelta=0.04512953473476955
New Minimum: -42.519051623672915 > -42.56422938391901
WOLFE (weak): th(4.308869380063768)=-42.56422938391901; dx=-0.014887951317703265 evalInputDelta=0.09030729498086743
New Minimum: -42.56422938391901 > -42.74487416729782
WOLFE (weak): th(12.926608140191302)=-42.74487416729782; dx=-0.01495563060116155 evalInputDelta=0.2709520783596773
New Minimum: -42.74487416729782 > -43.556182009112526
WOLFE (weak): th(51.70643256076521)=-43.556182009112526; dx=-0.014732031285119098 evalInputDelta=1.0822599201743799
New Minimum: -43.556182009112526 > -47.90696856114724
WOLFE (weak): th(258.53216280382605)=-47.90696856114724; dx=-0.014909483336653496 evalInputDelta=5.433046472209092
New Minimum: -47.90696856114724 > -73.30185278077049
END: th(1551.1929768229563)=-73.30185278077049; dx=-0.013049224021746431 evalInputDelta=30.827930691832343
Fitness changed from -42.473922088938146 to -73.30185278077049
Iteration 1 complete. Error: -73.30185278077049 Total: 9.5690; Orientation: 0.0164; Line Search: 6.8528
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 746392ce to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-73.30185278077049;dx=-0.01861324877427584
New Minimum: -73.30185278077049 > -143.41541905204963
END: th(3341.943960201201)=-143.41541905204963; dx=-0.010305271928766221 evalInputDelta=70.11356627127914
Fitness changed from -73.30185278077049 to -143.41541905204963
Iteration 2 complete. Error: -143.41541905204963 Total: 2.5804; Orientation: 0.0145; Line Search: 1.6745
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6f3d979d to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-143.41541905204963;dx=-0.019506406841578144
New Minimum: -143.41541905204963 > -252.8584167202073
END: th(7200.000000000001)=-252.8584167202073; dx=-0.006770955452980144 evalInputDelta=109.44299766815766
Fitness changed from -143.41541905204963 to -252.8584167202073
Iteration 3 complete. Error: -252.8584167202073 Total: 3.7750; Orientation: 0.0149; Line Search: 2.8568
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 74e6b0f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.397e+04, gradient 1.480e-01, dot -0.300; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -252.8584167202073, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 74e6b0f9 to history. Total: 3
th(0)=-252.8584167202073;dx=-0.02176301451159317
New Minimum: -252.8584167202073 > -336.5871915269504
END: th(15511.929768229566)=-336.5871915269504; dx=-0.002733255121074249 evalInputDelta=83.7287748067431
Fitness changed from -252.8584167202073 to -336.5871915269504
Iteration 4 complete. Error: -336.5871915269504 Total: 2.8890; Orientation: 0.2774; Line Search: 1.6858
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3f9c720 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.805e+04, gradient 1.837e-01, dot -0.210; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -336.5871915269504, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 3f9c720 to history. Total: 3
th(0)=-336.5871915269504;dx=-0.03356687549266739
New Minimum: -336.5871915269504 > -337.7496761656479
WOLF (strong): th(33419.43960201202)=-337.7496761656479; dx=0.0018180523411680197 evalInputDelta=1.1624846386974923
New Minimum: -337.7496761656479 > -494.1227548646186
END: th(16709.71980100601)=-494.1227548646186; dx=-8.8651673427935E-4 evalInputDelta=157.53556333766824
Fitness changed from -336.5871915269504 to -494.1227548646186
Iteration 5 complete. Error: -494.1227548646186 Total: 3.7865; Orientation: 0.2770; Line Search: 2.6083
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 385de356 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.613e+04, gradient 1.636e-01, dot -0.212; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -494.1227548646186, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 385de356 to history. Total: 3
th(0)=-494.1227548646186;dx=-0.026731912034088104
Armijo: th(36000.000000000015)=-485.64407941928005; dx=0.003194915011022888 evalInputDelta=-8.478675445338581
New Minimum: -494.1227548646186 > -665.1080034475603
END: th(18000.000000000007)=-665.1080034475603; dx=-0.0016044972207623447 evalInputDelta=170.98524858294172
Fitness changed from -494.1227548646186 to -665.1080034475603
Iteration 6 complete. Error: -665.1080034475603 Total: 4.0196; Orientation: 0.4264; Line Search: 2.6732
Adding measurement 726ddc27 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.860e+04, gradient 1.621e-01, dot -0.216; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -665.1080034475603, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 726ddc27 to history. Total: 3
th(0)=-665.1080034475603;dx=-0.02580500587568179
Armijo: th(38779.824420573925)=-624.8374538630027; dx=0.003111828460235366 evalInputDelta=-40.27054958455767
New Minimum: -665.1080034475603 > -799.5094198944751
END: th(19389.912210286962)=-799.5094198944751; dx=-0.0018820032133263457 evalInputDelta=134.4014164469147
Fitness changed from -665.1080034475603 to -799.5094198944751
Iteration 7 complete. Error: -799.5094198944751 Total: 3.6956; Orientation: 0.2730; Line Search: 2.6011
Adding measurement 4a449f40 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.019e+05, gradient 1.576e-01, dot -0.250; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -799.5094198944751, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 4a449f40 to history. Total: 3
th(0)=-799.5094198944751;dx=-0.024816523373795328
Armijo: th(41774.29950251503)=-749.9413727085605; dx=0.002378495345449149 evalInputDelta=-49.568047185914565
New Minimum: -799.5094198944751 > -919.0080930002739
END: th(20887.149751257515)=-919.0080930002739; dx=-0.0017217214450046302 evalInputDelta=119.49867310579884
Fitness changed from -799.5094198944751 to -919.0080930002739
Iteration 8 complete. Error: -919.0080930002739 Total: 3.7760; Orientation: 0.2707; Line Search: 2.6806
Adding measurement 5f9979f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.299e+06, gradient 1.583e-01, dot -0.258; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -919.0080930002739, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 5f9979f to history. Total: 3
th(0)=-919.0080930002739;dx=-0.02498170647695922
Armijo: th(45000.00000000002)=-903.4577328242866; dx=0.0022895209453964085 evalInputDelta=-15.550360175987294
New Minimum: -919.0080930002739 > -1056.2857686251386
END: th(22500.00000000001)=-1056.2857686251386; dx=-0.0015316865843387198 evalInputDelta=137.27767562486474
Fitness changed from -919.0080930002739 to -1056.2857686251386
Iteration 9 complete. Error: -1056.2857686251386 Total: 3.7246; Orientation: 0.2976; Line Search: 2.6011
Adding measurement 11df409e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.653e+05, gradient 1.635e-01, dot 0.284; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1056.2857686251386;dx=-7623.953345054199
Armijo: th(48474.78052571741)=-110.67015762644644; dx=0.0013452761499036718 evalInputDelta=-945.6156109986922
Armijo: th(24237.390262858706)=-110.5733548232801; dx=0.00275208506672597 evalInputDelta=-945.7124138018586
Armijo: th(8079.130087619568)=-110.34912193097641; dx=0.00841491643876123 evalInputDelta=-945.9366466941623
Armijo: th(2019.782521904892)=-109.1714132039038; dx=0.033295444882168665 evalInputDelta=-947.1143554212349
Armijo: th(403.9565043809784)=-111.07051884388584; dx=0.17224490704316814 evalInputDelta=-945.2152497812529
Armijo: th(67.3260840634964)=-116.02013985322763; dx=1.1050458662331866 evalInputDelta=-940.265628771911
Armijo: th(9.618012009070913)=-124.15046865472357; dx=8.232385093629077 evalInputDelta=-932.1352999704151
Armijo: th(1.2022515011338641)=-220.12754647874036; dx=-3.2053702098104324 evalInputDelta=-836.1582221463983
Armijo: th(0.1335835001259849)=-490.1622977392284; dx=1470.8654477341117 evalInputDelta=-566.1234708859103
Armijo: th(0.013358350012598491)=-1053.5797312021525; dx=485.1367963725019 evalInputDelta=-2.706037422986128
New Minimum: -1056.2857686251386 > -1060.5883390835484
END: th(0.0012143954556907718)=-1060.5883390835484; dx=-2850.045595061003 evalInputDelta=4.302570458409718
Fitness changed from -1056.2857686251386 to -1060.5883390835484
Iteration 10 complete. Error: -1060.5883390835484 Total: 11.5721; Orientation: 0.2678; Line Search: 10.4968
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 26977534 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 6.245e+05, gradient 1.630e-01, dot -0.271; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1060.5883390835484, -1056.2857686251386, -143.41541905204963, -73.30185278077049, -42.473922088938146
Rejected: LBFGS Orientation magnitude: 7.513e+05, gradient 1.630e-01, dot -0.325; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1060.5883390835484, -1056.2857686251386, -143.41541905204963, -73.30185278077049
LBFGS Accumulation History: 3 points
Removed measurement 26977534 to history. Total: 4
Removed measurement 11df409e to history. Total: 3
th(0)=-1060.5883390835484;dx=-0.0265540148027764
New Minimum: -1060.5883390835484 > -1060.588398013103
END: th(0.0026163356971572764)=-1060.588398013103; dx=-0.017800348494527073 evalInputDelta=5.8929554597852984E-5
Fitness changed from -1060.5883390835484 to -1060.588398013103
Iteration 11 complete. Error: -1060.588398013103 Total: 3.3969; Orientation: 0.5955; Line Search: 1.8634
Adding measurement 5a16c8a6 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.795e+05, gradient 1.629e-01, dot 0.274; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
th(0)=-1060.588398013103;dx=-8027.051590499861
New Minimum: -1060.588398013103 > -1067.8580195887466
END: th(0.0056367243867243896)=-1067.8580195887466; dx=-1085.198179763659 evalInputDelta=7.269621575643669
Fitness changed from -1060.588398013103 to -1067.8580195887466
Iteration 12 complete. Error: -1067.8580195887466 Total: 2.8377; Orientation: 0.2788; Line Search: 1.6984
Adding measurement 46e6b6bf to history. Total: 4
Rejected: LBFGS Orientation magnitude: 5.501e+05, gradient 1.630e-01, dot -0.202; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1067.8580195887466, -1060.588398013103, -143.41541905204963, -73.30185278077049, -42.473922088938146
Rejected: LBFGS Orientation magnitude: 5.034e+05, gradient 1.630e-01, dot -0.284; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1067.8580195887466, -1060.588398013103, -143.41541905204963, -73.30185278077049
LBFGS Accumulation History: 3 points
Removed measurement 46e6b6bf to history. Total: 4
Removed measurement 5a16c8a6 to history. Total: 3
th(0)=-1067.8580195887466;dx=-0.026470514215267596
New Minimum: -1067.8580195887466 > -1067.8583122306068
END: th(0.01214395455690772)=-1067.8583122306068; dx=-0.017552111077262796 evalInputDelta=2.9264186014188454E-4
Fitness changed from -1067.8580195887466 to -1067.8583122306068
Iteration 13 complete. Error: -1067.8583122306068 Total: 3.0901; Orientation: 0.5977; Line Search: 1.6966
Adding measurement 2470195a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.617e+05, gradient 1.628e-01, dot -0.207; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1067.8583122306068, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 2470195a to history. Total: 3
th(0)=-1067.8583122306068;dx=-0.02642455079907568
New Minimum: -1067.8583122306068 > -1067.8588389857528
END: th(0.026163356971572765)=-1067.8588389857528; dx=-0.017582665989594994 evalInputDelta=5.267551459837705E-4
Fitness changed from -1067.8583122306068 to -1067.8588389857528
Iteration 14 complete. Error: -1067.8588389857528 Total: 2.9456; Orientation: 0.3058; Line Search: 1.7965
Adding measurement 7dcc39dc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.279e+05, gradient 1.643e-01, dot -0.203; [a98ffd29-bcf3-4463-8268-9409b4885711 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1067.8588389857528, -143.41541905204963, -73.30185278077049, -42.473922088938146
LBFGS Accumulation History: 3 points
Removed measurement 7dcc39dc to history. Total: 3
th(0)=-1067.8588389857528;dx=-0.026428170087887323
New Minimum: -1067.8588389857528 > -1067.8599510235413
END: th(0.0563672438672439)=-1067.8599510235413; dx=-0.017450949825947238 evalInputDelta=0.001112037788516318
Fitness changed from -1067.8588389857528 to -1067.8599510235413
Iteration 15 complete. Error: -1067.8599510235413 Total: 3.0320; Orientation: 0.4308; Line Search: 1.8098
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1067.8599510235413 (> -Infinity) after 64.798s (< 5400.000s)
Returns
-1067.8599510235413