BasicOptimizer.scala:89 executed in 240.74 seconds (2.093 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2973226018781
Reset training subject: 2974577357888
Adding measurement 1153d7e7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.7028498331714668;dx=-1.8837997676781978E-5
New Minimum: -1.7028498331714668 > -1.7029050688607335
WOLFE (weak): th(2.154434690031884)=-1.7029050688607335; dx=-1.8791883782024045E-5 evalInputDelta=5.5235689266686094E-5
New Minimum: -1.7029050688607335 > -1.702959949606202
WOLFE (weak): th(4.308869380063768)=-1.702959949606202; dx=-1.8817467636063112E-5 evalInputDelta=1.1011643473524124E-4
New Minimum: -1.702959949606202 > -1.7031805848453243
WOLFE (weak): th(12.926608140191302)=-1.7031805848453243; dx=-1.878792164297993E-5 evalInputDelta=3.3075167385754334E-4
New Minimum: -1.7031805848453243 > -1.704169702076398
WOLFE (weak): th(51.70643256076521)=-1.704169702076398; dx=-1.8780249844320105E-5 evalInputDelta=0.0013198689049311874
New Minimum: -1.704169702076398 > -1.709410911964818
WOLFE (weak): th(258.53216280382605)=-1.709410911964818; dx=-1.864811007625721E-5 evalInputDelta=0.006561078793351216
New Minimum: -1.709410911964818 > -1.7417147065737286
WOLFE (weak): th(1551.1929768229563)=-1.7417147065737286; dx=-1.74932249717371E-5 evalInputDelta=0.03886487340226186
New Minimum: -1.7417147065737286 > -1.9225423456109907
END: th(10858.350837760694)=-1.9225423456109907; dx=-1.1813565741585464E-5 evalInputDelta=0.21969251243952392
Fitness changed from -1.7028498331714668 to -1.9225423456109907
Iteration 1 complete. Error: -1.9225423456109907 Total: 15.1942; Orientation: 0.0167; Line Search: 11.1259
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7c6d476d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.9225423456109907;dx=-1.2431291306290896E-5
New Minimum: -1.9225423456109907 > -2.187016096926548
END: th(23393.607721408407)=-2.187016096926548; dx=-8.187497505785824E-6 evalInputDelta=0.2644737513155573
Fitness changed from -1.9225423456109907 to -2.187016096926548
Iteration 2 complete. Error: -2.187016096926548 Total: 4.2628; Orientation: 0.0166; Line Search: 2.7788
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5d6155f1 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.187016096926548;dx=-1.1276365573104544E-5
New Minimum: -2.187016096926548 > -2.588032116441949
END: th(50400.00000000001)=-2.588032116441949; dx=-4.30770875460223E-6 evalInputDelta=0.4010160195154011
Fitness changed from -2.187016096926548 to -2.588032116441949
Iteration 3 complete. Error: -2.588032116441949 Total: 4.2741; Orientation: 0.0164; Line Search: 2.7875
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 705a6798 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.865e+02, gradient 3.633e-03, dot -0.456; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.588032116441949, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 705a6798 to history. Total: 3
th(0)=-2.588032116441949;dx=-1.283160077254108E-5
New Minimum: -2.588032116441949 > -2.711182088562721
WOLF (strong): th(108583.50837760697)=-2.711182088562721; dx=8.976599052095948E-7 evalInputDelta=0.12314997212077206
New Minimum: -2.711182088562721 > -2.887326410918231
END: th(54291.75418880348)=-2.887326410918231; dx=-2.5828940703495525E-6 evalInputDelta=0.29929429447628175
Fitness changed from -2.588032116441949 to -2.887326410918231
Iteration 4 complete. Error: -2.887326410918231 Total: 6.5920; Orientation: 0.4605; Line Search: 4.6566
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4db0fcba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.073e+03, gradient 5.415e-03, dot -0.303; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.887326410918231, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 4db0fcba to history. Total: 3
th(0)=-2.887326410918231;dx=-2.8995996823967157E-5
New Minimum: -2.887326410918231 > -3.1761364308015416
WOLF (strong): th(116968.03860704206)=-3.1761364308015416; dx=1.10071013580088E-6 evalInputDelta=0.2888100198833108
New Minimum: -3.1761364308015416 > -3.3715688793355523
END: th(58484.01930352103)=-3.3715688793355523; dx=-2.299157523047263E-6 evalInputDelta=0.48424246841732144
Fitness changed from -2.887326410918231 to -3.3715688793355523
Iteration 5 complete. Error: -3.3715688793355523 Total: 6.1015; Orientation: 0.4651; Line Search: 4.1824
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 397c7d9a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.571e+03, gradient 6.245e-03, dot -0.258; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.3715688793355523, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 397c7d9a to history. Total: 3
th(0)=-3.3715688793355523;dx=-3.8592782059860155E-5
New Minimum: -3.3715688793355523 > -3.478520565479839
WOLF (strong): th(126000.00000000004)=-3.478520565479839; dx=1.934576691389097E-6 evalInputDelta=0.10695168614428674
New Minimum: -3.478520565479839 > -3.9648300169310056
WOLF (strong): th(63000.00000000002)=-3.9648300169310056; dx=8.924501678372667E-7 evalInputDelta=0.5932611375954533
New Minimum: -3.9648300169310056 > -3.9825740713472566
END: th(21000.000000000007)=-3.9825740713472566; dx=-1.69817214455937E-5 evalInputDelta=0.6110051920117043
Fitness changed from -3.3715688793355523 to -3.9825740713472566
Iteration 6 complete. Error: -3.9825740713472566 Total: 7.5026; Orientation: 0.4643; Line Search: 5.5721
Adding measurement 250e7781 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.289e+05, gradient 5.671e-03, dot -0.322; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.9825740713472566, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 250e7781 to history. Total: 3
th(0)=-3.9825740713472566;dx=-3.1694488455821606E-5
New Minimum: -3.9825740713472566 > -4.150895171681125
END: th(45243.12849066957)=-4.150895171681125; dx=-5.04426091319484E-7 evalInputDelta=0.16832110033386805
Fitness changed from -3.9825740713472566 to -4.150895171681125
Iteration 7 complete. Error: -4.150895171681125 Total: 4.7477; Orientation: 0.5849; Line Search: 2.8199
Adding measurement 675bd6de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.386e+04, gradient 8.845e-03, dot -0.191; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.150895171681125, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 675bd6de to history. Total: 3
th(0)=-4.150895171681125;dx=-7.76608484462632E-5
Armijo: th(97473.3655058684)=-3.8641236539120105; dx=3.1895666211953028E-6 evalInputDelta=-0.28677151776911414
New Minimum: -4.150895171681125 > -4.438049394456653
WOLF (strong): th(48736.6827529342)=-4.438049394456653; dx=6.942027888133673E-6 evalInputDelta=0.28715422277552793
New Minimum: -4.438049394456653 > -4.912725338609399
END: th(16245.560917644732)=-4.912725338609399; dx=-9.632875622428225E-6 evalInputDelta=0.7618301669282745
Fitness changed from -4.150895171681125 to -4.912725338609399
Iteration 8 complete. Error: -4.912725338609399 Total: 7.4279; Orientation: 0.4757; Line Search: 5.5789
Adding measurement 3de27e3d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.745e+03, gradient 6.500e-03, dot -0.177; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Po
...skipping 18742 bytes...
dx=-1.5603208757487994E-5 evalInputDelta=7.130469444347028
Fitness changed from -93.64920965731378 to -100.7796791016608
Iteration 29 complete. Error: -100.7796791016608 Total: 7.5444; Orientation: 0.5889; Line Search: 5.6171
Adding measurement 1c4da38 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.858e+05, gradient 6.206e-03, dot -0.088; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -100.7796791016608, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 1c4da38 to history. Total: 3
th(0)=-100.7796791016608;dx=-3.738910193769096E-5
New Minimum: -100.7796791016608 > -104.46684985134986
END: th(625142.8898033845)=-104.46684985134986; dx=-9.149635943936136E-6 evalInputDelta=3.6871707496890593
Fitness changed from -100.7796791016608 to -104.46684985134986
Iteration 30 complete. Error: -104.46684985134986 Total: 4.7356; Orientation: 0.5929; Line Search: 2.8158
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Adding measurement 34fdb75a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.957e+05, gradient 7.395e-03, dot -0.075; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -104.46684985134986, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 34fdb75a to history. Total: 3
th(0)=-104.46684985134986;dx=-5.181854843189931E-5
Armijo: th(1346829.528019191)=-97.03767090803167; dx=6.0600201985703075E-6 evalInputDelta=-7.429178943318192
New Minimum: -104.46684985134986 > -107.09315316069046
WOLF (strong): th(673414.7640095955)=-107.09315316069046; dx=1.954265568404093E-6 evalInputDelta=2.6263033093405994
New Minimum: -107.09315316069046 > -108.90376509310727
END: th(224471.5880031985)=-108.90376509310727; dx=-1.2906277910909971E-5 evalInputDelta=4.436915241757404
Fitness changed from -104.46684985134986 to -108.90376509310727
Iteration 31 complete. Error: -108.90376509310727 Total: 7.6373; Orientation: 0.4620; Line Search: 5.5905
Adding measurement ac41dab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.546e+06, gradient 6.011e-03, dot -0.100; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -108.90376509310727, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement ac41dab to history. Total: 3
th(0)=-108.90376509310727;dx=-3.511585527623353E-5
New Minimum: -108.90376509310727 > -112.61990794927014
END: th(483609.3761206357)=-112.61990794927014; dx=-1.0885260813504154E-5 evalInputDelta=3.7161428561628753
Fitness changed from -108.90376509310727 to -112.61990794927014
Iteration 32 complete. Error: -112.61990794927014 Total: 4.5807; Orientation: 0.4623; Line Search: 2.7779
Adding measurement 151e325c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.657e+06, gradient 7.195e-03, dot -0.086; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -112.61990794927014, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
LBFGS Accumulation History: 3 points
Removed measurement 151e325c to history. Total: 3
th(0)=-112.61990794927014;dx=-5.032533666233201E-5
Armijo: th(1041904.8163389745)=-108.35772253321845; dx=3.603755092992855E-6 evalInputDelta=-4.262185416051693
New Minimum: -112.61990794927014 > -114.18932906739201
END: th(520952.40816948726)=-114.18932906739201; dx=-2.6725563854170923E-6 evalInputDelta=1.5694211181218662
Fitness changed from -112.61990794927014 to -114.18932906739201
Iteration 33 complete. Error: -114.18932906739201 Total: 5.9734; Orientation: 0.4521; Line Search: 4.1748
Adding measurement 5aca3244 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.336e+06, gradient 8.268e-03, dot 0.081; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
th(0)=-114.18932906739201;dx=-884.909760928027
Armijo: th(1122357.9400159926)=-55.89882821354068; dx=1.5755616683244317E-5 evalInputDelta=-58.29050085385133
Armijo: th(561178.9700079963)=-55.89882821354068; dx=3.115823987450237E-5 evalInputDelta=-58.29050085385133
Armijo: th(187059.6566693321)=-55.898447023753974; dx=9.495261746095889E-5 evalInputDelta=-58.290882043638035
Armijo: th(46764.91416733302)=-55.89598134610345; dx=3.755711958891828E-4 evalInputDelta=-58.29334772128856
Armijo: th(9352.982833466605)=-55.895304863248256; dx=0.0018929317666666696 evalInputDelta=-58.29402420414375
Armijo: th(1558.8304722444343)=-55.888751149442; dx=0.011462047710825908 evalInputDelta=-58.30057791795001
Armijo: th(222.6900674634906)=-55.93609734548274; dx=0.07956422744141298 evalInputDelta=-58.253231721909266
Armijo: th(27.836258432936326)=-56.15527119147787; dx=0.6409586955800628 evalInputDelta=-58.03405787591414
Armijo: th(3.0929176036595916)=-58.27742755640336; dx=5.999093429236774 evalInputDelta=-55.91190151098865
Armijo: th(0.3092917603659592)=-71.27060820747043; dx=75.39589020407251 evalInputDelta=-42.91872085992158
Armijo: th(0.028117432760541742)=-104.28480283768504; dx=461.9666218007345 evalInputDelta=-9.904526229706974
Armijo: th(0.0023431193967118117)=-114.05397189374158; dx=365.4566127749655 evalInputDelta=-0.13535717365043354
New Minimum: -114.18932906739201 > -114.20442883200269
WOLF (strong): th(1.8023995359321628E-4)=-114.20442883200269; dx=177.96674311169016 evalInputDelta=0.015099764610681632
WOLF (strong): th(1.2874282399515448E-5)=-114.19060615334709; dx=149.89011744125193 evalInputDelta=0.0012770859550812474
WOLF (strong): th(8.582854933010299E-7)=-114.18941535743349; dx=146.14079656883337 evalInputDelta=8.629004148019703E-5
WOLF (strong): th(5.364284333131437E-8)=-114.1893376963991; dx=147.14422998644994 evalInputDelta=8.629007083982287E-6
Armijo: th(3.155461372430257E-9)=-114.18932906739201; dx=147.9599784676305 evalInputDelta=0.0
WOLF (strong): th(1.753034095794587E-10)=-114.18933338189564; dx=152.87174706891645 evalInputDelta=4.314503627256272E-6
MIN ALPHA (9.226495241024143E-12): th(1.8023995359321628E-4)=-114.20442883200269
Fitness changed from -114.18932906739201 to -114.20442883200269
Iteration 34 complete. Error: -114.20442883200269 Total: 30.3548; Orientation: 0.4687; Line Search: 28.0599
Adding measurement 792c7d95 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.476e+07, gradient 8.216e-03, dot -0.072; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -114.20442883200269, -114.18932906739201, -2.187016096926548, -1.9225423456109907, -1.7028498331714668
Rejected: LBFGS Orientation magnitude: 2.013e+07, gradient 8.216e-03, dot -0.072; [a6a86fd2-e905-439a-999e-0755a16b2e01 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -114.20442883200269, -114.18932906739201, -2.187016096926548, -1.9225423456109907
LBFGS Accumulation History: 3 points
Removed measurement 792c7d95 to history. Total: 4
Removed measurement 5aca3244 to history. Total: 3
th(0)=-114.20442883200269;dx=-6.681346696042395E-5
END: th(2.1544346900318838E-10)=-114.20442883200269; dx=-5.500594742719019E-5 evalInputDelta=0.0
Fitness changed from -114.20442883200269 to -114.20442883200269
Static Iteration Total: 5.1456; Orientation: 1.0261; Line Search: 2.7782
Iteration 35 failed. Error: -114.20442883200269
Previous Error: 0.0 -> -114.20442883200269
Retrying iteration 35
Reset training subject: 3208465400443
Adding measurement 667f1dae to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-114.20442883200269;dx=-6.698545444860267E-5
END: th(4.6415888336127794E-10)=-114.20442883200269; dx=-5.531887447573582E-5 evalInputDelta=0.0
Fitness changed from -114.20442883200269 to -114.20442883200269
Static Iteration Total: 5.4991; Orientation: 0.0170; Line Search: 2.8099
Iteration 36 failed. Error: -114.20442883200269
Previous Error: 0.0 -> -114.20442883200269
Optimization terminated 36
Final threshold in iteration 36: -114.20442883200269 (> -Infinity) after 240.738s (< 3600.000s)
Returns
-114.20442883200269