BasicOptimizer.scala:89 executed in 84.48 seconds (1.570 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3002951405562
Reset training subject: 3003430035373
Adding measurement 34a805c0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-4.054976245866999;dx=-2.3813151188160735E-4
New Minimum: -4.054976245866999 > -4.054993265565109
WOLFE (weak): th(2.154434690031884)=-4.054993265565109; dx=-2.3810776387559645E-4 evalInputDelta=1.701969811040982E-5
New Minimum: -4.054993265565109 > -4.055010111522486
WOLFE (weak): th(4.308869380063768)=-4.055010111522486; dx=-2.3812017354731883E-4 evalInputDelta=3.386565548701981E-5
New Minimum: -4.055010111522486 > -4.055076973652876
WOLFE (weak): th(12.926608140191302)=-4.055076973652876; dx=-2.3796020273474282E-4 evalInputDelta=1.0072778587755238E-4
New Minimum: -4.055076973652876 > -4.055377926424528
WOLFE (weak): th(51.70643256076521)=-4.055377926424528; dx=-2.3814500565182785E-4 evalInputDelta=4.016805575295379E-4
New Minimum: -4.055377926424528 > -4.05702122434504
WOLFE (weak): th(258.53216280382605)=-4.05702122434504; dx=-2.3774144829651411E-4 evalInputDelta=0.002044978478041415
New Minimum: -4.05702122434504 > -4.067652632269336
WOLFE (weak): th(1551.1929768229563)=-4.067652632269336; dx=-2.3535835701873182E-4 evalInputDelta=0.012676386402337592
New Minimum: -4.067652632269336 > -4.14124561800471
END: th(10858.350837760694)=-4.14124561800471; dx=-2.072537926307146E-4 evalInputDelta=0.08626937213771146
Fitness changed from -4.054976245866999 to -4.14124561800471
Iteration 1 complete. Error: -4.14124561800471 Total: 6.2703; Orientation: 0.0163; Line Search: 4.8166
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 13e5ebca to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.14124561800471;dx=-2.3183918852419855E-4
New Minimum: -4.14124561800471 > -4.313439496653318
END: th(23393.607721408407)=-4.313439496653318; dx=-1.8280024342994937E-4 evalInputDelta=0.17219387864860813
Fitness changed from -4.14124561800471 to -4.313439496653318
Iteration 2 complete. Error: -4.313439496653318 Total: 1.5362; Orientation: 0.0161; Line Search: 1.0178
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7d7e4afe to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.313439496653318;dx=-2.446127923011159E-4
New Minimum: -4.313439496653318 > -4.754069815996515
END: th(50400.00000000001)=-4.754069815996515; dx=-1.3781607993003324E-4 evalInputDelta=0.44063031934319685
Fitness changed from -4.313439496653318 to -4.754069815996515
Iteration 3 complete. Error: -4.754069815996515 Total: 1.5458; Orientation: 0.0164; Line Search: 1.0301
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 48a7449e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.808e+04, gradient 1.630e-02, dot -0.489; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.754069815996515, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 48a7449e to history. Total: 3
th(0)=-4.754069815996515;dx=-2.6560855431150357E-4
New Minimum: -4.754069815996515 > -5.745269286460255
END: th(108583.50837760697)=-5.745269286460255; dx=-8.606131870706921E-5 evalInputDelta=0.9911994704637399
Fitness changed from -4.754069815996515 to -5.745269286460255
Iteration 4 complete. Error: -5.745269286460255 Total: 1.9522; Orientation: 0.4285; Line Search: 1.0266
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 29c374af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.354e+04, gradient 1.633e-02, dot -0.268; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.745269286460255, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 29c374af to history. Total: 3
th(0)=-5.745269286460255;dx=-2.666630200219054E-4
New Minimum: -5.745269286460255 > -6.291334816421232
END: th(233936.07721408413)=-6.291334816421232; dx=-3.350958279109908E-5 evalInputDelta=0.5460655299609769
Fitness changed from -5.745269286460255 to -6.291334816421232
Iteration 5 complete. Error: -6.291334816421232 Total: 2.1073; Orientation: 0.4272; Line Search: 1.1837
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 713d497b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.902e+03, gradient 1.410e-02, dot -0.138; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.291334816421232, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 713d497b to history. Total: 3
th(0)=-6.291334816421232;dx=-1.987867869301982E-4
New Minimum: -6.291334816421232 > -6.859837542532646
END: th(504000.0000000002)=-6.859837542532646; dx=-1.0878158903285522E-5 evalInputDelta=0.5685027261114142
Fitness changed from -6.291334816421232 to -6.859837542532646
Iteration 6 complete. Error: -6.859837542532646 Total: 2.0431; Orientation: 0.4905; Line Search: 1.0602
Adding measurement 7a19be69 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.107e+03, gradient 1.082e-02, dot -0.089; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.859837542532646, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 7a19be69 to history. Total: 3
th(0)=-6.859837542532646;dx=-1.1701040053209231E-4
Armijo: th(1085835.0837760698)=-6.463562724726249; dx=-4.263895505258188E-6 evalInputDelta=-0.39627481780639684
New Minimum: -6.859837542532646 > -7.176210482003285
END: th(542917.5418880349)=-7.176210482003285; dx=-1.5416595425649808E-5 evalInputDelta=0.3163729394706385
Fitness changed from -6.859837542532646 to -7.176210482003285
Iteration 7 complete. Error: -7.176210482003285 Total: 2.4590; Orientation: 0.4595; Line Search: 1.5450
Adding measurement 3b0205a0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.490e+03, gradient 9.366e-03, dot -0.111; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.176210482003285, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 3b0205a0 to history. Total: 3
th(0)=-7.176210482003285;dx=-8.765552978499415E-5
Armijo: th(1169680.3860704207)=-7.050421623011091; dx=-5.024629565930644E-6 evalInputDelta=-0.12578885899219383
New Minimum: -7.176210482003285 > -7.697584788489391
END: th(584840.1930352104)=-7.697584788489391; dx=-1.5623957586804707E-5 evalInputDelta=0.5213743064861065
Fitness changed from -7.176210482003285 to -7.697584788489391
Iteration 8 complete. Error: -7.697584788489391 Total: 2.4423; Orientation: 0.4391; Line Search: 1.5509
Adding measurement 6a94b563 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.365e+03, gradient 8.622e-03, dot -0.122; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.697584788489391, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 6a94b563 to history. Total: 3
th(0)=-7.697584788489391;dx=-7.423543529325855E-5
Armijo: th(1260000.0000000005)=-7.108210336283961; dx=-4.968025007401469E-6 evalInputDelta=-0.5893744522054298
Armijo: th(630000.0000000002)=-7.6220756007683015; dx=-1.5211123600030558E-5 evalInputDelta=-0.07550918772108961
New Minimum: -7.697584788489391 > -8.092094760376373
END: th(210000.0000000001)=-8.092094760376373; dx=-4.107184421817346E-5 evalInputDelta=0.3945099718869818
Fitness changed from -7.697584788489391 to -8.092094760376373
Iteration 9 complete. Error: -8.092094760376373 Total: 2.9721; Orientation: 0.4217; Line Search: 2.0801
Ad
...skipping 12224 bytes...
th(3172961.1167274886)=-10.674783734062702; dx=-3.177240715407544E-6 evalInputDelta=0.09047239759502546
Fitness changed from -10.584311336467676 to -10.674783734062702
Iteration 23 complete. Error: -10.674783734062702 Total: 2.6073; Orientation: 0.5089; Line Search: 1.6134
Adding measurement 69e60dc7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.157e+05, gradient 3.957e-03, dot -0.134; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.674783734062702, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 69e60dc7 to history. Total: 3
th(0)=-10.674783734062702;dx=-1.5611803764454892E-5
Armijo: th(6835937.5000000065)=-10.185598759159905; dx=-8.116525837419067E-7 evalInputDelta=-0.48918497490279655
New Minimum: -10.674783734062702 > -11.145780212791742
END: th(3417968.7500000033)=-11.145780212791742; dx=-2.7896412537597826E-6 evalInputDelta=0.47099647872903994
Fitness changed from -10.674783734062702 to -11.145780212791742
Iteration 24 complete. Error: -11.145780212791742 Total: 2.4416; Orientation: 0.4164; Line Search: 1.5683
Adding measurement 54a698e9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.561e+05, gradient 3.682e-03, dot -0.140; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.145780212791742, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 54a698e9 to history. Total: 3
th(0)=-11.145780212791742;dx=-1.3498795462177385E-5
Armijo: th(7363790.444444923)=-10.070411471213143; dx=-7.887089250041823E-7 evalInputDelta=-1.0753687415785986
New Minimum: -11.145780212791742 > -11.390056317723054
END: th(3681895.2222224614)=-11.390056317723054; dx=-2.625896748012738E-6 evalInputDelta=0.2442761049313127
Fitness changed from -11.145780212791742 to -11.390056317723054
Iteration 25 complete. Error: -11.390056317723054 Total: 3.0193; Orientation: 0.4198; Line Search: 2.1273
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 15dc696f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.892e+05, gradient 3.484e-03, dot -0.139; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.390056317723054, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 15dc696f to history. Total: 3
th(0)=-11.390056317723054;dx=-1.2086964039802078E-5
Armijo: th(7932402.791818722)=-10.179227603497015; dx=-7.24281448138978E-7 evalInputDelta=-1.2108287142260394
Armijo: th(3966201.395909361)=-11.016501839562785; dx=-2.400260548365698E-6 evalInputDelta=-0.3735544781602691
New Minimum: -11.390056317723054 > -11.471607085332066
END: th(1322067.131969787)=-11.471607085332066; dx=-6.072184623646063E-6 evalInputDelta=0.08155076760901103
Fitness changed from -11.390056317723054 to -11.471607085332066
Iteration 26 complete. Error: -11.471607085332066 Total: 3.0036; Orientation: 0.4289; Line Search: 2.0818
Adding measurement 521865fb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.546e+06, gradient 3.420e-03, dot -0.167; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.471607085332066, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 521865fb to history. Total: 3
th(0)=-11.471607085332066;dx=-1.1650390685320563E-5
Armijo: th(2848307.29166667)=-11.353324840477633; dx=-3.6246103967412817E-6 evalInputDelta=-0.1182822448544325
Armijo: th(1424153.645833335)=-11.460345265689176; dx=-5.748123642846409E-6 evalInputDelta=-0.011261819642889748
New Minimum: -11.471607085332066 > -11.489188198942031
END: th(474717.88194444496)=-11.489188198942031; dx=-7.996862208549778E-6 evalInputDelta=0.017581113609965726
Fitness changed from -11.471607085332066 to -11.489188198942031
Iteration 27 complete. Error: -11.489188198942031 Total: 2.9647; Orientation: 0.4198; Line Search: 2.0842
Adding measurement 87033c2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.198e+07, gradient 3.387e-03, dot -0.178; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.489188198942031, -4.313439496653318, -4.14124561800471, -4.054976245866999
LBFGS Accumulation History: 3 points
Removed measurement 87033c2 to history. Total: 3
th(0)=-11.489188198942031;dx=-1.1412728494754855E-5
New Minimum: -11.489188198942031 > -11.621564888456376
END: th(1022748.6728395727)=-11.621564888456376; dx=-6.462670800371912E-6 evalInputDelta=0.132376689514345
Fitness changed from -11.489188198942031 to -11.621564888456376
Iteration 28 complete. Error: -11.621564888456376 Total: 1.9305; Orientation: 0.4099; Line Search: 1.0494
Adding measurement d489ecb to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.378e+05, gradient 3.451e-03, dot 0.185; [329c4873-e1d4-4ec6-aa34-d2c2fa2092bd = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-11.621564888456376;dx=-88.12384162045467
Armijo: th(2203445.2199496455)=-7.910916191204353; dx=7.552050490661816E-7 evalInputDelta=-3.710648697252023
Armijo: th(1101722.6099748227)=-7.910916191204353; dx=1.514443229542779E-6 evalInputDelta=-3.710648697252023
Armijo: th(367240.86999160756)=-7.910647345846695; dx=4.541964935257463E-6 evalInputDelta=-3.710917542609681
Armijo: th(91810.21749790189)=-7.912052362541387; dx=1.8113059690878936E-5 evalInputDelta=-3.709512525914989
Armijo: th(18362.043499580377)=-7.914277945224142; dx=9.070139623246264E-5 evalInputDelta=-3.7072869432322344
Armijo: th(3060.340583263396)=-7.921160048619224; dx=5.408451572988372E-4 evalInputDelta=-3.7004048398371525
Armijo: th(437.1915118947709)=-7.9733351270258686; dx=0.00368902507173665 evalInputDelta=-3.6482297614305077
Armijo: th(54.64893898684636)=-8.154512088280981; dx=0.029091034907291573 evalInputDelta=-3.467052800175395
Armijo: th(6.0721043318718175)=-8.693907335625855; dx=0.2688723434681084 evalInputDelta=-2.927657552830521
Armijo: th(0.6072104331871817)=-10.300506521068511; dx=-1.2945392777439402 evalInputDelta=-1.3210583673878649
Armijo: th(0.05520094847156198)=-11.565386671810238; dx=-34.03120177558626 evalInputDelta=-0.056178216646138424
Armijo: th(0.004600079039296831)=-11.620305859109505; dx=-50.49227010268702 evalInputDelta=-0.0012590293468708325
Armijo: th(3.5385223379206393E-4)=-11.621336072317835; dx=-52.80695026283263 evalInputDelta=-2.2881613854153215E-4
Armijo: th(2.5275159556575994E-5)=-11.62154064967199; dx=-53.09603694344319 evalInputDelta=-2.4238784385843815E-5
Armijo: th(1.6850106371050663E-6)=-11.621563918905972; dx=-53.109055108755896 evalInputDelta=-9.695504044060499E-7
Armijo: th(1.0531316481906664E-7)=-11.621564888456376; dx=-53.01918588603194 evalInputDelta=0.0
Armijo: th(6.194892048180391E-9)=-11.621564403681184; dx=-53.122833996358246 evalInputDelta=-4.847751924330623E-7
Armijo: th(3.4416066934335504E-10)=-11.621564888456376; dx=-53.148804610314585 evalInputDelta=0.0
MIN ALPHA (1.811371943912395E-11): th(0.0)=-11.621564888456376
Fitness changed from -11.621564888456376 to -11.621564888456376
Static Iteration Total: 11.2416; Orientation: 0.4052; Line Search: 10.3824
Iteration 29 failed. Error: -11.621564888456376
Previous Error: 0.0 -> -11.621564888456376
Retrying iteration 29
Reset training subject: 3085474497905
Adding measurement 6b0ac6b4 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-11.621564888456376;dx=-1.1864828125797754E-5
END: th(2.1544346900318838E-10)=-11.621564888456376; dx=-9.634600669588189E-6 evalInputDelta=0.0
Fitness changed from -11.621564888456376 to -11.621564888456376
Static Iteration Total: 1.9569; Orientation: 0.0164; Line Search: 1.0344
Iteration 30 failed. Error: -11.621564888456376
Previous Error: 0.0 -> -11.621564888456376
Optimization terminated 30
Final threshold in iteration 30: -11.621564888456376 (> -Infinity) after 84.480s (< 3600.000s)
Returns
-11.621564888456376