BasicOptimizer.scala:89 executed in 138.32 seconds (1.120 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1983552250723
Reset training subject: 1984479608203
Adding measurement 4e46b407 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.266605529218525;dx=-0.055579514307734744
New Minimum: -15.266605529218525 > -15.378318366494108
WOLFE (weak): th(2.154434690031884)=-15.378318366494108; dx=-0.05658206084345925 evalInputDelta=0.1117128372755829
New Minimum: -15.378318366494108 > -15.491884629698312
WOLFE (weak): th(4.308869380063768)=-15.491884629698312; dx=-0.05645793015116144 evalInputDelta=0.22527910047978672
New Minimum: -15.491884629698312 > -15.935243693685655
WOLFE (weak): th(12.926608140191302)=-15.935243693685655; dx=-0.05741349644963424 evalInputDelta=0.6686381644671293
New Minimum: -15.935243693685655 > -18.050102863388513
WOLFE (weak): th(51.70643256076521)=-18.050102863388513; dx=-0.056612810439662174 evalInputDelta=2.7834973341699882
New Minimum: -18.050102863388513 > -28.094903601871238
END: th(258.53216280382605)=-28.094903601871238; dx=-0.034077809464234984 evalInputDelta=12.828298072652712
Fitness changed from -15.266605529218525 to -28.094903601871238
Iteration 1 complete. Error: -28.094903601871238 Total: 7.8267; Orientation: 0.0199; Line Search: 5.2384
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 400c7544 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-28.094903601871238;dx=-0.08525161019692594
New Minimum: -28.094903601871238 > -48.8150857589767
END: th(556.9906600335335)=-48.8150857589767; dx=-0.017427399833592656 evalInputDelta=20.720182157105462
Fitness changed from -28.094903601871238 to -48.8150857589767
Iteration 2 complete. Error: -48.8150857589767 Total: 2.6151; Orientation: 0.0203; Line Search: 1.7424
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3bc222f0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-48.8150857589767;dx=-0.10150924105498302
New Minimum: -48.8150857589767 > -76.58292457158775
END: th(1200.0)=-76.58292457158775; dx=-0.010961283110881985 evalInputDelta=27.767838812611053
Fitness changed from -48.8150857589767 to -76.58292457158775
Iteration 3 complete. Error: -76.58292457158775 Total: 2.6005; Orientation: 0.0202; Line Search: 1.7271
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 261429b8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.470e+03, gradient 3.572e-01, dot -0.240; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -76.58292457158775, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 261429b8 to history. Total: 3
th(0)=-76.58292457158775;dx=-0.12427480718714423
New Minimum: -76.58292457158775 > -102.62682226609913
END: th(2585.3216280382608)=-102.62682226609913; dx=-0.003160481047670036 evalInputDelta=26.043897694511372
Fitness changed from -76.58292457158775 to -102.62682226609913
Iteration 4 complete. Error: -102.62682226609913 Total: 3.1104; Orientation: 0.5237; Line Search: 1.7393
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 32bd24af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.882e+02, gradient 2.948e-01, dot -0.134; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -102.62682226609913, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 32bd24af to history. Total: 3
th(0)=-102.62682226609913;dx=-0.08485359084313795
New Minimum: -102.62682226609913 > -153.51269160683427
END: th(5569.9066003353355)=-153.51269160683427; dx=-0.0026001684104741823 evalInputDelta=50.88586934073514
Fitness changed from -102.62682226609913 to -153.51269160683427
Iteration 5 complete. Error: -153.51269160683427 Total: 3.1001; Orientation: 0.4959; Line Search: 1.7468
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement b9f8180 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.474e+02, gradient 2.627e-01, dot -0.103; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -153.51269160683427, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement b9f8180 to history. Total: 3
th(0)=-153.51269160683427;dx=-0.06783858527622663
New Minimum: -153.51269160683427 > -210.9028675273288
END: th(12000.000000000002)=-210.9028675273288; dx=-1.1531559371484883E-4 evalInputDelta=57.39017592049453
Fitness changed from -153.51269160683427 to -210.9028675273288
Iteration 6 complete. Error: -210.9028675273288 Total: 3.0890; Orientation: 0.4753; Line Search: 1.7710
Adding measurement 1732c769 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.156e+03, gradient 2.453e-01, dot -0.069; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -210.9028675273288, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 1732c769 to history. Total: 3
th(0)=-210.9028675273288;dx=-0.05920716543877095
New Minimum: -210.9028675273288 > -270.4886781364425
WOLF (strong): th(25853.21628038261)=-270.4886781364425; dx=9.493727888682619E-4 evalInputDelta=59.58581060911371
New Minimum: -270.4886781364425 > -370.03323330479384
END: th(12926.608140191305)=-370.03323330479384; dx=-9.772247995070696E-4 evalInputDelta=159.13036577746504
Fitness changed from -210.9028675273288 to -370.03323330479384
Iteration 7 complete. Error: -370.03323330479384 Total: 3.9847; Orientation: 0.4795; Line Search: 2.6870
Adding measurement 2ec74e8c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.095e+03, gradient 2.641e-01, dot -0.088; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -370.03323330479384, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 2ec74e8c to history. Total: 3
th(0)=-370.03323330479384;dx=-0.06869478865474936
New Minimum: -370.03323330479384 > -416.2230471754297
WOLF (strong): th(27849.533001676682)=-416.2230471754297; dx=0.0023028806307807516 evalInputDelta=46.189813870635874
New Minimum: -416.2230471754297 > -557.2790201057277
END: th(13924.766500838341)=-557.2790201057277; dx=-9.639757948174622E-4 evalInputDelta=187.24578680093384
Fitness changed from -370.03323330479384 to -557.2790201057277
Iteration 8 complete. Error: -557.2790201057277 Total: 3.9346; Orientation: 0.4753; Line Search: 2.6350
Adding measurement 418acc54 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.255e+04, gradient 2.651e-01, dot -0.108; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -557.2790201057277, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 418acc54 to history. Total: 3
th(0)=-557.2790201057277;dx=-0.06917287197091426
New Minimum: -557.2790201057277 > -573.6993441690516
WOLF (strong): th(30000.00000000001)=-573.6993441690516; dx=0.002986591324384083 evalInputDelta=16.42032406332396
New Minimum: -573.6993441690516 > -748.6244051592226
END: th(15000.000000000005)=-748.6244051592226; dx=-0.00208801327352798 evalInputDelta=191.34538505349497
Fitness changed from -557.2790201057277 to -748.6244051592226
Iteration 9 complete. Error: -748.6244051592226 Total: 3.9217; Orientation: 0.4739; Line Search: 2.6296
Adding measurement 6a318f23 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.119e+05, gradient 2.704e-01, dot -0.131; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -748.6244051592226, -48.815085758976
...skipping 13427 bytes...
ement 730f5a71 to history. Total: 3
th(0)=-1348.89839313419;dx=-0.04121573125732017
New Minimum: -1348.89839313419 > -1483.8290838233358
END: th(4452.783337532833)=-1483.8290838233358; dx=-0.03239086429836789 evalInputDelta=134.93069068914588
Fitness changed from -1348.89839313419 to -1483.8290838233358
Iteration 25 complete. Error: -1483.8290838233358 Total: 3.2060; Orientation: 0.6287; Line Search: 1.7632
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 500e2999 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.363e+04, gradient 2.037e-01, dot -0.078; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1483.8290838233358, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 500e2999 to history. Total: 3
th(0)=-1483.8290838233358;dx=-0.04127847661482185
New Minimum: -1483.8290838233358 > -1739.874779976996
END: th(9593.230889576686)=-1739.874779976996; dx=-0.027339936834444956 evalInputDelta=256.0456961536602
Fitness changed from -1483.8290838233358 to -1739.874779976996
Iteration 26 complete. Error: -1739.874779976996 Total: 3.7242; Orientation: 0.4739; Line Search: 2.4005
Adding measurement 1731c8a2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.637e+04, gradient 2.033e-01, dot -0.106; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1739.874779976996, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 1731c8a2 to history. Total: 3
th(0)=-1739.874779976996;dx=-0.04103432977052156
New Minimum: -1739.874779976996 > -2122.650583586474
END: th(20667.98941798944)=-2122.650583586474; dx=-0.018001431075600868 evalInputDelta=382.7758036094781
Fitness changed from -1739.874779976996 to -2122.650583586474
Iteration 27 complete. Error: -2122.650583586474 Total: 3.1005; Orientation: 0.4924; Line Search: 1.7752
Adding measurement 4e55b19d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.785e+05, gradient 2.058e-01, dot -0.141; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2122.650583586474, -48.8150857589767, -28.094903601871238, -15.266605529218525
LBFGS Accumulation History: 3 points
Removed measurement 4e55b19d to history. Total: 3
th(0)=-2122.650583586474;dx=-0.04204959403215982
New Minimum: -2122.650583586474 > -2476.72808358124
END: th(44527.83337532834)=-2476.72808358124; dx=-0.006778759219198396 evalInputDelta=354.07749999476573
Fitness changed from -2122.650583586474 to -2476.72808358124
Iteration 28 complete. Error: -2476.72808358124 Total: 3.0438; Orientation: 0.4813; Line Search: 1.7505
Adding measurement 265a5246 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.355e+05, gradient 2.316e-01, dot 0.153; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
th(0)=-2476.72808358124;dx=-8338.13058392208
Armijo: th(95932.30889576688)=-1807.7793698347152; dx=0.004784205867671356 evalInputDelta=-668.9487137465246
Armijo: th(47966.15444788344)=-1807.8012888589276; dx=0.009575741310336264 evalInputDelta=-668.9267947223123
Armijo: th(15988.71814929448)=-1807.8934564846459; dx=0.02876144094041345 evalInputDelta=-668.834627096594
Armijo: th(3997.17953732362)=-1807.962665543733; dx=0.11462162098657025 evalInputDelta=-668.7654180375068
Armijo: th(799.435907464724)=-1807.9750413100287; dx=0.5747139932720786 evalInputDelta=-668.7530422712111
Armijo: th(133.23931791078732)=-1811.3350739164744; dx=3.4249364367920587 evalInputDelta=-665.3930096647655
Armijo: th(19.03418827296962)=-1826.559402264268; dx=23.582810485610985 evalInputDelta=-650.1686813169717
Armijo: th(2.3792735341212023)=-1899.124469327906; dx=168.23254469189294 evalInputDelta=-577.603614253334
Armijo: th(0.2643637260134669)=-2328.4872771823343; dx=921.407011818899 evalInputDelta=-148.24080639890553
New Minimum: -2476.72808358124 > -2551.7887843628437
END: th(0.02643637260134669)=-2551.7887843628437; dx=-1484.4679408576126 evalInputDelta=75.06070078160383
Fitness changed from -2476.72808358124 to -2551.7887843628437
Iteration 29 complete. Error: -2551.7887843628437 Total: 11.0537; Orientation: 0.4627; Line Search: 9.7654
Adding measurement 7623330 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.943e+05, gradient 2.294e-01, dot 0.139; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
th(0)=-2551.7887843628437;dx=-6188.934251453773
New Minimum: -2551.7887843628437 > -2561.9073948915484
END: th(0.05695543821094974)=-2561.9073948915484; dx=-16.958855715276712 evalInputDelta=10.118610528704721
Fitness changed from -2551.7887843628437 to -2561.9073948915484
Iteration 30 complete. Error: -2561.9073948915484 Total: 3.1549; Orientation: 0.5778; Line Search: 1.7587
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Adding measurement 4792dd51 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 1.683e+05, gradient 2.220e-01, dot 0.122; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
th(0)=-2561.9073948915484;dx=-4565.668376765792
Armijo: th(0.12270677186763762)=-2460.5595908248188; dx=418.6796050231263 evalInputDelta=-101.34780406672962
Armijo: th(0.06135338593381881)=-2518.8035850379442; dx=285.22304308874425 evalInputDelta=-43.10380985360416
Armijo: th(0.02045112864460627)=-2550.5585662752383; dx=69.03564939777306 evalInputDelta=-11.34882861631013
Armijo: th(0.0051127821611515675)=-2559.45589920983; dx=-50.83744801761557 evalInputDelta=-2.45149568171837
Armijo: th(0.0010225564322303136)=-2561.4317285455804; dx=-109.59659016593712 evalInputDelta=-0.47566634596796575
Armijo: th(1.704260720383856E-4)=-2561.8287413486482; dx=-106.66180532111183 evalInputDelta=-0.07865354290015603
Armijo: th(2.434658171976937E-5)=-2561.8962703435127; dx=-112.69803688281426 evalInputDelta=-0.011124548035695625
Armijo: th(3.0433227149711713E-6)=-2561.906223888767; dx=-107.33286020535085 evalInputDelta=-0.0011710027815752255
Armijo: th(3.3814696833013015E-7)=-2561.9072973080038; dx=-117.26117931371749 evalInputDelta=-9.758354462974239E-5
Armijo: th(3.3814696833013015E-8)=-2561.9073948915484; dx=-107.92723094100668 evalInputDelta=0.0
Armijo: th(3.0740633484557285E-9)=-2561.9073948915484; dx=-111.85465341303791 evalInputDelta=0.0
New Minimum: -2561.9073948915484 > -2561.9075900586267
END: th(2.56171945704644E-10)=-2561.9075900586267; dx=-113.70864612482346 evalInputDelta=1.9516707834554836E-4
Fitness changed from -2561.9073948915484 to -2561.9075900586267
Iteration 31 complete. Error: -2561.9075900586267 Total: 12.9928; Orientation: 0.6912; Line Search: 11.4526
Adding measurement 48950326 to history. Total: 6
Accepted: LBFGS Orientation magnitude: 3.218e+05, gradient 2.219e-01, dot 0.041; [25e7bbb3-9e35-40e3-9917-df5b6155c942 = 1.000/1.000e+00]
th(0)=-2561.9075900586267;dx=-2933.3038448988073
Armijo: th(5.519057264390493E-10)=-2561.9075900586267; dx=308.4669044878026 evalInputDelta=0.0
Armijo: th(2.7595286321952465E-10)=-2561.9075900586267; dx=285.7319419072948 evalInputDelta=0.0
MIN ALPHA (9.198428773984155E-11): th(0.0)=-2561.9075900586267
Fitness changed from -2561.9075900586267 to -2561.9075900586267
Static Iteration Total: 5.1191; Orientation: 0.8208; Line Search: 3.4768
Iteration 32 failed. Error: -2561.9075900586267
Previous Error: 0.0 -> -2561.9075900586267
Retrying iteration 32
Reset training subject: 2118457167313
Adding measurement 4aec94fd to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-2561.9075900586267;dx=-0.04901206265519768
END: th(2.1544346900318838E-10)=-2561.9075900586267; dx=-0.0374838929672803 evalInputDelta=0.0
Fitness changed from -2561.9075900586267 to -2561.9075900586267
Static Iteration Total: 3.4118; Orientation: 0.0205; Line Search: 1.7523
Iteration 33 failed. Error: -2561.9075900586267
Previous Error: 0.0 -> -2561.9075900586267
Optimization terminated 33
Final threshold in iteration 33: -2561.9075900586267 (> -Infinity) after 138.317s (< 3600.000s)
Returns
-2561.9075900586267