BasicOptimizer.scala:89 executed in 62.84 seconds (0.742 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 159733485982000
Reset training subject: 159734412391200
Adding measurement 78103249 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-101.81915647362239;dx=-0.007688095068318798
New Minimum: -101.81915647362239 > -101.83597316119437
WOLFE (weak): th(2.154434690031884)=-101.83597316119437; dx=-0.007407113527930208 evalInputDelta=0.016816687571989064
New Minimum: -101.83597316119437 > -101.85277748421984
WOLFE (weak): th(4.308869380063768)=-101.85277748421984; dx=-0.0074047480062883805 evalInputDelta=0.03362101059745726
New Minimum: -101.85277748421984 > -101.92006288631302
WOLFE (weak): th(12.926608140191302)=-101.92006288631302; dx=-0.0073637857088679325 evalInputDelta=0.10090641269063383
New Minimum: -101.92006288631302 > -102.22288187155065
WOLFE (weak): th(51.70643256076521)=-102.22288187155065; dx=-0.007399695118068124 evalInputDelta=0.40372539792826956
New Minimum: -102.22288187155065 > -103.84571544935064
WOLFE (weak): th(258.53216280382605)=-103.84571544935064; dx=-0.007484595455511795 evalInputDelta=2.02655897572825
New Minimum: -103.84571544935064 > -114.1193522698933
WOLFE (weak): th(1551.1929768229563)=-114.1193522698933; dx=-0.007505179608586804 evalInputDelta=12.300195796270913
New Minimum: -114.1193522698933 > -180.99055785109343
END: th(10858.350837760694)=-180.99055785109343; dx=-0.0058764237733464255 evalInputDelta=79.17140137747104
Fitness changed from -101.81915647362239 to -180.99055785109343
Iteration 1 complete. Error: -180.99055785109343 Total: 9.4937; Orientation: 0.0149; Line Search: 6.9280
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 571838d1 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-180.99055785109343;dx=-0.009780889899446218
New Minimum: -180.99055785109343 > -325.6271712979431
END: th(23393.607721408407)=-325.6271712979431; dx=-0.003826296502313897 evalInputDelta=144.63661344684968
Fitness changed from -180.99055785109343 to -325.6271712979431
Iteration 2 complete. Error: -325.6271712979431 Total: 2.6423; Orientation: 0.0147; Line Search: 1.7056
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5022bfce to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-325.6271712979431;dx=-0.014901274479172208
New Minimum: -325.6271712979431 > -472.1445256751369
END: th(50400.00000000001)=-472.1445256751369; dx=-0.0011439213987881518 evalInputDelta=146.51735437719378
Fitness changed from -325.6271712979431 to -472.1445256751369
Iteration 3 complete. Error: -472.1445256751369 Total: 2.6271; Orientation: 0.0145; Line Search: 1.7260
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 27933872 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.923e+04, gradient 1.235e-01, dot -0.196; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -472.1445256751369, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 27933872 to history. Total: 3
th(0)=-472.1445256751369;dx=-0.015145130581908991
New Minimum: -472.1445256751369 > -602.431297950231
WOLF (strong): th(108583.50837760697)=-602.431297950231; dx=1.0939299037196517E-4 evalInputDelta=130.28677227509417
New Minimum: -602.431297950231 > -740.1244405503712
END: th(54291.75418880348)=-740.1244405503712; dx=-0.0011869225634450393 evalInputDelta=267.97991487523433
Fitness changed from -472.1445256751369 to -740.1244405503712
Iteration 4 complete. Error: -740.1244405503712 Total: 3.8185; Orientation: 0.2746; Line Search: 2.6228
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement f05bcde to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.706e+04, gradient 1.360e-01, dot -0.174; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -740.1244405503712, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement f05bcde to history. Total: 3
th(0)=-740.1244405503712;dx=-0.01835547165218611
New Minimum: -740.1244405503712 > -804.8544123007589
WOLF (strong): th(116968.03860704206)=-804.8544123007589; dx=8.92545679930453E-4 evalInputDelta=64.72997175038768
New Minimum: -804.8544123007589 > -1059.5831137763569
END: th(58484.01930352103)=-1059.5831137763569; dx=-4.3711837787786725E-4 evalInputDelta=319.45867322598565
Fitness changed from -740.1244405503712 to -1059.5831137763569
Iteration 5 complete. Error: -1059.5831137763569 Total: 3.7921; Orientation: 0.2821; Line Search: 2.6024
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3d7628f1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.028e+04, gradient 1.448e-01, dot -0.152; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1059.5831137763569, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 3d7628f1 to history. Total: 3
th(0)=-1059.5831137763569;dx=-0.020875016710018478
Armijo: th(126000.00000000004)=-925.4362147117433; dx=0.0014809596875080982 evalInputDelta=-134.14689906461354
New Minimum: -1059.5831137763569 > -1325.2237546920142
WOLF (strong): th(63000.00000000002)=-1325.2237546920142; dx=0.0011421663001596398 evalInputDelta=265.6406409156573
New Minimum: -1325.2237546920142 > -1360.947785552407
END: th(21000.000000000007)=-1360.947785552407; dx=-0.008257749188536305 evalInputDelta=301.36467177605005
Fitness changed from -1059.5831137763569 to -1360.947785552407
Iteration 6 complete. Error: -1360.947785552407 Total: 4.7270; Orientation: 0.2817; Line Search: 3.5023
Adding measurement 497a58d5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.513e+05, gradient 1.292e-01, dot -0.195; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1360.947785552407, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 497a58d5 to history. Total: 3
th(0)=-1360.947785552407;dx=-0.01657104073417445
New Minimum: -1360.947785552407 > -1636.4579890727412
END: th(45243.12849066957)=-1636.4579890727412; dx=-0.0036994506661837704 evalInputDelta=275.51020352033424
Fitness changed from -1360.947785552407 to -1636.4579890727412
Iteration 7 complete. Error: -1636.4579890727412 Total: 2.8010; Orientation: 0.2981; Line Search: 1.6924
Adding measurement 606135f6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.178e+05, gradient 1.414e-01, dot -0.211; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1636.4579890727412, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 606135f6 to history. Total: 3
th(0)=-1636.4579890727412;dx=-0.019947086660749166
Armijo: th(97473.3655058684)=-1627.1978982287312; dx=6.943310243738335E-4 evalInputDelta=-9.260090844009937
New Minimum: -1636.4579890727412 > -1836.0091230710157
END: th(48736.6827529342)=-1836.0091230710157; dx=-0.0014481060176030178 evalInputDelta=199.5511339982745
Fitness changed from -1636.4579890727412 to -1836.0091230710157
Iteration 8 complete. Error: -1836.0091230710157 Total: 3.6550; Orientation: 0.2767; Line Search: 2.5716
Adding measurement 333feb0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.746e+06, gradient 1.534e-01, dot -0.189; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1836.0091230710157, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 333feb0 to history. Total: 3
th(0)=-1836.0091230710157;dx=-0.023417557578702775
Armijo: th(105000.00000000004)=-1621.1403393907635; dx=0.0022096673506041492 evalInputDelta=-214.86878368025214
New Minimum: -1836.0091230710157 > -2029.534059827526
WOLF (strong): th(52500.00000000002)=-2029.534059827526; dx=0.0010853368941994464 evalInputDelta=193.52493675651044
New Minimum: -2029.534059827526 > -2072.197987645003
END: th(17500.000000000007)=-2072.197987645003; dx=-0.008358806209168191 evalInputDelta=236.18886457398753
Fitness changed from -1836.0091230710157 to -2072.197987645003
Iteration 9 complete. Error: -2072.197987645003 Total: 4.6551; Orientation: 0.2763; Line Search: 3.5747
Adding measurement 7f6cacf8 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.540e+04, gradient 1.306e-01, dot 0.363; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2072.197987645003;dx=-3075.581636776329
Armijo: th(37702.607075557986)=-379.9700440495277; dx=0.012853644558790765 evalInputDelta=-1692.2279435954756
Armijo: th(18851.303537778993)=-380.03437344535035; dx=0.025631465292946656 evalInputDelta=-1692.1636141996528
Armijo: th(6283.767845926331)=-380.211474872866; dx=0.07683863392373506 evalInputDelta=-1691.9865127721373
Armijo: th(1570.9419614815827)=-380.17935734334657; dx=0.30682557367767227 evalInputDelta=-1692.0186303016567
Armijo: th(314.1883922963165)=-381.3808586898404; dx=1.5298239561801716 evalInputDelta=-1690.8171289551628
Armijo: th(52.364732049386085)=-390.149050043698; dx=9.191295123041188 evalInputDelta=-1682.0489376013052
Armijo: th(7.480676007055155)=-465.505941154353; dx=69.88145109238326 evalInputDelta=-1606.6920464906502
Armijo: th(0.9350845008818943)=-1085.7569364273018; dx=537.4320469795103 evalInputDelta=-986.4410512177014
New Minimum: -2072.197987645003 > -2098.730687820617
END: th(0.10389827787576604)=-2098.730687820617; dx=-108.23059577784979 evalInputDelta=26.532700175613627
Fitness changed from -2072.197987645003 to -2098.730687820617
Iteration 10 complete. Error: -2098.730687820617 Total: 9.8142; Orientation: 0.2712; Line Search: 8.7022
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5d013356 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.184e+04, gradient 1.462e-01, dot -0.226; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2098.730687820617, -2072.197987645003, -325.6271712979431, -180.99055785109343, -101.81915647362239
Rejected: LBFGS Orientation magnitude: 1.943e+05, gradient 1.462e-01, dot -0.180; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2098.730687820617, -2072.197987645003, -325.6271712979431, -180.99055785109343
LBFGS Accumulation History: 3 points
Removed measurement 5d013356 to history. Total: 4
Removed measurement 7f6cacf8 to history. Total: 3
th(0)=-2098.730687820617;dx=-0.021346999714944685
New Minimum: -2098.730687820617 > -2098.7340231672997
END: th(0.22384205409012253)=-2098.7340231672997; dx=-0.012990002905284886 evalInputDelta=0.003335346682888485
Fitness changed from -2098.730687820617 to -2098.7340231672997
Iteration 11 complete. Error: -2098.7340231672997 Total: 3.1987; Orientation: 0.6092; Line Search: 1.6630
Adding measurement 44f382f1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.506e+04, gradient 1.463e-01, dot -0.108; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2098.7340231672997, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 44f382f1 to history. Total: 3
th(0)=-2098.7340231672997;dx=-0.021362911962777675
New Minimum: -2098.7340231672997 > -2098.7414085589485
END: th(0.4822530864197533)=-2098.7414085589485; dx=-0.013001112124670216 evalInputDelta=0.007385391648767836
Fitness changed from -2098.7340231672997 to -2098.7414085589485
Iteration 12 complete. Error: -2098.7414085589485 Total: 2.8504; Orientation: 0.2753; Line Search: 1.7183
Adding measurement 6114e04d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.427e+04, gradient 1.463e-01, dot -0.108; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2098.7414085589485, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 6114e04d to history. Total: 3
th(0)=-2098.7414085589485;dx=-0.021280249146669962
New Minimum: -2098.7414085589485 > -2098.7572513275563
END: th(1.0389827787576604)=-2098.7572513275563; dx=-0.01300696857026781 evalInputDelta=0.01584276860785394
Fitness changed from -2098.7414085589485 to -2098.7572513275563
Iteration 13 complete. Error: -2098.7572513275563 Total: 2.9355; Orientation: 0.2753; Line Search: 1.8666
Adding measurement 535da426 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.366e+04, gradient 1.463e-01, dot -0.108; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2098.7572513275563, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 535da426 to history. Total: 3
th(0)=-2098.7572513275563;dx=-0.021311361207618364
New Minimum: -2098.7572513275563 > -2098.7914379470867
END: th(2.2384205409012257)=-2098.7914379470867; dx=-0.01293558964922592 evalInputDelta=0.03418661953037372
Fitness changed from -2098.7572513275563 to -2098.7914379470867
Iteration 14 complete. Error: -2098.7914379470867 Total: 2.9212; Orientation: 0.2646; Line Search: 1.7472
Adding measurement 77630627 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.567e+04, gradient 1.462e-01, dot -0.108; [cb58a2f2-655d-4258-ab27-db95a3417cb0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2098.7914379470867, -325.6271712979431, -180.99055785109343, -101.81915647362239
LBFGS Accumulation History: 3 points
Removed measurement 77630627 to history. Total: 3
th(0)=-2098.7914379470867;dx=-0.021354358865922796
New Minimum: -2098.7914379470867 > -2098.8649313378887
END: th(4.8225308641975335)=-2098.8649313378887; dx=-0.013001304904507363 evalInputDelta=0.07349339080201389
Fitness changed from -2098.7914379470867 to -2098.8649313378887
Iteration 15 complete. Error: -2098.8649313378887 Total: 2.8067; Orientation: 0.2740; Line Search: 1.7212
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2098.8649313378887 (> -Infinity) after 62.836s (< 5400.000s)
Returns
-2098.8649313378887