BasicOptimizer.scala:89 executed in 8.11 seconds (0.239 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 151180617083100
Reset training subject: 151180749479700
Adding measurement 54b71236 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.410228586598917;dx=-1.8661601152059268E-4
New Minimum: -2.410228586598917 > -2.410682145940727
WOLFE (weak): th(2.154434690031884)=-2.410682145940727; dx=-1.883308770242855E-4 evalInputDelta=4.53559341810017E-4
New Minimum: -2.410682145940727 > -2.411136567573176
WOLFE (weak): th(4.308869380063768)=-2.411136567573176; dx=-1.85617506128006E-4 evalInputDelta=9.079809742589262E-4
New Minimum: -2.411136567573176 > -2.4129556708590734
WOLFE (weak): th(12.926608140191302)=-2.4129556708590734; dx=-1.8786440690081528E-4 evalInputDelta=0.0027270842601563317
New Minimum: -2.4129556708590734 > -2.4211798333154393
WOLFE (weak): th(51.70643256076521)=-2.4211798333154393; dx=-1.886240808776253E-4 evalInputDelta=0.010951246716522256
New Minimum: -2.4211798333154393 > -2.465369087431568
WOLFE (weak): th(258.53216280382605)=-2.465369087431568; dx=-1.8915092695564523E-4 evalInputDelta=0.055140500832651096
New Minimum: -2.465369087431568 > -2.7391607165116314
WOLFE (weak): th(1551.1929768229563)=-2.7391607165116314; dx=-1.7974887856351594E-4 evalInputDelta=0.32893212991271437
New Minimum: -2.7391607165116314 > -3.983446674881309
END: th(10858.350837760694)=-3.983446674881309; dx=-6.585873837627567E-5 evalInputDelta=1.5732180882823918
Fitness changed from -2.410228586598917 to -3.983446674881309
Iteration 1 complete. Error: -3.983446674881309 Total: 1.2524; Orientation: 0.0014; Line Search: 0.9026
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7d0c122a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.983446674881309;dx=-1.849738376647207E-4
New Minimum: -3.983446674881309 > -4.591151725688925
END: th(23393.607721408407)=-4.591151725688925; dx=-6.397426439430182E-6 evalInputDelta=0.607705050807616
Fitness changed from -3.983446674881309 to -4.591151725688925
Iteration 2 complete. Error: -4.591151725688925 Total: 0.3500; Orientation: 0.0015; Line Search: 0.2256
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7b17b7c to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.591151725688925;dx=-1.9668932067509368E-4
New Minimum: -4.591151725688925 > -5.214653583613771
WOLF (strong): th(50400.00000000001)=-5.214653583613771; dx=1.3010464394972004E-5 evalInputDelta=0.6235018579248459
New Minimum: -5.214653583613771 > -6.337998389497418
WOLF (strong): th(25200.000000000004)=-6.337998389497418; dx=8.099036551522114E-6 evalInputDelta=1.7468466638084932
END: th(8400.000000000002)=-6.13469892628542; dx=-8.460239679557849E-5 evalInputDelta=1.5435472005964952
Fitness changed from -4.591151725688925 to -6.337998389497418
Iteration 3 complete. Error: -6.337998389497418 Total: 0.8459; Orientation: 0.0015; Line Search: 0.7250
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 513555c2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.906e+02, gradient 1.374e-02, dot -0.331; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.337998389497418, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 513555c2 to history. Total: 3
th(0)=-6.337998389497418;dx=-1.7565873887108598E-4
New Minimum: -6.337998389497418 > -8.203737677407613
END: th(18097.251396267828)=-8.203737677407613; dx=-2.946343313686725E-5 evalInputDelta=1.8657392879101948
Fitness changed from -6.337998389497418 to -8.203737677407613
Iteration 4 complete. Error: -8.203737677407613 Total: 0.3923; Orientation: 0.0317; Line Search: 0.2359
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7661bc28 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.433e+02, gradient 1.079e-02, dot -0.397; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.203737677407613, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 7661bc28 to history. Total: 3
th(0)=-8.203737677407613;dx=-1.091116121869245E-4
New Minimum: -8.203737677407613 > -9.902549162556774
END: th(38989.346202347355)=-9.902549162556774; dx=-8.340052233631858E-6 evalInputDelta=1.698811485149161
Fitness changed from -8.203737677407613 to -9.902549162556774
Iteration 5 complete. Error: -9.902549162556774 Total: 0.4148; Orientation: 0.0325; Line Search: 0.2467
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 407b1c47 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.366e+02, gradient 1.363e-02, dot -0.301; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.902549162556774, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 407b1c47 to history. Total: 3
th(0)=-9.902549162556774;dx=-1.810134036028396E-4
Armijo: th(84000.00000000003)=-7.310010778032766; dx=1.3628015069492331E-5 evalInputDelta=-2.5925383845240075
New Minimum: -9.902549162556774 > -10.575909123939185
WOLF (strong): th(42000.000000000015)=-10.575909123939185; dx=1.7889852639421616E-5 evalInputDelta=0.6733599613824115
New Minimum: -10.575909123939185 > -10.958839744286474
END: th(14000.000000000005)=-10.958839744286474; dx=-5.151765442611982E-6 evalInputDelta=1.0562905817296997
Fitness changed from -9.902549162556774 to -10.958839744286474
Iteration 6 complete. Error: -10.958839744286474 Total: 0.6347; Orientation: 0.0323; Line Search: 0.4813
Adding measurement 4606bd28 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.063e+03, gradient 1.208e-02, dot -0.322; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.958839744286474, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 4606bd28 to history. Total: 3
th(0)=-10.958839744286474;dx=-1.3911650673481028E-4
Armijo: th(30162.085660446384)=-10.761498997362205; dx=1.8751445722349502E-5 evalInputDelta=-0.19734074692426873
New Minimum: -10.958839744286474 > -11.936797446904668
WOLF (strong): th(15081.042830223192)=-11.936797446904668; dx=4.572792477883053E-6 evalInputDelta=0.9779577026181947
END: th(5027.014276741064)=-11.72356022346703; dx=-9.505922631887508E-5 evalInputDelta=0.7647204791805571
Fitness changed from -10.958839744286474 to -11.936797446904668
Iteration 7 complete. Error: -11.936797446904668 Total: 0.5887; Orientation: 0.0275; Line Search: 0.4502
Adding measurement 707b833e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.583e+03, gradient 1.123e-02, dot -0.374; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.936797446904668, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 707b833e to history. Total: 3
th(0)=-11.936797446904668;dx=-1.1707331669224632E-4
New Minimum: -11.936797446904668 > -12.788261873361021
END: th(10830.37394509649)=-12.788261873361021; dx=-1.686156416053679E-5 evalInputDelta=0.8514644264563529
Fitness changed from -11.936797446904668 to -12.788261873361021
Iteration 8 complete. Error: -12.788261873361021 Total: 0.3603; Orientation: 0.0288; Line Search: 0.2244
Adding measurement d3be208 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.874e+03, gradient 9.490e-03, dot -0.440; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.788261873361021, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement d3be208 to history. Total: 3
th(0)=-12.788261873361021;dx=-8.432137641861516E-5
Armijo: th(23333.333333333347)=-12.565698620945064; dx=8.726145757607092E-6 evalInputDelta=-0.22256325241595754
New Minimum: -12.788261873361021 > -13.183760159791213
END: th(11666.666666666673)=-13.183760159791213; dx=-2.3115804386153377E-5 evalInputDelta=0.39549828643019147
Fitness changed from -12.788261873361021 to -13.183760159791213
Iteration 9 complete. Error: -13.183760159791213 Total: 0.4798; Orientation: 0.0268; Line Search: 0.3436
Adding measurement 6e068215 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.249e+03, gradient 9.533e-03, dot -0.457; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.183760159791213, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 6e068215 to history. Total: 3
th(0)=-13.183760159791213;dx=-8.54311202015818E-5
New Minimum: -13.183760159791213 > -13.778884865614577
END: th(25135.071383705326)=-13.778884865614577; dx=-4.659178531308063E-6 evalInputDelta=0.5951247058233644
Fitness changed from -13.183760159791213 to -13.778884865614577
Iteration 10 complete. Error: -13.778884865614577 Total: 0.3653; Orientation: 0.0270; Line Search: 0.2262
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 591060d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.691e+03, gradient 1.219e-02, dot -0.350; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.778884865614577, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 591060d6 to history. Total: 3
th(0)=-13.778884865614577;dx=-1.410547813230987E-4
Armijo: th(54151.86972548246)=-10.076837637914014; dx=3.817477245276302E-5 evalInputDelta=-3.7020472277005627
Armijo: th(27075.93486274123)=-13.338625201379068; dx=4.6504891902255976E-5 evalInputDelta=-0.44025966423550855
New Minimum: -13.778884865614577 > -14.787590050775535
END: th(9025.311620913742)=-14.787590050775535; dx=-3.506439342786772E-5 evalInputDelta=1.0087051851609576
Fitness changed from -13.778884865614577 to -14.787590050775535
Iteration 11 complete. Error: -14.787590050775535 Total: 0.6077; Orientation: 0.0287; Line Search: 0.4602
Adding measurement 3683b166 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.332e+03, gradient 8.508e-03, dot -0.521; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.787590050775535, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 3683b166 to history. Total: 3
th(0)=-14.787590050775535;dx=-6.778647894644001E-5
New Minimum: -14.787590050775535 > -15.382444098653393
END: th(19444.444444444456)=-15.382444098653393; dx=-3.41043537588844E-5 evalInputDelta=0.5948540478778579
Fitness changed from -14.787590050775535 to -15.382444098653393
Iteration 12 complete. Error: -15.382444098653393 Total: 0.3707; Orientation: 0.0285; Line Search: 0.2270
Adding measurement 4e3d3e5d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.004e+03, gradient 9.101e-03, dot -0.470; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.382444098653393, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 4e3d3e5d to history. Total: 3
th(0)=-15.382444098653393;dx=-7.750038735054326E-5
Armijo: th(41891.785639508875)=-14.973624477139792; dx=1.430666788511496E-5 evalInputDelta=-0.40881962151360085
New Minimum: -15.382444098653393 > -15.900139519943595
END: th(20945.892819754437)=-15.900139519943595; dx=-1.70013743659066E-6 evalInputDelta=0.5176954212902025
Fitness changed from -15.382444098653393 to -15.900139519943595
Iteration 13 complete. Error: -15.900139519943595 Total: 0.4769; Orientation: 0.0275; Line Search: 0.3394
Adding measurement 43c91b04 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.751e+03, gradient 1.165e-02, dot -0.388; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.900139519943595, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 43c91b04 to history. Total: 3
th(0)=-15.900139519943595;dx=-1.3014742420840652E-4
Armijo: th(45126.558104568714)=-15.490248532609865; dx=1.7607772150853703E-5 evalInputDelta=-0.4098909873337302
New Minimum: -15.900139519943595 > -16.343752594860973
WOLF (strong): th(22563.279052284357)=-16.343752594860973; dx=2.857231188912518E-5 evalInputDelta=0.4436130749173781
New Minimum: -16.343752594860973 > -16.81335117493841
END: th(7521.093017428119)=-16.81335117493841; dx=-4.278357593547432E-5 evalInputDelta=0.9132116549948162
Fitness changed from -15.900139519943595 to -16.81335117493841
Iteration 14 complete. Error: -16.81335117493841 Total: 0.5868; Orientation: 0.0270; Line Search: 0.4509
Adding measurement 7cc68d5f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.138e+03, gradient 7.867e-03, dot -0.545; [c45b095c-4d0d-4327-93cc-91279e4a45df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.81335117493841, -4.591151725688925, -3.983446674881309, -2.410228586598917
LBFGS Accumulation History: 3 points
Removed measurement 7cc68d5f to history. Total: 3
th(0)=-16.81335117493841;dx=-5.669616731871919E-5
New Minimum: -16.81335117493841 > -17.384845532494317
END: th(16203.703703703715)=-17.384845532494317; dx=-3.958880991291081E-5 evalInputDelta=0.5714943575559062
Fitness changed from -16.81335117493841 to -17.384845532494317
Iteration 15 complete. Error: -17.384845532494317 Total: 0.3759; Orientation: 0.0279; Line Search: 0.2372
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -17.384845532494317 (> -Infinity) after 8.113s (< 5400.000s)
Returns
-17.384845532494317