BasicOptimizer.scala:89 executed in 11.12 seconds (0.167 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 566381920252700
Reset training subject: 566382113388500
Adding measurement 3eab64ce to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.7615940269180341;dx=-1.1668159440999543E-5
New Minimum: -1.7615940269180341 > -1.7616483807063867
WOLFE (weak): th(2.154434690031884)=-1.7616483807063867; dx=-1.167740410915351E-5 evalInputDelta=5.43537883526124E-5
New Minimum: -1.7616483807063867 > -1.7617027328177624
WOLFE (weak): th(4.308869380063768)=-1.7617027328177624; dx=-1.167731068666665E-5 evalInputDelta=1.0870589972822309E-4
New Minimum: -1.7617027328177624 > -1.7619202030292127
WOLFE (weak): th(12.926608140191302)=-1.7619202030292127; dx=-1.16781913417962E-5 evalInputDelta=3.261761111785777E-4
New Minimum: -1.7619202030292127 > -1.7628989971751443
WOLFE (weak): th(51.70643256076521)=-1.7628989971751443; dx=-1.170154237620504E-5 evalInputDelta=0.0013049702571101829
New Minimum: -1.7628989971751443 > -1.7681162116925242
WOLFE (weak): th(258.53216280382605)=-1.7681162116925242; dx=-1.1672287808172966E-5 evalInputDelta=0.00652218477449007
New Minimum: -1.7681162116925242 > -1.8009720477922133
WOLFE (weak): th(1551.1929768229563)=-1.8009720477922133; dx=-1.1827324002429756E-5 evalInputDelta=0.03937802087417919
New Minimum: -1.8009720477922133 > -2.0434415367518453
WOLFE (weak): th(10858.350837760694)=-2.0434415367518453; dx=-1.1969490643226819E-5 evalInputDelta=0.2818475098338111
New Minimum: -2.0434415367518453 > -3.8148397835963315
END: th(86866.80670208555)=-3.8148397835963315; dx=-6.999326810877845E-6 evalInputDelta=2.0532457566782973
Fitness changed from -1.7615940269180341 to -3.8148397835963315
Iteration 1 complete. Error: -3.8148397835963315 Total: 1.9492; Orientation: 0.0026; Line Search: 1.4399
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 477db20f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.8148397835963315;dx=-1.4235938901305839E-5
New Minimum: -3.8148397835963315 > -6.559582866277024
END: th(187148.86177126726)=-6.559582866277024; dx=-1.6397479732840987E-6 evalInputDelta=2.7447430826806927
Fitness changed from -3.8148397835963315 to -6.559582866277024
Iteration 2 complete. Error: -6.559582866277024 Total: 0.4955; Orientation: 0.0029; Line Search: 0.3150
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7c04e9e3 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.559582866277024;dx=-1.2503398572543826E-5
New Minimum: -6.559582866277024 > -8.283430800745842
WOLF (strong): th(403200.00000000006)=-8.283430800745842; dx=1.2430670607334411E-6 evalInputDelta=1.723847934468818
New Minimum: -8.283430800745842 > -8.875485107980994
END: th(201600.00000000003)=-8.875485107980994; dx=-2.6046960202284685E-7 evalInputDelta=2.31590224170397
Fitness changed from -6.559582866277024 to -8.875485107980994
Iteration 3 complete. Error: -8.875485107980994 Total: 0.8685; Orientation: 0.0023; Line Search: 0.6924
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 598aa68e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.145e+03, gradient 3.430e-03, dot -0.467; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.875485107980994, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 598aa68e to history. Total: 3
th(0)=-8.875485107980994;dx=-1.172777768874733E-5
New Minimum: -8.875485107980994 > -9.951938046651856
WOLF (strong): th(434334.03351042786)=-9.951938046651856; dx=1.9208492738417032E-6 evalInputDelta=1.0764529386708617
New Minimum: -9.951938046651856 > -11.1259123444463
WOLF (strong): th(217167.01675521393)=-11.1259123444463; dx=6.195174671546997E-7 evalInputDelta=2.250427236465306
END: th(72389.00558507131)=-10.461099293575993; dx=-6.7975795659909915E-6 evalInputDelta=1.5856141855949986
Fitness changed from -8.875485107980994 to -11.1259123444463
Iteration 4 complete. Error: -11.1259123444463 Total: 0.8454; Orientation: 0.0415; Line Search: 0.6304
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 589eadc6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.220e+03, gradient 3.298e-03, dot -0.421; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.1259123444463, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 589eadc6 to history. Total: 3
th(0)=-11.1259123444463;dx=-1.0808448451905607E-5
New Minimum: -11.1259123444463 > -13.406850865570116
END: th(155957.38480938942)=-13.406850865570116; dx=-2.278054101481386E-6 evalInputDelta=2.280938521123815
Fitness changed from -11.1259123444463 to -13.406850865570116
Iteration 5 complete. Error: -13.406850865570116 Total: 0.5390; Orientation: 0.0453; Line Search: 0.3148
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 56cc60ca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.344e+03, gradient 2.567e-03, dot -0.595; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.406850865570116, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 56cc60ca to history. Total: 3
th(0)=-13.406850865570116;dx=-6.560544799725698E-6
New Minimum: -13.406850865570116 > -15.830349269018566
END: th(336000.0000000001)=-15.830349269018566; dx=-1.753539635465787E-6 evalInputDelta=2.42349840344845
Fitness changed from -13.406850865570116 to -15.830349269018566
Iteration 6 complete. Error: -15.830349269018566 Total: 0.5241; Orientation: 0.0412; Line Search: 0.3137
Adding measurement 1179cbdf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.414e+04, gradient 3.380e-03, dot -0.497; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.830349269018566, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 1179cbdf to history. Total: 3
th(0)=-15.830349269018566;dx=-1.1398328779512062E-5
New Minimum: -15.830349269018566 > -16.382009140967835
WOLF (strong): th(723890.0558507132)=-16.382009140967835; dx=1.8538517784087166E-6 evalInputDelta=0.5516598719492691
New Minimum: -16.382009140967835 > -18.20615209643752
WOLF (strong): th(361945.0279253566)=-18.20615209643752; dx=1.7073518107629887E-6 evalInputDelta=2.375802827418955
END: th(120648.34264178552)=-17.98919026093178; dx=-5.046183243584274E-6 evalInputDelta=2.158840991913216
Fitness changed from -15.830349269018566 to -18.20615209643752
Iteration 7 complete. Error: -18.20615209643752 Total: 0.8345; Orientation: 0.0427; Line Search: 0.6333
Adding measurement 3b308de1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.248e+04, gradient 3.858e-03, dot -0.419; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.20615209643752, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 3b308de1 to history. Total: 3
th(0)=-18.20615209643752;dx=-1.4818371206827205E-5
New Minimum: -18.20615209643752 > -22.623020429354717
END: th(259928.9746823157)=-22.623020429354717; dx=-8.988286537310265E-7 evalInputDelta=4.416868332917197
Fitness changed from -18.20615209643752 to -22.623020429354717
Iteration 8 complete. Error: -22.623020429354717 Total: 0.5133; Orientation: 0.0396; Line Search: 0.3145
Adding measurement 66ed35a9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.658e+04, gradient 2.917e-03, dot -0.595; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.623020429354717, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 66ed35a9 to history. Total: 3
th(0)=-22.623020429354717;dx=-8.473085034526307E-6
New Minimum: -22.623020429354717 > -26.322023393475114
WOLF (strong): th(560000.0000000002)=-26.322023393475114; dx=1.960135567641623E-7 evalInputDelta=3.699002964120396
END: th(280000.0000000001)=-25.613439998924164; dx=-2.3797075497125635E-6 evalInputDelta=2.990419569569447
Fitness changed from -22.623020429354717 to -26.322023393475114
Iteration 9 complete. Error: -26.322023393475114 Total: 0.6760; Orientation: 0.0407; Line Search: 0.4754
Adding measurement 5b5cd01d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.924e+04, gradient 3.863e-03, dot -0.431; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.322023393475114, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 5b5cd01d to history. Total: 3
th(0)=-26.322023393475114;dx=-1.4888017124989923E-5
New Minimum: -26.322023393475114 > -29.98694473158793
WOLF (strong): th(603241.7132089278)=-29.98694473158793; dx=4.454594371872049E-6 evalInputDelta=3.6649213381128156
New Minimum: -29.98694473158793 > -31.367380052872747
END: th(301620.8566044639)=-31.367380052872747; dx=-1.3179186463968474E-6 evalInputDelta=5.045356659397633
Fitness changed from -26.322023393475114 to -31.367380052872747
Iteration 10 complete. Error: -31.367380052872747 Total: 0.6587; Orientation: 0.0404; Line Search: 0.4681
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 54ca5216 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.358e+05, gradient 2.761e-03, dot -0.615; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.367380052872747, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 54ca5216 to history. Total: 3
th(0)=-31.367380052872747;dx=-7.583471467762661E-6
New Minimum: -31.367380052872747 > -35.71472664010252
END: th(649822.4367057894)=-35.71472664010252; dx=-3.9888087102911514E-7 evalInputDelta=4.347346587229772
Fitness changed from -31.367380052872747 to -35.71472664010252
Iteration 11 complete. Error: -35.71472664010252 Total: 0.5318; Orientation: 0.0450; Line Search: 0.3119
Adding measurement 3bcc11bf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.422e+04, gradient 3.516e-03, dot -0.439; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.71472664010252, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 3bcc11bf to history. Total: 3
th(0)=-35.71472664010252;dx=-1.2317395815999235E-5
Armijo: th(1400000.000000001)=-33.87721542522558; dx=3.70325241731156E-6 evalInputDelta=-1.8375112148769404
New Minimum: -35.71472664010252 > -39.845372571089285
WOLF (strong): th(700000.0000000005)=-39.845372571089285; dx=3.6306727259262346E-6 evalInputDelta=4.130645930986766
New Minimum: -39.845372571089285 > -40.10115584877626
END: th(233333.3333333335)=-40.10115584877626; dx=-5.010966928477276E-6 evalInputDelta=4.386429208673739
Fitness changed from -35.71472664010252 to -40.10115584877626
Iteration 12 complete. Error: -40.10115584877626 Total: 0.8240; Orientation: 0.0396; Line Search: 0.6247
Adding measurement a7b783a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.587e+04, gradient 2.269e-03, dot -0.668; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.10115584877626, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement a7b783a to history. Total: 3
th(0)=-40.10115584877626;dx=-5.122030803168797E-6
New Minimum: -40.10115584877626 > -44.231386526617946
END: th(502701.42767410656)=-44.231386526617946; dx=-2.587515847908953E-6 evalInputDelta=4.130230677841688
Fitness changed from -40.10115584877626 to -44.231386526617946
Iteration 13 complete. Error: -44.231386526617946 Total: 0.5059; Orientation: 0.0391; Line Search: 0.3111
Adding measurement 56e35aee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.671e+04, gradient 2.408e-03, dot -0.585; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -44.231386526617946, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 56e35aee to history. Total: 3
th(0)=-44.231386526617946;dx=-5.7519956536587845E-6
New Minimum: -44.231386526617946 > -48.70173183650648
END: th(1083037.3945096491)=-48.70173183650648; dx=-1.0248347542752087E-7 evalInputDelta=4.470345309888536
Fitness changed from -44.231386526617946 to -48.70173183650648
Iteration 14 complete. Error: -48.70173183650648 Total: 0.5001; Orientation: 0.0408; Line Search: 0.3051
Adding measurement 76369301 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.865e+04, gradient 3.085e-03, dot -0.370; [43a6b806-9115-4092-8d2a-9c6cbb9b7344 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.70173183650648, -6.559582866277024, -3.8148397835963315, -1.7615940269180341
LBFGS Accumulation History: 3 points
Removed measurement 76369301 to history. Total: 3
th(0)=-48.70173183650648;dx=-9.489457063470686E-6
Armijo: th(2333333.333333335)=-44.076775248319514; dx=3.053659298647487E-6 evalInputDelta=-4.6249565881869685
New Minimum: -48.70173183650648 > -53.35033715991711
WOLF (strong): th(1166666.6666666674)=-53.35033715991711; dx=3.5372173104921186E-6 evalInputDelta=4.648605323410628
New Minimum: -53.35033715991711 > -54.388531190354044
END: th(388888.88888888917)=-54.388531190354044; dx=-3.7904766791731003E-6 evalInputDelta=5.686799353847562
Fitness changed from -48.70173183650648 to -54.388531190354044
Iteration 15 complete. Error: -54.388531190354044 Total: 0.8327; Orientation: 0.0391; Line Search: 0.6397
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -54.388531190354044 (> -Infinity) after 11.116s (< 5400.000s)
Returns
-54.388531190354044