BasicOptimizer.scala:89 executed in 293.48 seconds (3.332 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 582408029265900
Reset training subject: 582414119233500
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -36.889303178270396 > -36.91278555271515
th(0)=-36.91278555271515;dx=-2.024377119932358E-6
New Minimum: -36.91278555271515 > -36.912787598815086
END: th(2.154434690031884)=-36.912787598815086; dx=-1.7811652436264918E-6 evalInputDelta=2.046099936592327E-6
Fitness changed from -36.889303178270396 to -36.912787598815086
Iteration 1 complete. Error: -36.912787598815086 Total: 29.3693; Orientation: 0.0921; Line Search: 11.7399
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3ad822d6 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-36.912787598815086;dx=-1.7962365676999567E-6
New Minimum: -36.912787598815086 > -36.912799875412304
WOLFE (weak): th(4.641588833612779)=-36.912799875412304; dx=-1.7949513083683358E-6 evalInputDelta=1.2276597217919516E-5
New Minimum: -36.912799875412304 > -36.91281010590687
WOLFE (weak): th(9.283177667225559)=-36.91281010590687; dx=-1.7937767635426197E-6 evalInputDelta=2.2507091784973454E-5
New Minimum: -36.91281010590687 > -36.91284488956718
WOLFE (weak): th(27.849533001676676)=-36.91284488956718; dx=-1.7930025047342872E-6 evalInputDelta=5.7290752096150754E-5
New Minimum: -36.91284488956718 > -36.913010623027944
WOLFE (weak): th(111.3981320067067)=-36.913010623027944; dx=-1.7921639114723945E-6 evalInputDelta=2.23024212857581E-4
New Minimum: -36.913010623027944 > -36.91388838413557
WOLFE (weak): th(556.9906600335335)=-36.91388838413557; dx=-1.7910847147082065E-6 evalInputDelta=0.0011007853204816342
New Minimum: -36.91388838413557 > -36.919379544018994
WOLFE (weak): th(3341.9439602012008)=-36.919379544018994; dx=-1.7916352195092043E-6 evalInputDelta=0.006591945203908267
New Minimum: -36.919379544018994 > -36.958890176517386
WOLFE (weak): th(23393.607721408407)=-36.958890176517386; dx=-1.7893338515547683E-6 evalInputDelta=0.04610257770229964
New Minimum: -36.958890176517386 > -37.279665450655145
WOLFE (weak): th(187148.86177126726)=-37.279665450655145; dx=-1.7696550504826387E-6 evalInputDelta=0.3668778518400586
New Minimum: -37.279665450655145 > -40.01869154924375
END: th(1684339.7559414054)=-40.01869154924375; dx=-1.5476499615785132E-6 evalInputDelta=3.1059039504286616
Fitness changed from -36.912787598815086 to -40.01869154924375
Iteration 2 complete. Error: -40.01869154924375 Total: 67.5236; Orientation: 0.1413; Line Search: 60.0404
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 51769a6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-40.01869154924375;dx=-1.507093322222779E-6
New Minimum: -40.01869154924375 > -45.00902628867295
END: th(3628800.0000000005)=-45.00902628867295; dx=-1.041357828629681E-6 evalInputDelta=4.990334739429201
Fitness changed from -40.01869154924375 to -45.00902628867295
Iteration 3 complete. Error: -45.00902628867295 Total: 19.0288; Orientation: 0.1355; Line Search: 11.7685
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 182bcc69 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-45.00902628867295;dx=-1.0123474903214091E-6
New Minimum: -45.00902628867295 > -51.77722017911258
END: th(7818012.603187701)=-51.77722017911258; dx=-5.940549933572366E-7 evalInputDelta=6.768193890439633
Fitness changed from -45.00902628867295 to -51.77722017911258
Iteration 4 complete. Error: -51.77722017911258 Total: 19.2287; Orientation: 0.1394; Line Search: 12.0597
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4fcfd5b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.872e+04, gradient 8.860e-04, dot -0.724; [7ce83fd5-c04a-40e4-9ce5-009b8701c457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -51.77722017911258, -45.00902628867295, -40.01869154924375, -36.912787598815086
LBFGS Accumulation History: 3 points
Removed measurement 4fcfd5b0 to history. Total: 3
th(0)=-51.77722017911258;dx=-7.848148638192978E-7
New Minimum: -51.77722017911258 > -59.60749030571547
END: th(1.6843397559414055E7)=-59.60749030571547; dx=-2.3122955341533899E-7 evalInputDelta=7.8302701266028905
Fitness changed from -51.77722017911258 to -59.60749030571547
Iteration 5 complete. Error: -59.60749030571547 Total: 21.7144; Orientation: 2.5406; Line Search: 12.1179
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4ecdf640 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.989e+04, gradient 9.927e-04, dot -0.491; [7ce83fd5-c04a-40e4-9ce5-009b8701c457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -59.60749030571547, -45.00902628867295, -40.01869154924375, -36.912787598815086
LBFGS Accumulation History: 3 points
Removed measurement 4ecdf640 to history. Total: 3
th(0)=-59.60749030571547;dx=-9.849581417984702E-7
New Minimum: -59.60749030571547 > -61.75092636610593
WOLF (strong): th(3.628800000000001E7)=-61.75092636610593; dx=1.206654567314686E-7 evalInputDelta=2.1434360603904565
New Minimum: -61.75092636610593 > -64.87730318822547
WOLF (strong): th(1.8144000000000004E7)=-64.87730318822547; dx=1.1020563459662802E-7 evalInputDelta=5.269812882509996
END: th(6048000.000000001)=-64.32498898759269; dx=-4.240445939886137E-7 evalInputDelta=4.717498681877217
Fitness changed from -59.60749030571547 to -64.87730318822547
Iteration 6 complete. Error: -64.87730318822547 Total: 33.9068; Orientation: 2.5024; Line Search: 24.4380
Adding measurement 799f7150 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.266e+04, gradient 1.078e-03, dot -0.371; [7ce83fd5-c04a-40e4-9ce5-009b8701c457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -64.87730318822547, -45.00902628867295, -40.01869154924375, -36.912787598815086
LBFGS Accumulation History: 3 points
Removed measurement 799f7150 to history. Total: 3
th(0)=-64.87730318822547;dx=-1.1622237716823583E-6
New Minimum: -64.87730318822547 > -73.23587888583363
WOLF (strong): th(1.3030021005312836E7)=-73.23587888583363; dx=2.1776630578153348E-8 evalInputDelta=8.35857569760816
END: th(6515010.502656418)=-71.39970016954548; dx=-6.021665179813217E-7 evalInputDelta=6.522396981320014
Fitness changed from -64.87730318822547 to -73.23587888583363
Iteration 7 complete. Error: -73.23587888583363 Total: 26.5456; Orientation: 2.3985; Line Search: 18.1559
Adding measurement 332a370e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.162e+04, gradient 8.280e-04, dot -0.505; [7ce83fd5-c04a-40e4-9ce5-009b8701c457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -73.23587888583363, -45.00902628867295, -40.01869154924375, -36.912787598815086
LBFGS Accumulation History: 3 points
Removed measurement 332a370e to history. Total: 3
th(0)=-73.23587888583363;dx=-6.852008779964645E-7
New Minimum: -73.23587888583363 > -78.03773742694798
END: th(1.4036164632845048E7)=-78.03773742694798; dx=-1.044060578636882E-8 evalInputDelta=4.801858541114356
Fitness changed from -73.23587888583363 to -78.03773742694798
Iteration 8 complete. Error: -78.03773742694798 Total: 21.1463; Orientation: 2.5865; Line Search: 12.2953
Adding measurement a400815 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.916e+04, gradient 8.140e-04, dot -0.489; [7ce83fd5-c04a-40e4-9ce5-009b8701c457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -78.03773742694798, -45.00902628867295, -40.01869154924375, -36.912787598815086
LBFGS Accumulation History: 3 points
Removed measurement a400815 to history. Total: 3
th(0)=-78.03773742694798;dx=-6.613964597717973E-7
New Minimum: -78.03773742694798 > -79.26896067301224
WOLF (strong): th(3.024000000000001E7)=-79.26896067301224; dx=1.9923764954601435E-7 evalInputDelta=1.2312232460642605
New Minimum: -79.26896067301224 > -82.26809701915978
WOLF (strong): th(1.5120000000000006E7)=-82.26809701915978; dx=5.8089298831405554E-8 evalInputDelta=4.230359592211798
END: th(5040000.000000002)=-80.97452719672027; dx=-3.907823521063043E-7 evalInputDelta=2.9367897697722896
Fitness changed from -78.03773742694798 to -82.26809701915978
Iteration 9 complete. Error: -82.26809701915978 Total: 32.9208; Orientation: 2.4617; Line Search: 24.5858
Adding measurement 7df4e846 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.504e+04, gradient 8.223e-04, dot -0.458; [7ce83fd5-c04a-40e4-9ce5-009b8701c457 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -82.26809701915978, -45.00902628867295, -40.01869154924375, -36.912787598815086
LBFGS Accumulation History: 3 points
Removed measurement 7df4e846 to history. Total: 3
th(0)=-82.26809701915978;dx=-6.756357109765179E-7
New Minimum: -82.26809701915978 > -86.81775332064772
END: th(1.0858350837760698E7)=-86.81775332064772; dx=-7.97166724866686E-8 evalInputDelta=4.549656301487943
Fitness changed from -82.26809701915978 to -86.81775332064772
Iteration 10 complete. Error: -86.81775332064772 Total: 20.9678; Orientation: 2.4516; Line Search: 12.3538
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -86.81775332064772 (> -Infinity) after 293.483s (< 5400.000s)
Returns
-86.81775332064772