BasicOptimizer.scala:89 executed in 94.23 seconds (1.351 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 285343112684100
Reset training subject: 285345118764700
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -56.070274358236354 > -56.138408441450395
th(0)=-56.138408441450395;dx=-5.442536223926597E-6
New Minimum: -56.138408441450395 > -56.138418457051095
END: th(2.154434690031884)=-56.138418457051095; dx=-3.5987738250534863E-6 evalInputDelta=1.0015600700796767E-5
Fitness changed from -56.070274358236354 to -56.138418457051095
Iteration 1 complete. Error: -56.138418457051095 Total: 9.4786; Orientation: 0.0356; Line Search: 3.7192
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5b60ae86 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-56.138418457051095;dx=-3.6076877657216714E-6
New Minimum: -56.138418457051095 > -56.13843848824714
WOLFE (weak): th(4.641588833612779)=-56.13843848824714; dx=-3.6076668393480915E-6 evalInputDelta=2.0031196044101307E-5
New Minimum: -56.13843848824714 > -56.13845184237386
WOLFE (weak): th(9.283177667225559)=-56.13845184237386; dx=-3.6076562041897847E-6 evalInputDelta=3.3385322765866476E-5
New Minimum: -56.13845184237386 > -56.13852195148706
WOLFE (weak): th(27.849533001676676)=-56.13852195148706; dx=-3.607654648557537E-6 evalInputDelta=1.0349443596169294E-4
New Minimum: -56.13852195148706 > -56.13882575663245
WOLFE (weak): th(111.3981320067067)=-56.13882575663245; dx=-3.607576868537775E-6 evalInputDelta=4.0729958135443667E-4
New Minimum: -56.13882575663245 > -56.14042821810457
WOLFE (weak): th(556.9906600335335)=-56.14042821810457; dx=-3.6073805442961737E-6 evalInputDelta=0.002009761053471948
New Minimum: -56.14042821810457 > -56.15047594435896
WOLFE (weak): th(3341.9439602012008)=-56.15047594435896; dx=-3.6065650952149E-6 evalInputDelta=0.012057487307863823
New Minimum: -56.15047594435896 > -56.22271322920217
WOLFE (weak): th(23393.607721408407)=-56.22271322920217; dx=-3.598975321228786E-6 evalInputDelta=0.08429477215107539
New Minimum: -56.22271322920217 > -56.806569215615774
WOLFE (weak): th(187148.86177126726)=-56.806569215615774; dx=-3.5315796302331833E-6 evalInputDelta=0.6681507585646784
New Minimum: -56.806569215615774 > -61.5812549100423
END: th(1684339.7559414054)=-61.5812549100423; dx=-2.834176104931298E-6 evalInputDelta=5.442836452991202
Fitness changed from -56.138418457051095 to -61.5812549100423
Iteration 2 complete. Error: -61.5812549100423 Total: 21.3762; Orientation: 0.0613; Line Search: 19.1759
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1bb21c8c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-61.5812549100423;dx=-2.546519206807127E-6
New Minimum: -61.5812549100423 > -69.09962794592349
END: th(3628800.0000000005)=-69.09962794592349; dx=-1.583654127139231E-6 evalInputDelta=7.51837303588119
Fitness changed from -61.5812549100423 to -69.09962794592349
Iteration 3 complete. Error: -69.09962794592349 Total: 6.1945; Orientation: 0.0583; Line Search: 3.9237
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement bce19ab to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-69.09962794592349;dx=-1.5450587039526957E-6
New Minimum: -69.09962794592349 > -77.8846390927847
END: th(7818012.603187701)=-77.8846390927847; dx=-7.439892356613898E-7 evalInputDelta=8.785011146861208
Fitness changed from -69.09962794592349 to -77.8846390927847
Iteration 4 complete. Error: -77.8846390927847 Total: 6.2012; Orientation: 0.0578; Line Search: 4.0073
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 16b33064 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.269e+04, gradient 1.080e-03, dot -0.715; [c363c097-efff-4f29-a34c-5c3d6d33d5ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -77.8846390927847, -69.09962794592349, -61.5812549100423, -56.138418457051095
LBFGS Accumulation History: 3 points
Removed measurement 16b33064 to history. Total: 3
th(0)=-77.8846390927847;dx=-1.166669522467353E-6
New Minimum: -77.8846390927847 > -85.61220451377979
END: th(1.6843397559414055E7)=-85.61220451377979; dx=-8.327844301102538E-8 evalInputDelta=7.7275654209950915
Fitness changed from -77.8846390927847 to -85.61220451377979
Iteration 5 complete. Error: -85.61220451377979 Total: 7.0900; Orientation: 1.0145; Line Search: 3.8764
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement a5949c9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.864e+04, gradient 1.323e-03, dot -0.451; [c363c097-efff-4f29-a34c-5c3d6d33d5ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -85.61220451377979, -69.09962794592349, -61.5812549100423, -56.138418457051095
LBFGS Accumulation History: 3 points
Removed measurement a5949c9 to history. Total: 3
th(0)=-85.61220451377979;dx=-1.7516116677854002E-6
Armijo: th(3.628800000000001E7)=-78.9931706239304; dx=4.243211363323514E-7 evalInputDelta=-6.619033889849391
New Minimum: -85.61220451377979 > -88.49195720488187
WOLF (strong): th(1.8144000000000004E7)=-88.49195720488187; dx=5.910508489624452E-7 evalInputDelta=2.879752691102084
New Minimum: -88.49195720488187 > -92.2763480109178
END: th(6048000.000000001)=-92.2763480109178; dx=-4.0125974158611593E-7 evalInputDelta=6.664143497138014
Fitness changed from -85.61220451377979 to -92.2763480109178
Iteration 6 complete. Error: -92.2763480109178 Total: 10.8799; Orientation: 1.0456; Line Search: 7.6724
Adding measurement 4f972c49 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.226e+04, gradient 9.059e-04, dot -0.607; [c363c097-efff-4f29-a34c-5c3d6d33d5ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.2763480109178, -69.09962794592349, -61.5812549100423, -56.138418457051095
LBFGS Accumulation History: 3 points
Removed measurement 4f972c49 to history. Total: 3
th(0)=-92.2763480109178;dx=-8.20572338581823E-7
New Minimum: -92.2763480109178 > -97.03250984862512
END: th(1.3030021005312836E7)=-97.03250984862512; dx=-4.2629386981734414E-8 evalInputDelta=4.7561618377073245
Fitness changed from -92.2763480109178 to -97.03250984862512
Iteration 7 complete. Error: -97.03250984862512 Total: 6.6523; Orientation: 0.9417; Line Search: 3.8419
Adding measurement 13d45722 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.516e+04, gradient 1.110e-03, dot -0.462; [c363c097-efff-4f29-a34c-5c3d6d33d5ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -97.03250984862512, -69.09962794592349, -61.5812549100423, -56.138418457051095
LBFGS Accumulation History: 3 points
Removed measurement 13d45722 to history. Total: 3
th(0)=-97.03250984862512;dx=-1.2329256173855198E-6
Armijo: th(2.8072329265690096E7)=-93.55866424947189; dx=3.7418586425658123E-7 evalInputDelta=-3.473845599153236
New Minimum: -97.03250984862512 > -99.63916815200835
WOLF (strong): th(1.4036164632845048E7)=-99.63916815200835; dx=4.36190984284111E-7 evalInputDelta=2.6066583033832273
New Minimum: -99.63916815200835 > -101.0537226811467
END: th(4678721.544281683)=-101.0537226811467; dx=-4.313198601924584E-7 evalInputDelta=4.02121283252157
Fitness changed from -97.03250984862512 to -101.0537226811467
Iteration 8 complete. Error: -101.0537226811467 Total: 10.7177; Orientation: 0.9901; Line Search: 7.8129
Adding measurement 6a37537b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.333e+04, gradient 7.217e-04, dot -0.680; [c363c097-efff-4f29-a34c-5c3d6d33d5ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -101.0537226811467, -69.09962794592349, -61.5812549100423, -56.138418457051095
LBFGS Accumulation History: 3 points
Removed measurement 6a37537b to history. Total: 3
th(0)=-101.0537226811467;dx=-5.208655847950937E-7
New Minimum: -101.0537226811467 > -104.76765121493024
END: th(1.0080000000000004E7)=-104.76765121493024; dx=-2.4206969061254033E-7 evalInputDelta=3.7139285337835446
Fitness changed from -101.0537226811467 to -104.76765121493024
Iteration 9 complete. Error: -104.76765121493024 Total: 6.6840; Orientation: 1.0367; Line Search: 3.8244
Adding measurement 72765914 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.588e+04, gradient 7.815e-04, dot -0.595; [c363c097-efff-4f29-a34c-5c3d6d33d5ac = 1.000/1.000e+00]
Orientation rejected. Popping history element from -104.76765121493024, -69.09962794592349, -61.5812549100423, -56.138418457051095
LBFGS Accumulation History: 3 points
Removed measurement 72765914 to history. Total: 3
th(0)=-104.76765121493024;dx=-6.107229476048953E-7
New Minimum: -104.76765121493024 > -107.21923845148352
WOLF (strong): th(2.1716701675521396E7)=-107.21923845148352; dx=9.702801679528869E-8 evalInputDelta=2.45158723655328
New Minimum: -107.21923845148352 > -107.71857555314313
END: th(1.0858350837760698E7)=-107.71857555314313; dx=-3.695629796495527E-8 evalInputDelta=2.950924338212886
Fitness changed from -104.76765121493024 to -107.71857555314313
Iteration 10 complete. Error: -107.71857555314313 Total: 8.6182; Orientation: 1.0541; Line Search: 5.7837
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -107.71857555314313 (> -Infinity) after 94.228s (< 5400.000s)
Returns
-107.71857555314313