BasicOptimizer.scala:89 executed in 235.09 seconds (2.572 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 8593999222084
Reset training subject: 8599472782949
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
th(0)=-14.01494762301445;dx=-4.091728659171109E-6
WOLFE (weak): th(2.154434690031884)=-14.014957219362259; dx=-3.824989760746793E-6 evalInputDelta=9.59634780883789E-6
WOLFE (weak): th(4.308869380063768)=-14.01496759057045; dx=-3.82590536955936E-6 evalInputDelta=1.996755599975586E-5
WOLFE (weak): th(12.926608140191302)=-14.01500716805458; dx=-3.825513085683908E-6 evalInputDelta=5.9545040130615234E-5
WOLFE (weak): th(51.70643256076521)=-14.015188187360764; dx=-3.825528824145571E-6 evalInputDelta=2.4056434631347656E-4
WOLFE (weak): th(258.53216280382605)=-14.016150444746017; dx=-3.822789560547188E-6 evalInputDelta=0.0012028217315673828
WOLFE (weak): th(1551.1929768229563)=-14.022159099578857; dx=-3.8171369323465965E-6 evalInputDelta=0.007211476564407349
WOLFE (weak): th(10858.350837760694)=-14.064997971057892; dx=-3.7476271797151126E-6 evalInputDelta=0.05005034804344177
New Minimum: -14.212331116199493 > -14.385092735290527
END: th(86866.80670208555)=-14.385092735290527; dx=-3.1568537557259825E-6 evalInputDelta=0.37014511227607727
Fitness changed from -14.212331116199493 to -14.385092735290527
Iteration 1 complete. Error: -14.385092735290527 Total: 61.4125; Orientation: 0.0211; Line Search: 45.6452
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement aa31f74 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-14.385092735290527;dx=-2.879728192648374E-6
New Minimum: -14.385092735290527 > -14.937229961156845
END: th(187148.86177126726)=-14.937229961156845; dx=-1.97815426840857E-6 evalInputDelta=0.5521372258663177
Fitness changed from -14.385092735290527 to -14.937229961156845
Iteration 2 complete. Error: -14.937229961156845 Total: 15.7273; Orientation: 0.0288; Line Search: 10.4900
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7ba5904b to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-14.937229961156845;dx=-1.6880076057057501E-6
New Minimum: -14.937229961156845 > -15.588094472885132
END: th(403200.00000000006)=-15.588094472885132; dx=-1.0080567650213705E-6 evalInputDelta=0.6508645117282867
Fitness changed from -14.937229961156845 to -15.588094472885132
Iteration 3 complete. Error: -15.588094472885132 Total: 15.6720; Orientation: 0.0296; Line Search: 10.3058
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1483f5ed to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-15.588094472885132;dx=-9.635385570390996E-7
New Minimum: -15.588094472885132 > -16.29670488834381
END: th(868668.0670208557)=-16.29670488834381; dx=-4.857690559842303E-7 evalInputDelta=0.7086104154586792
Fitness changed from -15.588094472885132 to -16.29670488834381
Iteration 4 complete. Error: -16.29670488834381 Total: 15.3883; Orientation: 0.0288; Line Search: 10.2587
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5df8d660 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.562e+03, gradient 8.059e-04, dot -0.780; [a2087c48-5a7f-45b7-b7fb-c069201a4d07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.29670488834381, -15.588094472885132, -14.937229961156845, -14.385092735290527
LBFGS Accumulation History: 3 points
Removed measurement 5df8d660 to history. Total: 3
th(0)=-16.29670488834381;dx=-6.487557876600094E-7
New Minimum: -16.29670488834381 > -17.020524591207504
END: th(1871488.617712673)=-17.020524591207504; dx=-1.629871574290413E-7 evalInputDelta=0.7238197028636932
Fitness changed from -16.29670488834381 to -17.020524591207504
Iteration 5 complete. Error: -17.020524591207504 Total: 16.0198; Orientation: 0.6177; Line Search: 9.8597
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 52b8de46 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.543e+03, gradient 8.131e-04, dot -0.537; [a2087c48-5a7f-45b7-b7fb-c069201a4d07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.020524591207504, -15.588094472885132, -14.937229961156845, -14.385092735290527
LBFGS Accumulation History: 3 points
Removed measurement 52b8de46 to history. Total: 3
th(0)=-17.020524591207504;dx=-6.604803618558919E-7
New Minimum: -17.020524591207504 > -17.088789522647858
WOLF (strong): th(4032000.0000000014)=-17.088789522647858; dx=1.6981493311528935E-7 evalInputDelta=0.0682649314403534
New Minimum: -17.088789522647858 > -17.42988520860672
WOLF (strong): th(2016000.0000000007)=-17.42988520860672; dx=6.097694224503249E-8 evalInputDelta=0.4093606173992157
END: th(672000.0000000002)=-17.370018005371094; dx=-2.385981710033496E-7 evalInputDelta=0.3494934141635895
Fitness changed from -17.020524591207504 to -17.42988520860672
Iteration 6 complete. Error: -17.42988520860672 Total: 26.9744; Orientation: 0.6174; Line Search: 20.8530
Adding measurement 7b592a4c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.669e+03, gradient 8.880e-04, dot -0.420; [a2087c48-5a7f-45b7-b7fb-c069201a4d07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.42988520860672, -15.588094472885132, -14.937229961156845, -14.385092735290527
LBFGS Accumulation History: 3 points
Removed measurement 7b592a4c to history. Total: 3
th(0)=-17.42988520860672;dx=-7.877276483310384E-7
New Minimum: -17.42988520860672 > -17.92203027009964
WOLF (strong): th(1447780.1117014263)=-17.92203027009964; dx=4.325856929897746E-8 evalInputDelta=0.4921450614929199
END: th(723890.0558507132)=-17.848662078380585; dx=-2.413411003903979E-7 evalInputDelta=0.41877686977386475
Fitness changed from -17.42988520860672 to -17.92203027009964
Iteration 7 complete. Error: -17.92203027009964 Total: 20.7308; Orientation: 0.6134; Line Search: 15.3664
Adding measurement e4f823e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.913e+03, gradient 6.441e-04, dot -0.479; [a2087c48-5a7f-45b7-b7fb-c069201a4d07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.92203027009964, -15.588094472885132, -14.937229961156845, -14.385092735290527
LBFGS Accumulation History: 3 points
Removed measurement e4f823e to history. Total: 3
th(0)=-17.92203027009964;dx=-4.143713149462233E-7
New Minimum: -17.92203027009964 > -18.286141872406006
END: th(1559573.8480938943)=-18.286141872406006; dx=-2.6916952770904035E-8 evalInputDelta=0.36411160230636597
Fitness changed from -17.92203027009964 to -18.286141872406006
Iteration 8 complete. Error: -18.286141872406006 Total: 15.3680; Orientation: 0.6135; Line Search: 9.7760
Adding measurement 577c39b3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.343e+03, gradient 5.300e-04, dot -0.541; [a2087c48-5a7f-45b7-b7fb-c069201a4d07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.286141872406006, -15.588094472885132, -14.937229961156845, -14.385092735290527
LBFGS Accumulation History: 3 points
Removed measurement 577c39b3 to history. Total: 3
th(0)=-18.286141872406006;dx=-2.806005871626146E-7
New Minimum: -18.286141872406006 > -18.42123317718506
WOLF (strong): th(3360000.0000000014)=-18.42123317718506; dx=1.0140310578355077E-7 evalInputDelta=0.13509130477905273
New Minimum: -18.42123317718506 > -18.548226296901703
END: th(1680000.0000000007)=-18.548226296901703; dx=-1.8354755993342387E-8 evalInputDelta=0.262084424495697
Fitness changed from -18.286141872406006 to -18.548226296901703
Iteration 9 complete. Error: -18.548226296901703 Total: 21.0015; Orientation: 0.6103; Line Search: 15.4712
Adding measurement 493b74f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.408e+03, gradient 5.106e-04, dot -0.524; [a2087c48-5a7f-45b7-b7fb-c069201a4d07 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.548226296901703, -15.588094472885132, -14.937229961156845, -14.385092735290527
LBFGS Accumulation History: 3 points
Removed measurement 493b74f7 to history. Total: 3
th(0)=-18.548226296901703;dx=-2.601724156573761E-7
Armijo: th(3619450.279253566)=-18.449749410152435; dx=1.640308948331582E-7 evalInputDelta=-0.09847688674926758
New Minimum: -18.548226296901703 > -18.741199851036072
WOLF (strong): th(1809725.139626783)=-18.741199851036072; dx=5.5972958446522564E-8 evalInputDelta=0.1929735541343689
END: th(603241.7132089277)=-18.693591058254242; dx=-1.4512433936087403E-7 evalInputDelta=0.14536476135253906
Fitness changed from -18.548226296901703 to -18.741199851036072
Iteration 10 complete. Error: -18.741199851036072 Total: 26.5183; Orientation: 0.6128; Line Search: 20.9413
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -18.741199851036072 (> -Infinity) after 235.091s (< 5400.000s)
Returns
-18.741199851036072