Code from BasicOptimizer.scala:75 executed in 122.45 seconds (2.006 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 900830240887100
Reset training subject: 900831417618400
Adding measurement 59df8444 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=3.2953154742717743;dx=-2.5035831673447433E-5
New Minimum: 3.2953154742717743 > 3.295270025730133
WOLFE (weak): th(2.154434690031884)=3.295270025730133; dx=-2.503355942736053E-5 evalInputDelta=4.544854164123535E-5
New Minimum: 3.295270025730133 > 3.295225292444229
WOLFE (weak): th(4.308869380063768)=3.295225292444229; dx=-2.5032186312334013E-5 evalInputDelta=9.018182754516602E-5
New Minimum: 3.295225292444229 > 3.2950439900159836
WOLFE (weak): th(12.926608140191302)=3.2950439900159836; dx=-2.5026142678043006E-5 evalInputDelta=2.7148425579071045E-4
New Minimum: 3.2950439900159836 > 3.2942307591438293
WOLFE (weak): th(51.70643256076521)=3.2942307591438293; dx=-2.5004696910461763E-5 evalInputDelta=0.0010847151279449463
New Minimum: 3.2942307591438293 > 3.289909526705742
WOLFE (weak): th(258.53216280382605)=3.289909526705742; dx=-2.475637804165024E-5 evalInputDelta=0.00540594756603241
New Minimum: 3.289909526705742 > 3.2640451937913895
WOLFE (weak): th(1551.1929768229563)=3.2640451937913895; dx=-2.287789762824562E-5 evalInputDelta=0.03127028048038483
New Minimum: 3.2640451937913895 > 3.098811812698841
END: th(10858.350837760694)=3.098811812698841; dx=-1.9080769009246117E-5 evalInputDelta=0.1965036615729332
Fitness changed from 3.2953154742717743 to 3.098811812698841
Iteration 1 complete. Error: 3.098811812698841 Total: 12.6478; Orientation: 0.0153; Line Search: 9.2061
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4170ee0f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=3.098811812698841;dx=-1.7211036900065558E-5
New Minimum: 3.098811812698841 > 2.7968299984931946
END: th(23393.607721408407)=2.7968299984931946; dx=-1.334966939889302E-5 evalInputDelta=0.3019818142056465
Fitness changed
...skipping 19045 bytes...
h: 4.6422
Adding measurement 411c5fa2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.583e+03, gradient 5.902e-05, dot -0.733; [0938a7fb-58be-4c91-82ea-e6661186e7a2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.23132720589637756, -0.2421588897705078, -0.2581919729709625, -0.26649209856987
LBFGS Accumulation History: 3 points
Removed measurement 33c976b1 to history. Total: 4
th(0)=-0.26649209856987;dx=-3.483572173836283E-9
New Minimum: -0.26649209856987 > -0.27393676340579987
END: th(8101851.851851857)=-0.27393676340579987; dx=-7.102209234926567E-10 evalInputDelta=0.007444664835929871
Fitness changed from -0.26649209856987 to -0.27393676340579987
Iteration 24 complete. Error: -0.27393676340579987 Total: 3.8441; Orientation: 0.4287; Line Search: 2.3045
Adding measurement 544502d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.519e+03, gradient 7.994e-05, dot -0.537; [0938a7fb-58be-4c91-82ea-e6661186e7a2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.2421588897705078, -0.2581919729709625, -0.26649209856987, -0.27393676340579987
LBFGS Accumulation History: 3 points
Removed measurement 473e0536 to history. Total: 4
th(0)=-0.27393676340579987;dx=-6.391060085673509E-9
Armijo: th(1.74549106831287E7)=-0.2222002148628235; dx=7.272016417529442E-9 evalInputDelta=-0.05173654854297638
Armijo: th(8727455.34156435)=-0.2701355218887329; dx=4.1182115206945705E-9 evalInputDelta=-0.0038012415170669556
New Minimum: -0.27393676340579987 > -0.281754732131958
END: th(2909151.78052145)=-0.281754732131958; dx=-1.370292400862851E-9 evalInputDelta=0.007817968726158142
Fitness changed from -0.27393676340579987 to -0.281754732131958
Iteration 25 complete. Error: -0.281754732131958 Total: 6.1340; Orientation: 0.4186; Line Search: 4.6043
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Final threshold in iteration 26: -0.281754732131958 (> -Infinity) after 122.446s (< 3600.000s)
-0.281754732131958