Code from BasicOptimizer.scala:75 executed in 2407.40 seconds (21.671 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1407863652293400
Reset training subject: 1407894063544800
Adding measurement 79445efb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=6.400889628256361;dx=-4.511498178141719E-5
New Minimum: 6.400889628256361 > 6.400870194037755
WOLFE (weak): th(2.154434690031884)=6.400870194037755; dx=-4.483037114668551E-5 evalInputDelta=1.9434218605951514E-5
New Minimum: 6.400870194037755 > 6.4008510534962015
WOLFE (weak): th(4.308869380063768)=6.4008510534962015; dx=-4.482892177191364E-5 evalInputDelta=3.857476015944883E-5
New Minimum: 6.4008510534962015 > 6.400773551935951
WOLFE (weak): th(12.926608140191302)=6.400773551935951; dx=-4.481605270516401E-5 evalInputDelta=1.1607632040977478E-4
New Minimum: 6.400773551935951 > 6.400425894806783
WOLFE (weak): th(51.70643256076521)=6.400425894806783; dx=-4.475517875965417E-5 evalInputDelta=4.637334495782852E-4
New Minimum: 6.400425894806783 > 6.398576867456238
WOLFE (weak): th(258.53216280382605)=6.398576867456238; dx=-4.460999873950391E-5 evalInputDelta=0.0023127608001232147
New Minimum: 6.398576867456238 > 6.387321283419927
WOLFE (weak): th(1551.1929768229563)=6.387321283419927; dx=-4.075828718122065E-5 evalInputDelta=0.01356834483643432
New Minimum: 6.387321283419927 > 6.325738343099753
END: th(10858.350837760694)=6.325738343099753; dx=-3.061535822900516E-5 evalInputDelta=0.07515128515660763
Fitness changed from 6.400889628256361 to 6.325738343099753
Iteration 1 complete. Error: 6.325738343099753 Total: 327.5089; Orientation: 0.0275; Line Search: 237.8400
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 26d24d7a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=6.325738343099753;dx=-3.054981980357473E-5
New Minimum: 6.325738343099753 > 6.2155499284466105
END: th(23393.607721408407)=6.2155499284466105; dx=-2.4374284518778252E-5 evalInputDelta=0.11018841465314289
Fitness changed from
...skipping 14306 bytes...
6751
END: th(40509.25925925929)=4.02574535086751; dx=-1.6279783531278336E-6 evalInputDelta=0.009242255240678787
Fitness changed from 4.034987606108189 to 4.02574535086751
Iteration 18 complete. Error: 4.02574535086751 Total: 89.3755; Orientation: 0.7975; Line Search: 59.0157
Adding measurement f08f8a9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.431e+03, gradient 1.276e-03, dot -0.871; [7cb338d5-ab18-41c9-8831-3a147d2015e5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.123672698934873, 4.03809011541307, 4.034987606108189, 4.02574535086751
LBFGS Accumulation History: 3 points
Removed measurement ca60688 to history. Total: 4
th(0)=4.02574535086751;dx=-1.6290125389879093E-6
New Minimum: 4.02574535086751 > 4.009100764989853
END: th(87274.55341564352)=4.009100764989853; dx=-1.4552450439066356E-6 evalInputDelta=0.016644585877656937
Fitness changed from 4.02574535086751 to 4.009100764989853
Iteration 19 complete. Error: 4.009100764989853 Total: 89.0712; Orientation: 0.7969; Line Search: 59.1104
Adding measurement 306c9b2c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.184e+03, gradient 1.209e-03, dot -0.985; [7cb338d5-ab18-41c9-8831-3a147d2015e5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.03809011541307, 4.034987606108189, 4.02574535086751, 4.009100764989853
LBFGS Accumulation History: 3 points
Removed measurement 2762253e to history. Total: 4
th(0)=4.009100764989853;dx=-1.460913218893522E-6
New Minimum: 4.009100764989853 > 3.980804141610861
END: th(188027.32543570304)=3.980804141610861; dx=-1.251147871010906E-6 evalInputDelta=0.02829662337899208
Fitness changed from 4.009100764989853 to 3.980804141610861
Iteration 20 complete. Error: 3.980804141610861 Total: 89.0834; Orientation: 0.8016; Line Search: 59.0661
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: 3.980804141610861 (> -Infinity) after 2407.395s (< 3600.000s)
3.980804141610861