Code from BasicOptimizer.scala:75 executed in 990.82 seconds (12.362 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1165418078639800
Reset training subject: 1165426669938700
Adding measurement 3c04ddda to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.278737924139166;dx=-3.150337793838257E-5
New Minimum: 5.278737924139166 > 5.278708096259414
WOLFE (weak): th(2.154434690031884)=5.278708096259414; dx=-3.15022439941585E-5 evalInputDelta=2.982787975192025E-5
New Minimum: 5.278708096259414 > 5.278678440906509
WOLFE (weak): th(4.308869380063768)=5.278678440906509; dx=-3.150138889027098E-5 evalInputDelta=5.948323265680955E-5
New Minimum: 5.278678440906509 > 5.27855958743388
WOLFE (weak): th(12.926608140191302)=5.27855958743388; dx=-3.149907696878612E-5 evalInputDelta=1.783367052858864E-4
New Minimum: 5.27855958743388 > 5.278024372467232
WOLFE (weak): th(51.70643256076521)=5.278024372467232; dx=-3.1483674828767793E-5 evalInputDelta=7.135516719332102E-4
New Minimum: 5.278024372467232 > 5.275174093503037
WOLFE (weak): th(258.53216280382605)=5.275174093503037; dx=-3.142265732965984E-5 evalInputDelta=0.003563830636128529
New Minimum: 5.275174093503037 > 5.257468748762001
WOLFE (weak): th(1551.1929768229563)=5.257468748762001; dx=-3.1106841468938274E-5 evalInputDelta=0.02126917537716455
New Minimum: 5.257468748762001 > 5.134822172602179
WOLFE (weak): th(10858.350837760694)=5.134822172602179; dx=-2.9152150583144852E-5 evalInputDelta=0.14391575153698621
New Minimum: 5.134822172602179 > 4.374227446398322
END: th(86866.80670208555)=4.374227446398322; dx=-1.7810851602925084E-5 evalInputDelta=0.9045104777408435
Fitness changed from 5.278737924139166 to 4.374227446398322
Iteration 1 complete. Error: 4.374227446398322 Total: 97.9095; Orientation: 0.0068; Line Search: 73.1909
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 72ea6fcb to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.374227446398322;dx=-1.336272124497231E-5
New Minimum: 4.374227446
...skipping 24290 bytes...
ot -0.770; [709595d3-97e9-4ed0-a79e-2a7053cf07b8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.14293682159114296, -0.1577051730157657, -0.1750178763442859, -0.20060195902270983
LBFGS Accumulation History: 3 points
Removed measurement c7cbc2d to history. Total: 4
th(0)=-0.20060195902270983;dx=-4.462497148907409E-8
Armijo: th(7834471.893154296)=-0.19529526608242564; dx=-1.2297703816145795E-8 evalInputDelta=-0.005306692940284186
New Minimum: -0.20060195902270983 > -0.2155194929960853
END: th(3917235.946577148)=-0.2155194929960853; dx=-1.3423367704749582E-8 evalInputDelta=0.014917533973375474
Fitness changed from -0.20060195902270983 to -0.2155194929960853
Iteration 29 complete. Error: -0.2155194929960853 Total: 33.6977; Orientation: 0.2014; Line Search: 25.3952
Adding measurement 14af57c3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.776e+03, gradient 3.070e-04, dot -0.520; [709595d3-97e9-4ed0-a79e-2a7053cf07b8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.1577051730157657, -0.1750178763442859, -0.20060195902270983, -0.2155194929960853
LBFGS Accumulation History: 3 points
Removed measurement 3e3a07ab to history. Total: 4
th(0)=-0.2155194929960853;dx=-9.425958026077285E-8
Armijo: th(8439429.01234569)=0.07857966438204289; dx=9.42215329376223E-8 evalInputDelta=-0.2940991573781282
Armijo: th(4219714.506172845)=-0.15235288469183916; dx=6.730695802153027E-8 evalInputDelta=-0.06316660830424614
New Minimum: -0.2155194929960853 > -0.23417274045871705
END: th(1406571.502057615)=-0.23417274045871705; dx=-4.2289294640539285E-9 evalInputDelta=0.01865324746263175
Fitness changed from -0.2155194929960853 to -0.23417274045871705
Iteration 30 complete. Error: -0.23417274045871705 Total: 41.0262; Orientation: 0.2018; Line Search: 32.7309
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: -0.23417274045871705 (> -Infinity) after 990.809s (< 3600.000s)
-0.23417274045871705