Code from BasicOptimizer.scala:75 executed in 989.69 seconds (8.411 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 992183828935900
Reset training subject: 992192629830100
Adding measurement 4283f29b to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=14.898160728731645;dx=-1.8593093571543288E-5
New Minimum: 14.898160728731645 > 14.898115766151822
WOLFE (weak): th(2.154434690031884)=14.898115766151822; dx=-1.8595461124428603E-5 evalInputDelta=4.4962579822893645E-5
New Minimum: 14.898115766151822 > 14.898069842447104
WOLFE (weak): th(4.308869380063768)=14.898069842447104; dx=-1.859483707803879E-5 evalInputDelta=9.08862845410141E-5
New Minimum: 14.898069842447104 > 14.897887903241992
WOLFE (weak): th(12.926608140191302)=14.897887903241992; dx=-1.8588739393462956E-5 evalInputDelta=2.728254896524618E-4
New Minimum: 14.897887903241992 > 14.897069601024644
WOLFE (weak): th(51.70643256076521)=14.897069601024644; dx=-1.858797358235602E-5 evalInputDelta=0.0010911277070011494
New Minimum: 14.897069601024644 > 14.892701855807617
WOLFE (weak): th(258.53216280382605)=14.892701855807617; dx=-1.8588627947863167E-5 evalInputDelta=0.005458872924027247
New Minimum: 14.892701855807617 > 14.865453494998016
WOLFE (weak): th(1551.1929768229563)=14.865453494998016; dx=-1.8531089564519324E-5 evalInputDelta=0.03270723373362827
New Minimum: 14.865453494998016 > 14.673384178603284
WOLFE (weak): th(10858.350837760694)=14.673384178603284; dx=-1.764818198876877E-5 evalInputDelta=0.22477655012836095
New Minimum: 14.673384178603284 > 13.416874870754343
END: th(86866.80670208555)=13.416874870754343; dx=-1.1346541750792574E-5 evalInputDelta=1.4812858579773014
Fitness changed from 14.898160728731645 to 13.416874870754343
Iteration 1 complete. Error: 13.416874870754343 Total: 98.0481; Orientation: 0.0252; Line Search: 73.1179
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 545ffe1 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=13.416874870754343;dx=-8.65877161132
...skipping 24023 bytes...
715)=2.9674279413003477; dx=-3.0734795835953076E-8 evalInputDelta=0.057424142658679234
Fitness changed from 3.024852083959027 to 2.9674279413003477
Iteration 28 complete. Error: 2.9674279413003477 Total: 40.4549; Orientation: 0.2277; Line Search: 32.1804
Adding measurement 2d3aa0f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.849e+03, gradient 1.785e-04, dot -0.858; [8414a67a-13bb-4dff-83bf-5e46ca1541de = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.128887830648912, 3.067983698532957, 3.024852083959027, 2.9674279413003477
LBFGS Accumulation History: 3 points
Removed measurement 3fa54a4b to history. Total: 4
th(0)=2.9674279413003477;dx=-3.180245443523335E-8
New Minimum: 2.9674279413003477 > 2.9164333323998695
END: th(2611490.631051432)=2.9164333323998695; dx=-1.800793560101324E-8 evalInputDelta=0.050994608900478244
Fitness changed from 2.9674279413003477 to 2.9164333323998695
Iteration 29 complete. Error: 2.9164333323998695 Total: 24.7140; Orientation: 0.2392; Line Search: 16.3048
Adding measurement 2e1911b3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.056e+03, gradient 1.762e-04, dot -0.800; [8414a67a-13bb-4dff-83bf-5e46ca1541de = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.067983698532957, 3.024852083959027, 2.9674279413003477, 2.9164333323998695
LBFGS Accumulation History: 3 points
Removed measurement 6cad16f8 to history. Total: 4
th(0)=2.9164333323998695;dx=-3.099351371777543E-8
New Minimum: 2.9164333323998695 > 2.8863014780885656
END: th(5626286.00823046)=2.8863014780885656; dx=-4.189891338188092E-9 evalInputDelta=0.03013185431130383
Fitness changed from 2.9164333323998695 to 2.8863014780885656
Iteration 30 complete. Error: 2.8863014780885656 Total: 24.5965; Orientation: 0.2488; Line Search: 16.4322
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 2.8863014780885656 (> -Infinity) after 989.661s (< 3600.000s)
2.8863014780885656