Code from BasicOptimizer.scala:75 executed in 3691.26 seconds (159.867 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1344970230384100
Reset training subject: 1345114311396700
Adding measurement 15c8b75c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=8.382369792088866;dx=-4.161573264295743E-6
New Minimum: 8.382369792088866 > 8.38236615806818
WOLFE (weak): th(2.154434690031884)=8.38236615806818; dx=-4.13366628472041E-6 evalInputDelta=3.634020686149597E-6
New Minimum: 8.38236615806818 > 8.38236252963543
WOLFE (weak): th(4.308869380063768)=8.38236252963543; dx=-4.133651072941861E-6 evalInputDelta=7.2624534368515015E-6
New Minimum: 8.38236252963543 > 8.382347764447331
WOLFE (weak): th(12.926608140191302)=8.382347764447331; dx=-4.133702928245593E-6 evalInputDelta=2.2027641534805298E-5
New Minimum: 8.382347764447331 > 8.382282445207238
WOLFE (weak): th(51.70643256076521)=8.382282445207238; dx=-4.132870917070566E-6 evalInputDelta=8.73468816280365E-5
New Minimum: 8.382282445207238 > 8.38193229585886
WOLFE (weak): th(258.53216280382605)=8.38193229585886; dx=-4.132448937343849E-6 evalInputDelta=4.3749623000621796E-4
New Minimum: 8.38193229585886 > 8.379744250327349
WOLFE (weak): th(1551.1929768229563)=8.379744250327349; dx=-4.131443149267039E-6 evalInputDelta=0.0026255417615175247
New Minimum: 8.379744250327349 > 8.364306267350912
WOLFE (weak): th(10858.350837760694)=8.364306267350912; dx=-4.024179544170466E-6 evalInputDelta=0.01806352473795414
New Minimum: 8.364306267350912 > 8.251010350883007
END: th(86866.80670208555)=8.251010350883007; dx=-3.3714666762969046E-6 evalInputDelta=0.13135944120585918
Fitness changed from 8.382369792088866 to 8.251010350883007
Iteration 1 complete. Error: 8.251010350883007 Total: 1869.6008; Orientation: 0.2038; Line Search: 1372.7719
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 156d7556 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=8.251010350883007;dx=-3.1336628042771597E-6
New Minimum: 8.25101035
...skipping 815 bytes...
6 Total: 463.6764; Orientation: 0.2013; Line Search: 321.9090
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 72ae1007 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.372e+03, gradient 1.258e-03, dot -0.971; [bfa28455-550f-4f4a-91cf-b994042190d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.382369792088866, 8.251010350883007, 8.049393564462662, 7.766712844371796
LBFGS Accumulation History: 3 points
Removed measurement 15c8b75c to history. Total: 4
th(0)=7.766712844371796;dx=-1.5825262974096566E-6
New Minimum: 7.766712844371796 > 7.388835467398167
END: th(868668.0670208557)=7.388835467398167; dx=-1.0191015624951078E-6 evalInputDelta=0.377877376973629
Fitness changed from 7.766712844371796 to 7.388835467398167
Iteration 4 complete. Error: 7.388835467398167 Total: 451.0250; Orientation: 5.8738; Line Search: 305.0891
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6e6ec71f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.444e+03, gradient 1.065e-03, dot -0.928; [bfa28455-550f-4f4a-91cf-b994042190d6 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.251010350883007, 8.049393564462662, 7.766712844371796, 7.388835467398167
LBFGS Accumulation History: 3 points
Removed measurement 156d7556 to history. Total: 4
th(0)=7.388835467398167;dx=-1.1346523937301504E-6
New Minimum: 7.388835467398167 > 6.933784440159798
END: th(1871488.617712673)=6.933784440159798; dx=-5.790050396988049E-7 evalInputDelta=0.455051027238369
Fitness changed from 7.388835467398167 to 6.933784440159798
Iteration 5 complete. Error: 6.933784440159798 Total: 449.2608; Orientation: 5.8502; Line Search: 301.5869
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Final threshold in iteration 5: 6.933784440159798 (> -Infinity) after 3691.244s (< 3600.000s)
6.933784440159798