Code from BasicOptimizer.scala:75 executed in 542.55 seconds (6.748 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1579935917562900
Reset training subject: 1579942299439800
Adding measurement 4e7bc939 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-21.877351120114326;dx=-8.379312056665357E-5
New Minimum: -21.877351120114326 > -21.877435728907585
WOLFE (weak): th(2.154434690031884)=-21.877435728907585; dx=-8.372221016202841E-5 evalInputDelta=8.460879325866699E-5
New Minimum: -21.877435728907585 > -21.877515375614166
WOLFE (weak): th(4.308869380063768)=-21.877515375614166; dx=-8.372200170052379E-5 evalInputDelta=1.6425549983978271E-4
New Minimum: -21.877515375614166 > -21.877854228019714
WOLFE (weak): th(12.926608140191302)=-21.877854228019714; dx=-8.372121186352727E-5 evalInputDelta=5.031079053878784E-4
New Minimum: -21.877854228019714 > -21.87937104701996
WOLFE (weak): th(51.70643256076521)=-21.87937104701996; dx=-8.371642435550403E-5 evalInputDelta=0.002019926905632019
New Minimum: -21.87937104701996 > -21.887460485100746
WOLFE (weak): th(258.53216280382605)=-21.887460485100746; dx=-8.369338205088897E-5 evalInputDelta=0.010109364986419678
New Minimum: -21.887460485100746 > -21.938501253724098
WOLFE (weak): th(1551.1929768229563)=-21.938501253724098; dx=-8.355604413805188E-5 evalInputDelta=0.06115013360977173
New Minimum: -21.938501253724098 > -22.299430757761
WOLFE (weak): th(10858.350837760694)=-22.299430757761; dx=-8.253759096020553E-5 evalInputDelta=0.4220796376466751
New Minimum: -22.299430757761 > -25.057385221123695
END: th(86866.80670208555)=-25.057385221123695; dx=-7.394662131810687E-5 evalInputDelta=3.180034101009369
Fitness changed from -21.877351120114326 to -25.057385221123695
Iteration 1 complete. Error: -25.057385221123695 Total: 76.8302; Orientation: 0.0124; Line Search: 57.5381
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1231a1be to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-25.057385221123695;dx=-6
...skipping 14900 bytes...
03, gradient 1.393e-03, dot -0.540; [c29a8ca0-9e92-4974-b03d-3334d4bb3e67 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -43.571241430938244, -43.771508276462555, -43.94185894727707, -43.949557546526194
LBFGS Accumulation History: 3 points
Removed measurement 3f3c5ecd to history. Total: 4
th(0)=-43.949557546526194;dx=-1.9409255647576565E-6
Armijo: th(4189178.563950889)=-40.599021082744; dx=2.2803699434870204E-6 evalInputDelta=-3.3505364637821913
Armijo: th(2094589.2819754444)=-43.35261050797999; dx=6.616382035967195E-7 evalInputDelta=-0.5969470385462046
New Minimum: -43.949557546526194 > -44.12723662704229
END: th(698196.4273251481)=-44.12723662704229; dx=-4.1796233705611755E-7 evalInputDelta=0.17767908051609993
Fitness changed from -43.949557546526194 to -44.12723662704229
Iteration 19 complete. Error: -44.12723662704229 Total: 32.2299; Orientation: 0.3430; Line Search: 25.6553
Adding measurement 615bad16 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.275e+03, gradient 1.356e-03, dot -0.683; [c29a8ca0-9e92-4974-b03d-3334d4bb3e67 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -43.771508276462555, -43.94185894727707, -43.949557546526194, -44.12723662704229
LBFGS Accumulation History: 3 points
Removed measurement 1ba35152 to history. Total: 4
th(0)=-44.12723662704229;dx=-1.8386695174994792E-6
Armijo: th(1504218.6034856243)=-43.85800343379378; dx=3.134874980193512E-7 evalInputDelta=-0.26923319324851036
New Minimum: -44.12723662704229 > -44.187656223773956
END: th(752109.3017428122)=-44.187656223773956; dx=-1.4578444764314858E-7 evalInputDelta=0.06041959673166275
Fitness changed from -44.12723662704229 to -44.187656223773956
Iteration 20 complete. Error: -44.187656223773956 Total: 25.7949; Orientation: 0.3448; Line Search: 19.1137
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: -44.187656223773956 (> -Infinity) after 542.541s (< 3600.000s)
-44.187656223773956