Code from BasicOptimizer.scala:75 executed in 888.50 seconds (5.812 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1024706753734700
Reset training subject: 1024714032743400
Adding measurement 19810310 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=11.281412150710821;dx=-2.991268195013721E-5
New Minimum: 11.281412150710821 > 11.28133899345994
WOLFE (weak): th(2.154434690031884)=11.28133899345994; dx=-2.991114652661375E-5 evalInputDelta=7.315725088119507E-5
New Minimum: 11.28133899345994 > 11.2812654748559
WOLFE (weak): th(4.308869380063768)=11.2812654748559; dx=-2.9912499343252183E-5 evalInputDelta=1.4667585492134094E-4
New Minimum: 11.2812654748559 > 11.280971217900515
WOLFE (weak): th(12.926608140191302)=11.280971217900515; dx=-2.990921525924575E-5 evalInputDelta=4.4093281030654907E-4
New Minimum: 11.280971217900515 > 11.279645957052708
WOLFE (weak): th(51.70643256076521)=11.279645957052708; dx=-2.9910602257991887E-5 evalInputDelta=0.0017661936581134796
New Minimum: 11.279645957052708 > 11.272581908851862
WOLFE (weak): th(258.53216280382605)=11.272581908851862; dx=-2.9884250937954217E-5 evalInputDelta=0.008830241858959198
New Minimum: 11.272581908851862 > 11.228577364236116
WOLFE (weak): th(1551.1929768229563)=11.228577364236116; dx=-2.9679368544132222E-5 evalInputDelta=0.05283478647470474
New Minimum: 11.228577364236116 > 10.9194574393332
WOLFE (weak): th(10858.350837760694)=10.9194574393332; dx=-2.8359680808398163E-5 evalInputDelta=0.3619547113776207
New Minimum: 10.9194574393332 > 9.00361966341734
END: th(86866.80670208555)=9.00361966341734; dx=-1.6250864508826988E-5 evalInputDelta=2.277792487293482
Fitness changed from 11.281412150710821 to 9.00361966341734
Iteration 1 complete. Error: 9.00361966341734 Total: 88.5407; Orientation: 0.0091; Line Search: 66.1713
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1c510921 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=9.00361966341734;dx=-1.195097442027848E-5
New Minimum:
...skipping 24369 bytes...
Adding measurement ceebc3b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.950e+03, gradient 1.254e-04, dot -0.621; [cc05f2ea-ca9d-4dae-89b6-23f4f97b37d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.335916079580784, -2.3669162318110466, -2.3836401253938675, -2.3986556753516197
LBFGS Accumulation History: 3 points
Removed measurement 35555daa to history. Total: 4
th(0)=-2.3986556753516197;dx=-1.569876668810856E-8
Armijo: th(1.566894378630859E7)=-2.1736185252666473; dx=1.9401136819406594E-8 evalInputDelta=-0.22503715008497238
Armijo: th(7834471.893154295)=-2.3666155263781548; dx=1.1949428558975487E-8 evalInputDelta=-0.032040148973464966
New Minimum: -2.3986556753516197 > -2.4197265952825546
END: th(2611490.631051432)=-2.4197265952825546; dx=-2.548234699908864E-9 evalInputDelta=0.021070919930934906
Fitness changed from -2.3986556753516197 to -2.4197265952825546
Iteration 29 complete. Error: -2.4197265952825546 Total: 37.6097; Orientation: 0.2568; Line Search: 29.9015
Adding measurement 24e84a4e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.195e+03, gradient 8.584e-05, dot -0.715; [cc05f2ea-ca9d-4dae-89b6-23f4f97b37d9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.3669162318110466, -2.3836401253938675, -2.3986556753516197, -2.4197265952825546
LBFGS Accumulation History: 3 points
Removed measurement 1efd5e0 to history. Total: 4
th(0)=-2.4197265952825546;dx=-7.3531736823937135E-9
New Minimum: -2.4197265952825546 > -2.4353460147976875
END: th(5626286.00823046)=-2.4353460147976875; dx=-2.4884026520828997E-9 evalInputDelta=0.015619419515132904
Fitness changed from -2.4197265952825546 to -2.4353460147976875
Iteration 30 complete. Error: -2.4353460147976875 Total: 22.4024; Orientation: 0.2622; Line Search: 14.8505
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: -2.4353460147976875 (> -Infinity) after 888.500s (< 3600.000s)
-2.4353460147976875