Code from BasicOptimizer.scala:75 executed in 16.56 seconds (0.212 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 904431007729300
Reset training subject: 904431501165600
Adding measurement 2037f12b to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=1.9808373600244522;dx=-1.3319788190654627E-7
New Minimum: 1.9808373600244522 > 1.980837106704712
WOLFE (weak): th(2.154434690031884)=1.980837106704712; dx=-1.3319790753034057E-7 evalInputDelta=2.5331974029541016E-7
New Minimum: 1.980837106704712 > 1.9808366298675537
WOLFE (weak): th(4.308869380063768)=1.9808366298675537; dx=-1.3319792726982912E-7 evalInputDelta=7.301568984985352E-7
New Minimum: 1.9808366298675537 > 1.980835646390915
WOLFE (weak): th(12.926608140191302)=1.980835646390915; dx=-1.331980169251923E-7 evalInputDelta=1.7136335372924805E-6
New Minimum: 1.980835646390915 > 1.9808305203914642
WOLFE (weak): th(51.70643256076521)=1.9808305203914642; dx=-1.3319841743652767E-7 evalInputDelta=6.839632987976074E-6
New Minimum: 1.9808305203914642 > 1.9808029234409332
WOLFE (weak): th(258.53216280382605)=1.9808029234409332; dx=-1.3320073557253227E-7 evalInputDelta=3.4436583518981934E-5
New Minimum: 1.9808029234409332 > 1.9806307405233383
WOLFE (weak): th(1551.1929768229563)=1.9806307405233383; dx=-1.3320337013016503E-7 evalInputDelta=2.066195011138916E-4
New Minimum: 1.9806307405233383 > 1.9793906509876251
WOLFE (weak): th(10858.350837760694)=1.9793906509876251; dx=-1.3328632295804474E-7 evalInputDelta=0.0014467090368270874
New Minimum: 1.9793906509876251 > 1.9692345410585403
WOLFE (weak): th(86866.80670208555)=1.9692345410585403; dx=-1.339619020725241E-7 evalInputDelta=0.011602818965911865
New Minimum: 1.9692345410585403 > 1.8745727986097336
WOLFE (weak): th(781801.26031877)=1.8745727986097336; dx=-1.378322382772286E-7 evalInputDelta=0.10626456141471863
New Minimum: 1.8745727986097336 > 1.0572738349437714
END: th(7818012.6031877)=1.0572738349437714; dx=-8.992033706832782E-8 evalInputDelta=0.9235635250806808
Fitness changed from 1.9808373600244522 to 1.0572738349437714
Iter
...skipping 2578 bytes...
measurement 72110818 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.158e+02, gradient 2.349e-05, dot -0.960; [b1b9e366-b5ee-49f1-9da4-26a47c868e8c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.0572738349437714, 0.010870784521102905, -0.3119158297777176, -0.3444915860891342
LBFGS Accumulation History: 3 points
Removed measurement 75e0a54c to history. Total: 4
th(0)=-0.3444915860891342;dx=-5.519628406469946E-10
New Minimum: -0.3444915860891342 > -0.3595847822725773
END: th(4.210849389853513E7)=-0.3595847822725773; dx=-2.012578131379388E-10 evalInputDelta=0.01509319618344307
Fitness changed from -0.3444915860891342 to -0.3595847822725773
Iteration 5 complete. Error: -0.3595847822725773 Total: 1.5677; Orientation: 0.1040; Line Search: 0.9935
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 33ebe4f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.722e+03, gradient 1.892e-05, dot -0.814; [b1b9e366-b5ee-49f1-9da4-26a47c868e8c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.010870784521102905, -0.3119158297777176, -0.3444915860891342, -0.3595847822725773
LBFGS Accumulation History: 3 points
Removed measurement 7e31062c to history. Total: 4
th(0)=-0.3595847822725773;dx=-3.581417216747995E-10
Armijo: th(9.072000000000001E7)=-0.3553927093744278; dx=3.329902045026451E-10 evalInputDelta=-0.00419207289814949
New Minimum: -0.3595847822725773 > -0.3652326390147209
WOLF (strong): th(4.536000000000001E7)=-0.3652326390147209; dx=7.441637909074252E-11 evalInputDelta=0.005647856742143631
END: th(1.5120000000000002E7)=-0.3635483793914318; dx=-1.921474940294564E-10 evalInputDelta=0.003963597118854523
Fitness changed from -0.3595847822725773 to -0.3652326390147209
Iteration 6 complete. Error: -0.3652326390147209 Total: 2.5107; Orientation: 0.1037; Line Search: 1.9112
Final threshold in iteration 7: -0.3652326390147209 (> -Infinity) after 16.558s (< 720.000s)
-0.3652326390147209