Code from BasicOptimizer.scala:75 executed in 210.14 seconds (5.777 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1494109020379100
Reset training subject: 1494111638471500
Adding measurement 33188612 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=76.51114066503942;dx=-0.0023056295065207863
New Minimum: 76.51114066503942 > 76.50855717528611
WOLFE (weak): th(2.154434690031884)=76.50855717528611; dx=-0.0023055917147439646 evalInputDelta=0.002583489753305912
New Minimum: 76.50855717528611 > 76.50342114083469
WOLFE (weak): th(4.308869380063768)=76.50342114083469; dx=-0.0023051231308714253 evalInputDelta=0.0077195242047309875
New Minimum: 76.50342114083469 > 76.49310003779829
WOLFE (weak): th(12.926608140191302)=76.49310003779829; dx=-0.0023054859179528864 evalInputDelta=0.018040627241134644
New Minimum: 76.49310003779829 > 76.44365459494293
WOLFE (weak): th(51.70643256076521)=76.44365459494293; dx=-0.0023038471312418853 evalInputDelta=0.06748607009649277
New Minimum: 76.44365459494293 > 76.19349173083901
WOLFE (weak): th(258.53216280382605)=76.19349173083901; dx=-0.0022838549893760608 evalInputDelta=0.3176489342004061
New Minimum: 76.19349173083901 > 74.7246900331229
WOLFE (weak): th(1551.1929768229563)=74.7246900331229; dx=-0.0022749275879614637 evalInputDelta=1.786450631916523
New Minimum: 74.7246900331229 > 65.37099837698042
END: th(10858.350837760694)=65.37099837698042; dx=-0.0017618261214426823 evalInputDelta=11.140142288058996
Fitness changed from 76.51114066503942 to 65.37099837698042
Iteration 1 complete. Error: 65.37099837698042 Total: 28.2251; Orientation: 0.0250; Line Search: 20.3695
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2e43c38d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=65.37099837698042;dx=-0.0015478234211869833
New Minimum: 65.37099837698042 > 50.797524970024824
END: th(23393.607721408407)=50.797524970024824; dx=-9.791591612154176E-4 evalInputDelta=14.5734734069556
Fitness changed from 65.3709983
...skipping 14813 bytes...
l: 3
Rejected: LBFGS Orientation magnitude: 1.222e+03, gradient 2.107e-03, dot -0.795; [a3f29a23-6c63-4733-a25f-61bc5798def2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.53116774559021, -40.555911019444466, -44.313790117390454, -46.17943731788546
LBFGS Accumulation History: 3 points
Removed measurement 2ce03e86 to history. Total: 4
th(0)=-46.17943731788546;dx=-4.43980178983486E-6
New Minimum: -46.17943731788546 > -46.50195067632012
WOLF (strong): th(523647.3204938611)=-46.50195067632012; dx=4.085295489384203E-7 evalInputDelta=0.3225133584346622
END: th(261823.66024693055)=-46.46118417894468; dx=-1.2955423378920234E-6 evalInputDelta=0.28174686105921865
Fitness changed from -46.17943731788546 to -46.50195067632012
Iteration 19 complete. Error: -46.50195067632012 Total: 9.8770; Orientation: 0.0865; Line Search: 7.3034
Adding measurement 1ac730cd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.258e+02, gradient 1.614e-03, dot -0.831; [a3f29a23-6c63-4733-a25f-61bc5798def2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.555911019444466, -44.313790117390454, -46.17943731788546, -46.50195067632012
LBFGS Accumulation History: 3 points
Removed measurement 4c731956 to history. Total: 4
th(0)=-46.50195067632012;dx=-2.606271734192403E-6
New Minimum: -46.50195067632012 > -46.57359539880417
WOLF (strong): th(564081.9763071091)=-46.57359539880417; dx=1.2288729848617964E-6 evalInputDelta=0.07164472248405218
New Minimum: -46.57359539880417 > -46.667495066300035
END: th(282040.98815355456)=-46.667495066300035; dx=-1.4016399494179536E-7 evalInputDelta=0.16554438997991383
Fitness changed from -46.50195067632012 to -46.667495066300035
Iteration 20 complete. Error: -46.667495066300035 Total: 10.3580; Orientation: 0.0851; Line Search: 7.4284
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: -46.667495066300035 (> -Infinity) after 210.119s (< 3600.000s)
-46.667495066300035