Code from BasicOptimizer.scala:75 executed in 981.49 seconds (10.753 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1236211952771000
Reset training subject: 1236219837288000
Adding measurement 1d444652 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.213316740807409;dx=-2.5425851195569275E-5
New Minimum: 5.213316740807409 > 5.213292636993221
WOLFE (weak): th(2.154434690031884)=5.213292636993221; dx=-2.542536532783622E-5 evalInputDelta=2.410381418815888E-5
New Minimum: 5.213292636993221 > 5.2132688464343415
WOLFE (weak): th(4.308869380063768)=5.2132688464343415; dx=-2.542488539240269E-5 evalInputDelta=4.7894373067514096E-5
New Minimum: 5.2132688464343415 > 5.2131731402718415
WOLFE (weak): th(12.926608140191302)=5.2131731402718415; dx=-2.542334762933502E-5 evalInputDelta=1.4360053556750074E-4
New Minimum: 5.2131731402718415 > 5.212742117000272
WOLFE (weak): th(51.70643256076521)=5.212742117000272; dx=-2.5416318321646588E-5 evalInputDelta=5.746238071369092E-4
New Minimum: 5.212742117000272 > 5.210444853943233
WOLFE (weak): th(258.53216280382605)=5.210444853943233; dx=-2.539100089334234E-5 evalInputDelta=0.002871886864175721
New Minimum: 5.210444853943233 > 5.196157743222452
WOLFE (weak): th(1551.1929768229563)=5.196157743222452; dx=-2.517778623804038E-5 evalInputDelta=0.017158997584957092
New Minimum: 5.196157743222452 > 5.095233363615147
WOLFE (weak): th(10858.350837760694)=5.095233363615147; dx=-2.3817154323434425E-5 evalInputDelta=0.11808337719226181
New Minimum: 5.095233363615147 > 4.466890346349041
END: th(86866.80670208555)=4.466890346349041; dx=-1.5014478451456646E-5 evalInputDelta=0.7464263944583678
Fitness changed from 5.213316740807409 to 4.466890346349041
Iteration 1 complete. Error: 4.466890346349041 Total: 95.4974; Orientation: 0.0067; Line Search: 71.6234
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 48d5ca17 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.466890346349041;dx=-1.1108298069994779E-5
New Minimu
...skipping 24013 bytes...
46c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.3282547237374093, 0.30300245710661666, 0.2855675451638132, 0.2697161018330542
LBFGS Accumulation History: 3 points
Removed measurement 46e6458d to history. Total: 4
th(0)=0.2697161018330542;dx=-5.203317634122389E-8
Armijo: th(6963975.016137152)=0.3214817687031732; dx=2.579616773635338E-8 evalInputDelta=-0.051765666870118965
New Minimum: 0.2697161018330542 > 0.25740611787125767
END: th(3481987.508068576)=0.25740611787125767; dx=-1.5925925902216759E-9 evalInputDelta=0.012309983961796556
Fitness changed from 0.2697161018330542 to 0.25740611787125767
Iteration 29 complete. Error: 0.25740611787125767 Total: 32.2097; Orientation: 0.2095; Line Search: 23.8736
Adding measurement 6badb08c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.549e+03, gradient 2.376e-04, dot -0.626; [4517efe9-8da1-4a40-8f3e-551d0745546c = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.30300245710661666, 0.2855675451638132, 0.2697161018330542, 0.25740611787125767
LBFGS Accumulation History: 3 points
Removed measurement 2d3c501b to history. Total: 4
th(0)=0.25740611787125767;dx=-5.6460325932148044E-8
Armijo: th(7501714.677640614)=0.33701935790517074; dx=4.0537933737746345E-8 evalInputDelta=-0.07961324003391307
New Minimum: 0.25740611787125767 > 0.24665125641070756
WOLF (strong): th(3750857.338820307)=0.24665125641070756; dx=7.438196021905137E-9 evalInputDelta=0.010754861460550103
New Minimum: 0.24665125641070756 > 0.23908588708514616
END: th(1250285.779606769)=0.23908588708514616; dx=-3.288491470241749E-8 evalInputDelta=0.018320230786111502
Fitness changed from 0.25740611787125767 to 0.23908588708514616
Iteration 30 complete. Error: 0.23908588708514616 Total: 40.0273; Orientation: 0.1998; Line Search: 31.7759
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 0.23908588708514616 (> -Infinity) after 981.486s (< 3600.000s)
0.23908588708514616