Code from BasicOptimizer.scala:75 executed in 1015.17 seconds (12.816 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1154913436807200
Reset training subject: 1154923326595200
Adding measurement 45424c0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.221549216513866;dx=-2.6453215830534575E-5
New Minimum: 5.221549216513866 > 5.221524360746223
WOLFE (weak): th(2.154434690031884)=5.221524360746223; dx=-2.645311384849587E-5 evalInputDelta=2.4855767643217064E-5
New Minimum: 5.221524360746223 > 5.221499557916017
WOLFE (weak): th(4.308869380063768)=5.221499557916017; dx=-2.6451500441564295E-5 evalInputDelta=4.9658597848889485E-5
New Minimum: 5.221499557916017 > 5.221400459390101
WOLFE (weak): th(12.926608140191302)=5.221400459390101; dx=-2.645047999072792E-5 evalInputDelta=1.4875712376483108E-4
New Minimum: 5.221400459390101 > 5.220954267812088
WOLFE (weak): th(51.70643256076521)=5.220954267812088; dx=-2.6443253972472902E-5 evalInputDelta=5.949487017780086E-4
New Minimum: 5.220954267812088 > 5.218576632752099
WOLFE (weak): th(258.53216280382605)=5.218576632752099; dx=-2.6413600264002657E-5 evalInputDelta=0.0029725837617666784
New Minimum: 5.218576632752099 > 5.204266047747687
WOLFE (weak): th(1551.1929768229563)=5.204266047747687; dx=-2.6218715204014246E-5 evalInputDelta=0.017283168766178747
New Minimum: 5.204266047747687 > 5.101153396026174
WOLFE (weak): th(10858.350837760694)=5.101153396026174; dx=-2.47263329296769E-5 evalInputDelta=0.12039582048769226
New Minimum: 5.101153396026174 > 4.461824922952173
END: th(86866.80670208555)=4.461824922952173; dx=-1.5220067431691656E-5 evalInputDelta=0.7597242935616926
Fitness changed from 5.221549216513866 to 4.461824922952173
Iteration 1 complete. Error: 4.461824922952173 Total: 99.3304; Orientation: 0.0068; Line Search: 73.3346
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 11c3ce6a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.461824922952173;dx=-1.1242766291425677E-5
New Minimum: 4
...skipping 23846 bytes...
6e+03, gradient 1.999e-04, dot -0.733; [d7045387-4e5c-4524-99f7-9eed46a9b7c5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.33716202111107474, 0.33031791078209116, 0.30390035667103854, 0.2885194835893412
LBFGS Accumulation History: 3 points
Removed measurement 54f25b42 to history. Total: 4
th(0)=0.2885194835893412;dx=-3.9976776990840576E-8
Armijo: th(9285300.021516202)=0.34392079915904294; dx=8.261817956875757E-9 evalInputDelta=-0.05540131556970174
New Minimum: 0.2885194835893412 > 0.2797715171177132
END: th(4642650.010758101)=0.2797715171177132; dx=-9.491790961969544E-9 evalInputDelta=0.00874796647162801
Fitness changed from 0.2885194835893412 to 0.2797715171177132
Iteration 29 complete. Error: 0.2797715171177132 Total: 34.4447; Orientation: 0.1963; Line Search: 25.5957
Adding measurement 311a09b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.622e+03, gradient 2.422e-04, dot -0.585; [d7045387-4e5c-4524-99f7-9eed46a9b7c5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.33031791078209116, 0.30390035667103854, 0.2885194835893412, 0.2797715171177132
LBFGS Accumulation History: 3 points
Removed measurement 27dc81be to history. Total: 4
th(0)=0.2797715171177132;dx=-5.866518534854752E-8
Armijo: th(1.000228623685415E7)=0.45988405174847147; dx=5.325125946995461E-8 evalInputDelta=-0.18011253463075827
Armijo: th(5001143.118427075)=0.2908371165153625; dx=1.6240499484075004E-8 evalInputDelta=-0.011065599397649284
New Minimum: 0.2797715171177132 > 0.2591427434452893
END: th(1667047.7061423585)=0.2591427434452893; dx=-2.7955217713039E-8 evalInputDelta=0.020628773672423883
Fitness changed from 0.2797715171177132 to 0.2591427434452893
Iteration 30 complete. Error: 0.2591427434452893 Total: 42.0663; Orientation: 0.1937; Line Search: 33.5645
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 0.2591427434452893 (> -Infinity) after 1015.158s (< 3600.000s)
0.2591427434452893