Code from BasicOptimizer.scala:75 executed in 3789.27 seconds (24.900 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1279825463866000
Reset training subject: 1279881208607300
Adding measurement 72ebfc21 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=1.3852362356832422;dx=-2.805583394793738E-7
New Minimum: 1.3852362356832422 > 1.3852360324227748
WOLFE (weak): th(2.154434690031884)=1.3852360324227748; dx=-2.7655407657661634E-7 evalInputDelta=2.032604673463112E-7
New Minimum: 1.3852360324227748 > 1.385235808905357
WOLFE (weak): th(4.308869380063768)=1.385235808905357; dx=-2.7655397692052734E-7 evalInputDelta=4.2677788525402605E-7
New Minimum: 1.385235808905357 > 1.3852350749412838
WOLFE (weak): th(12.926608140191302)=1.3852350749412838; dx=-2.7655375961985776E-7 evalInputDelta=1.1607419583281597E-6
New Minimum: 1.3852350749412838 > 1.3852311822169223
WOLFE (weak): th(51.70643256076521)=1.3852311822169223; dx=-2.7655213568030096E-7 evalInputDelta=5.0534663198487095E-6
New Minimum: 1.3852311822169223 > 1.3852116149088585
WOLFE (weak): th(258.53216280382605)=1.3852116149088585; dx=-2.765497553529021E-7 evalInputDelta=2.462077438369903E-5
New Minimum: 1.3852116149088585 > 1.385088531874112
WOLFE (weak): th(1551.1929768229563)=1.385088531874112; dx=-2.76536908444275E-7 evalInputDelta=1.477038091302063E-4
New Minimum: 1.385088531874112 > 1.3842030622208163
WOLFE (weak): th(10858.350837760694)=1.3842030622208163; dx=-2.7629272672140836E-7 evalInputDelta=0.0010331734624258182
New Minimum: 1.3842030622208163 > 1.3770144790148393
WOLFE (weak): th(86866.80670208555)=1.3770144790148393; dx=-2.74685199456825E-7 evalInputDelta=0.008221756668402858
New Minimum: 1.3770144790148393 > 1.314994370257614
WOLFE (weak): th(781801.26031877)=1.314994370257614; dx=-2.591331530960149E-7 evalInputDelta=0.07024186542562827
New Minimum: 1.314994370257614 > 0.980498317530515
END: th(7818012.6031877)=0.980498317530515; dx=-1.3901226924767836E-7 evalInputDelta=0.4047379181527272
Fitness changed from 1.3852362356832422 to 0.980498317530515
Iteration 1
...skipping 10978 bytes...
asurement 485262be to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.221e+04, gradient 1.369e-04, dot -0.767; [2b3f08bd-0fd3-4913-9688-08de56f1bb64 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.2621061750299045, 0.22079781754711839, 0.18651513643994777, 0.16045141899605608
LBFGS Accumulation History: 3 points
Removed measurement 4d088fd3 to history. Total: 4
th(0)=0.16045141899605608;dx=-1.8736632265971055E-8
Armijo: th(5.482876809705099E7)=0.2868422482906978; dx=-4.0431472078981455E-9 evalInputDelta=-0.12639082929464174
New Minimum: 0.16045141899605608 > 0.14233444089672942
END: th(2.7414384048525494E7)=0.14233444089672942; dx=-8.141930518054084E-9 evalInputDelta=0.018116978099326664
Fitness changed from 0.16045141899605608 to 0.14233444089672942
Iteration 14 complete. Error: 0.14233444089672942 Total: 217.8619; Orientation: 1.4899; Line Search: 161.8247
Adding measurement 53340a2d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.762e+03, gradient 1.372e-04, dot -0.722; [2b3f08bd-0fd3-4913-9688-08de56f1bb64 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.22079781754711839, 0.18651513643994777, 0.16045141899605608, 0.14233444089672942
LBFGS Accumulation History: 3 points
Removed measurement 2b3fe9e to history. Total: 4
th(0)=0.14233444089672942;dx=-1.8833197040087866E-8
Armijo: th(5.9062500000000045E7)=0.3408636692189673; dx=-1.5152356916875558E-9 evalInputDelta=-0.1985292283222379
New Minimum: 0.14233444089672942 > 0.1405133895280866
END: th(2.9531250000000022E7)=0.1405133895280866; dx=-5.814763541591001E-9 evalInputDelta=0.0018210513686428287
Fitness changed from 0.14233444089672942 to 0.1405133895280866
Iteration 15 complete. Error: 0.1405133895280866 Total: 215.8808; Orientation: 1.4690; Line Search: 161.7782
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 15: 0.1405133895280866 (> -Infinity) after 3789.260s (< 3600.000s)
0.1405133895280866