Code from BasicOptimizer.scala:75 executed in 3725.61 seconds (25.044 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1287819981930700
Reset training subject: 1287874810755200
Adding measurement 46a7a069 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=1.4310736118205991;dx=-3.1420256173564014E-7
New Minimum: 1.4310736118205991 > 1.4310734749477807
WOLFE (weak): th(2.154434690031884)=1.4310734749477807; dx=-3.0997884782990406E-7 evalInputDelta=1.3687281841257004E-7
New Minimum: 1.4310734749477807 > 1.4310732093919596
WOLFE (weak): th(4.308869380063768)=1.4310732093919596; dx=-3.099786165927042E-7 evalInputDelta=4.024286395232224E-7
New Minimum: 1.4310732093919596 > 1.4310721997541738
WOLFE (weak): th(12.926608140191302)=1.4310721997541738; dx=-3.09978550724104E-7 evalInputDelta=1.4120664253436388E-6
New Minimum: 1.4310721997541738 > 1.4310679607925296
WOLFE (weak): th(51.70643256076521)=1.4310679607925296; dx=-3.099776402296735E-7 evalInputDelta=5.6510280694865855E-6
New Minimum: 1.4310679607925296 > 1.431045578995119
WOLFE (weak): th(258.53216280382605)=1.431045578995119; dx=-3.099730757837409E-7 evalInputDelta=2.8032825480028833E-5
New Minimum: 1.431045578995119 > 1.4309058334654754
WOLFE (weak): th(1551.1929768229563)=1.4309058334654754; dx=-3.099451859989998E-7 evalInputDelta=1.6777835512371553E-4
New Minimum: 1.4309058334654754 > 1.4299006260910823
WOLFE (weak): th(10858.350837760694)=1.4299006260910823; dx=-3.0972521756284193E-7 evalInputDelta=0.001172985729516851
New Minimum: 1.4299006260910823 > 1.4217366269375522
WOLFE (weak): th(86866.80670208555)=1.4217366269375522; dx=-3.0795164885421097E-7 evalInputDelta=0.009336984883046906
New Minimum: 1.4217366269375522 > 1.3510456743312798
WOLFE (weak): th(781801.26031877)=1.3510456743312798; dx=-2.90150727275436E-7 evalInputDelta=0.08002793748931936
New Minimum: 1.3510456743312798 > 0.9666616217355293
END: th(7818012.6031877)=0.9666616217355293; dx=-1.5257902006064346E-7 evalInputDelta=0.46441199008506984
Fitness changed from 1.4310736118205991 to 0.9666616217355293
I
...skipping 10981 bytes...
omplete. Error: 0.07405409555750841 Total: 267.2914; Orientation: 1.4742; Line Search: 211.9384
Adding measurement 70d18b82 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.235e+03, gradient 1.338e-04, dot -0.858; [305b5a42-9760-4d75-b103-db698fdeadaa = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.17130277161578644, 0.13855775432468476, 0.12269722973110575, 0.07405409555750841
LBFGS Accumulation History: 3 points
Removed measurement 4660248 to history. Total: 4
th(0)=0.07405409555750841;dx=-1.7909023356912456E-8
New Minimum: 0.07405409555750841 > 0.03890452861388338
END: th(1.827625603235033E7)=0.03890452861388338; dx=-1.1718067265206038E-8 evalInputDelta=0.03514956694362503
Fitness changed from 0.07405409555750841 to 0.03890452861388338
Iteration 14 complete. Error: 0.03890452861388338 Total: 163.9284; Orientation: 1.4906; Line Search: 109.1542
Adding measurement 6250d1d3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.308e+04, gradient 1.237e-04, dot -0.871; [305b5a42-9760-4d75-b103-db698fdeadaa = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.13855775432468476, 0.12269722973110575, 0.07405409555750841, 0.03890452861388338
LBFGS Accumulation History: 3 points
Removed measurement 5c80bd18 to history. Total: 4
th(0)=0.03890452861388338;dx=-1.5298185617711855E-8
Armijo: th(3.937500000000003E7)=0.06340610232966049; dx=-6.706829138525364E-9 evalInputDelta=-0.024501573715777103
New Minimum: 0.03890452861388338 > 0.010173863788961124
END: th(1.9687500000000015E7)=0.010173863788961124; dx=-9.875533382077116E-9 evalInputDelta=0.02873066482492226
Fitness changed from 0.03890452861388338 to 0.010173863788961124
Iteration 15 complete. Error: 0.010173863788961124 Total: 214.0910; Orientation: 1.4800; Line Search: 159.9134
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 15: 0.010173863788961124 (> -Infinity) after 3725.608s (< 3600.000s)
0.010173863788961124