Code from BasicOptimizer.scala:75 executed in 993.93 seconds (11.457 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1162300304843700
Reset training subject: 1162310157897700
Adding measurement 2aa5bd48 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.171821542666278;dx=-2.4620286178424777E-5
New Minimum: 5.171821542666278 > 5.171798276461171
WOLFE (weak): th(2.154434690031884)=5.171798276461171; dx=-2.4619811353445884E-5 evalInputDelta=2.326620510650912E-5
New Minimum: 5.171798276461171 > 5.171775133657985
WOLFE (weak): th(4.308869380063768)=5.171775133657985; dx=-2.4620436390018517E-5 evalInputDelta=4.6409008292336296E-5
New Minimum: 5.171775133657985 > 5.171682325982818
WOLFE (weak): th(12.926608140191302)=5.171682325982818; dx=-2.4619706084235787E-5 evalInputDelta=1.3921668345950877E-4
New Minimum: 5.171682325982818 > 5.17126498047157
WOLFE (weak): th(51.70643256076521)=5.17126498047157; dx=-2.4616372064070457E-5 evalInputDelta=5.565621947081212E-4
New Minimum: 5.17126498047157 > 5.169041522065828
WOLFE (weak): th(258.53216280382605)=5.169041522065828; dx=-2.457599833348576E-5 evalInputDelta=0.002780020600449795
New Minimum: 5.169041522065828 > 5.15547328134104
WOLFE (weak): th(1551.1929768229563)=5.15547328134104; dx=-2.440327403323198E-5 evalInputDelta=0.0163482613252377
New Minimum: 5.15547328134104 > 5.058798167788129
WOLFE (weak): th(10858.350837760694)=5.058798167788129; dx=-2.308224554337413E-5 evalInputDelta=0.11302337487814906
New Minimum: 5.058798167788129 > 4.448319956114609
END: th(86866.80670208555)=4.448319956114609; dx=-1.4560620441850726E-5 evalInputDelta=0.7235015865516683
Fitness changed from 5.171821542666278 to 4.448319956114609
Iteration 1 complete. Error: 4.448319956114609 Total: 102.0188; Orientation: 0.0070; Line Search: 75.1662
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 401e02b4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.448319956114609;dx=-1.0838354473058838E-5
New Minimum: 4.448319
...skipping 23900 bytes...
rch: 24.1701
Adding measurement 2c674d58 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.493e+03, gradient 2.012e-04, dot -0.718; [d8b498b9-ac37-4580-8fad-e3b4679192c3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.32176117720982766, 0.3149285540118185, 0.29237500522676874, 0.2744310209080949
LBFGS Accumulation History: 3 points
Removed measurement 69ce14e6 to history. Total: 4
th(0)=0.2744310209080949;dx=-4.047589374855525E-8
Armijo: th(6963975.016137152)=0.29124662440459304; dx=-1.6046643784575964E-9 evalInputDelta=-0.016815603496498133
New Minimum: 0.2744310209080949 > 0.26091122233835323
END: th(3481987.508068576)=0.26091122233835323; dx=-1.6254966573889093E-8 evalInputDelta=0.01351979856974167
Fitness changed from 0.2744310209080949 to 0.26091122233835323
Iteration 29 complete. Error: 0.26091122233835323 Total: 32.9171; Orientation: 0.2067; Line Search: 24.5378
Adding measurement 2e929182 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.009e+03, gradient 2.030e-04, dot -0.716; [d8b498b9-ac37-4580-8fad-e3b4679192c3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.3149285540118185, 0.29237500522676874, 0.2744310209080949, 0.26091122233835323
LBFGS Accumulation History: 3 points
Removed measurement 7433ca19 to history. Total: 4
th(0)=0.26091122233835323;dx=-4.122903771012604E-8
Armijo: th(7501714.677640614)=0.29533573489103054; dx=9.72182537594955E-9 evalInputDelta=-0.03442451255267731
New Minimum: 0.26091122233835323 > 0.2487358114120217
END: th(3750857.338820307)=0.2487358114120217; dx=-9.767439227425579E-9 evalInputDelta=0.012175410926331537
Fitness changed from 0.26091122233835323 to 0.2487358114120217
Iteration 30 complete. Error: 0.2487358114120217 Total: 32.8802; Orientation: 0.2020; Line Search: 24.5055
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 0.2487358114120217 (> -Infinity) after 993.921s (< 3600.000s)
0.2487358114120217