Code from BasicOptimizer.scala:75 executed in 3665.34 seconds (19.201 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1003216532786200
Reset training subject: 1003269624640700
Adding measurement 76f29779 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.71000498253566;dx=-1.6360287800302033E-7
New Minimum: 5.71000498253566 > 5.710004788820564
WOLFE (weak): th(2.154434690031884)=5.710004788820564; dx=-1.6328295023116318E-7 evalInputDelta=1.9371509552001953E-7
New Minimum: 5.710004788820564 > 5.710004252378761
WOLFE (weak): th(4.308869380063768)=5.710004252378761; dx=-1.63253594375824E-7 evalInputDelta=7.301568984985352E-7
New Minimum: 5.710004252378761 > 5.710002288504361
WOLFE (weak): th(12.926608140191302)=5.710002288504361; dx=-1.6326749420091534E-7 evalInputDelta=2.694031298489108E-6
New Minimum: 5.710002288504361 > 5.709994269093989
WOLFE (weak): th(51.70643256076521)=5.709994269093989; dx=-1.6324448892774084E-7 evalInputDelta=1.0713441670695545E-5
New Minimum: 5.709994269093989 > 5.709952494122053
WOLFE (weak): th(258.53216280382605)=5.709952494122053; dx=-1.6324280489585892E-7 evalInputDelta=5.248841360661061E-5
New Minimum: 5.709952494122053 > 5.70969012154171
WOLFE (weak): th(1551.1929768229563)=5.70969012154171; dx=-1.6321319857267378E-7 evalInputDelta=3.1486099394939515E-4
New Minimum: 5.70969012154171 > 5.7078019741217
WOLFE (weak): th(10858.350837760694)=5.7078019741217; dx=-1.631370393890763E-7 evalInputDelta=0.002203008413959928
New Minimum: 5.7078019741217 > 5.692421410541863
WOLFE (weak): th(86866.80670208555)=5.692421410541863; dx=-1.6260026694104704E-7 evalInputDelta=0.017583571993796454
New Minimum: 5.692421410541863 > 5.554982033464742
WOLFE (weak): th(781801.26031877)=5.554982033464742; dx=-1.5659614442410548E-7 evalInputDelta=0.15502294907091763
New Minimum: 5.554982033464742 > 4.658717894736098
END: th(7818012.6031877)=4.658717894736098; dx=-6.703125953085462E-8 evalInputDelta=1.0512870877995617
Fitness changed from 5.71000498253566 to 4.658717894736098
Iteration 1 complete. Error: 4.65871789473
...skipping 10773 bytes...
2.6607723582417817
Iteration 13 complete. Error: 2.6607723582417817 Total: 268.7698; Orientation: 1.8278; Line Search: 213.8776
Adding measurement cbed52b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.042e+03, gradient 9.045e-05, dot -0.836; [9e6e4c60-dcf2-4b1d-86ec-1299e1866385 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.7901547138658462, 2.744809008206452, 2.733498867458683, 2.6607723582417817
LBFGS Accumulation History: 3 points
Removed measurement 52f8e8af to history. Total: 4
th(0)=2.6607723582417817;dx=-8.172648024825057E-9
New Minimum: 2.6607723582417817 > 2.6238748931948224
END: th(1.6245560917644735E7)=2.6238748931948224; dx=-3.831109098335801E-9 evalInputDelta=0.036897465046959343
Fitness changed from 2.6607723582417817 to 2.6238748931948224
Iteration 14 complete. Error: 2.6238748931948224 Total: 161.9432; Orientation: 1.8205; Line Search: 106.9931
Adding measurement 7d060593 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.446e+03, gradient 9.217e-05, dot -0.732; [9e6e4c60-dcf2-4b1d-86ec-1299e1866385 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.744809008206452, 2.733498867458683, 2.6607723582417817, 2.6238748931948224
LBFGS Accumulation History: 3 points
Removed measurement 5084c525 to history. Total: 4
th(0)=2.6238748931948224;dx=-8.486507955753628E-9
Armijo: th(3.500000000000002E7)=2.7060979395946347; dx=2.6644624826135304E-10 evalInputDelta=-0.08222304639981237
New Minimum: 2.6238748931948224 > 2.5999902209681185
END: th(1.750000000000001E7)=2.5999902209681185; dx=-2.760067895309604E-9 evalInputDelta=0.023884672226703874
Fitness changed from 2.6238748931948224 to 2.5999902209681185
Iteration 15 complete. Error: 2.5999902209681185 Total: 213.8007; Orientation: 1.7961; Line Search: 158.3883
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 15: 2.5999902209681185 (> -Infinity) after 3665.329s (< 3600.000s)
2.5999902209681185