Code from BasicOptimizer.scala:75 executed in 3727.24 seconds (32.507 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1047974017959600
Reset training subject: 1048025069901300
Adding measurement 6d4b0ce9 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.5361071825027466;dx=-7.125224451128806E-8
New Minimum: -0.5361071825027466 > -0.5361072421073914
WOLFE (weak): th(2.154434690031884)=-0.5361072421073914; dx=-7.090090448618717E-8 evalInputDelta=5.9604644775390625E-8
New Minimum: -0.5361072421073914 > -0.5361073911190033
WOLFE (weak): th(4.308869380063768)=-0.5361073911190033; dx=-7.09074655005068E-8 evalInputDelta=2.086162567138672E-7
New Minimum: -0.5361073911190033 > -0.5361081063747406
WOLFE (weak): th(12.926608140191302)=-0.5361081063747406; dx=-7.091086343616852E-8 evalInputDelta=9.238719940185547E-7
New Minimum: -0.5361081063747406 > -0.5361111536622047
WOLFE (weak): th(51.70643256076521)=-0.5361111536622047; dx=-7.090148595456944E-8 evalInputDelta=3.9711594581604E-6
New Minimum: -0.5361111536622047 > -0.5361268445849419
WOLFE (weak): th(258.53216280382605)=-0.5361268445849419; dx=-7.089129082855524E-8 evalInputDelta=1.9662082195281982E-5
New Minimum: -0.5361268445849419 > -0.5362248942255974
WOLFE (weak): th(1551.1929768229563)=-0.5362248942255974; dx=-7.091188999563252E-8 evalInputDelta=1.1771172285079956E-4
New Minimum: -0.5362248942255974 > -0.5369306579232216
WOLFE (weak): th(10858.350837760694)=-0.5369306579232216; dx=-7.086932612796847E-8 evalInputDelta=8.234754204750061E-4
New Minimum: -0.5369306579232216 > -0.5426826998591423
WOLFE (weak): th(86866.80670208555)=-0.5426826998591423; dx=-7.065174748792813E-8 evalInputDelta=0.0065755173563957214
New Minimum: -0.5426826998591423 > -0.5942291468381882
WOLFE (weak): th(781801.26031877)=-0.5942291468381882; dx=-6.834905557502851E-8 evalInputDelta=0.05812196433544159
New Minimum: -0.5942291468381882 > -0.9970283880829811
END: th(7818012.6031877)=-0.9970283880829811; dx=-4.409452351215819E-8 evalInputDelta=0.4609212055802345
Fitness changed from -0.53610718250274
...skipping 11856 bytes...
: LBFGS Orientation magnitude: 5.354e+03, gradient 4.767e-05, dot -0.694; [79da38cb-7bc4-4b46-a6d6-430655e82192 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.3752291798591614, -2.3856198228895664, -2.3953576758503914, -2.40158624202013
LBFGS Accumulation History: 3 points
Removed measurement 616b5b05 to history. Total: 4
th(0)=-2.40158624202013;dx=-2.2713159744163905E-9
Armijo: th(5.250000000000004E7)=-2.372871320694685; dx=-1.3869546752068067E-10 evalInputDelta=-0.028714921325445175
New Minimum: -2.40158624202013 > -2.4059020951390266
WOLF (strong): th(2.625000000000002E7)=-2.4059020951390266; dx=5.853811195165813E-11 evalInputDelta=0.004315853118896484
New Minimum: -2.4059020951390266 > -2.412243865430355
END: th(8750000.000000006)=-2.412243865430355; dx=-1.411125254398774E-9 evalInputDelta=0.010657623410224915
Fitness changed from -2.40158624202013 to -2.412243865430355
Iteration 15 complete. Error: -2.412243865430355 Total: 262.8554; Orientation: 1.9959; Line Search: 209.7917
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 553eb8fc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.833e+03, gradient 3.541e-05, dot -0.942; [79da38cb-7bc4-4b46-a6d6-430655e82192 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.3856198228895664, -2.3953576758503914, -2.40158624202013, -2.412243865430355
LBFGS Accumulation History: 3 points
Removed measurement 45399077 to history. Total: 4
th(0)=-2.412243865430355;dx=-1.2528007800048562E-9
New Minimum: -2.412243865430355 > -2.4194544665515423
END: th(1.8851303537778996E7)=-2.4194544665515423; dx=-1.078279539198863E-9 evalInputDelta=0.00721060112118721
Fitness changed from -2.412243865430355 to -2.4194544665515423
Iteration 16 complete. Error: -2.4194544665515423 Total: 158.1120; Orientation: 1.9794; Line Search: 104.1267
Final threshold in iteration 16: -2.4194544665515423 (> -Infinity) after 3727.241s (< 3600.000s)
-2.4194544665515423