Code from BasicOptimizer.scala:75 executed in 2430.61 seconds (21.267 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1495327711296800
Reset training subject: 1495357984638200
Adding measurement 7769d9b6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-27.336309427705903;dx=-5.379167453629633E-5
New Minimum: -27.336309427705903 > -27.336328386329114
WOLFE (weak): th(2.154434690031884)=-27.336328386329114; dx=-5.3246834137986525E-5 evalInputDelta=1.895862321177333E-5
New Minimum: -27.336328386329114 > -27.33634745174398
WOLFE (weak): th(4.308869380063768)=-27.33634745174398; dx=-5.268556165157001E-5 evalInputDelta=3.802403807640076E-5
New Minimum: -27.33634745174398 > -27.33642171261211
WOLFE (weak): th(12.926608140191302)=-27.33642171261211; dx=-5.05094841185754E-5 evalInputDelta=1.1228490620851517E-4
New Minimum: -27.33642171261211 > -27.33674331040432
END: th(51.70643256076521)=-27.33674331040432; dx=-4.473244398467986E-5 evalInputDelta=4.338826984167099E-4
Fitness changed from -27.336309427705903 to -27.33674331040432
Iteration 1 complete. Error: -27.33674331040432 Total: 242.6520; Orientation: 0.0267; Line Search: 152.0136
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 71a2df1 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-27.33674331040432;dx=-4.558758503925887E-5
New Minimum: -27.33674331040432 > -27.33767196815461
WOLFE (weak): th(111.39813200670669)=-27.33767196815461; dx=-4.573689848750243E-5 evalInputDelta=9.286577502898297E-4
New Minimum: -27.33767196815461 > -27.33860244570921
WOLFE (weak): th(222.79626401341338)=-27.33860244570921; dx=-4.5822890062803736E-5 evalInputDelta=0.001859135304890458
New Minimum: -27.33860244570921 > -27.342421618600685
WOLFE (weak): th(668.3887920402401)=-27.342421618600685; dx=-4.626466383985079E-5 evalInputDelta=0.005678308196365833
New Minimum: -27.342421618600685 > -27.35940447015067
WOLFE (weak): th(2673.5551681609604)=-27.35940447015067; dx=-4.126490439741989E-5 evalInputDelta=0
...skipping 14605 bytes...
ejected: LBFGS Orientation magnitude: 3.063e+03, gradient 1.773e-03, dot -0.700; [3494d270-965c-4121-b5c4-feb811a428d0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.84780288177232, -29.888255865002673, -30.0276975814874, -30.10441246132056
LBFGS Accumulation History: 3 points
Removed measurement 65015a49 to history. Total: 4
th(0)=-30.10441246132056;dx=-3.1436736586773524E-6
Armijo: th(3590724.483386475)=-29.847423534529906; dx=5.422333807698824E-7 evalInputDelta=-0.25698892679065466
New Minimum: -30.10441246132056 > -30.148948666950066
WOLF (strong): th(1795362.2416932376)=-30.148948666950066; dx=1.824528550324691E-7 evalInputDelta=0.04453620562950533
New Minimum: -30.148948666950066 > -30.218081952693563
END: th(598454.0805644125)=-30.218081952693563; dx=-7.248158903683043E-7 evalInputDelta=0.11366949137300253
Fitness changed from -30.10441246132056 to -30.218081952693563
Iteration 19 complete. Error: -30.218081952693563 Total: 152.2539; Orientation: 0.7916; Line Search: 121.1655
Adding measurement 539ee811 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.351e+03, gradient 1.332e-03, dot -0.776; [3494d270-965c-4121-b5c4-feb811a428d0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.888255865002673, -30.0276975814874, -30.10441246132056, -30.218081952693563
LBFGS Accumulation History: 3 points
Removed measurement 1b632442 to history. Total: 4
th(0)=-30.218081952693563;dx=-1.774394546117872E-6
New Minimum: -30.218081952693563 > -30.287115995772183
END: th(1289330.2315591061)=-30.287115995772183; dx=-2.74176272863235E-7 evalInputDelta=0.06903404307862004
Fitness changed from -30.218081952693563 to -30.287115995772183
Iteration 20 complete. Error: -30.287115995772183 Total: 91.1518; Orientation: 0.7829; Line Search: 60.1135
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: -30.287115995772183 (> -Infinity) after 2430.606s (< 3600.000s)
-30.287115995772183