Code from BasicOptimizer.scala:75 executed in 1025.63 seconds (12.010 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1231843430108700
Reset training subject: 1231852391297700
Adding measurement 642ee49c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.206609284961616;dx=-2.831520595990937E-5
New Minimum: 5.206609284961616 > 5.2065829502309
WOLFE (weak): th(2.154434690031884)=5.2065829502309; dx=-2.8314834583365667E-5 evalInputDelta=2.6334730716115473E-5
New Minimum: 5.2065829502309 > 5.20655654738059
WOLFE (weak): th(4.308869380063768)=5.20655654738059; dx=-2.8314180913670587E-5 evalInputDelta=5.273758102664061E-5
New Minimum: 5.20655654738059 > 5.2064512431663985
WOLFE (weak): th(12.926608140191302)=5.2064512431663985; dx=-2.8314131388584254E-5 evalInputDelta=1.5804179521783368E-4
New Minimum: 5.2064512431663985 > 5.205976209103993
WOLFE (weak): th(51.70643256076521)=5.205976209103993; dx=-2.830317812097491E-5 evalInputDelta=6.330758576229556E-4
New Minimum: 5.205976209103993 > 5.20344752753793
WOLFE (weak): th(258.53216280382605)=5.20344752753793; dx=-2.8231052613388907E-5 evalInputDelta=0.0031617574236859625
New Minimum: 5.20344752753793 > 5.187751876281193
WOLFE (weak): th(1551.1929768229563)=5.187751876281193; dx=-2.7881309350006446E-5 evalInputDelta=0.018857408680423404
New Minimum: 5.187751876281193 > 5.078314576366033
WOLFE (weak): th(10858.350837760694)=5.078314576366033; dx=-2.6451664949867478E-5 evalInputDelta=0.12829470859558345
New Minimum: 5.078314576366033 > 4.3809905500951585
END: th(86866.80670208555)=4.3809905500951585; dx=-1.6860952608015207E-5 evalInputDelta=0.8256187348664579
Fitness changed from 5.206609284961616 to 4.3809905500951585
Iteration 1 complete. Error: 4.3809905500951585 Total: 100.8357; Orientation: 0.0206; Line Search: 75.0008
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 53d9826f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.3809905500951585;dx=-1.3042430238198761E-5
New Minimum: 4.
...skipping 24179 bytes...
1703E-8 evalInputDelta=0.0189709294066292
Fitness changed from -0.1585123729849766 to -0.18217983874912477
Iteration 28 complete. Error: -0.18217983874912477 Total: 50.5607; Orientation: 0.2152; Line Search: 42.1084
Adding measurement 7a04f730 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.691e+03, gradient 2.948e-04, dot -0.434; [0321ac39-da10-4073-b4b7-8e2a2cae0145 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.13356605462295992, -0.15715354192075323, -0.1585123729849766, -0.18217983874912477
LBFGS Accumulation History: 3 points
Removed measurement 18709cb2 to history. Total: 4
th(0)=-0.18217983874912477;dx=-8.688639804235495E-8
New Minimum: -0.18217983874912477 > -0.2043412171387307
END: th(979308.9866442868)=-0.2043412171387307; dx=-3.8100486568789976E-8 evalInputDelta=0.02216137838960594
Fitness changed from -0.18217983874912477 to -0.2043412171387307
Iteration 29 complete. Error: -0.2043412171387307 Total: 25.4318; Orientation: 0.2120; Line Search: 16.9501
Adding measurement 74b00247 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.854e+02, gradient 1.987e-04, dot -0.828; [0321ac39-da10-4073-b4b7-8e2a2cae0145 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.15715354192075323, -0.1585123729849766, -0.18217983874912477, -0.2043412171387307
LBFGS Accumulation History: 3 points
Removed measurement 7d2c345d to history. Total: 4
th(0)=-0.2043412171387307;dx=-3.949765703656532E-8
New Minimum: -0.2043412171387307 > -0.2213646632389524
END: th(2109857.253086422)=-0.2213646632389524; dx=-2.7929023324492885E-8 evalInputDelta=0.01702344610022169
Fitness changed from -0.2043412171387307 to -0.2213646632389524
Iteration 30 complete. Error: -0.2213646632389524 Total: 25.0073; Orientation: 0.2169; Line Search: 16.7231
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: -0.2213646632389524 (> -Infinity) after 1025.618s (< 3600.000s)
-0.2213646632389524