Code from BasicOptimizer.scala:75 executed in 981.42 seconds (10.822 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1234149711580400
Reset training subject: 1234158256891200
Adding measurement 5d3b58ca to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=5.1561573490079535;dx=-2.426132182546419E-5
New Minimum: 5.1561573490079535 > 5.156134398610975
WOLFE (weak): th(2.154434690031884)=5.156134398610975; dx=-2.426110982829732E-5 evalInputDelta=2.2950396978416165E-5
New Minimum: 5.156134398610975 > 5.156111998980197
WOLFE (weak): th(4.308869380063768)=5.156111998980197; dx=-2.4260976297328376E-5 evalInputDelta=4.5350027756185796E-5
New Minimum: 5.156111998980197 > 5.156020872886941
WOLFE (weak): th(12.926608140191302)=5.156020872886941; dx=-2.4260434162895896E-5 evalInputDelta=1.3647612101230777E-4
New Minimum: 5.156020872886941 > 5.155612190460412
WOLFE (weak): th(51.70643256076521)=5.155612190460412; dx=-2.425308420579512E-5 evalInputDelta=5.451585475411136E-4
New Minimum: 5.155612190460412 > 5.1534330099113514
WOLFE (weak): th(258.53216280382605)=5.1534330099113514; dx=-2.4227249723269397E-5 evalInputDelta=0.0027243390966020797
New Minimum: 5.1534330099113514 > 5.139876201085166
WOLFE (weak): th(1551.1929768229563)=5.139876201085166; dx=-2.40326755657117E-5 evalInputDelta=0.01628114792278712
New Minimum: 5.139876201085166 > 5.045020031467437
WOLFE (weak): th(10858.350837760694)=5.045020031467437; dx=-2.2874650903117126E-5 evalInputDelta=0.11113731754051681
New Minimum: 5.045020031467437 > 4.441147934873828
END: th(86866.80670208555)=4.441147934873828; dx=-1.4586931679779035E-5 evalInputDelta=0.7150094141341254
Fitness changed from 5.1561573490079535 to 4.441147934873828
Iteration 1 complete. Error: 4.441147934873828 Total: 97.4930; Orientation: 0.0187; Line Search: 72.9260
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7b2ccba5 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=4.441147934873828;dx=-1.1413232592324706E-5
New Minim
...skipping 24120 bytes...
754; [8cce2303-05da-4ae0-a41a-11c0e13ef481 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.0827835409669055, 0.07318051893568799, 0.04892514907706233, 0.03320144386549599
LBFGS Accumulation History: 3 points
Removed measurement 4a6c0f38 to history. Total: 4
th(0)=0.03320144386549599;dx=-4.1117579124384445E-8
Armijo: th(6963975.01613715)=0.05509933650815481; dx=3.5077557775421326E-9 evalInputDelta=-0.021897892642658817
New Minimum: 0.03320144386549599 > 0.022867153776466503
END: th(3481987.508068575)=0.022867153776466503; dx=-1.067046650363563E-8 evalInputDelta=0.010334290089029488
Fitness changed from 0.03320144386549599 to 0.022867153776466503
Iteration 29 complete. Error: 0.022867153776466503 Total: 32.2954; Orientation: 0.2020; Line Search: 24.0917
Adding measurement 19fec3d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.869e+03, gradient 2.360e-04, dot -0.633; [8cce2303-05da-4ae0-a41a-11c0e13ef481 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.07318051893568799, 0.04892514907706233, 0.03320144386549599, 0.022867153776466503
LBFGS Accumulation History: 3 points
Removed measurement 64021427 to history. Total: 4
th(0)=0.022867153776466503;dx=-5.571128954330931E-8
Armijo: th(7501714.677640612)=0.13005190384364396; dx=4.946365524442517E-8 evalInputDelta=-0.10718475006717745
Armijo: th(3750857.338820306)=0.023973255473082503; dx=1.845056985526655E-8 evalInputDelta=-0.0011061016966159992
New Minimum: 0.022867153776466503 > 0.006625885042367385
END: th(1250285.7796067686)=0.006625885042367385; dx=-2.9107311693218372E-8 evalInputDelta=0.016241268734099118
Fitness changed from 0.022867153776466503 to 0.006625885042367385
Iteration 30 complete. Error: 0.006625885042367385 Total: 39.8002; Orientation: 0.2019; Line Search: 31.7907
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: 0.006625885042367385 (> -Infinity) after 981.405s (< 3600.000s)
0.006625885042367385