BasicOptimizer.scala:89 executed in 130.85 seconds (1.629 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 335746249722500
Reset training subject: 335748863951300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -97.3985370002842 > -97.59855589590991
th(0)=-97.59855589590991;dx=-2.5136234007236444E-5
New Minimum: -97.59855589590991 > -97.59860530709348
END: th(2.154434690031884)=-97.59860530709348; dx=-2.147713163111122E-5 evalInputDelta=4.941118356782681E-5
Fitness changed from -97.3985370002842 to -97.59860530709348
Iteration 1 complete. Error: -97.59860530709348 Total: 12.9407; Orientation: 0.0368; Line Search: 5.0872
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4e19d9cb to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-97.59860530709348;dx=-2.15391295364934E-5
New Minimum: -97.59860530709348 > -97.59871895272077
WOLFE (weak): th(4.641588833612779)=-97.59871895272077; dx=-2.153460930204629E-5 evalInputDelta=1.1364562729454519E-4
New Minimum: -97.59871895272077 > -97.59883753932121
WOLFE (weak): th(9.283177667225559)=-97.59883753932121; dx=-2.1525357025427717E-5 evalInputDelta=2.3223222773083307E-4
New Minimum: -97.59883753932121 > -97.59930200211889
WOLFE (weak): th(27.849533001676676)=-97.59930200211889; dx=-2.1515253404707225E-5 evalInputDelta=6.966950254110316E-4
New Minimum: -97.59930200211889 > -97.60138217540383
WOLFE (weak): th(111.3981320067067)=-97.60138217540383; dx=-2.1509360878960677E-5 evalInputDelta=0.0027768683103488456
New Minimum: -97.60138217540383 > -97.61248391827061
WOLFE (weak): th(556.9906600335335)=-97.61248391827061; dx=-2.1505216087650884E-5 evalInputDelta=0.013878611177133848
New Minimum: -97.61248391827061 > -97.68182764849176
WOLFE (weak): th(3341.9439602012008)=-97.68182764849176; dx=-2.1484824475520475E-5 evalInputDelta=0.0832223413982831
New Minimum: -97.68182764849176 > -98.17918418987149
WOLFE (weak): th(23393.607721408407)=-98.17918418987149; dx=-2.1312442055487388E-5 evalInputDelta=0.5805788827780134
New Minimum: -98.17918418987149 > -102.08390650010762
WOLFE (weak): th(187148.86177126726)=-102.08390650010762; dx=-1.9835077289363395E-5 evalInputDelta=4.485301193014138
New Minimum: -102.08390650010762 > -124.4677851051101
END: th(1684339.7559414054)=-124.4677851051101; dx=-7.418359811448195E-6 evalInputDelta=26.869179798016617
Fitness changed from -97.59860530709348 to -124.4677851051101
Iteration 2 complete. Error: -124.4677851051101 Total: 29.1510; Orientation: 0.0583; Line Search: 25.9873
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7de12019 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-124.4677851051101;dx=-1.1305452169615255E-5
New Minimum: -124.4677851051101 > -147.35909394812217
END: th(3628800.0000000005)=-147.35909394812217; dx=-1.91270279146074E-6 evalInputDelta=22.89130884301207
Fitness changed from -124.4677851051101 to -147.35909394812217
Iteration 3 complete. Error: -147.35909394812217 Total: 8.1562; Orientation: 0.0619; Line Search: 5.1194
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 45834d59 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-147.35909394812217;dx=-1.0143700053708549E-5
New Minimum: -147.35909394812217 > -152.3771045100238
WOLF (strong): th(7818012.603187701)=-152.3771045100238; dx=2.215069846074516E-6 evalInputDelta=5.018010561901633
New Minimum: -152.3771045100238 > -161.79949599562167
WOLF (strong): th(3909006.3015938504)=-161.79949599562167; dx=6.640948575816282E-7 evalInputDelta=14.440402047499504
END: th(1303002.1005312835)=-158.2434135978341; dx=-4.756085081447426E-6 evalInputDelta=10.884319649711927
Fitness changed from -147.35909394812217 to -161.79949599562167
Iteration 4 complete. Error: -161.79949599562167 Total: 13.8997; Orientation: 0.0627; Line Search: 10.4464
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5e131b84 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.374e+04, gradient 3.219e-03, dot -0.540; [1b5822cf-aa9d-482a-9872-24adc17db34d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -161.79949599562167, -147.35909394812217, -124.4677851051101, -97.59860530709348
LBFGS Accumulation History: 3 points
Removed measurement 5e131b84 to history. Total: 3
th(0)=-161.79949599562167;dx=-1.0342399364854395E-5
New Minimum: -161.79949599562167 > -177.891725843946
END: th(2807232.926569009)=-177.891725843946; dx=-2.8637430756221843E-7 evalInputDelta=16.092229848324337
Fitness changed from -161.79949599562167 to -177.891725843946
Iteration 5 complete. Error: -177.891725843946 Total: 9.2953; Orientation: 1.0714; Line Search: 5.2801
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6567668a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.544e+04, gradient 2.556e-03, dot -0.575; [1b5822cf-aa9d-482a-9872-24adc17db34d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -177.891725843946, -147.35909394812217, -124.4677851051101, -97.59860530709348
LBFGS Accumulation History: 3 points
Removed measurement 6567668a to history. Total: 3
th(0)=-177.891725843946;dx=-6.530436831203293E-6
New Minimum: -177.891725843946 > -182.75843430777815
WOLF (strong): th(6048000.000000001)=-182.75843430777815; dx=2.2602936355432582E-6 evalInputDelta=4.86670846383214
New Minimum: -182.75843430777815 > -188.1674926551345
END: th(3024000.0000000005)=-188.1674926551345; dx=-3.0896946325800915E-7 evalInputDelta=10.275766811188504
Fitness changed from -177.891725843946 to -188.1674926551345
Iteration 6 complete. Error: -188.1674926551345 Total: 11.6911; Orientation: 1.0315; Line Search: 7.6039
Adding measurement 65a0931e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.094e+04, gradient 2.472e-03, dot -0.573; [1b5822cf-aa9d-482a-9872-24adc17db34d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -188.1674926551345, -147.35909394812217, -124.4677851051101, -97.59860530709348
LBFGS Accumulation History: 3 points
Removed measurement 65a0931e to history. Total: 3
th(0)=-188.1674926551345;dx=-6.100274290413049E-6
New Minimum: -188.1674926551345 > -188.23009793181515
WOLF (strong): th(6515010.502656418)=-188.23009793181515; dx=2.6415897992986733E-6 evalInputDelta=0.06260527668064242
New Minimum: -188.23009793181515 > -196.51073528285045
WOLF (strong): th(3257505.251328209)=-196.51073528285045; dx=6.045730721004046E-7 evalInputDelta=8.343242627715938
END: th(1085835.0837760696)=-193.99464299160843; dx=-3.5297901662154265E-6 evalInputDelta=5.82715033647392
Fitness changed from -188.1674926551345 to -196.51073528285045
Iteration 7 complete. Error: -196.51073528285045 Total: 14.0226; Orientation: 1.0410; Line Search: 10.4902
Adding measurement 6e9ea75 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.736e+04, gradient 2.585e-03, dot -0.506; [1b5822cf-aa9d-482a-9872-24adc17db34d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -196.51073528285045, -147.35909394812217, -124.4677851051101, -97.59860530709348
LBFGS Accumulation History: 3 points
Removed measurement 6e9ea75 to history. Total: 3
th(0)=-196.51073528285045;dx=-6.679734531428755E-6
New Minimum: -196.51073528285045 > -205.13164674482698
END: th(2339360.772140841)=-205.13164674482698; dx=-2.664664694285089E-7 evalInputDelta=8.620911461976533
Fitness changed from -196.51073528285045 to -205.13164674482698
Iteration 8 complete. Error: -205.13164674482698 Total: 8.6429; Orientation: 1.0080; Line Search: 5.1142
Adding measurement 12f59276 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.282e+04, gradient 2.004e-03, dot -0.626; [1b5822cf-aa9d-482a-9872-24adc17db34d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -205.13164674482698, -147.35909394812217, -124.4677851051101, -97.59860530709348
LBFGS Accumulation History: 3 points
Removed measurement 12f59276 to history. Total: 3
th(0)=-205.13164674482698;dx=-4.011858787745603E-6
New Minimum: -205.13164674482698 > -210.54623302476895
WOLF (strong): th(5040000.000000001)=-210.54623302476895; dx=9.55001925213436E-7 evalInputDelta=5.414586279941972
New Minimum: -210.54623302476895 > -211.61635231069988
END: th(2520000.0000000005)=-211.61635231069988; dx=-9.321303494356957E-7 evalInputDelta=6.4847055658729005
Fitness changed from -205.13164674482698 to -211.61635231069988
Iteration 9 complete. Error: -211.61635231069988 Total: 11.1446; Orientation: 1.0753; Line Search: 7.6625
Adding measurement 3637ffa0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.647e+04, gradient 1.876e-03, dot -0.632; [1b5822cf-aa9d-482a-9872-24adc17db34d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -211.61635231069988, -147.35909394812217, -124.4677851051101, -97.59860530709348
LBFGS Accumulation History: 3 points
Removed measurement 3637ffa0 to history. Total: 3
th(0)=-211.61635231069988;dx=-3.517749914960117E-6
New Minimum: -211.61635231069988 > -216.29163849525514
WOLF (strong): th(5429175.418880348)=-216.29163849525514; dx=6.82683194641766E-7 evalInputDelta=4.675286184555262
New Minimum: -216.29163849525514 > -217.25188715231494
END: th(2714587.709440174)=-217.25188715231494; dx=-6.87994125032247E-7 evalInputDelta=5.635534841615055
Fitness changed from -211.61635231069988 to -217.25188715231494
Iteration 10 complete. Error: -217.25188715231494 Total: 11.4717; Orientation: 1.0585; Line Search: 7.8869
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -217.25188715231494 (> -Infinity) after 130.849s (< 5400.000s)
Returns
-217.25188715231494