BasicOptimizer.scala:89 executed in 135.00 seconds (1.687 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 72646735705400
Reset training subject: 72649385810000
Adding measurement 7cd9e078 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-22.71501090122418;dx=-8.133337401509086E-7
New Minimum: -22.71501090122418 > -22.71501203764086
END: th(2.154434690031884)=-22.71501203764086; dx=-6.417984898059866E-7 evalInputDelta=1.136416681646324E-6
Fitness changed from -22.71501090122418 to -22.71501203764086
Iteration 1 complete. Error: -22.71501203764086 Total: 12.9962; Orientation: 0.0589; Line Search: 5.1135
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 595d41be to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-22.71501203764086;dx=-6.395171757682857E-7
New Minimum: -22.71501203764086 > -22.715015446890572
WOLFE (weak): th(4.641588833612779)=-22.715015446890572; dx=-6.396051776265488E-7 evalInputDelta=3.409249710983886E-6
New Minimum: -22.715015446890572 > -22.71501999255606
WOLFE (weak): th(9.283177667225559)=-22.71501999255606; dx=-6.392601186147784E-7 evalInputDelta=7.954915197672108E-6
New Minimum: -22.71501999255606 > -22.715033629547058
WOLFE (weak): th(27.849533001676676)=-22.715033629547058; dx=-6.38880710768682E-7 evalInputDelta=2.159190619721585E-5
New Minimum: -22.715033629547058 > -22.715092723080115
WOLFE (weak): th(111.3981320067067)=-22.715092723080115; dx=-6.384799051503021E-7 evalInputDelta=8.068543925432436E-5
New Minimum: -22.715092723080115 > -22.715412052781495
WOLFE (weak): th(556.9906600335335)=-22.715412052781495; dx=-6.381931671522713E-7 evalInputDelta=4.0001514063447985E-4
New Minimum: -22.715412052781495 > -22.71740747764398
WOLFE (weak): th(3341.9439602012008)=-22.71740747764398; dx=-6.381806769959524E-7 evalInputDelta=0.002395440003120086
New Minimum: -22.71740747764398 > -22.731738478133053
WOLFE (weak): th(23393.607721408407)=-22.731738478133053; dx=-6.35403182222871E-7 evalInputDelta=0.016726440492192296
New Minimum: -22.731738478133053 > -22.846282056916497
WOLFE (weak): th(187148.86177126726)=-22.846282056916497; dx=-6.102888718506448E-7 evalInputDelta=0.13127001927563597
New Minimum: -22.846282056916497 > -23.53835226330812
END: th(1684339.7559414054)=-23.53835226330812; dx=-2.1067385046785077E-7 evalInputDelta=0.8233402256672591
Fitness changed from -22.71501203764086 to -23.53835226330812
Iteration 2 complete. Error: -23.53835226330812 Total: 29.2837; Orientation: 0.0590; Line Search: 26.0553
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 11390527 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-23.53835226330812;dx=-5.73907976951279E-7
New Minimum: -23.53835226330812 > -24.23410346156714
END: th(3628800.0000000005)=-24.23410346156714; dx=-4.9608989460403763E-8 evalInputDelta=0.6957511982590212
Fitness changed from -23.53835226330812 to -24.23410346156714
Iteration 3 complete. Error: -24.23410346156714 Total: 7.9673; Orientation: 0.0518; Line Search: 4.9990
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 268709c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.672e+03, gradient 8.043e-04, dot -0.484; [f6b1dcb8-5097-4fb8-9a64-937279256773 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.23410346156714, -23.53835226330812, -22.71501203764086, -22.71501090122418
LBFGS Accumulation History: 3 points
Removed measurement 268709c4 to history. Total: 3
th(0)=-24.23410346156714;dx=-6.462997744849163E-7
New Minimum: -24.23410346156714 > -24.972457816488248
WOLF (strong): th(7818012.603187701)=-24.972457816488248; dx=2.2059214200337296E-8 evalInputDelta=0.7383543549211069
New Minimum: -24.972457816488248 > -25.25344069966546
WOLF (strong): th(3909006.3015938504)=-25.25344069966546; dx=7.677031795954027E-8 evalInputDelta=1.0193372380983199
END: th(1303002.1005312835)=-25.032661633272387; dx=-3.973940901280752E-7 evalInputDelta=0.7985581717052455
Fitness changed from -24.23410346156714 to -25.25344069966546
Iteration 4 complete. Error: -25.25344069966546 Total: 14.8566; Orientation: 1.0413; Line Search: 10.6586
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2d8cf70f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.214e+03, gradient 8.109e-04, dot -0.294; [f6b1dcb8-5097-4fb8-9a64-937279256773 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.25344069966546, -23.53835226330812, -22.71501203764086, -22.71501090122418
LBFGS Accumulation History: 3 points
Removed measurement 2d8cf70f to history. Total: 3
th(0)=-25.25344069966546;dx=-6.568473958326349E-7
New Minimum: -25.25344069966546 > -26.346604671872647
WOLF (strong): th(2807232.926569009)=-26.346604671872647; dx=6.689117503644698E-8 evalInputDelta=1.0931639722071864
END: th(1403616.4632845046)=-26.12828138797953; dx=-3.893128498172653E-7 evalInputDelta=0.8748406883140696
Fitness changed from -25.25344069966546 to -26.346604671872647
Iteration 5 complete. Error: -26.346604671872647 Total: 11.6016; Orientation: 1.0334; Line Search: 7.5872
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2d2652a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.188e+03, gradient 7.255e-04, dot -0.483; [f6b1dcb8-5097-4fb8-9a64-937279256773 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.346604671872647, -23.53835226330812, -22.71501203764086, -22.71501090122418
LBFGS Accumulation History: 3 points
Removed measurement 2d2652a5 to history. Total: 3
th(0)=-26.346604671872647;dx=-5.261714032497357E-7
New Minimum: -26.346604671872647 > -26.81551671792185
WOLF (strong): th(3024000.0000000005)=-26.81551671792185; dx=1.2062650810279518E-7 evalInputDelta=0.46891204604920134
New Minimum: -26.81551671792185 > -26.912266661747168
END: th(1512000.0000000002)=-26.912266661747168; dx=-8.532777721588503E-8 evalInputDelta=0.5656619898745205
Fitness changed from -26.346604671872647 to -26.912266661747168
Iteration 6 complete. Error: -26.912266661747168 Total: 11.9110; Orientation: 1.0086; Line Search: 7.7146
Adding measurement 6e5771f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.314e+04, gradient 5.730e-04, dot -0.675; [f6b1dcb8-5097-4fb8-9a64-937279256773 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.912266661747168, -23.53835226330812, -22.71501203764086, -22.71501090122418
LBFGS Accumulation History: 3 points
Removed measurement 6e5771f9 to history. Total: 3
th(0)=-26.912266661747168;dx=-3.2775669261884247E-7
New Minimum: -26.912266661747168 > -27.33425026154104
END: th(3257505.251328209)=-27.33425026154104; dx=-2.7282044658820962E-8 evalInputDelta=0.42198359979387234
Fitness changed from -26.912266661747168 to -27.33425026154104
Iteration 7 complete. Error: -27.33425026154104 Total: 8.9727; Orientation: 0.9725; Line Search: 5.3030
Adding measurement 7970605f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.119e+04, gradient 7.062e-04, dot -0.473; [f6b1dcb8-5097-4fb8-9a64-937279256773 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.33425026154104, -23.53835226330812, -22.71501203764086, -22.71501090122418
LBFGS Accumulation History: 3 points
Removed measurement 7970605f to history. Total: 3
th(0)=-27.33425026154104;dx=-4.987223497482456E-7
New Minimum: -27.33425026154104 > -27.344394718721304
WOLF (strong): th(7018082.316422524)=-27.344394718721304; dx=5.772710674860922E-8 evalInputDelta=0.0101444571802638
New Minimum: -27.344394718721304 > -27.780629271229706
WOLF (strong): th(3509041.158211262)=-27.780629271229706; dx=1.3918253162306534E-7 evalInputDelta=0.4463790096886662
New Minimum: -27.780629271229706 > -27.85622264690657
END: th(1169680.3860704207)=-27.85622264690657; dx=-2.505388507379127E-7 evalInputDelta=0.5219723853655296
Fitness changed from -27.33425026154104 to -27.85622264690657
Iteration 8 complete. Error: -27.85622264690657 Total: 13.9231; Orientation: 1.0043; Line Search: 10.4481
Adding measurement 31e8a822 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.628e+04, gradient 5.308e-04, dot -0.696; [f6b1dcb8-5097-4fb8-9a64-937279256773 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.85622264690657, -23.53835226330812, -22.71501203764086, -22.71501090122418
LBFGS Accumulation History: 3 points
Removed measurement 31e8a822 to history. Total: 3
th(0)=-27.85622264690657;dx=-2.813767380447736E-7
New Minimum: -27.85622264690657 > -28.303714247577368
END: th(2520000.000000001)=-28.303714247577368; dx=-6.452784557514715E-8 evalInputDelta=0.44749160067079785
Fitness changed from -27.85622264690657 to -28.303714247577368
Iteration 9 complete. Error: -28.303714247577368 Total: 8.9042; Orientation: 1.0895; Line Search: 5.3113
Adding measurement 33d363a3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.656e+04, gradient 6.244e-04, dot -0.560; [f6b1dcb8-5097-4fb8-9a64-937279256773 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.303714247577368, -23.53835226330812, -22.71501203764086, -22.71501090122418
LBFGS Accumulation History: 3 points
Removed measurement 33d363a3 to history. Total: 3
th(0)=-28.303714247577368;dx=-3.8984890253681687E-7
New Minimum: -28.303714247577368 > -28.42338263022725
WOLF (strong): th(5429175.418880349)=-28.42338263022725; dx=4.325998928736051E-8 evalInputDelta=0.11966838264988056
New Minimum: -28.42338263022725 > -28.62954785744675
WOLF (strong): th(2714587.7094401745)=-28.62954785744675; dx=6.995063090387496E-8 evalInputDelta=0.32583360986938104
END: th(904862.5698133915)=-28.614191412105242; dx=-1.9658172981776574E-7 evalInputDelta=0.31047716452787455
Fitness changed from -28.303714247577368 to -28.62954785744675
Iteration 10 complete. Error: -28.62954785744675 Total: 14.0857; Orientation: 0.9627; Line Search: 10.4237
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -28.62954785744675 (> -Infinity) after 135.003s (< 5400.000s)
Returns
-28.62954785744675