BasicOptimizer.scala:89 executed in 131.32 seconds (2.091 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 37266582979600
Reset training subject: 37269346718000
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -80.35599419403307 > -80.37813063315616
th(0)=-80.37813063315616;dx=-4.98477308395106E-6
New Minimum: -80.37813063315616 > -80.37813914763576
END: th(2.154434690031884)=-80.37813914763576; dx=-3.5904621698061713E-6 evalInputDelta=8.514479603149994E-6
Fitness changed from -80.35599419403307 to -80.37813914763576
Iteration 1 complete. Error: -80.37813914763576 Total: 13.4102; Orientation: 0.0422; Line Search: 5.3288
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 64993d0e to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-80.37813914763576;dx=-3.5681167286998473E-6
New Minimum: -80.37813914763576 > -80.37815617659226
WOLFE (weak): th(4.641588833612779)=-80.37815617659226; dx=-3.5653406556593387E-6 evalInputDelta=1.7028956492026737E-5
New Minimum: -80.37815617659226 > -80.3781774627828
WOLFE (weak): th(9.283177667225559)=-80.3781774627828; dx=-3.5650681845218237E-6 evalInputDelta=3.831514703733774E-5
New Minimum: -80.3781774627828 > -80.3782540930221
WOLFE (weak): th(27.849533001676676)=-80.3782540930221; dx=-3.5617739509086E-6 evalInputDelta=1.1494538632916829E-4
New Minimum: -80.3782540930221 > -80.37859892819482
WOLFE (weak): th(111.3981320067067)=-80.37859892819482; dx=-3.5581099316790113E-6 evalInputDelta=4.597805590549342E-4
New Minimum: -80.37859892819482 > -80.38041673853941
WOLFE (weak): th(556.9906600335335)=-80.38041673853941; dx=-3.558170925305489E-6 evalInputDelta=0.0022775909036454323
New Minimum: -80.38041673853941 > -80.39182926424836
WOLFE (weak): th(3341.9439602012008)=-80.39182926424836; dx=-3.5577295782130425E-6 evalInputDelta=0.013690116612593783
New Minimum: -80.39182926424836 > -80.47391255175964
WOLFE (weak): th(23393.607721408407)=-80.47391255175964; dx=-3.552246313744421E-6 evalInputDelta=0.09577340412387514
New Minimum: -80.47391255175964 > -81.13760180228255
WOLFE (weak): th(187148.86177126726)=-81.13760180228255; dx=-3.488320791697933E-6 evalInputDelta=0.7594626546467822
New Minimum: -81.13760180228255 > -86.61698800272136
END: th(1684339.7559414054)=-86.61698800272136; dx=-2.850268942081912E-6 evalInputDelta=6.2388488550855925
Fitness changed from -80.37813914763576 to -86.61698800272136
Iteration 2 complete. Error: -86.61698800272136 Total: 29.0537; Orientation: 0.0626; Line Search: 25.7482
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 43e6029f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-86.61698800272136;dx=-2.8255080158230537E-6
New Minimum: -86.61698800272136 > -95.76859189316254
END: th(3628800.0000000005)=-95.76859189316254; dx=-1.6969325575095454E-6 evalInputDelta=9.15160389044118
Fitness changed from -86.61698800272136 to -95.76859189316254
Iteration 3 complete. Error: -95.76859189316254 Total: 8.8617; Orientation: 0.0614; Line Search: 5.4606
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4c5a4eb0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-95.76859189316254;dx=-1.7561907271888034E-6
New Minimum: -95.76859189316254 > -107.09586484289122
END: th(7818012.603187701)=-107.09586484289122; dx=-8.295031253160713E-7 evalInputDelta=11.32727294972868
Fitness changed from -95.76859189316254 to -107.09586484289122
Iteration 4 complete. Error: -107.09586484289122 Total: 8.0766; Orientation: 0.0592; Line Search: 5.0998
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7bae20de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.056e+04, gradient 1.213e-03, dot -0.633; [58da565d-7f94-4ca2-aa1b-3683973f2434 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.09586484289122, -95.76859189316254, -86.61698800272136, -80.37813914763576
LBFGS Accumulation History: 3 points
Removed measurement 7bae20de to history. Total: 3
th(0)=-107.09586484289122;dx=-1.469760299692774E-6
New Minimum: -107.09586484289122 > -113.1294185853603
WOLF (strong): th(1.6843397559414055E7)=-113.1294185853603; dx=1.944314156456311E-7 evalInputDelta=6.033553742469081
New Minimum: -113.1294185853603 > -113.72427020097624
END: th(8421698.779707028)=-113.72427020097624; dx=-1.8497227788980146E-7 evalInputDelta=6.628405358085018
Fitness changed from -107.09586484289122 to -113.72427020097624
Iteration 5 complete. Error: -113.72427020097624 Total: 12.2458; Orientation: 1.0754; Line Search: 8.0995
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1860e4e1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.041e+04, gradient 1.413e-03, dot -0.482; [58da565d-7f94-4ca2-aa1b-3683973f2434 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -113.72427020097624, -95.76859189316254, -86.61698800272136, -80.37813914763576
LBFGS Accumulation History: 3 points
Removed measurement 1860e4e1 to history. Total: 3
th(0)=-113.72427020097624;dx=-1.9972149703274596E-6
Armijo: th(1.8144000000000004E7)=-108.75119435783063; dx=8.104751499150596E-7 evalInputDelta=-4.973075843145608
New Minimum: -113.72427020097624 > -117.5957161267584
WOLF (strong): th(9072000.000000002)=-117.5957161267584; dx=6.794771922143734E-7 evalInputDelta=3.8714459257821687
New Minimum: -117.5957161267584 > -118.68242816168576
END: th(3024000.0000000005)=-118.68242816168576; dx=-8.31419768892389E-7 evalInputDelta=4.958157960709528
Fitness changed from -113.72427020097624 to -118.68242816168576
Iteration 6 complete. Error: -118.68242816168576 Total: 14.2832; Orientation: 1.0849; Line Search: 10.2921
Adding measurement 7652d60e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.364e+04, gradient 9.394e-04, dot -0.725; [58da565d-7f94-4ca2-aa1b-3683973f2434 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -118.68242816168576, -95.76859189316254, -86.61698800272136, -80.37813914763576
LBFGS Accumulation History: 3 points
Removed measurement 7652d60e to history. Total: 3
th(0)=-118.68242816168576;dx=-8.812082890452221E-7
New Minimum: -118.68242816168576 > -123.54592699088323
END: th(6515010.502656418)=-123.54592699088323; dx=-4.841646353026183E-7 evalInputDelta=4.863498829197468
Fitness changed from -118.68242816168576 to -123.54592699088323
Iteration 7 complete. Error: -123.54592699088323 Total: 8.6843; Orientation: 1.0593; Line Search: 5.1330
Adding measurement a20802f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.705e+04, gradient 9.735e-04, dot -0.648; [58da565d-7f94-4ca2-aa1b-3683973f2434 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -123.54592699088323, -95.76859189316254, -86.61698800272136, -80.37813914763576
LBFGS Accumulation History: 3 points
Removed measurement a20802f to history. Total: 3
th(0)=-123.54592699088323;dx=-9.47581113607211E-7
New Minimum: -123.54592699088323 > -127.67608591747654
WOLF (strong): th(1.4036164632845048E7)=-127.67608591747654; dx=5.560620916808235E-8 evalInputDelta=4.13015892659331
END: th(7018082.316422524)=-127.45635099040831; dx=-1.9007588560152727E-7 evalInputDelta=3.9104239995250794
Fitness changed from -123.54592699088323 to -127.67608591747654
Iteration 8 complete. Error: -127.67608591747654 Total: 11.1313; Orientation: 1.0526; Line Search: 7.5886
Adding measurement 6aa41dbc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.509e+04, gradient 1.439e-03, dot -0.355; [58da565d-7f94-4ca2-aa1b-3683973f2434 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -127.67608591747654, -95.76859189316254, -86.61698800272136, -80.37813914763576
LBFGS Accumulation History: 3 points
Removed measurement 6aa41dbc to history. Total: 3
th(0)=-127.67608591747654;dx=-2.0684222823540607E-6
Armijo: th(1.5120000000000006E7)=-124.1858391930978; dx=1.2559690101891843E-6 evalInputDelta=-3.4902467243787356
New Minimum: -127.67608591747654 > -133.97816706582924
WOLF (strong): th(7560000.000000003)=-133.97816706582924; dx=5.852426660429948E-7 evalInputDelta=6.302081148352698
END: th(2520000.000000001)=-132.5209997889; dx=-1.274684597361496E-6 evalInputDelta=4.844913871423472
Fitness changed from -127.67608591747654 to -133.97816706582924
Iteration 9 complete. Error: -133.97816706582924 Total: 13.7145; Orientation: 1.0334; Line Search: 10.3082
Adding measurement 24eb3b40 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.420e+04, gradient 1.280e-03, dot -0.388; [58da565d-7f94-4ca2-aa1b-3683973f2434 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -133.97816706582924, -95.76859189316254, -86.61698800272136, -80.37813914763576
LBFGS Accumulation History: 3 points
Removed measurement 24eb3b40 to history. Total: 3
th(0)=-133.97816706582924;dx=-1.6372939406626458E-6
New Minimum: -133.97816706582924 > -138.62819854551347
WOLF (strong): th(5429175.418880349)=-138.62819854551347; dx=1.4898840909829612E-7 evalInputDelta=4.65003147968423
END: th(2714587.7094401745)=-137.77121881823757; dx=-7.762960050482335E-7 evalInputDelta=3.7930517524083314
Fitness changed from -133.97816706582924 to -138.62819854551347
Iteration 10 complete. Error: -138.62819854551347 Total: 11.4150; Orientation: 1.0117; Line Search: 7.8243
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -138.62819854551347 (> -Infinity) after 131.323s (< 5400.000s)
Returns
-138.62819854551347