BasicOptimizer.scala:89 executed in 8.50 seconds (0.501 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 149619680266600
Reset training subject: 149619810794400
Adding measurement 3f2838de to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.6094063673953358;dx=-4.119966479717844E-4
New Minimum: -1.6094063673953358 > -1.6108117992119628
WOLFE (weak): th(2.154434690031884)=-1.6108117992119628; dx=-4.0700874868519873E-4 evalInputDelta=0.0014054318166270008
New Minimum: -1.6108117992119628 > -1.6122149218045387
WOLFE (weak): th(4.308869380063768)=-1.6122149218045387; dx=-4.050562789301837E-4 evalInputDelta=0.0028085544092029124
New Minimum: -1.6122149218045387 > -1.6178335445012682
WOLFE (weak): th(12.926608140191302)=-1.6178335445012682; dx=-4.113191863655354E-4 evalInputDelta=0.008427177105932326
New Minimum: -1.6178335445012682 > -1.643129174234029
WOLFE (weak): th(51.70643256076521)=-1.643129174234029; dx=-4.1085721961991973E-4 evalInputDelta=0.03372280683869322
New Minimum: -1.643129174234029 > -1.7763367090622353
WOLFE (weak): th(258.53216280382605)=-1.7763367090622353; dx=-4.0891012695553014E-4 evalInputDelta=0.16693034166689946
New Minimum: -1.7763367090622353 > -2.59870721685953
WOLFE (weak): th(1551.1929768229563)=-2.59870721685953; dx=-3.8801729221966095E-4 evalInputDelta=0.9893008494641942
New Minimum: -2.59870721685953 > -4.454672216055102
END: th(10858.350837760694)=-4.454672216055102; dx=-1.1707994924061055E-5 evalInputDelta=2.8452658486597664
Fitness changed from -1.6094063673953358 to -4.454672216055102
Iteration 1 complete. Error: -4.454672216055102 Total: 1.2482; Orientation: 0.0016; Line Search: 0.8944
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 65e4784c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.454672216055102;dx=-5.012226063344388E-4
Armijo: th(23393.607721408407)=-3.6787350830230405; dx=5.605551894674644E-5 evalInputDelta=-0.7759371330320617
New Minimum: -4.454672216055102 > -6.363384434669582
WOLF (strong): th(11696.803860704204)=-6.363384434669582; dx=1.1186980774119486E-4 evalInputDelta=1.9087122186144798
New Minimum: -6.363384434669582 > -6.8410190878949
END: th(3898.9346202347347)=-6.8410190878949; dx=-2.764482938538104E-4 evalInputDelta=2.386346871839798
Fitness changed from -4.454672216055102 to -6.8410190878949
Iteration 2 complete. Error: -6.8410190878949 Total: 0.5714; Orientation: 0.0015; Line Search: 0.4483
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1f35f576 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.8410190878949;dx=-3.6240634437395493E-4
New Minimum: -6.8410190878949 > -8.843294186177989
END: th(8400.000000000002)=-8.843294186177989; dx=-5.077543575473702E-5 evalInputDelta=2.0022750982830884
Fitness changed from -6.8410190878949 to -8.843294186177989
Iteration 3 complete. Error: -8.843294186177989 Total: 0.3463; Orientation: 0.0015; Line Search: 0.2253
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3d685a7c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.082e+02, gradient 2.062e-02, dot -0.487; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.843294186177989, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 3d685a7c to history. Total: 3
th(0)=-8.843294186177989;dx=-4.1524361985222514E-4
Armijo: th(18097.251396267828)=-7.0984277243571405; dx=1.073088745703691E-4 evalInputDelta=-1.7448664618208483
New Minimum: -8.843294186177989 > -9.633509644710381
WOLF (strong): th(9048.625698133914)=-9.633509644710381; dx=1.3125371593186377E-4 evalInputDelta=0.7902154585323924
New Minimum: -9.633509644710381 > -10.08200078542548
END: th(3016.208566044638)=-10.08200078542548; dx=-1.5661543918967408E-4 evalInputDelta=1.2387065992474913
Fitness changed from -8.843294186177989 to -10.08200078542548
Iteration 4 complete. Error: -10.08200078542548 Total: 0.6022; Orientation: 0.0285; Line Search: 0.4534
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 46eb64c2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.087e+03, gradient 1.541e-02, dot -0.600; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.08200078542548, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 46eb64c2 to history. Total: 3
th(0)=-10.08200078542548;dx=-2.2542422271545307E-4
New Minimum: -10.08200078542548 > -11.42078403340408
END: th(6498.224367057892)=-11.42078403340408; dx=-1.1990116121255645E-4 evalInputDelta=1.3387832479786006
Fitness changed from -10.08200078542548 to -11.42078403340408
Iteration 5 complete. Error: -11.42078403340408 Total: 0.3846; Orientation: 0.0301; Line Search: 0.2314
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1b2cc726 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.698e+03, gradient 1.640e-02, dot -0.593; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.42078403340408, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 1b2cc726 to history. Total: 3
th(0)=-11.42078403340408;dx=-2.565259155482643E-4
New Minimum: -11.42078403340408 > -11.50612258905171
WOLF (strong): th(14000.000000000004)=-11.50612258905171; dx=6.21095869609575E-5 evalInputDelta=0.08533855564762938
New Minimum: -11.50612258905171 > -12.416387383299089
END: th(7000.000000000002)=-12.416387383299089; dx=-3.751891669992536E-5 evalInputDelta=0.9956033498950081
Fitness changed from -11.42078403340408 to -12.416387383299089
Iteration 6 complete. Error: -12.416387383299089 Total: 0.5054; Orientation: 0.0274; Line Search: 0.3552
Adding measurement 638be51b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.605e+03, gradient 1.946e-02, dot -0.433; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.416387383299089, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 638be51b to history. Total: 3
th(0)=-12.416387383299089;dx=-3.680039132057009E-4
Armijo: th(15081.04283022319)=-9.30575411333463; dx=1.4403463089112702E-4 evalInputDelta=-3.110633269964458
New Minimum: -12.416387383299089 > -12.46256508927147
WOLF (strong): th(7540.521415111595)=-12.46256508927147; dx=1.9249973189827183E-4 evalInputDelta=0.04617770597238113
New Minimum: -12.46256508927147 > -13.231204175341864
END: th(2513.5071383705317)=-13.231204175341864; dx=-1.592570261075256E-4 evalInputDelta=0.8148167920427749
Fitness changed from -12.416387383299089 to -13.231204175341864
Iteration 7 complete. Error: -13.231204175341864 Total: 0.6238; Orientation: 0.0276; Line Search: 0.4828
Adding measurement 7b6f51dc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.012e+03, gradient 1.381e-02, dot -0.628; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.231204175341864, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 7b6f51dc to history. Total: 3
th(0)=-13.231204175341864;dx=-1.7728357562827348E-4
New Minimum: -13.231204175341864 > -14.09951660165344
END: th(5415.186972548244)=-14.09951660165344; dx=-1.1736429456887672E-4 evalInputDelta=0.8683124263115758
Fitness changed from -13.231204175341864 to -14.09951660165344
Iteration 8 complete. Error: -14.09951660165344 Total: 0.3917; Orientation: 0.0345; Line Search: 0.2382
Adding measurement 4fdf5f6a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.422e+03, gradient 1.397e-02, dot -0.625; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.09951660165344, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 4fdf5f6a to history. Total: 3
th(0)=-14.09951660165344;dx=-1.8747273008976574E-4
New Minimum: -14.09951660165344 > -15.046511235906598
END: th(11666.666666666672)=-15.046511235906598; dx=-3.7677447367987475E-5 evalInputDelta=0.9469946342531586
Fitness changed from -14.09951660165344 to -15.046511235906598
Iteration 9 complete. Error: -15.046511235906598 Total: 0.3647; Orientation: 0.0271; Line Search: 0.2297
Adding measurement 6258d280 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.872e+03, gradient 2.076e-02, dot -0.421; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.046511235906598, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 6258d280 to history. Total: 3
th(0)=-15.046511235906598;dx=-4.221516810726508E-4
Armijo: th(25135.071383705323)=-8.424514992924715; dx=1.2213993040603412E-4 evalInputDelta=-6.621996242981883
Armijo: th(12567.535691852661)=-12.85279134711207; dx=2.2470116541399134E-4 evalInputDelta=-2.1937198887945275
New Minimum: -15.046511235906598 > -16.13778788740272
WOLF (strong): th(4189.178563950887)=-16.13778788740272; dx=2.2272790267271903E-5 evalInputDelta=1.0912766514961216
END: th(1047.2946409877218)=-15.54263567594422; dx=-3.1400280451042363E-4 evalInputDelta=0.49612444003762235
Fitness changed from -15.046511235906598 to -16.13778788740272
Iteration 10 complete. Error: -16.13778788740272 Total: 1.2459; Orientation: 0.0288; Line Search: 1.1055
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 720237f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.603e+03, gradient 1.796e-02, dot -0.473; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.13778788740272, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 720237f7 to history. Total: 3
th(0)=-16.13778788740272;dx=-3.140102373839193E-4
New Minimum: -16.13778788740272 > -16.700024000319427
END: th(2256.3279052284356)=-16.700024000319427; dx=-1.0360294288938544E-4 evalInputDelta=0.5622361129167075
Fitness changed from -16.13778788740272 to -16.700024000319427
Iteration 11 complete. Error: -16.700024000319427 Total: 0.3816; Orientation: 0.0268; Line Search: 0.2267
Adding measurement 40490d72 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.058e+03, gradient 1.292e-02, dot -0.611; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.700024000319427, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 40490d72 to history. Total: 3
th(0)=-16.700024000319427;dx=-1.5750007708997552E-4
New Minimum: -16.700024000319427 > -17.349752807033248
END: th(4861.111111111114)=-17.349752807033248; dx=-9.296251828355873E-5 evalInputDelta=0.6497288067138207
Fitness changed from -16.700024000319427 to -17.349752807033248
Iteration 12 complete. Error: -17.349752807033248 Total: 0.3749; Orientation: 0.0383; Line Search: 0.2264
Adding measurement 6e930408 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.482e+03, gradient 1.425e-02, dot -0.599; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.349752807033248, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 6e930408 to history. Total: 3
th(0)=-17.349752807033248;dx=-1.95229088085621E-4
New Minimum: -17.349752807033248 > -17.54345128403212
WOLF (strong): th(10472.946409877219)=-17.54345128403212; dx=4.122487892101251E-5 evalInputDelta=0.19369847699887188
New Minimum: -17.54345128403212 > -17.866496035538816
END: th(5236.473204938609)=-17.866496035538816; dx=-2.9829842417288147E-5 evalInputDelta=0.5167432285055682
Fitness changed from -17.349752807033248 to -17.866496035538816
Iteration 13 complete. Error: -17.866496035538816 Total: 0.4765; Orientation: 0.0266; Line Search: 0.3406
Adding measurement acc4e10 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.935e+03, gradient 1.787e-02, dot -0.467; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.866496035538816, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement acc4e10 to history. Total: 3
th(0)=-17.866496035538816;dx=-3.096078379933589E-4
Armijo: th(11281.639526142179)=-16.099457523375918; dx=1.6446073769998073E-4 evalInputDelta=-1.7670385121628982
New Minimum: -17.866496035538816 > -18.173161326069007
WOLF (strong): th(5640.819763071089)=-18.173161326069007; dx=1.357634552655295E-4 evalInputDelta=0.30666529053019076
New Minimum: -18.173161326069007 > -18.419336689195735
END: th(1880.2732543570298)=-18.419336689195735; dx=-1.4118659794762998E-4 evalInputDelta=0.5528406536569186
Fitness changed from -17.866496035538816 to -18.419336689195735
Iteration 14 complete. Error: -18.419336689195735 Total: 0.6020; Orientation: 0.0259; Line Search: 0.4636
Adding measurement 6f7323b3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.463e+03, gradient 1.239e-02, dot -0.644; [a1551424-ecf5-47df-97eb-3cc4bbc007a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.419336689195735, -6.8410190878949, -4.454672216055102, -1.6094063673953358
LBFGS Accumulation History: 3 points
Removed measurement 6f7323b3 to history. Total: 3
th(0)=-18.419336689195735;dx=-1.4603613040294305E-4
New Minimum: -18.419336689195735 > -18.931897681006536
END: th(4050.925925925929)=-18.931897681006536; dx=-1.0894586835075352E-4 evalInputDelta=0.5125609918108012
Fitness changed from -18.419336689195735 to -18.931897681006536
Iteration 15 complete. Error: -18.931897681006536 Total: 0.3706; Orientation: 0.0277; Line Search: 0.2296
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -18.931897681006536 (> -Infinity) after 8.500s (< 5400.000s)
Returns
-18.931897681006536