BasicOptimizer.scala:89 executed in 37.66 seconds (0.641 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 639007034906700
Reset training subject: 639007669990100
Adding measurement 3ca2fbca to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-20.170299375263475;dx=-9.010597639871724E-7
New Minimum: -20.170299375263475 > -20.17030311974152
WOLFE (weak): th(2.154434690031884)=-20.17030311974152; dx=-8.983372559597892E-7 evalInputDelta=3.7444780431883373E-6
New Minimum: -20.17030311974152 > -20.17030686421887
WOLFE (weak): th(4.308869380063768)=-20.17030686421887; dx=-8.99584311734797E-7 evalInputDelta=7.488955393597507E-6
New Minimum: -20.17030686421887 > -20.17031996988412
WOLFE (weak): th(12.926608140191302)=-20.17031996988412; dx=-8.974729965212939E-7 evalInputDelta=2.0594620643521466E-5
New Minimum: -20.17031996988412 > -20.17037800915643
WOLFE (weak): th(51.70643256076521)=-20.17037800915643; dx=-8.971862808798807E-7 evalInputDelta=7.863389295437173E-5
New Minimum: -20.17037800915643 > -20.170688797257007
WOLFE (weak): th(258.53216280382605)=-20.170688797257007; dx=-9.008702291173444E-7 evalInputDelta=3.894219935318688E-4
New Minimum: -20.170688797257007 > -20.17263392243847
WOLFE (weak): th(1551.1929768229563)=-20.17263392243847; dx=-9.004962022827381E-7 evalInputDelta=0.00233454717499626
New Minimum: -20.17263392243847 > -20.186617763895118
WOLFE (weak): th(10858.350837760694)=-20.186617763895118; dx=-8.966666465155566E-7 evalInputDelta=0.016318388631642478
New Minimum: -20.186617763895118 > -20.299727651904576
WOLFE (weak): th(86866.80670208555)=-20.299727651904576; dx=-8.916817624076651E-7 evalInputDelta=0.12942827664110013
New Minimum: -20.299727651904576 > -21.247408209856694
END: th(781801.26031877)=-21.247408209856694; dx=-7.842279849144874E-7 evalInputDelta=1.0771088345932185
Fitness changed from -20.170299375263475 to -21.247408209856694
Iteration 1 complete. Error: -21.247408209856694 Total: 7.2248; Orientation: 0.0077; Line Search: 5.4628
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5a9b177f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-21.247408209856694;dx=-8.365157644313728E-7
New Minimum: -21.247408209856694 > -22.96025715730497
END: th(1684339.7559414052)=-22.96025715730497; dx=-6.360466318012412E-7 evalInputDelta=1.7128489474482755
Fitness changed from -21.247408209856694 to -22.96025715730497
Iteration 2 complete. Error: -22.96025715730497 Total: 1.9496; Orientation: 0.0077; Line Search: 1.1405
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2841610c to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-22.96025715730497;dx=-7.506977171902886E-7
New Minimum: -22.96025715730497 > -25.615908783773065
END: th(3628800.0)=-25.615908783773065; dx=-4.7479697663899237E-7 evalInputDelta=2.655651626468096
Fitness changed from -22.96025715730497 to -25.615908783773065
Iteration 3 complete. Error: -25.615908783773065 Total: 1.6735; Orientation: 0.0071; Line Search: 1.0690
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 53396134 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.273e+04, gradient 8.112e-04, dot -0.630; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.615908783773065, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 53396134 to history. Total: 3
th(0)=-25.615908783773065;dx=-6.532685143369366E-7
New Minimum: -25.615908783773065 > -29.486402834214143
END: th(7818012.6031877)=-29.486402834214143; dx=-3.1076961032431037E-7 evalInputDelta=3.870494050441078
Fitness changed from -25.615908783773065 to -29.486402834214143
Iteration 4 complete. Error: -29.486402834214143 Total: 1.8293; Orientation: 0.1374; Line Search: 1.1135
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 646302cf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.587e+04, gradient 7.475e-04, dot -0.483; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.486402834214143, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 646302cf to history. Total: 3
th(0)=-29.486402834214143;dx=-5.5333940662489E-7
New Minimum: -29.486402834214143 > -36.46212483723474
END: th(1.684339755941405E7)=-36.46212483723474; dx=-1.7692433634749032E-7 evalInputDelta=6.975722003020596
Fitness changed from -29.486402834214143 to -36.46212483723474
Iteration 5 complete. Error: -36.46212483723474 Total: 1.9355; Orientation: 0.1605; Line Search: 1.1615
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 548f63b5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.347e+04, gradient 6.955e-04, dot -0.366; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.46212483723474, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 548f63b5 to history. Total: 3
th(0)=-36.46212483723474;dx=-4.778222037170494E-7
New Minimum: -36.46212483723474 > -44.84962492055352
END: th(3.6288E7)=-44.84962492055352; dx=-5.53287814748034E-8 evalInputDelta=8.387500083318777
Fitness changed from -36.46212483723474 to -44.84962492055352
Iteration 6 complete. Error: -44.84962492055352 Total: 1.8482; Orientation: 0.1444; Line Search: 1.0926
Adding measurement 3a0ad407 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.720e+04, gradient 5.733e-04, dot -0.229; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -44.84962492055352, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 3a0ad407 to history. Total: 3
th(0)=-44.84962492055352;dx=-3.2629369129526376E-7
New Minimum: -44.84962492055352 > -54.77444066236001
END: th(7.8180126031877E7)=-54.77444066236001; dx=-2.6019907614641652E-9 evalInputDelta=9.92481574180649
Fitness changed from -44.84962492055352 to -54.77444066236001
Iteration 7 complete. Error: -54.77444066236001 Total: 1.9962; Orientation: 0.1371; Line Search: 1.3030
Adding measurement 1c095b1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.387e+04, gradient 4.193e-04, dot -0.123; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -54.77444066236001, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 1c095b1a to history. Total: 3
th(0)=-54.77444066236001;dx=-1.7332701664524625E-7
New Minimum: -54.77444066236001 > -64.579943758822
WOLF (strong): th(1.6843397559414053E8)=-64.579943758822; dx=1.7860579758307593E-9 evalInputDelta=9.80550309646199
New Minimum: -64.579943758822 > -69.45508512989392
END: th(8.421698779707026E7)=-69.45508512989392; dx=-1.3100963232227755E-8 evalInputDelta=14.68064446753391
Fitness changed from -54.77444066236001 to -69.45508512989392
Iteration 8 complete. Error: -69.45508512989392 Total: 2.3238; Orientation: 0.1321; Line Search: 1.6496
Adding measurement 3a671ef7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.282e+04, gradient 3.115e-04, dot -0.123; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -69.45508512989392, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 3a671ef7 to history. Total: 3
th(0)=-69.45508512989392;dx=-9.631284516700915E-8
New Minimum: -69.45508512989392 > -72.43248431575489
WOLF (strong): th(1.8144000000000003E8)=-72.43248431575489; dx=2.544545424185657E-10 evalInputDelta=2.9773991858609747
New Minimum: -72.43248431575489 > -75.8053610465724
END: th(9.072000000000001E7)=-75.8053610465724; dx=-1.2364615627954137E-8 evalInputDelta=6.350275916678484
Fitness changed from -69.45508512989392 to -75.8053610465724
Iteration 9 complete. Error: -75.8053610465724 Total: 2.3715; Orientation: 0.1331; Line Search: 1.6871
Adding measurement 5f2d18c3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.253e+04, gradient 2.526e-04, dot -0.130; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -75.8053610465724, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 5f2d18c3 to history. Total: 3
th(0)=-75.8053610465724;dx=-6.25425754657966E-8
New Minimum: -75.8053610465724 > -77.39087279201492
WOLF (strong): th(1.9545031507969254E8)=-77.39087279201492; dx=4.875191429408109E-10 evalInputDelta=1.585511745442517
New Minimum: -77.39087279201492 > -79.97543305743223
END: th(9.772515753984627E7)=-79.97543305743223; dx=-9.03866080374962E-9 evalInputDelta=4.170072010859826
Fitness changed from -75.8053610465724 to -79.97543305743223
Iteration 10 complete. Error: -79.97543305743223 Total: 2.3059; Orientation: 0.1397; Line Search: 1.6123
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4d849fc7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.247e+04, gradient 2.134e-04, dot -0.136; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.97543305743223, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 4d849fc7 to history. Total: 3
th(0)=-79.97543305743223;dx=-4.463873704609949E-8
Armijo: th(2.1054246949267572E8)=-79.73356274155324; dx=6.282717591085778E-10 evalInputDelta=-0.2418703158789839
New Minimum: -79.97543305743223 > -82.24042563564272
END: th(1.0527123474633786E8)=-82.24042563564272; dx=-6.222400271514637E-9 evalInputDelta=2.264992578210496
Fitness changed from -79.97543305743223 to -82.24042563564272
Iteration 11 complete. Error: -82.24042563564272 Total: 2.5511; Orientation: 0.1310; Line Search: 1.8253
Adding measurement 2a7ce179 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.196e+04, gradient 1.862e-04, dot -0.137; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -82.24042563564272, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 2a7ce179 to history. Total: 3
th(0)=-82.24042563564272;dx=-3.398116117226846E-8
Armijo: th(2.268000000000001E8)=-81.32727787915177; dx=8.524292977742792E-10 evalInputDelta=-0.9131477564909574
New Minimum: -82.24042563564272 > -83.8066315727271
END: th(1.1340000000000004E8)=-83.8066315727271; dx=-4.4873582654533485E-9 evalInputDelta=1.5662059370843764
Fitness changed from -82.24042563564272 to -83.8066315727271
Iteration 12 complete. Error: -83.8066315727271 Total: 2.3217; Orientation: 0.1311; Line Search: 1.6523
Adding measurement 22024919 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.137e+04, gradient 1.640e-04, dot -0.138; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -83.8066315727271, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 22024919 to history. Total: 3
th(0)=-83.8066315727271;dx=-2.6465067094244046E-8
Armijo: th(2.4431289384961572E8)=-82.28877653080959; dx=6.23831582443138E-10 evalInputDelta=-1.5178550419175139
New Minimum: -83.8066315727271 > -84.58722431634436
END: th(1.2215644692480786E8)=-84.58722431634436; dx=-3.593459358625474E-9 evalInputDelta=0.7805927436172624
Fitness changed from -83.8066315727271 to -84.58722431634436
Iteration 13 complete. Error: -84.58722431634436 Total: 2.3504; Orientation: 0.1379; Line Search: 1.6762
Adding measurement 44094cd2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.110e+04, gradient 1.444e-04, dot -0.142; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.58722431634436, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 44094cd2 to history. Total: 3
th(0)=-84.58722431634436;dx=-2.0560820449350145E-8
Armijo: th(2.631780868658447E8)=-83.79714311232365; dx=7.52542371296226E-10 evalInputDelta=-0.7900812040207086
New Minimum: -84.58722431634436 > -85.52392107595892
END: th(1.3158904343292235E8)=-85.52392107595892; dx=-2.5541095983147616E-9 evalInputDelta=0.9366967596145628
Fitness changed from -84.58722431634436 to -85.52392107595892
Iteration 14 complete. Error: -85.52392107595892 Total: 2.3547; Orientation: 0.1369; Line Search: 1.6506
Adding measurement 30cf960 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.050e+04, gradient 1.316e-04, dot -0.140; [c2bf55b7-7c23-4712-9194-20521b39b47a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -85.52392107595892, -22.96025715730497, -21.247408209856694, -20.170299375263475
LBFGS Accumulation History: 3 points
Removed measurement 30cf960 to history. Total: 3
th(0)=-85.52392107595892;dx=-1.6857483043182744E-8
Armijo: th(2.835000000000002E8)=-84.64759798096145; dx=9.252121538179574E-10 evalInputDelta=-0.8763230949974741
New Minimum: -85.52392107595892 > -86.22162363235535
END: th(1.417500000000001E8)=-86.22162363235535; dx=-1.6101981575842936E-9 evalInputDelta=0.6977025563964219
Fitness changed from -85.52392107595892 to -86.22162363235535
Iteration 15 complete. Error: -86.22162363235535 Total: 2.5709; Orientation: 0.1317; Line Search: 1.8947
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -86.22162363235535 (> -Infinity) after 37.656s (< 5400.000s)
Returns
-86.22162363235535