BasicOptimizer.scala:89 executed in 165.54 seconds (1.948 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 6590887477916
Reset training subject: 6592488705815
Adding measurement 341baec3 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-4.969856457846868;dx=-4.682144902238476E-5
New Minimum: -4.969856457846868 > -4.969863343120976
WOLFE (weak): th(2.154434690031884)=-4.969863343120976; dx=-4.64290954721358E-5 evalInputDelta=6.885274108192618E-6
New Minimum: -4.969863343120976 > -4.969869834942112
WOLFE (weak): th(4.308869380063768)=-4.969869834942112; dx=-4.707160149451088E-5 evalInputDelta=1.3377095243960468E-5
New Minimum: -4.969869834942112 > -4.9698973759071805
WOLFE (weak): th(12.926608140191302)=-4.9698973759071805; dx=-4.651717537630735E-5 evalInputDelta=4.091806031247813E-5
New Minimum: -4.9698973759071805 > -4.970020521498387
WOLFE (weak): th(51.70643256076521)=-4.970020521498387; dx=-4.690314339007936E-5 evalInputDelta=1.640636515185534E-4
New Minimum: -4.970020521498387 > -4.970680262368086
WOLFE (weak): th(258.53216280382605)=-4.970680262368086; dx=-4.6647799619431964E-5 evalInputDelta=8.238045212181078E-4
New Minimum: -4.970680262368086 > -4.9748481388991905
WOLFE (weak): th(1551.1929768229563)=-4.9748481388991905; dx=-4.630425911277596E-5 evalInputDelta=0.004991681052322505
New Minimum: -4.9748481388991905 > -5.007957151638768
WOLFE (weak): th(10858.350837760694)=-5.007957151638768; dx=-4.289392380575758E-5 evalInputDelta=0.0381006937919004
New Minimum: -5.007957151638768 > -5.268030598909295
END: th(86866.80670208555)=-5.268030598909295; dx=-2.1754081851395923E-5 evalInputDelta=0.298174141062427
Fitness changed from -4.969856457846868 to -5.268030598909295
Iteration 1 complete. Error: -5.268030598909295 Total: 24.1646; Orientation: 0.0164; Line Search: 19.5506
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 48e28da7 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-5.268030598909295;dx=-4.511230381101341E-5
New Minimum: -5.268030598909295 > -5.65084861700724
END: th(187148.86177126726)=-5.65084861700724; dx=-9.054270328236794E-6 evalInputDelta=0.3828180180979448
Fitness changed from -5.268030598909295 to -5.65084861700724
Iteration 2 complete. Error: -5.65084861700724 Total: 3.7961; Orientation: 0.0177; Line Search: 2.3219
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7a828f77 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-5.65084861700724;dx=-3.429661871209967E-5
New Minimum: -5.65084861700724 > -6.408307633354075
END: th(403200.00000000006)=-6.408307633354075; dx=-3.1885284692763215E-6 evalInputDelta=0.7574590163468349
Fitness changed from -5.65084861700724 to -6.408307633354075
Iteration 3 complete. Error: -6.408307633354075 Total: 3.8644; Orientation: 0.0183; Line Search: 2.4374
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 312d5438 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.780e+03, gradient 5.831e-03, dot -0.262; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.408307633354075, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 312d5438 to history. Total: 3
th(0)=-6.408307633354075;dx=-3.2983486698785094E-5
New Minimum: -6.408307633354075 > -7.657969176474945
END: th(868668.0670208557)=-7.657969176474945; dx=-1.0187927281752513E-6 evalInputDelta=1.2496615431208706
Fitness changed from -6.408307633354075 to -7.657969176474945
Iteration 4 complete. Error: -7.657969176474945 Total: 4.1762; Orientation: 0.4769; Line Search: 2.3230
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 696f08eb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.206e+03, gradient 4.079e-03, dot -0.220; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.657969176474945, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 696f08eb to history. Total: 3
th(0)=-7.657969176474945;dx=-1.6275678241421118E-5
New Minimum: -7.657969176474945 > -8.4611487670742
END: th(1871488.617712673)=-8.4611487670742; dx=-4.993712401683727E-7 evalInputDelta=0.803179590599254
Fitness changed from -7.657969176474945 to -8.4611487670742
Iteration 5 complete. Error: -8.4611487670742 Total: 4.4348; Orientation: 0.4977; Line Search: 2.3803
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 12f0a8f2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.153e+03, gradient 3.021e-03, dot -0.166; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.4611487670742, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 12f0a8f2 to history. Total: 3
th(0)=-8.4611487670742;dx=-8.93060254176145E-6
New Minimum: -8.4611487670742 > -8.58462890681087
END: th(4032000.0000000014)=-8.58462890681087; dx=-1.9750401655517993E-7 evalInputDelta=0.12348013973667094
Fitness changed from -8.4611487670742 to -8.58462890681087
Iteration 6 complete. Error: -8.58462890681087 Total: 4.3431; Orientation: 0.4732; Line Search: 2.4567
Adding measurement 5926d10c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.611e+03, gradient 2.221e-03, dot -0.106; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.58462890681087, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 5926d10c to history. Total: 3
th(0)=-8.58462890681087;dx=-4.839438505631797E-6
Armijo: th(8686680.670208558)=-8.138680697620012; dx=-4.752057790443779E-8 evalInputDelta=-0.445948209190858
New Minimum: -8.58462890681087 > -8.627609030298132
END: th(4343340.335104279)=-8.627609030298132; dx=-4.194984501130839E-7 evalInputDelta=0.042980123487261324
Fitness changed from -8.58462890681087 to -8.627609030298132
Iteration 7 complete. Error: -8.627609030298132 Total: 5.2216; Orientation: 0.4765; Line Search: 3.6150
Adding measurement 4892f3e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.523e+03, gradient 1.884e-03, dot -0.105; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.627609030298132, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 4892f3e0 to history. Total: 3
th(0)=-8.627609030298132;dx=-3.457040725012737E-6
New Minimum: -8.627609030298132 > -9.49138886104833
END: th(9357443.088563366)=-9.49138886104833; dx=-9.726621065097728E-8 evalInputDelta=0.8637798307501985
Fitness changed from -8.627609030298132 to -9.49138886104833
Iteration 8 complete. Error: -9.49138886104833 Total: 4.2099; Orientation: 0.4922; Line Search: 2.4474
Adding measurement 5e6a9163 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.413e+03, gradient 1.518e-03, dot -0.075; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.49138886104833, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 5e6a9163 to history. Total: 3
th(0)=-9.49138886104833;dx=-2.2641133531506504E-6
Armijo: th(2.0160000000000007E7)=-8.890260909383684; dx=-2.547973576066287E-8 evalInputDelta=-0.6011279516646457
Armijo: th(1.0080000000000004E7)=-9.315402390955725; dx=-1.7924823219648545E-7 evalInputDelta=-0.17598647009260482
New Minimum: -9.49138886104833 > -9.755415396339156
END: th(3360000.0000000014)=-9.755415396339156; dx=-9.270543281274203E-7 evalInputDelta=0.2640265352908262
Fitness changed from -9.49138886104833 to -9.755415396339156
Iteration 9 complete. Error: -9.755415396339156 Total: 6.5422; Orientation: 0.47
...skipping 7840 bytes...
(1.750000000000001E7)=-11.843715963663751; dx=-1.7115596511373877E-7 evalInputDelta=0.02465087108380537
Fitness changed from -11.819065092579946 to -11.843715963663751
Iteration 18 complete. Error: -11.843715963663751 Total: 5.2299; Orientation: 0.4735; Line Search: 3.6535
Adding measurement 5fb185de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.609e+05, gradient 1.042e-03, dot -0.158; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.843715963663751, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 5fb185de to history. Total: 3
th(0)=-11.843715963663751;dx=-1.0619711627801452E-6
Armijo: th(3.770260707555799E7)=-10.868752908569975; dx=-3.502885826391702E-8 evalInputDelta=-0.9749630550937756
Armijo: th(1.8851303537778996E7)=-11.80871650369217; dx=-1.5861540864757436E-7 evalInputDelta=-0.034999459971581715
New Minimum: -11.843715963663751 > -12.05192820806055
END: th(6283767.845926332)=-12.05192820806055; dx=-4.913989893512396E-7 evalInputDelta=0.2082122443967993
Fitness changed from -11.843715963663751 to -12.05192820806055
Iteration 19 complete. Error: -12.05192820806055 Total: 6.3012; Orientation: 0.4776; Line Search: 4.7043
Adding measurement 7437a4c0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.466e+06, gradient 1.059e-03, dot -0.196; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.05192820806055, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 7437a4c0 to history. Total: 3
th(0)=-12.05192820806055;dx=-1.085482341180683E-6
New Minimum: -12.05192820806055 > -12.263054457053899
END: th(1.3537967431370616E7)=-12.263054457053899; dx=-2.407699741355547E-7 evalInputDelta=0.21112624899334875
Fitness changed from -12.05192820806055 to -12.263054457053899
Iteration 20 complete. Error: -12.263054457053899 Total: 4.0235; Orientation: 0.5564; Line Search: 2.3501
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Adding measurement 7e282457 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.572e+07, gradient 1.041e-03, dot -0.183; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.263054457053899, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 7e282457 to history. Total: 3
th(0)=-12.263054457053899;dx=-1.0515888114385413E-6
Armijo: th(2.916666666666669E7)=-11.152500207418614; dx=-6.864046403273822E-8 evalInputDelta=-1.1105542496352854
New Minimum: -12.263054457053899 > -12.268804844202291
END: th(1.4583333333333345E7)=-12.268804844202291; dx=-2.378857553491283E-7 evalInputDelta=0.005750387148392377
Fitness changed from -12.263054457053899 to -12.268804844202291
Iteration 21 complete. Error: -12.268804844202291 Total: 5.4800; Orientation: 0.4790; Line Search: 3.6037
Adding measurement 1e59563 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.350e+05, gradient 1.032e-03, dot -0.190; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.268804844202291, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 1e59563 to history. Total: 3
th(0)=-12.268804844202291;dx=-1.0457198845268997E-6
Armijo: th(3.1418839229631666E7)=-11.821620555127026; dx=-5.382068782599256E-8 evalInputDelta=-0.4471842890752651
New Minimum: -12.268804844202291 > -12.526895029307875
END: th(1.5709419614815833E7)=-12.526895029307875; dx=-2.050543321604916E-7 evalInputDelta=0.25809018510558346
Fitness changed from -12.268804844202291 to -12.526895029307875
Iteration 22 complete. Error: -12.526895029307875 Total: 5.1542; Orientation: 0.4742; Line Search: 3.5384
Adding measurement 20fe91e8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.930e+05, gradient 9.840e-04, dot -0.172; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.526895029307875, -5.65084861700724, -5.268030598909295, -4.969856457846868
LBFGS Accumulation History: 3 points
Removed measurement 20fe91e8 to history. Total: 3
th(0)=-12.526895029307875;dx=-9.422579045047984E-7
Armijo: th(3.384491857842655E7)=-11.384082535250762; dx=-5.947938976358458E-8 evalInputDelta=-1.1428124940571127
New Minimum: -12.526895029307875 > -12.582355164258109
END: th(1.6922459289213274E7)=-12.582355164258109; dx=-1.9632895638512548E-7 evalInputDelta=0.05546013495023416
Fitness changed from -12.526895029307875 to -12.582355164258109
Iteration 23 complete. Error: -12.582355164258109 Total: 5.7482; Orientation: 0.4916; Line Search: 4.1330
Adding measurement 4f670318 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.479e+04, gradient 1.017e-03, dot 0.179; [84ee2940-1014-4486-abac-996067cb2411 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-12.582355164258109;dx=-15.468436817850705
Armijo: th(3.645833333333337E7)=-10.16970830705849; dx=1.5688890739378293E-8 evalInputDelta=-2.4126468571996185
Armijo: th(1.8229166666666687E7)=-10.16970830705849; dx=3.073364877107233E-8 evalInputDelta=-2.4126468571996185
Armijo: th(6076388.8888888955)=-10.16970830705849; dx=9.503049908777888E-8 evalInputDelta=-2.4126468571996185
Armijo: th(1519097.2222222239)=-10.16957871407173; dx=3.665166032805736E-7 evalInputDelta=-2.4127764501863798
Armijo: th(303819.4444444448)=-10.168899573671158; dx=1.8799727859665227E-6 evalInputDelta=-2.413455590586951
Armijo: th(50636.57407407413)=-10.168562678590328; dx=1.1098354353346636E-5 evalInputDelta=-2.4137924856677806
Armijo: th(7233.796296296305)=-10.174483253258913; dx=7.779923206931482E-5 evalInputDelta=-2.407871910999196
Armijo: th(904.2245370370381)=-10.189845910779953; dx=6.4031189528011E-4 evalInputDelta=-2.3925092534781562
Armijo: th(100.46939300411535)=-10.229746830973182; dx=0.005397012947634418 evalInputDelta=-2.352608333284927
Armijo: th(10.046939300411534)=-10.61500542573331; dx=0.04516080970004638 evalInputDelta=-1.967349738524799
Armijo: th(0.9133581182192304)=-11.521620771371703; dx=-0.4225873379229025 evalInputDelta=-1.0607343928864061
Armijo: th(0.0761131765182692)=-12.495627383159285; dx=-4.394741661491289 evalInputDelta=-0.08672778109882451
Armijo: th(0.005854859732174553)=-12.577396763148576; dx=-6.286071201738335 evalInputDelta=-0.0049584011095333835
Armijo: th(4.1820426658389665E-4)=-12.582030674131738; dx=-6.552941964845473 evalInputDelta=-3.2449012637059127E-4
Armijo: th(2.7880284438926443E-5)=-12.582334029140675; dx=-6.462204196262078 evalInputDelta=-2.1135117433956907E-5
Armijo: th(1.7425177774329027E-6)=-12.58235392101689; dx=-6.502101524032156 evalInputDelta=-1.243241218418234E-6
Armijo: th(1.0250104573134722E-7)=-12.582355164258109; dx=-6.243719172314252 evalInputDelta=0.0
Armijo: th(5.694502540630401E-9)=-12.582355164258109; dx=-6.505732285930007 evalInputDelta=0.0
Armijo: th(2.9971066003317903E-10)=-12.582355164258109; dx=-6.511928564877985 evalInputDelta=0.0
MIN ALPHA (1.498553300165895E-11): th(0.0)=-12.582355164258109
Fitness changed from -12.582355164258109 to -12.582355164258109
Static Iteration Total: 28.0651; Orientation: 0.5122; Line Search: 26.3982
Iteration 24 failed. Error: -12.582355164258109
Previous Error: 0.0 -> -12.582355164258109
Retrying iteration 24
Reset training subject: 6751638130655
Adding measurement 120482c6 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-12.582355164258109;dx=-1.0050124209841573E-6
END: th(2.1544346900318838E-10)=-12.582355164258109; dx=-8.360958190477542E-7 evalInputDelta=0.0
Fitness changed from -12.582355164258109 to -12.582355164258109
Static Iteration Total: 4.8058; Orientation: 0.0254; Line Search: 2.3653
Iteration 25 failed. Error: -12.582355164258109
Previous Error: 0.0 -> -12.582355164258109
Optimization terminated 25
Final threshold in iteration 25: -12.582355164258109 (> -Infinity) after 165.543s (< 3600.000s)
Returns
-12.582355164258109