BasicOptimizer.scala:89 executed in 68.64 seconds (1.052 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 149334999664700
Reset training subject: 149335880066900
Adding measurement 4a9f0ecd to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-7.254326417388421;dx=-0.038130640631298375
New Minimum: -7.254326417388421 > -7.320794512045646
WOLFE (weak): th(2.154434690031884)=-7.320794512045646; dx=-0.038134987137316785 evalInputDelta=0.0664680946572247
New Minimum: -7.320794512045646 > -7.387374324342782
WOLFE (weak): th(4.308869380063768)=-7.387374324342782; dx=-0.03836172463319341 evalInputDelta=0.13304790695436086
New Minimum: -7.387374324342782 > -7.656933314134296
WOLFE (weak): th(12.926608140191302)=-7.656933314134296; dx=-0.0386638184222106 evalInputDelta=0.40260689674587447
New Minimum: -7.656933314134296 > -8.914172480286702
WOLFE (weak): th(51.70643256076521)=-8.914172480286702; dx=-0.039889913672413464 evalInputDelta=1.6598460628982803
New Minimum: -8.914172480286702 > -15.854070368243322
END: th(258.53216280382605)=-15.854070368243322; dx=-0.033977875974153554 evalInputDelta=8.5997439508549
Fitness changed from -7.254326417388421 to -15.854070368243322
Iteration 1 complete. Error: -15.854070368243322 Total: 7.8600; Orientation: 0.0151; Line Search: 5.3738
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 20aea960 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-15.854070368243322;dx=-0.049936835460854
New Minimum: -15.854070368243322 > -32.43972964699787
END: th(556.9906600335335)=-32.43972964699787; dx=-0.020842378147229595 evalInputDelta=16.585659278754548
Fitness changed from -15.854070368243322 to -32.43972964699787
Iteration 2 complete. Error: -32.43972964699787 Total: 2.5738; Orientation: 0.0146; Line Search: 1.6672
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 56ddbc88 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-32.43972964699787;dx=-0.04993828680035575
New Minimum: -32.43972964699787 > -54.018585397614544
END: th(1200.0)=-54.018585397614544; dx=-0.009469145712759428 evalInputDelta=21.578855750616675
Fitness changed from -32.43972964699787 to -54.018585397614544
Iteration 3 complete. Error: -54.018585397614544 Total: 2.5947; Orientation: 0.0144; Line Search: 1.7038
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 351442a1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.372e+03, gradient 2.935e-01, dot -0.233; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -54.018585397614544, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 351442a1 to history. Total: 3
th(0)=-54.018585397614544;dx=-0.0828349970492937
Armijo: th(2585.3216280382608)=-39.20672593316659; dx=1.8673513627204925E-4 evalInputDelta=-14.811859464447956
New Minimum: -54.018585397614544 > -63.114543146889886
END: th(1292.6608140191304)=-63.114543146889886; dx=-0.0014962423421441692 evalInputDelta=9.095957749275343
Fitness changed from -54.018585397614544 to -63.114543146889886
Iteration 4 complete. Error: -63.114543146889886 Total: 3.8354; Orientation: 0.2670; Line Search: 2.6407
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 407d8907 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.410e+03, gradient 2.956e-01, dot -0.144; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -63.114543146889886, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 407d8907 to history. Total: 3
th(0)=-63.114543146889886;dx=-0.08722621067593246
Armijo: th(2784.9533001676677)=-54.56768124024971; dx=0.005048189807567513 evalInputDelta=-8.546861906640174
New Minimum: -63.114543146889886 > -90.57221523113476
WOLF (strong): th(1392.4766500838339)=-90.57221523113476; dx=0.0015846210881441786 evalInputDelta=27.457672084244876
END: th(464.15888336127796)=-89.62783021594771; dx=-0.04230058296070583 evalInputDelta=26.513287069057824
Fitness changed from -63.114543146889886 to -90.57221523113476
Iteration 5 complete. Error: -90.57221523113476 Total: 4.6579; Orientation: 0.2678; Line Search: 3.4275
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 20e26824 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.755e+03, gradient 2.712e-01, dot -0.107; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -90.57221523113476, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 20e26824 to history. Total: 3
th(0)=-90.57221523113476;dx=-0.07335228349573683
New Minimum: -90.57221523113476 > -126.34368043530115
END: th(1000.0000000000002)=-126.34368043530115; dx=-0.021890370130713675 evalInputDelta=35.77146520416639
Fitness changed from -90.57221523113476 to -126.34368043530115
Iteration 6 complete. Error: -126.34368043530115 Total: 3.1671; Orientation: 0.2914; Line Search: 1.9752
Adding measurement 58687405 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.777e+03, gradient 2.061e-01, dot -0.110; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -126.34368043530115, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 58687405 to history. Total: 3
th(0)=-126.34368043530115;dx=-0.04244262352655819
New Minimum: -126.34368043530115 > -145.62372585279502
WOLF (strong): th(2154.434690031884)=-145.62372585279502; dx=1.9780114888334126E-4 evalInputDelta=19.280045417493866
New Minimum: -145.62372585279502 > -149.82536057648917
END: th(1077.217345015942)=-149.82536057648917; dx=-0.016099247968041797 evalInputDelta=23.481680141188022
Fitness changed from -126.34368043530115 to -149.82536057648917
Iteration 7 complete. Error: -149.82536057648917 Total: 3.6855; Orientation: 0.2650; Line Search: 2.5799
Adding measurement 5529050 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.747e+03, gradient 2.049e-01, dot -0.149; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -149.82536057648917, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 5529050 to history. Total: 3
th(0)=-149.82536057648917;dx=-0.041597246900303614
New Minimum: -149.82536057648917 > -167.84359581541977
END: th(2320.79441680639)=-167.84359581541977; dx=-0.00259999379989126 evalInputDelta=18.0182352389306
Fitness changed from -149.82536057648917 to -167.84359581541977
Iteration 8 complete. Error: -167.84359581541977 Total: 2.8119; Orientation: 0.2883; Line Search: 1.7363
Adding measurement 1de45f49 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.737e+03, gradient 2.528e-01, dot -0.112; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -167.84359581541977, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 1de45f49 to history. Total: 3
th(0)=-167.84359581541977;dx=-0.0638524220924801
Armijo: th(5000.000000000002)=-129.49017367400896; dx=0.006445930613997939 evalInputDelta=-38.35342214141082
New Minimum: -167.84359581541977 > -183.30382693427325
WOLF (strong): th(2500.000000000001)=-183.30382693427325; dx=0.005296465914049571 evalInputDelta=15.460231118853471
New Minimum: -183.30382693427325 > -199.26742787645952
END: th(833.3333333333336)=-199.26742787645952; dx=-0.025311339558466203 evalInputDelta=31.423832061039747
Fitness changed from -167.84359581541977 to -199.26742787645952
Iteration 9 complete. Error: -199.26742787645952 Total: 4.6579; Orientation: 0.2755; Line Search: 3.5447
Adding measurement 582cae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.398e+04, gradient 1.902e-01, dot -0.185; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -199.26742787645952, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 582cae to history. Total: 3
th(0)=-199.26742787645952;dx=-0.036114972162677866
New Minimum: -199.26742787645952 > -222.59245796690865
END: th(1795.3622416932371)=-222.59245796690865; dx=-0.0068266860985583926 evalInputDelta=23.325030090449133
Fitness changed from -199.26742787645952 to -222.59245796690865
Iteration 10 complete. Error: -222.59245796690865 Total: 2.7839; Orientation: 0.2656; Line Search: 1.7098
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 72c060ee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.449e+04, gradient 2.452e-01, dot -0.176; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -222.59245796690865, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 72c060ee to history. Total: 3
th(0)=-222.59245796690865;dx=-0.06002316276270586
Armijo: th(3867.9906946773176)=-200.60060209904654; dx=0.00803938351802488 evalInputDelta=-21.991855867862114
New Minimum: -222.59245796690865 > -242.5663971601384
WOLF (strong): th(1933.9953473386588)=-242.5663971601384; dx=0.004881118547556837 evalInputDelta=19.97393919322974
New Minimum: -242.5663971601384 > -245.31380572900906
END: th(644.6651157795529)=-245.31380572900906; dx=-0.018971304153217836 evalInputDelta=22.721347762100407
Fitness changed from -222.59245796690865 to -245.31380572900906
Iteration 11 complete. Error: -245.31380572900906 Total: 4.6656; Orientation: 0.2659; Line Search: 3.4064
Adding measurement 67f4476e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.032e+05, gradient 1.657e-01, dot -0.254; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -245.31380572900906, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 67f4476e to history. Total: 3
th(0)=-245.31380572900906;dx=-0.027404638712852605
New Minimum: -245.31380572900906 > -266.06175857308017
END: th(1388.8888888888896)=-266.06175857308017; dx=-0.01388590138476984 evalInputDelta=20.747952844071108
Fitness changed from -245.31380572900906 to -266.06175857308017
Iteration 12 complete. Error: -266.06175857308017 Total: 2.9327; Orientation: 0.3865; Line Search: 1.7546
Adding measurement 5c1e5663 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.967e+03, gradient 1.801e-01, dot 0.338; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-266.06175857308017;dx=-487.84803924958254
Armijo: th(2992.2704028220624)=-93.32999668514941; dx=0.031619213805821646 evalInputDelta=-172.73176188793076
Armijo: th(1496.1352014110312)=-97.68029703930829; dx=0.06012954337759451 evalInputDelta=-168.38146153377187
Armijo: th(498.7117338036771)=-125.30474960446432; dx=0.10454092034415499 evalInputDelta=-140.75700896861585
Armijo: th(124.67793345091927)=-161.80686307292407; dx=0.05664517312590163 evalInputDelta=-104.2548955001561
Armijo: th(24.935586690183854)=-138.40290199721247; dx=0.2783687096540598 evalInputDelta=-127.6588565758677
Armijo: th(4.155931115030643)=-148.06743343406072; dx=0.9052247098382744 evalInputDelta=-117.99432513901945
Armijo: th(0.5937044450043775)=-187.4812803155291; dx=-25.50351526611948 evalInputDelta=-78.58047825755108
New Minimum: -266.06175857308017 > -276.482015124854
WOLF (strong): th(0.07421305562554718)=-276.482015124854; dx=35.73371934164923 evalInputDelta=10.420256551773832
END: th(0.008245895069505243)=-270.38703452181284; dx=-384.8071709406636 evalInputDelta=4.325275948732667
Fitness changed from -266.06175857308017 to -276.482015124854
Iteration 13 complete. Error: -276.482015124854 Total: 9.7690; Orientation: 0.2618; Line Search: 8.6938
Adding measurement dc54710 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.294e+04, gradient 2.470e-01, dot -0.111; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -276.482015124854, -266.06175857308017, -32.43972964699787, -15.854070368243322, -7.254326417388421
Rejected: LBFGS Orientation magnitude: 1.428e+05, gradient 2.470e-01, dot -0.217; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -276.482015124854, -266.06175857308017, -32.43972964699787, -15.854070368243322
LBFGS Accumulation History: 3 points
Removed measurement dc54710 to history. Total: 4
Removed measurement 5c1e5663 to history. Total: 3
th(0)=-276.482015124854;dx=-0.06113309850271049
New Minimum: -276.482015124854 > -276.48304649652573
WOLFE (weak): th(0.01776524238810497)=-276.48304649652573; dx=-0.05969824153414506 evalInputDelta=0.0010313716717291754
New Minimum: -276.48304649652573 > -276.4840919926497
WOLFE (weak): th(0.03553048477620994)=-276.4840919926497; dx=-0.06050462033096818 evalInputDelta=0.002076867795722137
New Minimum: -276.4840919926497 > -276.4883728341935
WOLFE (weak): th(0.10659145432862982)=-276.4883728341935; dx=-0.060492996059061936 evalInputDelta=0.006357709339511075
New Minimum: -276.4883728341935 > -276.50750159353726
WOLFE (weak): th(0.4263658173145193)=-276.50750159353726; dx=-0.06049500156026889 evalInputDelta=0.02548646868325477
New Minimum: -276.50750159353726 > -276.60959344805815
WOLFE (weak): th(2.1318290865725964)=-276.60959344805815; dx=-0.060266832690059016 evalInputDelta=0.12757832320414764
New Minimum: -276.60959344805815 > -277.2417012959991
WOLFE (weak): th(12.79097451943558)=-277.2417012959991; dx=-0.05951162211695063 evalInputDelta=0.7596861711451197
New Minimum: -277.2417012959991 > -281.58918436261007
WOLFE (weak): th(89.53682163604905)=-281.58918436261007; dx=-0.055611177702382594 evalInputDelta=5.107169237756068
New Minimum: -281.58918436261007 > -305.65259333514575
END: th(716.2945730883924)=-305.65259333514575; dx=-0.028247598409715344 evalInputDelta=29.17057821029175
Fitness changed from -276.482015124854 to -305.65259333514575
Iteration 14 complete. Error: -305.65259333514575 Total: 9.6930; Orientation: 0.5785; Line Search: 8.3415
Adding measurement 1c5dcf58 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.095e+04, gradient 1.800e-01, dot -0.194; [161f636c-729e-4666-8dc8-335864703dc5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -305.65259333514575, -32.43972964699787, -15.854070368243322, -7.254326417388421
LBFGS Accumulation History: 3 points
Removed measurement 1c5dcf58 to history. Total: 3
th(0)=-305.65259333514575;dx=-0.032319469861944956
New Minimum: -305.65259333514575 > -335.51288863097346
END: th(1543.2098765432113)=-335.51288863097346; dx=-0.018304804513136746 evalInputDelta=29.86029529582771
Fitness changed from -305.65259333514575 to -335.51288863097346
Iteration 15 complete. Error: -335.51288863097346 Total: 2.8503; Orientation: 0.2584; Line Search: 1.7134
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -335.51288863097346 (> -Infinity) after 68.641s (< 5400.000s)
Returns
-335.51288863097346