BasicOptimizer.scala:89 executed in 58.88 seconds (1.357 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 208456768849000
Reset training subject: 208457515879600
Adding measurement 7b71c9b1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=8.609679251383747;dx=-7.010066337253364E-4
New Minimum: 8.609679251383747 > 8.608542361129162
WOLFE (weak): th(2.154434690031884)=8.608542361129162; dx=-7.034207692315647E-4 evalInputDelta=0.0011368902545854098
New Minimum: 8.608542361129162 > 8.607405981771405
WOLFE (weak): th(4.308869380063768)=8.607405981771405; dx=-7.03243430695564E-4 evalInputDelta=0.002273269612341977
New Minimum: 8.607405981771405 > 8.602856706245811
WOLFE (weak): th(12.926608140191302)=8.602856706245811; dx=-7.054453800304566E-4 evalInputDelta=0.006822545137936231
New Minimum: 8.602856706245811 > 8.582345314597998
WOLFE (weak): th(51.70643256076521)=8.582345314597998; dx=-7.05986340916557E-4 evalInputDelta=0.027333936785749913
New Minimum: 8.582345314597998 > 8.472071568582873
WOLFE (weak): th(258.53216280382605)=8.472071568582873; dx=-7.132056027947707E-4 evalInputDelta=0.13760768280087454
New Minimum: 8.472071568582873 > 7.759355927136565
WOLFE (weak): th(1551.1929768229563)=7.759355927136565; dx=-7.424196446275651E-4 evalInputDelta=0.8503233242471824
New Minimum: 7.759355927136565 > 2.771537051946222
END: th(10858.350837760694)=2.771537051946222; dx=-6.183444979477108E-4 evalInputDelta=5.8381421994375255
Fitness changed from 8.609679251383747 to 2.771537051946222
Iteration 1 complete. Error: 2.771537051946222 Total: 8.4233; Orientation: 0.0160; Line Search: 6.2475
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement cec6c1 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=2.771537051946222;dx=-9.659135156369151E-4
New Minimum: 2.771537051946222 > -9.364033266438803
END: th(23393.607721408407)=-9.364033266438803; dx=-4.590197925325406E-4 evalInputDelta=12.135570318385025
Fitness changed from 2.771537051946222 to -9.364033266438803
Iteration 2 complete. Error: -9.364033266438803 Total: 2.3788; Orientation: 0.0164; Line Search: 1.6100
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5dd3c659 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-9.364033266438803;dx=-9.524229362842887E-4
New Minimum: -9.364033266438803 > -25.10008701025197
END: th(50400.00000000001)=-25.10008701025197; dx=-2.556041078841866E-4 evalInputDelta=15.736053743813166
Fitness changed from -9.364033266438803 to -25.10008701025197
Iteration 3 complete. Error: -25.10008701025197 Total: 2.2629; Orientation: 0.0174; Line Search: 1.5379
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3edd1573 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.498e+04, gradient 3.491e-02, dot -0.174; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.10008701025197, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 3edd1573 to history. Total: 3
th(0)=-25.10008701025197;dx=-0.0012172670780649071
New Minimum: -25.10008701025197 > -33.9292631422555
END: th(108583.50837760697)=-33.9292631422555; dx=-6.182329799531169E-5 evalInputDelta=8.829176132003532
Fitness changed from -25.10008701025197 to -33.9292631422555
Iteration 4 complete. Error: -33.9292631422555 Total: 2.8210; Orientation: 0.3211; Line Search: 1.5005
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 68c1eaf1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.361e+04, gradient 3.454e-02, dot -0.085; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.9292631422555, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 68c1eaf1 to history. Total: 3
th(0)=-33.9292631422555;dx=-0.001190491287687551
New Minimum: -33.9292631422555 > -41.25705659922195
WOLF (strong): th(233936.07721408413)=-41.25705659922195; dx=1.3936283795637833E-5 evalInputDelta=7.327793456966447
New Minimum: -41.25705659922195 > -52.997359331344114
END: th(116968.03860704206)=-52.997359331344114; dx=-7.498247044189744E-5 evalInputDelta=19.068096189088614
Fitness changed from -33.9292631422555 to -52.997359331344114
Iteration 5 complete. Error: -52.997359331344114 Total: 3.3296; Orientation: 0.2998; Line Search: 2.2396
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1f7539af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.139e+04, gradient 3.189e-02, dot -0.065; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -52.997359331344114, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 1f7539af to history. Total: 3
th(0)=-52.997359331344114;dx=-0.0010148456285796665
New Minimum: -52.997359331344114 > -61.67357608543915
WOLF (strong): th(252000.0000000001)=-61.67357608543915; dx=2.406137450800771E-5 evalInputDelta=8.676216754095037
New Minimum: -61.67357608543915 > -78.16645202531089
END: th(126000.00000000004)=-78.16645202531089; dx=-1.1699459378914841E-4 evalInputDelta=25.169092693966775
Fitness changed from -52.997359331344114 to -78.16645202531089
Iteration 6 complete. Error: -78.16645202531089 Total: 3.6770; Orientation: 0.3207; Line Search: 2.5808
Adding measurement 3c22ebb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.615e+04, gradient 3.135e-02, dot -0.072; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -78.16645202531089, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 3c22ebb to history. Total: 3
th(0)=-78.16645202531089;dx=-9.796767959820265E-4
New Minimum: -78.16645202531089 > -80.12423489746352
WOLF (strong): th(271458.77094401745)=-80.12423489746352; dx=2.3954441417804708E-5 evalInputDelta=1.9577828721526345
New Minimum: -80.12423489746352 > -101.06180275870918
END: th(135729.38547200873)=-101.06180275870918; dx=-1.1256205422437118E-4 evalInputDelta=22.89535073339829
Fitness changed from -78.16645202531089 to -101.06180275870918
Iteration 7 complete. Error: -101.06180275870918 Total: 3.2174; Orientation: 0.2859; Line Search: 2.2190
Adding measurement 144d3914 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.294e+04, gradient 3.170e-02, dot -0.077; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -101.06180275870918, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 144d3914 to history. Total: 3
th(0)=-101.06180275870918;dx=-0.0010022601824077144
Armijo: th(292420.0965176052)=-93.1254377349389; dx=4.197004334941655E-5 evalInputDelta=-7.9363650237702785
New Minimum: -101.06180275870918 > -120.99590510698857
END: th(146210.0482588026)=-120.99590510698857; dx=-9.369304375413676E-5 evalInputDelta=19.934102348279396
Fitness changed from -101.06180275870918 to -120.99590510698857
Iteration 8 complete. Error: -120.99590510698857 Total: 3.4713; Orientation: 0.2992; Line Search: 2.4156
Adding measurement b5ba9b9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.484e+04, gradient 3.207e-02, dot -0.085; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -120.99590510698857, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement b5ba9b9 to history. Total: 3
th(0)=-120.99590510698857;dx=-0.0010276261894238802
Armijo: th(315000.0000000001)=-104.97841561877175; dx=5.275870060695392E-5 evalInputDelta=-16.017489488216825
New Minimum: -120.99590510698857 > -139.4400136643701
END: th(157500.00000000006)=-139.4400136643701; dx=-8.074785493210217E-5 evalInputDelta=18.444108557381526
Fitness changed from -120.99590510698857 to -139.4400136643701
Iteration 9 complete. Error: -139.4400136643701 Total: 3.2378; Orientation: 0.3064; Line Search: 2.1956
Adding measurement 40aba69e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.492e+04, gradient 3.199e-02, dot -0.089; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -139.4400136643701, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 40aba69e to history. Total: 3
th(0)=-139.4400136643701;dx=-0.001022038358967403
Armijo: th(339323.4636800218)=-116.44408503632391; dx=6.87325073032641E-5 evalInputDelta=-22.995928628046187
New Minimum: -139.4400136643701 > -156.91972805060377
END: th(169661.7318400109)=-156.91972805060377; dx=-7.676025479805398E-5 evalInputDelta=17.479714386233667
Fitness changed from -139.4400136643701 to -156.91972805060377
Iteration 10 complete. Error: -156.91972805060377 Total: 3.2392; Orientation: 0.2967; Line Search: 2.2620
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6081c5cf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.054e+05, gradient 3.137e-02, dot -0.096; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -156.91972805060377, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 6081c5cf to history. Total: 3
th(0)=-156.91972805060377;dx=-9.827002363484005E-4
Armijo: th(365525.1206470065)=-130.05286820454583; dx=6.838686766162185E-5 evalInputDelta=-26.866859846057935
New Minimum: -156.91972805060377 > -174.87392755153212
END: th(182762.56032350325)=-174.87392755153212; dx=-8.396074758970569E-5 evalInputDelta=17.954199500928354
Fitness changed from -156.91972805060377 to -174.87392755153212
Iteration 11 complete. Error: -174.87392755153212 Total: 3.4502; Orientation: 0.2919; Line Search: 2.1978
Adding measurement 17a3e7e8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.391e+05, gradient 3.103e-02, dot -0.108; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -174.87392755153212, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 17a3e7e8 to history. Total: 3
th(0)=-174.87392755153212;dx=-9.615029394529539E-4
Armijo: th(393750.0000000002)=-137.68130488046253; dx=7.809942388373353E-5 evalInputDelta=-37.192622671069586
New Minimum: -174.87392755153212 > -189.2714183249998
END: th(196875.0000000001)=-189.2714183249998; dx=-7.22350986746086E-5 evalInputDelta=14.397490773467666
Fitness changed from -174.87392755153212 to -189.2714183249998
Iteration 12 complete. Error: -189.2714183249998 Total: 3.2877; Orientation: 0.3317; Line Search: 2.2926
Adding measurement 62c28d8b to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.246e+05, gradient 3.122e-02, dot 0.123; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-189.2714183249998;dx=-477.621254417649
Armijo: th(424154.3296000273)=259.0695160107748; dx=5.609129545444273E-4 evalInputDelta=-448.3409343357746
Armijo: th(212077.16480001365)=259.0666283479434; dx=0.0011237643151595059 evalInputDelta=-448.33804667294316
Armijo: th(70692.38826667122)=259.0632774740311; dx=0.003363997864579767 evalInputDelta=-448.3346957990309
Armijo: th(17673.097066667804)=259.0606329818172; dx=0.013476253271400397 evalInputDelta=-448.332051306817
Armijo: th(3534.6194133335607)=259.06583489738966; dx=0.0674514305899876 evalInputDelta=-448.33725322238945
Armijo: th(589.1032355555934)=258.9251718923339; dx=0.4046079425111051 evalInputDelta=-448.19659021733366
Armijo: th(84.15760507937048)=257.3157974151013; dx=2.846236142376175 evalInputDelta=-446.5872157401011
Armijo: th(10.51970063492131)=243.34780779870533; dx=23.62450904228872 evalInputDelta=-432.6192261237051
Armijo: th(1.168855626102368)=88.1331866660764; dx=168.89583386290684 evalInputDelta=-277.4046049910762
Armijo: th(0.11688556261023679)=-162.27565585923892; dx=244.9981659582459 evalInputDelta=-26.995762465760862
New Minimum: -189.2714183249998 > -191.2537125737147
END: th(0.010625960237294254)=-191.2537125737147; dx=-110.56546333624318 evalInputDelta=1.9822942487149078
Fitness changed from -189.2714183249998 to -191.2537125737147
Iteration 13 complete. Error: -191.2537125737147 Total: 10.5704; Orientation: 0.2904; Line Search: 9.6054
Adding measurement 710915d4 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.708e+06, gradient 3.114e-02, dot -0.095; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -191.2537125737147, -189.2714183249998, -9.364033266438803, 2.771537051946222, 8.609679251383747
Rejected: LBFGS Orientation magnitude: 3.009e+06, gradient 3.114e-02, dot -0.158; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -191.2537125737147, -189.2714183249998, -9.364033266438803, 2.771537051946222
LBFGS Accumulation History: 3 points
Removed measurement 710915d4 to history. Total: 4
Removed measurement 62c28d8b to history. Total: 3
th(0)=-191.2537125737147;dx=-9.677107907403047E-4
New Minimum: -191.2537125737147 > -191.25372423881706
END: th(0.02289293735012617)=-191.25372423881706; dx=-8.338925428174446E-4 evalInputDelta=1.1665102363167534E-5
Fitness changed from -191.2537125737147 to -191.25372423881706
Iteration 14 complete. Error: -191.25372423881706 Total: 2.8149; Orientation: 0.6491; Line Search: 1.5036
Adding measurement 7b36f798 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.091e+05, gradient 3.115e-02, dot -0.099; [710d9da8-62d7-4843-8999-f4e4b0710df0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -191.25372423881706, -9.364033266438803, 2.771537051946222, 8.609679251383747
LBFGS Accumulation History: 3 points
Removed measurement 7b36f798 to history. Total: 3
th(0)=-191.25372423881706;dx=-9.69166412968918E-4
New Minimum: -191.25372423881706 > -191.25374756902
END: th(0.04932133838383841)=-191.25374756902; dx=-8.341885493727522E-4 evalInputDelta=2.3330202935767375E-5
Fitness changed from -191.25372423881706 to -191.25374756902
Iteration 15 complete. Error: -191.25374756902 Total: 2.6793; Orientation: 0.2941; Line Search: 1.7012
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -191.25374756902 (> -Infinity) after 58.884s (< 5400.000s)
Returns
-191.25374756902