BasicOptimizer.scala:89 executed in 64.71 seconds (0.900 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140203900334300
Reset training subject: 140204809195400
Adding measurement 3bc11667 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.648166606283005;dx=-3.794259167173986E-6
New Minimum: -3.648166606283005 > -3.648174611556515
WOLFE (weak): th(2.154434690031884)=-3.648174611556515; dx=-3.680630093652605E-6 evalInputDelta=8.005273509859023E-6
New Minimum: -3.648174611556515 > -3.6481822966025588
WOLFE (weak): th(4.308869380063768)=-3.6481822966025588; dx=-3.698809580017646E-6 evalInputDelta=1.569031955384048E-5
New Minimum: -3.6481822966025588 > -3.648213837142799
WOLFE (weak): th(12.926608140191302)=-3.648213837142799; dx=-3.691395713938286E-6 evalInputDelta=4.723085979385644E-5
New Minimum: -3.648213837142799 > -3.6483548856640584
WOLFE (weak): th(51.70643256076521)=-3.6483548856640584; dx=-3.698247108315771E-6 evalInputDelta=1.8827938105347997E-4
New Minimum: -3.6483548856640584 > -3.6491069456574365
WOLFE (weak): th(258.53216280382605)=-3.6491069456574365; dx=-3.6968781247009977E-6 evalInputDelta=9.403393744316091E-4
New Minimum: -3.6491069456574365 > -3.6538117265143457
WOLFE (weak): th(1551.1929768229563)=-3.6538117265143457; dx=-3.686774311973529E-6 evalInputDelta=0.005645120231340783
New Minimum: -3.6538117265143457 > -3.6877539229969467
WOLFE (weak): th(10858.350837760694)=-3.6877539229969467; dx=-3.7110912564708813E-6 evalInputDelta=0.03958731671394178
New Minimum: -3.6877539229969467 > -3.968364325992494
WOLFE (weak): th(86866.80670208555)=-3.968364325992494; dx=-3.7585783113703407E-6 evalInputDelta=0.3201977197094892
New Minimum: -3.968364325992494 > -6.332629486185812
END: th(781801.26031877)=-6.332629486185812; dx=-3.022677324702072E-6 evalInputDelta=2.6844628799028074
Fitness changed from -3.648166606283005 to -6.332629486185812
Iteration 1 complete. Error: -6.332629486185812 Total: 11.4949; Orientation: 0.0166; Line Search: 8.9857
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 32067db7 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-6.332629486185812;dx=-5.3140065300834455E-6
New Minimum: -6.332629486185812 > -10.886919200799976
END: th(1684339.7559414052)=-10.886919200799976; dx=-1.288726724948749E-6 evalInputDelta=4.554289714614164
Fitness changed from -6.332629486185812 to -10.886919200799976
Iteration 2 complete. Error: -10.886919200799976 Total: 2.6642; Orientation: 0.0146; Line Search: 1.7440
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 39392199 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-10.886919200799976;dx=-8.194050240763975E-6
New Minimum: -10.886919200799976 > -13.016666089854448
WOLF (strong): th(3628800.0)=-13.016666089854448; dx=7.263952311312086E-9 evalInputDelta=2.1297468890544717
New Minimum: -13.016666089854448 > -13.912537223732391
END: th(1814400.0)=-13.912537223732391; dx=-8.192567292720529E-7 evalInputDelta=3.0256180229324148
Fitness changed from -10.886919200799976 to -13.912537223732391
Iteration 3 complete. Error: -13.912537223732391 Total: 3.6137; Orientation: 0.0158; Line Search: 2.6823
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7bce4b53 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.597e+04, gradient 2.965e-03, dot -0.202; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.912537223732391, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 7bce4b53 to history. Total: 3
th(0)=-13.912537223732391;dx=-8.777308098918869E-6
New Minimum: -13.912537223732391 > -15.623918314976956
WOLF (strong): th(3909006.30159385)=-15.623918314976956; dx=1.2356391663130358E-7 evalInputDelta=1.711381091244565
New Minimum: -15.623918314976956 > -17.800354543147915
END: th(1954503.150796925)=-17.800354543147915; dx=-1.0003166923584042E-7 evalInputDelta=3.8878173194155234
Fitness changed from -13.912537223732391 to -17.800354543147915
Iteration 4 complete. Error: -17.800354543147915 Total: 4.2381; Orientation: 0.3101; Line Search: 2.7375
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 550d655f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.325e+04, gradient 3.000e-03, dot -0.127; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.800354543147915, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 550d655f to history. Total: 3
th(0)=-17.800354543147915;dx=-8.984439104086881E-6
Armijo: th(4210849.389853513)=-17.385203255328; dx=5.095038123605203E-7 evalInputDelta=-0.4151512878199135
New Minimum: -17.800354543147915 > -21.03734643557738
WOLF (strong): th(2105424.6949267564)=-21.03734643557738; dx=5.701005130016143E-7 evalInputDelta=3.236991892429465
New Minimum: -21.03734643557738 > -21.94960410463613
END: th(701808.2316422522)=-21.94960410463613; dx=-3.221062786922744E-6 evalInputDelta=4.149249561488215
Fitness changed from -17.800354543147915 to -21.94960410463613
Iteration 5 complete. Error: -21.94960410463613 Total: 4.7483; Orientation: 0.3191; Line Search: 3.4163
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 525d3470 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.468e+04, gradient 2.273e-03, dot -0.193; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.94960410463613, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 525d3470 to history. Total: 3
th(0)=-21.94960410463613;dx=-5.1571760614921285E-6
New Minimum: -21.94960410463613 > -25.217724978689525
END: th(1512000.0)=-25.217724978689525; dx=-1.6057422428757656E-6 evalInputDelta=3.2681208740533947
Fitness changed from -21.94960410463613 to -25.217724978689525
Iteration 6 complete. Error: -25.217724978689525 Total: 3.0984; Orientation: 0.4069; Line Search: 1.7964
Adding measurement 76dee0bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.640e+04, gradient 2.410e-03, dot -0.183; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.217724978689525, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 76dee0bb to history. Total: 3
th(0)=-25.217724978689525;dx=-5.7723442555884515E-6
New Minimum: -25.217724978689525 > -25.771241863645717
WOLF (strong): th(3257505.2513282085)=-25.771241863645717; dx=4.202642817768056E-8 evalInputDelta=0.553516884956192
New Minimum: -25.771241863645717 > -27.481533317712202
END: th(1628752.6256641042)=-27.481533317712202; dx=-5.11346405788198E-7 evalInputDelta=2.2638083390226775
Fitness changed from -25.217724978689525 to -27.481533317712202
Iteration 7 complete. Error: -27.481533317712202 Total: 3.6823; Orientation: 0.2832; Line Search: 2.6153
Adding measurement 3809c072 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.672e+04, gradient 2.672e-03, dot -0.169; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.481533317712202, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 3809c072 to history. Total: 3
th(0)=-27.481533317712202;dx=-7.094465940135103E-6
Armijo: th(3509041.1582112615)=-26.033425514616688; dx=4.728988623581063E-7 evalInputDelta=-1.448107803095514
New Minimum: -27.481533317712202 > -29.244827667372366
WOLF (strong): th(1754520.5791056307)=-29.244827667372366; dx=5.092667940746133E-7 evalInputDelta=1.763294349660164
New Minimum: -29.244827667372366 > -30.145194043681904
END: th(584840.1930352102)=-30.145194043681904; dx=-2.727434955132262E-6 evalInputDelta=2.6636607259697023
Fitness changed from -27.481533317712202 to -30.145194043681904
Iteration 8 complete. Error: -30.145194043681904 Total: 4.6616; Orientation: 0.2798; Line Search: 3.5558
Adding measurement 680452dc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.104e+05, gradient 2.015e-03, dot -0.230; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.145194043681904, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 680452dc to history. Total: 3
th(0)=-30.145194043681904;dx=-4.044666924708993E-6
New Minimum: -30.145194043681904 > -32.3864373886634
END: th(1260000.0000000002)=-32.3864373886634; dx=-1.852302661806797E-6 evalInputDelta=2.2412433449814984
Fitness changed from -30.145194043681904 to -32.3864373886634
Iteration 9 complete. Error: -32.3864373886634 Total: 3.0146; Orientation: 0.2676; Line Search: 1.7040
Adding measurement 69cdfc7d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.507e+05, gradient 2.096e-03, dot -0.242; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -32.3864373886634, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 69cdfc7d to history. Total: 3
th(0)=-32.3864373886634;dx=-4.385852542609179E-6
New Minimum: -32.3864373886634 > -34.227605794619116
END: th(2714587.709440174)=-34.227605794619116; dx=-7.372389369821914E-7 evalInputDelta=1.8411684059557132
Fitness changed from -32.3864373886634 to -34.227605794619116
Iteration 10 complete. Error: -34.227605794619116 Total: 2.8932; Orientation: 0.2940; Line Search: 1.7391
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 3491a2f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.677e+05, gradient 2.747e-03, dot -0.175; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.227605794619116, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 3491a2f0 to history. Total: 3
th(0)=-34.227605794619116;dx=-7.523169289604625E-6
Armijo: th(5848401.930352103)=-26.53907880882893; dx=7.311267993708142E-7 evalInputDelta=-7.688526985790187
Armijo: th(2924200.9651760515)=-32.959646267769365; dx=9.263020665776896E-7 evalInputDelta=-1.2679595268497508
New Minimum: -34.227605794619116 > -37.28408430760159
END: th(974733.6550586838)=-37.28408430760159; dx=-1.0081755857938899E-6 evalInputDelta=3.0564785129824728
Fitness changed from -34.227605794619116 to -37.28408430760159
Iteration 11 complete. Error: -37.28408430760159 Total: 4.7543; Orientation: 0.2821; Line Search: 3.5651
Adding measurement 23d4c515 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.588e+05, gradient 2.275e-03, dot -0.226; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.28408430760159, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 23d4c515 to history. Total: 3
th(0)=-37.28408430760159;dx=-5.165433220896282E-6
New Minimum: -37.28408430760159 > -37.88699579143616
WOLF (strong): th(2100000.0000000005)=-37.88699579143616; dx=9.69098845094769E-8 evalInputDelta=0.6029114838345748
New Minimum: -37.88699579143616 > -39.08191802488397
END: th(1050000.0000000002)=-39.08191802488397; dx=-9.76544869380624E-7 evalInputDelta=1.79783371728238
Fitness changed from -37.28408430760159 to -39.08191802488397
Iteration 12 complete. Error: -39.08191802488397 Total: 3.6803; Orientation: 0.2699; Line Search: 2.5959
Adding measurement 418dfbd6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.618e+05, gradient 2.254e-03, dot -0.232; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.08191802488397, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 418dfbd6 to history. Total: 3
th(0)=-39.08191802488397;dx=-5.070133792153206E-6
New Minimum: -39.08191802488397 > -39.11398017680783
WOLF (strong): th(2262156.4245334785)=-39.11398017680783; dx=1.4626486346006213E-7 evalInputDelta=0.03206215192385997
New Minimum: -39.11398017680783 > -40.487546772646816
END: th(1131078.2122667392)=-40.487546772646816; dx=-5.155246328135254E-7 evalInputDelta=1.4056287477628473
Fitness changed from -39.08191802488397 to -40.487546772646816
Iteration 13 complete. Error: -40.487546772646816 Total: 3.6542; Orientation: 0.2811; Line Search: 2.5261
Adding measurement 1e7ec304 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.182e+05, gradient 2.365e-03, dot -0.228; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.487546772646816, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 1e7ec304 to history. Total: 3
th(0)=-40.487546772646816;dx=-5.570432089268259E-6
Armijo: th(2436834.1376467096)=-39.68613105620208; dx=4.4144669895220496E-7 evalInputDelta=-0.8014157164447369
New Minimum: -40.487546772646816 > -41.74879627377385
END: th(1218417.0688233548)=-41.74879627377385; dx=-5.036345408154229E-8 evalInputDelta=1.2612495011270326
Fitness changed from -40.487546772646816 to -41.74879627377385
Iteration 14 complete. Error: -41.74879627377385 Total: 3.9068; Orientation: 0.2858; Line Search: 2.7915
Adding measurement 2fec071 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.979e+05, gradient 2.459e-03, dot -0.219; [b724c7a6-d0cf-4b2a-aad8-e5d982db07a1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.74879627377385, -10.886919200799976, -6.332629486185812, -3.648166606283005
LBFGS Accumulation History: 3 points
Removed measurement 2fec071 to history. Total: 3
th(0)=-41.74879627377385;dx=-6.030239562300423E-6
Armijo: th(2625000.000000001)=-39.81445485920488; dx=8.397325686170281E-7 evalInputDelta=-1.9343414145689692
New Minimum: -41.74879627377385 > -42.91616620468274
WOLF (strong): th(1312500.0000000005)=-42.91616620468274; dx=4.959862350000836E-7 evalInputDelta=1.1673699309088903
New Minimum: -42.91616620468274 > -43.30606201427904
END: th(437500.0000000002)=-43.30606201427904; dx=-2.4072400138335344E-6 evalInputDelta=1.5572657405051942
Fitness changed from -41.74879627377385 to -43.30606201427904
Iteration 15 complete. Error: -43.30606201427904 Total: 4.5101; Orientation: 0.2803; Line Search: 3.4161
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -43.30606201427904 (> -Infinity) after 64.715s (< 5400.000s)
Returns
-43.30606201427904