BasicOptimizer.scala:89 executed in 66.19 seconds (2.745 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 162680421471000
Reset training subject: 162682122115100
Adding measurement 7ee0fb4c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-22.10862203333556;dx=-0.003358700798783342
New Minimum: -22.10862203333556 > -22.11499141892179
WOLFE (weak): th(2.154434690031884)=-22.11499141892179; dx=-0.0032848494262615524 evalInputDelta=0.006369385586229015
New Minimum: -22.11499141892179 > -22.12134586482237
WOLFE (weak): th(4.308869380063768)=-22.12134586482237; dx=-0.003284485898674112 evalInputDelta=0.012723831486809445
New Minimum: -22.12134586482237 > -22.146810869501124
WOLFE (weak): th(12.926608140191302)=-22.146810869501124; dx=-0.0032952152088398566 evalInputDelta=0.038188836165563345
New Minimum: -22.146810869501124 > -22.26155733129318
WOLFE (weak): th(51.70643256076521)=-22.26155733129318; dx=-0.00330514907855245 evalInputDelta=0.15293529795761884
New Minimum: -22.26155733129318 > -22.87664222033784
WOLFE (weak): th(258.53216280382605)=-22.87664222033784; dx=-0.00333181829870941 evalInputDelta=0.7680201870022785
New Minimum: -22.87664222033784 > -26.856110503695778
WOLFE (weak): th(1551.1929768229563)=-26.856110503695778; dx=-0.003426990700656026 evalInputDelta=4.747488470360217
New Minimum: -26.856110503695778 > -56.38887370860496
WOLFE (weak): th(10858.350837760694)=-56.38887370860496; dx=-0.003162202533310699 evalInputDelta=34.2802516752694
New Minimum: -56.38887370860496 > -147.13120606069603
END: th(86866.80670208555)=-147.13120606069603; dx=-2.916152505381606E-4 evalInputDelta=125.02258402736047
Fitness changed from -22.10862203333556 to -147.13120606069603
Iteration 1 complete. Error: -147.13120606069603 Total: 11.1577; Orientation: 0.0146; Line Search: 7.6613
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3c7c7df3 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-147.13120606069603;dx=-0.010575267160275563
New Minimum: -147.13120606069603 > -197.5464443017135
WOLF (strong): th(187148.86177126726)=-197.5464443017135; dx=1.9906674159303543E-4 evalInputDelta=50.41523824101748
New Minimum: -197.5464443017135 > -324.1171181070201
END: th(93574.43088563363)=-324.1171181070201; dx=-9.056934813369737E-5 evalInputDelta=176.98591204632405
Fitness changed from -147.13120606069603 to -324.1171181070201
Iteration 2 complete. Error: -324.1171181070201 Total: 3.6795; Orientation: 0.0147; Line Search: 2.5327
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 679e763f to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-324.1171181070201;dx=-0.01382795151610424
New Minimum: -324.1171181070201 > -372.6641207240107
WOLF (strong): th(201600.00000000003)=-372.6641207240107; dx=5.537127588897516E-4 evalInputDelta=48.547002616990596
New Minimum: -372.6641207240107 > -611.9631575920237
WOLF (strong): th(100800.00000000001)=-611.9631575920237; dx=3.3733919997575024E-4 evalInputDelta=287.8460394850036
New Minimum: -611.9631575920237 > -688.0731928726768
END: th(33600.00000000001)=-688.0731928726768; dx=-0.004699838730395161 evalInputDelta=363.95607476565675
Fitness changed from -324.1171181070201 to -688.0731928726768
Iteration 3 complete. Error: -688.0731928726768 Total: 5.7227; Orientation: 0.0150; Line Search: 4.7765
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5754bcdc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.327e+04, gradient 1.307e-01, dot -0.283; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -688.0731928726768, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 5754bcdc to history. Total: 3
th(0)=-688.0731928726768;dx=-0.017054587949639623
New Minimum: -688.0731928726768 > -989.0711551753999
END: th(72389.00558507131)=-989.0711551753999; dx=-4.7554933387619446E-4 evalInputDelta=300.99796230272307
Fitness changed from -688.0731928726768 to -989.0711551753999
Iteration 4 complete. Error: -989.0711551753999 Total: 2.8572; Orientation: 0.2753; Line Search: 1.6836
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 13083659 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.799e+04, gradient 1.655e-01, dot -0.241; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -989.0711551753999, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 13083659 to history. Total: 3
th(0)=-989.0711551753999;dx=-0.027393968175701
Armijo: th(155957.38480938942)=-781.5962432739809; dx=0.001906587504241935 evalInputDelta=-207.47491190141898
New Minimum: -989.0711551753999 > -1234.6721224681473
WOLF (strong): th(77978.69240469471)=-1234.6721224681473; dx=0.0024752373629391175 evalInputDelta=245.60096729274744
New Minimum: -1234.6721224681473 > -1481.3941912941336
END: th(25992.89746823157)=-1481.3941912941336; dx=-0.00534626039744856 evalInputDelta=492.32303611873374
Fitness changed from -989.0711551753999 to -1481.3941912941336
Iteration 5 complete. Error: -1481.3941912941336 Total: 4.5315; Orientation: 0.2775; Line Search: 3.3244
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 56057765 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.591e+05, gradient 1.571e-01, dot -0.263; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1481.3941912941336, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 56057765 to history. Total: 3
th(0)=-1481.3941912941336;dx=-0.024599488728465298
New Minimum: -1481.3941912941336 > -1616.520105040454
WOLF (strong): th(56000.00000000002)=-1616.520105040454; dx=0.001920985830456631 evalInputDelta=135.1259137463203
New Minimum: -1616.520105040454 > -1770.381809102206
END: th(28000.00000000001)=-1770.381809102206; dx=-0.0024933002675534134 evalInputDelta=288.9876178080724
Fitness changed from -1481.3941912941336 to -1770.381809102206
Iteration 6 complete. Error: -1770.381809102206 Total: 3.8786; Orientation: 0.2671; Line Search: 2.5354
Adding measurement 5b38494f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.849e+05, gradient 1.756e-01, dot -0.241; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1770.381809102206, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 5b38494f to history. Total: 3
th(0)=-1770.381809102206;dx=-0.030720789916127735
Armijo: th(60324.17132089277)=-1656.3120931756794; dx=0.00259388029532267 evalInputDelta=-114.06971592652667
New Minimum: -1770.381809102206 > -1915.6603300167803
WOLF (strong): th(30162.085660446384)=-1915.6603300167803; dx=0.0037945050568680343 evalInputDelta=145.2785209145743
New Minimum: -1915.6603300167803 > -2013.2309852572805
END: th(10054.028553482129)=-2013.2309852572805; dx=-0.010751000716763121 evalInputDelta=242.84917615507447
Fitness changed from -1770.381809102206 to -2013.2309852572805
Iteration 7 complete. Error: -2013.2309852572805 Total: 4.4307; Orientation: 0.2754; Line Search: 3.3650
Adding measurement 1547fa0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.723e+06, gradient 1.545e-01, dot -0.288; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2013.2309852572805, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 1547fa0b to history. Total: 3
th(0)=-2013.2309852572805;dx=-0.02370249989131712
New Minimum: -2013.2309852572805 > -2160.484089272587
END: th(21660.74789019298)=-2160.484089272587; dx=-0.0030490948708520775 evalInputDelta=147.25310401530646
Fitness changed from -2013.2309852572805 to -2160.484089272587
Iteration 8 complete. Error: -2160.484089272587 Total: 2.9467; Orientation: 0.2705; Line Search: 1.9125
Adding measurement 1585938e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.974e+06, gradient 1.852e-01, dot -0.248; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2160.484089272587, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 1585938e to history. Total: 3
th(0)=-2160.484089272587;dx=-0.03411649867650632
Armijo: th(46666.66666666669)=-1936.3006610544758; dx=0.0029956940319331273 evalInputDelta=-224.1834282181112
New Minimum: -2160.484089272587 > -2192.833669022801
WOLF (strong): th(23333.333333333347)=-2192.833669022801; dx=0.006014086932325156 evalInputDelta=32.34957975021416
New Minimum: -2192.833669022801 > -2327.7739366184164
END: th(7777.777777777782)=-2327.7739366184164; dx=-0.005986037964659269 evalInputDelta=167.28984734582946
Fitness changed from -2160.484089272587 to -2327.7739366184164
Iteration 9 complete. Error: -2327.7739366184164 Total: 4.5347; Orientation: 0.2961; Line Search: 3.4447
Adding measurement 227ff412 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.431e+06, gradient 1.656e-01, dot -0.266; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2327.7739366184164, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 227ff412 to history. Total: 3
th(0)=-2327.7739366184164;dx=-0.027396881167086544
New Minimum: -2327.7739366184164 > -2348.55891984851
WOLF (strong): th(16756.71425580355)=-2348.55891984851; dx=0.0026131440999268996 evalInputDelta=20.78498323009353
New Minimum: -2348.55891984851 > -2420.246578346925
END: th(8378.357127901774)=-2420.246578346925; dx=-0.001841294511051716 evalInputDelta=92.47264172850873
Fitness changed from -2327.7739366184164 to -2420.246578346925
Iteration 10 complete. Error: -2420.246578346925 Total: 3.9028; Orientation: 0.2750; Line Search: 2.7995
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 712051a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.082e+07, gradient 1.803e-01, dot -0.250; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2420.246578346925, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 712051a to history. Total: 3
th(0)=-2420.246578346925;dx=-0.032473605650522264
Armijo: th(18050.623241827485)=-2329.6349499438747; dx=0.004613195655463843 evalInputDelta=-90.61162840305042
New Minimum: -2420.246578346925 > -2455.754161148872
WOLF (strong): th(9025.311620913742)=-2455.754161148872; dx=0.005305040058345817 evalInputDelta=35.50758280194668
New Minimum: -2455.754161148872 > -2490.274282082197
END: th(3008.4372069712476)=-2490.274282082197; dx=-0.012079380918777244 evalInputDelta=70.02770373527164
Fitness changed from -2420.246578346925 to -2490.274282082197
Iteration 11 complete. Error: -2490.274282082197 Total: 4.6316; Orientation: 0.2750; Line Search: 3.4418
Adding measurement 15e6cc78 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.722e+05, gradient 1.391e-01, dot -0.312; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2490.274282082197, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 15e6cc78 to history. Total: 3
th(0)=-2490.274282082197;dx=-0.019334875377579712
New Minimum: -2490.274282082197 > -2553.298846590426
END: th(6481.481481481486)=-2553.298846590426; dx=-0.009097326443589394 evalInputDelta=63.024564508229105
Fitness changed from -2490.274282082197 to -2553.298846590426
Iteration 12 complete. Error: -2553.298846590426 Total: 2.7695; Orientation: 0.2710; Line Search: 1.6892
Adding measurement 1ae2c8e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.146e+05, gradient 1.432e-01, dot -0.297; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2553.298846590426, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 1ae2c8e3 to history. Total: 3
th(0)=-2553.298846590426;dx=-0.020347601176554077
New Minimum: -2553.298846590426 > -2595.974768752578
END: th(13963.92854650296)=-2595.974768752578; dx=-0.0022585823367281136 evalInputDelta=42.67592216215189
Fitness changed from -2553.298846590426 to -2595.974768752578
Iteration 13 complete. Error: -2595.974768752578 Total: 2.9139; Orientation: 0.2713; Line Search: 1.8288
Adding measurement 4f42daba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.750e+06, gradient 1.991e-01, dot -0.221; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2595.974768752578, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 4f42daba to history. Total: 3
th(0)=-2595.974768752578;dx=-0.03946009338233101
Armijo: th(30084.37206971248)=-2338.0862901099267; dx=0.0018245805096546266 evalInputDelta=-257.8884786426511
Armijo: th(15042.18603485624)=-2483.3103913929085; dx=0.006988869416145876 evalInputDelta=-112.66437735966929
New Minimum: -2595.974768752578 > -2704.385789786657
WOLF (strong): th(5014.0620116187465)=-2704.385789786657; dx=0.0017998012689842372 evalInputDelta=108.41102103407911
END: th(1253.5155029046866)=-2646.9698713812368; dx=-0.027287845574092043 evalInputDelta=50.99510262865897
Fitness changed from -2595.974768752578 to -2704.385789786657
Iteration 14 complete. Error: -2704.385789786657 Total: 5.3568; Orientation: 0.2692; Line Search: 4.2949
Adding measurement 39f5c8cd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.670e+05, gradient 1.619e-01, dot -0.254; [967e6dce-bc43-434a-b3b3-fbcaa8e2c80a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2704.385789786657, -324.1171181070201, -147.13120606069603, -22.10862203333556
LBFGS Accumulation History: 3 points
Removed measurement 39f5c8cd to history. Total: 3
th(0)=-2704.385789786657;dx=-0.026089093451470292
New Minimum: -2704.385789786657 > -2743.2298481899034
END: th(2700.6172839506194)=-2743.2298481899034; dx=-0.0071826594755991045 evalInputDelta=38.844058403246436
Fitness changed from -2704.385789786657 to -2743.2298481899034
Iteration 15 complete. Error: -2743.2298481899034 Total: 2.7706; Orientation: 0.2709; Line Search: 1.6788
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -2743.2298481899034 (> -Infinity) after 66.186s (< 5400.000s)
Returns
-2743.2298481899034