BasicOptimizer.scala:89 executed in 60.84 seconds (1.075 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140383179013499
Reset training subject: 140384074632000
Adding measurement 1b34d839 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.294549595867679;dx=-5.377283353089399E-5
New Minimum: -3.294549595867679 > -3.29467211372349
WOLFE (weak): th(2.154434690031884)=-3.29467211372349; dx=-5.3864266751480115E-5 evalInputDelta=1.2251785581085528E-4
New Minimum: -3.29467211372349 > -3.294794236238658
WOLFE (weak): th(4.308869380063768)=-3.294794236238658; dx=-5.368536509751472E-5 evalInputDelta=2.446403709788214E-4
New Minimum: -3.294794236238658 > -3.2952836578580293
WOLFE (weak): th(12.926608140191302)=-3.2952836578580293; dx=-5.366075379494622E-5 evalInputDelta=7.340619903501988E-4
New Minimum: -3.2952836578580293 > -3.2974850585930837
WOLFE (weak): th(51.70643256076521)=-3.2974850585930837; dx=-5.364824563080387E-5 evalInputDelta=0.0029354627254045695
New Minimum: -3.2974850585930837 > -3.309210402787413
WOLFE (weak): th(258.53216280382605)=-3.309210402787413; dx=-5.364207790047527E-5 evalInputDelta=0.014660806919733815
New Minimum: -3.309210402787413 > -3.382147646084852
WOLFE (weak): th(1551.1929768229563)=-3.382147646084852; dx=-5.326867312339954E-5 evalInputDelta=0.08759805021717293
New Minimum: -3.382147646084852 > -3.877851276969923
WOLFE (weak): th(10858.350837760694)=-3.877851276969923; dx=-4.870078924219711E-5 evalInputDelta=0.583301681102244
New Minimum: -3.877851276969923 > -6.065079425678976
END: th(86866.80670208555)=-6.065079425678976; dx=-1.9277366284518943E-5 evalInputDelta=2.7705298298112973
Fitness changed from -3.294549595867679 to -6.065079425678976
Iteration 1 complete. Error: -6.065079425678976 Total: 10.7198; Orientation: 0.0150; Line Search: 8.2211
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2d082b6a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-6.065079425678976;dx=-5.167014231080169E-5
New Minimum: -6.065079425678976 > -9.310981074082017
END: th(187148.86177126726)=-9.310981074082017; dx=-1.2158590430006848E-5 evalInputDelta=3.2459016484030405
Fitness changed from -6.065079425678976 to -9.310981074082017
Iteration 2 complete. Error: -9.310981074082017 Total: 2.6185; Orientation: 0.0146; Line Search: 1.6719
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6ddf1b59 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-9.310981074082017;dx=-6.715786606716142E-5
New Minimum: -9.310981074082017 > -13.8478852597829
END: th(403200.00000000006)=-13.8478852597829; dx=-1.2896422672778592E-6 evalInputDelta=4.536904185700884
Fitness changed from -9.310981074082017 to -13.8478852597829
Iteration 3 complete. Error: -13.8478852597829 Total: 2.6755; Orientation: 0.0146; Line Search: 1.7220
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 699053cb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.419e+03, gradient 8.820e-03, dot -0.224; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.8478852597829, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 699053cb to history. Total: 3
th(0)=-13.8478852597829;dx=-7.720032147097311E-5
New Minimum: -13.8478852597829 > -16.50326306633911
WOLF (strong): th(868668.0670208557)=-16.50326306633911; dx=3.984919152442049E-6 evalInputDelta=2.655377806556208
New Minimum: -16.50326306633911 > -20.15448155187881
WOLF (strong): th(434334.03351042786)=-20.15448155187881; dx=4.895889255208455E-7 evalInputDelta=6.306596292095909
New Minimum: -20.15448155187881 > -20.469236547891654
END: th(144778.01117014262)=-20.469236547891654; dx=-2.3011279814369863E-5 evalInputDelta=6.6213512881087535
Fitness changed from -13.8478852597829 to -20.469236547891654
Iteration 4 complete. Error: -20.469236547891654 Total: 4.7413; Orientation: 0.2792; Line Search: 3.4246
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2ddc1454 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.374e+03, gradient 7.322e-03, dot -0.287; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.469236547891654, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 2ddc1454 to history. Total: 3
th(0)=-20.469236547891654;dx=-5.3446655163181366E-5
New Minimum: -20.469236547891654 > -26.326493488432437
END: th(311914.76961877884)=-26.326493488432437; dx=-8.034199254790117E-6 evalInputDelta=5.857256940540783
Fitness changed from -20.469236547891654 to -26.326493488432437
Iteration 5 complete. Error: -26.326493488432437 Total: 2.8932; Orientation: 0.2814; Line Search: 1.7139
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 39ecba78 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.794e+03, gradient 9.205e-03, dot -0.231; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.326493488432437, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 39ecba78 to history. Total: 3
th(0)=-26.326493488432437;dx=-8.437529699301657E-5
Armijo: th(672000.0000000002)=-24.16781303801923; dx=3.581054417574066E-6 evalInputDelta=-2.1586804504132076
New Minimum: -26.326493488432437 > -28.999815977300074
WOLF (strong): th(336000.0000000001)=-28.999815977300074; dx=3.026270383367648E-6 evalInputDelta=2.673322488867637
New Minimum: -28.999815977300074 > -31.194217002425994
END: th(112000.00000000004)=-31.194217002425994; dx=-1.989193407036695E-5 evalInputDelta=4.867723513993557
Fitness changed from -26.326493488432437 to -31.194217002425994
Iteration 6 complete. Error: -31.194217002425994 Total: 4.7713; Orientation: 0.2794; Line Search: 3.5728
Adding measurement 3c4b771c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.550e+04, gradient 7.389e-03, dot -0.338; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.194217002425994, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 3c4b771c to history. Total: 3
th(0)=-31.194217002425994;dx=-5.444045911885403E-5
New Minimum: -31.194217002425994 > -34.94980410893042
END: th(241296.68528357107)=-34.94980410893042; dx=-6.621538581672409E-6 evalInputDelta=3.7555871065044286
Fitness changed from -31.194217002425994 to -34.94980410893042
Iteration 7 complete. Error: -34.94980410893042 Total: 2.7982; Orientation: 0.2818; Line Search: 1.6865
Adding measurement 77a778ae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.478e+04, gradient 9.944e-03, dot -0.224; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -34.94980410893042, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 77a778ae to history. Total: 3
th(0)=-34.94980410893042;dx=-9.839514603428389E-5
Armijo: th(519857.9493646315)=-30.659391183692403; dx=7.269396836822403E-6 evalInputDelta=-4.290412925238019
New Minimum: -34.94980410893042 > -36.126654583104795
WOLF (strong): th(259928.97468231575)=-36.126654583104795; dx=8.395469346209543E-6 evalInputDelta=1.1768504741743726
New Minimum: -36.126654583104795 > -39.417668956796625
END: th(86642.99156077192)=-39.417668956796625; dx=-1.3613789623978039E-5 evalInputDelta=4.467864847866203
Fitness changed from -34.94980410893042 to -39.417668956796625
Iteration 8 complete. Error: -39.417668956796625 Total: 4.4710; Orientation: 0.2749; Line Search: 3.4197
Adding measurement 2f987f9a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.631e+04, gradient 7.214e-03, dot -0.373; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.417668956796625, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 2f987f9a to history. Total: 3
th(0)=-39.417668956796625;dx=-5.191113620126414E-5
New Minimum: -39.417668956796625 > -42.82499368142587
END: th(186666.66666666677)=-42.82499368142587; dx=-1.101772333291119E-5 evalInputDelta=3.407324724629248
Fitness changed from -39.417668956796625 to -42.82499368142587
Iteration 9 complete. Error: -42.82499368142587 Total: 3.1135; Orientation: 0.2968; Line Search: 1.9838
Adding measurement 213804d8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.716e+04, gradient 9.036e-03, dot -0.271; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.82499368142587, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 213804d8 to history. Total: 3
th(0)=-42.82499368142587;dx=-8.152821100519794E-5
Armijo: th(402161.1421392852)=-39.747942638177626; dx=2.997681161967394E-6 evalInputDelta=-3.0770510432482467
New Minimum: -42.82499368142587 > -43.79029790221673
WOLF (strong): th(201080.5710696426)=-43.79029790221673; dx=3.8063039949268457E-6 evalInputDelta=0.9653042207908555
New Minimum: -43.79029790221673 > -45.77042388218835
END: th(67026.85702321421)=-45.77042388218835; dx=-1.792770569258634E-5 evalInputDelta=2.9454302007624733
Fitness changed from -42.82499368142587 to -45.77042388218835
Iteration 10 complete. Error: -45.77042388218835 Total: 4.4944; Orientation: 0.2772; Line Search: 3.3739
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 47f00d9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.556e+04, gradient 6.739e-03, dot -0.426; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.77042388218835, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 47f00d9 to history. Total: 3
th(0)=-45.77042388218835;dx=-4.515751487663529E-5
New Minimum: -45.77042388218835 > -48.504357676772706
END: th(144404.9859346199)=-48.504357676772706; dx=-1.8760357451398636E-5 evalInputDelta=2.7339337945843596
Fitness changed from -45.77042388218835 to -48.504357676772706
Iteration 11 complete. Error: -48.504357676772706 Total: 3.1740; Orientation: 0.5054; Line Search: 1.7455
Adding measurement 791f2de0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.672e+04, gradient 7.624e-03, dot -0.349; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -48.504357676772706, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 791f2de0 to history. Total: 3
th(0)=-48.504357676772706;dx=-5.796819271604848E-5
Armijo: th(311111.11111111136)=-48.27580271432048; dx=-2.138560019964646E-6 evalInputDelta=-0.22855496245222895
New Minimum: -48.504357676772706 > -50.20213654349237
END: th(155555.55555555568)=-50.20213654349237; dx=-5.4954384223220205E-6 evalInputDelta=1.6977788667196663
Fitness changed from -48.504357676772706 to -50.20213654349237
Iteration 12 complete. Error: -50.20213654349237 Total: 3.8551; Orientation: 0.2740; Line Search: 2.7471
Adding measurement 71526683 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.683e+04, gradient 8.245e-03, dot -0.391; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -50.20213654349237, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 71526683 to history. Total: 3
th(0)=-50.20213654349237;dx=-6.777833285987912E-5
Armijo: th(335134.28511607106)=-47.711198401681244; dx=1.2076986906034408E-5 evalInputDelta=-2.490938141811128
New Minimum: -50.20213654349237 > -52.84546263675712
WOLF (strong): th(167567.14255803553)=-52.84546263675712; dx=5.0893378158622865E-6 evalInputDelta=2.643326093264747
New Minimum: -52.84546263675712 > -53.138992934761546
END: th(55855.71418601184)=-53.138992934761546; dx=-3.832238553921466E-5 evalInputDelta=2.936856391269174
Fitness changed from -50.20213654349237 to -53.138992934761546
Iteration 13 complete. Error: -53.138992934761546 Total: 4.6062; Orientation: 0.2761; Line Search: 3.4993
Adding measurement 45d3c46c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.119e+05, gradient 6.670e-03, dot -0.461; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -53.138992934761546, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 45d3c46c to history. Total: 3
th(0)=-53.138992934761546;dx=-4.424244181863305E-5
New Minimum: -53.138992934761546 > -55.77179282772598
END: th(120337.48827884992)=-55.77179282772598; dx=-2.318604624395683E-5 evalInputDelta=2.632799892964435
Fitness changed from -53.138992934761546 to -55.77179282772598
Iteration 14 complete. Error: -55.77179282772598 Total: 2.7949; Orientation: 0.2767; Line Search: 1.7382
Adding measurement 30402ee5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.286e+05, gradient 6.699e-03, dot -0.437; [5bebc2b5-080a-49d8-a0a9-023a7e6b6417 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.77179282772598, -9.310981074082017, -6.065079425678976, -3.294549595867679
LBFGS Accumulation History: 3 points
Removed measurement 30402ee5 to history. Total: 3
th(0)=-55.77179282772598;dx=-4.4689882953018254E-5
New Minimum: -55.77179282772598 > -58.38124293110325
END: th(259259.25925925947)=-58.38124293110325; dx=-1.327803485902363E-5 evalInputDelta=2.60945010337727
Fitness changed from -55.77179282772598 to -58.38124293110325
Iteration 15 complete. Error: -58.38124293110325 Total: 3.0009; Orientation: 0.4036; Line Search: 1.7933
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -58.38124293110325 (> -Infinity) after 60.834s (< 5400.000s)
Returns
-58.38124293110325