BasicOptimizer.scala:89 executed in 63.17 seconds (0.790 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 144525710838800
Reset training subject: 144526698827500
Adding measurement 7afe30cb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-99.11311305618193;dx=-0.007603405177592237
New Minimum: -99.11311305618193 > -99.12742352018437
WOLFE (weak): th(2.154434690031884)=-99.12742352018437; dx=-0.0072872252470506955 evalInputDelta=0.014310464002434742
New Minimum: -99.12742352018437 > -99.14175654397596
WOLFE (weak): th(4.308869380063768)=-99.14175654397596; dx=-0.007254128996849755 evalInputDelta=0.028643487794028033
New Minimum: -99.14175654397596 > -99.19909746313219
WOLFE (weak): th(12.926608140191302)=-99.19909746313219; dx=-0.007313318364245926 evalInputDelta=0.08598440695026
New Minimum: -99.19909746313219 > -99.45693850701167
WOLFE (weak): th(51.70643256076521)=-99.45693850701167; dx=-0.007236685957485774 evalInputDelta=0.3438254508297405
New Minimum: -99.45693850701167 > -100.82824969967494
WOLFE (weak): th(258.53216280382605)=-100.82824969967494; dx=-0.007235227031407751 evalInputDelta=1.715136643493011
New Minimum: -100.82824969967494 > -109.16683762709259
WOLFE (weak): th(1551.1929768229563)=-109.16683762709259; dx=-0.007139365248402688 evalInputDelta=10.05372457091066
New Minimum: -109.16683762709259 > -158.78614868944015
END: th(10858.350837760694)=-158.78614868944015; dx=-0.005209328848961423 evalInputDelta=59.67303563325822
Fitness changed from -99.11311305618193 to -158.78614868944015
Iteration 1 complete. Error: -158.78614868944015 Total: 9.9678; Orientation: 0.0151; Line Search: 7.2708
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 53d5a1d8 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-158.78614868944015;dx=-0.009222672131964793
New Minimum: -158.78614868944015 > -242.95759821674645
END: th(23393.607721408407)=-242.95759821674645; dx=-0.003587283990514899 evalInputDelta=84.1714495273063
Fitness changed from -158.78614868944015 to -242.95759821674645
Iteration 2 complete. Error: -242.95759821674645 Total: 2.7002; Orientation: 0.0146; Line Search: 1.7295
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 74e1a0f2 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-242.95759821674645;dx=-0.009359708463655548
New Minimum: -242.95759821674645 > -384.9806001605795
END: th(50400.00000000001)=-384.9806001605795; dx=-0.0025490691590552535 evalInputDelta=142.02300194383307
Fitness changed from -242.95759821674645 to -384.9806001605795
Iteration 3 complete. Error: -384.9806001605795 Total: 2.6942; Orientation: 0.0154; Line Search: 1.7411
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 44322bba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.761e+04, gradient 1.080e-01, dot -0.203; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -384.9806001605795, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 44322bba to history. Total: 3
th(0)=-384.9806001605795;dx=-0.011566946555215538
New Minimum: -384.9806001605795 > -492.81059368686465
END: th(108583.50837760697)=-492.81059368686465; dx=-0.0011920755535697511 evalInputDelta=107.82999352628514
Fitness changed from -384.9806001605795 to -492.81059368686465
Iteration 4 complete. Error: -492.81059368686465 Total: 3.0492; Orientation: 0.3038; Line Search: 1.8064
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 68bda59a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.397e+04, gradient 1.161e-01, dot -0.195; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -492.81059368686465, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 68bda59a to history. Total: 3
th(0)=-492.81059368686465;dx=-0.013433086100996944
New Minimum: -492.81059368686465 > -542.7699501169901
END: th(233936.07721408413)=-542.7699501169901; dx=-6.034606527324649E-5 evalInputDelta=49.95935643012547
Fitness changed from -492.81059368686465 to -542.7699501169901
Iteration 5 complete. Error: -542.7699501169901 Total: 2.9867; Orientation: 0.2966; Line Search: 1.7351
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 65be8cee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.560e+04, gradient 1.003e-01, dot -0.110; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -542.7699501169901, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 65be8cee to history. Total: 3
th(0)=-542.7699501169901;dx=-0.010012875681211258
Armijo: th(504000.0000000002)=-511.430148578474; dx=4.2634324156201305E-4 evalInputDelta=-31.3398015385161
New Minimum: -542.7699501169901 > -905.6854103384906
END: th(252000.0000000001)=-905.6854103384906; dx=-1.5024581535672527E-4 evalInputDelta=362.91546022150044
Fitness changed from -542.7699501169901 to -905.6854103384906
Iteration 6 complete. Error: -905.6854103384906 Total: 4.0340; Orientation: 0.4814; Line Search: 2.6037
Adding measurement 78abb028 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.968e+04, gradient 9.371e-02, dot -0.124; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -905.6854103384906, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 78abb028 to history. Total: 3
th(0)=-905.6854103384906;dx=-0.008746042333490791
Armijo: th(542917.5418880349)=-727.0397169343639; dx=6.878570462313805E-4 evalInputDelta=-178.64569340412663
New Minimum: -905.6854103384906 > -1247.165837008054
WOLF (strong): th(271458.77094401745)=-1247.165837008054; dx=1.1305612841588038E-5 evalInputDelta=341.4804266695635
New Minimum: -1247.165837008054 > -1313.6193322268061
END: th(90486.25698133915)=-1313.6193322268061; dx=-0.003169109191819366 evalInputDelta=407.9339218883156
Fitness changed from -905.6854103384906 to -1313.6193322268061
Iteration 7 complete. Error: -1313.6193322268061 Total: 4.7167; Orientation: 0.2816; Line Search: 3.6136
Adding measurement 57c08418 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.773e+05, gradient 8.464e-02, dot -0.202; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1313.6193322268061, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 57c08418 to history. Total: 3
th(0)=-1313.6193322268061;dx=-0.007129528515862484
New Minimum: -1313.6193322268061 > -1633.208115948485
END: th(194946.7310117368)=-1633.208115948485; dx=-0.0012200772865479798 evalInputDelta=319.58878372167896
Fitness changed from -1313.6193322268061 to -1633.208115948485
Iteration 8 complete. Error: -1633.208115948485 Total: 2.7818; Orientation: 0.2728; Line Search: 1.7005
Adding measurement 8bd69b8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.265e+05, gradient 9.582e-02, dot -0.149; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1633.208115948485, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 8bd69b8 to history. Total: 3
th(0)=-1633.208115948485;dx=-0.00913998378652044
Armijo: th(420000.0000000002)=-1237.0739670690673; dx=6.810745684019043E-4 evalInputDelta=-396.1341488794178
New Minimum: -1633.208115948485 > -1743.6368601288516
END: th(210000.0000000001)=-1743.6368601288516; dx=-1.1646971562753366E-4 evalInputDelta=110.4287441803665
Fitness changed from -1633.208115948485 to -1743.6368601288516
Iteration 9 complete. Error: -1743.6368601288516 Total: 3.7003; Orientation: 0.2747; Line Search: 2.6210
Adding measurement e4ad631 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.697e+05, gradient 1.053e-01, dot 0.205; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1743.6368601288516;dx=-3646.53297244478
Armijo: th(452431.2849066958)=-681.9988544711787; dx=8.236871624302097E-4 evalInputDelta=-1061.638005657673
Armijo: th(226215.6424533479)=-682.00586507742; dx=0.001648754599401176 evalInputDelta=-1061.6309950514315
Armijo: th(75405.21415111596)=-682.019313326243; dx=0.0049388723776866515 evalInputDelta=-1061.6175468026086
Armijo: th(18851.30353777899)=-682.0412377561931; dx=0.0197602252586708 evalInputDelta=-1061.5956223726585
Armijo: th(3770.260707555798)=-682.1079689096148; dx=0.09943639196677816 evalInputDelta=-1061.5288912192368
Armijo: th(628.376784592633)=-683.3064603455758; dx=0.5932745504414337 evalInputDelta=-1060.3303997832759
Armijo: th(89.76811208466187)=-687.5543842126236; dx=4.09604750526596 evalInputDelta=-1056.0824759162278
Armijo: th(11.221014010582733)=-719.0716236926611; dx=32.7118135715853 evalInputDelta=-1024.5652364361904
Armijo: th(1.2467793345091926)=-946.1783658486385; dx=263.30905742927456 evalInputDelta=-797.4584942802131
Armijo: th(0.12467793345091926)=-1624.2468254548014; dx=457.04265115368315 evalInputDelta=-119.39003467405018
New Minimum: -1743.6368601288516 > -1746.0739531875504
END: th(0.011334357586447207)=-1746.0739531875504; dx=-881.1181593620158 evalInputDelta=2.4370930586987924
Fitness changed from -1743.6368601288516 to -1746.0739531875504
Iteration 10 complete. Error: -1746.0739531875504 Total: 11.8278; Orientation: 0.2826; Line Search: 10.7448
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 635a792 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 7.650e+05, gradient 1.041e-01, dot -0.163; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.0739531875504, -1743.6368601288516, -242.95759821674645, -158.78614868944015, -99.11311305618193
Rejected: LBFGS Orientation magnitude: 1.773e+06, gradient 1.041e-01, dot -0.185; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.0739531875504, -1743.6368601288516, -242.95759821674645, -158.78614868944015
LBFGS Accumulation History: 3 points
Removed measurement 635a792 to history. Total: 4
Removed measurement e4ad631 to history. Total: 3
th(0)=-1746.0739531875504;dx=-0.010806290648939742
New Minimum: -1746.0739531875504 > -1746.0741679550729
END: th(0.02441913317346792)=-1746.0741679550729; dx=-0.007584278247547697 evalInputDelta=2.1476752249327546E-4
Fitness changed from -1746.0739531875504 to -1746.0741679550729
Iteration 11 complete. Error: -1746.0741679550729 Total: 3.3903; Orientation: 0.5877; Line Search: 1.7352
Adding measurement 7ac43b28 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.759e+05, gradient 1.043e-01, dot -0.153; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.0741679550729, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 7ac43b28 to history. Total: 3
th(0)=-1746.0741679550729;dx=-0.010833411933095092
New Minimum: -1746.0741679550729 > -1746.074525900885
END: th(0.05260942760942765)=-1746.074525900885; dx=-0.0075780194198081675 evalInputDelta=3.579458120839263E-4
Fitness changed from -1746.0741679550729 to -1746.074525900885
Iteration 12 complete. Error: -1746.074525900885 Total: 2.7799; Orientation: 0.2782; Line Search: 1.6596
Adding measurement 485f3ed9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.047e+05, gradient 1.043e-01, dot -0.152; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.074525900885, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 485f3ed9 to history. Total: 3
th(0)=-1746.074525900885;dx=-0.010822694670873064
New Minimum: -1746.074525900885 > -1746.0753849705345
END: th(0.11334357586447208)=-1746.0753849705345; dx=-0.007593857878263525 evalInputDelta=8.590696495502925E-4
Fitness changed from -1746.074525900885 to -1746.0753849705345
Iteration 13 complete. Error: -1746.0753849705345 Total: 2.7691; Orientation: 0.2724; Line Search: 1.6771
Adding measurement 33a76b4c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.773e+05, gradient 1.043e-01, dot -0.151; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.0753849705345, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 33a76b4c to history. Total: 3
th(0)=-1746.0753849705345;dx=-0.010836714297890314
New Minimum: -1746.0753849705345 > -1746.0772462866585
END: th(0.24419133173467922)=-1746.0772462866585; dx=-0.007560335275548546 evalInputDelta=0.0018613161239500187
Fitness changed from -1746.0753849705345 to -1746.0772462866585
Iteration 14 complete. Error: -1746.0772462866585 Total: 2.8650; Orientation: 0.2740; Line Search: 1.8062
Adding measurement 197d2f07 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.692e+05, gradient 1.041e-01, dot -0.154; [7b836e09-b63f-436c-aaa0-4de29905e22e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1746.0772462866585, -242.95759821674645, -158.78614868944015, -99.11311305618193
LBFGS Accumulation History: 3 points
Removed measurement 197d2f07 to history. Total: 3
th(0)=-1746.0772462866585;dx=-0.010788937341196583
New Minimum: -1746.0772462866585 > -1746.0814700351184
END: th(0.5260942760942765)=-1746.0814700351184; dx=-0.0075746307799958775 evalInputDelta=0.00422374845993545
Fitness changed from -1746.0772462866585 to -1746.0814700351184
Iteration 15 complete. Error: -1746.0814700351184 Total: 2.8064; Orientation: 0.2683; Line Search: 1.7000
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1746.0814700351184 (> -Infinity) after 63.169s (< 5400.000s)
Returns
-1746.0814700351184