BasicOptimizer.scala:89 executed in 46.45 seconds (0.886 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 223438969300400
Reset training subject: 223439592207900
Adding measurement 63f4b166 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-14.983964851687203;dx=-0.0017815408341877895
New Minimum: -14.983964851687203 > -14.98685251744252
WOLFE (weak): th(2.154434690031884)=-14.98685251744252; dx=-0.0017843442432267569 evalInputDelta=0.0028876657553169593
New Minimum: -14.98685251744252 > -14.989746440974237
WOLFE (weak): th(4.308869380063768)=-14.989746440974237; dx=-0.0017882504345658374 evalInputDelta=0.005781589287034095
New Minimum: -14.989746440974237 > -15.001314335327624
WOLFE (weak): th(12.926608140191302)=-15.001314335327624; dx=-0.0017842662788284808 evalInputDelta=0.017349483640421326
New Minimum: -15.001314335327624 > -15.053272167486622
WOLFE (weak): th(51.70643256076521)=-15.053272167486622; dx=-0.0017812100633554746 evalInputDelta=0.06930731579941884
New Minimum: -15.053272167486622 > -15.329669357488765
WOLFE (weak): th(258.53216280382605)=-15.329669357488765; dx=-0.001772867716585945 evalInputDelta=0.345704505801562
New Minimum: -15.329669357488765 > -16.962057947178376
END: th(1551.1929768229563)=-16.962057947178376; dx=-0.0016015627906387352 evalInputDelta=1.9780930954911735
Fitness changed from -14.983964851687203 to -16.962057947178376
Iteration 1 complete. Error: -16.962057947178376 Total: 6.0420; Orientation: 0.0169; Line Search: 4.2437
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 66b37785 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-16.962057947178376;dx=-0.0017774404939765805
New Minimum: -16.962057947178376 > -21.10707877343815
END: th(3341.943960201201)=-21.10707877343815; dx=-0.001533459474568451 evalInputDelta=4.145020826259774
Fitness changed from -16.962057947178376 to -21.10707877343815
Iteration 2 complete. Error: -21.10707877343815 Total: 1.8194; Orientation: 0.0163; Line Search: 1.2125
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 843eca5 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-21.10707877343815;dx=-0.002020255911590009
New Minimum: -21.10707877343815 > -29.860572584862684
END: th(7200.000000000001)=-29.860572584862684; dx=-0.0013690715919328841 evalInputDelta=8.753493811424534
Fitness changed from -21.10707877343815 to -29.860572584862684
Iteration 3 complete. Error: -29.860572584862684 Total: 2.0584; Orientation: 0.0172; Line Search: 1.4416
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7c7c1517 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.234e+04, gradient 4.803e-02, dot -0.607; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.860572584862684, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 7c7c1517 to history. Total: 3
th(0)=-29.860572584862684;dx=-0.002281104878542824
New Minimum: -29.860572584862684 > -42.23966198582841
END: th(15511.929768229566)=-42.23966198582841; dx=-8.935642020783819E-4 evalInputDelta=12.379089400965729
Fitness changed from -29.860572584862684 to -42.23966198582841
Iteration 4 complete. Error: -42.23966198582841 Total: 2.0736; Orientation: 0.2959; Line Search: 1.1721
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 439c6606 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.485e+04, gradient 4.251e-02, dot -0.398; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.23966198582841, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 439c6606 to history. Total: 3
th(0)=-42.23966198582841;dx=-0.0018024323896363172
New Minimum: -42.23966198582841 > -61.584595932485676
END: th(33419.43960201202)=-61.584595932485676; dx=-5.332091156706384E-4 evalInputDelta=19.344933946657264
Fitness changed from -42.23966198582841 to -61.584595932485676
Iteration 5 complete. Error: -61.584595932485676 Total: 2.0832; Orientation: 0.2977; Line Search: 1.2060
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 72ee53f4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.165e+04, gradient 4.324e-02, dot -0.223; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -61.584595932485676, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 72ee53f4 to history. Total: 3
th(0)=-61.584595932485676;dx=-0.0018594005459921692
New Minimum: -61.584595932485676 > -67.39870155982533
END: th(72000.00000000003)=-67.39870155982533; dx=-1.83693595956769E-4 evalInputDelta=5.81410562733965
Fitness changed from -61.584595932485676 to -67.39870155982533
Iteration 6 complete. Error: -67.39870155982533 Total: 2.1050; Orientation: 0.2942; Line Search: 1.2492
Adding measurement 21b6d950 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.697e+03, gradient 4.265e-02, dot -0.138; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -67.39870155982533, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 21b6d950 to history. Total: 3
th(0)=-67.39870155982533;dx=-0.0018151322231827458
Armijo: th(155119.2976822957)=-45.860263321857666; dx=-1.328612821750925E-5 evalInputDelta=-21.53843823796766
New Minimum: -67.39870155982533 > -86.64241930148492
END: th(77559.64884114785)=-86.64241930148492; dx=-2.4790731447008605E-4 evalInputDelta=19.243717741659594
Fitness changed from -67.39870155982533 to -86.64241930148492
Iteration 7 complete. Error: -86.64241930148492 Total: 2.7087; Orientation: 0.2998; Line Search: 1.8296
Adding measurement ce95853 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.216e+04, gradient 4.398e-02, dot -0.147; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -86.64241930148492, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement ce95853 to history. Total: 3
th(0)=-86.64241930148492;dx=-0.0019317582700381093
Armijo: th(167097.19801006012)=-69.46174862549054; dx=-3.0588362293722914E-5 evalInputDelta=-17.180670675994378
New Minimum: -86.64241930148492 > -113.2281858523094
END: th(83548.59900503006)=-113.2281858523094; dx=-3.0566073311669585E-4 evalInputDelta=26.585766550824474
Fitness changed from -86.64241930148492 to -113.2281858523094
Iteration 8 complete. Error: -113.2281858523094 Total: 3.0313; Orientation: 0.2911; Line Search: 2.2002
Adding measurement 516c6704 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.776e+04, gradient 3.997e-02, dot -0.160; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -113.2281858523094, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 516c6704 to history. Total: 3
th(0)=-113.2281858523094;dx=-0.0015945635833056688
Armijo: th(180000.0000000001)=-92.16613705871356; dx=-3.269410015268349E-5 evalInputDelta=-21.062048793595835
New Minimum: -113.2281858523094 > -143.0116971489953
END: th(90000.00000000004)=-143.0116971489953; dx=-3.0514403699229316E-4 evalInputDelta=29.7835112966859
Fitness changed from -113.2281858523094 to -143.0116971489953
Iteration 9 complete. Error: -143.0116971489953 Total: 2.6920; Orientation: 0.2832; Line Search: 1.8261
Adding measurement 9ad4e13 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.822e+04, gradient 4.026e-02, dot -0.162; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -143.0116971489953, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 9ad4e13 to history. Total: 3
th(0)=-143.0116971489953;dx=-0.0016142273849442596
Armijo: th(193899.12210286965)=-109.16749421777953; dx=-2.2819614443248048E-6 evalInputDelta=-33.84420293121576
New Minimum: -143.0116971489953 > -164.9692303485916
END: th(96949.56105143482)=-164.9692303485916; dx=-2.46654538440369E-4 evalInputDelta=21.9575331995963
Fitness changed from -143.0116971489953 to -164.9692303485916
Iteration 10 complete. Error: -164.9692303485916 Total: 2.7020; Orientation: 0.3049; Line Search: 1.7998
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6cd7d94b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.121e+04, gradient 3.893e-02, dot -0.176; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -164.9692303485916, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 6cd7d94b to history. Total: 3
th(0)=-164.9692303485916;dx=-0.001517254065531199
Armijo: th(208871.49751257518)=-126.00934132607637; dx=3.033447482498752E-5 evalInputDelta=-38.95988902251523
New Minimum: -164.9692303485916 > -189.21189968422416
END: th(104435.74875628759)=-189.21189968422416; dx=-2.211560260537709E-4 evalInputDelta=24.24266933563257
Fitness changed from -164.9692303485916 to -189.21189968422416
Iteration 11 complete. Error: -189.21189968422416 Total: 2.6760; Orientation: 0.2963; Line Search: 1.8163
Adding measurement 4924fe62 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.177e+05, gradient 3.835e-02, dot -0.194; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -189.21189968422416, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 4924fe62 to history. Total: 3
th(0)=-189.21189968422416;dx=-0.0014684258437618431
Armijo: th(225000.00000000015)=-140.38430934058331; dx=4.1794482452691146E-5 evalInputDelta=-48.82759034364085
New Minimum: -189.21189968422416 > -210.9251919775124
END: th(112500.00000000007)=-210.9251919775124; dx=-1.763670204207265E-4 evalInputDelta=21.713292293288248
Fitness changed from -189.21189968422416 to -210.9251919775124
Iteration 12 complete. Error: -210.9251919775124 Total: 3.1976; Orientation: 0.2937; Line Search: 2.3235
Adding measurement cad7e56 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.096e+05, gradient 3.871e-02, dot 0.209; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-210.9251919775124;dx=-889.4211833714321
Armijo: th(242373.9026285871)=652.8447891705432; dx=0.0018669834044196887 evalInputDelta=-863.7699811480556
Armijo: th(121186.95131429355)=649.0051627272757; dx=0.003699564248037109 evalInputDelta=-859.9303547047881
Armijo: th(40395.65043809785)=642.44624121752; dx=0.010905814960813085 evalInputDelta=-853.3714331950324
Armijo: th(10098.912609524463)=630.3247469835661; dx=0.04224223222420448 evalInputDelta=-841.2499389610786
Armijo: th(2019.7825219048925)=609.4533600698844; dx=0.200992538175544 evalInputDelta=-820.3785520473969
Armijo: th(336.6304203174821)=575.3882248179607; dx=1.102031064852508 evalInputDelta=-786.3134167954731
Armijo: th(48.09006004535458)=515.8561940399692; dx=6.6477407785076155 evalInputDelta=-726.7813860174816
Armijo: th(6.011257505669323)=385.36019950292604; dx=39.50977033572245 evalInputDelta=-596.2853914804384
Armijo: th(0.6679175006299247)=85.94455775175392; dx=152.18026906754957 evalInputDelta=-296.86974972926635
Armijo: th(0.06679175006299247)=-182.80256837284006; dx=-18.60532872573978 evalInputDelta=-28.122623604672356
New Minimum: -210.9251919775124 > -211.46435015295586
END: th(0.006071977278453861)=-211.46435015295586; dx=-430.48889397445475 evalInputDelta=0.5391581754434469
Fitness changed from -210.9251919775124 to -211.46435015295586
Iteration 13 complete. Error: -211.46435015295586 Total: 8.4949; Orientation: 0.2930; Line Search: 7.5763
Adding measurement e2eda59 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.472e+07, gradient 3.838e-02, dot -0.177; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -211.46435015295586, -210.9251919775124, -21.10707877343815, -16.962057947178376, -14.983964851687203
Rejected: LBFGS Orientation magnitude: 2.291e+07, gradient 3.838e-02, dot -0.215; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -211.46435015295586, -210.9251919775124, -21.10707877343815, -16.962057947178376
LBFGS Accumulation History: 3 points
Removed measurement e2eda59 to history. Total: 4
Removed measurement cad7e56 to history. Total: 3
th(0)=-211.46435015295586;dx=-0.0014688756183155866
New Minimum: -211.46435015295586 > -211.4643729828131
END: th(0.013081678485786384)=-211.4643729828131; dx=-0.001215874678337863 evalInputDelta=2.282985724377795E-5
Fitness changed from -211.46435015295586 to -211.4643729828131
Iteration 14 complete. Error: -211.4643729828131 Total: 2.6972; Orientation: 0.6365; Line Search: 1.2337
Adding measurement 4dac59b7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.711e+05, gradient 3.833e-02, dot -0.181; [efd97447-faeb-4738-b882-0991d3a6b9aa = 1.000/1.000e+00]
Orientation rejected. Popping history element from -211.4643729828131, -21.10707877343815, -16.962057947178376, -14.983964851687203
LBFGS Accumulation History: 3 points
Removed measurement 4dac59b7 to history. Total: 3
th(0)=-211.4643729828131;dx=-0.0014650274750270399
New Minimum: -211.4643729828131 > -211.4644173903981
END: th(0.028183621933621953)=-211.4644173903981; dx=-0.001215771687354883 evalInputDelta=4.4407584994132776E-5
Fitness changed from -211.4643729828131 to -211.4644173903981
Iteration 15 complete. Error: -211.4644173903981 Total: 2.0428; Orientation: 0.2939; Line Search: 1.1868
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -211.4644173903981 (> -Infinity) after 46.449s (< 5400.000s)
Returns
-211.4644173903981