BasicOptimizer.scala:89 executed in 65.90 seconds (0.870 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 168319426082400
Reset training subject: 168320349105000
Adding measurement 4f138eee to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.0418072434971343;dx=-1.2079957781316653E-7
New Minimum: -0.0418072434971343 > -0.041807470329788064
WOLFE (weak): th(2.154434690031884)=-0.041807470329788064; dx=-1.2047327746577584E-7 evalInputDelta=2.268326537643861E-7
New Minimum: -0.041807470329788064 > -0.04180771959502125
WOLFE (weak): th(4.308869380063768)=-0.04180771959502125; dx=-1.2073700509151942E-7 evalInputDelta=4.7609788694780164E-7
New Minimum: -0.04180771959502125 > -0.04180867052823631
WOLFE (weak): th(12.926608140191302)=-0.04180867052823631; dx=-1.2063791306012192E-7 evalInputDelta=1.4270311020081472E-6
New Minimum: -0.04180867052823631 > -0.04181293512915636
WOLFE (weak): th(51.70643256076521)=-0.04181293512915636; dx=-1.2031711460165501E-7 evalInputDelta=5.691632022061799E-6
New Minimum: -0.04181293512915636 > -0.041835691422957
WOLFE (weak): th(258.53216280382605)=-0.041835691422957; dx=-1.2061920093667771E-7 evalInputDelta=2.8447925822697173E-5
New Minimum: -0.041835691422957 > -0.04197787150464756
WOLFE (weak): th(1551.1929768229563)=-0.04197787150464756; dx=-1.208485506228579E-7 evalInputDelta=1.706280075132599E-4
New Minimum: -0.04197787150464756 > -0.04300744371874128
WOLFE (weak): th(10858.350837760694)=-0.04300744371874128; dx=-1.2147028579374564E-7 evalInputDelta=0.0012002002216069813
New Minimum: -0.04300744371874128 > -0.05139645405672322
WOLFE (weak): th(86866.80670208555)=-0.05139645405672322; dx=-1.1593019433172144E-7 evalInputDelta=0.009589210559588918
New Minimum: -0.05139645405672322 > -0.10353436569302185
END: th(781801.26031877)=-0.10353436569302185; dx=-3.997943113451405E-8 evalInputDelta=0.06172712219588755
Fitness changed from -0.0418072434971343 to -0.10353436569302185
Iteration 1 complete. Error: -0.10353436569302185 Total: 11.0914; Orientation: 0.0145; Line Search: 8.5532
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 44bc2a95 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.10353436569302185;dx=-1.792742172888963E-7
New Minimum: -0.10353436569302185 > -0.18557194670829533
END: th(1684339.7559414052)=-0.18557194670829533; dx=-5.344911894785279E-9 evalInputDelta=0.08203758101527348
Fitness changed from -0.10353436569302185 to -0.18557194670829533
Iteration 2 complete. Error: -0.18557194670829533 Total: 2.5653; Orientation: 0.0144; Line Search: 1.6518
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7ad42726 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.18557194670829533;dx=-2.1650430872819388E-7
New Minimum: -0.18557194670829533 > -0.22565895386466595
WOLF (strong): th(3628800.0)=-0.22565895386466595; dx=9.204368652173849E-9 evalInputDelta=0.04008700715637062
New Minimum: -0.22565895386466595 > -0.29808006789874897
WOLF (strong): th(1814400.0)=-0.29808006789874897; dx=1.4262441434835848E-9 evalInputDelta=0.11250812119045364
New Minimum: -0.29808006789874897 > -0.30663375605184906
END: th(604800.0)=-0.30663375605184906; dx=-7.974583310604807E-8 evalInputDelta=0.12106180934355373
Fitness changed from -0.18557194670829533 to -0.30663375605184906
Iteration 3 complete. Error: -0.30663375605184906 Total: 4.3829; Orientation: 0.0161; Line Search: 3.4669
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2c50b174 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.450e+03, gradient 4.299e-04, dot -0.346; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.30663375605184906, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 2c50b174 to history. Total: 3
th(0)=-0.30663375605184906;dx=-1.839259862866554E-7
New Minimum: -0.30663375605184906 > -0.4189320962326587
END: th(1303002.1005312833)=-0.4189320962326587; dx=-3.1050323159944924E-8 evalInputDelta=0.11229834018080964
Fitness changed from -0.30663375605184906 to -0.4189320962326587
Iteration 4 complete. Error: -0.4189320962326587 Total: 2.7981; Orientation: 0.2677; Line Search: 1.6587
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3b9e8549 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.754e+03, gradient 5.377e-04, dot -0.280; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.4189320962326587, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 3b9e8549 to history. Total: 3
th(0)=-0.4189320962326587;dx=-2.853301763807153E-7
New Minimum: -0.4189320962326587 > -0.43139690725333196
WOLF (strong): th(2807232.9265690087)=-0.43139690725333196; dx=8.897822425253431E-9 evalInputDelta=0.012464811020673261
New Minimum: -0.43139690725333196 > -0.5820550977925849
END: th(1403616.4632845044)=-0.5820550977925849; dx=-1.3761493591619127E-8 evalInputDelta=0.1631230015599262
Fitness changed from -0.4189320962326587 to -0.5820550977925849
Iteration 5 complete. Error: -0.5820550977925849 Total: 3.6340; Orientation: 0.2690; Line Search: 2.4909
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement a2280d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.545e+03, gradient 5.506e-04, dot -0.247; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.5820550977925849, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement a2280d2 to history. Total: 3
th(0)=-0.5820550977925849;dx=-3.0086363733791407E-7
New Minimum: -0.5820550977925849 > -0.6056125388282179
WOLF (strong): th(3024000.0)=-0.6056125388282179; dx=4.0211471627033176E-8 evalInputDelta=0.023557441035632953
New Minimum: -0.6056125388282179 > -0.8290349436488409
END: th(1512000.0)=-0.8290349436488409; dx=-4.289411263598684E-8 evalInputDelta=0.246979845856256
Fitness changed from -0.5820550977925849 to -0.8290349436488409
Iteration 6 complete. Error: -0.8290349436488409 Total: 3.8977; Orientation: 0.2646; Line Search: 2.5710
Adding measurement 25b71c94 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.907e+03, gradient 5.740e-04, dot -0.226; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.8290349436488409, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 25b71c94 to history. Total: 3
th(0)=-0.8290349436488409;dx=-3.2905900130820706E-7
New Minimum: -0.8290349436488409 > -0.8579929070233124
WOLF (strong): th(3257505.2513282085)=-0.8579929070233124; dx=3.640211268970071E-8 evalInputDelta=0.02895796337447154
New Minimum: -0.8579929070233124 > -1.1371208067590828
END: th(1628752.6256641042)=-1.1371208067590828; dx=-5.0078504713599604E-8 evalInputDelta=0.3080858631102419
Fitness changed from -0.8290349436488409 to -1.1371208067590828
Iteration 7 complete. Error: -1.1371208067590828 Total: 3.5716; Orientation: 0.2674; Line Search: 2.4877
Adding measurement 43566994 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.819e+04, gradient 6.316e-04, dot -0.240; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.1371208067590828, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 43566994 to history. Total: 3
th(0)=-1.1371208067590828;dx=-3.9861211200276533E-7
Armijo: th(3509041.1582112615)=-1.053110839965581; dx=4.776187256156021E-8 evalInputDelta=-0.0840099667935017
New Minimum: -1.1371208067590828 > -1.422627924422763
END: th(1754520.5791056307)=-1.422627924422763; dx=-1.254922768040664E-8 evalInputDelta=0.2855071176636803
Fitness changed from -1.1371208067590828 to -1.422627924422763
Iteration 8 complete. Error: -1.422627924422763 Total: 3.7468; Orientation: 0.2694; Line Search: 2.6707
Adding measurement 3d0f8c01 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.706e+06, gradient 6.714e-04, dot -0.218; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.422627924422763, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 3d0f8c01 to history. Total: 3
th(0)=-1.422627924422763;dx=-4.5016862259065835E-7
Armijo: th(3780000.000000001)=-1.2390718055134609; dx=7.132002215279183E-8 evalInputDelta=-0.18355611890930223
New Minimum: -1.422627924422763 > -1.7735803811715716
WOLF (strong): th(1890000.0000000005)=-1.7735803811715716; dx=1.67831996722368E-8 evalInputDelta=0.3509524567488085
END: th(630000.0000000001)=-1.746787516096783; dx=-2.45313936991668E-7 evalInputDelta=0.32415959167401986
Fitness changed from -1.422627924422763 to -1.7735803811715716
Iteration 9 complete. Error: -1.7735803811715716 Total: 4.3660; Orientation: 0.2709; Line Search: 3.3159
Adding measurement 19ba871a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.045e+03, gradient 7.038e-04, dot -0.100; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.7735803811715716, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 19ba871a to history. Total: 3
th(0)=-1.7735803811715716;dx=-4.948571180694469E-7
New Minimum: -1.7735803811715716 > -2.22009267674158
END: th(1357293.854720087)=-2.22009267674158; dx=-7.583348153018684E-8 evalInputDelta=0.44651229557000827
Fitness changed from -1.7735803811715716 to -2.22009267674158
Iteration 10 complete. Error: -2.22009267674158 Total: 2.8781; Orientation: 0.2721; Line Search: 1.7725
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5b172bb0 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.620e+03, gradient 5.779e-04, dot 0.403; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2.22009267674158;dx=-1.539100962730741
Armijo: th(2924200.9651760515)=-2.0983199541432285; dx=6.176392858002572E-7 evalInputDelta=-0.12177272259835137
Armijo: th(1462100.4825880257)=-2.098319242154286; dx=1.2286231341975914E-6 evalInputDelta=-0.12177343458729384
Armijo: th(487366.8275293419)=-2.0983477215235458; dx=3.6863336414457173E-6 evalInputDelta=-0.12174495521803408
Armijo: th(121841.70688233548)=-2.0981341168314676; dx=1.4729103612153871E-5 evalInputDelta=-0.12195855991011229
Armijo: th(24368.341376467095)=-2.098714257190235; dx=7.358514146710456E-5 evalInputDelta=-0.12137841955134476
Armijo: th(4061.3902294111826)=-2.103935524174802; dx=4.423086464228446E-4 evalInputDelta=-0.11615715256677772
Armijo: th(580.1986042015975)=-2.129316703195212; dx=0.0030354891504943637 evalInputDelta=-0.0907759735463678
New Minimum: -2.22009267674158 > -2.2724974020008206
WOLF (strong): th(72.52482552519969)=-2.2724974020008206; dx=0.02286294868013913 evalInputDelta=0.05240472525924078
New Minimum: -2.2724974020008206 > -3.415721494932346
WOLF (strong): th(8.05831394724441)=-3.415721494932346; dx=0.09805324088396608 evalInputDelta=1.195628818190766
END: th(0.8058313947244409)=-3.3807873556110857; dx=-0.6569647861978682 evalInputDelta=1.1606946788695058
Fitness changed from -2.22009267674158 to -3.415721494932346
Iteration 11 complete. Error: -3.415721494932346 Total: 10.7754; Orientation: 0.2834; Line Search: 9.5708
Adding measurement 29c0ebd0 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 7.457e+03, gradient 4.512e-04, dot -0.187; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.415721494932346, -2.22009267674158, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
Rejected: LBFGS Orientation magnitude: 1.002e+04, gradient 4.512e-04, dot -0.276; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.415721494932346, -2.22009267674158, -0.18557194670829533, -0.10353436569302185
LBFGS Accumulation History: 3 points
Removed measurement 29c0ebd0 to history. Total: 4
Removed measurement 5b172bb0 to history. Total: 3
th(0)=-3.415721494932346;dx=-2.027935777850083E-7
New Minimum: -3.415721494932346 > -3.415722244732614
END: th(1.7361111111111114)=-3.415722244732614; dx=-1.2671010391468492E-7 evalInputDelta=7.498002680961235E-7
Fitness changed from -3.415721494932346 to -3.415722244732614
Iteration 12 complete. Error: -3.415722244732614 Total: 3.0734; Orientation: 0.6032; Line Search: 1.6520
Adding measurement b1194a7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.685e+04, gradient 4.526e-04, dot -0.084; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.415722244732614, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement b1194a7 to history. Total: 3
th(0)=-3.415722244732614;dx=-2.0443314483642307E-7
New Minimum: -3.415722244732614 > -3.415722619632686
END: th(3.740338003527577)=-3.415722619632686; dx=-1.2752852447341778E-7 evalInputDelta=3.749000723196616E-7
Fitness changed from -3.415722244732614 to -3.415722619632686
Iteration 13 complete. Error: -3.415722619632686 Total: 2.9377; Orientation: 0.3017; Line Search: 1.7737
Adding measurement 3939601 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.684e+04, gradient 4.526e-04, dot -0.084; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.415722619632686, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 3939601 to history. Total: 3
th(0)=-3.415722619632686;dx=-2.0479436724214976E-7
New Minimum: -3.415722619632686 > -3.4157246190990436
END: th(8.058313947244411)=-3.4157246190990436; dx=-1.273044466883149E-7 evalInputDelta=1.999466357371915E-6
Fitness changed from -3.415722619632686 to -3.4157246190990436
Iteration 14 complete. Error: -3.4157246190990436 Total: 2.9807; Orientation: 0.3282; Line Search: 1.7345
Adding measurement 1d6c86ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.685e+04, gradient 4.526e-04, dot -0.084; [a0ffb363-1df6-46a5-9b3e-019abd9110b9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.4157246190990436, -0.18557194670829533, -0.10353436569302185, -0.0418072434971343
LBFGS Accumulation History: 3 points
Removed measurement 1d6c86ea to history. Total: 3
th(0)=-3.4157246190990436;dx=-2.0372418957442373E-7
New Minimum: -3.4157246190990436 > -3.4157281181623533
END: th(17.361111111111118)=-3.4157281181623533; dx=-1.2713051198987801E-7 evalInputDelta=3.4990633097642387E-6
Fitness changed from -3.4157246190990436 to -3.4157281181623533
Iteration 15 complete. Error: -3.4157281181623533 Total: 3.0882; Orientation: 0.2764; Line Search: 1.7116
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -3.4157281181623533 (> -Infinity) after 65.897s (< 5400.000s)
Returns
-3.4157281181623533