BasicOptimizer.scala:89 executed in 66.00 seconds (0.842 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 166653489449100
Reset training subject: 166654402064200
Adding measurement 6315141e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-132.78653423360882;dx=-0.038875679238527805
New Minimum: -132.78653423360882 > -132.8615718233686
WOLFE (weak): th(2.154434690031884)=-132.8615718233686; dx=-0.038704067824284566 evalInputDelta=0.07503758975977348
New Minimum: -132.8615718233686 > -132.93664009978696
WOLFE (weak): th(4.308869380063768)=-132.93664009978696; dx=-0.038676746463517125 evalInputDelta=0.15010586617813715
New Minimum: -132.93664009978696 > -133.2366279140137
WOLFE (weak): th(12.926608140191302)=-133.2366279140137; dx=-0.03868483003250659 evalInputDelta=0.4500936804048763
New Minimum: -133.2366279140137 > -134.5871243924975
WOLFE (weak): th(51.70643256076521)=-134.5871243924975; dx=-0.03867181667924929 evalInputDelta=1.8005901588886672
New Minimum: -134.5871243924975 > -141.85403719250075
WOLFE (weak): th(258.53216280382605)=-141.85403719250075; dx=-0.039010256316140614 evalInputDelta=9.067502958891936
New Minimum: -141.85403719250075 > -189.58064115379224
WOLFE (weak): th(1551.1929768229563)=-189.58064115379224; dx=-0.03881205669060803 evalInputDelta=56.794106920183424
New Minimum: -189.58064115379224 > -405.0208222814305
END: th(10858.350837760694)=-405.0208222814305; dx=-0.013054082218753447 evalInputDelta=272.2342880478217
Fitness changed from -132.78653423360882 to -405.0208222814305
Iteration 1 complete. Error: -405.0208222814305 Total: 9.4174; Orientation: 0.0184; Line Search: 6.8938
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 604f949 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-405.0208222814305;dx=-0.044352005276000334
New Minimum: -405.0208222814305 > -858.0436688952831
END: th(23393.607721408407)=-858.0436688952831; dx=-0.007181799670397137 evalInputDelta=453.02284661385255
Fitness changed from -405.0208222814305 to -858.0436688952831
Iteration 2 complete. Error: -858.0436688952831 Total: 2.6212; Orientation: 0.0145; Line Search: 1.6913
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 465b1112 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-858.0436688952831;dx=-0.04331149030453633
New Minimum: -858.0436688952831 > -1197.5065684602457
WOLF (strong): th(50400.00000000001)=-1197.5065684602457; dx=0.0014475141478011992 evalInputDelta=339.46289956496264
New Minimum: -1197.5065684602457 > -1324.7856621988228
END: th(25200.000000000004)=-1324.7856621988228; dx=-0.006340569037743786 evalInputDelta=466.74199330353974
Fitness changed from -858.0436688952831 to -1324.7856621988228
Iteration 3 complete. Error: -1324.7856621988228 Total: 3.6104; Orientation: 0.0155; Line Search: 2.7198
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5776a31b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.424e+04, gradient 2.060e-01, dot -0.328; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1324.7856621988228, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 5776a31b to history. Total: 3
th(0)=-1324.7856621988228;dx=-0.042382650922424575
New Minimum: -1324.7856621988228 > -1574.468828627354
WOLF (strong): th(54291.75418880348)=-1574.468828627354; dx=0.0018398483561026906 evalInputDelta=249.6831664285312
New Minimum: -1574.468828627354 > -1760.7286216239459
END: th(27145.87709440174)=-1760.7286216239459; dx=-0.004934023665669899 evalInputDelta=435.94295942512304
Fitness changed from -1324.7856621988228 to -1760.7286216239459
Iteration 4 complete. Error: -1760.7286216239459 Total: 3.7655; Orientation: 0.2876; Line Search: 2.5634
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3645b595 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.922e+04, gradient 2.026e-01, dot -0.307; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1760.7286216239459, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 3645b595 to history. Total: 3
th(0)=-1760.7286216239459;dx=-0.041003763007699694
New Minimum: -1760.7286216239459 > -1841.0808754108832
WOLF (strong): th(58484.01930352103)=-1841.0808754108832; dx=0.004377409103881893 evalInputDelta=80.35225378693735
New Minimum: -1841.0808754108832 > -2160.760749205596
END: th(29242.009651760516)=-2160.760749205596; dx=-0.0026785406938400758 evalInputDelta=400.03212758165
Fitness changed from -1760.7286216239459 to -2160.760749205596
Iteration 5 complete. Error: -2160.760749205596 Total: 3.7415; Orientation: 0.2835; Line Search: 2.5181
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2218386b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.163e+04, gradient 2.041e-01, dot -0.328; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2160.760749205596, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 2218386b to history. Total: 3
th(0)=-2160.760749205596;dx=-0.041392955316220066
New Minimum: -2160.760749205596 > -2163.6321872174817
WOLF (strong): th(63000.00000000002)=-2163.6321872174817; dx=0.005027018248822995 evalInputDelta=2.871438011885857
New Minimum: -2163.6321872174817 > -2533.6153243359167
END: th(31500.00000000001)=-2533.6153243359167; dx=-5.74053507495129E-4 evalInputDelta=372.8545751303209
Fitness changed from -2160.760749205596 to -2533.6153243359167
Iteration 6 complete. Error: -2533.6153243359167 Total: 3.7914; Orientation: 0.2632; Line Search: 2.5292
Adding measurement 5fcf68a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.225e+04, gradient 1.975e-01, dot -0.320; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2533.6153243359167, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 5fcf68a to history. Total: 3
th(0)=-2533.6153243359167;dx=-0.03895813462034126
Armijo: th(67864.69273600436)=-2342.7674413634286; dx=0.007821509537268982 evalInputDelta=-190.84788297248815
New Minimum: -2533.6153243359167 > -2891.339504949353
WOLF (strong): th(33932.34636800218)=-2891.339504949353; dx=1.7445146990916318E-4 evalInputDelta=357.72418061343615
END: th(11310.782122667393)=-2839.3361192280154; dx=-0.018548413632681844 evalInputDelta=305.72079489209864
Fitness changed from -2533.6153243359167 to -2891.339504949353
Iteration 7 complete. Error: -2891.339504949353 Total: 4.4746; Orientation: 0.2730; Line Search: 3.3850
Adding measurement 55925814 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.070e+04, gradient 1.952e-01, dot -0.355; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2891.339504949353, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 55925814 to history. Total: 3
th(0)=-2891.339504949353;dx=-0.037953573464319365
New Minimum: -2891.339504949353 > -3280.7020275592877
END: th(24368.3413764671)=-3280.7020275592877; dx=-0.005172573541300334 evalInputDelta=389.3625226099348
Fitness changed from -2891.339504949353 to -3280.7020275592877
Iteration 8 complete. Error: -3280.7020275592877 Total: 2.9203; Orientation: 0.2725; Line Search: 1.8336
Adding measurement 2be8beb1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.490e+05, gradient 1.671e-01, dot -0.410; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3280.7020275592877, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 2be8beb1 to history. Total: 3
th(0)=-3280.7020275592877;dx=-0.027907244033626488
New Minimum: -3280.7020275592877 > -3411.6831109387404
WOLF (strong): th(52500.00000000002)=-3411.6831109387404; dx=5.185073953910322E-4 evalInputDelta=130.98108337945268
New Minimum: -3411.6831109387404 > -3531.765733825059
END: th(26250.00000000001)=-3531.765733825059; dx=-0.006743491208065335 evalInputDelta=251.06370626577154
Fitness changed from -3280.7020275592877 to -3531.765733825059
Iteration 9 complete. Error: -3531.765733825059 Total: 3.6428; Orientation: 0.2636; Line Search: 2.5530
Adding measurement 77fcde2f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.620e+05, gradient 1.599e-01, dot -0.460; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3531.765733825059, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 77fcde2f to history. Total: 3
th(0)=-3531.765733825059;dx=-0.025405295886226458
New Minimum: -3531.765733825059 > -3665.106406168927
END: th(56553.91061333697)=-3665.106406168927; dx=-9.857789614214898E-4 evalInputDelta=133.34067234386794
Fitness changed from -3531.765733825059 to -3665.106406168927
Iteration 10 complete. Error: -3665.106406168927 Total: 2.6280; Orientation: 0.2687; Line Search: 1.5957
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 38175650 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.820e+05, gradient 2.036e-01, dot -0.323; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3665.106406168927, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 38175650 to history. Total: 3
th(0)=-3665.106406168927;dx=-0.04133739137039773
Armijo: th(121841.70688233551)=-2367.1772512745065; dx=0.008131049698595544 evalInputDelta=-1297.9291548944207
Armijo: th(60920.853441167754)=-3547.911616668369; dx=0.009677385819158502 evalInputDelta=-117.19478950055827
New Minimum: -3665.106406168927 > -4051.534565487898
END: th(20306.95114705592)=-4051.534565487898; dx=-0.008200957376178255 evalInputDelta=386.42815931897076
Fitness changed from -3665.106406168927 to -4051.534565487898
Iteration 11 complete. Error: -4051.534565487898 Total: 4.8161; Orientation: 0.2658; Line Search: 3.6108
Adding measurement 5d17b6d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.703e+08, gradient 1.551e-01, dot -0.459; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4051.534565487898, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 5d17b6d2 to history. Total: 3
th(0)=-4051.534565487898;dx=-0.024011189199288095
New Minimum: -4051.534565487898 > -4275.541102638661
END: th(43750.00000000003)=-4275.541102638661; dx=-0.005114426294332967 evalInputDelta=224.00653715076305
Fitness changed from -4051.534565487898 to -4275.541102638661
Iteration 12 complete. Error: -4275.541102638661 Total: 2.7888; Orientation: 0.2703; Line Search: 1.7105
Adding measurement 5d6bf5a6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.170e+05, gradient 1.656e-01, dot -0.403; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4275.541102638661, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 5d6bf5a6 to history. Total: 3
th(0)=-4275.541102638661;dx=-0.027381429172738392
Armijo: th(94256.51768889498)=-3997.3029462486465; dx=0.0031686521403084646 evalInputDelta=-278.2381563900144
New Minimum: -4275.541102638661 > -4406.502339012033
END: th(47128.25884444749)=-4406.502339012033; dx=-0.0011367088779767006 evalInputDelta=130.96123637337223
Fitness changed from -4275.541102638661 to -4406.502339012033
Iteration 13 complete. Error: -4406.502339012033 Total: 3.5271; Orientation: 0.2697; Line Search: 2.4605
Adding measurement 1763c57a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.126e+05, gradient 1.786e-01, dot -0.349; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4406.502339012033, -858.0436688952831, -405.0208222814305, -132.78653423360882
LBFGS Accumulation History: 3 points
Removed measurement 1763c57a to history. Total: 3
th(0)=-4406.502339012033;dx=-0.03183164418722029
Armijo: th(101534.75573527961)=-3706.5235201737473; dx=0.007957655569752654 evalInputDelta=-699.9788188382859
New Minimum: -4406.502339012033 > -4510.764027870675
WOLF (strong): th(50767.37786763981)=-4510.764027870675; dx=0.004087251238005763 evalInputDelta=104.26168885864172
New Minimum: -4510.764027870675 > -4651.673405108656
END: th(16922.459289213268)=-4651.673405108656; dx=-0.009641641577920971 evalInputDelta=245.17106609662278
Fitness changed from -4406.502339012033 to -4651.673405108656
Iteration 14 complete. Error: -4651.673405108656 Total: 4.5933; Orientation: 0.2667; Line Search: 3.5099
Adding measurement 1dfc5336 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.468e+04, gradient 1.488e-01, dot 0.558; [89e6d420-c9f0-412b-a1a7-82106ae1ea78 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-4651.673405108656;dx=-5371.292542945463
Armijo: th(36458.33333333336)=-2438.6337169432745; dx=0.037631901126293034 evalInputDelta=-2213.0396881653814
Armijo: th(18229.16666666668)=-2438.668826844277; dx=0.07524803592506676 evalInputDelta=-2213.004578264379
Armijo: th(6076.388888888893)=-2438.5303096851303; dx=0.2254860976492226 evalInputDelta=-2213.1430954235257
Armijo: th(1519.0972222222233)=-2440.458875190786; dx=0.902923352509083 evalInputDelta=-2211.21452991787
Armijo: th(303.8194444444447)=-2451.579127596277; dx=4.511918141286828 evalInputDelta=-2200.094277512379
Armijo: th(50.63657407407411)=-2500.780963800764; dx=27.031055218867618 evalInputDelta=-2150.892441307892
Armijo: th(7.233796296296302)=-2840.183160008247; dx=182.75129600873578 evalInputDelta=-1811.490245100409
Armijo: th(0.9042245370370378)=-4239.247085802152; dx=578.6914737850085 evalInputDelta=-412.4263193065044
New Minimum: -4651.673405108656 > -4811.89502062816
END: th(0.10046939300411531)=-4811.89502062816; dx=-1631.6004868566697 evalInputDelta=160.22161551950376
Fitness changed from -4651.673405108656 to -4811.89502062816
Iteration 15 complete. Error: -4811.89502062816 Total: 9.5609; Orientation: 0.2630; Line Search: 8.5228
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -4811.89502062816 (> -Infinity) after 65.996s (< 5400.000s)
Returns
-4811.89502062816