BasicOptimizer.scala:89 executed in 13.68 seconds (0.858 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 564416297041900
Reset training subject: 564416744344100
Adding measurement 5773d271 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.7109171247307253;dx=-7.807795442707102E-6
New Minimum: -1.7109171247307253 > -1.7109595020215516
WOLFE (weak): th(2.154434690031884)=-1.7109595020215516; dx=-7.808806991097268E-6 evalInputDelta=4.2377290826234315E-5
New Minimum: -1.7109595020215516 > -1.711001959132483
WOLFE (weak): th(4.308869380063768)=-1.711001959132483; dx=-7.81755140828033E-6 evalInputDelta=8.483440175766077E-5
New Minimum: -1.711001959132483 > -1.711171857903527
WOLFE (weak): th(12.926608140191302)=-1.711171857903527; dx=-7.821472164309267E-6 evalInputDelta=2.5473317280177987E-4
New Minimum: -1.711171857903527 > -1.7119363149520874
WOLFE (weak): th(51.70643256076521)=-1.7119363149520874; dx=-7.751959906204247E-6 evalInputDelta=0.0010191902213620985
New Minimum: -1.7119363149520874 > -1.7160187950550367
WOLFE (weak): th(258.53216280382605)=-1.7160187950550367; dx=-7.8308727848052E-6 evalInputDelta=0.005101670324311414
New Minimum: -1.7160187950550367 > -1.7416666195920434
WOLFE (weak): th(1551.1929768229563)=-1.7416666195920434; dx=-7.903486525478864E-6 evalInputDelta=0.03074949486131806
New Minimum: -1.7416666195920434 > -1.9331216890870555
WOLFE (weak): th(10858.350837760694)=-1.9331216890870555; dx=-8.40817561557138E-6 evalInputDelta=0.22220456435633018
New Minimum: -1.9331216890870555 > -3.5376381165488566
WOLFE (weak): th(86866.80670208555)=-3.5376381165488566; dx=-7.082544582437511E-6 evalInputDelta=1.8267209918181313
New Minimum: -3.5376381165488566 > -5.070008911355466
END: th(781801.26031877)=-5.070008911355466; dx=-4.2263650628906864E-8 evalInputDelta=3.3590917866247407
Fitness changed from -1.7109171247307253 to -5.070008911355466
Iteration 1 complete. Error: -5.070008911355466 Total: 3.0023; Orientation: 0.0303; Line Search: 2.0790
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 10f397d0 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-5.070008911355466;dx=-1.1630138187561061E-5
New Minimum: -5.070008911355466 > -10.645087885520852
WOLF (strong): th(1684339.7559414052)=-10.645087885520852; dx=2.526517923427327E-7 evalInputDelta=5.575078974165385
New Minimum: -10.645087885520852 > -11.877540632881308
WOLF (strong): th(842169.8779707026)=-11.877540632881308; dx=3.9754990422276043E-7 evalInputDelta=6.807531721525842
END: th(280723.2926569009)=-10.792273987997262; dx=-3.623284436329503E-6 evalInputDelta=5.722265076641796
Fitness changed from -5.070008911355466 to -11.877540632881308
Iteration 2 complete. Error: -11.877540632881308 Total: 0.9168; Orientation: 0.0027; Line Search: 0.7219
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7f9e6167 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-11.877540632881308;dx=-8.960341588269317E-6
New Minimum: -11.877540632881308 > -19.93379382328393
END: th(604800.0000000001)=-19.93379382328393; dx=-1.761488668940494E-6 evalInputDelta=8.056253190402623
Fitness changed from -11.877540632881308 to -19.93379382328393
Iteration 3 complete. Error: -19.93379382328393 Total: 0.5596; Orientation: 0.0021; Line Search: 0.3597
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 528c4353 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.612e+03, gradient 2.590e-03, dot -0.589; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.93379382328393, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 528c4353 to history. Total: 3
th(0)=-19.93379382328393;dx=-6.661642640278485E-6
New Minimum: -19.93379382328393 > -27.318243633052703
END: th(1303002.1005312835)=-27.318243633052703; dx=-1.3841586268531743E-7 evalInputDelta=7.384449809768771
Fitness changed from -19.93379382328393 to -27.318243633052703
Iteration 4 complete. Error: -27.318243633052703 Total: 0.6389; Orientation: 0.1025; Line Search: 0.3426
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 22ba30b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.462e+04, gradient 3.231e-03, dot -0.467; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.318243633052703, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 22ba30b0 to history. Total: 3
th(0)=-27.318243633052703;dx=-1.0379729196468456E-5
New Minimum: -27.318243633052703 > -29.032392434491733
WOLF (strong): th(2807232.926569009)=-29.032392434491733; dx=1.917990333979276E-6 evalInputDelta=1.7141488014390305
New Minimum: -29.032392434491733 > -36.87004483889603
WOLF (strong): th(1403616.4632845046)=-36.87004483889603; dx=1.7335872916002113E-6 evalInputDelta=9.551801205843326
END: th(467872.1544281682)=-36.045259397762344; dx=-4.562655472521336E-6 evalInputDelta=8.727015764709641
Fitness changed from -27.318243633052703 to -36.87004483889603
Iteration 5 complete. Error: -36.87004483889603 Total: 0.8979; Orientation: 0.0483; Line Search: 0.6699
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 38224635 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.468e+04, gradient 3.248e-03, dot -0.409; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.87004483889603, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 38224635 to history. Total: 3
th(0)=-36.87004483889603;dx=-1.0515299311365144E-5
New Minimum: -36.87004483889603 > -50.77948732552934
END: th(1008000.0000000002)=-50.77948732552934; dx=-7.237181956250286E-7 evalInputDelta=13.90944248663331
Fitness changed from -36.87004483889603 to -50.77948732552934
Iteration 6 complete. Error: -50.77948732552934 Total: 0.6128; Orientation: 0.0463; Line Search: 0.3743
Adding measurement 194a653d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.726e+04, gradient 2.373e-03, dot -0.533; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -50.77948732552934, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 194a653d to history. Total: 3
th(0)=-50.77948732552934;dx=-5.6168206729503055E-6
New Minimum: -50.77948732552934 > -58.42706708328407
WOLF (strong): th(2171670.1675521396)=-58.42706708328407; dx=9.762206464054518E-7 evalInputDelta=7.647579757754734
New Minimum: -58.42706708328407 > -58.740779146034505
END: th(1085835.0837760698)=-58.740779146034505; dx=-8.495632110303375E-7 evalInputDelta=7.961291820505167
Fitness changed from -50.77948732552934 to -58.740779146034505
Iteration 7 complete. Error: -58.740779146034505 Total: 1.2069; Orientation: 0.0530; Line Search: 0.9727
Adding measurement 16576a6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.597e+04, gradient 2.237e-03, dot -0.512; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -58.740779146034505, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 16576a6 to history. Total: 3
th(0)=-58.740779146034505;dx=-4.986323802980842E-6
New Minimum: -58.740779146034505 > -63.18064446265581
WOLF (strong): th(2339360.7721408415)=-63.18064446265581; dx=1.3074177013558725E-6 evalInputDelta=4.439865316621308
New Minimum: -63.18064446265581 > -65.03980307117818
WOLF (strong): th(1169680.3860704207)=-65.03980307117818; dx=4.5082657868857863E-8 evalInputDelta=6.299023925143679
END: th(389893.4620234736)=-62.388371270851266; dx=-3.169757066959681E-6 evalInputDelta=3.647592124816761
Fitness changed from -58.740779146034505 to -65.03980307117818
Iteration 8 complete. Error: -65.03980307117818 Total: 0.8544; Orientation: 0.0432; Line Search: 0.6529
Adding measurement 181909f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.442e+04, gradient 2.318e-03, dot -0.446; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -65.03980307117818, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 181909f0 to history. Total: 3
th(0)=-65.03980307117818;dx=-5.348483926871019E-6
New Minimum: -65.03980307117818 > -70.3383962619093
END: th(840000.0000000003)=-70.3383962619093; dx=-3.053021195371508E-7 evalInputDelta=5.298593190731111
Fitness changed from -65.03980307117818 to -70.3383962619093
Iteration 9 complete. Error: -70.3383962619093 Total: 0.5238; Orientation: 0.0429; Line Search: 0.3171
Adding measurement 71c45428 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.216e+04, gradient 1.844e-03, dot -0.507; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.3383962619093, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 71c45428 to history. Total: 3
th(0)=-70.3383962619093;dx=-3.390957696681704E-6
New Minimum: -70.3383962619093 > -74.45023705948313
WOLF (strong): th(1809725.139626783)=-74.45023705948313; dx=5.712990403896776E-7 evalInputDelta=4.111840797573834
New Minimum: -74.45023705948313 > -74.48928680884103
END: th(904862.5698133915)=-74.48928680884103; dx=-6.706909830519127E-7 evalInputDelta=4.150890546931734
Fitness changed from -70.3383962619093 to -74.48928680884103
Iteration 10 complete. Error: -74.48928680884103 Total: 0.6826; Orientation: 0.0407; Line Search: 0.4868
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 59464f08 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.224e+04, gradient 1.779e-03, dot -0.498; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -74.48928680884103, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 59464f08 to history. Total: 3
th(0)=-74.48928680884103;dx=-3.1446357861707707E-6
New Minimum: -74.48928680884103 > -77.9377721411525
WOLF (strong): th(1949467.3101173681)=-77.9377721411525; dx=6.187174974462838E-7 evalInputDelta=3.4484853323114635
New Minimum: -77.9377721411525 > -78.21170698368769
END: th(974733.6550586841)=-78.21170698368769; dx=-3.898794167140434E-7 evalInputDelta=3.722420174846661
Fitness changed from -74.48928680884103 to -78.21170698368769
Iteration 11 complete. Error: -78.21170698368769 Total: 0.7094; Orientation: 0.0456; Line Search: 0.4833
Adding measurement 19bd1f98 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.018e+04, gradient 1.764e-03, dot -0.462; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -78.21170698368769, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 19bd1f98 to history. Total: 3
th(0)=-78.21170698368769;dx=-3.0941221605649266E-6
New Minimum: -78.21170698368769 > -80.48433420525089
WOLF (strong): th(2100000.000000001)=-80.48433420525089; dx=9.792155336253371E-7 evalInputDelta=2.272627221563198
New Minimum: -80.48433420525089 > -81.71610293477433
WOLF (strong): th(1050000.0000000005)=-81.71610293477433; dx=4.578879035688486E-8 evalInputDelta=3.504395951086636
END: th(350000.0000000002)=-80.20025882962284; dx=-2.0055325393465264E-6 evalInputDelta=1.9885518459351488
Fitness changed from -78.21170698368769 to -81.71610293477433
Iteration 12 complete. Error: -81.71610293477433 Total: 0.8604; Orientation: 0.0416; Line Search: 0.6521
Adding measurement 9478683 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.727e+04, gradient 1.766e-03, dot -0.413; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -81.71610293477433, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 9478683 to history. Total: 3
th(0)=-81.71610293477433;dx=-3.0945694432840984E-6
New Minimum: -81.71610293477433 > -84.67937440858921
END: th(754052.1415111597)=-84.67937440858921; dx=-4.099538542134274E-7 evalInputDelta=2.963271473814885
Fitness changed from -81.71610293477433 to -84.67937440858921
Iteration 13 complete. Error: -84.67937440858921 Total: 0.7789; Orientation: 0.0411; Line Search: 0.5785
Adding measurement 2f75c3f3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.644e+04, gradient 1.387e-03, dot -0.486; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.67937440858921, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 2f75c3f3 to history. Total: 3
th(0)=-84.67937440858921;dx=-1.902227990716761E-6
New Minimum: -84.67937440858921 > -87.74808038231194
END: th(1624556.0917644736)=-87.74808038231194; dx=-1.2832155927089623E-8 evalInputDelta=3.0687059737227287
Fitness changed from -84.67937440858921 to -87.74808038231194
Iteration 14 complete. Error: -87.74808038231194 Total: 0.5531; Orientation: 0.0431; Line Search: 0.3334
Adding measurement 3990c36c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.577e+04, gradient 1.818e-03, dot -0.356; [d76ab561-b26d-4c7c-a36e-ec5f53ca9793 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -87.74808038231194, -11.877540632881308, -5.070008911355466, -1.7109171247307253
LBFGS Accumulation History: 3 points
Removed measurement 3990c36c to history. Total: 3
th(0)=-87.74808038231194;dx=-3.288484501100947E-6
Armijo: th(3500000.0000000023)=-81.15590622324225; dx=1.9862862208762657E-6 evalInputDelta=-6.592174159069685
New Minimum: -87.74808038231194 > -88.99489077048125
WOLF (strong): th(1750000.0000000012)=-88.99489077048125; dx=1.5594789016227158E-6 evalInputDelta=1.2468103881693082
New Minimum: -88.99489077048125 > -90.56117119046404
END: th(583333.3333333337)=-90.56117119046404; dx=-1.020319470136049E-6 evalInputDelta=2.813090808152097
Fitness changed from -87.74808038231194 to -90.56117119046404
Iteration 15 complete. Error: -90.56117119046404 Total: 0.8445; Orientation: 0.0411; Line Search: 0.6479
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -90.56117119046404 (> -Infinity) after 13.662s (< 5400.000s)
Returns
-90.56117119046404