BasicOptimizer.scala:88 executed in 696.08 seconds (12.503 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1522468919431
Reset training subject: 1529255001355
Adding measurement 3d8776de to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=32.818611800670624;dx=-1.6175803800100481E-4
New Minimum: 32.818611800670624 > 32.81843423843384
WOLFE (weak): th(2.154434690031884)=32.81843423843384; dx=-1.6163895162483614E-4 evalInputDelta=1.7756223678588867E-4
New Minimum: 32.81843423843384 > 32.818258225917816
WOLFE (weak): th(4.308869380063768)=32.818258225917816; dx=-1.6163499931436164E-4 evalInputDelta=3.535747528076172E-4
New Minimum: 32.818258225917816 > 32.81755155324936
WOLFE (weak): th(12.926608140191302)=32.81755155324936; dx=-1.6153034554177738E-4 evalInputDelta=0.0010602474212646484
New Minimum: 32.81755155324936 > 32.81437569856644
WOLFE (weak): th(51.70643256076521)=32.81437569856644; dx=-1.616842102191505E-4 evalInputDelta=0.004236102104187012
New Minimum: 32.81437569856644 > 32.797401547431946
WOLFE (weak): th(258.53216280382605)=32.797401547431946; dx=-1.616125797808369E-4 evalInputDelta=0.02121025323867798
New Minimum: 32.797401547431946 > 32.69252300262451
WOLFE (weak): th(1551.1929768229563)=32.69252300262451; dx=-1.5745791394537392E-4 evalInputDelta=0.12608879804611206
New Minimum: 32.69252300262451 > 32.01976352930069
END: th(10858.350837760694)=32.01976352930069; dx=-1.2254693626919175E-4 evalInputDelta=0.7988482713699341
Fitness changed from 32.818611800670624 to 32.01976352930069
Iteration 1 complete. Error: 32.01976352930069 Total: 76.1353; Orientation: 0.1069; Line Search: 56.0305
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 44694fdc to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=32.01976352930069;dx=-1.1158537465966482E-4
New Minimum: 32.01976352930069 > 30.958445012569427
END: th(23393.607721408407)=30.958445012569427; dx=-7.174150935569666E-5 evalInputDelta=1.0613185167312622
Fitness changed from 32.01976352930069 to 30.958445012569427
Iteration 2 complete. Error: 30.958445012569427 Total: 20.9622; Orientation: 0.0382; Line Search: 14.4509
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 32fbdb56 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=30.958445012569427;dx=-8.476044414497229E-5
New Minimum: 30.958445012569427 > 29.709693372249603
END: th(50400.00000000001)=29.709693372249603; dx=-3.3646980313419514E-5 evalInputDelta=1.2487516403198242
Fitness changed from 30.958445012569427 to 29.709693372249603
Iteration 3 complete. Error: 29.709693372249603 Total: 21.0382; Orientation: 0.0376; Line Search: 14.3994
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3c56f506 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.133e+03, gradient 9.143e-03, dot -0.596; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 29.709693372249603, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 3c56f506 to history. Total: 3
th(0)=29.709693372249603;dx=-8.359638495651864E-5
New Minimum: 29.709693372249603 > 28.297121584415436
END: th(108583.50837760697)=28.297121584415436; dx=-1.9254179104696527E-5 evalInputDelta=1.4125717878341675
Fitness changed from 29.709693372249603 to 28.297121584415436
Iteration 4 complete. Error: 28.297121584415436 Total: 21.7864; Orientation: 0.9330; Line Search: 13.8604
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 53796f65 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.257e+03, gradient 6.036e-03, dot -0.536; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 28.297121584415436, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 53796f65 to history. Total: 3
th(0)=28.297121584415436;dx=-3.643063732084767E-5
New Minimum: 28.297121584415436 > 26.62329253554344
END: th(233936.07721408413)=26.62329253554344; dx=-8.882218536860172E-6 evalInputDelta=1.673829048871994
Fitness changed from 28.297121584415436 to 26.62329253554344
Iteration 5 complete. Error: 26.62329253554344 Total: 21.8522; Orientation: 1.0021; Line Search: 14.2738
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 746a6374 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.193e+03, gradient 5.345e-03, dot -0.414; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 26.62329253554344, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 746a6374 to history. Total: 3
th(0)=26.62329253554344;dx=-2.857126584022518E-5
New Minimum: 26.62329253554344 > 25.00369292497635
END: th(504000.0000000002)=25.00369292497635; dx=-2.6814244092332955E-6 evalInputDelta=1.619599610567093
Fitness changed from 26.62329253554344 to 25.00369292497635
Iteration 6 complete. Error: 25.00369292497635 Total: 21.5916; Orientation: 0.9625; Line Search: 14.0710
Adding measurement 7e2378a3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.389e+03, gradient 5.513e-03, dot -0.266; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 25.00369292497635, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 7e2378a3 to history. Total: 3
th(0)=25.00369292497635;dx=-3.0396915650442866E-5
New Minimum: 25.00369292497635 > 23.766172498464584
WOLF (strong): th(1085835.0837760698)=23.766172498464584; dx=8.963845658859732E-7 evalInputDelta=1.2375204265117645
New Minimum: 23.766172498464584 > 23.607106864452362
END: th(542917.5418880349)=23.607106864452362; dx=-1.0597170736619302E-6 evalInputDelta=1.3965860605239868
Fitness changed from 25.00369292497635 to 23.607106864452362
Iteration 7 complete. Error: 23.607106864452362 Total: 28.6911; Orientation: 0.9225; Line Search: 20.6776
Adding measurement 16769e3e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.484e+03, gradient 4.222e-03, dot -0.248; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 23.607106864452362, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 16769e3e to history. Total: 3
th(0)=23.607106864452362;dx=-1.7827606843642703E-5
New Minimum: 23.607106864452362 > 22.630328476428986
WOLF (strong): th(1169680.3860704207)=22.630328476428986; dx=7.36144965628167E-7 evalInputDelta=0.9767783880233765
New Minimum: 22.630328476428986 > 22.49492508172989
END: th(584840.1930352104)=22.49492508172989; dx=-1.0153276268619203E-6 evalInputDelta=1.1121817827224731
Fitness changed from 23.607106864452362 to 22.49492508172989
Iteration 8 complete. Error: 22.49492508172989 Total: 30.3738; Orientation: 0.9114; Line Search: 22.1681
Adding measurement 7024a65c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.445e+03, gradient 3.461e-03, dot -0.244; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 22.49492508172989, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 7024a65c to history. Total: 3
th(0)=22.49492508172989;dx=-1.1977233211842975E-5
New Minimum: 22.49492508172989 > 22.024592339992523
WOLF (strong): th(1260000.0000000005)=22.024592339992523; dx=8.413177415050861E-7 evalInputDelta=0.4703327417373657
New Minimum: 22.024592339992523 > 21.70474398136139
END: th(630000.0000000002)=21.70474398136139; dx=-3.6746587551973154E-7 evalInputDelta=0.7901811003684998
Fitness changed from 22.49492508172989 to 21.70474398136139
Iteration 9 complete. Error: 21.70474398136139 Total: 29.0006; Orien
...skipping 4506 bytes...
w Minimum: 19.757584542036057 > 19.66109949350357
END: th(203069.51147055917)=19.66109949350357; dx=-8.972686243378755E-7 evalInputDelta=0.22324874997138977
Fitness changed from 19.88434824347496 to 19.66109949350357
Iteration 14 complete. Error: 19.66109949350357 Total: 35.6036; Orientation: 0.9106; Line Search: 27.9517
Adding measurement 69b4bfbd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.866e+03, gradient 1.752e-03, dot -0.324; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 19.66109949350357, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 69b4bfbd to history. Total: 3
th(0)=19.66109949350357;dx=-3.0683543023867855E-6
New Minimum: 19.66109949350357 > 19.376048922538757
END: th(437500.00000000023)=19.376048922538757; dx=-6.710501322531563E-7 evalInputDelta=0.28505057096481323
Fitness changed from 19.66109949350357 to 19.376048922538757
Iteration 15 complete. Error: 19.376048922538757 Total: 21.4563; Orientation: 0.8933; Line Search: 13.4332
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 7016ef29 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.836e+03, gradient 1.781e-03, dot -0.304; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 19.376048922538757, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 7016ef29 to history. Total: 3
th(0)=19.376048922538757;dx=-3.171532270582812E-6
New Minimum: 19.376048922538757 > 19.353221505880356
WOLF (strong): th(942565.1768889497)=19.353221505880356; dx=6.321524021971525E-7 evalInputDelta=0.02282741665840149
New Minimum: 19.353221505880356 > 19.187804698944092
WOLF (strong): th(471282.58844447485)=19.187804698944092; dx=8.010485014719347E-6 evalInputDelta=0.18824422359466553
END: th(157094.19614815828)=19.235532730817795; dx=-1.3980134686046199E-6 evalInputDelta=0.14051619172096252
Fitness changed from 19.376048922538757 to 19.187804698944092
Iteration 16 complete. Error: 19.187804698944092 Total: 36.2651; Orientation: 0.8970; Line Search: 27.6928
Adding measurement 225150a9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.812e+06, gradient 4.021e-01, dot -0.008; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 19.187804698944092, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 225150a9 to history. Total: 3
th(0)=19.187804698944092;dx=-0.1616574301475993
Armijo: th(338449.18578426534)=37.052318051457405; dx=1.2872455086522473E-4 evalInputDelta=-17.864513352513313
Armijo: th(169224.59289213267)=32.17830926179886; dx=2.0100802782823638E-4 evalInputDelta=-12.990504562854767
Armijo: th(56408.19763071089)=24.432296752929688; dx=2.971437546025885E-4 evalInputDelta=-5.244492053985596
Armijo: th(14102.049407677723)=19.879771798849106; dx=1.3800794087177762E-4 evalInputDelta=-0.691967099905014
Armijo: th(2820.4098815355446)=19.279484897851944; dx=7.429027623079733E-5 evalInputDelta=-0.09168019890785217
Armijo: th(470.06831358925746)=19.19528478384018; dx=4.180178803210956E-5 evalInputDelta=-0.0074800848960876465
Armijo: th(67.15261622703677)=19.188689053058624; dx=4.5151538993434E-5 evalInputDelta=-8.843541145324707E-4
New Minimum: 19.187804698944092 > 19.187410950660706
WOLF (strong): th(8.394077028379597)=19.187410950660706; dx=3.5538355113821836E-5 evalInputDelta=3.9374828338623047E-4
New Minimum: 19.187410950660706 > 19.187295228242874
WOLF (strong): th(0.9326752253755107)=19.187295228242874; dx=3.165698061601829E-5 evalInputDelta=5.094707012176514E-4
New Minimum: 19.187295228242874 > 19.18728333711624
WOLF (strong): th(0.09326752253755108)=19.18728333711624; dx=3.1047128496214844E-5 evalInputDelta=5.213618278503418E-4
END: th(0.008478865685231917)=19.187369883060455; dx=-0.0586981808421934 evalInputDelta=4.348158836364746E-4
Fitness changed from 19.187804698944092 to 19.18728333711624
Iteration 17 complete. Error: 19.18728333711624 Total: 93.2193; Orientation: 0.9089; Line Search: 85.8412
Adding measurement 12fffddf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.991e+03, gradient 2.355e-03, dot -0.242; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 19.18728333711624, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 12fffddf to history. Total: 3
th(0)=19.18728333711624;dx=-5.5470715928930074E-6
New Minimum: 19.18728333711624 > 19.187283277511597
WOLFE (weak): th(0.018267162364384602)=19.187283277511597; dx=-5.533083518597067E-6 evalInputDelta=5.9604644775390625E-8
New Minimum: 19.187283277511597 > 19.187283098697662
WOLFE (weak): th(0.036534324728769205)=19.187283098697662; dx=-5.533063513775751E-6 evalInputDelta=2.384185791015625E-7
New Minimum: 19.187283098697662 > 19.187283039093018
WOLFE (weak): th(0.10960297418630761)=19.187283039093018; dx=-5.533089517055377E-6 evalInputDelta=2.980232238769531E-7
New Minimum: 19.187283039093018 > 19.18728205561638
WOLFE (weak): th(0.43841189674523046)=19.18728205561638; dx=-5.533035390923161E-6 evalInputDelta=1.2814998626708984E-6
New Minimum: 19.18728205561638 > 19.187277734279633
WOLFE (weak): th(2.1920594837261524)=19.187277734279633; dx=-5.532415756490992E-6 evalInputDelta=5.602836608886719E-6
New Minimum: 19.187277734279633 > 19.187247961759567
WOLFE (weak): th(13.152356902356914)=19.187247961759567; dx=-5.531167242530751E-6 evalInputDelta=3.5375356674194336E-5
New Minimum: 19.187247961759567 > 19.187035262584686
WOLFE (weak): th(92.0664983164984)=19.187035262584686; dx=-5.524253238106223E-6 evalInputDelta=2.480745315551758E-4
New Minimum: 19.187035262584686 > 19.186891078948975
WOLFE (weak): th(736.5319865319872)=19.186891078948975; dx=-5.5210968563825E-6 evalInputDelta=3.922581672668457E-4
New Minimum: 19.186891078948975 > 19.17439803481102
END: th(6628.787878787884)=19.17439803481102; dx=-3.879243418688268E-6 evalInputDelta=0.012885302305221558
Fitness changed from 19.18728333711624 to 19.17439803481102
Iteration 18 complete. Error: 19.17439803481102 Total: 79.2689; Orientation: 0.8798; Line Search: 71.3755
Adding measurement 417cce81 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.008e+03, gradient 1.866e-03, dot -0.299; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 19.17439803481102, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 417cce81 to history. Total: 3
th(0)=19.17439803481102;dx=-3.481340059424711E-6
New Minimum: 19.17439803481102 > 19.1515009701252
END: th(14281.290558923485)=19.1515009701252; dx=-3.123940370864927E-6 evalInputDelta=0.022897064685821533
Fitness changed from 19.17439803481102 to 19.1515009701252
Iteration 19 complete. Error: 19.1515009701252 Total: 21.8172; Orientation: 0.8976; Line Search: 14.4408
Adding measurement 262a6c8f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.989e+03, gradient 1.787e-03, dot -0.309; [c5de46de-4976-4f70-8a7a-f6bcdf8ab65d = 1.000/1.000e+00]
Orientation rejected. Popping history element from 19.1515009701252, 30.958445012569427, 32.01976352930069, 32.818611800670624
LBFGS Accumulation History: 3 points
Removed measurement 262a6c8f to history. Total: 3
th(0)=19.1515009701252;dx=-3.192766262562871E-6
New Minimum: 19.1515009701252 > 19.11322659254074
END: th(30768.10779856959)=19.11322659254074; dx=-2.2716424087045087E-6 evalInputDelta=0.0382743775844574
Fitness changed from 19.1515009701252 to 19.11322659254074
Iteration 20 complete. Error: 19.11322659254074 Total: 21.8575; Orientation: 0.8879; Line Search: 14.2287
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: 19.11322659254074 (> -Infinity) after 696.062s (< 5400.000s)
Returns
19.11322659254074