BasicOptimizer.scala:89 executed in 54.76 seconds (1.155 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 209235597089400
Reset training subject: 209236408578800
Adding measurement 559cdd50 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=11.10892836099608;dx=-1.5619986719023332E-6
New Minimum: 11.10892836099608 > 11.108925896530591
WOLFE (weak): th(2.154434690031884)=11.108925896530591; dx=-1.562376879104556E-6 evalInputDelta=2.464465488571932E-6
New Minimum: 11.108925896530591 > 11.108922250007186
WOLFE (weak): th(4.308869380063768)=11.108922250007186; dx=-1.5621316932555985E-6 evalInputDelta=6.110988893581748E-6
New Minimum: 11.108922250007186 > 11.108909987889808
WOLFE (weak): th(12.926608140191302)=11.108909987889808; dx=-1.5622340542311016E-6 evalInputDelta=1.8373106271951656E-5
New Minimum: 11.108909987889808 > 11.108851171877852
WOLFE (weak): th(51.70643256076521)=11.108851171877852; dx=-1.561864912579028E-6 evalInputDelta=7.718911822784946E-5
New Minimum: 11.108851171877852 > 11.108538819188173
WOLFE (weak): th(258.53216280382605)=11.108538819188173; dx=-1.5616000992478967E-6 evalInputDelta=3.8954180790717885E-4
New Minimum: 11.108538819188173 > 11.106585004126897
WOLFE (weak): th(1551.1929768229563)=11.106585004126897; dx=-1.561299995455477E-6 evalInputDelta=0.0023433568691828555
New Minimum: 11.106585004126897 > 11.092530872726453
WOLFE (weak): th(10858.350837760694)=11.092530872726453; dx=-1.559895892660174E-6 evalInputDelta=0.016397488269626947
New Minimum: 11.092530872726453 > 10.97880547239396
WOLFE (weak): th(86866.80670208555)=10.97880547239396; dx=-1.5422878200261743E-6 evalInputDelta=0.13012288860211996
New Minimum: 10.97880547239396 > 10.036005623667895
END: th(781801.26031877)=10.036005623667895; dx=-1.325955944485715E-6 evalInputDelta=1.0729227373281844
Fitness changed from 11.10892836099608 to 10.036005623667895
Iteration 1 complete. Error: 10.036005623667895 Total: 9.8923; Orientation: 0.0155; Line Search: 7.6494
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4cc4d1b3 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=10.036005623667895;dx=-1.301336065674697E-6
New Minimum: 10.036005623667895 > 8.388654490761919
END: th(1684339.7559414052)=8.388654490761919; dx=-8.162945816770032E-7 evalInputDelta=1.6473511329059765
Fitness changed from 10.036005623667895 to 8.388654490761919
Iteration 2 complete. Error: 8.388654490761919 Total: 2.2103; Orientation: 0.0158; Line Search: 1.4954
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6cdfeb4a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=8.388654490761919;dx=-9.686711540364998E-7
New Minimum: 8.388654490761919 > 7.254683896827959
END: th(3628800.0)=7.254683896827959; dx=-2.0010690101357724E-7 evalInputDelta=1.1339705939339595
Fitness changed from 8.388654490761919 to 7.254683896827959
Iteration 3 complete. Error: 7.254683896827959 Total: 2.4479; Orientation: 0.0157; Line Search: 1.6746
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 159bc7bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.672e+03, gradient 9.200e-04, dot -0.355; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.254683896827959, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 159bc7bb to history. Total: 3
th(0)=7.254683896827959;dx=-8.444810806013622E-7
Armijo: th(7818012.6031877)=10.160277983865532; dx=4.061872858339655E-7 evalInputDelta=-2.9055940870375725
New Minimum: 7.254683896827959 > 6.990534002092399
WOLF (strong): th(3909006.30159385)=6.990534002092399; dx=1.779159986218049E-8 evalInputDelta=0.26414989473556005
New Minimum: 6.990534002092399 > 6.571490320175632
END: th(1303002.1005312833)=6.571490320175632; dx=-5.116132953624871E-7 evalInputDelta=0.6831935766523278
Fitness changed from 7.254683896827959 to 6.571490320175632
Iteration 4 complete. Error: 6.571490320175632 Total: 4.1125; Orientation: 0.3164; Line Search: 3.0625
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2059ee1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.112e+03, gradient 7.521e-04, dot -0.403; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.571490320175632, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 2059ee1 to history. Total: 3
th(0)=6.571490320175632;dx=-5.639076219048908E-7
New Minimum: 6.571490320175632 > 6.143665999262576
END: th(2807232.9265690087)=6.143665999262576; dx=-2.048034595131117E-7 evalInputDelta=0.42782432091305544
Fitness changed from 6.571490320175632 to 6.143665999262576
Iteration 5 complete. Error: 6.143665999262576 Total: 2.6287; Orientation: 0.3184; Line Search: 1.5574
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 36daa049 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.268e+03, gradient 7.473e-04, dot -0.357; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.143665999262576, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 36daa049 to history. Total: 3
th(0)=6.143665999262576;dx=-5.568480765824429E-7
Armijo: th(6048000.0)=7.441274854516076; dx=1.718066891725647E-7 evalInputDelta=-1.2976088552534994
New Minimum: 6.143665999262576 > 6.037115763122747
END: th(3024000.0)=6.037115763122747; dx=-5.301624124417487E-8 evalInputDelta=0.10655023613982895
Fitness changed from 6.143665999262576 to 6.037115763122747
Iteration 6 complete. Error: 6.037115763122747 Total: 3.5589; Orientation: 0.4664; Line Search: 2.3786
Adding measurement 3b79a8ad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.459e+03, gradient 7.997e-04, dot -0.307; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.037115763122747, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 3b79a8ad to history. Total: 3
th(0)=6.037115763122747;dx=-6.377672736258929E-7
Armijo: th(6515010.502656417)=8.282178894260445; dx=3.745498958322949E-7 evalInputDelta=-2.245063131137698
Armijo: th(3257505.2513282085)=6.070801297958336; dx=9.697476138306346E-8 evalInputDelta=-0.03368553483558845
New Minimum: 6.037115763122747 > 5.6408271642232
END: th(1085835.0837760696)=5.6408271642232; dx=-3.5181083620734735E-7 evalInputDelta=0.39628859889954704
Fitness changed from 6.037115763122747 to 5.6408271642232
Iteration 7 complete. Error: 5.6408271642232 Total: 3.9878; Orientation: 0.2939; Line Search: 2.9736
Adding measurement 6a1282b9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.373e+03, gradient 6.314e-04, dot -0.351; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.6408271642232, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 6a1282b9 to history. Total: 3
th(0)=5.6408271642232;dx=-3.969890057531025E-7
New Minimum: 5.6408271642232 > 5.48064191522015
END: th(2339360.772140841)=5.48064191522015; dx=-1.1625090085807208E-7 evalInputDelta=0.16018524900305042
Fitness changed from 5.6408271642232 to 5.48064191522015
Iteration 8 complete. Error: 5.48064191522015 Total: 2.6676; Orientation: 0.2823; Line Search: 1.5148
Adding measurement 7fc43357 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.500e+03, gradient 6.536e-04, dot -0.326; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.48064191522015, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 7fc43357 to history. Total: 3
th(0)=5.48064191522015;dx=-4.249827515274782E-7
Armijo: th(5040000.000000001)=6.352823601681035; dx=1.2808081739423986E-7 evalInputDelta=-0.8721816864608849
New Minimum: 5.48064191522015 > 5.439635136375333
END: th(2520000.0000000005)=5.439635136375333; dx=-2.816633460609091E-8 evalInputDelta=0.041006778844816516
Fitness changed from 5.48064191522015 to 5.439635136375333
Iteration 9 complete. Error: 5.439635136375333 Total: 3.2222; Orientation: 0.3109; Line Search: 2.2022
Adding measurement 5f7957cd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.118e+03, gradient 7.077e-04, dot -0.296; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.439635136375333, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 5f7957cd to history. Total: 3
th(0)=5.439635136375333;dx=-4.989577356766715E-7
Armijo: th(5429175.418880348)=6.882274243524598; dx=2.6093824632510246E-7 evalInputDelta=-1.4426391071492644
Armijo: th(2714587.709440174)=5.494208998695658; dx=7.208128176033516E-8 evalInputDelta=-0.054573862320324906
New Minimum: 5.439635136375333 > 5.200167582295202
END: th(904862.5698133913)=5.200167582295202; dx=-2.730848076445233E-7 evalInputDelta=0.23946755408013143
Fitness changed from 5.439635136375333 to 5.200167582295202
Iteration 10 complete. Error: 5.200167582295202 Total: 4.1989; Orientation: 0.3112; Line Search: 3.2216
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4ac08db5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.341e+03, gradient 5.571e-04, dot -0.338; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.200167582295202, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 4ac08db5 to history. Total: 3
th(0)=5.200167582295202;dx=-3.086806357322922E-7
New Minimum: 5.200167582295202 > 5.09788462955654
END: th(1949467.3101173674)=5.09788462955654; dx=-1.0028706925690098E-7 evalInputDelta=0.10228295273866195
Fitness changed from 5.200167582295202 to 5.09788462955654
Iteration 11 complete. Error: 5.09788462955654 Total: 2.5337; Orientation: 0.2854; Line Search: 1.5527
Adding measurement 6256b644 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.299e+03, gradient 5.709e-04, dot -0.322; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.09788462955654, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 6256b644 to history. Total: 3
th(0)=5.09788462955654;dx=-3.2495039164892687E-7
Armijo: th(4200000.000000001)=5.6169823142634305; dx=7.354139776117182E-8 evalInputDelta=-0.5190976847068907
New Minimum: 5.09788462955654 > 5.0796640394604236
END: th(2100000.0000000005)=5.0796640394604236; dx=-3.691975625349112E-8 evalInputDelta=0.018220590096116318
Fitness changed from 5.09788462955654 to 5.0796640394604236
Iteration 12 complete. Error: 5.0796640394604236 Total: 3.5077; Orientation: 0.3019; Line Search: 2.5188
Adding measurement 37790c00 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.281e+03, gradient 6.188e-04, dot -0.298; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.0796640394604236, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 37790c00 to history. Total: 3
th(0)=5.0796640394604236;dx=-3.799276420983832E-7
Armijo: th(4524312.849066957)=5.916143759456776; dx=1.6296399234912522E-7 evalInputDelta=-0.8364797199963521
Armijo: th(2262156.4245334785)=5.106997852824114; dx=3.517646130087235E-8 evalInputDelta=-0.027333813363690673
New Minimum: 5.0796640394604236 > 4.936378407306023
END: th(754052.1415111595)=4.936378407306023; dx=-2.1232423728340704E-7 evalInputDelta=0.1432856321544005
Fitness changed from 5.0796640394604236 to 4.936378407306023
Iteration 13 complete. Error: 4.936378407306023 Total: 3.9188; Orientation: 0.2833; Line Search: 2.9607
Adding measurement 1b7c4d49 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.337e+03, gradient 4.954e-04, dot -0.338; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.936378407306023, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 1b7c4d49 to history. Total: 3
th(0)=4.936378407306023;dx=-2.4061951943643626E-7
New Minimum: 4.936378407306023 > 4.876497108551278
END: th(1624556.0917644731)=4.876497108551278; dx=-9.279890002322043E-8 evalInputDelta=0.05988129875474524
Fitness changed from 4.936378407306023 to 4.876497108551278
Iteration 14 complete. Error: 4.876497108551278 Total: 2.7078; Orientation: 0.2965; Line Search: 1.7423
Adding measurement 18474d04 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.319e+03, gradient 5.056e-04, dot -0.326; [55c94ba3-ed53-47ac-a1f8-7c63ec2b94bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.876497108551278, 8.388654490761919, 10.036005623667895, 11.10892836099608
LBFGS Accumulation History: 3 points
Removed measurement 18474d04 to history. Total: 3
th(0)=4.876497108551278;dx=-2.536853926935391E-7
Armijo: th(3500000.0000000014)=5.166551004510748; dx=2.76722217868562E-8 evalInputDelta=-0.2900538959594705
New Minimum: 4.876497108551278 > 4.858272542216157
END: th(1750000.0000000007)=4.858272542216157; dx=-4.9744969338987806E-8 evalInputDelta=0.01822456633512104
Fitness changed from 4.876497108551278 to 4.858272542216157
Iteration 15 complete. Error: 4.858272542216157 Total: 3.1470; Orientation: 0.3070; Line Search: 2.1917
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: 4.858272542216157 (> -Infinity) after 54.763s (< 5400.000s)
Returns
4.858272542216157