BasicOptimizer.scala:89 executed in 137.42 seconds (1.787 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 69262067395299
Reset training subject: 69264764239100
Adding measurement 3c45bfb2 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-16.988316324112528;dx=-6.938238271418083E-7
New Minimum: -16.988316324112528 > -16.98831708386167
END: th(2.154434690031884)=-16.98831708386167; dx=-5.508915037095422E-7 evalInputDelta=7.597491418209756E-7
Fitness changed from -16.988316324112528 to -16.98831708386167
Iteration 1 complete. Error: -16.98831708386167 Total: 13.9632; Orientation: 0.0905; Line Search: 5.8555
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 120d3c5b to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-16.98831708386167;dx=-5.494782285385425E-7
New Minimum: -16.98831708386167 > -16.988319363108886
WOLFE (weak): th(4.641588833612779)=-16.988319363108886; dx=-5.493012454666051E-7 evalInputDelta=2.27924721585282E-6
New Minimum: -16.988319363108886 > -16.9883224021047
WOLFE (weak): th(9.283177667225559)=-16.9883224021047; dx=-5.488662783077845E-7 evalInputDelta=5.3182430299614225E-6
New Minimum: -16.9883224021047 > -16.988333798334153
WOLFE (weak): th(27.849533001676676)=-16.988333798334153; dx=-5.491914093481333E-7 evalInputDelta=1.671447248341451E-5
New Minimum: -16.988333798334153 > -16.988384701399056
WOLFE (weak): th(111.3981320067067)=-16.988384701399056; dx=-5.490360367769181E-7 evalInputDelta=6.761753738615539E-5
New Minimum: -16.988384701399056 > -16.9886559285913
WOLFE (weak): th(556.9906600335335)=-16.9886559285913; dx=-5.488583640268648E-7 evalInputDelta=3.388447296295283E-4
New Minimum: -16.9886559285913 > -16.990351570190263
WOLFE (weak): th(3341.9439602012008)=-16.990351570190263; dx=-5.487026114801478E-7 evalInputDelta=0.0020344863285934878
New Minimum: -16.990351570190263 > -17.002530602477076
WOLFE (weak): th(23393.607721408407)=-17.002530602477076; dx=-5.463435473409408E-7 evalInputDelta=0.014213518615406429
New Minimum: -17.002530602477076 > -17.100053645810892
WOLFE (weak): th(187148.86177126726)=-17.100053645810892; dx=-5.266335701001103E-7 evalInputDelta=0.11173656194922188
New Minimum: -17.100053645810892 > -17.740857982590647
END: th(1684339.7559414054)=-17.740857982590647; dx=-2.43231232479667E-7 evalInputDelta=0.7525408987289772
Fitness changed from -16.98831708386167 to -17.740857982590647
Iteration 2 complete. Error: -17.740857982590647 Total: 29.8593; Orientation: 0.0702; Line Search: 26.6753
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3f327c15 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-17.740857982590647;dx=-4.986654335867577E-7
New Minimum: -17.740857982590647 > -18.5389424179001
END: th(3628800.0000000005)=-18.5389424179001; dx=-9.269370507364201E-8 evalInputDelta=0.7980844353094518
Fitness changed from -17.740857982590647 to -18.5389424179001
Iteration 3 complete. Error: -18.5389424179001 Total: 8.6228; Orientation: 0.0635; Line Search: 5.4688
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3b1698d4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.276e+03, gradient 7.094e-04, dot -0.492; [d5296cf6-2b9b-43bb-848c-18c6668bcacc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.5389424179001, -17.740857982590647, -16.98831708386167, -16.988316324112528
LBFGS Accumulation History: 3 points
Removed measurement 3b1698d4 to history. Total: 3
th(0)=-18.5389424179001;dx=-5.029476429725646E-7
New Minimum: -18.5389424179001 > -19.327505833077463
WOLF (strong): th(7818012.603187701)=-19.327505833077463; dx=1.2412197851942391E-9 evalInputDelta=0.7885634151773644
New Minimum: -19.327505833077463 > -19.412372861366848
WOLF (strong): th(3909006.3015938504)=-19.412372861366848; dx=1.7133606939939915E-8 evalInputDelta=0.8734304434667486
END: th(1303002.1005312835)=-19.148842250956633; dx=-3.0412106244936764E-7 evalInputDelta=0.6098998330565344
Fitness changed from -18.5389424179001 to -19.412372861366848
Iteration 4 complete. Error: -19.412372861366848 Total: 14.4565; Orientation: 1.1094; Line Search: 10.3864
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5bbf00c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.885e+03, gradient 7.419e-04, dot -0.340; [d5296cf6-2b9b-43bb-848c-18c6668bcacc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.412372861366848, -17.740857982590647, -16.98831708386167, -16.988316324112528
LBFGS Accumulation History: 3 points
Removed measurement 5bbf00c4 to history. Total: 3
th(0)=-19.412372861366848;dx=-5.502409239027139E-7
New Minimum: -19.412372861366848 > -20.32379922294126
WOLF (strong): th(2807232.926569009)=-20.32379922294126; dx=3.883287964558154E-8 evalInputDelta=0.9114263615744136
END: th(1403616.4632845046)=-20.1373947167968; dx=-3.198415461712705E-7 evalInputDelta=0.7250218554299508
Fitness changed from -19.412372861366848 to -20.32379922294126
Iteration 5 complete. Error: -20.32379922294126 Total: 12.0179; Orientation: 1.0994; Line Search: 7.8502
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7ec3650 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.079e+03, gradient 6.998e-04, dot -0.460; [d5296cf6-2b9b-43bb-848c-18c6668bcacc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.32379922294126, -17.740857982590647, -16.98831708386167, -16.988316324112528
LBFGS Accumulation History: 3 points
Removed measurement 7ec3650 to history. Total: 3
th(0)=-20.32379922294126;dx=-4.895364515512564E-7
New Minimum: -20.32379922294126 > -20.75102608583416
WOLF (strong): th(3024000.0000000005)=-20.75102608583416; dx=1.2010929304640715E-7 evalInputDelta=0.4272268628928977
New Minimum: -20.75102608583416 > -20.85556377128078
END: th(1512000.0000000002)=-20.85556377128078; dx=-7.713393852582481E-8 evalInputDelta=0.5317645483395204
Fitness changed from -20.32379922294126 to -20.85556377128078
Iteration 6 complete. Error: -20.85556377128078 Total: 11.9825; Orientation: 1.0500; Line Search: 7.6998
Adding measurement 2504a5a1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.281e+04, gradient 5.762e-04, dot -0.635; [d5296cf6-2b9b-43bb-848c-18c6668bcacc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.85556377128078, -17.740857982590647, -16.98831708386167, -16.988316324112528
LBFGS Accumulation History: 3 points
Removed measurement 2504a5a1 to history. Total: 3
th(0)=-20.85556377128078;dx=-3.3198256946349827E-7
New Minimum: -20.85556377128078 > -21.17058144030865
END: th(3257505.251328209)=-21.17058144030865; dx=-9.206738655936595E-9 evalInputDelta=0.315017669027867
Fitness changed from -20.85556377128078 to -21.17058144030865
Iteration 7 complete. Error: -21.17058144030865 Total: 8.9238; Orientation: 1.1864; Line Search: 5.3406
Adding measurement 363fcda2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.075e+04, gradient 6.858e-04, dot -0.448; [d5296cf6-2b9b-43bb-848c-18c6668bcacc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.17058144030865, -17.740857982590647, -16.98831708386167, -16.988316324112528
LBFGS Accumulation History: 3 points
Removed measurement 363fcda2 to history. Total: 3
th(0)=-21.17058144030865;dx=-4.7013397583932194E-7
New Minimum: -21.17058144030865 > -21.3222810354725
WOLF (strong): th(7018082.316422524)=-21.3222810354725; dx=4.045334038985528E-8 evalInputDelta=0.15169959516385134
New Minimum: -21.3222810354725 > -21.68032323438033
WOLF (strong): th(3509041.158211262)=-21.68032323438033; dx=1.2454489842945567E-7 evalInputDelta=0.5097417940716831
New Minimum: -21.68032323438033 > -21.686957722285335
END: th(1169680.3860704207)=-21.686957722285335; dx=-2.7618961719220477E-7 evalInputDelta=0.5163762819766866
Fitness changed from -21.17058144030865 to -21.686957722285335
Iteration 8 complete. Error: -21.686957722285335 Total: 14.0107; Orientation: 1.0296; Line Search: 10.3855
Adding measurement 60f5b4a2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.617e+04, gradient 5.580e-04, dot -0.625; [d5296cf6-2b9b-43bb-848c-18c6668bcacc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.686957722285335, -17.740857982590647, -16.98831708386167, -16.988316324112528
LBFGS Accumulation History: 3 points
Removed measurement 60f5b4a2 to history. Total: 3
th(0)=-21.686957722285335;dx=-3.1105794059706333E-7
New Minimum: -21.686957722285335 > -22.10842465721125
END: th(2520000.000000001)=-22.10842465721125; dx=-2.4015767202380047E-8 evalInputDelta=0.42146693492591325
Fitness changed from -21.686957722285335 to -22.10842465721125
Iteration 9 complete. Error: -22.10842465721125 Total: 8.8984; Orientation: 1.0316; Line Search: 5.2909
Adding measurement 4d1d7bd9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.625e+04, gradient 6.489e-04, dot -0.508; [d5296cf6-2b9b-43bb-848c-18c6668bcacc = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.10842465721125, -17.740857982590647, -16.98831708386167, -16.988316324112528
LBFGS Accumulation History: 3 points
Removed measurement 4d1d7bd9 to history. Total: 3
th(0)=-22.10842465721125;dx=-4.2110157402169074E-7
New Minimum: -22.10842465721125 > -22.205037662286994
WOLF (strong): th(5429175.418880349)=-22.205037662286994; dx=3.2062293343985344E-8 evalInputDelta=0.09661300507574566
New Minimum: -22.205037662286994 > -22.421199360212817
WOLF (strong): th(2714587.7094401745)=-22.421199360212817; dx=1.0015790252139078E-7 evalInputDelta=0.31277470300156907
New Minimum: -22.421199360212817 > -22.45091796392727
END: th(904862.5698133915)=-22.45091796392727; dx=-2.1687160180916588E-7 evalInputDelta=0.3424933067160225
Fitness changed from -22.10842465721125 to -22.45091796392727
Iteration 10 complete. Error: -22.45091796392727 Total: 14.2110; Orientation: 1.0441; Line Search: 10.3947
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -22.45091796392727 (> -Infinity) after 137.420s (< 5400.000s)
Returns
-22.45091796392727