Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 2493.88 seconds (15.559 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1073127010494800
    Reset training subject: 1073147547903300
    Adding measurement 6abaa14b to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=2.2433596000323055;dx=-2.884456026527891E-7
    New Minimum: 2.2433596000323055 > 2.243358884786676
    WOLFE (weak): th(2.154434690031884)=2.243358884786676; dx=-2.8573598752498797E-7 evalInputDelta=7.152456293901821E-7
    New Minimum: 2.243358884786676 > 2.2433579907271124
    WOLFE (weak): th(4.308869380063768)=2.2433579907271124; dx=-2.8573592437986617E-7 evalInputDelta=1.609305193106536E-6
    New Minimum: 2.2433579907271124 > 2.243355681067343
    WOLFE (weak): th(12.926608140191302)=2.243355681067343; dx=-2.8573565816197153E-7 evalInputDelta=3.918964962323912E-6
    New Minimum: 2.243355681067343 > 2.2433441625708195
    WOLFE (weak): th(51.70643256076521)=2.2433441625708195; dx=-2.8573549321811614E-7 evalInputDelta=1.5437461486023096E-5
    New Minimum: 2.2433441625708195 > 2.2432826064096463
    WOLFE (weak): th(258.53216280382605)=2.2432826064096463; dx=-2.8571215296799E-7 evalInputDelta=7.699362265922716E-5
    New Minimum: 2.2432826064096463 > 2.242898204489935
    WOLFE (weak): th(1551.1929768229563)=2.242898204489935; dx=-2.856571277567165E-7 evalInputDelta=4.6139554237045743E-4
    New Minimum: 2.242898204489935 > 2.2401329306368085
    WOLFE (weak): th(10858.350837760694)=2.2401329306368085; dx=-2.8520575504349155E-7 evalInputDelta=0.003226669395496984
    New Minimum: 2.2401329306368085 > 2.217707623301864
    WOLFE (weak): th(86866.80670208555)=2.217707623301864; dx=-2.817503786746542E-7 evalInputDelta=0.025651976730441284
    New Minimum: 2.217707623301864 > 2.026360319213692
    END: th(781801.26031877)=2.026360319213692; dx=-2.4713347454403673E-7 evalInputDelta=0.21699928081861364
    Fitness changed from 2.2433596000323055 to 2.026360319213692
    Iteration 1 complete. Error: 2.026360319213692 Total: 255.1763; Orientation: 0.0351; Line Search: 196.2275
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/f255383a-3041

...skipping 25347 bytes...

    tion rejected. Popping history element from -0.1289216779480027, -0.14582446329155552, -0.1574404556237825, -0.16565239849454239
    LBFGS Accumulation History: 3 points
    Removed measurement 7a79a5d6 to history. Total: 4
    th(0)=-0.16565239849454239;dx=-6.946089723166365E-9
    Armijo: th(1.7627561759597164E7)=-0.14669720714017576; dx=8.34502236512255E-10 evalInputDelta=-0.01895519135436663
    New Minimum: -0.16565239849454239 > -0.17261200688845735
    END: th(8813780.879798582)=-0.17261200688845735; dx=-3.3715396598437683E-10 evalInputDelta=0.006959608393914962
    Fitness changed from -0.16565239849454239 to -0.17261200688845735
    Iteration 29 complete. Error: -0.17261200688845735 Total: 78.7949; Orientation: 1.0358; Line Search: 58.3363
    Adding measurement 4b407ac7 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.898e+03, gradient 8.977e-05, dot -0.609; [3eb2f189-c2d2-41bc-a712-8eb7545de4b8 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.14582446329155552, -0.1574404556237825, -0.16565239849454239, -0.17261200688845735
    LBFGS Accumulation History: 3 points
    Removed measurement 650997b0 to history. Total: 4
    th(0)=-0.17261200688845735;dx=-8.057785168986459E-9
    Armijo: th(1.8988715277777802E7)=-0.14138342904689383; dx=1.6723235542134386E-9 evalInputDelta=-0.03122857784156352
    New Minimum: -0.17261200688845735 > -0.1787698449869386
    WOLF (strong): th(9494357.638888901)=-0.1787698449869386; dx=5.26282291328651E-10 evalInputDelta=0.00615783809848125
    New Minimum: -0.1787698449869386 > -0.1861096250286569
    END: th(3164785.879629634)=-0.1861096250286569; dx=-4.483479678687648E-9 evalInputDelta=0.013497618140199563
    Fitness changed from -0.17261200688845735 to -0.1861096250286569
    Iteration 30 complete. Error: -0.1861096250286569 Total: 97.9625; Orientation: 1.0346; Line Search: 77.6000
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/b96a5063-3527-45e2-8ebf-1f0b7a216e5e.jpg)
    
    Final threshold in iteration 31: -0.1861096250286569 (> -Infinity) after 2493.875s (< 3600.000s)
    

Returns:

    -0.1861096250286569