Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 3622.28 seconds (47.174 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1337030401114800
    Reset training subject: 1337074000703300
    Adding measurement 58c36104 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=6.883337376639247;dx=-4.9086824542370144E-6
    New Minimum: 6.883337376639247 > 6.883331434801221
    WOLFE (weak): th(2.154434690031884)=6.883331434801221; dx=-4.877739032932398E-6 evalInputDelta=5.941838026046753E-6
    New Minimum: 6.883331434801221 > 6.883325330913067
    WOLFE (weak): th(4.308869380063768)=6.883325330913067; dx=-4.877501914243836E-6 evalInputDelta=1.2045726180076599E-5
    New Minimum: 6.883325330913067 > 6.883302783593535
    WOLFE (weak): th(12.926608140191302)=6.883302783593535; dx=-4.877255934505355E-6 evalInputDelta=3.4593045711517334E-5
    New Minimum: 6.883302783593535 > 6.88319842889905
    WOLFE (weak): th(51.70643256076521)=6.88319842889905; dx=-4.8715603565858205E-6 evalInputDelta=1.389477401971817E-4
    New Minimum: 6.88319842889905 > 6.882642045617104
    WOLFE (weak): th(258.53216280382605)=6.882642045617104; dx=-4.869073731822072E-6 evalInputDelta=6.95331022143364E-4
    New Minimum: 6.882642045617104 > 6.879176141694188
    WOLFE (weak): th(1551.1929768229563)=6.879176141694188; dx=-4.842230389705609E-6 evalInputDelta=0.004161234945058823
    New Minimum: 6.879176141694188 > 6.8547482173889875
    WOLFE (weak): th(10858.350837760694)=6.8547482173889875; dx=-4.646654461490087E-6 evalInputDelta=0.0285891592502594
    New Minimum: 6.8547482173889875 > 6.689243447035551
    END: th(86866.80670208555)=6.689243447035551; dx=-3.3786892272627292E-6 evalInputDelta=0.19409392960369587
    Fitness changed from 6.883337376639247 to 6.689243447035551
    Iteration 1 complete. Error: 6.689243447035551 Total: 561.2924; Orientation: 0.0851; Line Search: 421.2146
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/5dbbb619-2c9e-48bf-9e8d-4f3023b3534a.jpg)
    
    Adding measurement 4023d23f to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=6.689243447035551;dx=-3.1487652200876885E-6
    New Minimum: 6.6

...skipping 13970 bytes...

    elta=0.04063594713807106
    New Minimum: 3.1279969587922096 > 3.0371288657188416
    END: th(451265.5810456872)=3.0371288657188416; dx=-2.0628926804361433E-7 evalInputDelta=0.13150404021143913
    Fitness changed from 3.1686329059302807 to 3.0371288657188416
    Iteration 17 complete. Error: 3.0371288657188416 Total: 230.9783; Orientation: 2.4738; Line Search: 182.7900
    Adding measurement 736f2fb8 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.326e+03, gradient 5.518e-04, dot -0.671; [4c9c185a-1ad9-4587-832d-894e9ab5a3a2 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 3.3917146660387516, 3.2694899328052998, 3.1686329059302807, 3.0371288657188416
    LBFGS Accumulation History: 3 points
    Removed measurement 3ab70d34 to history. Total: 4
    th(0)=3.0371288657188416;dx=-3.044803928293884E-7
    New Minimum: 3.0371288657188416 > 2.9388365671038628
    END: th(972222.2222222231)=2.9388365671038628; dx=-1.2888400318087337E-7 evalInputDelta=0.09829229861497879
    Fitness changed from 3.0371288657188416 to 2.9388365671038628
    Iteration 18 complete. Error: 2.9388365671038628 Total: 140.7757; Orientation: 2.3743; Line Search: 92.7488
    Adding measurement 64921450 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.358e+03, gradient 4.832e-04, dot -0.764; [4c9c185a-1ad9-4587-832d-894e9ab5a3a2 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 3.2694899328052998, 3.1686329059302807, 3.0371288657188416, 2.9388365671038628
    LBFGS Accumulation History: 3 points
    Removed measurement 5216532a to history. Total: 4
    th(0)=2.9388365671038628;dx=-2.3348061960216932E-7
    New Minimum: 2.9388365671038628 > 2.835347130894661
    END: th(2094589.2819754444)=2.835347130894661; dx=-2.7320206364056017E-8 evalInputDelta=0.10348943620920181
    Fitness changed from 2.9388365671038628 to 2.835347130894661
    Iteration 19 complete. Error: 2.835347130894661 Total: 138.9204; Orientation: 2.4064; Line Search: 90.5198
    Final threshold in iteration 19: 2.835347130894661 (> -Infinity) after 3622.281s (< 3600.000s)
    

Returns:

    2.835347130894661