Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 238.24 seconds (2.700 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 920986505123500
    Reset training subject: 920989655884300
    Adding measurement 17e11783 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.439131811261177;dx=-3.0381445646717703E-7
    New Minimum: 1.439131811261177 > 1.4391316920518875
    WOLFE (weak): th(2.154434690031884)=1.4391316920518875; dx=-3.0381444944575555E-7 evalInputDelta=1.1920928955078125E-7
    New Minimum: 1.4391316920518875 > 1.439131185412407
    WOLFE (weak): th(4.308869380063768)=1.439131185412407; dx=-3.0381432800072236E-7 evalInputDelta=6.258487701416016E-7
    New Minimum: 1.439131185412407 > 1.4391289874911308
    WOLFE (weak): th(12.926608140191302)=1.4391289874911308; dx=-3.0381563045127926E-7 evalInputDelta=2.823770046234131E-6
    New Minimum: 1.4391289874911308 > 1.439119592308998
    WOLFE (weak): th(51.70643256076521)=1.439119592308998; dx=-3.038267966831627E-7 evalInputDelta=1.2218952178955078E-5
    New Minimum: 1.439119592308998 > 1.4390705898404121
    WOLFE (weak): th(258.53216280382605)=1.4390705898404121; dx=-3.037914823532227E-7 evalInputDelta=6.12214207649231E-5
    New Minimum: 1.4390705898404121 > 1.4387643486261368
    WOLFE (weak): th(1551.1929768229563)=1.4387643486261368; dx=-3.035679473318348E-7 evalInputDelta=3.674626350402832E-4
    New Minimum: 1.4387643486261368 > 1.4365623518824577
    WOLFE (weak): th(10858.350837760694)=1.4365623518824577; dx=-3.0264985236238644E-7 evalInputDelta=0.00256945937871933
    New Minimum: 1.4365623518824577 > 1.4189695566892624
    WOLFE (weak): th(86866.80670208555)=1.4189695566892624; dx=-2.8989706499258265E-7 evalInputDelta=0.020162254571914673
    New Minimum: 1.4189695566892624 > 1.286973349750042
    END: th(781801.26031877)=1.286973349750042; dx=-1.9510994685283072E-7 evalInputDelta=0.1521584615111351
    Fitness changed from 1.439131811261177 to 1.286973349750042
    Iteration 1 complete. Error: 1.286973349750042 Total: 41.3411; Orientation: 0.0611; Line Search: 32.2300
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/aae08f8a-3b87-

...skipping 10913 bytes...

    35fae6-a26e-4b54-82ed-87ca4b6a39c3 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.11370128393173218, 0.09268498420715332, 0.07192167639732361, 0.05448263883590698
    LBFGS Accumulation History: 3 points
    Removed measurement 1617b329 to history. Total: 4
    th(0)=0.05448263883590698;dx=-3.6899104966079555E-9
    New Minimum: 0.05448263883590698 > 0.035633206367492676
    END: th(1.462100482588026E7)=0.035633206367492676; dx=-5.422674736134608E-10 evalInputDelta=0.018849432468414307
    Fitness changed from 0.05448263883590698 to 0.035633206367492676
    Iteration 14 complete. Error: 0.035633206367492676 Total: 12.1608; Orientation: 2.1365; Line Search: 6.8624
    Adding measurement 32c6ffba to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.919e+03, gradient 9.779e-05, dot -0.497; [c935fae6-a26e-4b54-82ed-87ca4b6a39c3 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.09268498420715332, 0.07192167639732361, 0.05448263883590698, 0.035633206367492676
    LBFGS Accumulation History: 3 points
    Removed measurement 13f746e8 to history. Total: 4
    th(0)=0.035633206367492676;dx=-9.56362712019709E-9
    Armijo: th(3.1500000000000015E7)=0.2706812769174576; dx=8.753643539816068E-9 evalInputDelta=-0.2350480705499649
    Armijo: th(1.5750000000000007E7)=0.10928228497505188; dx=1.032374722297794E-8 evalInputDelta=-0.0736490786075592
    New Minimum: 0.035633206367492676 > 0.022318005561828613
    WOLF (strong): th(5250000.000000003)=0.022318005561828613; dx=2.406692574765279E-9 evalInputDelta=0.013315200805664062
    END: th(1312500.0000000007)=0.026348143815994263; dx=-6.093316754257272E-9 evalInputDelta=0.009285062551498413
    Fitness changed from 0.035633206367492676 to 0.022318005561828613
    Iteration 15 complete. Error: 0.022318005561828613 Total: 21.7209; Orientation: 1.7783; Line Search: 16.6716
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/63d54c50-d5b4-4db1-9e69-e457bd31b1b2.jpg)
    
    Final threshold in iteration 16: 0.022318005561828613 (> -Infinity) after 238.235s (< 1800.000s)
    

Returns:

    0.022318005561828613