Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Code from BasicOptimizer.scala:75 executed in 30.85 seconds (0.491 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 911541501564900
    Reset training subject: 911542589536200
    Adding measurement 6a2d9ede to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.6831014752388;dx=-8.956818122552765E-8
    New Minimum: 0.6831014752388 > 0.6831013560295105
    WOLFE (weak): th(2.154434690031884)=0.6831013560295105; dx=-8.951469212943937E-8 evalInputDelta=1.1920928955078125E-7
    New Minimum: 0.6831013560295105 > 0.6831012964248657
    WOLFE (weak): th(4.308869380063768)=0.6831012964248657; dx=-8.951682589517616E-8 evalInputDelta=1.7881393432617188E-7
    New Minimum: 0.6831012964248657 > 0.683100700378418
    WOLFE (weak): th(12.926608140191302)=0.683100700378418; dx=-8.951663446360999E-8 evalInputDelta=7.748603820800781E-7
    New Minimum: 0.683100700378418 > 0.6830980777740479
    WOLFE (weak): th(51.70643256076521)=0.6830980777740479; dx=-8.951594462901391E-8 evalInputDelta=3.3974647521972656E-6
    New Minimum: 0.6830980777740479 > 0.6830849051475525
    WOLFE (weak): th(258.53216280382605)=0.6830849051475525; dx=-8.951453026656297E-8 evalInputDelta=1.6570091247558594E-5
    New Minimum: 0.6830849051475525 > 0.6830018758773804
    WOLFE (weak): th(1551.1929768229563)=0.6830018758773804; dx=-8.949267654371911E-8 evalInputDelta=9.959936141967773E-5
    New Minimum: 0.6830018758773804 > 0.6824046075344086
    WOLFE (weak): th(10858.350837760694)=0.6824046075344086; dx=-8.946212845458453E-8 evalInputDelta=6.968677043914795E-4
    New Minimum: 0.6824046075344086 > 0.6775610446929932
    WOLFE (weak): th(86866.80670208555)=0.6775610446929932; dx=-8.839908826698627E-8 evalInputDelta=0.005540430545806885
    New Minimum: 0.6775610446929932 > 0.6364290714263916
    END: th(781801.26031877)=0.6364290714263916; dx=-7.954321450533162E-8 evalInputDelta=0.04667240381240845
    Fitness changed from 0.6831014752388 to 0.6364290714263916
    Iteration 1 complete. Error: 0.6364290714263916 Total: 14.1725; Orientation: 0.0141; Line Search: 10.9515
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/e8c2aaa9-e294-4

...skipping 1841 bytes...

    74 to 0.22493520379066467
    Iteration 4 complete. Error: 0.22493520379066467 Total: 3.5146; Orientation: 0.4243; Line Search: 2.0602
    <a id="p-9"></a>Iteration 4
    <a id="p-8"></a>![Iteration 4](etc/4b0109cc-a3e8-45da-b9c7-35a488358298.jpg)
    
    Adding measurement 5649bcb6 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.100e+03, gradient 1.941e-04, dot -0.848; [05aaf650-ae11-49c8-a065-3523e8413b28 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.6364290714263916, 0.5521142482757568, 0.41476261615753174, 0.22493520379066467
    LBFGS Accumulation History: 3 points
    Removed measurement 23481b1a to history. Total: 4
    th(0)=0.22493520379066467;dx=-3.76824978088181E-8
    New Minimum: 0.22493520379066467 > 0.026435941457748413
    END: th(1.684339755941405E7)=0.026435941457748413; dx=-1.1882978959818217E-8 evalInputDelta=0.19849926233291626
    Fitness changed from 0.22493520379066467 to 0.026435941457748413
    Iteration 5 complete. Error: 0.026435941457748413 Total: 3.5072; Orientation: 0.4239; Line Search: 2.0620
    <a id="p-11"></a>Iteration 5
    <a id="p-10"></a>![Iteration 5](etc/13337b69-1c79-46d5-8fe1-f393c2b99923.jpg)
    
    Adding measurement 2321821d to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.675e+03, gradient 1.513e-04, dot -0.816; [05aaf650-ae11-49c8-a065-3523e8413b28 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.5521142482757568, 0.41476261615753174, 0.22493520379066467, 0.026435941457748413
    LBFGS Accumulation History: 3 points
    Removed measurement 6e05ec93 to history. Total: 4
    th(0)=0.026435941457748413;dx=-2.28820206354005E-8
    New Minimum: 0.026435941457748413 > -0.08057776093482971
    END: th(3.6288E7)=-0.08057776093482971; dx=-3.17927625326067E-9 evalInputDelta=0.10701370239257812
    Fitness changed from 0.026435941457748413 to -0.08057776093482971
    Iteration 6 complete. Error: -0.08057776093482971 Total: 3.4877; Orientation: 0.4203; Line Search: 2.0749
    Final threshold in iteration 7: -0.08057776093482971 (> -Infinity) after 30.853s (< 720.000s)
    

Returns:

    -0.08057776093482971