Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:88 executed in 65.60 seconds (1.582 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 674699697525600
    Reset training subject: 674701129697700
    Adding measurement 657e11b0 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    New Minimum: 185.29831187427044 > 17.904027435928583
    th(0)=17.904027435928583;dx=2.919530364280147E-7 (ERROR: Starting derivative negative)
    Fitness changed from 185.29831187427044 to 17.904027435928583
    Iteration 1 complete. Error: 17.904027435928583 Total: 6.9766; Orientation: 0.0284; Line Search: 2.9659
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/1e2d5199-bb6a-4524-a468-2cb6417edc46.jpg)
    
    Adding measurement 4ba5272e to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=17.904027435928583;dx=-6.13437677082424E-7
    New Minimum: 17.904027435928583 > 17.904022879898548
    END: th(4.641588833612779)=17.904022879898548; dx=-4.099031480694962E-7 evalInputDelta=4.556030035018921E-6
    Fitness changed from 17.904027435928583 to 17.904022879898548
    Iteration 2 complete. Error: 17.904022879898548 Total: 3.9259; Orientation: 0.0034; Line Search: 2.6474
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/8dc87420-38b6-4e8a-bc15-861cc8bea3ef.jpg)
    
    Adding measurement 2f6bf9fb to history. Total: 2
    LBFGS Accumulation History: 3 points
    th(0)=17.904022879898548;dx=-6.155490962960682E-7
    New Minimum: 17.904022879898548 > 17.904015075415373
    END: th(10.000000000000002)=17.904015075415373; dx=-4.119276032156045E-7 evalInputDelta=7.80448317527771E-6
    Fitness changed from 17.904022879898548 to 17.904015075415373
    Iteration 3 complete. Error: 17.904015075415373 Total: 4.6890; Orientation: 0.0046; Line Search: 3.3462
    <a id="p-7"></a>Iteration 3
    <a id="p-6"></a>![Iteration 3](etc/e063f081-1d0f-41b7-989f-848d312753f9.jpg)
    
    Adding measurement d50bdc8 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.696e+05, gradient 7.892e-04, dot -0.844; [518adf61-7a8e-4f71-8691-9a2fecee6893 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 17.904015075415373, 1

...skipping 7693 bytes...

    690031885)=17.869978167116642; dx=-4.769567909486727E-7 evalInputDelta=0.01898404210805893
    Fitness changed from 17.8889622092247 to 17.869978167116642
    Iteration 13 complete. Error: 17.869978167116642 Total: 3.9091; Orientation: 0.0407; Line Search: 2.5794
    Adding measurement 71b6ccab to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.325e+05, gradient 8.859e-04, dot -0.801; [518adf61-7a8e-4f71-8691-9a2fecee6893 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 17.869978167116642, 17.904022879898548, 17.904027435928583, 185.29831187427044
    LBFGS Accumulation History: 3 points
    Removed measurement 71b6ccab to history. Total: 3
    th(0)=17.869978167116642;dx=-7.793455180853334E-7
    New Minimum: 17.869978167116642 > 17.821091160178185
    END: th(46415.88833612782)=17.821091160178185; dx=-5.898433325606904E-7 evalInputDelta=0.04888700693845749
    Fitness changed from 17.869978167116642 to 17.821091160178185
    Iteration 14 complete. Error: 17.821091160178185 Total: 4.0069; Orientation: 0.0381; Line Search: 2.6755
    Adding measurement 42019b47 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.163e+05, gradient 1.074e-03, dot -0.731; [518adf61-7a8e-4f71-8691-9a2fecee6893 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 17.821091160178185, 17.904022879898548, 17.904027435928583, 185.29831187427044
    LBFGS Accumulation History: 3 points
    Removed measurement 42019b47 to history. Total: 3
    th(0)=17.821091160178185;dx=-1.1451351170494155E-6
    New Minimum: 17.821091160178185 > 17.645315593108535
    END: th(100000.00000000007)=17.645315593108535; dx=-9.6493057941921E-7 evalInputDelta=0.1757755670696497
    Fitness changed from 17.821091160178185 to 17.645315593108535
    Iteration 15 complete. Error: 17.645315593108535 Total: 3.9643; Orientation: 0.0380; Line Search: 2.5892
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/5f0cfdbe-ff06-4de4-8bfd-6d4a03d3f127.jpg)
    
    Final threshold in iteration 16: 17.645315593108535 (> -Infinity) after 65.581s (< 3600.000s)
    

Returns:

    17.645315593108535