Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 51.83 seconds (2.141 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 1930972495920
    Reset training subject: 1932369144536
    Adding measurement 146c5463 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=32.98892036592588;dx=-0.005125102434027751
    New Minimum: 32.98892036592588 > 32.9778825417161
    WOLFE (weak): th(2.154434690031884)=32.9778825417161; dx=-0.005122568680310094 evalInputDelta=0.011037824209779501
    New Minimum: 32.9778825417161 > 32.96684611681849
    WOLFE (weak): th(4.308869380063768)=32.96684611681849; dx=-0.005122354046776467 evalInputDelta=0.022074249107390642
    New Minimum: 32.96684611681849 > 32.923143356107175
    WOLFE (weak): th(12.926608140191302)=32.923143356107175; dx=-0.005113651909608608 evalInputDelta=0.06577700981870294
    New Minimum: 32.923143356107175 > 32.72464380413294
    WOLFE (weak): th(51.70643256076521)=32.72464380413294; dx=-0.005061948050139393 evalInputDelta=0.2642765617929399
    New Minimum: 32.72464380413294 > 31.71752834226936
    WOLFE (weak): th(258.53216280382605)=31.71752834226936; dx=-0.004915619782515112 evalInputDelta=1.2713920236565173
    New Minimum: 31.71752834226936 > 27.26048848964274
    END: th(1551.1929768229563)=27.26048848964274; dx=-0.002455968652736792 evalInputDelta=5.728431876283139
    Fitness changed from 32.98892036592588 to 27.26048848964274
    Iteration 1 complete. Error: 27.26048848964274 Total: 13.6062; Orientation: 0.0035; Line Search: 8.9534
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/88652e67-93fb-4dd6-814f-d66f4781c85e.jpg)
    
    Adding measurement 76460f85 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=27.26048848964274;dx=-0.0015577971137451067
    New Minimum: 27.26048848964274 > 23.67448614910245
    END: th(3341.943960201201)=23.67448614910245; dx=-7.070270650906591E-4 evalInputDelta=3.58600234054029
    Fitness changed from 27.26048848964274 to 23.67448614910245
    Iteration 2 complete. Error: 23.67448614910245 Total: 3.7160; Orientation: 0.0035; Line Search: 2.4395
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/058e1432-a824-402

...skipping 4488 bytes...

    a=2.654758097603917
    Fitness changed from 13.305667037144303 to 10.650908939540386
    Iteration 8 complete. Error: 10.650908939540386 Total: 5.5395; Orientation: 0.0741; Line Search: 4.2280
    Adding measurement 6d78ebf2 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.329e+02, gradient 4.958e-03, dot -0.296; [6211d2f7-88b2-4804-91fa-67c21092b30c = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 10.650908939540386, 23.67448614910245, 27.26048848964274, 32.98892036592588
    LBFGS Accumulation History: 3 points
    Removed measurement 6d78ebf2 to history. Total: 3
    th(0)=10.650908939540386;dx=-2.4580200844006985E-5
    New Minimum: 10.650908939540386 > 7.223209850490093
    WOLF (strong): th(720000.0000000003)=7.223209850490093; dx=1.1949760394109698E-7 evalInputDelta=3.427699089050293
    END: th(360000.0000000002)=7.546474426984787; dx=-2.1133701020471032E-6 evalInputDelta=3.104434512555599
    Fitness changed from 10.650908939540386 to 7.223209850490093
    Iteration 9 complete. Error: 7.223209850490093 Total: 5.2931; Orientation: 0.0767; Line Search: 3.9255
    Adding measurement 661c8685 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.308e+02, gradient 2.884e-03, dot -0.203; [6211d2f7-88b2-4804-91fa-67c21092b30c = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 7.223209850490093, 23.67448614910245, 27.26048848964274, 32.98892036592588
    LBFGS Accumulation History: 3 points
    Removed measurement 661c8685 to history. Total: 3
    th(0)=7.223209850490093;dx=-8.316779519236574E-6
    New Minimum: 7.223209850490093 > 5.010092280805111
    END: th(775596.4884114786)=5.010092280805111; dx=-1.9115103520035718E-7 evalInputDelta=2.2131175696849823
    Fitness changed from 7.223209850490093 to 5.010092280805111
    Iteration 10 complete. Error: 5.010092280805111 Total: 4.0356; Orientation: 0.1376; Line Search: 2.5696
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/0e1cb354-3a92-4e43-8888-5c4f8753f34c.jpg)
    
    Final threshold in iteration 11: 5.010092280805111 (> -Infinity) after 51.826s (< 3600.000s)
    

Returns:

    5.010092280805111