Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:88 executed in 287.88 seconds (5.378 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 863758416874
    Reset training subject: 866903126918
    Adding measurement 54877260 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=13.669359520077705;dx=-6.398112302717962E-7
    New Minimum: 13.669359520077705 > 13.669358305633068
    WOLFE (weak): th(2.154434690031884)=13.669358305633068; dx=-6.224739066085761E-7 evalInputDelta=1.214444637298584E-6
    New Minimum: 13.669358305633068 > 13.669356912374496
    WOLFE (weak): th(4.308869380063768)=13.669356912374496; dx=-6.224534590815456E-7 evalInputDelta=2.60770320892334E-6
    New Minimum: 13.669356912374496 > 13.669352188706398
    WOLFE (weak): th(12.926608140191302)=13.669352188706398; dx=-6.219442869696115E-7 evalInputDelta=7.331371307373047E-6
    New Minimum: 13.669352188706398 > 13.669331356883049
    WOLFE (weak): th(51.70643256076521)=13.669331356883049; dx=-6.228592784187855E-7 evalInputDelta=2.816319465637207E-5
    New Minimum: 13.669331356883049 > 13.669218197464943
    WOLFE (weak): th(258.53216280382605)=13.669218197464943; dx=-6.231253144227222E-7 evalInputDelta=1.4132261276245117E-4
    New Minimum: 13.669218197464943 > 13.668513432145119
    WOLFE (weak): th(1551.1929768229563)=13.668513432145119; dx=-6.224186982732449E-7 evalInputDelta=8.460879325866699E-4
    New Minimum: 13.668513432145119 > 13.66343966126442
    WOLFE (weak): th(10858.350837760694)=13.66343966126442; dx=-6.220068122773089E-7 evalInputDelta=0.005919858813285828
    New Minimum: 13.66343966126442 > 13.622385881841183
    WOLFE (weak): th(86866.80670208555)=13.622385881841183; dx=-6.144073621215866E-7 evalInputDelta=0.046973638236522675
    New Minimum: 13.622385881841183 > 13.27863471955061
    END: th(781801.26031877)=13.27863471955061; dx=-5.405325919679775E-7 evalInputDelta=0.3907248005270958
    Fitness changed from 13.669359520077705 to 13.27863471955061
    Iteration 1 complete. Error: 13.27863471955061 Total: 44.6048; Orientation: 0.0082; Line Search: 34.9345
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/728cc19f-e454-469c-a251-c38375abc4ac.jpg)
    
    Addi

...skipping 14534 bytes...

    84732708334923 Total: 13.2609; Orientation: 0.1509; Line Search: 9.7978
    Adding measurement 31f4ae1d to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.425e+04, gradient 2.683e-04, dot -0.104; [ec5b4336-129b-4744-a7e9-245c83708643 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 4.884732708334923, 12.569645300507545, 13.27863471955061, 13.669359520077705
    LBFGS Accumulation History: 3 points
    Removed measurement 31f4ae1d to history. Total: 3
    th(0)=4.884732708334923;dx=-7.101607842807385E-8
    Armijo: th(4.241543296000274E7)=6.955566816031933; dx=-3.584197214973707E-9 evalInputDelta=-2.07083410769701
    Armijo: th(2.120771648000137E7)=4.961918592453003; dx=-2.22578840276236E-8 evalInputDelta=-0.07718588411808014
    New Minimum: 4.884732708334923 > 4.606091678142548
    END: th(7069238.8266671235)=4.606091678142548; dx=-4.682440521098473E-8 evalInputDelta=0.2786410301923752
    Fitness changed from 4.884732708334923 to 4.606091678142548
    Iteration 19 complete. Error: 4.606091678142548 Total: 17.2832; Orientation: 0.1537; Line Search: 13.0915
    Adding measurement 5b631c03 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.942e+04, gradient 2.425e-04, dot -0.129; [ec5b4336-129b-4744-a7e9-245c83708643 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 4.606091678142548, 12.569645300507545, 13.27863471955061, 13.669359520077705
    LBFGS Accumulation History: 3 points
    Removed measurement 5b631c03 to history. Total: 3
    th(0)=4.606091678142548;dx=-5.809966987833873E-8
    New Minimum: 4.606091678142548 > 4.499795824289322
    END: th(1.5230213360291943E7)=4.499795824289322; dx=-2.858116131267633E-8 evalInputDelta=0.10629585385322571
    Fitness changed from 4.606091678142548 to 4.499795824289322
    Iteration 20 complete. Error: 4.499795824289322 Total: 10.1486; Orientation: 0.2031; Line Search: 6.5400
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/854500a3-17f1-4730-82d7-483a8844534a.jpg)
    
    Final threshold in iteration 21: 4.499795824289322 (> -Infinity) after 287.882s (< 3600.000s)
    

Returns:

    4.499795824289322