Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 60.49 seconds (2.039 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 2114393210087
    Reset training subject: 2115773948427
    Adding measurement 49843c80 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=11.10774253308773;dx=-3.27230207991042E-5
    New Minimum: 11.10774253308773 > 11.107671961188316
    WOLFE (weak): th(2.154434690031884)=11.107671961188316; dx=-3.272484582306559E-5 evalInputDelta=7.05718994140625E-5
    New Minimum: 11.107671961188316 > 11.107601307332516
    WOLFE (weak): th(4.308869380063768)=11.107601307332516; dx=-3.272221076533827E-5 evalInputDelta=1.4122575521469116E-4
    New Minimum: 11.107601307332516 > 11.107318948954344
    WOLFE (weak): th(12.926608140191302)=11.107318948954344; dx=-3.2719378909323544E-5 evalInputDelta=4.2358413338661194E-4
    New Minimum: 11.107318948954344 > 11.10605076700449
    WOLFE (weak): th(51.70643256076521)=11.10605076700449; dx=-3.2712319747543826E-5 evalInputDelta=0.001691766083240509
    New Minimum: 11.10605076700449 > 11.099290080368519
    WOLFE (weak): th(258.53216280382605)=11.099290080368519; dx=-3.26611807634127E-5 evalInputDelta=0.008452452719211578
    New Minimum: 11.099290080368519 > 11.05727032572031
    WOLFE (weak): th(1551.1929768229563)=11.05727032572031; dx=-3.236861168928052E-5 evalInputDelta=0.0504722073674202
    New Minimum: 11.05727032572031 > 10.766674421727657
    WOLFE (weak): th(10858.350837760694)=10.766674421727657; dx=-3.0297598007490645E-5 evalInputDelta=0.3410681113600731
    New Minimum: 10.766674421727657 > 9.01782776787877
    END: th(86866.80670208555)=9.01782776787877; dx=-1.6690873351036443E-5 evalInputDelta=2.0899147652089596
    Fitness changed from 11.10774253308773 to 9.01782776787877
    Iteration 1 complete. Error: 9.01782776787877 Total: 15.7529; Orientation: 0.0034; Line Search: 11.2211
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/1051e6bd-787b-40c6-8bcd-36818f4ab282.jpg)
    
    Adding measurement a8ca765 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=9.01782776787877;dx=-1.2943466862357284E-5
    New Minimum: 9.01782776787877 > 7.2973741143941

...skipping 5537 bytes...

    h: 2.4451
    Adding measurement 443b018a to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 8.786e+02, gradient 6.146e-04, dot -0.695; [21f80312-3c9a-461d-9cc1-b0ca7a9bf470 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 2.847277492284775, 7.297374114394188, 9.01782776787877, 11.10774253308773
    LBFGS Accumulation History: 3 points
    Removed measurement 443b018a to history. Total: 3
    th(0)=2.847277492284775;dx=-3.7776436090813484E-7
    New Minimum: 2.847277492284775 > 2.5531891882419586
    END: th(1120000.0000000005)=2.5531891882419586; dx=-1.8130034889877836E-7 evalInputDelta=0.29408830404281616
    Fitness changed from 2.847277492284775 to 2.5531891882419586
    Iteration 9 complete. Error: 2.5531891882419586 Total: 3.9438; Orientation: 0.0947; Line Search: 2.4818
    Adding measurement 58d45192 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 8.589e+02, gradient 6.909e-04, dot -0.570; [21f80312-3c9a-461d-9cc1-b0ca7a9bf470 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 2.5531891882419586, 7.297374114394188, 9.01782776787877, 11.10774253308773
    LBFGS Accumulation History: 3 points
    Removed measurement 58d45192 to history. Total: 3
    th(0)=2.5531891882419586;dx=-4.774020284781603E-7
    Armijo: th(2412966.852835711)=3.111394226551056; dx=5.694774891639224E-7 evalInputDelta=-0.5582050383090973
    New Minimum: 2.5531891882419586 > 2.5284893959760666
    WOLF (strong): th(1206483.4264178555)=2.5284893959760666; dx=3.125922935293096E-7 evalInputDelta=0.02469979226589203
    New Minimum: 2.5284893959760666 > 2.4300830960273743
    END: th(402161.14213928516)=2.4300830960273743; dx=-1.5211514988938334E-7 evalInputDelta=0.12310609221458435
    Fitness changed from 2.5531891882419586 to 2.4300830960273743
    Iteration 10 complete. Error: 2.4300830960273743 Total: 6.6661; Orientation: 0.2062; Line Search: 5.2004
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/8047109e-d59f-4b52-982e-d1d5b196abf0.jpg)
    
    Final threshold in iteration 11: 2.4300830960273743 (> -Infinity) after 60.494s (< 3600.000s)
    

Returns:

    2.4300830960273743