Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Iteration 35 Iteration 35

Iteration 40 Iteration 40

Iteration 45 Iteration 45

Iteration 50 Iteration 50

Code from BasicOptimizer.scala:75 executed in 301.06 seconds (4.192 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 2326056795305
    Reset training subject: 2326964170323
    Adding measurement 1fff3dbf to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.8316777888090856;dx=-4.185461205562743E-8
    New Minimum: 0.8316777888090856 > 0.8316777195220282
    WOLFE (weak): th(2.154434690031884)=0.8316777195220282; dx=-4.1854612971153485E-8 evalInputDelta=6.928705742481611E-8
    New Minimum: 0.8316777195220282 > 0.8316776502347796
    WOLFE (weak): th(4.308869380063768)=0.8316776502347796; dx=-4.185473667532484E-8 evalInputDelta=1.3857430603003706E-7
    New Minimum: 0.8316776502347796 > 0.8316773730864763
    WOLFE (weak): th(12.926608140191302)=0.8316773730864763; dx=-4.185436354412113E-8 evalInputDelta=4.1572260933708804E-7
    New Minimum: 0.8316773730864763 > 0.8316761259169184
    WOLFE (weak): th(51.70643256076521)=0.8316761259169184; dx=-4.185414404761752E-8 evalInputDelta=1.6628921671868468E-6
    New Minimum: 0.8316761259169184 > 0.8316694746943302
    WOLFE (weak): th(258.53216280382605)=0.8316694746943302; dx=-4.1851781123704235E-8 evalInputDelta=8.314114755436641E-6
    New Minimum: 0.8316694746943302 > 0.8316279098541842
    WOLFE (weak): th(1551.1929768229563)=0.8316279098541842; dx=-4.1836399337175344E-8 evalInputDelta=4.987895490138339E-5
    New Minimum: 0.8316279098541842 > 0.8313287694877669
    WOLFE (weak): th(10858.350837760694)=0.8313287694877669; dx=-4.1781915741964094E-8 evalInputDelta=3.4901932131869984E-4
    New Minimum: 0.8313287694877669 > 0.8288825965295638
    WOLFE (weak): th(86866.80670208555)=0.8288825965295638; dx=-4.1459607723351E-8 evalInputDelta=0.002795192279521852
    New Minimum: 0.8288825965295638 > 0.808075498650146
    END: th(781801.26031877)=0.808075498650146; dx=-3.32191989719668E-8 evalInputDelta=0.023602290158939554
    Fitness changed from 0.8316777888090856 to 0.808075498650146
    Iteration 1 complete. Error: 0.808075498650146 Total: 13.8619; Orientation: 0.0678; Line Search: 10.9631
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/f0ff021d-f06a-4846-9e59-b3af4d89e15

...skipping 43416 bytes...

    arch: 3.3550
    Adding measurement 33014f6a to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 7.043e+02, gradient 3.461e-05, dot -0.591; [3496a743-cce5-4593-bd8c-a72ae96e99eb = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.22903698638061218, 0.22783595105313267, 0.22699288195084918, 0.22664594896880616
    LBFGS Accumulation History: 3 points
    Removed measurement 1105d0a7 to history. Total: 4
    th(0)=0.22664594896880616;dx=-1.1981460312517312E-9
    Armijo: th(2.3119861130090192E7)=0.23668773376725513; dx=3.6072782116437015E-10 evalInputDelta=-0.010041784798448972
    Armijo: th(1.1559930565045096E7)=0.22665676279017843; dx=6.488904557611144E-11 evalInputDelta=-1.0813821372274157E-5
    New Minimum: 0.22664594896880616 > 0.22447677295551485
    END: th(3853310.1883483655)=0.22447677295551485; dx=-6.904839659955659E-10 evalInputDelta=0.0021691760132913074
    Fitness changed from 0.22664594896880616 to 0.22447677295551485
    Iteration 49 complete. Error: 0.22447677295551485 Total: 7.0491; Orientation: 1.6307; Line Search: 4.4718
    Adding measurement 7ef4fd5b to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.481e+02, gradient 2.692e-05, dot -0.858; [3496a743-cce5-4593-bd8c-a72ae96e99eb = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.22783595105313267, 0.22699288195084918, 0.22664594896880616, 0.22447677295551485
    LBFGS Accumulation History: 3 points
    Removed measurement 4f0c6558 to history. Total: 4
    th(0)=0.22447677295551485;dx=-7.246160958414041E-10
    New Minimum: 0.22447677295551485 > 0.22318216419001713
    END: th(8301705.141231011)=0.22318216419001713; dx=-3.951854442416296E-10 evalInputDelta=0.0012946087654977234
    Fitness changed from 0.22447677295551485 to 0.22318216419001713
    Iteration 50 complete. Error: 0.22318216419001713 Total: 4.7696; Orientation: 1.6130; Line Search: 2.2097
    <a id="p-29"></a>Iteration 50
    <a id="p-28"></a>![Iteration 50](etc/58edeb87-dd16-4afe-a196-90ba5b9cdc8b.jpg)
    
    Final threshold in iteration 51: 0.22318216419001713 (> -Infinity) after 301.061s (< 3600.000s)
    

Returns:

    0.22318216419001713