Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 695.13 seconds (6.507 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 16509532034526
    Reset training subject: 16523309504645
    Adding measurement 5d714e79 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.0931093618273735;dx=-5.515449431849722E-8
    New Minimum: 1.0931093618273735 > 1.093109093606472
    WOLFE (weak): th(2.154434690031884)=1.093109093606472; dx=-5.515450625140043E-8 evalInputDelta=2.682209014892578E-7
    New Minimum: 1.093109093606472 > 1.0931089743971825
    WOLFE (weak): th(4.308869380063768)=1.0931089743971825; dx=-5.515450474842072E-8 evalInputDelta=3.8743019104003906E-7
    New Minimum: 1.0931089743971825 > 1.0931084379553795
    WOLFE (weak): th(12.926608140191302)=1.0931084379553795; dx=-5.5154378523536004E-8 evalInputDelta=9.238719940185547E-7
    New Minimum: 1.0931084379553795 > 1.0931059271097183
    WOLFE (weak): th(51.70643256076521)=1.0931059271097183; dx=-5.5154219841657166E-8 evalInputDelta=3.4347176551818848E-6
    New Minimum: 1.0931059271097183 > 1.0930930078029633
    WOLFE (weak): th(258.53216280382605)=1.0930930078029633; dx=-5.5153068765243194E-8 evalInputDelta=1.6354024410247803E-5
    New Minimum: 1.0930930078029633 > 1.093011274933815
    WOLFE (weak): th(1551.1929768229563)=1.093011274933815; dx=-5.514792532207449E-8 evalInputDelta=9.80868935585022E-5
    New Minimum: 1.093011274933815 > 1.0924238413572311
    WOLFE (weak): th(10858.350837760694)=1.0924238413572311; dx=-5.510372139883789E-8 evalInputDelta=6.855204701423645E-4
    New Minimum: 1.0924238413572311 > 1.087644837796688
    WOLFE (weak): th(86866.80670208555)=1.087644837796688; dx=-5.473615596119606E-8 evalInputDelta=0.005464524030685425
    New Minimum: 1.087644837796688 > 1.0455980524420738
    WOLFE (weak): th(781801.26031877)=1.0455980524420738; dx=-5.132431011229362E-8 evalInputDelta=0.04751130938529968
    New Minimum: 1.0455980524420738 > 0.787270113825798
    END: th(7818012.6031877)=0.787270113825798; dx=-1.869474082582425E-8 evalInputDelta=0.30583924800157547
    Fitness changed from 1.0931093618273735 to 0.787270113825798
    Iteration 1 complete. Error: 0.7872701138257

...skipping 5962 bytes...

    51841950416565; dx=-5.056487209028988E-9 evalInputDelta=0.0756581723690033
    Fitness changed from -0.2195260226726532 to -0.2951841950416565
    Iteration 8 complete. Error: -0.2951841950416565 Total: 44.7489; Orientation: 1.6331; Line Search: 29.4843
    Adding measurement 3fa9fb6a to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.013e+04, gradient 9.719e-05, dot -0.685; [64d6e027-9fd3-4b1d-859b-f0540566520b = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.2951841950416565, 0.5203494131565094, 0.787270113825798, 1.0931093618273735
    LBFGS Accumulation History: 3 points
    Removed measurement 3fa9fb6a to history. Total: 3
    th(0)=-0.2951841950416565;dx=-9.446720287026281E-9
    New Minimum: -0.2951841950416565 > -0.37451016902923584
    END: th(1.2600000000000004E7)=-0.37451016902923584; dx=-4.825819837814562E-9 evalInputDelta=0.07932597398757935
    Fitness changed from -0.2951841950416565 to -0.37451016902923584
    Iteration 9 complete. Error: -0.37451016902923584 Total: 42.9616; Orientation: 1.6196; Line Search: 27.6976
    Adding measurement 52f21c0c to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.124e+04, gradient 9.855e-05, dot -0.550; [64d6e027-9fd3-4b1d-859b-f0540566520b = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.37451016902923584, 0.5203494131565094, 0.787270113825798, 1.0931093618273735
    LBFGS Accumulation History: 3 points
    Removed measurement 52f21c0c to history. Total: 3
    th(0)=-0.37451016902923584;dx=-9.711915242401276E-9
    New Minimum: -0.37451016902923584 > -0.4137292951345444
    END: th(2.7145877094401743E7)=-0.4137292951345444; dx=-2.4326982863823853E-9 evalInputDelta=0.03921912610530853
    Fitness changed from -0.37451016902923584 to -0.4137292951345444
    Iteration 10 complete. Error: -0.4137292951345444 Total: 42.8902; Orientation: 1.6293; Line Search: 27.9479
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/e95f5ffc-c7f9-4d99-ba6a-dc87855ce890.jpg)
    
    Final threshold in iteration 11: -0.4137292951345444 (> -Infinity) after 695.125s (< 3600.000s)
    

Returns:

    -0.4137292951345444