Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 3793.29 seconds (32.629 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1275800746566100
    Reset training subject: 1275854833519900
    Adding measurement 5bb37521 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.2533806667720517;dx=-2.670675090283578E-7
    New Minimum: 1.2533806667720517 > 1.253380567891372
    WOLFE (weak): th(2.154434690031884)=1.253380567891372; dx=-2.639355106177119E-7 evalInputDelta=9.888067964425318E-8
    New Minimum: 1.253380567891372 > 1.2533803049440175
    WOLFE (weak): th(4.308869380063768)=1.2533803049440175; dx=-2.63935692846688E-7 evalInputDelta=3.6182803420103937E-7
    New Minimum: 1.2533803049440175 > 1.253379604394867
    WOLFE (weak): th(12.926608140191302)=1.253379604394867; dx=-2.6393545433134584E-7 evalInputDelta=1.062377184668506E-6
    New Minimum: 1.253379604394867 > 1.2533760987982518
    WOLFE (weak): th(51.70643256076521)=1.2533760987982518; dx=-2.639349139373571E-7 evalInputDelta=4.567973799884939E-6
    New Minimum: 1.2533760987982518 > 1.2533573986864732
    WOLFE (weak): th(258.53216280382605)=1.2533573986864732; dx=-2.639292620606916E-7 evalInputDelta=2.3268085578509456E-5
    New Minimum: 1.2533573986864732 > 1.2532408260473589
    WOLFE (weak): th(1551.1929768229563)=1.2532408260473589; dx=-2.638837030530821E-7 evalInputDelta=1.3984072469286346E-4
    New Minimum: 1.2532408260473589 > 1.2524018572337892
    WOLFE (weak): th(10858.350837760694)=1.2524018572337892; dx=-2.636932196260915E-7 evalInputDelta=9.788095382625706E-4
    New Minimum: 1.2524018572337892 > 1.2455927219759881
    WOLFE (weak): th(86866.80670208555)=1.2455927219759881; dx=-2.620752583117629E-7 evalInputDelta=0.007787944796063595
    New Minimum: 1.2455927219759881 > 1.1867175242224772
    WOLFE (weak): th(781801.26031877)=1.1867175242224772; dx=-2.475238499321444E-7 evalInputDelta=0.0666631425495745
    New Minimum: 1.1867175242224772 > 0.8748478736389947
    END: th(7818012.6031877)=0.8748478736389947; dx=-1.336467718313745E-7 evalInputDelta=0.378532793133057
    Fitness changed from 1.2533806667720517 to 0.8748478736389947
    Iteration 1 

...skipping 10974 bytes...

     measurement 7af48227 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.270e+04, gradient 1.341e-04, dot -0.795; [21632e19-c634-4c88-982d-31c36fd04558 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.2468390854377357, 0.1742975566107158, 0.12954589580803846, 0.09726602509752774
    LBFGS Accumulation History: 3 points
    Removed measurement 2feea37d to history. Total: 4
    th(0)=0.09726602509752774;dx=-1.799262863346557E-8
    Armijo: th(5.482876809705099E7)=0.2032779182757248; dx=-5.156767912193799E-9 evalInputDelta=-0.10601189317819706
    New Minimum: 0.09726602509752774 > 0.07098144937626681
    END: th(2.7414384048525494E7)=0.07098144937626681; dx=-9.091865194361313E-9 evalInputDelta=0.02628457572126093
    Fitness changed from 0.09726602509752774 to 0.07098144937626681
    Iteration 14 complete. Error: 0.07098144937626681 Total: 220.3967; Orientation: 1.4949; Line Search: 162.9585
    Adding measurement 5c6d0169 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.098e+04, gradient 1.314e-04, dot -0.761; [21632e19-c634-4c88-982d-31c36fd04558 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.1742975566107158, 0.12954589580803846, 0.09726602509752774, 0.07098144937626681
    LBFGS Accumulation History: 3 points
    Removed measurement 65c7a786 to history. Total: 4
    th(0)=0.07098144937626681;dx=-1.727576638028554E-8
    Armijo: th(5.9062500000000045E7)=0.2230245688344899; dx=-3.323436224215069E-9 evalInputDelta=-0.15204311945822307
    New Minimum: 0.07098144937626681 > 0.05639626123057384
    END: th(2.9531250000000022E7)=0.05639626123057384; dx=-7.58036876356739E-9 evalInputDelta=0.01458518814569297
    Fitness changed from 0.07098144937626681 to 0.05639626123057384
    Iteration 15 complete. Error: 0.05639626123057384 Total: 221.0744; Orientation: 1.4961; Line Search: 165.9329
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/8a0a8bf5-9ecc-45c5-aad7-6d72550697b6.jpg)
    
    Final threshold in iteration 15: 0.05639626123057384 (> -Infinity) after 3793.282s (< 3600.000s)
    

Returns:

    0.05639626123057384