Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 3725.61 seconds (25.044 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1287819981930700
    Reset training subject: 1287874810755200
    Adding measurement 46a7a069 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.4310736118205991;dx=-3.1420256173564014E-7
    New Minimum: 1.4310736118205991 > 1.4310734749477807
    WOLFE (weak): th(2.154434690031884)=1.4310734749477807; dx=-3.0997884782990406E-7 evalInputDelta=1.3687281841257004E-7
    New Minimum: 1.4310734749477807 > 1.4310732093919596
    WOLFE (weak): th(4.308869380063768)=1.4310732093919596; dx=-3.099786165927042E-7 evalInputDelta=4.024286395232224E-7
    New Minimum: 1.4310732093919596 > 1.4310721997541738
    WOLFE (weak): th(12.926608140191302)=1.4310721997541738; dx=-3.09978550724104E-7 evalInputDelta=1.4120664253436388E-6
    New Minimum: 1.4310721997541738 > 1.4310679607925296
    WOLFE (weak): th(51.70643256076521)=1.4310679607925296; dx=-3.099776402296735E-7 evalInputDelta=5.6510280694865855E-6
    New Minimum: 1.4310679607925296 > 1.431045578995119
    WOLFE (weak): th(258.53216280382605)=1.431045578995119; dx=-3.099730757837409E-7 evalInputDelta=2.8032825480028833E-5
    New Minimum: 1.431045578995119 > 1.4309058334654754
    WOLFE (weak): th(1551.1929768229563)=1.4309058334654754; dx=-3.099451859989998E-7 evalInputDelta=1.6777835512371553E-4
    New Minimum: 1.4309058334654754 > 1.4299006260910823
    WOLFE (weak): th(10858.350837760694)=1.4299006260910823; dx=-3.0972521756284193E-7 evalInputDelta=0.001172985729516851
    New Minimum: 1.4299006260910823 > 1.4217366269375522
    WOLFE (weak): th(86866.80670208555)=1.4217366269375522; dx=-3.0795164885421097E-7 evalInputDelta=0.009336984883046906
    New Minimum: 1.4217366269375522 > 1.3510456743312798
    WOLFE (weak): th(781801.26031877)=1.3510456743312798; dx=-2.90150727275436E-7 evalInputDelta=0.08002793748931936
    New Minimum: 1.3510456743312798 > 0.9666616217355293
    END: th(7818012.6031877)=0.9666616217355293; dx=-1.5257902006064346E-7 evalInputDelta=0.46441199008506984
    Fitness changed from 1.4310736118205991 to 0.9666616217355293
    I

...skipping 10981 bytes...

    omplete. Error: 0.07405409555750841 Total: 267.2914; Orientation: 1.4742; Line Search: 211.9384
    Adding measurement 70d18b82 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.235e+03, gradient 1.338e-04, dot -0.858; [305b5a42-9760-4d75-b103-db698fdeadaa = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.17130277161578644, 0.13855775432468476, 0.12269722973110575, 0.07405409555750841
    LBFGS Accumulation History: 3 points
    Removed measurement 4660248 to history. Total: 4
    th(0)=0.07405409555750841;dx=-1.7909023356912456E-8
    New Minimum: 0.07405409555750841 > 0.03890452861388338
    END: th(1.827625603235033E7)=0.03890452861388338; dx=-1.1718067265206038E-8 evalInputDelta=0.03514956694362503
    Fitness changed from 0.07405409555750841 to 0.03890452861388338
    Iteration 14 complete. Error: 0.03890452861388338 Total: 163.9284; Orientation: 1.4906; Line Search: 109.1542
    Adding measurement 6250d1d3 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.308e+04, gradient 1.237e-04, dot -0.871; [305b5a42-9760-4d75-b103-db698fdeadaa = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.13855775432468476, 0.12269722973110575, 0.07405409555750841, 0.03890452861388338
    LBFGS Accumulation History: 3 points
    Removed measurement 5c80bd18 to history. Total: 4
    th(0)=0.03890452861388338;dx=-1.5298185617711855E-8
    Armijo: th(3.937500000000003E7)=0.06340610232966049; dx=-6.706829138525364E-9 evalInputDelta=-0.024501573715777103
    New Minimum: 0.03890452861388338 > 0.010173863788961124
    END: th(1.9687500000000015E7)=0.010173863788961124; dx=-9.875533382077116E-9 evalInputDelta=0.02873066482492226
    Fitness changed from 0.03890452861388338 to 0.010173863788961124
    Iteration 15 complete. Error: 0.010173863788961124 Total: 214.0910; Orientation: 1.4800; Line Search: 159.9134
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/fda1593b-3b16-4889-9849-9bc2976e6b43.jpg)
    
    Final threshold in iteration 15: 0.010173863788961124 (> -Infinity) after 3725.608s (< 3600.000s)
    

Returns:

    0.010173863788961124