Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:75 executed in 4317.28 seconds (49.839 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1587623284471800
    Reset training subject: 1587672366547100
    Adding measurement 3368cf02 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=4.651180795170826;dx=-2.64373281575794E-6
    New Minimum: 4.651180795170826 > 4.6511804305776225
    WOLFE (weak): th(2.154434690031884)=4.6511804305776225; dx=-2.6326715692917426E-6 evalInputDelta=3.6459320362780545E-7
    New Minimum: 4.6511804305776225 > 4.651179984971616
    WOLFE (weak): th(4.308869380063768)=4.651179984971616; dx=-2.6326751640870004E-6 evalInputDelta=8.101992099440736E-7
    New Minimum: 4.651179984971616 > 4.651177409944322
    WOLFE (weak): th(12.926608140191302)=4.651177409944322; dx=-2.632670254313605E-6 evalInputDelta=3.385226504448724E-6
    New Minimum: 4.651177409944322 > 4.651163935659245
    WOLFE (weak): th(51.70643256076521)=4.651163935659245; dx=-2.632648853117102E-6 evalInputDelta=1.685951158147958E-5
    New Minimum: 4.651163935659245 > 4.651099498528243
    WOLFE (weak): th(258.53216280382605)=4.651099498528243; dx=-2.6323558034414395E-6 evalInputDelta=8.129664258316183E-5
    New Minimum: 4.651099498528243 > 4.6506951815468875
    WOLFE (weak): th(1551.1929768229563)=4.6506951815468875; dx=-2.631536272316224E-6 evalInputDelta=4.8561362393861884E-4
    New Minimum: 4.6506951815468875 > 4.647786989663715
    WOLFE (weak): th(10858.350837760694)=4.647786989663715; dx=-2.626066402069887E-6 evalInputDelta=0.0033938055071107698
    New Minimum: 4.647786989663715 > 4.624218764723517
    WOLFE (weak): th(86866.80670208555)=4.624218764723517; dx=-2.5782036687252816E-6 evalInputDelta=0.026962030447308827
    New Minimum: 4.624218764723517 > 4.4273147303097815
    END: th(781801.26031877)=4.4273147303097815; dx=-2.1186068825555186E-6 evalInputDelta=0.22386606486104466
    Fitness changed from 4.651180795170826 to 4.4273147303097815
    Iteration 1 complete. Error: 4.4273147303097815 Total: 640.7964; Orientation: 0.1272; Line Search: 494.5431
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/03073c34-e47c-40d

...skipping 14996 bytes...

    847; Line Search: 201.9357
    Adding measurement 55448710 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.176e+04, gradient 4.904e-04, dot -0.743; [672e037d-ba60-4eae-8d6c-0bee1e13fe68 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 1.866343596692382, 1.8184638968840867, 1.7924714956069414, 1.7576350015135456
    LBFGS Accumulation History: 3 points
    Removed measurement c925ae1 to history. Total: 4
    th(0)=1.7576350015135456;dx=-2.4048732812949863E-7
    New Minimum: 1.7576350015135456 > 1.7340797920042714
    END: th(4189178.563950887)=1.7340797920042714; dx=-5.558555636996063E-8 evalInputDelta=0.023555209509274144
    Fitness changed from 1.7576350015135456 to 1.7340797920042714
    Iteration 19 complete. Error: 1.7340797920042714 Total: 152.5009; Orientation: 3.8637; Line Search: 100.4793
    Adding measurement 1f5aac08 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 7.087e+03, gradient 6.069e-04, dot -0.609; [672e037d-ba60-4eae-8d6c-0bee1e13fe68 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 1.8184638968840867, 1.7924714956069414, 1.7576350015135456, 1.7340797920042714
    LBFGS Accumulation History: 3 points
    Removed measurement 25d5327 to history. Total: 4
    th(0)=1.7340797920042714;dx=-3.6836791529359794E-7
    Armijo: th(9025311.620913742)=1.869525571727671; dx=1.1854909418029562E-7 evalInputDelta=-0.13544577972339966
    Armijo: th(4512655.810456871)=1.7436440432513505; dx=8.305254073542306E-8 evalInputDelta=-0.009564251247079048
    New Minimum: 1.7340797920042714 > 1.708555993557053
    END: th(1504218.6034856236)=1.708555993557053; dx=-1.194498413619127E-7 evalInputDelta=0.025523798447218526
    Fitness changed from 1.7340797920042714 to 1.708555993557053
    Iteration 20 complete. Error: 1.708555993557053 Total: 251.6519; Orientation: 3.7313; Line Search: 198.8686
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/f39c1f4e-04ac-464c-915d-3c994b327d70.jpg)
    
    Final threshold in iteration 21: 1.708555993557053 (> -Infinity) after 4317.270s (< 5400.000s)
    

Returns:

    1.708555993557053