Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:75 executed in 978.97 seconds (2.904 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 829442285174800
    Reset training subject: 829452633033300
    Adding measurement ac4915e to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=3.5570440123116462;dx=-6.45291267304736E-8
    New Minimum: 3.5570440123116462 > 3.557043919423275
    WOLFE (weak): th(2.154434690031884)=3.557043919423275; dx=-6.4529096688824E-8 evalInputDelta=9.288837121701476E-8
    WOLFE (weak): th(4.308869380063768)=3.557043919423275; dx=-6.45292056981619E-8 evalInputDelta=9.288837121701476E-8
    New Minimum: 3.557043919423275 > 3.557043498606747
    WOLFE (weak): th(12.926608140191302)=3.557043498606747; dx=-6.45291870374763E-8 evalInputDelta=5.137048990633275E-7
    New Minimum: 3.557043498606747 > 3.5570416071047197
    WOLFE (weak): th(51.70643256076521)=3.5570416071047197; dx=-6.452919970432402E-8 evalInputDelta=2.4052069265145803E-6
    New Minimum: 3.5570416071047197 > 3.557032076349186
    WOLFE (weak): th(258.53216280382605)=3.557032076349186; dx=-6.452900789782452E-8 evalInputDelta=1.1935962460185578E-5
    New Minimum: 3.557032076349186 > 3.5569732300924803
    WOLFE (weak): th(1551.1929768229563)=3.5569732300924803; dx=-6.452722352680744E-8 evalInputDelta=7.078221916589911E-5
    New Minimum: 3.5569732300924803 > 3.556548325780826
    WOLFE (weak): th(10858.350837760694)=3.556548325780826; dx=-6.451094249172678E-8 evalInputDelta=4.956865308201053E-4
    New Minimum: 3.556548325780826 > 3.553078964844167
    WOLFE (weak): th(86866.80670208555)=3.553078964844167; dx=-6.437397808547635E-8 evalInputDelta=0.003965047467479277
    New Minimum: 3.553078964844167 > 3.521426083506656
    WOLFE (weak): th(781801.26031877)=3.521426083506656; dx=-6.307850729063065E-8 evalInputDelta=0.03561792880499004
    New Minimum: 3.521426083506656 > 3.216086944417034
    END: th(7818012.6031877)=3.216086944417034; dx=-5.0605168974516934E-8 evalInputDelta=0.3409570678946121
    Fitness changed from 3.5570440123116462 to 3.216086944417034
    Iteration 1 complete. Error: 3.216086944417034 Total: 146.2701; Orientation: 0.0982; Lin

...skipping 15554 bytes...

    earch: 31.2114
    Adding measurement 5885e231 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.742e+04, gradient 1.185e-04, dot -0.933; [77052347-53c4-4efb-b373-6ea19e0058f4 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 1.6028509158156274, 1.595749785487695, 1.5758164280359275, 1.5614950867227044
    LBFGS Accumulation History: 3 points
    Removed measurement 7845b21a to history. Total: 4
    th(0)=1.5614950867227044;dx=-1.4039916188071925E-8
    Armijo: th(4.7128258844447486E7)=1.6372517190352616; dx=-1.1188612967011495E-8 evalInputDelta=-0.07575663231255714
    New Minimum: 1.5614950867227044 > 1.5516369897266364
    END: th(2.3564129422223743E7)=1.5516369897266364; dx=-1.1991600698265757E-8 evalInputDelta=0.009858096996067989
    Fitness changed from 1.5614950867227044 to 1.5516369897266364
    Iteration 19 complete. Error: 1.5516369897266364 Total: 44.5897; Orientation: 3.1359; Line Search: 31.2990
    Adding measurement 2e29f28e to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.641e+04, gradient 1.179e-04, dot -0.924; [77052347-53c4-4efb-b373-6ea19e0058f4 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 1.595749785487695, 1.5758164280359275, 1.5614950867227044, 1.5516369897266364
    LBFGS Accumulation History: 3 points
    Removed measurement 3212d853 to history. Total: 4
    th(0)=1.5516369897266364;dx=-1.3904894595377348E-8
    Armijo: th(5.07673778676398E7)=1.655231684240696; dx=-1.0922255891339168E-8 evalInputDelta=-0.10359469451405956
    New Minimum: 1.5516369897266364 > 1.5477036915000124
    END: th(2.53836889338199E7)=1.5477036915000124; dx=-1.1713363092062317E-8 evalInputDelta=0.003933298226624027
    Fitness changed from 1.5516369897266364 to 1.5477036915000124
    Iteration 20 complete. Error: 1.5477036915000124 Total: 43.9156; Orientation: 2.7099; Line Search: 31.0875
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/40c96693-077d-4754-997e-772265cd94b1.jpg)
    
    Final threshold in iteration 21: 1.5477036915000124 (> -Infinity) after 978.974s (< 3600.000s)
    

Returns:

    1.5477036915000124