Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 888.50 seconds (5.812 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1024706753734700
    Reset training subject: 1024714032743400
    Adding measurement 19810310 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=11.281412150710821;dx=-2.991268195013721E-5
    New Minimum: 11.281412150710821 > 11.28133899345994
    WOLFE (weak): th(2.154434690031884)=11.28133899345994; dx=-2.991114652661375E-5 evalInputDelta=7.315725088119507E-5
    New Minimum: 11.28133899345994 > 11.2812654748559
    WOLFE (weak): th(4.308869380063768)=11.2812654748559; dx=-2.9912499343252183E-5 evalInputDelta=1.4667585492134094E-4
    New Minimum: 11.2812654748559 > 11.280971217900515
    WOLFE (weak): th(12.926608140191302)=11.280971217900515; dx=-2.990921525924575E-5 evalInputDelta=4.4093281030654907E-4
    New Minimum: 11.280971217900515 > 11.279645957052708
    WOLFE (weak): th(51.70643256076521)=11.279645957052708; dx=-2.9910602257991887E-5 evalInputDelta=0.0017661936581134796
    New Minimum: 11.279645957052708 > 11.272581908851862
    WOLFE (weak): th(258.53216280382605)=11.272581908851862; dx=-2.9884250937954217E-5 evalInputDelta=0.008830241858959198
    New Minimum: 11.272581908851862 > 11.228577364236116
    WOLFE (weak): th(1551.1929768229563)=11.228577364236116; dx=-2.9679368544132222E-5 evalInputDelta=0.05283478647470474
    New Minimum: 11.228577364236116 > 10.9194574393332
    WOLFE (weak): th(10858.350837760694)=10.9194574393332; dx=-2.8359680808398163E-5 evalInputDelta=0.3619547113776207
    New Minimum: 10.9194574393332 > 9.00361966341734
    END: th(86866.80670208555)=9.00361966341734; dx=-1.6250864508826988E-5 evalInputDelta=2.277792487293482
    Fitness changed from 11.281412150710821 to 9.00361966341734
    Iteration 1 complete. Error: 9.00361966341734 Total: 88.5407; Orientation: 0.0091; Line Search: 66.1713
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/78c9f68c-3730-4a58-a6bd-0b5fb53b739b.jpg)
    
    Adding measurement 1c510921 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=9.00361966341734;dx=-1.195097442027848E-5
    New Minimum: 

...skipping 24369 bytes...

    
    Adding measurement ceebc3b to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.950e+03, gradient 1.254e-04, dot -0.621; [cc05f2ea-ca9d-4dae-89b6-23f4f97b37d9 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -2.335916079580784, -2.3669162318110466, -2.3836401253938675, -2.3986556753516197
    LBFGS Accumulation History: 3 points
    Removed measurement 35555daa to history. Total: 4
    th(0)=-2.3986556753516197;dx=-1.569876668810856E-8
    Armijo: th(1.566894378630859E7)=-2.1736185252666473; dx=1.9401136819406594E-8 evalInputDelta=-0.22503715008497238
    Armijo: th(7834471.893154295)=-2.3666155263781548; dx=1.1949428558975487E-8 evalInputDelta=-0.032040148973464966
    New Minimum: -2.3986556753516197 > -2.4197265952825546
    END: th(2611490.631051432)=-2.4197265952825546; dx=-2.548234699908864E-9 evalInputDelta=0.021070919930934906
    Fitness changed from -2.3986556753516197 to -2.4197265952825546
    Iteration 29 complete. Error: -2.4197265952825546 Total: 37.6097; Orientation: 0.2568; Line Search: 29.9015
    Adding measurement 24e84a4e to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.195e+03, gradient 8.584e-05, dot -0.715; [cc05f2ea-ca9d-4dae-89b6-23f4f97b37d9 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -2.3669162318110466, -2.3836401253938675, -2.3986556753516197, -2.4197265952825546
    LBFGS Accumulation History: 3 points
    Removed measurement 1efd5e0 to history. Total: 4
    th(0)=-2.4197265952825546;dx=-7.3531736823937135E-9
    New Minimum: -2.4197265952825546 > -2.4353460147976875
    END: th(5626286.00823046)=-2.4353460147976875; dx=-2.4884026520828997E-9 evalInputDelta=0.015619419515132904
    Fitness changed from -2.4197265952825546 to -2.4353460147976875
    Iteration 30 complete. Error: -2.4353460147976875 Total: 22.4024; Orientation: 0.2622; Line Search: 14.8505
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/2847479f-5758-4d39-81c3-29b2653b3805.jpg)
    
    Final threshold in iteration 31: -2.4353460147976875 (> -Infinity) after 888.500s (< 3600.000s)
    

Returns:

    -2.4353460147976875