Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 1015.17 seconds (12.816 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1154913436807200
    Reset training subject: 1154923326595200
    Adding measurement 45424c0 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=5.221549216513866;dx=-2.6453215830534575E-5
    New Minimum: 5.221549216513866 > 5.221524360746223
    WOLFE (weak): th(2.154434690031884)=5.221524360746223; dx=-2.645311384849587E-5 evalInputDelta=2.4855767643217064E-5
    New Minimum: 5.221524360746223 > 5.221499557916017
    WOLFE (weak): th(4.308869380063768)=5.221499557916017; dx=-2.6451500441564295E-5 evalInputDelta=4.9658597848889485E-5
    New Minimum: 5.221499557916017 > 5.221400459390101
    WOLFE (weak): th(12.926608140191302)=5.221400459390101; dx=-2.645047999072792E-5 evalInputDelta=1.4875712376483108E-4
    New Minimum: 5.221400459390101 > 5.220954267812088
    WOLFE (weak): th(51.70643256076521)=5.220954267812088; dx=-2.6443253972472902E-5 evalInputDelta=5.949487017780086E-4
    New Minimum: 5.220954267812088 > 5.218576632752099
    WOLFE (weak): th(258.53216280382605)=5.218576632752099; dx=-2.6413600264002657E-5 evalInputDelta=0.0029725837617666784
    New Minimum: 5.218576632752099 > 5.204266047747687
    WOLFE (weak): th(1551.1929768229563)=5.204266047747687; dx=-2.6218715204014246E-5 evalInputDelta=0.017283168766178747
    New Minimum: 5.204266047747687 > 5.101153396026174
    WOLFE (weak): th(10858.350837760694)=5.101153396026174; dx=-2.47263329296769E-5 evalInputDelta=0.12039582048769226
    New Minimum: 5.101153396026174 > 4.461824922952173
    END: th(86866.80670208555)=4.461824922952173; dx=-1.5220067431691656E-5 evalInputDelta=0.7597242935616926
    Fitness changed from 5.221549216513866 to 4.461824922952173
    Iteration 1 complete. Error: 4.461824922952173 Total: 99.3304; Orientation: 0.0068; Line Search: 73.3346
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/9817dd40-e81c-4866-b4f7-39e3f23e1434.jpg)
    
    Adding measurement 11c3ce6a to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=4.461824922952173;dx=-1.1242766291425677E-5
    New Minimum: 4

...skipping 23846 bytes...

    6e+03, gradient 1.999e-04, dot -0.733; [d7045387-4e5c-4524-99f7-9eed46a9b7c5 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.33716202111107474, 0.33031791078209116, 0.30390035667103854, 0.2885194835893412
    LBFGS Accumulation History: 3 points
    Removed measurement 54f25b42 to history. Total: 4
    th(0)=0.2885194835893412;dx=-3.9976776990840576E-8
    Armijo: th(9285300.021516202)=0.34392079915904294; dx=8.261817956875757E-9 evalInputDelta=-0.05540131556970174
    New Minimum: 0.2885194835893412 > 0.2797715171177132
    END: th(4642650.010758101)=0.2797715171177132; dx=-9.491790961969544E-9 evalInputDelta=0.00874796647162801
    Fitness changed from 0.2885194835893412 to 0.2797715171177132
    Iteration 29 complete. Error: 0.2797715171177132 Total: 34.4447; Orientation: 0.1963; Line Search: 25.5957
    Adding measurement 311a09b2 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.622e+03, gradient 2.422e-04, dot -0.585; [d7045387-4e5c-4524-99f7-9eed46a9b7c5 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.33031791078209116, 0.30390035667103854, 0.2885194835893412, 0.2797715171177132
    LBFGS Accumulation History: 3 points
    Removed measurement 27dc81be to history. Total: 4
    th(0)=0.2797715171177132;dx=-5.866518534854752E-8
    Armijo: th(1.000228623685415E7)=0.45988405174847147; dx=5.325125946995461E-8 evalInputDelta=-0.18011253463075827
    Armijo: th(5001143.118427075)=0.2908371165153625; dx=1.6240499484075004E-8 evalInputDelta=-0.011065599397649284
    New Minimum: 0.2797715171177132 > 0.2591427434452893
    END: th(1667047.7061423585)=0.2591427434452893; dx=-2.7955217713039E-8 evalInputDelta=0.020628773672423883
    Fitness changed from 0.2797715171177132 to 0.2591427434452893
    Iteration 30 complete. Error: 0.2591427434452893 Total: 42.0663; Orientation: 0.1937; Line Search: 33.5645
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/25503a52-e357-483f-aed2-0fcdf901e9a6.jpg)
    
    Final threshold in iteration 31: 0.2591427434452893 (> -Infinity) after 1015.158s (< 3600.000s)
    

Returns:

    0.2591427434452893