Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Code from BasicOptimizer.scala:75 executed in 135.90 seconds (4.848 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1804135644241
    Reset training subject: 1804909954693
    Adding measurement 1aa590f7 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.0973255813943918;dx=-5.056056990974963E-8
    New Minimum: 1.0973255813943918 > 1.097325514111961
    WOLFE (weak): th(2.154434690031884)=1.097325514111961; dx=-5.0560465914196867E-8 evalInputDelta=6.728243073794715E-8
    New Minimum: 1.097325514111961 > 1.0973253122646687
    WOLFE (weak): th(4.308869380063768)=1.0973253122646687; dx=-5.056050275645599E-8 evalInputDelta=2.691297231738332E-7
    New Minimum: 1.0973253122646687 > 1.0973248412876528
    WOLFE (weak): th(12.926608140191302)=1.0973248412876528; dx=-5.056044136255366E-8 evalInputDelta=7.401067390055971E-7
    New Minimum: 1.0973248412876528 > 1.0973230246620203
    WOLFE (weak): th(51.70643256076521)=1.0973230246620203; dx=-5.055941873836144E-8 evalInputDelta=2.5567323715947055E-6
    New Minimum: 1.0973230246620203 > 1.0973132687095493
    WOLFE (weak): th(258.53216280382605)=1.0973132687095493; dx=-5.054797865103046E-8 evalInputDelta=1.2312684842585853E-5
    New Minimum: 1.0973132687095493 > 1.0972517052853354
    WOLFE (weak): th(1551.1929768229563)=1.0972517052853354; dx=-5.0495227984622285E-8 evalInputDelta=7.38761090564033E-5
    New Minimum: 1.0972517052853354 > 1.0968116781876773
    WOLFE (weak): th(10858.350837760694)=1.0968116781876773; dx=-5.006610207762667E-8 evalInputDelta=5.139032067145166E-4
    New Minimum: 1.0968116781876773 > 1.0933370788945176
    WOLFE (weak): th(86866.80670208555)=1.0933370788945176; dx=-4.7289393906784144E-8 evalInputDelta=0.003988502499874258
    New Minimum: 1.0933370788945176 > 1.0693667036732337
    END: th(781801.26031877)=1.0693667036732337; dx=-2.7633554605424622E-8 evalInputDelta=0.027958877721158126
    Fitness changed from 1.0973255813943918 to 1.0693667036732337
    Iteration 1 complete. Error: 1.0693667036732337 Total: 12.5485; Orientation: 0.0676; Line Search: 10.1623
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/8a2bc996-3ee6-4bf5-95a3-56859a258c

...skipping 19947 bytes...

     4.2441; Orientation: 1.6359; Line Search: 1.8457
    Adding measurement 2a64a736 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.297e+02, gradient 1.078e-04, dot -0.861; [12e21c12-a7a7-4c61-8da0-0debeb3943a8 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.9434616963943498, 0.9407138819190654, 0.9380126941683492, 0.9352667636011281
    LBFGS Accumulation History: 3 points
    Removed measurement 7a9966a to history. Total: 4
    th(0)=0.9352667636011281;dx=-1.1612179999619187E-8
    New Minimum: 0.9352667636011281 > 0.9317640402518795
    END: th(1012731.481481482)=0.9317640402518795; dx=-4.706503411315053E-9 evalInputDelta=0.0035027233492486154
    Fitness changed from 0.9352667636011281 to 0.9317640402518795
    Iteration 24 complete. Error: 0.9317640402518795 Total: 4.2425; Orientation: 1.6286; Line Search: 1.8400
    Adding measurement 75f56458 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.816e+02, gradient 1.474e-04, dot -0.576; [12e21c12-a7a7-4c61-8da0-0debeb3943a8 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.9407138819190654, 0.9380126941683492, 0.9352667636011281, 0.9317640402518795
    LBFGS Accumulation History: 3 points
    Removed measurement 73d788b7 to history. Total: 4
    th(0)=0.9317640402518795;dx=-2.173556892285585E-8
    Armijo: th(2181863.8353910875)=0.9473194018486817; dx=1.1094290129075312E-8 evalInputDelta=-0.015555361596802153
    Armijo: th(1090931.9176955437)=0.9322793563896417; dx=5.739339932235609E-9 evalInputDelta=-5.153161377621229E-4
    New Minimum: 0.9317640402518795 > 0.9278206169806644
    END: th(363643.97256518126)=0.9278206169806644; dx=-6.280215144587665E-9 evalInputDelta=0.003943423271215107
    Fitness changed from 0.9317640402518795 to 0.9278206169806644
    Iteration 25 complete. Error: 0.9278206169806644 Total: 6.0365; Orientation: 1.6326; Line Search: 3.6395
    <a id="p-19"></a>Iteration 25
    <a id="p-18"></a>![Iteration 25](etc/0f0ac99d-c486-46e9-a285-740801bb4b1f.jpg)
    
    Final threshold in iteration 26: 0.9278206169806644 (> -Infinity) after 135.904s (< 3600.000s)
    

Returns:

    0.9278206169806644