Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Code from BasicOptimizer.scala:75 executed in 756.78 seconds (5.371 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 943405320745900
    Reset training subject: 943443160358200
    Adding measurement 3852046d to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=4.646856017410755;dx=-3.7603484681930427E-5
    New Minimum: 4.646856017410755 > 4.646821543574333
    WOLFE (weak): th(2.154434690031884)=4.646821543574333; dx=-3.760026056494143E-5 evalInputDelta=3.447383642196655E-5
    New Minimum: 4.646821543574333 > 4.646785628050566
    WOLFE (weak): th(4.308869380063768)=4.646785628050566; dx=-3.759869228301794E-5 evalInputDelta=7.038936018943787E-5
    New Minimum: 4.646785628050566 > 4.646643910557032
    WOLFE (weak): th(12.926608140191302)=4.646643910557032; dx=-3.758019418761013E-5 evalInputDelta=2.12106853723526E-4
    New Minimum: 4.646643910557032 > 4.646007619798183
    WOLFE (weak): th(51.70643256076521)=4.646007619798183; dx=-3.750035046482374E-5 evalInputDelta=8.483976125717163E-4
    New Minimum: 4.646007619798183 > 4.642506346106529
    WOLFE (weak): th(258.53216280382605)=4.642506346106529; dx=-3.7637648934490934E-5 evalInputDelta=0.004349671304225922
    New Minimum: 4.642506346106529 > 4.621114540845156
    WOLFE (weak): th(1551.1929768229563)=4.621114540845156; dx=-3.587640590444014E-5 evalInputDelta=0.02574147656559944
    New Minimum: 4.621114540845156 > 4.489714723080397
    END: th(10858.350837760694)=4.489714723080397; dx=-2.6548000621483636E-5 evalInputDelta=0.1571412943303585
    Fitness changed from 4.646856017410755 to 4.489714723080397
    Iteration 1 complete. Error: 4.489714723080397 Total: 415.9559; Orientation: 0.0313; Line Search: 302.8468
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/0ae9b179-65e7-4108-8dd0-63c3cd3ac0d6.jpg)
    
    Adding measurement 736f858d to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=4.489714723080397;dx=-2.4268687138523732E-5
    New Minimum: 4.489714723080397 > 4.289900898933411
    END: th(23393.607721408407)=4.289900898933411; dx=-1.4482128011312777E-5 evalInputDelta=0.199813824146986
    Fitness changed from 4.489714723080397 to 4.289900898933411
    Iteration 2 complete. Error: 4.289900898933411 Total: 113.6054; Orientation: 0.0347; Line Search: 75.8585
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/0639131f-f9cc-4422-8f80-1aa88b266b10.jpg)
    
    Adding measurement 1ca476a7 to history. Total: 2
    LBFGS Accumulation History: 3 points
    th(0)=4.289900898933411;dx=-1.6331918800252535E-5
    New Minimum: 4.289900898933411 > 4.104253016412258
    END: th(50400.00000000001)=4.104253016412258; dx=-6.562357957010955E-6 evalInputDelta=0.1856478825211525
    Fitness changed from 4.289900898933411 to 4.104253016412258
    Iteration 3 complete. Error: 4.104253016412258 Total: 112.7458; Orientation: 0.0319; Line Search: 75.1794
    <a id="p-7"></a>Iteration 3
    <a id="p-6"></a>![Iteration 3](etc/5e8295c4-db69-4a1e-82f1-1bb04d872de3.jpg)
    
    Adding measurement 7ef28ab2 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.768e+02, gradient 3.334e-03, dot -0.823; [3facb081-0181-495e-9f51-67e1c0c831c2 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 4.646856017410755, 4.489714723080397, 4.289900898933411, 4.104253016412258
    LBFGS Accumulation History: 3 points
    Removed measurement 3852046d to history. Total: 4
    th(0)=4.104253016412258;dx=-1.1117584533204965E-5
    New Minimum: 4.104253016412258 > 3.9633858539164066
    END: th(108583.50837760697)=3.9633858539164066; dx=-2.9191167515858817E-6 evalInputDelta=0.14086716249585152
    Fitness changed from 4.104253016412258 to 3.9633858539164066
    Iteration 4 complete. Error: 3.9633858539164066 Total: 114.4490; Orientation: 0.9660; Line Search: 75.9354
    <a id="p-9"></a>Iteration 4
    <a id="p-8"></a>![Iteration 4](etc/2ec95da0-d230-4175-91bf-ac56447bd7fc.jpg)
    
    Final threshold in iteration 4: 3.9633858539164066 (> -Infinity) after 756.783s (< 720.000s)
    

Returns:

    3.9633858539164066