Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:75 executed in 4502.64 seconds (49.254 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1504829178125400
    Reset training subject: 1504882100776900
    Adding measurement 68aa1164 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=4.408724554880986;dx=-2.6386448873956384E-6
    New Minimum: 4.408724554880986 > 4.408724371237786
    WOLFE (weak): th(2.154434690031884)=4.408724371237786; dx=-2.6382934127692364E-6 evalInputDelta=1.8364320020225477E-7
    New Minimum: 4.408724371237786 > 4.408723655101028
    WOLFE (weak): th(4.308869380063768)=4.408723655101028; dx=-2.6382713018008046E-6 evalInputDelta=8.99779958096758E-7
    New Minimum: 4.408723655101028 > 4.408720526206795
    WOLFE (weak): th(12.926608140191302)=4.408720526206795; dx=-2.638275270114378E-6 evalInputDelta=4.028674190692527E-6
    New Minimum: 4.408720526206795 > 4.408707408073993
    WOLFE (weak): th(51.70643256076521)=4.408707408073993; dx=-2.638256738872305E-6 evalInputDelta=1.7146806992940355E-5
    New Minimum: 4.408707408073993 > 4.408634807215438
    WOLFE (weak): th(258.53216280382605)=4.408634807215438; dx=-2.6379523111788877E-6 evalInputDelta=8.97476655481455E-5
    New Minimum: 4.408634807215438 > 4.408186203275895
    WOLFE (weak): th(1551.1929768229563)=4.408186203275895; dx=-2.635585289092902E-6 evalInputDelta=5.383516050905257E-4
    New Minimum: 4.408186203275895 > 4.4049603246633025
    WOLFE (weak): th(10858.350837760694)=4.4049603246633025; dx=-2.6240456077109602E-6 evalInputDelta=0.003764230217683462
    New Minimum: 4.4049603246633025 > 4.379136832654441
    WOLFE (weak): th(86866.80670208555)=4.379136832654441; dx=-2.528052495352557E-6 evalInputDelta=0.0295877222265446
    New Minimum: 4.379136832654441 > 4.197168749606953
    END: th(781801.26031877)=4.197168749606953; dx=-1.4960898192771932E-6 evalInputDelta=0.21155580527403295
    Fitness changed from 4.408724554880986 to 4.197168749606953
    Iteration 1 complete. Error: 4.197168749606953 Total: 687.6801; Orientation: 0.1277; Line Search: 529.5907
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/c8e678ce-d216-439d-bc6e-212

...skipping 14854 bytes...

    2226)=2.786889966393345; dx=-2.325296272505912E-7 evalInputDelta=0.03507632918888959
    Fitness changed from 2.8219662955822344 to 2.786889966393345
    Iteration 18 complete. Error: 2.786889966393345 Total: 160.1037; Orientation: 3.6380; Line Search: 104.8724
    Adding measurement 4a5e0faf to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.242e+04, gradient 4.758e-04, dot -0.921; [34262f92-0137-4367-a50f-17421835ab75 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 2.923896197752511, 2.85232153505428, 2.8219662955822344, 2.786889966393345
    LBFGS Accumulation History: 3 points
    Removed measurement 135f760a to history. Total: 4
    th(0)=2.786889966393345;dx=-2.2638334985619993E-7
    New Minimum: 2.786889966393345 > 2.7541150535203616
    END: th(2094589.2819754435)=2.7541150535203616; dx=-2.0142957399552477E-7 evalInputDelta=0.032774912872983286
    Fitness changed from 2.786889966393345 to 2.7541150535203616
    Iteration 19 complete. Error: 2.7541150535203616 Total: 162.6298; Orientation: 3.6456; Line Search: 106.2549
    Adding measurement 7f59f4e4 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.056e+04, gradient 4.631e-04, dot -0.964; [34262f92-0137-4367-a50f-17421835ab75 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 2.85232153505428, 2.8219662955822344, 2.786889966393345, 2.7541150535203616
    LBFGS Accumulation History: 3 points
    Removed measurement 1e8c9083 to history. Total: 4
    th(0)=2.7541150535203616;dx=-2.145019323115516E-7
    New Minimum: 2.7541150535203616 > 2.695939034544209
    END: th(4512655.810456871)=2.695939034544209; dx=-1.7895962937581286E-7 evalInputDelta=0.058176018976152655
    Fitness changed from 2.7541150535203616 to 2.695939034544209
    Iteration 20 complete. Error: 2.695939034544209 Total: 162.9431; Orientation: 3.6506; Line Search: 106.8117
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/38625be3-4831-4ff7-bf64-79a4dcbaa287.jpg)
    
    Final threshold in iteration 21: 2.695939034544209 (> -Infinity) after 4502.634s (< 5400.000s)
    

Returns:

    2.695939034544209