Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Code from BasicOptimizer.scala:75 executed in 132.29 seconds (1.552 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 899764863633400
    Reset training subject: 899766055309500
    Adding measurement 6870cfac to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.8700454384088516;dx=-6.019906636806742E-8
    New Minimum: 0.8700454384088516 > 0.8700453191995621
    WOLFE (weak): th(2.154434690031884)=0.8700453191995621; dx=-6.019903762733195E-8 evalInputDelta=1.1920928955078125E-7
    New Minimum: 0.8700453191995621 > 0.8700453042984009
    WOLFE (weak): th(4.308869380063768)=0.8700453042984009; dx=-6.019901499366456E-8 evalInputDelta=1.341104507446289E-7
    New Minimum: 0.8700453042984009 > 0.8700447976589203
    WOLFE (weak): th(12.926608140191302)=0.8700447976589203; dx=-6.019889261889508E-8 evalInputDelta=6.407499313354492E-7
    New Minimum: 0.8700447976589203 > 0.8700428903102875
    WOLFE (weak): th(51.70643256076521)=0.8700428903102875; dx=-6.019842803508405E-8 evalInputDelta=2.5480985641479492E-6
    New Minimum: 0.8700428903102875 > 0.8700327575206757
    WOLFE (weak): th(258.53216280382605)=0.8700327575206757; dx=-6.019658789819888E-8 evalInputDelta=1.2680888175964355E-5
    New Minimum: 0.8700327575206757 > 0.8699692487716675
    WOLFE (weak): th(1551.1929768229563)=0.8699692487716675; dx=-6.01802951204271E-8 evalInputDelta=7.618963718414307E-5
    New Minimum: 0.8699692487716675 > 0.8695130050182343
    WOLFE (weak): th(10858.350837760694)=0.8695130050182343; dx=-6.007579528052929E-8 evalInputDelta=5.324333906173706E-4
    New Minimum: 0.8695130050182343 > 0.8658049702644348
    WOLFE (weak): th(86866.80670208555)=0.8658049702644348; dx=-5.945462087777701E-8 evalInputDelta=0.004240468144416809
    New Minimum: 0.8658049702644348 > 0.8335308283567429
    END: th(781801.26031877)=0.8335308283567429; dx=-5.364366654536032E-8 evalInputDelta=0.036514610052108765
    Fitness changed from 0.8700454384088516 to 0.8335308283567429
    Iteration 1 complete. Error: 0.8335308283567429 Total: 15.7025; Orientation: 0.0151; Line Search: 12.1337
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/954e

...skipping 20137 bytes...

    412E-10 evalInputDelta=0.0016901567578315735
    Fitness changed from -0.4563176780939102 to -0.4580078348517418
    Iteration 23 complete. Error: -0.4580078348517418 Total: 6.2959; Orientation: 0.4378; Line Search: 4.7034
    Adding measurement 239b98cb to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.424e+03, gradient 1.080e-05, dot -0.936; [ee815948-f439-4925-941d-6d378ec1cdcc = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.4520881921052933, -0.45426400750875473, -0.4563176780939102, -0.4580078348517418
    LBFGS Accumulation History: 3 points
    Removed measurement 6a96d639 to history. Total: 4
    th(0)=-0.4580078348517418;dx=-1.1664070160477168E-10
    New Minimum: -0.4580078348517418 > -0.4598141759634018
    END: th(3.645833333333337E7)=-0.4598141759634018; dx=-7.133251578245851E-11 evalInputDelta=0.0018063411116600037
    Fitness changed from -0.4580078348517418 to -0.4598141759634018
    Iteration 24 complete. Error: -0.4598141759634018 Total: 3.8559; Orientation: 0.4230; Line Search: 2.3157
    Adding measurement 23f60b7d to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.825e+03, gradient 1.019e-05, dot -0.849; [ee815948-f439-4925-941d-6d378ec1cdcc = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.45426400750875473, -0.4563176780939102, -0.4580078348517418, -0.4598141759634018
    LBFGS Accumulation History: 3 points
    Removed measurement 570b2057 to history. Total: 4
    th(0)=-0.4598141759634018;dx=-1.038107277361544E-10
    New Minimum: -0.4598141759634018 > -0.4636022299528122
    END: th(7.854709807407919E7)=-0.4636022299528122; dx=-4.8349557453206916E-11 evalInputDelta=0.0037880539894104004
    Fitness changed from -0.4598141759634018 to -0.4636022299528122
    Iteration 25 complete. Error: -0.4636022299528122 Total: 3.9351; Orientation: 0.4375; Line Search: 2.3263
    <a id="p-19"></a>Iteration 25
    <a id="p-18"></a>![Iteration 25](etc/f47d155e-90ba-4f88-9a19-5248a2c2aa11.jpg)
    
    Final threshold in iteration 26: -0.4636022299528122 (> -Infinity) after 132.293s (< 3600.000s)
    

Returns:

    -0.4636022299528122