Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Iteration 35 Iteration 35

Iteration 40 Iteration 40

Iteration 45 Iteration 45

Iteration 50 Iteration 50

Code from BasicOptimizer.scala:75 executed in 406.81 seconds (6.202 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 2994381160063
    Reset training subject: 2995811909984
    Adding measurement 5c025122 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.9366366700557319;dx=-9.012874634113232E-8
    New Minimum: 0.9366366700557319 > 0.9366365310438761
    WOLFE (weak): th(2.154434690031884)=0.9366365310438761; dx=-9.012638737808244E-8 evalInputDelta=1.3901185580422748E-7
    New Minimum: 0.9366365310438761 > 0.9366363920347074
    WOLFE (weak): th(4.308869380063768)=0.9366363920347074; dx=-9.012660942376756E-8 evalInputDelta=2.7802102453566846E-7
    New Minimum: 0.9366363920347074 > 0.9366358359984814
    WOLFE (weak): th(12.926608140191302)=0.9366358359984814; dx=-9.012675520215603E-8 evalInputDelta=8.340572504872412E-7
    New Minimum: 0.9366358359984814 > 0.9366333338510475
    WOLFE (weak): th(51.70643256076521)=0.9366333338510475; dx=-9.012716245368812E-8 evalInputDelta=3.3362046844009896E-6
    New Minimum: 0.9366333338510475 > 0.9366199842922445
    WOLFE (weak): th(258.53216280382605)=0.9366199842922445; dx=-9.015834887826646E-8 evalInputDelta=1.6685763487456207E-5
    New Minimum: 0.9366199842922445 > 0.936536425217887
    WOLFE (weak): th(1551.1929768229563)=0.936536425217887; dx=-9.030391566407131E-8 evalInputDelta=1.0024483784487614E-4
    New Minimum: 0.936536425217887 > 0.9359296667759847
    WOLFE (weak): th(10858.350837760694)=0.9359296667759847; dx=-9.102017699812285E-8 evalInputDelta=7.070032797472647E-4
    New Minimum: 0.9359296667759847 > 0.9307047794094278
    WOLFE (weak): th(86866.80670208555)=0.9307047794094278; dx=-9.584365939925764E-8 evalInputDelta=0.005931890646304172
    New Minimum: 0.9307047794094278 > 0.8803466167689128
    END: th(781801.26031877)=0.8803466167689128; dx=-7.588710995972393E-8 evalInputDelta=0.05629005328681913
    Fitness changed from 0.9366366700557319 to 0.8803466167689128
    Iteration 1 complete. Error: 0.8803466167689128 Total: 20.5660; Orientation: 0.0636; Line Search: 16.1249
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/58e86ea7-eb63-43a1-a8ab-4e50125f5e9

...skipping 43139 bytes...

    2, gradient 2.214e-05, dot -0.660; [65b8d454-07aa-484c-ad3a-f8dbb5803c80 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.16503159322777441, 0.16330205744971485, 0.1622800929454408, 0.16152863785320556
    LBFGS Accumulation History: 3 points
    Removed measurement 2d1c3cd3 to history. Total: 4
    th(0)=0.16152863785320556;dx=-4.902964567004407E-10
    Armijo: th(2.4356726128819294E7)=0.16458478053777836; dx=1.5719517941809875E-10 evalInputDelta=-0.003056142684572799
    New Minimum: 0.16152863785320556 > 0.16102926868856743
    END: th(1.2178363064409647E7)=0.16102926868856743; dx=-2.5063399158123887E-11 evalInputDelta=4.99369164638136E-4
    Fitness changed from 0.16152863785320556 to 0.16102926868856743
    Iteration 49 complete. Error: 0.16102926868856743 Total: 8.0810; Orientation: 1.6481; Line Search: 4.9421
    Adding measurement 40f65bf4 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.616e+02, gradient 2.588e-05, dot -0.559; [65b8d454-07aa-484c-ad3a-f8dbb5803c80 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.16330205744971485, 0.1622800929454408, 0.16152863785320556, 0.16102926868856743
    LBFGS Accumulation History: 3 points
    Removed measurement 622559d9 to history. Total: 4
    th(0)=0.16102926868856743;dx=-6.697993333755044E-10
    Armijo: th(2.623748785376714E7)=0.1673722114638111; dx=3.3003216450908937E-10 evalInputDelta=-0.006342942775243665
    Armijo: th(1.311874392688357E7)=0.1612124430097911; dx=1.4076185067385871E-10 evalInputDelta=-1.8317432122366695E-4
    New Minimum: 0.16102926868856743 > 0.1598110001500463
    END: th(4372914.642294523)=0.1598110001500463; dx=-2.7734464951949257E-10 evalInputDelta=0.001218268538521139
    Fitness changed from 0.16102926868856743 to 0.1598110001500463
    Iteration 50 complete. Error: 0.1598110001500463 Total: 11.1970; Orientation: 1.6271; Line Search: 8.0868
    <a id="p-29"></a>Iteration 50
    <a id="p-28"></a>![Iteration 50](etc/31b0880d-c3fe-45b2-8286-b356d1fecb4b.jpg)
    
    Final threshold in iteration 51: 0.1598110001500463 (> -Infinity) after 406.809s (< 3600.000s)
    

Returns:

    0.1598110001500463