Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Iteration 35 Iteration 35

Iteration 40 Iteration 40

Iteration 45 Iteration 45

Iteration 50 Iteration 50

Code from BasicOptimizer.scala:75 executed in 620.97 seconds (8.682 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 7204605161101
    Reset training subject: 7207884443229
    Adding measurement 671bbcc1 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.0113056861275587;dx=-3.605820984539476E-8
    New Minimum: 1.0113056861275587 > 1.011305617900821
    WOLFE (weak): th(2.154434690031884)=1.011305617900821; dx=-3.605808698222674E-8 evalInputDelta=6.822673759288023E-8
    New Minimum: 1.011305617900821 > 1.0113055496680172
    WOLFE (weak): th(4.308869380063768)=1.0113055496680172; dx=-3.605805921949431E-8 evalInputDelta=1.36459541444367E-7
    New Minimum: 1.0113055496680172 > 1.011305276774386
    WOLFE (weak): th(12.926608140191302)=1.011305276774386; dx=-3.608001127238054E-8 evalInputDelta=4.093531726923061E-7
    New Minimum: 1.011305276774386 > 1.011304046293785
    WOLFE (weak): th(51.70643256076521)=1.011304046293785; dx=-3.621541854754129E-8 evalInputDelta=1.6398337736589497E-6
    New Minimum: 1.011304046293785 > 1.011297464744096
    WOLFE (weak): th(258.53216280382605)=1.011297464744096; dx=-3.621300123141384E-8 evalInputDelta=8.221383462680976E-6
    New Minimum: 1.011297464744096 > 1.011255951185659
    WOLFE (weak): th(1551.1929768229563)=1.011255951185659; dx=-3.63075794250805E-8 evalInputDelta=4.9734941899748364E-5
    New Minimum: 1.011255951185659 > 1.0109498709959355
    WOLFE (weak): th(10858.350837760694)=1.0109498709959355; dx=-3.6739029080499836E-8 evalInputDelta=3.558151316231939E-4
    New Minimum: 1.0109498709959355 > 1.0085998598049737
    END: th(86866.80670208555)=1.0085998598049737; dx=-3.220870311873634E-8 evalInputDelta=0.002705826322584981
    Fitness changed from 1.0113056861275587 to 1.0085998598049737
    Iteration 1 complete. Error: 1.0085998598049737 Total: 32.3363; Orientation: 0.0684; Line Search: 23.9136
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/e4ff0614-d381-43b6-b5ef-43c5938c6982.jpg)
    
    Adding measurement 603f15e9 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=1.0085998598049737;dx=-4.339749153895871E-8
    New Minimum: 1.0085998598049737 > 1.005

...skipping 42673 bytes...

    dot -0.752; [16adf665-938a-4e3d-a81a-83faab550cf9 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.9497811259012165, 0.9488016836193991, 0.9481916648710145, 0.9476800822109106
    LBFGS Accumulation History: 3 points
    Removed measurement 13df02f7 to history. Total: 4
    th(0)=0.9476800822109106;dx=-2.5568339477671383E-8
    Armijo: th(150350.161289008)=0.9477468260477143; dx=-2.4182906156436304E-9 evalInputDelta=-6.674383680371765E-5
    New Minimum: 0.9476800822109106 > 0.9474237809504988
    END: th(75175.080644504)=0.9474237809504988; dx=-7.892329912399027E-9 evalInputDelta=2.5630126041176204E-4
    Fitness changed from 0.9476800822109106 to 0.9474237809504988
    Iteration 49 complete. Error: 0.9474237809504988 Total: 12.0826; Orientation: 1.6073; Line Search: 7.9559
    Adding measurement 3ccee4b1 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.400e+01, gradient 1.891e-04, dot -0.638; [16adf665-938a-4e3d-a81a-83faab550cf9 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.9488016836193991, 0.9481916648710145, 0.9476800822109106, 0.9474237809504988
    LBFGS Accumulation History: 3 points
    Removed measurement 5060aced to history. Total: 4
    th(0)=0.9474237809504988;dx=-3.577436963455127E-8
    Armijo: th(161959.80156646387)=0.9484189266161194; dx=6.910203917499077E-9 evalInputDelta=-9.951456656205337E-4
    New Minimum: 0.9474237809504988 > 0.947079266103337
    WOLF (strong): th(80979.90078323193)=0.947079266103337; dx=4.043784673816638E-9 evalInputDelta=3.4451484716180314E-4
    New Minimum: 0.947079266103337 > 0.9468834849777135
    END: th(26993.30026107731)=0.9468834849777135; dx=-1.9310658394705267E-8 evalInputDelta=5.402959727853762E-4
    Fitness changed from 0.9474237809504988 to 0.9468834849777135
    Iteration 50 complete. Error: 0.9468834849777135 Total: 17.1412; Orientation: 3.9477; Line Search: 10.6822
    <a id="p-29"></a>Iteration 50
    <a id="p-28"></a>![Iteration 50](etc/14db731b-af6c-415b-9bfb-35b906ac0525.jpg)
    
    Final threshold in iteration 51: 0.9468834849777135 (> -Infinity) after 620.968s (< 3600.000s)
    

Returns:

    0.9468834849777135