Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 3794.90 seconds (43.027 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1271754932904200
    Reset training subject: 1271815380402500
    Adding measurement 51f0c6d1 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.1091851280240506;dx=-2.627605329409182E-7
    New Minimum: 1.1091851280240506 > 1.1091850545075506
    WOLFE (weak): th(2.154434690031884)=1.1091850545075506; dx=-2.5968561043754453E-7 evalInputDelta=7.351649999876031E-8
    New Minimum: 1.1091850545075506 > 1.1091848353801403
    WOLFE (weak): th(4.308869380063768)=1.1091848353801403; dx=-2.596861923705508E-7 evalInputDelta=2.9264391021577296E-7
    New Minimum: 1.1091848353801403 > 1.1091841499267254
    WOLFE (weak): th(12.926608140191302)=1.1091841499267254; dx=-2.5968588300497257E-7 evalInputDelta=9.780973251327651E-7
    New Minimum: 1.1091841499267254 > 1.1091806576610295
    WOLFE (weak): th(51.70643256076521)=1.1091806576610295; dx=-2.596841942994334E-7 evalInputDelta=4.470363021091828E-6
    New Minimum: 1.1091806576610295 > 1.1091622423432024
    WOLFE (weak): th(258.53216280382605)=1.1091622423432024; dx=-2.59682692870946E-7 evalInputDelta=2.2885680848183654E-5
    New Minimum: 1.1091622423432024 > 1.1090468360651777
    WOLFE (weak): th(1551.1929768229563)=1.1090468360651777; dx=-2.5966133479734976E-7 evalInputDelta=1.3829195887282708E-4
    New Minimum: 1.1090468360651777 > 1.1082170094818338
    WOLFE (weak): th(10858.350837760694)=1.1082170094818338; dx=-2.594466385929116E-7 evalInputDelta=9.681185422167893E-4
    New Minimum: 1.1082170094818338 > 1.1014828930685279
    WOLFE (weak): th(86866.80670208555)=1.1014828930685279; dx=-2.5779676108597993E-7 evalInputDelta=0.007702234955522691
    New Minimum: 1.1014828930685279 > 1.0433142598279717
    WOLFE (weak): th(781801.26031877)=1.0433142598279717; dx=-2.434014863579786E-7 evalInputDelta=0.0658708681960789
    New Minimum: 1.0433142598279717 > 0.7351460135149409
    END: th(7818012.6031877)=0.7351460135149409; dx=-1.2949823731622284E-7 evalInputDelta=0.37403911450910965
    Fitness changed from 1.1091851280240506 to 0.7351460135149409

...skipping 11084 bytes...

    otal: 3
    Rejected: LBFGS Orientation magnitude: 1.102e+04, gradient 1.382e-04, dot -0.769; [c5c38ff7-7a6e-44f4-8044-9f5871b68788 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.11741711149797464, 0.04923264556323345, 0.007433952264505814, -0.021124911439867368
    LBFGS Accumulation History: 3 points
    Removed measurement 72f97b38 to history. Total: 4
    th(0)=-0.021124911439867368;dx=-1.9089343769138163E-8
    Armijo: th(5.482876809705099E7)=0.11872719302415201; dx=-3.0801686261616144E-9 evalInputDelta=-0.13985210446401938
    New Minimum: -0.021124911439867368 > -0.04290865235789443
    END: th(2.7414384048525494E7)=-0.04290865235789443; dx=-8.528020786653719E-9 evalInputDelta=0.02178374091802706
    Fitness changed from -0.021124911439867368 to -0.04290865235789443
    Iteration 14 complete. Error: -0.04290865235789443 Total: 218.9357; Orientation: 1.4752; Line Search: 163.4941
    Adding measurement 784d12b6 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 9.283e+03, gradient 1.387e-04, dot -0.726; [c5c38ff7-7a6e-44f4-8044-9f5871b68788 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.04923264556323345, 0.007433952264505814, -0.021124911439867368, -0.04290865235789443
    LBFGS Accumulation History: 3 points
    Removed measurement 2a117d87 to history. Total: 4
    th(0)=-0.04290865235789443;dx=-1.9244249889122213E-8
    Armijo: th(5.9062500000000045E7)=0.17498034463963602; dx=5.682356682018853E-10 evalInputDelta=-0.21788899699753045
    New Minimum: -0.04290865235789443 > -0.04732631789318079
    END: th(2.9531250000000022E7)=-0.04732631789318079; dx=-5.807904915358828E-9 evalInputDelta=0.004417665535286365
    Fitness changed from -0.04290865235789443 to -0.04732631789318079
    Iteration 15 complete. Error: -0.04732631789318079 Total: 223.8278; Orientation: 1.5369; Line Search: 167.5596
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/877282a6-5306-474a-aa77-2b8aabcb698c.jpg)
    
    Final threshold in iteration 15: -0.04732631789318079 (> -Infinity) after 3794.884s (< 3600.000s)
    

Returns:

    -0.04732631789318079