Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Code from BasicOptimizer.scala:75 executed in 61.02 seconds (1.049 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 919668710738300
    Reset training subject: 919670497060500
    Adding measurement 2ad1363a to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.3525277376174927;dx=-3.288521990739654E-8
    New Minimum: 0.3525277376174927 > 0.3525276780128479
    WOLFE (weak): th(2.154434690031884)=0.3525276780128479; dx=-3.2765216328918066E-8 evalInputDelta=5.9604644775390625E-8
    WOLFE (weak): th(4.308869380063768)=0.3525276780128479; dx=-3.27651587030937E-8 evalInputDelta=5.9604644775390625E-8
    New Minimum: 0.3525276780128479 > 0.3525274693965912
    WOLFE (weak): th(12.926608140191302)=0.3525274693965912; dx=-3.2765245228683415E-8 evalInputDelta=2.682209014892578E-7
    New Minimum: 0.3525274693965912 > 0.35252657532691956
    WOLFE (weak): th(51.70643256076521)=0.35252657532691956; dx=-3.276528424990389E-8 evalInputDelta=1.1622905731201172E-6
    New Minimum: 0.35252657532691956 > 0.3525221347808838
    WOLFE (weak): th(258.53216280382605)=0.3525221347808838; dx=-3.276481381039784E-8 evalInputDelta=5.602836608886719E-6
    New Minimum: 0.3525221347808838 > 0.35249438881874084
    WOLFE (weak): th(1551.1929768229563)=0.35249438881874084; dx=-3.2760623995835785E-8 evalInputDelta=3.3348798751831055E-5
    New Minimum: 0.35249438881874084 > 0.35229456424713135
    WOLFE (weak): th(10858.350837760694)=0.35229456424713135; dx=-3.27436794934171E-8 evalInputDelta=2.3317337036132812E-4
    New Minimum: 0.35229456424713135 > 0.35066941380500793
    WOLFE (weak): th(86866.80670208555)=0.35066941380500793; dx=-3.257148576819967E-8 evalInputDelta=0.0018583238124847412
    New Minimum: 0.35066941380500793 > 0.3363179564476013
    WOLFE (weak): th(781801.26031877)=0.3363179564476013; dx=-3.0736813287992243E-8 evalInputDelta=0.016209781169891357
    New Minimum: 0.3363179564476013 > 0.24074941873550415
    END: th(7818012.6031877)=0.24074941873550415; dx=-1.5692474172514014E-8 evalInputDelta=0.11177831888198853
    Fitness changed from 0.3525277376174927 to 0.24074941873550415
    Iteration 1 complete. Error: 0.240749

...skipping 2380 bytes...

    nt 43eff895 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.872e+03, gradient 6.573e-05, dot -0.794; [b217f4ed-9dc1-4f67-ae79-78ec2c828ae9 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.24074941873550415, 0.16160979866981506, 0.09584474563598633, 0.04646626114845276
    LBFGS Accumulation History: 3 points
    Removed measurement 5ce90fcf to history. Total: 4
    th(0)=0.04646626114845276;dx=-4.319849782495211E-9
    Armijo: th(8.421698779707026E7)=0.0552123486995697; dx=-8.790785764141741E-10 evalInputDelta=-0.008746087551116943
    New Minimum: 0.04646626114845276 > 0.005486994981765747
    END: th(4.210849389853513E7)=0.005486994981765747; dx=-1.8184160399123416E-9 evalInputDelta=0.04097926616668701
    Fitness changed from 0.04646626114845276 to 0.005486994981765747
    Iteration 5 complete. Error: 0.005486994981765747 Total: 8.0133; Orientation: 0.8691; Line Search: 5.2824
    <a id="p-11"></a>Iteration 5
    <a id="p-10"></a>![Iteration 5](etc/19071bc1-a019-4ad9-8dce-ecae872c3dc2.jpg)
    
    Adding measurement 24302b86 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.807e+03, gradient 5.975e-05, dot -0.779; [b217f4ed-9dc1-4f67-ae79-78ec2c828ae9 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.16160979866981506, 0.09584474563598633, 0.04646626114845276, 0.005486994981765747
    LBFGS Accumulation History: 3 points
    Removed measurement 5996ffd3 to history. Total: 4
    th(0)=0.005486994981765747;dx=-3.569562942383111E-9
    Armijo: th(9.072000000000001E7)=0.019954919815063477; dx=-5.736913135688077E-10 evalInputDelta=-0.01446792483329773
    New Minimum: 0.005486994981765747 > -0.025663286447525024
    END: th(4.536000000000001E7)=-0.025663286447525024; dx=-1.3241559101114213E-9 evalInputDelta=0.03115028142929077
    Fitness changed from 0.005486994981765747 to -0.025663286447525024
    Iteration 6 complete. Error: -0.025663286447525024 Total: 7.9813; Orientation: 0.8654; Line Search: 5.4116
    Final threshold in iteration 7: -0.025663286447525024 (> -Infinity) after 61.017s (< 720.000s)
    

Returns:

    -0.025663286447525024