Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 414.02 seconds (4.108 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 916040044462600
    Reset training subject: 916043751429400
    Adding measurement 6476dd40 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=2.774860866367817;dx=-2.4268483182015705E-6
    New Minimum: 2.774860866367817 > 2.7748580016195774
    WOLFE (weak): th(2.154434690031884)=2.7748580016195774; dx=-2.426885689893337E-6 evalInputDelta=2.864748239517212E-6
    New Minimum: 2.7748580016195774 > 2.7748539336025715
    WOLFE (weak): th(4.308869380063768)=2.7748539336025715; dx=-2.4269254476120466E-6 evalInputDelta=6.932765245437622E-6
    New Minimum: 2.7748539336025715 > 2.7748379185795784
    WOLFE (weak): th(12.926608140191302)=2.7748379185795784; dx=-2.4268664499120124E-6 evalInputDelta=2.294778823852539E-5
    New Minimum: 2.7748379185795784 > 2.774768356233835
    WOLFE (weak): th(51.70643256076521)=2.774768356233835; dx=-2.425413381119049E-6 evalInputDelta=9.251013398170471E-5
    New Minimum: 2.774768356233835 > 2.7743985392153263
    WOLFE (weak): th(258.53216280382605)=2.7743985392153263; dx=-2.417409205997664E-6 evalInputDelta=4.6232715249061584E-4
    New Minimum: 2.7743985392153263 > 2.772092293947935
    WOLFE (weak): th(1551.1929768229563)=2.772092293947935; dx=-2.3943521881106734E-6 evalInputDelta=0.0027685724198818207
    New Minimum: 2.772092293947935 > 2.7559372037649155
    WOLFE (weak): th(10858.350837760694)=2.7559372037649155; dx=-2.222417014863633E-6 evalInputDelta=0.01892366260290146
    New Minimum: 2.7559372037649155 > 2.642168965190649
    END: th(86866.80670208555)=2.642168965190649; dx=-1.4911823802010598E-6 evalInputDelta=0.1326919011771679
    Fitness changed from 2.774860866367817 to 2.642168965190649
    Iteration 1 complete. Error: 2.642168965190649 Total: 39.0661; Orientation: 0.0577; Line Search: 28.9516
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/3cc86ad8-8048-499b-997e-fe1eb9e2ecb1.jpg)
    
    Adding measurement 2a8ecf5b to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=2.642168965190649;dx=-4.170815404972474E-6
    New

...skipping 24473 bytes...

    c4b87b to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.070e+03, gradient 2.885e-05, dot -0.571; [1e49c655-0394-43d7-ba31-e281390738c7 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.26074594259262085, -0.2665611803531647, -0.27189382910728455, -0.2776503413915634
    LBFGS Accumulation History: 3 points
    Removed measurement 1c03675b to history. Total: 4
    th(0)=-0.2776503413915634;dx=-8.321545980767102E-10
    Armijo: th(9.401366271785153E7)=-0.2479754090309143; dx=4.25281438037948E-10 evalInputDelta=-0.02967493236064911
    Armijo: th(4.700683135892577E7)=-0.2760430872440338; dx=2.6265814318127E-10 evalInputDelta=-0.001607254147529602
    New Minimum: -0.2776503413915634 > -0.28392502665519714
    END: th(1.5668943786308588E7)=-0.28392502665519714; dx=-2.7420589750867216E-10 evalInputDelta=0.006274685263633728
    Fitness changed from -0.2776503413915634 to -0.28392502665519714
    Iteration 29 complete. Error: -0.28392502665519714 Total: 17.6063; Orientation: 1.7637; Line Search: 12.5715
    Adding measurement 50cc08f5 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.114e+03, gradient 2.195e-05, dot -0.731; [1e49c655-0394-43d7-ba31-e281390738c7 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.2665611803531647, -0.27189382910728455, -0.2776503413915634, -0.28392502665519714
    LBFGS Accumulation History: 3 points
    Removed measurement 47c158c1 to history. Total: 4
    th(0)=-0.28392502665519714;dx=-4.818339657996292E-10
    New Minimum: -0.28392502665519714 > -0.2887333035469055
    END: th(3.375771604938275E7)=-0.2887333035469055; dx=-1.0032588991569155E-10 evalInputDelta=0.004808276891708374
    Fitness changed from -0.28392502665519714 to -0.2887333035469055
    Iteration 30 complete. Error: -0.2887333035469055 Total: 10.7402; Orientation: 1.7334; Line Search: 6.0791
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/62dd8286-7b61-4a73-90fe-a2bbbe5d5a63.jpg)
    
    Final threshold in iteration 31: -0.2887333035469055 (> -Infinity) after 414.023s (< 3600.000s)
    

Returns:

    -0.2887333035469055