Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Code from BasicOptimizer.scala:75 executed in 3645.56 seconds (39.322 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1060670399031500
    Reset training subject: 1060700975233000
    Adding measurement 141b3d18 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=-0.5660821124911308;dx=-6.869178560685675E-8
    New Minimum: -0.5660821124911308 > -0.5660821869969368
    WOLFE (weak): th(2.154434690031884)=-0.5660821869969368; dx=-6.844563405952946E-8 evalInputDelta=7.450580596923828E-8
    New Minimum: -0.5660821869969368 > -0.5660823658108711
    WOLFE (weak): th(4.308869380063768)=-0.5660823658108711; dx=-6.843935637092656E-8 evalInputDelta=2.5331974029541016E-7
    New Minimum: -0.5660823658108711 > -0.566083088517189
    WOLFE (weak): th(12.926608140191302)=-0.566083088517189; dx=-6.844611576137764E-8 evalInputDelta=9.760260581970215E-7
    New Minimum: -0.566083088517189 > -0.5660858899354935
    WOLFE (weak): th(51.70643256076521)=-0.5660858899354935; dx=-6.844570564369768E-8 evalInputDelta=3.777444362640381E-6
    New Minimum: -0.5660858899354935 > -0.5661011636257172
    WOLFE (weak): th(258.53216280382605)=-0.5661011636257172; dx=-6.844071073613417E-8 evalInputDelta=1.905113458633423E-5
    New Minimum: -0.5661011636257172 > -0.566195972263813
    WOLFE (weak): th(1551.1929768229563)=-0.566195972263813; dx=-6.843931322020202E-8 evalInputDelta=1.1385977268218994E-4
    New Minimum: -0.566195972263813 > -0.5668798685073853
    WOLFE (weak): th(10858.350837760694)=-0.5668798685073853; dx=-6.841843477646597E-8 evalInputDelta=7.97756016254425E-4
    New Minimum: -0.5668798685073853 > -0.5724527463316917
    WOLFE (weak): th(86866.80670208555)=-0.5724527463316917; dx=-6.822220054299394E-8 evalInputDelta=0.006370633840560913
    New Minimum: -0.5724527463316917 > -0.6223822981119156
    WOLFE (weak): th(781801.26031877)=-0.6223822981119156; dx=-6.592785380605657E-8 evalInputDelta=0.05630018562078476
    New Minimum: -0.6223822981119156 > -1.008657194674015
    END: th(7818012.6031877)=-1.008657194674015; dx=-4.236964374935977E-8 evalInputDelta=0.4425750821828842
    Fitness changed from -0.5660821124911308 to -1

...skipping 24208 bytes...

    on magnitude: 1.078e+04, gradient 3.395e-05, dot -0.617; [af1d341f-b9c5-456b-ae0b-3173fb7f688b = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -2.5012378469109535, -2.5051437318325043, -2.51191334053874, -2.5153335742652416
    LBFGS Accumulation History: 3 points
    Removed measurement 45ee6854 to history. Total: 4
    th(0)=-2.5153335742652416;dx=-1.151808769958106E-9
    Armijo: th(1.0909319176955444E8)=-2.307514436542988; dx=1.8178941803761978E-9 evalInputDelta=-0.2078191377222538
    Armijo: th(5.454659588477722E7)=-2.4590324237942696; dx=1.3069720044043857E-9 evalInputDelta=-0.05630115047097206
    New Minimum: -2.5153335742652416 > -2.5185775570571423
    WOLF (strong): th(1.8182198628259074E7)=-2.5185775570571423; dx=2.7326415134623702E-11 evalInputDelta=0.0032439827919006348
    New Minimum: -2.5185775570571423 > -2.5189139433205128
    END: th(4545549.6570647685)=-2.5189139433205128; dx=-7.995228688290571E-10 evalInputDelta=0.0035803690552711487
    Fitness changed from -2.5153335742652416 to -2.5189139433205128
    Iteration 28 complete. Error: -2.5189139433205128 Total: 172.8060; Orientation: 1.6473; Line Search: 143.8709
    Adding measurement 4b5a252c to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 8.893e+03, gradient 2.554e-05, dot -0.777; [af1d341f-b9c5-456b-ae0b-3173fb7f688b = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -2.5051437318325043, -2.51191334053874, -2.5153335742652416, -2.5189139433205128
    LBFGS Accumulation History: 3 points
    Removed measurement 7f78363f to history. Total: 4
    th(0)=-2.5189139433205128;dx=-6.514893454462278E-10
    New Minimum: -2.5189139433205128 > -2.5214243568480015
    END: th(9793089.86644287)=-2.5214243568480015; dx=-4.941134093916063E-10 evalInputDelta=0.0025104135274887085
    Fitness changed from -2.5189139433205128 to -2.5214243568480015
    Iteration 29 complete. Error: -2.5214243568480015 Total: 86.4598; Orientation: 1.6455; Line Search: 57.0095
    Final threshold in iteration 29: -2.5214243568480015 (> -Infinity) after 3645.561s (< 3600.000s)
    

Returns:

    -2.5214243568480015