Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:75 executed in 3301.73 seconds (31.768 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 854008762160100
    Reset training subject: 854047320648500
    Adding measurement 7614378d to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=6.498792231082916;dx=-1.3886808014689673E-6
    New Minimum: 6.498792231082916 > 6.498789668083191
    WOLFE (weak): th(2.154434690031884)=6.498789668083191; dx=-1.3886806829662498E-6 evalInputDelta=2.562999725341797E-6
    New Minimum: 6.498789668083191 > 6.498786568641663
    WOLFE (weak): th(4.308869380063768)=6.498786568641663; dx=-1.3886531570937815E-6 evalInputDelta=5.662441253662109E-6
    New Minimum: 6.498786568641663 > 6.49877542257309
    WOLFE (weak): th(12.926608140191302)=6.49877542257309; dx=-1.3886517318898946E-6 evalInputDelta=1.6808509826660156E-5
    New Minimum: 6.49877542257309 > 6.498724043369293
    WOLFE (weak): th(51.70643256076521)=6.498724043369293; dx=-1.3886317015238474E-6 evalInputDelta=6.818771362304688E-5
    New Minimum: 6.498724043369293 > 6.498451888561249
    WOLFE (weak): th(258.53216280382605)=6.498451888561249; dx=-1.3885746259977911E-6 evalInputDelta=3.4034252166748047E-4
    New Minimum: 6.498451888561249 > 6.496750593185425
    WOLFE (weak): th(1551.1929768229563)=6.496750593185425; dx=-1.3872523964768899E-6 evalInputDelta=0.002041637897491455
    New Minimum: 6.496750593185425 > 6.48453676700592
    WOLFE (weak): th(10858.350837760694)=6.48453676700592; dx=-1.3773677436827286E-6 evalInputDelta=0.01425546407699585
    New Minimum: 6.48453676700592 > 6.3861348032951355
    WOLFE (weak): th(86866.80670208555)=6.3861348032951355; dx=-1.2944072919867116E-6 evalInputDelta=0.11265742778778076
    New Minimum: 6.3861348032951355 > 5.72378146648407
    END: th(781801.26031877)=5.72378146648407; dx=-7.248597744339219E-7 evalInputDelta=0.7750107645988464
    Fitness changed from 6.498792231082916 to 5.72378146648407
    Iteration 1 complete. Error: 5.72378146648407 Total: 512.7336; Orientation: 0.0669; Line Search: 393.9783
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/8edd322d-4702-41dd-a65f-56ec8e3ac15

...skipping 15145 bytes...

    77.3262
    Adding measurement 5016f1a to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.193e+03, gradient 4.287e-05, dot -0.618; [16bdde5e-c893-4257-9786-1238934b9073 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.7260234132409096, 0.7076123729348183, 0.6880819350481033, 0.6622244790196419
    LBFGS Accumulation History: 3 points
    Removed measurement 706d2bae to history. Total: 4
    th(0)=0.6622244790196419;dx=-1.8376582566120024E-9
    Armijo: th(1.1310782122667396E8)=0.7651916369795799; dx=1.8877517166379224E-9 evalInputDelta=-0.10296715795993805
    Armijo: th(5.655391061333698E7)=0.66483985632658; dx=8.416144084418766E-10 evalInputDelta=-0.0026153773069381714
    New Minimum: 0.6622244790196419 > 0.6442935988306999
    END: th(1.8851303537778992E7)=0.6442935988306999; dx=-5.363466543186074E-10 evalInputDelta=0.017930880188941956
    Fitness changed from 0.6622244790196419 to 0.6442935988306999
    Iteration 19 complete. Error: 0.6442935988306999 Total: 198.6161; Orientation: 1.8065; Line Search: 158.3995
    Adding measurement 699bb304 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.071e+03, gradient 3.542e-05, dot -0.719; [16bdde5e-c893-4257-9786-1238934b9073 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.7076123729348183, 0.6880819350481033, 0.6622244790196419, 0.6442935988306999
    LBFGS Accumulation History: 3 points
    Removed measurement 14b9df65 to history. Total: 4
    th(0)=0.6442935988306999;dx=-1.2547093596097618E-9
    New Minimum: 0.6442935988306999 > 0.6257985569536686
    END: th(4.061390229411184E7)=0.6257985569536686; dx=-1.9484736398192914E-10 evalInputDelta=0.018495041877031326
    Fitness changed from 0.6442935988306999 to 0.6257985569536686
    Iteration 20 complete. Error: 0.6257985569536686 Total: 119.1080; Orientation: 1.8044; Line Search: 77.8021
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/1f2786fc-bf3d-47c0-a6ad-c438821e4755.jpg)
    
    Final threshold in iteration 21: 0.6257985569536686 (> -Infinity) after 3301.727s (< 3600.000s)
    

Returns:

    0.6257985569536686