Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 29.43 seconds (0.468 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 909172083414500
    Reset training subject: 909172546147300
    Adding measurement 47dd7b9a to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.5669799149036407;dx=-1.029998747439374E-6
    New Minimum: 1.5669799149036407 > 1.5669785737991333
    WOLFE (weak): th(2.154434690031884)=1.5669785737991333; dx=-1.0299730288346564E-6 evalInputDelta=1.341104507446289E-6
    New Minimum: 1.5669785737991333 > 1.5669772326946259
    WOLFE (weak): th(4.308869380063768)=1.5669772326946259; dx=-1.029970085409071E-6 evalInputDelta=2.682209014892578E-6
    New Minimum: 1.5669772326946259 > 1.566971480846405
    WOLFE (weak): th(12.926608140191302)=1.566971480846405; dx=-1.0299485680767241E-6 evalInputDelta=8.434057235717773E-6
    New Minimum: 1.566971480846405 > 1.566946193575859
    WOLFE (weak): th(51.70643256076521)=1.566946193575859; dx=-1.0301579578493423E-6 evalInputDelta=3.3721327781677246E-5
    New Minimum: 1.566946193575859 > 1.5668113082647324
    WOLFE (weak): th(258.53216280382605)=1.5668113082647324; dx=-1.030324808078514E-6 evalInputDelta=1.6860663890838623E-4
    New Minimum: 1.5668113082647324 > 1.5659658312797546
    WOLFE (weak): th(1551.1929768229563)=1.5659658312797546; dx=-1.0280068385247058E-6 evalInputDelta=0.0010140836238861084
    New Minimum: 1.5659658312797546 > 1.5597572326660156
    WOLFE (weak): th(10858.350837760694)=1.5597572326660156; dx=-1.014976721485608E-6 evalInputDelta=0.007222682237625122
    New Minimum: 1.5597572326660156 > 1.5047556161880493
    END: th(86866.80670208555)=1.5047556161880493; dx=-8.549839180837226E-7 evalInputDelta=0.06222429871559143
    Fitness changed from 1.5669799149036407 to 1.5047556161880493
    Iteration 1 complete. Error: 1.5047556161880493 Total: 5.4537; Orientation: 0.0036; Line Search: 4.0968
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/3846a88a-abc2-4016-a8b2-93433ae65544.jpg)
    
    Adding measurement 717d6487 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=1.5047556161880493;dx=-1.095810753724622

...skipping 10114 bytes...

    ng measurement 2c1e93ff to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.026e+03, gradient 4.595e-05, dot -0.592; [56d0c476-7614-4751-8687-1aa265313238 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.35483314096927643, -0.3687223792076111, -0.3790566623210907, -0.3852917402982712
    LBFGS Accumulation History: 3 points
    Removed measurement 22bfbb14 to history. Total: 4
    th(0)=-0.3852917402982712;dx=-2.1114092639180546E-9
    Armijo: th(3.898934620234736E7)=-0.3203709274530411; dx=3.330640618031059E-9 evalInputDelta=-0.0649208128452301
    Armijo: th(1.949467310117368E7)=-0.38142943382263184; dx=1.4371852094891505E-9 evalInputDelta=-0.0038623064756393433
    New Minimum: -0.3852917402982712 > -0.3925749361515045
    END: th(6498224.367057893)=-0.3925749361515045; dx=-9.170990049109762E-10 evalInputDelta=0.007283195853233337
    Fitness changed from -0.3852917402982712 to -0.3925749361515045
    Iteration 14 complete. Error: -0.3925749361515045 Total: 2.3814; Orientation: 0.1055; Line Search: 1.8309
    Adding measurement 38cd4663 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.951e+03, gradient 3.069e-05, dot -0.912; [56d0c476-7614-4751-8687-1aa265313238 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.3687223792076111, -0.3790566623210907, -0.3852917402982712, -0.3925749361515045
    LBFGS Accumulation History: 3 points
    Removed measurement 6e3a9e71 to history. Total: 4
    th(0)=-0.3925749361515045;dx=-9.416986176193715E-10
    New Minimum: -0.3925749361515045 > -0.3981310725212097
    END: th(1.4000000000000007E7)=-0.3981310725212097; dx=-4.066950036441075E-10 evalInputDelta=0.0055561363697052
    Fitness changed from -0.3925749361515045 to -0.3981310725212097
    Iteration 15 complete. Error: -0.3981310725212097 Total: 1.4513; Orientation: 0.1047; Line Search: 0.9072
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/db2edf5a-1c9d-418f-83f8-184263bf9630.jpg)
    
    Final threshold in iteration 16: -0.3981310725212097 (> -Infinity) after 29.432s (< 1800.000s)
    

Returns:

    -0.3981310725212097