Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 222.27 seconds (4.014 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 2917942808050
    Reset training subject: 2922091873175
    Adding measurement 5d7e7801 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=-1.22454484552145;dx=-7.000610307067112E-8
    New Minimum: -1.22454484552145 > -1.2245450466871262
    WOLFE (weak): th(2.154434690031884)=-1.2245450466871262; dx=-7.000000663901318E-8 evalInputDelta=2.0116567611694336E-7
    New Minimum: -1.2245450466871262 > -1.224545195698738
    WOLFE (weak): th(4.308869380063768)=-1.224545195698738; dx=-7.00299448425113E-8 evalInputDelta=3.501772880554199E-7
    New Minimum: -1.224545195698738 > -1.224546141922474
    WOLFE (weak): th(12.926608140191302)=-1.224546141922474; dx=-7.001043757913985E-8 evalInputDelta=1.296401023864746E-6
    New Minimum: -1.224546141922474 > -1.2245501428842545
    WOLFE (weak): th(51.70643256076521)=-1.2245501428842545; dx=-7.001431494164067E-8 evalInputDelta=5.297362804412842E-6
    New Minimum: -1.2245501428842545 > -1.2245714962482452
    WOLFE (weak): th(258.53216280382605)=-1.2245714962482452; dx=-7.000070096982947E-8 evalInputDelta=2.6650726795196533E-5
    New Minimum: -1.2245714962482452 > -1.2247045189142227
    WOLFE (weak): th(1551.1929768229563)=-1.2247045189142227; dx=-6.998890325614134E-8 evalInputDelta=1.5967339277267456E-4
    New Minimum: -1.2247045189142227 > -1.225662000477314
    WOLFE (weak): th(10858.350837760694)=-1.225662000477314; dx=-6.98159845946275E-8 evalInputDelta=0.0011171549558639526
    New Minimum: -1.225662000477314 > -1.2334035709500313
    WOLFE (weak): th(86866.80670208555)=-1.2334035709500313; dx=-6.875305677583149E-8 evalInputDelta=0.008858725428581238
    New Minimum: -1.2334035709500313 > -1.2977240830659866
    END: th(781801.26031877)=-1.2977240830659866; dx=-5.830684881593304E-8 evalInputDelta=0.07317923754453659
    Fitness changed from -1.22454484552145 to -1.2977240830659866
    Iteration 1 complete. Error: -1.2977240830659866 Total: 59.3623; Orientation: 0.0306; Line Search: 46.3715
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/814df041-156b-48c6-

...skipping 6039 bytes...

    56-4be5-8455-0d0a3955a3c2 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -1.651194915175438, -1.3922896981239319, -1.2977240830659866, -1.22454484552145
    LBFGS Accumulation History: 3 points
    Removed measurement 7acd9466 to history. Total: 3
    th(0)=-1.651194915175438;dx=-8.066849855974484E-9
    Armijo: th(1.5120000000000004E7)=-1.4294291585683823; dx=2.3515397473431123E-8 evalInputDelta=-0.22176575660705566
    Armijo: th(7560000.000000002)=-1.6377484649419785; dx=6.947838749270676E-9 evalInputDelta=-0.013446450233459473
    New Minimum: -1.651194915175438 > -1.6672967001795769
    END: th(2520000.0000000005)=-1.6672967001795769; dx=-3.142776888392246E-9 evalInputDelta=0.016101785004138947
    Fitness changed from -1.651194915175438 to -1.6672967001795769
    Iteration 9 complete. Error: -1.6672967001795769 Total: 22.9664; Orientation: 0.4803; Line Search: 18.0095
    Adding measurement 7cbdcf2d to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.123e+03, gradient 5.675e-05, dot -0.903; [feeabb50-ac56-4be5-8455-0d0a3955a3c2 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -1.6672967001795769, -1.3922896981239319, -1.2977240830659866, -1.22454484552145
    LBFGS Accumulation History: 3 points
    Removed measurement 7cbdcf2d to history. Total: 3
    th(0)=-1.6672967001795769;dx=-3.1977198838983305E-9
    New Minimum: -1.6672967001795769 > -1.685145191848278
    WOLFE (weak): th(5429175.418880348)=-1.685145191848278; dx=-2.9606911385179498E-9 evalInputDelta=0.017848491668701172
    New Minimum: -1.685145191848278 > -1.7011962048709393
    END: th(1.0858350837760696E7)=-1.7011962048709393; dx=-2.710979270665077E-9 evalInputDelta=0.03389950469136238
    Fitness changed from -1.6672967001795769 to -1.7011962048709393
    Iteration 10 complete. Error: -1.7011962048709393 Total: 19.8390; Orientation: 0.5506; Line Search: 14.4102
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/ffee554b-1aeb-4f96-b870-aadffd118e79.jpg)
    
    Final threshold in iteration 11: -1.7011962048709393 (> -Infinity) after 222.265s (< 3600.000s)
    

Returns:

    -1.7011962048709393