Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Code from BasicOptimizer.scala:88 executed in 533.07 seconds (9.128 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 4383705260918
    Reset training subject: 4387099741097
    Adding measurement 7165845e to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=4.7952825129032135;dx=-2.4724729935112336E-7
    New Minimum: 4.7952825129032135 > 4.795282065868378
    WOLFE (weak): th(2.154434690031884)=4.795282065868378; dx=-2.472476233563785E-7 evalInputDelta=4.470348358154297E-7
    New Minimum: 4.795282065868378 > 4.795281857252121
    WOLFE (weak): th(4.308869380063768)=4.795281857252121; dx=-2.472492549715528E-7 evalInputDelta=6.556510925292969E-7
    New Minimum: 4.795281857252121 > 4.795280531048775
    WOLFE (weak): th(12.926608140191302)=4.795280531048775; dx=-2.472474008337848E-7 evalInputDelta=1.9818544387817383E-6
    New Minimum: 4.795280531048775 > 4.7952739000320435
    WOLFE (weak): th(51.70643256076521)=4.7952739000320435; dx=-2.4724793734161884E-7 evalInputDelta=8.612871170043945E-6
    New Minimum: 4.7952739000320435 > 4.795240134000778
    WOLFE (weak): th(258.53216280382605)=4.795240134000778; dx=-2.47236454476307E-7 evalInputDelta=4.2378902435302734E-5
    New Minimum: 4.795240134000778 > 4.795028924942017
    WOLFE (weak): th(1551.1929768229563)=4.795028924942017; dx=-2.471962012799092E-7 evalInputDelta=2.535879611968994E-4
    New Minimum: 4.795028924942017 > 4.793508321046829
    WOLFE (weak): th(10858.350837760694)=4.793508321046829; dx=-2.470035576692392E-7 evalInputDelta=0.0017741918563842773
    New Minimum: 4.793508321046829 > 4.781184121966362
    WOLFE (weak): th(86866.80670208555)=4.781184121966362; dx=-2.4518339208910543E-7 evalInputDelta=0.014098390936851501
    New Minimum: 4.781184121966362 > 4.676478534936905
    WOLFE (weak): th(781801.26031877)=4.676478534936905; dx=-2.268129196889803E-7 evalInputDelta=0.1188039779663086
    New Minimum: 4.676478534936905 > 4.40969155728817
    END: th(7818012.6031877)=4.40969155728817; dx=-8.512102913471359E-8 evalInputDelta=0.38559095561504364
    Fitness changed from 4.7952825129032135 to 4.40969155728817
    Iteration 1 complete. Error: 4.40969155728817 Total: 48.0529; Ori

...skipping 16317 bytes...

    231613E-4)=4.325307443737984; dx=-1.0894038254535755E-7 evalInputDelta=-1.1920928955078125E-7
    Armijo: th(5.140051204052078E-4)=4.325307443737984; dx=-1.0894038239612542E-7 evalInputDelta=-1.1920928955078125E-7
    Armijo: th(4.988873227462312E-4)=4.325307443737984; dx=-1.0894038269524573E-7 evalInputDelta=-1.1920928955078125E-7
    WOLFE (weak): th(4.913284239167428E-4)=4.325307324528694; dx=-1.0894038219874054E-7 evalInputDelta=0.0
    WOLFE (weak): th(4.95107873331487E-4)=4.325307324528694; dx=-1.0894038220710615E-7 evalInputDelta=0.0
    Armijo: th(4.969975980388591E-4)=4.325307443737984; dx=-1.0894038232440975E-7 evalInputDelta=-1.1920928955078125E-7
    Armijo: th(4.96052735685173E-4)=4.325307443737984; dx=-1.0894038227930824E-7 evalInputDelta=-1.1920928955078125E-7
    Armijo: th(4.9558030450833E-4)=4.325307443737984; dx=-1.0894038244213206E-7 evalInputDelta=-1.1920928955078125E-7
    Armijo: th(4.953440889199085E-4)=4.325307443737984; dx=-1.0894038218294021E-7 evalInputDelta=-1.1920928955078125E-7
    Armijo: th(4.952259811256978E-4)=4.325307443737984; dx=-1.0894038225377481E-7 evalInputDelta=-1.1920928955078125E-7
    Armijo: th(4.951669272285924E-4)=4.325307443737984; dx=-1.0894038225330891E-7 evalInputDelta=-1.1920928955078125E-7
    WOLFE (weak): th(4.951374002800397E-4)=4.325307324528694; dx=-1.0894038236696214E-7 evalInputDelta=0.0
    WOLFE (weak): th(4.951521637543161E-4)=4.325307324528694; dx=-1.0894038231735918E-7 evalInputDelta=0.0
    Armijo: th(4.951595454914543E-4)=4.325307443737984; dx=-1.0894038232869616E-7 evalInputDelta=-1.1920928955078125E-7
    WOLFE (weak): th(4.951558546228852E-4)=4.325307324528694; dx=-1.0894038224236691E-7 evalInputDelta=0.0
    mu ~= nu (4.951558546228852E-4): th(0.0)=4.325307324528694
    Fitness changed from 4.325307324528694 to 4.325307324528694
    Static Iteration Total: 80.1010; Orientation: 0.0125; Line Search: 73.3098
    Iteration 8 failed. Error: 4.325307324528694
    Previous Error: 0.0 -> 4.325307324528694
    Optimization terminated 8
    Final threshold in iteration 8: 4.325307324528694 (> -Infinity) after 533.070s (< 3600.000s)
    

Returns:

    4.325307324528694