Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 271.56 seconds (5.096 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 3393964553816
    Reset training subject: 3397425985986
    Adding measurement 7937c8d to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=12.072415869683027;dx=-2.054740034667741E-6
    New Minimum: 12.072415869683027 > 12.072412729263306
    WOLFE (weak): th(2.154434690031884)=12.072412729263306; dx=-2.0547733402837184E-6 evalInputDelta=3.1404197216033936E-6
    New Minimum: 12.072412729263306 > 12.072409346699715
    WOLFE (weak): th(4.308869380063768)=12.072409346699715; dx=-2.0547615331801363E-6 evalInputDelta=6.5229833126068115E-6
    New Minimum: 12.072409346699715 > 12.072395619004965
    WOLFE (weak): th(12.926608140191302)=12.072395619004965; dx=-2.054737278075657E-6 evalInputDelta=2.0250678062438965E-5
    New Minimum: 12.072395619004965 > 12.072334792464972
    WOLFE (weak): th(51.70643256076521)=12.072334792464972; dx=-2.0546510202131874E-6 evalInputDelta=8.10772180557251E-5
    New Minimum: 12.072334792464972 > 12.072011083364487
    WOLFE (weak): th(258.53216280382605)=12.072011083364487; dx=-2.0545278877686862E-6 evalInputDelta=4.047863185405731E-4
    New Minimum: 12.072011083364487 > 12.069987565279007
    WOLFE (weak): th(1551.1929768229563)=12.069987565279007; dx=-2.054024308826367E-6 evalInputDelta=0.0024283044040203094
    New Minimum: 12.069987565279007 > 12.0554402731359
    WOLFE (weak): th(10858.350837760694)=12.0554402731359; dx=-2.043906816933875E-6 evalInputDelta=0.01697559654712677
    New Minimum: 12.0554402731359 > 11.938690897077322
    WOLFE (weak): th(86866.80670208555)=11.938690897077322; dx=-1.9644652384137043E-6 evalInputDelta=0.13372497260570526
    New Minimum: 11.938690897077322 > 11.089171182364225
    END: th(781801.26031877)=11.089171182364225; dx=-1.2225429145985624E-6 evalInputDelta=0.9832446873188019
    Fitness changed from 12.072415869683027 to 11.089171182364225
    Iteration 1 complete. Error: 11.089171182364225 Total: 44.7014; Orientation: 0.0121; Line Search: 34.4402
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/af954f87-3e7d-4664-8d9c-5c4db0735d86.j

...skipping 9108 bytes...

    4.421188667416573; dx=-1.0281558684960461E-7 evalInputDelta=2.9802322387695312E-8
    Armijo: th(0.35543836993749267)=4.421188697218895; dx=-1.0281559538604798E-7 evalInputDelta=0.0
    Armijo: th(0.35145363484850733)=4.421188846230507; dx=-1.0281559506501407E-7 evalInputDelta=-1.4901161193847656E-7
    Armijo: th(0.34946126730401467)=4.421188727021217; dx=-1.0281559486337962E-7 evalInputDelta=-2.9802322387695312E-8
    Armijo: th(0.3484650835317683)=4.421188697218895; dx=-1.028155859994535E-7 evalInputDelta=0.0
    Armijo: th(0.34796699164564515)=4.421188727021217; dx=-1.0281558597492149E-7 evalInputDelta=-2.9802322387695312E-8
    Armijo: th(0.3477179457025836)=4.421188727021217; dx=-1.0281558604262891E-7 evalInputDelta=-2.9802322387695312E-8
    WOLFE (weak): th(0.34759342273105276)=4.421188667416573; dx=-1.0281558609758773E-7 evalInputDelta=2.9802322387695312E-8
    New Minimum: 4.421188667416573 > 4.421188607811928
    WOLFE (weak): th(0.34765568421681814)=4.421188607811928; dx=-1.0281558619815397E-7 evalInputDelta=8.940696716308594E-8
    WOLFE (weak): th(0.3476868149597009)=4.421188607811928; dx=-1.0281558602798775E-7 evalInputDelta=8.940696716308594E-8
    Armijo: th(0.3477023803311422)=4.421188727021217; dx=-1.0281558615996346E-7 evalInputDelta=-2.9802322387695312E-8
    WOLFE (weak): th(0.34769459764542154)=4.421188607811928; dx=-1.0281558621960101E-7 evalInputDelta=8.940696716308594E-8
    WOLFE (weak): th(0.3476984889882819)=4.421188607811928; dx=-1.0281558614205433E-7 evalInputDelta=8.940696716308594E-8
    Armijo: th(0.347700434659712)=4.421188727021217; dx=-1.0281558605964846E-7 evalInputDelta=-2.9802322387695312E-8
    mu ~= nu (0.3476984889882819): th(0.34765568421681814)=4.421188607811928
    Fitness changed from 4.421188697218895 to 4.421188607811928
    Iteration 10 complete. Error: 4.421188607811928 Total: 120.2815; Orientation: 0.3076; Line Search: 116.6761
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/7b08d6dd-93e0-4058-9f2f-db88ff0b13f1.jpg)
    
    Final threshold in iteration 11: 4.421188607811928 (> -Infinity) after 271.558s (< 3600.000s)
    

Returns:

    4.421188607811928