Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 399.17 seconds (3.277 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 6417201401225
    Reset training subject: 6425118645363
    Adding measurement 76cc2700 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=7.0040093511343;dx=-2.1244385257089654E-7
    New Minimum: 7.0040093511343 > 7.00400872528553
    WOLFE (weak): th(2.154434690031884)=7.00400872528553; dx=-2.0991896963874832E-7 evalInputDelta=6.258487701416016E-7
    New Minimum: 7.00400872528553 > 7.004008382558823
    WOLFE (weak): th(4.308869380063768)=7.004008382558823; dx=-2.0989379977860682E-7 evalInputDelta=9.685754776000977E-7
    New Minimum: 7.004008382558823 > 7.004006281495094
    WOLFE (weak): th(12.926608140191302)=7.004006281495094; dx=-2.0985005071914014E-7 evalInputDelta=3.069639205932617E-6
    New Minimum: 7.004006281495094 > 7.0039976835250854
    WOLFE (weak): th(51.70643256076521)=7.0039976835250854; dx=-2.0982290174071186E-7 evalInputDelta=1.1667609214782715E-5
    New Minimum: 7.0039976835250854 > 7.003951355814934
    WOLFE (weak): th(258.53216280382605)=7.003951355814934; dx=-2.0980920127962332E-7 evalInputDelta=5.799531936645508E-5
    New Minimum: 7.003951355814934 > 7.0036613047122955
    WOLFE (weak): th(1551.1929768229563)=7.0036613047122955; dx=-2.0974579963359527E-7 evalInputDelta=3.480464220046997E-4
    New Minimum: 7.0036613047122955 > 7.0015749335289
    WOLFE (weak): th(10858.350837760694)=7.0015749335289; dx=-2.0984472887356623E-7 evalInputDelta=0.0024344176054000854
    New Minimum: 7.0015749335289 > 6.984563127160072
    WOLFE (weak): th(86866.80670208555)=6.984563127160072; dx=-2.0948023894354866E-7 evalInputDelta=0.019446223974227905
    New Minimum: 6.984563127160072 > 6.831221893429756
    WOLFE (weak): th(781801.26031877)=6.831221893429756; dx=-2.0407367412233664E-7 evalInputDelta=0.17278745770454407
    New Minimum: 6.831221893429756 > 5.601209685206413
    END: th(7818012.6031877)=5.601209685206413; dx=-1.3816833314906315E-7 evalInputDelta=1.402799665927887
    Fitness changed from 7.0040093511343 to 5.601209685206413
    Iteration 1 complete. Error: 5.601209685206413 Total: 116.4360; Orient

...skipping 5848 bytes...

    c5750 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 8.421e+03, gradient 1.174e-04, dot -0.249; [23c82323-72e7-481a-8ce8-0522ac93c031 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.4403597116470337, 4.053332477807999, 5.601209685206413, 7.0040093511343
    LBFGS Accumulation History: 3 points
    Removed measurement 679c5750 to history. Total: 3
    th(0)=-0.4403597116470337;dx=-1.3766672445792102E-8
    Armijo: th(2.268000000000001E8)=0.9655492305755615; dx=3.971899001789251E-9 evalInputDelta=-1.4059089422225952
    New Minimum: -0.4403597116470337 > -0.5107216835021973
    WOLF (strong): th(1.1340000000000004E8)=-0.5107216835021973; dx=5.486335728073191E-10 evalInputDelta=0.07036197185516357
    New Minimum: -0.5107216835021973 > -0.7662067413330078
    END: th(3.7800000000000015E7)=-0.7662067413330078; dx=-7.192843838205739E-9 evalInputDelta=0.3258470296859741
    Fitness changed from -0.4403597116470337 to -0.7662067413330078
    Iteration 9 complete. Error: -0.7662067413330078 Total: 42.3238; Orientation: 0.8074; Line Search: 32.9134
    Adding measurement 408d47a9 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 9.078e+03, gradient 8.903e-05, dot -0.282; [23c82323-72e7-481a-8ce8-0522ac93c031 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.7662067413330078, 4.053332477807999, 5.601209685206413, 7.0040093511343
    LBFGS Accumulation History: 3 points
    Removed measurement 408d47a9 to history. Total: 3
    th(0)=-0.7662067413330078;dx=-7.91427760980425E-9
    New Minimum: -0.7662067413330078 > -0.9542307108640671
    END: th(8.143763128320524E7)=-0.9542307108640671; dx=-2.868301559062652E-9 evalInputDelta=0.18802396953105927
    Fitness changed from -0.7662067413330078 to -0.9542307108640671
    Iteration 10 complete. Error: -0.9542307108640671 Total: 26.5649; Orientation: 0.8068; Line Search: 17.5537
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/b0f03fff-8d02-485c-ac0c-ab31786d0bce.jpg)
    
    Final threshold in iteration 11: -0.9542307108640671 (> -Infinity) after 399.169s (< 3600.000s)
    

Returns:

    -0.9542307108640671