Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 703.43 seconds (7.350 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 14544561609941
    Reset training subject: 14561317170221
    Adding measurement 782a4fb9 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.1580933406949043;dx=-5.248732247869983E-8
    New Minimum: 1.1580933406949043 > 1.1580931767821312
    WOLFE (weak): th(2.154434690031884)=1.1580931767821312; dx=-5.248723694893465E-8 evalInputDelta=1.6391277313232422E-7
    New Minimum: 1.1580931767821312 > 1.158093087375164
    WOLFE (weak): th(4.308869380063768)=1.158093087375164; dx=-5.248720741186225E-8 evalInputDelta=2.5331974029541016E-7
    New Minimum: 1.158093087375164 > 1.1580926701426506
    WOLFE (weak): th(12.926608140191302)=1.1580926701426506; dx=-5.2487220415576575E-8 evalInputDelta=6.705522537231445E-7
    New Minimum: 1.1580926701426506 > 1.1580903828144073
    WOLFE (weak): th(51.70643256076521)=1.1580903828144073; dx=-5.248706797740811E-8 evalInputDelta=2.9578804969787598E-6
    New Minimum: 1.1580903828144073 > 1.1580781191587448
    WOLFE (weak): th(258.53216280382605)=1.1580781191587448; dx=-5.248671162571148E-8 evalInputDelta=1.5221536159515381E-5
    New Minimum: 1.1580781191587448 > 1.1580024361610413
    WOLFE (weak): th(1551.1929768229563)=1.1580024361610413; dx=-5.248208593610478E-8 evalInputDelta=9.090453386306763E-5
    New Minimum: 1.1580024361610413 > 1.157456986606121
    WOLFE (weak): th(10858.350837760694)=1.157456986606121; dx=-5.2441001585063315E-8 evalInputDelta=6.363540887832642E-4
    New Minimum: 1.157456986606121 > 1.1530180722475052
    WOLFE (weak): th(86866.80670208555)=1.1530180722475052; dx=-5.214968839710373E-8 evalInputDelta=0.005075268447399139
    New Minimum: 1.1530180722475052 > 1.1136450916528702
    WOLFE (weak): th(781801.26031877)=1.1136450916528702; dx=-4.9543614799317907E-8 evalInputDelta=0.04444824904203415
    New Minimum: 1.1136450916528702 > 0.8345713838934898
    END: th(7818012.6031877)=0.8345713838934898; dx=-2.6040887297648796E-8 evalInputDelta=0.3235219568014145
    Fitness changed from 1.1580933406949043 to 0.8345713838934898
    Iteration 1 complete. Error: 0.83457

...skipping 6018 bytes...

    18036961555481; dx=-5.7560448475283246E-9 evalInputDelta=0.11545246839523315
    Fitness changed from -0.13635122776031494 to -0.2518036961555481
    Iteration 8 complete. Error: -0.2518036961555481 Total: 46.2841; Orientation: 1.6121; Line Search: 28.2493
    Adding measurement 62f6f9c4 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.523e+04, gradient 9.356e-05, dot -0.704; [4374cd13-ef9c-4bcc-86a9-812898edc6cf = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.2518036961555481, 0.533400759100914, 0.8345713838934898, 1.1580933406949043
    LBFGS Accumulation History: 3 points
    Removed measurement 62f6f9c4 to history. Total: 3
    th(0)=-0.2518036961555481;dx=-8.754078993893988E-9
    New Minimum: -0.2518036961555481 > -0.3337986469268799
    END: th(1.2600000000000004E7)=-0.3337986469268799; dx=-6.077427025225796E-9 evalInputDelta=0.08199495077133179
    Fitness changed from -0.2518036961555481 to -0.3337986469268799
    Iteration 9 complete. Error: -0.3337986469268799 Total: 45.0432; Orientation: 1.6166; Line Search: 29.4828
    Adding measurement 67c08c10 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.324e+04, gradient 8.541e-05, dot -0.696; [4374cd13-ef9c-4bcc-86a9-812898edc6cf = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.3337986469268799, 0.533400759100914, 0.8345713838934898, 1.1580933406949043
    LBFGS Accumulation History: 3 points
    Removed measurement 67c08c10 to history. Total: 3
    th(0)=-0.3337986469268799;dx=-7.295425654611409E-9
    New Minimum: -0.3337986469268799 > -0.43813616037368774
    END: th(2.7145877094401743E7)=-0.43813616037368774; dx=-3.6989227714438855E-9 evalInputDelta=0.10433751344680786
    Fitness changed from -0.3337986469268799 to -0.43813616037368774
    Iteration 10 complete. Error: -0.43813616037368774 Total: 47.1222; Orientation: 1.6270; Line Search: 30.2755
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/6e4d87b1-d794-4f31-b52b-453cb561798c.jpg)
    
    Final threshold in iteration 11: -0.43813616037368774 (> -Infinity) after 703.430s (< 3600.000s)
    

Returns:

    -0.43813616037368774