Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 3003.27 seconds (19.659 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1174460182989300
    Reset training subject: 1174480873039400
    Adding measurement 1724c649 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.8795898986723659;dx=-2.7408997503063936E-7
    New Minimum: 0.8795898986723659 > 0.8795897688049796
    WOLFE (weak): th(2.154434690031884)=0.8795897688049796; dx=-2.7113972005974983E-7 evalInputDelta=1.2986738628928407E-7
    New Minimum: 0.8795897688049796 > 0.8795894357303211
    WOLFE (weak): th(4.308869380063768)=0.8795894357303211; dx=-2.7113954972152314E-7 evalInputDelta=4.6294204480989976E-7
    New Minimum: 0.8795894357303211 > 0.8795885906871338
    WOLFE (weak): th(12.926608140191302)=0.8795885906871338; dx=-2.711392214774729E-7 evalInputDelta=1.3079852321240537E-6
    New Minimum: 0.8795885906871338 > 0.8795852747642474
    WOLFE (weak): th(51.70643256076521)=0.8795852747642474; dx=-2.711380676994539E-7 evalInputDelta=4.623908118506215E-6
    New Minimum: 0.8795852747642474 > 0.8795668523721158
    WOLFE (weak): th(258.53216280382605)=0.8795668523721158; dx=-2.711327847292018E-7 evalInputDelta=2.3046300250073593E-5
    New Minimum: 0.8795668523721158 > 0.8794518403469582
    WOLFE (weak): th(1551.1929768229563)=0.8794518403469582; dx=-2.71042356070991E-7 evalInputDelta=1.3805832540769725E-4
    New Minimum: 0.8794518403469582 > 0.8786247371860912
    WOLFE (weak): th(10858.350837760694)=0.8786247371860912; dx=-2.7063684812207565E-7 evalInputDelta=9.651614862746793E-4
    New Minimum: 0.8786247371860912 > 0.8719188145679395
    WOLFE (weak): th(86866.80670208555)=0.8719188145679395; dx=-2.6817784714761464E-7 evalInputDelta=0.007671084104426407
    New Minimum: 0.8719188145679395 > 0.815022763098312
    END: th(781801.26031877)=0.815022763098312; dx=-2.4305353337332967E-7 evalInputDelta=0.06456713557405391
    Fitness changed from 0.8795898986723659 to 0.815022763098312
    Iteration 1 complete. Error: 0.815022763098312 Total: 270.1928; Orientation: 0.0186; Line Search: 209.1861
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](e

...skipping 26653 bytes...

    d9e0c to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.213e+04, gradient 1.290e-04, dot -0.963; [be250050-4d13-419b-aef8-35017c48af85 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.3313488982989969, 0.3308108701417309, 0.3305343744837662, 0.33025674766832747
    LBFGS Accumulation History: 3 points
    Removed measurement 60adea01 to history. Total: 4
    th(0)=0.33025674766832747;dx=-1.663887464474044E-8
    Armijo: th(5274643.132716058)=0.3325397232725953; dx=-1.4284117224535752E-8 evalInputDelta=-0.002282975604267823
    Armijo: th(2637321.566358029)=0.33032819289814436; dx=-1.4922275476084613E-8 evalInputDelta=-7.144522981689105E-5
    New Minimum: 0.33025674766832747 > 0.3300456029272191
    WOLFE (weak): th(879107.1887860097)=0.3300456029272191; dx=-1.543356135787674E-8 evalInputDelta=2.1114474110839332E-4
    WOLFE (weak): th(1758214.3775720194)=0.3300690709527035; dx=-1.515907468080541E-8 evalInputDelta=1.8767671562397448E-4
    WOLFE (weak): th(2197767.9719650242)=0.33016961142278867; dx=-1.5042818478489245E-8 evalInputDelta=8.713624553879917E-5
    WOLFE (weak): th(2417544.7691615266)=0.33024115974714396; dx=-1.4984981427426194E-8 evalInputDelta=1.558792118350194E-5
    Armijo: th(2527433.167759778)=0.33028279509701636; dx=-1.495127737713736E-8 evalInputDelta=-2.6047428688891205E-5
    Armijo: th(2472488.968460652)=0.3302614053999386; dx=-1.496876972100119E-8 evalInputDelta=-4.6577316111240386E-6
    WOLFE (weak): th(2445016.8688110895)=0.33025117300155016; dx=-1.4976351664658995E-8 evalInputDelta=5.574666777308224E-6
    END: th(2458752.918635871)=0.3302563836305294; dx=-1.4970626090708522E-8 evalInputDelta=3.640377980729248E-7
    Fitness changed from 0.33025674766832747 to 0.3300456029272191
    Iteration 30 complete. Error: 0.3300456029272191 Total: 247.7560; Orientation: 0.5452; Line Search: 226.1000
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/2f16fc2f-be42-4f6d-b13d-978449a4a50f.jpg)
    
    Final threshold in iteration 31: 0.3300456029272191 (> -Infinity) after 3003.265s (< 3600.000s)

Returns:

    0.3300456029272191