Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 3166.63 seconds (32.365 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1261406656467200
    Reset training subject: 1261427468512400
    Adding measurement 4a256169 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.9152823313776063;dx=-2.897872276066674E-7
    New Minimum: 0.9152823313776063 > 0.9152821985510524
    WOLFE (weak): th(2.154434690031884)=0.9152821985510524; dx=-2.8698644142247935E-7 evalInputDelta=1.328265538669271E-7
    New Minimum: 0.9152821985510524 > 0.9152818955860529
    WOLFE (weak): th(4.308869380063768)=0.9152818955860529; dx=-2.8698639273157716E-7 evalInputDelta=4.3579155339923403E-7
    New Minimum: 0.9152818955860529 > 0.9152811673932979
    WOLFE (weak): th(12.926608140191302)=0.9152811673932979; dx=-2.8698558774854936E-7 evalInputDelta=1.1639843083832702E-6
    New Minimum: 0.9152811673932979 > 0.9152774444899828
    WOLFE (weak): th(51.70643256076521)=0.9152774444899828; dx=-2.8698847444930754E-7 evalInputDelta=4.8868876234564596E-6
    New Minimum: 0.9152774444899828 > 0.9152578085038658
    WOLFE (weak): th(258.53216280382605)=0.9152578085038658; dx=-2.86967991581005E-7 evalInputDelta=2.4522873740440154E-5
    New Minimum: 0.9152578085038658 > 0.9151354642658279
    WOLFE (weak): th(1551.1929768229563)=0.9151354642658279; dx=-2.868936750023674E-7 evalInputDelta=1.4686711177835932E-4
    New Minimum: 0.9151354642658279 > 0.9142550725394406
    WOLFE (weak): th(10858.350837760694)=0.9142550725394406; dx=-2.865314674255181E-7 evalInputDelta=0.0010272588381656789
    New Minimum: 0.9142550725394406 > 0.9071230056493776
    WOLFE (weak): th(86866.80670208555)=0.9071230056493776; dx=-2.8357597851447783E-7 evalInputDelta=0.008159325728228639
    New Minimum: 0.9071230056493776 > 0.8469319064072494
    END: th(781801.26031877)=0.8469319064072494; dx=-2.557675046428731E-7 evalInputDelta=0.06835042497035693
    Fitness changed from 0.9152823313776063 to 0.8469319064072494
    Iteration 1 complete. Error: 0.8469319064072494 Total: 269.1174; Orientation: 0.0185; Line Search: 206.6731
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration

...skipping 27510 bytes...

    2365; dx=-1.523380420706227E-8 evalInputDelta=1.9711262587207568E-5
    WOLFE (weak): th(1714259.0181327178)=0.3305219672579777; dx=-1.5224037658049186E-8 evalInputDelta=9.177510333135963E-6
    WOLFE (weak): th(1736236.697852368)=0.330527450454505; dx=-1.5218268884863732E-8 evalInputDelta=3.6943138058687097E-6
    WOLFE (weak): th(1747225.537712193)=0.33053032459818765; dx=-1.5216381936095568E-8 evalInputDelta=8.201701232057879E-7
    Armijo: th(1752719.9576421059)=0.33053168359366764; dx=-1.5214825591818325E-8 evalInputDelta=-5.388253567772949E-7
    WOLFE (weak): th(1749972.7476771495)=0.33053106745535704; dx=-1.5215997001640168E-8 evalInputDelta=7.731295381674386E-8
    Armijo: th(1751346.3526596278)=0.3305313642513543; dx=-1.521526096286675E-8 evalInputDelta=-2.1948304346208758E-7
    Armijo: th(1750659.5501683885)=0.33053116936957605; dx=-1.5216350226757047E-8 evalInputDelta=-2.4601265191659394E-8
    WOLFE (weak): th(1750316.148922769)=0.33053102080533825; dx=-1.5216139204425205E-8 evalInputDelta=1.2396297260908185E-7
    WOLFE (weak): th(1750487.8495455787)=0.3305311124300119; dx=-1.5216380235822497E-8 evalInputDelta=3.233829898308116E-8
    Armijo: th(1750573.6998569835)=0.33053114030648917; dx=-1.5216333425076936E-8 evalInputDelta=4.461821689893952E-9
    Armijo: th(1750530.774701281)=0.33053114968291497; dx=-1.5216346761477898E-8 evalInputDelta=-4.9146041125602835E-9
    Armijo: th(1750509.31212343)=0.3305311273311731; dx=-1.5216347604890696E-8 evalInputDelta=1.7437137733722352E-8
    Armijo: th(1750498.5808345042)=0.3305311273311731; dx=-1.5216359629239208E-8 evalInputDelta=1.7437137733722352E-8
    mu ~= nu (1750487.8495455787): th(703285.7510288074)=0.330421140969504
    Fitness changed from 0.33053114476831086 to 0.330421140969504
    Iteration 30 complete. Error: 0.330421140969504 Total: 476.7153; Orientation: 0.5452; Line Search: 456.0395
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/6445ba7d-ea9c-4922-8615-bb6d6f8c9d37.jpg)
    
    Final threshold in iteration 31: 0.330421140969504 (> -Infinity) after 3166.627s (< 3600.000s)

Returns:

    0.330421140969504