Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 31.31 seconds (0.883 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 914480169607900
    Reset training subject: 914480651037900
    Adding measurement 1c226852 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=2.044800192117691;dx=-4.811804295524754E-6
    New Minimum: 2.044800192117691 > 2.044792801141739
    WOLFE (weak): th(2.154434690031884)=2.044792801141739; dx=-4.812259331600041E-6 evalInputDelta=7.3909759521484375E-6
    New Minimum: 2.044792801141739 > 2.044785164296627
    WOLFE (weak): th(4.308869380063768)=2.044785164296627; dx=-4.812008025196706E-6 evalInputDelta=1.5027821063995361E-5
    New Minimum: 2.044785164296627 > 2.044755332171917
    WOLFE (weak): th(12.926608140191302)=2.044755332171917; dx=-4.812444363735187E-6 evalInputDelta=4.485994577407837E-5
    New Minimum: 2.044755332171917 > 2.0446204990148544
    WOLFE (weak): th(51.70643256076521)=2.0446204990148544; dx=-4.809122555764752E-6 evalInputDelta=1.796931028366089E-4
    New Minimum: 2.0446204990148544 > 2.0439023673534393
    WOLFE (weak): th(258.53216280382605)=2.0439023673534393; dx=-4.800048984281362E-6 evalInputDelta=8.97824764251709E-4
    New Minimum: 2.0439023673534393 > 2.0393850952386856
    WOLFE (weak): th(1551.1929768229563)=2.0393850952386856; dx=-4.806922061696125E-6 evalInputDelta=0.005415096879005432
    New Minimum: 2.0393850952386856 > 2.006393790245056
    WOLFE (weak): th(10858.350837760694)=2.006393790245056; dx=-4.5696273263607484E-6 evalInputDelta=0.03840640187263489
    New Minimum: 2.006393790245056 > 1.780654028058052
    END: th(86866.80670208555)=1.780654028058052; dx=-2.4225970950910358E-6 evalInputDelta=0.264146164059639
    Fitness changed from 2.044800192117691 to 1.780654028058052
    Iteration 1 complete. Error: 1.780654028058052 Total: 5.7777; Orientation: 0.0035; Line Search: 4.4892
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/e2ebfecb-fcd9-4490-a069-8c6664c97a7b.jpg)
    
    Adding measurement 78968490 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=1.780654028058052;dx=-3.7426619282106594E-6
    New Minimum: 1.

...skipping 10211 bytes...

    .569; [10f53cd0-6787-413e-936f-47547a9b4a6d = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.01671135425567627, -0.06705808639526367, -0.10188791155815125, -0.11159953474998474
    LBFGS Accumulation History: 3 points
    Removed measurement badaf9c to history. Total: 4
    th(0)=-0.11159953474998474;dx=-2.1486807931367224E-8
    Armijo: th(2.599289746823158E7)=0.30027735233306885; dx=2.4456701869762487E-8 evalInputDelta=-0.4118768870830536
    Armijo: th(1.299644873411579E7)=-0.021436870098114014; dx=1.782157751239748E-8 evalInputDelta=-0.09016266465187073
    New Minimum: -0.11159953474998474 > -0.14194396138191223
    END: th(4332149.578038597)=-0.14194396138191223; dx=-4.4213213292065643E-10 evalInputDelta=0.03034442663192749
    Fitness changed from -0.11159953474998474 to -0.14194396138191223
    Iteration 14 complete. Error: -0.14194396138191223 Total: 2.4351; Orientation: 0.1046; Line Search: 1.8621
    Adding measurement 4519c13f to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 8.642e+02, gradient 1.130e-04, dot -0.584; [10f53cd0-6787-413e-936f-47547a9b4a6d = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.06705808639526367, -0.10188791155815125, -0.11159953474998474, -0.14194396138191223
    LBFGS Accumulation History: 3 points
    Removed measurement a74ee7b to history. Total: 4
    th(0)=-0.14194396138191223;dx=-1.2768585452956163E-8
    Armijo: th(9333333.33333334)=-0.1410488784313202; dx=4.447771234331065E-9 evalInputDelta=-8.95082950592041E-4
    New Minimum: -0.14194396138191223 > -0.1594504714012146
    END: th(4666666.66666667)=-0.1594504714012146; dx=-7.90525013374974E-10 evalInputDelta=0.017506510019302368
    Fitness changed from -0.14194396138191223 to -0.1594504714012146
    Iteration 15 complete. Error: -0.1594504714012146 Total: 1.9881; Orientation: 0.1093; Line Search: 1.4228
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/67b0c5ef-0be1-498b-948a-acb292fd5f34.jpg)
    
    Final threshold in iteration 16: -0.1594504714012146 (> -Infinity) after 31.305s (< 1800.000s)
    

Returns:

    -0.1594504714012146