Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:75 executed in 210.14 seconds (5.777 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1494109020379100
    Reset training subject: 1494111638471500
    Adding measurement 33188612 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=76.51114066503942;dx=-0.0023056295065207863
    New Minimum: 76.51114066503942 > 76.50855717528611
    WOLFE (weak): th(2.154434690031884)=76.50855717528611; dx=-0.0023055917147439646 evalInputDelta=0.002583489753305912
    New Minimum: 76.50855717528611 > 76.50342114083469
    WOLFE (weak): th(4.308869380063768)=76.50342114083469; dx=-0.0023051231308714253 evalInputDelta=0.0077195242047309875
    New Minimum: 76.50342114083469 > 76.49310003779829
    WOLFE (weak): th(12.926608140191302)=76.49310003779829; dx=-0.0023054859179528864 evalInputDelta=0.018040627241134644
    New Minimum: 76.49310003779829 > 76.44365459494293
    WOLFE (weak): th(51.70643256076521)=76.44365459494293; dx=-0.0023038471312418853 evalInputDelta=0.06748607009649277
    New Minimum: 76.44365459494293 > 76.19349173083901
    WOLFE (weak): th(258.53216280382605)=76.19349173083901; dx=-0.0022838549893760608 evalInputDelta=0.3176489342004061
    New Minimum: 76.19349173083901 > 74.7246900331229
    WOLFE (weak): th(1551.1929768229563)=74.7246900331229; dx=-0.0022749275879614637 evalInputDelta=1.786450631916523
    New Minimum: 74.7246900331229 > 65.37099837698042
    END: th(10858.350837760694)=65.37099837698042; dx=-0.0017618261214426823 evalInputDelta=11.140142288058996
    Fitness changed from 76.51114066503942 to 65.37099837698042
    Iteration 1 complete. Error: 65.37099837698042 Total: 28.2251; Orientation: 0.0250; Line Search: 20.3695
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/7b568b24-17c2-4ff9-9a3f-2092d540eadc.jpg)
    
    Adding measurement 2e43c38d to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=65.37099837698042;dx=-0.0015478234211869833
    New Minimum: 65.37099837698042 > 50.797524970024824
    END: th(23393.607721408407)=50.797524970024824; dx=-9.791591612154176E-4 evalInputDelta=14.5734734069556
    Fitness changed from 65.3709983

...skipping 14813 bytes...

    l: 3
    Rejected: LBFGS Orientation magnitude: 1.222e+03, gradient 2.107e-03, dot -0.795; [a3f29a23-6c63-4733-a25f-61bc5798def2 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -35.53116774559021, -40.555911019444466, -44.313790117390454, -46.17943731788546
    LBFGS Accumulation History: 3 points
    Removed measurement 2ce03e86 to history. Total: 4
    th(0)=-46.17943731788546;dx=-4.43980178983486E-6
    New Minimum: -46.17943731788546 > -46.50195067632012
    WOLF (strong): th(523647.3204938611)=-46.50195067632012; dx=4.085295489384203E-7 evalInputDelta=0.3225133584346622
    END: th(261823.66024693055)=-46.46118417894468; dx=-1.2955423378920234E-6 evalInputDelta=0.28174686105921865
    Fitness changed from -46.17943731788546 to -46.50195067632012
    Iteration 19 complete. Error: -46.50195067632012 Total: 9.8770; Orientation: 0.0865; Line Search: 7.3034
    Adding measurement 1ac730cd to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.258e+02, gradient 1.614e-03, dot -0.831; [a3f29a23-6c63-4733-a25f-61bc5798def2 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -40.555911019444466, -44.313790117390454, -46.17943731788546, -46.50195067632012
    LBFGS Accumulation History: 3 points
    Removed measurement 4c731956 to history. Total: 4
    th(0)=-46.50195067632012;dx=-2.606271734192403E-6
    New Minimum: -46.50195067632012 > -46.57359539880417
    WOLF (strong): th(564081.9763071091)=-46.57359539880417; dx=1.2288729848617964E-6 evalInputDelta=0.07164472248405218
    New Minimum: -46.57359539880417 > -46.667495066300035
    END: th(282040.98815355456)=-46.667495066300035; dx=-1.4016399494179536E-7 evalInputDelta=0.16554438997991383
    Fitness changed from -46.50195067632012 to -46.667495066300035
    Iteration 20 complete. Error: -46.667495066300035 Total: 10.3580; Orientation: 0.0851; Line Search: 7.4284
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/c5353fa3-c5c5-46a6-aefd-2332bf43f13b.jpg)
    
    Final threshold in iteration 21: -46.667495066300035 (> -Infinity) after 210.119s (< 3600.000s)
    

Returns:

    -46.667495066300035