Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 913.92 seconds (8.244 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1069200704226899
    Reset training subject: 1069209223596900
    Adding measurement 71166348 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=9.917843228406136;dx=-9.772815014688597E-6
    New Minimum: 9.917843228406136 > 9.917824814186469
    WOLFE (weak): th(2.154434690031884)=9.917824814186469; dx=-9.77264509317264E-6 evalInputDelta=1.8414219667306497E-5
    New Minimum: 9.917824814186469 > 9.917806325460996
    WOLFE (weak): th(4.308869380063768)=9.917806325460996; dx=-9.7726556343324E-6 evalInputDelta=3.690294514058223E-5
    New Minimum: 9.917806325460996 > 9.917731740875324
    WOLFE (weak): th(12.926608140191302)=9.917731740875324; dx=-9.772413839274592E-6 evalInputDelta=1.1148753081258178E-4
    New Minimum: 9.917731740875324 > 9.917397470409643
    WOLFE (weak): th(51.70643256076521)=9.917397470409643; dx=-9.771383657361025E-6 evalInputDelta=4.4575799649315684E-4
    New Minimum: 9.917397470409643 > 9.915612224012898
    WOLFE (weak): th(258.53216280382605)=9.915612224012898; dx=-9.764005767851347E-6 evalInputDelta=0.0022310043932378676
    New Minimum: 9.915612224012898 > 9.904481466560489
    WOLFE (weak): th(1551.1929768229563)=9.904481466560489; dx=-9.726060991209318E-6 evalInputDelta=0.013361761845647635
    New Minimum: 9.904481466560489 > 9.825549386909318
    WOLFE (weak): th(10858.350837760694)=9.825549386909318; dx=-9.480893444588707E-6 evalInputDelta=0.09229384149681863
    New Minimum: 9.825549386909318 > 9.252083280096233
    END: th(86866.80670208555)=9.252083280096233; dx=-7.762261649285677E-6 evalInputDelta=0.665759948309903
    Fitness changed from 9.917843228406136 to 9.252083280096233
    Iteration 1 complete. Error: 9.252083280096233 Total: 89.4279; Orientation: 0.0134; Line Search: 66.6199
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/3261bf11-b2ec-4a5e-b52e-325d6a15a5c4.jpg)
    
    Adding measurement 6f15f52a to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=9.252083280096233;dx=-6.835609749285827E-6
    New Minimum: 9.25208328

...skipping 23868 bytes...

    3, gradient 2.039e-04, dot -0.728; [545273eb-5236-4b15-ae5b-fed953e8be55 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.40565836626832263, 0.36014279993412757, 0.3252542972593351, 0.29721309340924185
    LBFGS Accumulation History: 3 points
    Removed measurement 15ac02d5 to history. Total: 4
    th(0)=0.29721309340924185;dx=-4.1564918678440365E-8
    Armijo: th(1.0445962524205728E7)=0.3933992202290484; dx=2.299777530908384E-9 evalInputDelta=-0.09618612681980654
    New Minimum: 0.29721309340924185 > 0.2779049455220681
    END: th(5222981.262102864)=0.2779049455220681; dx=-1.30038680592715E-8 evalInputDelta=0.01930814788717372
    Fitness changed from 0.29721309340924185 to 0.2779049455220681
    Iteration 29 complete. Error: 0.2779049455220681 Total: 29.3813; Orientation: 0.3818; Line Search: 21.9613
    Adding measurement 78545d40 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.731e+03, gradient 2.317e-04, dot -0.613; [545273eb-5236-4b15-ae5b-fed953e8be55 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.36014279993412757, 0.3252542972593351, 0.29721309340924185, 0.2779049455220681
    LBFGS Accumulation History: 3 points
    Removed measurement 4597e6e3 to history. Total: 4
    th(0)=0.2779049455220681;dx=-5.369941569474354E-8
    Armijo: th(1.125257201646092E7)=0.5834117798472231; dx=3.446177255420041E-8 evalInputDelta=-0.305506834325155
    Armijo: th(5626286.00823046)=0.29444877423083415; dx=7.115232216584643E-9 evalInputDelta=-0.01654382870876603
    New Minimum: 0.2779049455220681 > 0.23912036632373646
    END: th(1875428.6694101533)=0.23912036632373646; dx=-2.995322834694966E-8 evalInputDelta=0.03878457919833167
    Fitness changed from 0.2779049455220681 to 0.23912036632373646
    Iteration 30 complete. Error: 0.23912036632373646 Total: 38.1703; Orientation: 0.4331; Line Search: 30.0779
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/3f9a45b0-bc9f-4ca1-8aab-dd95a35778be.jpg)
    
    Final threshold in iteration 31: 0.23912036632373646 (> -Infinity) after 913.911s (< 3600.000s)
    

Returns:

    0.23912036632373646