Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:75 executed in 2038.34 seconds (13.407 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 951880983294600
    Reset training subject: 951929434366400
    Adding measurement 652af187 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=4.978843554854393;dx=-5.23246812533169E-4
    New Minimum: 4.978843554854393 > 4.978191085159779
    WOLFE (weak): th(2.154434690031884)=4.978191085159779; dx=-5.224569628649777E-4 evalInputDelta=6.524696946144104E-4
    New Minimum: 4.978191085159779 > 4.977540038526058
    WOLFE (weak): th(4.308869380063768)=4.977540038526058; dx=-5.1976105566554E-4 evalInputDelta=0.0013035163283348083
    New Minimum: 4.977540038526058 > 4.974967814981937
    WOLFE (weak): th(12.926608140191302)=4.974967814981937; dx=-5.127869919174235E-4 evalInputDelta=0.003875739872455597
    New Minimum: 4.974967814981937 > 4.963103346526623
    WOLFE (weak): th(51.70643256076521)=4.963103346526623; dx=-5.012527503757878E-4 evalInputDelta=0.015740208327770233
    New Minimum: 4.963103346526623 > 4.903527051210403
    END: th(258.53216280382605)=4.903527051210403; dx=-4.686759495783387E-4 evalInputDelta=0.07531650364398956
    Fitness changed from 4.978843554854393 to 4.903527051210403
    Iteration 1 complete. Error: 4.903527051210403 Total: 433.7483; Orientation: 0.0649; Line Search: 289.5712
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/5077d0a0-690d-48cd-81e2-abe0c7b5cbdc.jpg)
    
    Adding measurement 1b96af82 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=4.903527051210403;dx=-4.468955869261969E-4
    New Minimum: 4.903527051210403 > 4.76745043694973
    END: th(556.9906600335335)=4.76745043694973; dx=-3.8638505224205947E-4 evalInputDelta=0.13607661426067352
    Fitness changed from 4.903527051210403 to 4.76745043694973
    Iteration 2 complete. Error: 4.76745043694973 Total: 144.4298; Orientation: 0.0618; Line Search: 96.1653
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/e214b185-cbbc-4b78-92b1-64efa5b64577.jpg)
    
    Adding measurement 6dee22dd to history. Total: 2
    LBFGS Accumulation History: 3 points
    th(0)=4.76745

...skipping 5262 bytes...

    n rejected. Popping history element from 3.9021017998456955, 3.483529895544052, 3.257017731666565, 3.0945203974843025
    LBFGS Accumulation History: 3 points
    Removed measurement 4be66e1a to history. Total: 4
    th(0)=3.0945203974843025;dx=-4.916967481245273E-5
    New Minimum: 3.0945203974843025 > 3.063801795244217
    WOLF (strong): th(10000.000000000004)=3.063801795244217; dx=1.4847507519102643E-5 evalInputDelta=0.030718602240085602
    New Minimum: 3.063801795244217 > 3.0234205052256584
    END: th(5000.000000000002)=3.0234205052256584; dx=-3.084402899819889E-6 evalInputDelta=0.0710998922586441
    Fitness changed from 3.0945203974843025 to 3.0234205052256584
    Iteration 9 complete. Error: 3.0234205052256584 Total: 194.5903; Orientation: 1.9195; Line Search: 144.6326
    Adding measurement 64ab49f4 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.505e+01, gradient 6.628e-03, dot -0.628; [76ea1118-34df-4890-9d12-2da0a4fe7ad7 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 3.483529895544052, 3.257017731666565, 3.0945203974843025, 3.0234205052256584
    LBFGS Accumulation History: 3 points
    Removed measurement d5085f1 to history. Total: 4
    th(0)=3.0234205052256584;dx=-4.3933756768486426E-5
    New Minimum: 3.0234205052256584 > 3.012283533811569
    WOLF (strong): th(10772.173450159424)=3.012283533811569; dx=2.058123190392515E-5 evalInputDelta=0.011136971414089203
    New Minimum: 3.012283533811569 > 2.950475297868252
    WOLF (strong): th(5386.086725079712)=2.950475297868252; dx=7.791314289309258E-7 evalInputDelta=0.07294520735740662
    END: th(1795.3622416932374)=2.978096403181553; dx=-2.5236948524231663E-5 evalInputDelta=0.04532410204410553
    Fitness changed from 3.0234205052256584 to 2.950475297868252
    Iteration 10 complete. Error: 2.950475297868252 Total: 242.5471; Orientation: 1.8916; Line Search: 192.6219
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/a43caffe-153c-4579-977d-d3c41f482bda.jpg)
    
    Final threshold in iteration 10: 2.950475297868252 (> -Infinity) after 2038.335s (< 1800.000s)
    

Returns:

    2.950475297868252