Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 30.36 seconds (1.043 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 910935254431700
    Reset training subject: 910935724510900
    Adding measurement 4453299e to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.9787459075450897;dx=-2.6186084981108507E-6
    New Minimum: 1.9787459075450897 > 1.9787417948246002
    WOLFE (weak): th(2.154434690031884)=1.9787417948246002; dx=-2.6186107342998077E-6 evalInputDelta=4.112720489501953E-6
    New Minimum: 1.9787417948246002 > 1.9787379205226898
    WOLFE (weak): th(4.308869380063768)=1.9787379205226898; dx=-2.6186359370894077E-6 evalInputDelta=7.987022399902344E-6
    New Minimum: 1.9787379205226898 > 1.9787226617336273
    WOLFE (weak): th(12.926608140191302)=1.9787226617336273; dx=-2.618531406778196E-6 evalInputDelta=2.3245811462402344E-5
    New Minimum: 1.9787226617336273 > 1.9786540865898132
    WOLFE (weak): th(51.70643256076521)=1.9786540865898132; dx=-2.6178000017019392E-6 evalInputDelta=9.182095527648926E-5
    New Minimum: 1.9786540865898132 > 1.978287249803543
    WOLFE (weak): th(258.53216280382605)=1.978287249803543; dx=-2.6187267286613023E-6 evalInputDelta=4.5865774154663086E-4
    New Minimum: 1.978287249803543 > 1.975990578532219
    WOLFE (weak): th(1551.1929768229563)=1.975990578532219; dx=-2.6160368151255077E-6 evalInputDelta=0.0027553290128707886
    New Minimum: 1.975990578532219 > 1.9592942744493484
    WOLFE (weak): th(10858.350837760694)=1.9592942744493484; dx=-2.5747738983727147E-6 evalInputDelta=0.019451633095741272
    New Minimum: 1.9592942744493484 > 1.8304488956928253
    END: th(86866.80670208555)=1.8304488956928253; dx=-1.9124453418940705E-6 evalInputDelta=0.1482970118522644
    Fitness changed from 1.9787459075450897 to 1.8304488956928253
    Iteration 1 complete. Error: 1.8304488956928253 Total: 5.4685; Orientation: 0.0035; Line Search: 4.1028
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/44e940ea-740b-4d5c-9b02-73f84abeb26f.jpg)
    
    Adding measurement 7d9bc433 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=1.8304488956928253;dx=-2.985395524686

...skipping 10170 bytes...

    ot -0.706; [6380a884-012c-4bb6-90a5-d70b147e253a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.10395655035972595, -0.14811304211616516, -0.18633633852005005, -0.2091374695301056
    LBFGS Accumulation History: 3 points
    Removed measurement 28c77795 to history. Total: 4
    th(0)=-0.2091374695301056;dx=-9.59052972962211E-9
    Armijo: th(3.898934620234736E7)=-0.11420723795890808; dx=1.979029645057973E-9 evalInputDelta=-0.09493023157119751
    New Minimum: -0.2091374695301056 > -0.2227778434753418
    END: th(1.949467310117368E7)=-0.2227778434753418; dx=-1.4541013011991122E-9 evalInputDelta=0.013640373945236206
    Fitness changed from -0.2091374695301056 to -0.2227778434753418
    Iteration 14 complete. Error: -0.2227778434753418 Total: 2.1414; Orientation: 0.1051; Line Search: 1.5916
    Adding measurement 67bde84e to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.209e+03, gradient 1.048e-04, dot -0.641; [6380a884-012c-4bb6-90a5-d70b147e253a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.14811304211616516, -0.18633633852005005, -0.2091374695301056, -0.2227778434753418
    LBFGS Accumulation History: 3 points
    Removed measurement 261b4fa to history. Total: 4
    th(0)=-0.2227778434753418;dx=-1.0974133979587653E-8
    Armijo: th(4.200000000000002E7)=-0.008796513080596924; dx=6.771079206166433E-9 evalInputDelta=-0.21398133039474487
    Armijo: th(2.100000000000001E7)=-0.21026578545570374; dx=1.9274206840522793E-9 evalInputDelta=-0.012512058019638062
    New Minimum: -0.2227778434753418 > -0.2541656345129013
    END: th(7000000.000000004)=-0.2541656345129013; dx=-4.683838255829687E-9 evalInputDelta=0.03138779103755951
    Fitness changed from -0.2227778434753418 to -0.2541656345129013
    Iteration 15 complete. Error: -0.2541656345129013 Total: 2.3696; Orientation: 0.1056; Line Search: 1.8083
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/c60743fd-5fd5-475e-babc-d079da38202d.jpg)
    
    Final threshold in iteration 16: -0.2541656345129013 (> -Infinity) after 30.359s (< 1800.000s)
    

Returns:

    -0.2541656345129013