Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Code from BasicOptimizer.scala:75 executed in 3753.29 seconds (40.826 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1187318733744200
    Reset training subject: 1187372703958200
    Adding measurement a09fe7e to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.267790514197229;dx=-2.638604003746722E-7
    New Minimum: 1.267790514197229 > 1.2677903208667143
    WOLFE (weak): th(2.154434690031884)=1.2677903208667143; dx=-2.6036998361808776E-7 evalInputDelta=1.933305147083786E-7
    New Minimum: 1.2677903208667143 > 1.267790071689169
    WOLFE (weak): th(4.308869380063768)=1.267790071689169; dx=-2.6036996515694655E-7 evalInputDelta=4.4250805997236853E-7
    New Minimum: 1.267790071689169 > 1.2677894000509315
    WOLFE (weak): th(12.926608140191302)=1.2677894000509315; dx=-2.603697747962987E-7 evalInputDelta=1.1141462974428862E-6
    New Minimum: 1.2677894000509315 > 1.2677859197605306
    WOLFE (weak): th(51.70643256076521)=1.2677859197605306; dx=-2.603703256573426E-7 evalInputDelta=4.594436698424076E-6
    New Minimum: 1.2677859197605306 > 1.2677675827744221
    WOLFE (weak): th(258.53216280382605)=1.2677675827744221; dx=-2.6036501223353694E-7 evalInputDelta=2.29314228068489E-5
    New Minimum: 1.2677675827744221 > 1.2676526775688264
    WOLFE (weak): th(1551.1929768229563)=1.2676526775688264; dx=-2.603413338379829E-7 evalInputDelta=1.3783662840261002E-4
    New Minimum: 1.2676526775688264 > 1.2668257780936842
    WOLFE (weak): th(10858.350837760694)=1.2668257780936842; dx=-2.6016495756247186E-7 evalInputDelta=9.647361035447499E-4
    New Minimum: 1.2668257780936842 > 1.2601102100841732
    WOLFE (weak): th(86866.80670208555)=1.2601102100841732; dx=-2.5869393427124803E-7 evalInputDelta=0.007680304113055758
    New Minimum: 1.2601102100841732 > 1.2020853565648304
    WOLFE (weak): th(781801.26031877)=1.2020853565648304; dx=-2.4474495218901595E-7 evalInputDelta=0.06570515763239859
    New Minimum: 1.2020853565648304 > 0.890924745920601
    END: th(7818012.6031877)=0.890924745920601; dx=-1.3306970027760307E-7 evalInputDelta=0.376865768276628
    Fitness changed from 1.267790514197229 to 0.890924745920601
    Iteration

...skipping 10990 bytes...

    easurement 21d1f97f to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.067e+04, gradient 1.400e-04, dot -0.744; [720c5a8e-10c8-4a28-b228-7616b7dd1bc0 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.2045452964156549, 0.16146093700171527, 0.1263330167051701, 0.10288565754773729
    LBFGS Accumulation History: 3 points
    Removed measurement 68812a5c to history. Total: 4
    th(0)=0.10288565754773729;dx=-1.9603630638727868E-8
    Armijo: th(5.482876809705099E7)=0.24940199641503233; dx=-2.54632713845967E-9 evalInputDelta=-0.14651633886729504
    New Minimum: 0.10288565754773729 > 0.08792990649217541
    END: th(2.7414384048525494E7)=0.08792990649217541; dx=-7.2013616416018434E-9 evalInputDelta=0.014955751055561878
    Fitness changed from 0.10288565754773729 to 0.08792990649217541
    Iteration 14 complete. Error: 0.08792990649217541 Total: 219.0442; Orientation: 1.5082; Line Search: 164.1269
    Adding measurement 8d902f to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 7.467e+03, gradient 1.414e-04, dot -0.698; [720c5a8e-10c8-4a28-b228-7616b7dd1bc0 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.16146093700171527, 0.1263330167051701, 0.10288565754773729, 0.08792990649217541
    LBFGS Accumulation History: 3 points
    Removed measurement 3657201c to history. Total: 4
    th(0)=0.08792990649217541;dx=-1.9995528701692146E-8
    Armijo: th(5.9062500000000045E7)=0.30297197127655134; dx=-2.423468491627841E-10 evalInputDelta=-0.21504206478437593
    New Minimum: 0.08792990649217541 > 0.0875170307479206
    END: th(2.9531250000000022E7)=0.0875170307479206; dx=-4.859166247289371E-9 evalInputDelta=4.1287574425481754E-4
    Fitness changed from 0.08792990649217541 to 0.0875170307479206
    Iteration 15 complete. Error: 0.0875170307479206 Total: 213.9987; Orientation: 1.4830; Line Search: 159.3337
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/b3fd2e44-5aa1-40c6-b45c-0e0b74501b67.jpg)
    
    Final threshold in iteration 15: 0.0875170307479206 (> -Infinity) after 3753.284s (< 3600.000s)
    

Returns:

    0.0875170307479206