Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Code from BasicOptimizer.scala:75 executed in 677.14 seconds (6.037 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 940027225963700
    Reset training subject: 940050637690200
    Adding measurement 11dba94d to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=4.182647597044706;dx=-9.783506073032324E-5
    New Minimum: 4.182647597044706 > 4.182593207806349
    END: th(2.154434690031884)=4.182593207806349; dx=-2.1967720364372247E-5 evalInputDelta=5.4389238357543945E-5
    Fitness changed from 4.182647597044706 to 4.182593207806349
    Iteration 1 complete. Error: 4.182593207806349 Total: 114.0046; Orientation: 0.0043; Line Search: 45.4009
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/2b201646-ee94-4a2b-8ad7-1c0cb659c984.jpg)
    
    Adding measurement 4891d2ab to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=4.182593207806349;dx=-1.6222244394322808E-5
    New Minimum: 4.182593207806349 > 4.182538729161024
    WOLFE (weak): th(4.641588833612779)=4.182538729161024; dx=-1.622456049114257E-5 evalInputDelta=5.447864532470703E-5
    New Minimum: 4.182538729161024 > 4.182484725490212
    WOLFE (weak): th(9.283177667225559)=4.182484725490212; dx=-1.6242341374764568E-5 evalInputDelta=1.0848231613636017E-4
    New Minimum: 4.182484725490212 > 4.182267274707556
    WOLFE (weak): th(27.849533001676676)=4.182267274707556; dx=-1.6207206796346414E-5 evalInputDelta=3.259330987930298E-4
    New Minimum: 4.182267274707556 > 4.181291026994586
    WOLFE (weak): th(111.3981320067067)=4.181291026994586; dx=-1.6262611132064796E-5 evalInputDelta=0.0013021808117628098
    New Minimum: 4.181291026994586 > 4.175157058984041
    WOLFE (weak): th(556.9906600335335)=4.175157058984041; dx=-1.5575915551986868E-5 evalInputDelta=0.007436148822307587
    New Minimum: 4.175157058984041 > 4.146064827218652
    WOLFE (weak): th(3341.9439602012008)=4.146064827218652; dx=-1.5365403071944944E-5 evalInputDelta=0.03652838058769703
    New Minimum: 4.146064827218652 > 3.9848637394607067
    END: th(23393.607721408407)=3.9848637394607067; dx=-7.677859008661745E-6 evalInputDelta=0.1977294683456421
    Fitness changed from 4

...skipping 1995 bytes...

     3.9848637394607067, 3.7164139337837696, 3.5575170777738094
    LBFGS Accumulation History: 3 points
    Removed measurement 4891d2ab to history. Total: 4
    th(0)=3.5575170777738094;dx=-1.657412642892832E-4
    Armijo: th(233936.07721408413)=3.9531471580266953; dx=1.8740632456161618E-6 evalInputDelta=-0.3956300802528858
    Armijo: th(116968.03860704206)=3.707739770412445; dx=7.423955167035101E-7 evalInputDelta=-0.15022269263863564
    New Minimum: 3.5575170777738094 > 3.4861150607466698
    WOLF (strong): th(38989.346202347355)=3.4861150607466698; dx=1.301240920906189E-6 evalInputDelta=0.07140201702713966
    New Minimum: 3.4861150607466698 > 3.4761022944003344
    WOLF (strong): th(9747.336550586839)=3.4761022944003344; dx=1.1718223545623601E-6 evalInputDelta=0.08141478337347507
    END: th(1949.4673101173678)=3.5294554084539413; dx=-5.337502162761014E-6 evalInputDelta=0.028061669319868088
    Fitness changed from 3.5575170777738094 to 3.4761022944003344
    Iteration 5 complete. Error: 3.4761022944003344 Total: 157.2783; Orientation: 0.1066; Line Search: 134.6628
    <a id="p-11"></a>Iteration 5
    <a id="p-10"></a>![Iteration 5](etc/978f21c0-bdda-4f1a-90a7-26fdef370117.jpg)
    
    Adding measurement 5da44962 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.961e+01, gradient 2.516e-03, dot -0.820; [a6417127-e10f-4b7b-b134-dcf333b212a8 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 3.9848637394607067, 3.7164139337837696, 3.5575170777738094, 3.4761022944003344
    LBFGS Accumulation History: 3 points
    Removed measurement 773878b2 to history. Total: 4
    th(0)=3.4761022944003344;dx=-6.3282309700195454E-6
    New Minimum: 3.4761022944003344 > 3.4573720693588257
    END: th(4200.000000000002)=3.4573720693588257; dx=-2.071135865822464E-6 evalInputDelta=0.018730225041508675
    Fitness changed from 3.4761022944003344 to 3.4573720693588257
    Iteration 6 complete. Error: 3.4573720693588257 Total: 67.3946; Orientation: 0.1148; Line Search: 44.9421
    Final threshold in iteration 7: 3.4573720693588257 (> -Infinity) after 677.139s (< 720.000s)
    

Returns:

    3.4573720693588257