Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 141.96 seconds (5.099 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 1109035059638
    Reset training subject: 1111666580329
    Adding measurement 49814f27 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=11.312715165317059;dx=-1.2016681362639817E-5
    New Minimum: 11.312715165317059 > 11.312679331749678
    WOLFE (weak): th(2.154434690031884)=11.312679331749678; dx=-1.2008099190353486E-5 evalInputDelta=3.583356738090515E-5
    New Minimum: 11.312679331749678 > 11.312643885612488
    WOLFE (weak): th(4.308869380063768)=11.312643885612488; dx=-1.2013573060997078E-5 evalInputDelta=7.127970457077026E-5
    New Minimum: 11.312643885612488 > 11.312502451241016
    WOLFE (weak): th(12.926608140191302)=11.312502451241016; dx=-1.201316217614852E-5 evalInputDelta=2.127140760421753E-4
    New Minimum: 11.312502451241016 > 11.311864607036114
    WOLFE (weak): th(51.70643256076521)=11.311864607036114; dx=-1.2017928457239582E-5 evalInputDelta=8.505582809448242E-4
    New Minimum: 11.311864607036114 > 11.308459386229515
    WOLFE (weak): th(258.53216280382605)=11.308459386229515; dx=-1.2027867651885488E-5 evalInputDelta=0.0042557790875434875
    New Minimum: 11.308459386229515 > 11.287142008543015
    WOLFE (weak): th(1551.1929768229563)=11.287142008543015; dx=-1.2068600920890883E-5 evalInputDelta=0.025573156774044037
    New Minimum: 11.287142008543015 > 11.132015395909548
    WOLFE (weak): th(10858.350837760694)=11.132015395909548; dx=-1.2273720142792644E-5 evalInputDelta=0.18069976940751076
    New Minimum: 11.132015395909548 > 9.866149514913559
    WOLFE (weak): th(86866.80670208555)=9.866149514913559; dx=-1.1643759689066015E-5 evalInputDelta=1.4465656504034996
    New Minimum: 9.866149514913559 > 4.136226713657379
    END: th(781801.26031877)=4.136226713657379; dx=-1.2774237670205958E-6 evalInputDelta=7.176488451659679
    Fitness changed from 11.312715165317059 to 4.136226713657379
    Iteration 1 complete. Error: 4.136226713657379 Total: 37.0468; Orientation: 0.0232; Line Search: 29.0338
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/e03213f4-adec-4730-b358-424f1af24250.jp

...skipping 5853 bytes...

    .7186; Orientation: 0.3317; Line Search: 5.6314
    Adding measurement 19176fe5 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.129e+03, gradient 4.000e-04, dot -0.758; [6ad00860-0554-4cfd-8352-ba21973b24be = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.5479568541049957, 2.018716186285019, 4.136226713657379, 11.312715165317059
    LBFGS Accumulation History: 3 points
    Removed measurement 19176fe5 to history. Total: 3
    th(0)=-0.5479568541049957;dx=-1.5906516238080908E-7
    New Minimum: -0.5479568541049957 > -0.6589077115058899
    END: th(1260000.0)=-0.6589077115058899; dx=-2.775235479977007E-8 evalInputDelta=0.11095085740089417
    Fitness changed from -0.5479568541049957 to -0.6589077115058899
    Iteration 9 complete. Error: -0.6589077115058899 Total: 10.4748; Orientation: 0.3039; Line Search: 7.2270
    Adding measurement 29b2f924 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.584e+03, gradient 5.612e-04, dot -0.663; [6ad00860-0554-4cfd-8352-ba21973b24be = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.6589077115058899, 2.018716186285019, 4.136226713657379, 11.312715165317059
    LBFGS Accumulation History: 3 points
    Removed measurement 29b2f924 to history. Total: 3
    th(0)=-0.6589077115058899;dx=-3.1431943294104236E-7
    Armijo: th(2714587.7094401736)=2.4235540330410004; dx=1.9805781802300954E-6 evalInputDelta=-3.0824617445468903
    Armijo: th(1357293.8547200868)=-0.2529496103525162; dx=8.119063807112998E-7 evalInputDelta=-0.4059581011533737
    New Minimum: -0.6589077115058899 > -0.7465513274073601
    END: th(452431.2849066956)=-0.7465513274073601; dx=-2.170372252182595E-8 evalInputDelta=0.08764361590147018
    Fitness changed from -0.6589077115058899 to -0.7465513274073601
    Iteration 10 complete. Error: -0.7465513274073601 Total: 14.2793; Orientation: 0.3720; Line Search: 10.7116
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/0679fccc-330b-49bc-947b-bbde54d41f56.jpg)
    
    Final threshold in iteration 11: -0.7465513274073601 (> -Infinity) after 141.957s (< 3600.000s)
    

Returns:

    -0.7465513274073601