Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 244.74 seconds (5.745 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 1695976390875
    Reset training subject: 1700452085151
    Adding measurement 62265cf1 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=4.0573520809412;dx=-3.830755982502676E-7
    New Minimum: 4.0573520809412 > 4.057351008057594
    WOLFE (weak): th(2.154434690031884)=4.057351008057594; dx=-3.831399362094895E-7 evalInputDelta=1.0728836059570312E-6
    New Minimum: 4.057351008057594 > 4.057349771261215
    WOLFE (weak): th(4.308869380063768)=4.057349771261215; dx=-3.831013822143576E-7 evalInputDelta=2.3096799850463867E-6
    New Minimum: 4.057349771261215 > 4.0573447942733765
    WOLFE (weak): th(12.926608140191302)=4.0573447942733765; dx=-3.829613239874673E-7 evalInputDelta=7.286667823791504E-6
    New Minimum: 4.0573447942733765 > 4.057322010397911
    WOLFE (weak): th(51.70643256076521)=4.057322010397911; dx=-3.8316418024532485E-7 evalInputDelta=3.007054328918457E-5
    New Minimum: 4.057322010397911 > 4.057201385498047
    WOLFE (weak): th(258.53216280382605)=4.057201385498047; dx=-3.831777953092027E-7 evalInputDelta=1.5069544315338135E-4
    New Minimum: 4.057201385498047 > 4.056446999311447
    WOLFE (weak): th(1551.1929768229563)=4.056446999311447; dx=-3.8315944515458076E-7 evalInputDelta=9.050816297531128E-4
    New Minimum: 4.056446999311447 > 4.051018074154854
    WOLFE (weak): th(10858.350837760694)=4.051018074154854; dx=-3.829154214475461E-7 evalInputDelta=0.0063340067863464355
    New Minimum: 4.051018074154854 > 4.006748020648956
    WOLFE (weak): th(86866.80670208555)=4.006748020648956; dx=-3.818603969650656E-7 evalInputDelta=0.05060406029224396
    New Minimum: 4.006748020648956 > 3.6102137714624405
    WOLFE (weak): th(781801.26031877)=3.6102137714624405; dx=-3.6729602803636064E-7 evalInputDelta=0.44713830947875977
    New Minimum: 3.6102137714624405 > 1.6283849030733109
    END: th(7818012.6031877)=1.6283849030733109; dx=-2.494270664718057E-8 evalInputDelta=2.4289671778678894
    Fitness changed from 4.0573520809412 to 1.6283849030733109
    Iteration 1 complete. Error: 1.6283849030733109 Total: 66.2340; 

...skipping 6660 bytes...

    tion rejected. Popping history element from -1.1644974499940872, 0.6140206009149551, 1.6283849030733109, 4.0573520809412
    LBFGS Accumulation History: 3 points
    Removed measurement 253cf61d to history. Total: 3
    th(0)=-1.1644974499940872;dx=-2.3607507026094336E-8
    Armijo: th(1.2600000000000004E7)=-0.7880789786577225; dx=5.3945348590452774E-8 evalInputDelta=-0.37641847133636475
    New Minimum: -1.1644974499940872 > -1.1754840910434723
    WOLF (strong): th(6300000.000000002)=-1.1754840910434723; dx=1.4095911856221892E-8 evalInputDelta=0.01098664104938507
    New Minimum: -1.1754840910434723 > -1.2113217189908028
    END: th(2100000.0000000005)=-1.2113217189908028; dx=-1.1122344886149388E-8 evalInputDelta=0.046824268996715546
    Fitness changed from -1.1644974499940872 to -1.2113217189908028
    Iteration 9 complete. Error: -1.2113217189908028 Total: 22.6595; Orientation: 0.4889; Line Search: 18.0905
    Adding measurement 2c670cba to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.712e+03, gradient 1.035e-04, dot -0.795; [67ba4518-c610-4430-b92a-fdaca925002b = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -1.2113217189908028, 0.6140206009149551, 1.6283849030733109, 4.0573520809412
    LBFGS Accumulation History: 3 points
    Removed measurement 2c670cba to history. Total: 3
    th(0)=-1.2113217189908028;dx=-1.067367394288838E-8
    New Minimum: -1.2113217189908028 > -1.2677194252610207
    WOLFE (weak): th(4524312.849066957)=-1.2677194252610207; dx=-9.649203713590465E-9 evalInputDelta=0.056397706270217896
    New Minimum: -1.2677194252610207 > -1.317408137023449
    END: th(9048625.698133914)=-1.317408137023449; dx=-8.531287675920989E-9 evalInputDelta=0.10608641803264618
    Fitness changed from -1.2113217189908028 to -1.317408137023449
    Iteration 10 complete. Error: -1.317408137023449 Total: 20.5351; Orientation: 0.5338; Line Search: 14.3829
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/d2853c2e-ec90-4102-ac37-771057c4d789.jpg)
    
    Final threshold in iteration 11: -1.317408137023449 (> -Infinity) after 244.740s (< 3600.000s)
    

Returns:

    -1.317408137023449