Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 531.43 seconds (8.383 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 609141520246900
    Reset training subject: 609152445873200
    Adding measurement 18542438 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=9.373434767127037;dx=-1.143875320577291E-6
    New Minimum: 9.373434767127037 > 9.373430952429771
    WOLFE (weak): th(2.154434690031884)=9.373430952429771; dx=-1.142246502225933E-6 evalInputDelta=3.814697265625E-6
    New Minimum: 9.373430952429771 > 9.373427629470825
    WOLFE (weak): th(4.308869380063768)=9.373427629470825; dx=-1.1423344732181665E-6 evalInputDelta=7.137656211853027E-6
    New Minimum: 9.373427629470825 > 9.373413041234016
    WOLFE (weak): th(12.926608140191302)=9.373413041234016; dx=-1.142351095846459E-6 evalInputDelta=2.1725893020629883E-5
    New Minimum: 9.373413041234016 > 9.373348653316498
    WOLFE (weak): th(51.70643256076521)=9.373348653316498; dx=-1.1423731365778101E-6 evalInputDelta=8.61138105392456E-5
    New Minimum: 9.373348653316498 > 9.37300580739975
    WOLFE (weak): th(258.53216280382605)=9.37300580739975; dx=-1.1421453003650145E-6 evalInputDelta=4.289597272872925E-4
    New Minimum: 9.37300580739975 > 9.370864197611809
    WOLFE (weak): th(1551.1929768229563)=9.370864197611809; dx=-1.1419658808955448E-6 evalInputDelta=0.0025705695152282715
    New Minimum: 9.370864197611809 > 9.355461731553078
    WOLFE (weak): th(10858.350837760694)=9.355461731553078; dx=-1.1396903270890052E-6 evalInputDelta=0.01797303557395935
    New Minimum: 9.355461731553078 > 9.230639547109604
    WOLFE (weak): th(86866.80670208555)=9.230639547109604; dx=-1.1224224770590544E-6 evalInputDelta=0.14279522001743317
    New Minimum: 9.230639547109604 > 8.23781767487526
    END: th(781801.26031877)=8.23781767487526; dx=-8.229816976722729E-7 evalInputDelta=1.1356170922517776
    Fitness changed from 9.373434767127037 to 8.23781767487526
    Iteration 1 complete. Error: 8.23781767487526 Total: 143.6854; Orientation: 0.0287; Line Search: 110.3849
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/633ae134-cfc3-4600-8a0c-d7f4a6a9c667.jpg)

...skipping 6094 bytes...

    LBFGS Orientation magnitude: 2.543e+03, gradient 2.293e-04, dot -0.368; [ae17e3d2-8ca8-4bde-b353-c3167b84db9f = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.010116517543792725, 6.836750775575638, 8.23781767487526, 9.373434767127037
    LBFGS Accumulation History: 3 points
    Removed measurement 1c06e7b7 to history. Total: 3
    th(0)=-0.010116517543792725;dx=-5.255148693118352E-8
    New Minimum: -0.010116517543792725 > -0.2999553978443146
    END: th(7560000.000000001)=-0.2999553978443146; dx=-6.4427993936911645E-9 evalInputDelta=0.28983888030052185
    Fitness changed from -0.010116517543792725 to -0.2999553978443146
    Iteration 9 complete. Error: -0.2999553978443146 Total: 33.5702; Orientation: 0.6524; Line Search: 22.0035
    Adding measurement 4c487ef8 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.285e+03, gradient 2.038e-04, dot -0.430; [ae17e3d2-8ca8-4bde-b353-c3167b84db9f = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.2999553978443146, 6.836750775575638, 8.23781767487526, 9.373434767127037
    LBFGS Accumulation History: 3 points
    Removed measurement 4c487ef8 to history. Total: 3
    th(0)=-0.2999553978443146;dx=-4.148864151987935E-8
    New Minimum: -0.2999553978443146 > -0.3527531176805496
    WOLF (strong): th(1.6287526256641043E7)=-0.3527531176805496; dx=1.3663410560183919E-8 evalInputDelta=0.052797719836235046
    New Minimum: -0.3527531176805496 > -0.4849175661802292
    WOLF (strong): th(8143763.128320522)=-0.4849175661802292; dx=6.83112172369873E-10 evalInputDelta=0.1849621683359146
    END: th(2714587.709440174)=-0.4218137562274933; dx=-2.438171881270582E-8 evalInputDelta=0.12185835838317871
    Fitness changed from -0.2999553978443146 to -0.4849175661802292
    Iteration 10 complete. Error: -0.4849175661802292 Total: 56.1020; Orientation: 0.3790; Line Search: 44.2545
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/d6405277-98aa-412c-8c07-e8367c986d29.jpg)
    
    Final threshold in iteration 11: -0.4849175661802292 (> -Infinity) after 531.423s (< 3600.000s)
    

Returns:

    -0.4849175661802292