Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:88 executed in 158.87 seconds (3.871 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 1855404168187
    Reset training subject: 1856583073620
    Adding measurement 770ede38 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.7009885013103485;dx=-5.406999640728108E-7
    New Minimum: 1.7009885013103485 > 1.7009871006011963
    WOLFE (weak): th(2.154434690031884)=1.7009871006011963; dx=-5.407863158805261E-7 evalInputDelta=1.4007091522216797E-6
    New Minimum: 1.7009871006011963 > 1.700985610485077
    WOLFE (weak): th(4.308869380063768)=1.700985610485077; dx=-5.406951412875904E-7 evalInputDelta=2.8908252716064453E-6
    New Minimum: 1.700985610485077 > 1.7009795606136322
    WOLFE (weak): th(12.926608140191302)=1.7009795606136322; dx=-5.404741021771445E-7 evalInputDelta=8.940696716308594E-6
    New Minimum: 1.7009795606136322 > 1.7009533643722534
    WOLFE (weak): th(51.70643256076521)=1.7009533643722534; dx=-5.411962432103236E-7 evalInputDelta=3.5136938095092773E-5
    New Minimum: 1.7009533643722534 > 1.7008123844861984
    WOLFE (weak): th(258.53216280382605)=1.7008123844861984; dx=-5.407117370378012E-7 evalInputDelta=1.7611682415008545E-4
    New Minimum: 1.7008123844861984 > 1.6999315023422241
    WOLFE (weak): th(1551.1929768229563)=1.6999315023422241; dx=-5.402291793477146E-7 evalInputDelta=0.0010569989681243896
    New Minimum: 1.6999315023422241 > 1.6935985535383224
    WOLFE (weak): th(10858.350837760694)=1.6935985535383224; dx=-5.39272874637256E-7 evalInputDelta=0.007389947772026062
    New Minimum: 1.6935985535383224 > 1.642488032579422
    WOLFE (weak): th(86866.80670208555)=1.642488032579422; dx=-5.276098587843011E-7 evalInputDelta=0.058500468730926514
    New Minimum: 1.642488032579422 > 1.2322756350040436
    END: th(781801.26031877)=1.2322756350040436; dx=-4.1031401869570923E-7 evalInputDelta=0.46871286630630493
    Fitness changed from 1.7009885013103485 to 1.2322756350040436
    Iteration 1 complete. Error: 1.2322756350040436 Total: 15.8752; Orientation: 0.0060; Line Search: 12.4134
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/858dbfff-6be9-41fe-8141-c1d20af66767.j

...skipping 25106 bytes...

    itude: 1.682e+03, gradient 6.684e-05, dot -0.542; [31a9088b-4027-4910-919f-5cf7a7faec39 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -1.409107118844986, 0.5690070688724518, 1.2322756350040436, 1.7009885013103485
    LBFGS Accumulation History: 3 points
    Removed measurement 6f8df288 to history. Total: 3
    th(0)=-1.409107118844986;dx=-4.45051105226352E-9
    New Minimum: -1.409107118844986 > -1.4182622879743576
    WOLF (strong): th(6610335.659848933)=-1.4182622879743576; dx=8.507556995003657E-10 evalInputDelta=0.009155169129371643
    New Minimum: -1.4182622879743576 > -1.4190524816513062
    END: th(3305167.8299244666)=-1.4190524816513062; dx=-9.001163041185653E-10 evalInputDelta=0.00994536280632019
    Fitness changed from -1.409107118844986 to -1.4190524816513062
    Iteration 29 complete. Error: -1.4190524816513062 Total: 5.5560; Orientation: 0.1747; Line Search: 4.1636
    Adding measurement 6ff2188f to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.736e+03, gradient 6.659e-05, dot -0.539; [31a9088b-4027-4910-919f-5cf7a7faec39 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -1.4190524816513062, 0.5690070688724518, 1.2322756350040436, 1.7009885013103485
    LBFGS Accumulation History: 3 points
    Removed measurement 6ff2188f to history. Total: 3
    th(0)=-1.4190524816513062;dx=-4.41252855882184E-9
    New Minimum: -1.4190524816513062 > -1.4256672337651253
    WOLF (strong): th(7120768.229166673)=-1.4256672337651253; dx=1.1457919115061868E-9 evalInputDelta=0.006614752113819122
    New Minimum: -1.4256672337651253 > -1.428412564098835
    END: th(3560384.1145833363)=-1.428412564098835; dx=-3.3117513489770653E-10 evalInputDelta=0.009360082447528839
    Fitness changed from -1.4190524816513062 to -1.428412564098835
    Iteration 30 complete. Error: -1.428412564098835 Total: 4.6294; Orientation: 0.1505; Line Search: 3.3705
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/de137a3b-3b90-4591-a08b-33bdc7f8bf9b.jpg)
    
    Final threshold in iteration 31: -1.428412564098835 (> -Infinity) after 158.868s (< 3600.000s)
    

Returns:

    -1.428412564098835