Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:75 executed in 1313.24 seconds (15.643 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1406304766703600
    Reset training subject: 1406310794648500
    Adding measurement 117b2cc6 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=11.181547485291958;dx=-7.511670397824067E-6
    New Minimum: 11.181547485291958 > 11.181543081998825
    WOLFE (weak): th(2.154434690031884)=11.181543081998825; dx=-7.476546751709633E-6 evalInputDelta=4.403293132781982E-6
    New Minimum: 11.181543081998825 > 11.181539334356785
    WOLFE (weak): th(4.308869380063768)=11.181539334356785; dx=-7.4765386760368015E-6 evalInputDelta=8.150935173034668E-6
    New Minimum: 11.181539334356785 > 11.181522622704506
    WOLFE (weak): th(12.926608140191302)=11.181522622704506; dx=-7.476770831434389E-6 evalInputDelta=2.4862587451934814E-5
    New Minimum: 11.181522622704506 > 11.18144816160202
    WOLFE (weak): th(51.70643256076521)=11.18144816160202; dx=-7.477013057111486E-6 evalInputDelta=9.932368993759155E-5
    New Minimum: 11.18144816160202 > 11.181053154170513
    WOLFE (weak): th(258.53216280382605)=11.181053154170513; dx=-7.474977187351988E-6 evalInputDelta=4.943311214447021E-4
    New Minimum: 11.181053154170513 > 11.178585529327393
    WOLFE (weak): th(1551.1929768229563)=11.178585529327393; dx=-7.46975038525376E-6 evalInputDelta=0.002961955964565277
    New Minimum: 11.178585529327393 > 11.160980612039566
    WOLFE (weak): th(10858.350837760694)=11.160980612039566; dx=-7.4100033651780485E-6 evalInputDelta=0.020566873252391815
    New Minimum: 11.160980612039566 > 11.027503713965416
    WOLFE (weak): th(86866.80670208555)=11.027503713965416; dx=-6.967026115192731E-6 evalInputDelta=0.1540437713265419
    New Minimum: 11.027503713965416 > 10.611618392169476
    END: th(781801.26031877)=10.611618392169476; dx=-3.7593319572429735E-6 evalInputDelta=0.5699290931224823
    Fitness changed from 11.181547485291958 to 10.611618392169476
    Iteration 1 complete. Error: 10.611618392169476 Total: 80.1278; Orientation: 0.0136; Line Search: 61.8794
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/4c62ae3a

...skipping 29285 bytes...

    Armijo: th(281362.2635955604)=9.866066440939903; dx=-2.293250201124707E-6 evalInputDelta=-7.458552718162537E-4
    Armijo: th(275245.6926478308)=9.865651689469814; dx=-2.295146327471784E-6 evalInputDelta=-3.311038017272949E-4
    Armijo: th(272187.40717396606)=9.865450397133827; dx=-2.2956384476569362E-6 evalInputDelta=-1.2981146574020386E-4
    Armijo: th(270658.2644370337)=9.865351155400276; dx=-2.296864486700047E-6 evalInputDelta=-3.056973218917847E-5
    WOLFE (weak): th(269893.69306856743)=9.865302138030529; dx=-2.2972234600856483E-6 evalInputDelta=1.84476375579834E-5
    Armijo: th(270275.97875280055)=9.865325957536697; dx=-2.2968980326861874E-6 evalInputDelta=-5.37186861038208E-6
    WOLFE (weak): th(270084.83591068396)=9.865314401686192; dx=-2.297059822956166E-6 evalInputDelta=6.183981895446777E-6
    WOLFE (weak): th(270180.40733174223)=9.865320362150669; dx=-2.2970611542738363E-6 evalInputDelta=2.2351741790771484E-7
    Armijo: th(270228.1930422714)=9.86532423645258; dx=-2.296994730530351E-6 evalInputDelta=-3.6507844924926758E-6
    Armijo: th(270204.3001870068)=9.865321449935436; dx=-2.297016456254632E-6 evalInputDelta=-8.642673492431641E-7
    Armijo: th(270192.3537593745)=9.865321338176727; dx=-2.2970473248097867E-6 evalInputDelta=-7.525086402893066E-7
    Armijo: th(270186.3805455584)=9.865321703255177; dx=-2.297050814346556E-6 evalInputDelta=-1.1175870895385742E-6
    WOLFE (weak): th(270183.3939386503)=9.865320079028606; dx=-2.297050953972133E-6 evalInputDelta=5.066394805908203E-7
    Armijo: th(270184.8872421044)=9.865321576595306; dx=-2.2970487221121274E-6 evalInputDelta=-9.909272193908691E-7
    mu ~= nu (270183.3939386503): th(97865.13516367319)=9.861402094364166
    Fitness changed from 9.865320585668087 to 9.861402094364166
    Iteration 20 complete. Error: 9.861402094364166 Total: 141.0010; Orientation: 0.3552; Line Search: 134.6170
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/8c0ee7a0-3641-43fd-959f-698a18a3d978.jpg)
    
    Final threshold in iteration 21: 9.861402094364166 (> -Infinity) after 1313.231s (< 3600.000s)
    

Returns:

    9.861402094364166