Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 981.42 seconds (10.822 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1234149711580400
    Reset training subject: 1234158256891200
    Adding measurement 5d3b58ca to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=5.1561573490079535;dx=-2.426132182546419E-5
    New Minimum: 5.1561573490079535 > 5.156134398610975
    WOLFE (weak): th(2.154434690031884)=5.156134398610975; dx=-2.426110982829732E-5 evalInputDelta=2.2950396978416165E-5
    New Minimum: 5.156134398610975 > 5.156111998980197
    WOLFE (weak): th(4.308869380063768)=5.156111998980197; dx=-2.4260976297328376E-5 evalInputDelta=4.5350027756185796E-5
    New Minimum: 5.156111998980197 > 5.156020872886941
    WOLFE (weak): th(12.926608140191302)=5.156020872886941; dx=-2.4260434162895896E-5 evalInputDelta=1.3647612101230777E-4
    New Minimum: 5.156020872886941 > 5.155612190460412
    WOLFE (weak): th(51.70643256076521)=5.155612190460412; dx=-2.425308420579512E-5 evalInputDelta=5.451585475411136E-4
    New Minimum: 5.155612190460412 > 5.1534330099113514
    WOLFE (weak): th(258.53216280382605)=5.1534330099113514; dx=-2.4227249723269397E-5 evalInputDelta=0.0027243390966020797
    New Minimum: 5.1534330099113514 > 5.139876201085166
    WOLFE (weak): th(1551.1929768229563)=5.139876201085166; dx=-2.40326755657117E-5 evalInputDelta=0.01628114792278712
    New Minimum: 5.139876201085166 > 5.045020031467437
    WOLFE (weak): th(10858.350837760694)=5.045020031467437; dx=-2.2874650903117126E-5 evalInputDelta=0.11113731754051681
    New Minimum: 5.045020031467437 > 4.441147934873828
    END: th(86866.80670208555)=4.441147934873828; dx=-1.4586931679779035E-5 evalInputDelta=0.7150094141341254
    Fitness changed from 5.1561573490079535 to 4.441147934873828
    Iteration 1 complete. Error: 4.441147934873828 Total: 97.4930; Orientation: 0.0187; Line Search: 72.9260
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/5a9c7c3f-9ccb-4548-924f-658c615d84c7.jpg)
    
    Adding measurement 7b2ccba5 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=4.441147934873828;dx=-1.1413232592324706E-5
    New Minim

...skipping 24120 bytes...

    754; [8cce2303-05da-4ae0-a41a-11c0e13ef481 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.0827835409669055, 0.07318051893568799, 0.04892514907706233, 0.03320144386549599
    LBFGS Accumulation History: 3 points
    Removed measurement 4a6c0f38 to history. Total: 4
    th(0)=0.03320144386549599;dx=-4.1117579124384445E-8
    Armijo: th(6963975.01613715)=0.05509933650815481; dx=3.5077557775421326E-9 evalInputDelta=-0.021897892642658817
    New Minimum: 0.03320144386549599 > 0.022867153776466503
    END: th(3481987.508068575)=0.022867153776466503; dx=-1.067046650363563E-8 evalInputDelta=0.010334290089029488
    Fitness changed from 0.03320144386549599 to 0.022867153776466503
    Iteration 29 complete. Error: 0.022867153776466503 Total: 32.2954; Orientation: 0.2020; Line Search: 24.0917
    Adding measurement 19fec3d6 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.869e+03, gradient 2.360e-04, dot -0.633; [8cce2303-05da-4ae0-a41a-11c0e13ef481 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.07318051893568799, 0.04892514907706233, 0.03320144386549599, 0.022867153776466503
    LBFGS Accumulation History: 3 points
    Removed measurement 64021427 to history. Total: 4
    th(0)=0.022867153776466503;dx=-5.571128954330931E-8
    Armijo: th(7501714.677640612)=0.13005190384364396; dx=4.946365524442517E-8 evalInputDelta=-0.10718475006717745
    Armijo: th(3750857.338820306)=0.023973255473082503; dx=1.845056985526655E-8 evalInputDelta=-0.0011061016966159992
    New Minimum: 0.022867153776466503 > 0.006625885042367385
    END: th(1250285.7796067686)=0.006625885042367385; dx=-2.9107311693218372E-8 evalInputDelta=0.016241268734099118
    Fitness changed from 0.022867153776466503 to 0.006625885042367385
    Iteration 30 complete. Error: 0.006625885042367385 Total: 39.8002; Orientation: 0.2019; Line Search: 31.7907
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/d4121ccf-c981-4eac-95ce-e46c05131bbe.jpg)
    
    Final threshold in iteration 31: 0.006625885042367385 (> -Infinity) after 981.405s (< 3600.000s)
    

Returns:

    0.006625885042367385