Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Code from BasicOptimizer.scala:75 executed in 939.43 seconds (6.552 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1023725099230400
    Reset training subject: 1023733465489200
    Adding measurement b35ac04 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=11.453930355608463;dx=-3.0361119256459914E-5
    New Minimum: 11.453930355608463 > 11.453856319189072
    WOLFE (weak): th(2.154434690031884)=11.453856319189072; dx=-3.035853573603472E-5 evalInputDelta=7.403641939163208E-5
    New Minimum: 11.453856319189072 > 11.453781925141811
    WOLFE (weak): th(4.308869380063768)=11.453781925141811; dx=-3.03242944341072E-5 evalInputDelta=1.484304666519165E-4
    New Minimum: 11.453781925141811 > 11.453485660254955
    WOLFE (weak): th(12.926608140191302)=11.453485660254955; dx=-3.035984863847101E-5 evalInputDelta=4.446953535079956E-4
    New Minimum: 11.453485660254955 > 11.452152665704489
    WOLFE (weak): th(51.70643256076521)=11.452152665704489; dx=-3.034580601899306E-5 evalInputDelta=0.001777689903974533
    New Minimum: 11.452152665704489 > 11.445044957101345
    WOLFE (weak): th(258.53216280382605)=11.445044957101345; dx=-3.031198130297643E-5 evalInputDelta=0.008885398507118225
    New Minimum: 11.445044957101345 > 11.400783024728298
    WOLFE (weak): th(1551.1929768229563)=11.400783024728298; dx=-3.0106097881505946E-5 evalInputDelta=0.0531473308801651
    New Minimum: 11.400783024728298 > 11.090523645281792
    WOLFE (weak): th(10858.350837760694)=11.090523645281792; dx=-2.8513313558500192E-5 evalInputDelta=0.3634067103266716
    New Minimum: 11.090523645281792 > 9.145930178463459
    END: th(86866.80670208555)=9.145930178463459; dx=-1.6499448849892308E-5 evalInputDelta=2.3080001771450043
    Fitness changed from 11.453930355608463 to 9.145930178463459
    Iteration 1 complete. Error: 9.145930178463459 Total: 94.2148; Orientation: 0.0089; Line Search: 70.2931
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/e811f6c5-ec0c-463d-b53a-c8f518c93322.jpg)
    
    Adding measurement 6225569 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=9.145930178463459;dx=-1.242854831572865E-5
    N

...skipping 24344 bytes...

    037
    Adding measurement 768112e5 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.153e+03, gradient 9.598e-05, dot -0.638; [4682485d-b219-40d6-93d2-b0d87df83d7f = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -2.372233025729656, -2.394582986831665, -2.4160471856594086, -2.433436118066311
    LBFGS Accumulation History: 3 points
    Removed measurement 595aed34 to history. Total: 4
    th(0)=-2.433436118066311;dx=-9.195041554697241E-9
    New Minimum: -2.433436118066311 > -2.4476461485028267
    END: th(1.3927950032274304E7)=-2.4476461485028267; dx=-2.1654967649365489E-10 evalInputDelta=0.014210030436515808
    Fitness changed from -2.433436118066311 to -2.4476461485028267
    Iteration 29 complete. Error: -2.4476461485028267 Total: 23.6821; Orientation: 0.2466; Line Search: 15.8087
    Adding measurement 42048ec5 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.711e+03, gradient 1.056e-04, dot -0.619; [4682485d-b219-40d6-93d2-b0d87df83d7f = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -2.394582986831665, -2.4160471856594086, -2.433436118066311, -2.4476461485028267
    LBFGS Accumulation History: 3 points
    Removed measurement 1074fef2 to history. Total: 4
    th(0)=-2.4476461485028267;dx=-1.1143765600294486E-8
    Armijo: th(3.0006858710562456E7)=-2.1111018508672714; dx=1.7051696165745348E-8 evalInputDelta=-0.33654429763555527
    Armijo: th(1.5003429355281228E7)=-2.439915206283331; dx=6.719778675666441E-9 evalInputDelta=-0.007730942219495773
    New Minimum: -2.4476461485028267 > -2.478965789079666
    END: th(5001143.118427076)=-2.478965789079666; dx=-3.783557366325524E-9 evalInputDelta=0.03131964057683945
    Fitness changed from -2.4476461485028267 to -2.478965789079666
    Iteration 30 complete. Error: -2.478965789079666 Total: 38.7243; Orientation: 0.2789; Line Search: 30.7797
    <a id="p-21"></a>Iteration 30
    <a id="p-20"></a>![Iteration 30](etc/c0ab3d9e-aa9a-434a-a727-e273ea2c2816.jpg)
    
    Final threshold in iteration 31: -2.478965789079666 (> -Infinity) after 939.434s (< 3600.000s)
    

Returns:

    -2.478965789079666