Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Code from BasicOptimizer.scala:75 executed in 3696.43 seconds (39.465 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1257593230918600
    Reset training subject: 1257613894329300
    Adding measurement e1619a to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=1.0116279445283325;dx=-2.9335869700326935E-7
    New Minimum: 1.0116279445283325 > 1.0116278315551546
    WOLFE (weak): th(2.154434690031884)=1.0116278315551546; dx=-2.903693976778283E-7 evalInputDelta=1.1297317792369199E-7
    New Minimum: 1.0116278315551546 > 1.0116275657631906
    WOLFE (weak): th(4.308869380063768)=1.0116275657631906; dx=-2.9036938235493365E-7 evalInputDelta=3.787651419173699E-7
    New Minimum: 1.0116275657631906 > 1.0116265461510194
    WOLFE (weak): th(12.926608140191302)=1.0116265461510194; dx=-2.9036916922091914E-7 evalInputDelta=1.3983773130554766E-6
    New Minimum: 1.0116265461510194 > 1.0116227672979914
    WOLFE (weak): th(51.70643256076521)=1.0116227672979914; dx=-2.9036900986023155E-7 evalInputDelta=5.177230341102401E-6
    New Minimum: 1.0116227672979914 > 1.0116024469546168
    WOLFE (weak): th(258.53216280382605)=1.0116024469546168; dx=-2.9036411222969697E-7 evalInputDelta=2.549757371572703E-5
    New Minimum: 1.0116024469546168 > 1.0114746479250367
    WOLFE (weak): th(1551.1929768229563)=1.0114746479250367; dx=-2.90310786882197E-7 evalInputDelta=1.5329660329577166E-4
    New Minimum: 1.0114746479250367 > 1.010556377989466
    WOLFE (weak): th(10858.350837760694)=1.010556377989466; dx=-2.9004303018992956E-7 evalInputDelta=0.0010715665388665307
    New Minimum: 1.010556377989466 > 1.0031040640372384
    WOLFE (weak): th(86866.80670208555)=1.0031040640372384; dx=-2.8738517405067475E-7 evalInputDelta=0.008523880491094094
    New Minimum: 1.0031040640372384 > 0.939388304613318
    END: th(781801.26031877)=0.939388304613318; dx=-2.612956131067515E-7 evalInputDelta=0.07223963991501448
    Fitness changed from 1.0116279445283325 to 0.939388304613318
    Iteration 1 complete. Error: 0.939388304613318 Total: 266.6563; Orientation: 0.0180; Line Search: 204.9834
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/8e

...skipping 29028 bytes...

    009804889555E-8 evalInputDelta=5.3042175426953886E-5
    WOLFE (weak): th(2122617.583626558)=0.3823106846890462; dx=-1.5323648406433562E-8 evalInputDelta=1.9631998797342565E-5
    WOLFE (weak): th(2178475.9410904148)=0.3823290329214756; dx=-1.5307017334840133E-8 evalInputDelta=1.2837663679876599E-6
    Armijo: th(2206405.119822343)=0.3823385190345563; dx=-1.5295870241429303E-8 evalInputDelta=-8.202346712726793E-6
    Armijo: th(2192440.530456379)=0.382333634474594; dx=-1.5300450052696883E-8 evalInputDelta=-3.3177867504452507E-6
    Armijo: th(2185458.2357733967)=0.38233129031970875; dx=-1.5303389396506214E-8 evalInputDelta=-9.736318651865083E-7
    WOLFE (weak): th(2181967.088431906)=0.38233016265426994; dx=-1.5305462708820304E-8 evalInputDelta=1.5403357361831738E-7
    Armijo: th(2183712.6621026513)=0.3823307856363861; dx=-1.5304447364673246E-8 evalInputDelta=-4.689485425535267E-7
    Armijo: th(2182839.8752672784)=0.3823305501525524; dx=-1.5305067762677547E-8 evalInputDelta=-2.3346470884311188E-7
    Armijo: th(2182403.481849592)=0.38233033595525656; dx=-1.5305238779802123E-8 evalInputDelta=-1.9267412998402733E-8
    WOLFE (weak): th(2182185.285140749)=0.3823302454975888; dx=-1.5305299618882674E-8 evalInputDelta=7.119025474677798E-8
    WOLFE (weak): th(2182294.3834951706)=0.38233030372399673; dx=-1.5305235802063534E-8 evalInputDelta=1.2963846829361358E-8
    WOLFE (weak): th(2182348.9326723814)=0.38233030060594086; dx=-1.5305237804383075E-8 evalInputDelta=1.608190269841714E-8
    Armijo: th(2182376.207260987)=0.38233033283720064; dx=-1.5305235922409674E-8 evalInputDelta=-1.6149357073835802E-8
    Armijo: th(2182362.569966684)=0.38233031550710206; dx=-1.5305232888489024E-8 evalInputDelta=1.1807415045694825E-9
    mu ~= nu (2182348.9326723814): th(893733.7194217085)=0.38215299013236487
    Fitness changed from 0.38233031668784356 to 0.38215299013236487
    Iteration 27 complete. Error: 0.38215299013236487 Total: 481.7684; Orientation: 0.5626; Line Search: 460.3889
    Final threshold in iteration 27: 0.38215299013236487 (> -Infinity) after 3696.422s (< 3600.000s)
    

Returns:

    0.38215299013236487