Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:75 executed in 2683.83 seconds (28.016 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1502040640560900
    Reset training subject: 1502071654917100
    Adding measurement 2a79da3a to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=2.384229107916477;dx=-2.395249273904452E-6
    New Minimum: 2.384229107916477 > 2.3842288409965007
    WOLFE (weak): th(2.154434690031884)=2.3842288409965007; dx=-2.3947637422227306E-6 evalInputDelta=2.669199763438712E-7
    New Minimum: 2.3842288409965007 > 2.384227721696225
    WOLFE (weak): th(4.308869380063768)=2.384227721696225; dx=-2.3947714002637444E-6 evalInputDelta=1.386220251919923E-6
    New Minimum: 2.384227721696225 > 2.384223619154601
    WOLFE (weak): th(12.926608140191302)=2.384223619154601; dx=-2.394739590385006E-6 evalInputDelta=5.488761876115689E-6
    New Minimum: 2.384223619154601 > 2.3842061701310127
    WOLFE (weak): th(51.70643256076521)=2.3842061701310127; dx=-2.3949738399695423E-6 evalInputDelta=2.293778546436087E-5
    New Minimum: 2.3842061701310127 > 2.384114177872069
    WOLFE (weak): th(258.53216280382605)=2.384114177872069; dx=-2.3950515505752908E-6 evalInputDelta=1.149300444081014E-4
    New Minimum: 2.384114177872069 > 2.3835398338426477
    WOLFE (weak): th(1551.1929768229563)=2.3835398338426477; dx=-2.393089494228349E-6 evalInputDelta=6.89274073829349E-4
    New Minimum: 2.3835398338426477 > 2.3794216947862066
    WOLFE (weak): th(10858.350837760694)=2.3794216947862066; dx=-2.3747803246234763E-6 evalInputDelta=0.004807413130270444
    New Minimum: 2.3794216947862066 > 2.3466797783964686
    WOLFE (weak): th(86866.80670208555)=2.3466797783964686; dx=-2.246900682520834E-6 evalInputDelta=0.037549329520008445
    New Minimum: 2.3466797783964686 > 2.171274761416871
    END: th(781801.26031877)=2.171274761416871; dx=-5.323953568784267E-7 evalInputDelta=0.2129543464996062
    Fitness changed from 2.384229107916477 to 2.171274761416871
    Iteration 1 complete. Error: 2.171274761416871 Total: 388.5792; Orientation: 0.0779; Line Search: 297.0567
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/863a9784-78d6-4e

...skipping 15643 bytes...

    tude: 8.361e+03, gradient 3.496e-04, dot -0.948; [776bddba-8b34-46fa-b53a-64151a2187f5 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 1.4539081207994682, 1.4047076942435377, 1.3827672231660582, 1.3427513734338674
    LBFGS Accumulation History: 3 points
    Removed measurement 3ba815ee to history. Total: 4
    th(0)=1.3427513734338674;dx=-1.2223093970182863E-7
    New Minimum: 1.3427513734338674 > 1.2911897368386043
    END: th(4189178.563950888)=1.2911897368386043; dx=-2.5202564187744384E-8 evalInputDelta=0.05156163659526314
    Fitness changed from 1.3427513734338674 to 1.2911897368386043
    Iteration 19 complete. Error: 1.2911897368386043 Total: 94.6900; Orientation: 2.3298; Line Search: 61.8032
    Adding measurement 43cf97a8 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.104e+03, gradient 9.115e-04, dot -0.335; [776bddba-8b34-46fa-b53a-64151a2187f5 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 1.4047076942435377, 1.3827672231660582, 1.3427513734338674, 1.2911897368386043
    LBFGS Accumulation History: 3 points
    Removed measurement 769f6023 to history. Total: 4
    th(0)=1.2911897368386043;dx=-8.307864823610833E-7
    Armijo: th(9025311.620913744)=1.6591540373698364; dx=8.647726808572431E-8 evalInputDelta=-0.3679643005312321
    Armijo: th(4512655.810456872)=1.541326669431543; dx=2.2653822625724794E-7 evalInputDelta=-0.2501369325929388
    Armijo: th(1504218.603485624)=1.3376247960344776; dx=4.653398271842893E-7 evalInputDelta=-0.04643505919587332
    New Minimum: 1.2911897368386043 > 1.2649211851577802
    END: th(376054.650871406)=1.2649211851577802; dx=-4.657288338729641E-8 evalInputDelta=0.026268551680824093
    Fitness changed from 1.2911897368386043 to 1.2649211851577802
    Iteration 20 complete. Error: 1.2649211851577802 Total: 182.6468; Orientation: 2.2992; Line Search: 149.8647
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/11c27f5b-cc17-4671-b824-654f7538c176.jpg)
    
    Final threshold in iteration 21: 1.2649211851577802 (> -Infinity) after 2683.835s (< 5400.000s)
    

Returns:

    1.2649211851577802