Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Iteration 25 Iteration 25

Iteration 30 Iteration 30

Iteration 35 Iteration 35

Iteration 40 Iteration 40

Iteration 45 Iteration 45

Iteration 50 Iteration 50

Code from BasicOptimizer.scala:75 executed in 294.36 seconds (3.715 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    IterativeTrainer.wrap(trainable)
      .setOrientation(orientation())
      .setMonitor(new TrainingMonitor() {
        override def clear(): Unit = trainingMonitor.clear()
  
        override def log(msg: String): Unit = trainingMonitor.log(msg)
  
        override def onStepFail(currentPoint: Step): Boolean = {
          BasicOptimizer.this.onStepFail(trainable, currentPoint)
        }
  
        override def onStepComplete(currentPoint: Step): Unit = {
          if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
            val image = currentImage
            timelineAnimation += image
            val caption = "Iteration " + currentPoint.iteration
            out.p(caption + "\n" + out.jpg(image, caption))
          }
          BasicOptimizer.this.onStepComplete(trainable, currentPoint)
          trainingMonitor.onStepComplete(currentPoint)
          super.onStepComplete(currentPoint)
        }
      })
      .setTimeout(trainingMinutes, TimeUnit.MINUTES)
      .setMaxIterations(trainingIterations)
      .setLineSearchFactory((_: CharSequence) => lineSearchInstance)
      .setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
      .runAndFree
      .asInstanceOf[lang.Double]

Logging:

    Reset training subject: 1762590294014
    Reset training subject: 1763541837797
    Adding measurement 5b81bb04 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=0.9305623571834871;dx=-9.94513782647717E-9
    New Minimum: 0.9305623571834871 > 0.9305623398812042
    WOLFE (weak): th(2.154434690031884)=0.9305623398812042; dx=-9.945137068325808E-9 evalInputDelta=1.7302282939724023E-8
    New Minimum: 0.9305623398812042 > 0.930562322578923
    WOLFE (weak): th(4.308869380063768)=0.930562322578923; dx=-9.945136310174448E-9 evalInputDelta=3.4604564103091207E-8
    New Minimum: 0.930562322578923 > 0.9305622533698105
    WOLFE (weak): th(12.926608140191302)=0.9305622533698105; dx=-9.945131028042428E-9 evalInputDelta=1.0381367665512897E-7
    New Minimum: 0.9305622533698105 > 0.9305619419268306
    WOLFE (weak): th(51.70643256076521)=0.9305619419268306; dx=-9.945155076424137E-9 evalInputDelta=4.1525665650521404E-7
    New Minimum: 0.9305619419268306 > 0.9305602808190174
    WOLFE (weak): th(258.53216280382605)=0.9305602808190174; dx=-9.945595213498527E-9 evalInputDelta=2.0763644696941697E-6
    New Minimum: 0.9305602808190174 > 0.9305498971702386
    WOLFE (weak): th(1551.1929768229563)=0.9305498971702386; dx=-9.947297592442254E-9 evalInputDelta=1.246001324850532E-5
    New Minimum: 0.9305498971702386 > 0.9304750723934052
    WOLFE (weak): th(10858.350837760694)=0.9304750723934052; dx=-9.956467810985013E-9 evalInputDelta=8.728479008190959E-5
    New Minimum: 0.9304750723934052 > 0.9298591128921492
    WOLFE (weak): th(86866.80670208555)=0.9298591128921492; dx=-1.0048276524245235E-8 evalInputDelta=7.032442913379455E-4
    New Minimum: 0.9298591128921492 > 0.9238576365226459
    WOLFE (weak): th(781801.26031877)=0.9238576365226459; dx=-1.0719328878248866E-8 evalInputDelta=0.006704720660841268
    New Minimum: 0.9238576365226459 > 0.8502610342067672
    WOLFE (weak): th(7818012.6031877)=0.8502610342067672; dx=-1.092857606335615E-8 evalInputDelta=0.08030132297671988
    New Minimum: 0.8502610342067672 > 0.7541555366107379
    WOLF (strong): th(8.599813863506469

...skipping 43816 bytes...

    +03, gradient 2.166e-05, dot -0.822; [0ebad2b1-6e9d-489b-9eaa-5ea76a976592 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.3552111039173597, 0.35461922747671965, 0.3542191206590147, 0.3539523527689336
    LBFGS Accumulation History: 3 points
    Removed measurement 7aa2ca31 to history. Total: 4
    th(0)=0.3539523527689336;dx=-4.693039611684737E-10
    Armijo: th(3.853310188348365E7)=0.3583624885067259; dx=-1.9044736126985474E-10 evalInputDelta=-0.004410135737792309
    New Minimum: 0.3539523527689336 > 0.35383974059799567
    END: th(1.9266550941741824E7)=0.35383974059799567; dx=-2.86543584933459E-10 evalInputDelta=1.126121709379424E-4
    Fitness changed from 0.3539523527689336 to 0.35383974059799567
    Iteration 49 complete. Error: 0.35383974059799567 Total: 5.6084; Orientation: 1.6156; Line Search: 3.1180
    Adding measurement 5f95c7c0 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.631e+03, gradient 2.231e-05, dot -0.790; [0ebad2b1-6e9d-489b-9eaa-5ea76a976592 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.35461922747671965, 0.3542191206590147, 0.3539523527689336, 0.35383974059799567
    LBFGS Accumulation History: 3 points
    Removed measurement 1cd07175 to history. Total: 4
    th(0)=0.35383974059799567;dx=-4.977589217104601E-10
    Armijo: th(4.150852570615505E7)=0.3599316635567249; dx=-1.5946347019432012E-10 evalInputDelta=-0.0060919229587292345
    Armijo: th(2.0754262853077523E7)=0.3539657094999698; dx=-2.662865577625684E-10 evalInputDelta=-1.2596890197413835E-4
    New Minimum: 0.35383974059799567 > 0.35297414014460077
    END: th(6918087.617692508)=0.35297414014460077; dx=-4.093615843175677E-10 evalInputDelta=8.656004533948991E-4
    Fitness changed from 0.35383974059799567 to 0.35297414014460077
    Iteration 50 complete. Error: 0.35297414014460077 Total: 6.6679; Orientation: 1.6179; Line Search: 4.1718
    <a id="p-29"></a>Iteration 50
    <a id="p-28"></a>![Iteration 50](etc/f62acb59-7b6a-4e4d-b412-ced449ae56a5.jpg)
    
    Final threshold in iteration 51: 0.35297414014460077 (> -Infinity) after 294.365s (< 3600.000s)
    

Returns:

    0.35297414014460077