Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

Code from BasicOptimizer.scala:88 executed in 321.48 seconds (8.239 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 779459207902
    Reset training subject: 783046492664
    Adding measurement 617a4adf to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=12.021105624735355;dx=-6.382098679171443E-7
    New Minimum: 12.021105624735355 > 12.021104671061039
    WOLFE (weak): th(2.154434690031884)=12.021104671061039; dx=-6.328255509513413E-7 evalInputDelta=9.5367431640625E-7
    New Minimum: 12.021104671061039 > 12.02110342681408
    WOLFE (weak): th(4.308869380063768)=12.02110342681408; dx=-6.332805529429379E-7 evalInputDelta=2.1979212760925293E-6
    New Minimum: 12.02110342681408 > 12.021098732948303
    WOLFE (weak): th(12.926608140191302)=12.021098732948303; dx=-6.333035456467112E-7 evalInputDelta=6.891787052154541E-6
    New Minimum: 12.021098732948303 > 12.021078377962112
    WOLFE (weak): th(51.70643256076521)=12.021078377962112; dx=-6.334961747990198E-7 evalInputDelta=2.724677324295044E-5
    New Minimum: 12.021078377962112 > 12.020968809723854
    WOLFE (weak): th(258.53216280382605)=12.020968809723854; dx=-6.326574040234294E-7 evalInputDelta=1.3681501150131226E-4
    New Minimum: 12.020968809723854 > 12.020286709070206
    WOLFE (weak): th(1551.1929768229563)=12.020286709070206; dx=-6.329802000353754E-7 evalInputDelta=8.189156651496887E-4
    New Minimum: 12.020286709070206 > 12.015380784869194
    WOLFE (weak): th(10858.350837760694)=12.015380784869194; dx=-6.33314548991315E-7 evalInputDelta=0.0057248398661613464
    New Minimum: 12.015380784869194 > 11.97572736442089
    WOLFE (weak): th(86866.80670208555)=11.97572736442089; dx=-6.266421874616964E-7 evalInputDelta=0.04537826031446457
    New Minimum: 11.97572736442089 > 11.64126193523407
    END: th(781801.26031877)=11.64126193523407; dx=-5.632681006980963E-7 evalInputDelta=0.37984368950128555
    Fitness changed from 12.021105624735355 to 11.64126193523407
    Iteration 1 complete. Error: 11.64126193523407 Total: 49.4090; Orientation: 0.0087; Line Search: 37.1439
    <a id="p-3"></a>Iteration 1
    <a id="p-2"></a>![Iteration 1](etc/d46b9038-2f3c-4a78-a7a5-2c467c9c10b4.jpg)
    
    Adding 

...skipping 14603 bytes...

     Total: 10.7390; Orientation: 0.1612; Line Search: 7.2353
    Adding measurement 659d4e83 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 7.667e+03, gradient 2.447e-04, dot -0.116; [eac5b4b3-2a14-4f3f-8948-ef84b2c4969a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 2.26608407497406, 10.900565281510353, 11.64126193523407, 12.021105624735355
    LBFGS Accumulation History: 3 points
    Removed measurement 659d4e83 to history. Total: 3
    th(0)=2.26608407497406;dx=-5.910996020502325E-8
    Armijo: th(2.8276955306668498E7)=2.9970371425151825; dx=-1.504831583805777E-8 evalInputDelta=-0.7309530675411224
    New Minimum: 2.26608407497406 > 2.1870853900909424
    END: th(1.4138477653334249E7)=2.1870853900909424; dx=-2.99751079368993E-8 evalInputDelta=0.07899868488311768
    Fitness changed from 2.26608407497406 to 2.1870853900909424
    Iteration 19 complete. Error: 2.1870853900909424 Total: 14.8584; Orientation: 0.2533; Line Search: 10.9556
    Adding measurement 2700b31 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 8.271e+03, gradient 2.468e-04, dot -0.113; [eac5b4b3-2a14-4f3f-8948-ef84b2c4969a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 2.1870853900909424, 10.900565281510353, 11.64126193523407, 12.021105624735355
    LBFGS Accumulation History: 3 points
    Removed measurement 2700b31 to history. Total: 3
    th(0)=2.1870853900909424;dx=-6.02038484437193E-8
    Armijo: th(3.046042672058389E7)=3.19332954287529; dx=-9.587662664652574E-9 evalInputDelta=-1.0062441527843475
    New Minimum: 2.1870853900909424 > 2.1598561853170395
    END: th(1.5230213360291945E7)=2.1598561853170395; dx=-2.6093831892186524E-8 evalInputDelta=0.027229204773902893
    Fitness changed from 2.1870853900909424 to 2.1598561853170395
    Iteration 20 complete. Error: 2.1598561853170395 Total: 15.9954; Orientation: 0.1596; Line Search: 12.1104
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/4e96e8e7-084f-4b67-aed4-63ffe5bd6b2c.jpg)
    
    Final threshold in iteration 21: 2.1598561853170395 (> -Infinity) after 321.480s (< 3600.000s)
    

Returns:

    2.1598561853170395