Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Code from BasicOptimizer.scala:88 executed in 1913.85 seconds (9.625 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result

Logging:

    Reset training subject: 3030656383230
    Reset training subject: 3058689366624
    Adding measurement 2673ce79 to history. Total: 0
    LBFGS Accumulation History: 1 points
    Constructing line search parameters: GD+Trust
    th(0)=2.2836386218349425;dx=-1.4756934463273207E-7
    New Minimum: 2.2836386218349425 > 2.283638581578166
    WOLFE (weak): th(2.154434690031884)=2.283638581578166; dx=-1.4654904740433725E-7 evalInputDelta=4.0256776578928566E-8
    Armijo: th(4.308869380063768)=2.2836386234721657; dx=-1.4645992511492294E-7 evalInputDelta=-1.637223245865016E-9
    Armijo: th(3.2316520350478255)=2.2836386620917186; dx=-1.4646938841799622E-7 evalInputDelta=-4.0256776134839356E-8
    Armijo: th(2.6930433625398544)=2.2836386620917186; dx=-1.464688588979692E-7 evalInputDelta=-4.0256776134839356E-8
    WOLFE (weak): th(2.423739026285869)=2.2836386189822937; dx=-1.4665378233079904E-7 evalInputDelta=2.852648783857603E-9
    Armijo: th(2.5583911944128617)=2.283638648784616; dx=-1.4655065109384562E-7 evalInputDelta=-2.694967360383771E-8
    Armijo: th(2.491065110349365)=2.283638648784616; dx=-1.4648510010253122E-7 evalInputDelta=-2.694967360383771E-8
    WOLFE (weak): th(2.457402068317617)=2.2836386189822937; dx=-1.4667006103850727E-7 evalInputDelta=2.852648783857603E-9
    Armijo: th(2.4742335893334912)=2.283638648784616; dx=-1.4652526775760158E-7 evalInputDelta=-2.694967360383771E-8
    Armijo: th(2.4658178288255543)=2.283638648784616; dx=-1.4644740032431503E-7 evalInputDelta=-2.694967360383771E-8
    WOLFE (weak): th(2.461609948571586)=2.2836386189822937; dx=-1.464696198131799E-7 evalInputDelta=2.852648783857603E-9
    WOLFE (weak): th(2.46371388869857)=2.2836386189822937; dx=-1.4636521102101695E-7 evalInputDelta=2.852648783857603E-9
    Armijo: th(2.4647658587620622)=2.283638648784616; dx=-1.4666115311729613E-7 evalInputDelta=-2.694967360383771E-8
    Armijo: th(2.464239873730316)=2.283638648784616; dx=-1.4650296938182156E-7 evalInputDelta=-2.694967360383771E-8
    WOLFE (weak): th(2.4639768812144434)=2.2836386189822937; dx=-1.4654458878157375E-7 evalInputDelta=2.852648783857603E-9
    Armijo: th(2

...skipping 7864 bytes...

    9395; Orientation: 6.5619; Line Search: 58.6623
    Adding measurement 38c18964 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.592e+03, gradient 9.113e-05, dot -0.182; [f41d2960-b03b-4bc4-ba7e-98fe24d51f8e = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -9.440489570205921, -0.6477191575682495, 2.283638581578166, 2.2836386218349425
    LBFGS Accumulation History: 3 points
    Removed measurement 38c18964 to history. Total: 3
    th(0)=-9.440489570205921;dx=-8.284801026310204E-9
    Armijo: th(3.4586054077148473E8)=-6.728520123123286; dx=8.948745201365451E-9 evalInputDelta=-2.7119694470826348
    Armijo: th(1.7293027038574237E8)=-8.93229208535208; dx=8.35429198071889E-9 evalInputDelta=-0.5081974848538415
    New Minimum: -9.440489570205921 > -9.734009299396813
    END: th(5.764342346191412E7)=-9.734009299396813; dx=1.6439519559155327E-10 evalInputDelta=0.2935197291908924
    Fitness changed from -9.440489570205921 to -9.734009299396813
    Iteration 9 complete. Error: -9.734009299396813 Total: 149.0428; Orientation: 6.6072; Line Search: 115.0796
    Adding measurement ce324dc to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.619e+03, gradient 6.063e-05, dot -0.348; [f41d2960-b03b-4bc4-ba7e-98fe24d51f8e = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -9.734009299396813, -0.6477191575682495, 2.283638581578166, 2.2836386218349425
    LBFGS Accumulation History: 3 points
    Removed measurement ce324dc to history. Total: 3
    th(0)=-9.734009299396813;dx=-3.670165447020463E-9
    New Minimum: -9.734009299396813 > -9.838074330583378
    END: th(1.2418899115854557E8)=-9.838074330583378; dx=5.708810857583811E-10 evalInputDelta=0.10406503118656474
    Fitness changed from -9.734009299396813 to -9.838074330583378
    Iteration 10 complete. Error: -9.838074330583378 Total: 91.8628; Orientation: 6.5929; Line Search: 57.6044
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/4afe63eb-936e-43c4-bb66-c37432bcd553.jpg)
    
    Final threshold in iteration 11: -9.838074330583378 (> -Infinity) after 1913.850s (< 5400.000s)
    

Returns:

    -9.838074330583378