Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

BasicOptimizer.scala:89 executed in 8.31 seconds (0.378 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 446494503322500
Reset training subject: 446494871008000
Adding measurement 1c5d3a37 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.0066951265005224;dx=-1.0978449298006212E-6
New Minimum: -1.0066951265005224 > -1.006701787683383
WOLFE (weak): th(2.154434690031884)=-1.006701787683383; dx=-1.0837290611816519E-6 evalInputDelta=6.66118286063444E-6
New Minimum: -1.006701787683383 > -1.006708448822168
WOLFE (weak): th(4.308869380063768)=-1.006708448822168; dx=-1.097411900466488E-6 evalInputDelta=1.3322321645636848E-5
New Minimum: -1.006708448822168 > -1.0067348780674994
WOLFE (weak): th(12.926608140191302)=-1.0067348780674994; dx=-1.0841137454009178E-6 evalInputDelta=3.9751566976997665E-5
New Minimum: -1.0067348780674994 > -1.0068543075035292
WOLFE (weak): th(51.70643256076521)=-1.0068543075035292; dx=-1.0953457385806384E-6 evalInputDelta=1.5918100300682525E-4
New Minimum: -1.0068543075035292 > -1.0074904426154623
WOLFE (weak): th(258.53216280382605)=-1.0074904426154623; dx=-1.0942743583527553E-6 evalInputDelta=7.953161149398813E-4
New Minimum: -1.0074904426154623 > -1.0114797929406498
WOLFE (weak): th(1551.1929768229563)=-1.0114797929406498; dx=-1.0854534245655318E-6 evalInputDelta=0.0047846664401274275
New Minimum: -1.0114797929406498 > -1.0403218508345702
WOLFE (weak): th(10858.350837760694)=-1.0403218508345702; dx=-1.1068962900666763E-6 evalInputDelta=0.033626724334047786
New Minimum: -1.0403218508345702 > -1.2751147217084362
WOLFE (weak): th(86866.80670208555)=-1.2751147217084362; dx=-1.0714615041631588E-6 evalInputDelta=0.2684195952079138
New Minimum: -1.2751147217084362 > -2.1587367516634446
END: th(781801.26031877)=-2.1587367516634446; dx=-1.7824862945340263E-7 evalInputDelta=1.1520416251629222
Fitness changed from -1.0066951265005224 to -2.1587367516634446
Iteration 1 complete. Error: -2.1587367516634446 Total: 2.5067; Orientation: 0.0308; Line Search: 1.6204
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/5cf3d0c2-f1df-4983-b452-cf4353cc0eb1.jpg)
Adding measurement 4cadd4d4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.1587367516634446;dx=-1.1165075980053495E-6
New Minimum: -2.1587367516634446 > -3.079386675727713
WOLF (strong): th(1684339.7559414052)=-3.079386675727713; dx=1.3603273079162856E-8 evalInputDelta=0.9206499240642683
New Minimum: -3.079386675727713 > -3.146929886474298
END: th(842169.8779707026)=-3.146929886474298; dx=-6.214820651105181E-8 evalInputDelta=0.9881931348108535
Fitness changed from -2.1587367516634446 to -3.146929886474298
Iteration 2 complete. Error: -3.146929886474298 Total: 0.6098; Orientation: 0.0034; Line Search: 0.4434
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>![Iteration 2](etc/8f29674f-3ac7-4cf7-ac82-268e522dd2bb.jpg)
Adding measurement 3e71a1f8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.146929886474298;dx=-1.1317725480209148E-6
New Minimum: -3.146929886474298 > -3.675333267845406
WOLF (strong): th(1814400.0)=-3.675333267845406; dx=1.0353758358931873E-7 evalInputDelta=0.528403381371108
New Minimum: -3.675333267845406 > -4.028796412581741
WOLF (strong): th(907200.0)=-4.028796412581741; dx=9.498351745001933E-8 evalInputDelta=0.881866526107443
END: th(302400.0)=-3.8135541699965634; dx=-6.366646887710687E-7 evalInputDelta=0.6666242835222653
Fitness changed from -3.146929886474298 to -4.028796412581741
Iteration 3 complete. Error: -4.028796412581741 Total: 0.7712; Orientation: 0.0022; Line Search: 0.6017
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>![Iteration 3](etc/b6c1e749-588f-4f20-b18a-14a7c7e846cc.jpg)
Adding measurement 75ef1392 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.864e+03, gradient 1.130e-03, dot -0.501; [9da66dab-04af-415b-bd3b-acd2a09dd95b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.028796412581741, -3.146929886474298, -2.1587367516634446, -1.0066951265005224
LBFGS Accumulation History: 3 points
Removed measurement 75ef1392 to history. Total: 3
th(0)=-4.028796412581741;dx=-1.2710353938484584E-6
New Minimum: -4.028796412581741 > -5.059616157487303
WOLF (strong): th(651501.0502656417)=-5.059616157487303; dx=1.2134309886965926E-9 evalInputDelta=1.0308197449055623
END: th(325750.5251328208)=-4.819492252961399; dx=-6.599172170268978E-7 evalInputDelta=0.7906958403796578
Fitness changed from -4.028796412581741 to -5.059616157487303
Iteration 4 complete. Error: -5.059616157487303 Total: 0.6614; Orientation: 0.0816; Line Search: 0.4223
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>![Iteration 4](etc/bc808fe1-2c4b-4f42-aef5-054f0de79b3f.jpg)
Adding measurement 4587f0f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.372e+03, gradient 1.027e-03, dot -0.546; [9da66dab-04af-415b-bd3b-acd2a09dd95b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.059616157487303, -3.146929886474298, -2.1587367516634446, -1.0066951265005224
LBFGS Accumulation History: 3 points
Removed measurement 4587f0f9 to history. Total: 3
th(0)=-5.059616157487303;dx=-1.0479352364815303E-6
New Minimum: -5.059616157487303 > -5.7017036058874835
WOLF (strong): th(701808.2316422522)=-5.7017036058874835; dx=6.28658650618854E-8 evalInputDelta=0.6420874484001802
END: th(350904.1158211261)=-5.639079761994886; dx=-3.3717170072885096E-7 evalInputDelta=0.5794636045075823
Fitness changed from -5.059616157487303 to -5.7017036058874835
Iteration 5 complete. Error: -5.7017036058874835 Total: 0.6135; Orientation: 0.0472; Line Search: 0.4164
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>![Iteration 5](etc/3322bb14-1aed-4dd9-b6f8-b031c91b42e3.jpg)
Adding measurement 73502d5e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.894e+03, gradient 1.079e-03, dot -0.508; [9da66dab-04af-415b-bd3b-acd2a09dd95b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.7017036058874835, -3.146929886474298, -2.1587367516634446, -1.0066951265005224
LBFGS Accumulation History: 3 points
Removed measurement 73502d5e to history. Total: 3
th(0)=-5.7017036058874835;dx=-1.157849295146606E-6
New Minimum: -5.7017036058874835 > -6.431739860081312
WOLF (strong): th(756000.0)=-6.431739860081312; dx=2.0641206353187472E-7 evalInputDelta=0.7300362541938288
END: th(378000.0)=-6.42709003702442; dx=-3.476070571873938E-7 evalInputDelta=0.7253864311369362
Fitness changed from -5.7017036058874835 to -6.431739860081312
Iteration 6 complete. Error: -6.431739860081312 Total: 0.6135; Orientation: 0.0443; Line Search: 0.4197
Adding measurement 549b26d7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.683e+04, gradient 1.131e-03, dot -0.487; [9da66dab-04af-415b-bd3b-acd2a09dd95b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.431739860081312, -3.146929886474298, -2.1587367516634446, -1.0066951265005224
LBFGS Accumulation History: 3 points
Removed measurement 549b26d7 to history. Total: 3
th(0)=-6.431739860081312;dx=-1.2743493680315293E-6
New Minimum: -6.431739860081312 > -7.106911322579351
WOLF (strong): th(814376.3128320521)=-7.106911322579351; dx=3.265197980060566E-7 evalInputDelta=0.6751714624980387
New Minimum: -7.106911322579351 > -7.2279892589804495
END: th(407188.15641602606)=-7.2279892589804495; dx=-2.863070751116718E-7 evalInputDelta=0.7962493988991373
Fitness changed from -6.431739860081312 to -7.2279892589804495
Iteration 7 complete. Error: -7.2279892589804495 Total: 0.9262; Orientation: 0.0415; Line Search: 0.4156
Adding measurement 174da9a0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.824e+04, gradient 8.016e-04, dot -0.674; [9da66dab-04af-415b-bd3b-acd2a09dd95b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.2279892589804495, -3.146929886474298, -2.1587367516634446, -1.0066951265005224
LBFGS Accumulation History: 3 points
Removed measurement 174da9a0 to history. Total: 3
th(0)=-7.2279892589804495;dx=-6.344569429333751E-7
New Minimum: -7.2279892589804495 > -7.916146731587388
END: th(877260.2895528154)=-7.916146731587388; dx=-1.3636470843485773E-7 evalInputDelta=0.6881574726069388
Fitness changed from -7.2279892589804495 to -7.916146731587388
Iteration 8 complete. Error: -7.916146731587388 Total: 0.4347; Orientation: 0.0405; Line Search: 0.2588
Adding measurement 7e9b68c5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.465e+04, gradient 1.013e-03, dot -0.535; [9da66dab-04af-415b-bd3b-acd2a09dd95b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.916146731587388, -3.146929886474298, -2.1587367516634446, -1.0066951265005224
LBFGS Accumulation History: 3 points
Removed measurement 7e9b68c5 to history. Total: 3
th(0)=-7.916146731587388;dx=-1.0160391441737116E-6
Armijo: th(1890000.0000000005)=-7.898108668623769; dx=1.8936048775137618E-7 evalInputDelta=-0.018038062963618984
New Minimum: -7.916146731587388 > -8.416694681761033
WOLF (strong): th(945000.0000000002)=-8.416694681761033; dx=2.0274044783541595E-7 evalInputDelta=0.5005479501736447
New Minimum: -8.416694681761033 > -8.477937261270267
END: th(315000.00000000006)=-8.477937261270267; dx=-4.0138486658813286E-7 evalInputDelta=0.5617905296828782
Fitness changed from -7.916146731587388 to -8.477937261270267
Iteration 9 complete. Error: -8.477937261270267 Total: 0.7061; Orientation: 0.0396; Line Search: 0.5377
Adding measurement 5ffae42f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.995e+04, gradient 7.533e-04, dot -0.710; [9da66dab-04af-415b-bd3b-acd2a09dd95b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.477937261270267, -3.146929886474298, -2.1587367516634446, -1.0066951265005224
LBFGS Accumulation History: 3 points
Removed measurement 5ffae42f to history. Total: 3
th(0)=-8.477937261270267;dx=-5.618763072865669E-7
New Minimum: -8.477937261270267 > -9.047106166676803
END: th(678646.9273600435)=-9.047106166676803; dx=-1.9883185088277589E-7 evalInputDelta=0.5691689054065368
Fitness changed from -8.477937261270267 to -9.047106166676803
Iteration 10 complete. Error: -9.047106166676803 Total: 0.4400; Orientation: 0.0421; Line Search: 0.2718
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>![Iteration 10](etc/26775cee-b942-4f52-8e2c-0e7a7a65287a.jpg)
Final threshold in iteration 11: -9.047106166676803 (> -Infinity) after 8.298s (< 5400.000s)

Returns

    -9.047106166676803