Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

BasicOptimizer.scala:88 executed in 134.00 seconds (2.944 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 1168690403605
Reset training subject: 1173186994302
Adding measurement 35d9229 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=30.109142376109958;dx=-1.185250053649868E-4
New Minimum: 30.109142376109958 > 30.108236907050014
WOLFE (weak): th(2.154434690031884)=30.108236907050014; dx=-1.1841232759728825E-4 evalInputDelta=9.054690599441528E-4
New Minimum: 30.108236907050014 > 30.107331505045295
WOLFE (weak): th(4.308869380063768)=30.107331505045295; dx=-1.2048297854106859E-4 evalInputDelta=0.0018108710646629333
New Minimum: 30.107331505045295 > 30.103701585903764
WOLFE (weak): th(12.926608140191302)=30.103701585903764; dx=-1.1303919964105674E-4 evalInputDelta=0.005440790206193924
New Minimum: 30.103701585903764 > 30.088263055309653
WOLFE (weak): th(51.70643256076521)=30.088263055309653; dx=-1.2288003182949682E-4 evalInputDelta=0.020879320800304413
New Minimum: 30.088263055309653 > 30.01368905417621
WOLFE (weak): th(258.53216280382605)=30.01368905417621; dx=-1.2091818638659044E-4 evalInputDelta=0.09545332193374634
New Minimum: 30.01368905417621 > 29.500001680105925
WOLFE (weak): th(1551.1929768229563)=29.500001680105925; dx=-1.1685367164146422E-4 evalInputDelta=0.6091406960040331
New Minimum: 29.500001680105925 > 25.946674779057503
END: th(10858.350837760694)=25.946674779057503; dx=-6.625710299903924E-5 evalInputDelta=4.162467597052455
Fitness changed from 30.109142376109958 to 25.946674779057503
Iteration 1 complete. Error: 25.946674779057503 Total: 36.3646; Orientation: 0.0389; Line Search: 25.3132
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/226d8979-8933-446a-abd5-6086b9a8046c.jpg)

    Adding measurement 63aa4738 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=25.946674779057503;dx=-7.503130623517352E-5
    New Minimum: 25.946674779057503 > 21.86539688706398
    END: th(23393.607721408407)=21.86539688706398; dx=-2.4224088050606063E-5 evalInputDelta=4.081277891993523
    Fitness changed from 25.946674779057503 to 21.86539688706398
    Iteration 2 complete. Error: 21.86539688706398 Total: 10.1631; Orientation: 0.0062; Line Search: 6.2243
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/c2e97b1b-c24d-42f6-9f7a-7fc126fda642.jpg)

    Adding measurement 42d001bc to history. Total: 2
    LBFGS Accumulation History: 3 points
    th(0)=21.86539688706398;dx=-1.9366944373431327E-5
    New Minimum: 21.86539688706398 > 19.703789092600346
    END: th(50400.00000000001)=19.703789092600346; dx=-1.0925482898606915E-5 evalInputDelta=2.1616077944636345
    Fitness changed from 21.86539688706398 to 19.703789092600346
    Iteration 3 complete. Error: 19.703789092600346 Total: 9.5108; Orientation: 0.0058; Line Search: 6.3668
    <a id="p-7"></a>Iteration 3
    <a id="p-6"></a>![Iteration 3](etc/52be156c-76d5-40f2-812b-a39eaa5f5c16.jpg)

    Adding measurement 3a29a15a to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.118e+02, gradient 3.499e-03, dot -0.782; [57e40a79-1afd-4e3c-b351-c6fbe463945a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 19.703789092600346, 21.86539688706398, 25.946674779057503, 30.109142376109958
    LBFGS Accumulation History: 3 points
    Removed measurement 3a29a15a to history. Total: 3
    th(0)=19.703789092600346;dx=-1.1104445715578116E-5
    New Minimum: 19.703789092600346 > 16.90454263985157
    END: th(108583.50837760697)=16.90454263985157; dx=-6.0171991277678105E-6 evalInputDelta=2.7992464527487755
    Fitness changed from 19.703789092600346 to 16.90454263985157
    Iteration 4 complete. Error: 16.90454263985157 Total: 9.7196; Orientation: 0.2684; Line Search: 6.2220
    <a id="p-9"></a>Iteration 4
    <a id="p-8"></a>![Iteration 4](etc/2e03dfc8-e487-45dc-9279-45f52ab5b97c.jpg)

    Adding measurement 4c35bec5 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.277e+02, gradient 2.686e-03, dot -0.699; [57e40a79-1afd-4e3c-b351-c6fbe463945a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 16.90454263985157, 21.86539688706398, 25.946674779057503, 30.109142376109958
    LBFGS Accumulation History: 3 points
    Removed measurement 4c35bec5 to history. Total: 3
    th(0)=16.90454263985157;dx=-6.698241687853834E-6
    New Minimum: 16.90454263985157 > 14.00636813044548
    END: th(233936.07721408413)=14.00636813044548; dx=-3.431972460507445E-6 evalInputDelta=2.89817450940609
    Fitness changed from 16.90454263985157 to 14.00636813044548
    Iteration 5 complete. Error: 14.00636813044548 Total: 10.2830; Orientation: 0.0697; Line Search: 7.0448
    <a id="p-11"></a>Iteration 5
    <a id="p-10"></a>![Iteration 5](etc/f60dcc49-3917-4f7a-94f2-bb06121577b4.jpg)

    Adding measurement 706b60c2 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.967e+02, gradient 2.314e-03, dot -0.600; [57e40a79-1afd-4e3c-b351-c6fbe463945a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 14.00636813044548, 21.86539688706398, 25.946674779057503, 30.109142376109958
    LBFGS Accumulation History: 3 points
    Removed measurement 706b60c2 to history. Total: 3
    th(0)=14.00636813044548;dx=-4.906395860297038E-6
    New Minimum: 14.00636813044548 > 10.524693816900253
    END: th(504000.0000000002)=10.524693816900253; dx=-1.7519245346106946E-6 evalInputDelta=3.481674313545227
    Fitness changed from 14.00636813044548 to 10.524693816900253
    Iteration 6 complete. Error: 10.524693816900253 Total: 9.5966; Orientation: 0.0656; Line Search: 6.3557
    Adding measurement 2e7ee553 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.601e+02, gradient 1.751e-03, dot -0.488; [57e40a79-1afd-4e3c-b351-c6fbe463945a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 10.524693816900253, 21.86539688706398, 25.946674779057503, 30.109142376109958
    LBFGS Accumulation History: 3 points
    Removed measurement 2e7ee553 to history. Total: 3
    th(0)=10.524693816900253;dx=-2.8716982697556617E-6
    New Minimum: 10.524693816900253 > 8.291742146015167
    END: th(1085835.0837760698)=8.291742146015167; dx=-7.70349609946341E-7 evalInputDelta=2.232951670885086
    Fitness changed from 10.524693816900253 to 8.291742146015167
    Iteration 7 complete. Error: 8.291742146015167 Total: 9.6829; Orientation: 0.1487; Line Search: 6.3554
    Adding measurement 2e748bc0 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.912e+02, gradient 2.167e-03, dot -0.267; [57e40a79-1afd-4e3c-b351-c6fbe463945a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 8.291742146015167, 21.86539688706398, 25.946674779057503, 30.109142376109958
    LBFGS Accumulation History: 3 points
    Removed measurement 2e748bc0 to history. Total: 3
    th(0)=8.291742146015167;dx=-4.318618686845753E-6
    New Minimum: 8.291742146015167 > 7.206093862652779
    WOLF (strong): th(2339360.7721408415)=7.206093862652779; dx=8.86764762019764E-8 evalInputDelta=1.0856482833623886
    New Minimum: 7.206093862652779 > 6.176423326134682
    END: th(1169680.3860704207)=6.176423326134682; dx=-1.9271595808815662E-7 evalInputDelta=2.1153188198804855
    Fitness changed from 8.291742146015167 to 6.176423326134682
    Iteration 8 complete. Error: 6.176423326134682 Total: 13.3783; Orientation: 0.0654; Line Search: 10.1439
    Adding measurement 540952ca to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.833e+02, gradient 1.069e-03, dot -0.204; [57e40a79-1afd-4e3c-b351-c6fbe463945a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 6.176423326134682, 21.86539688706398, 25.946674779057503, 30.109142376109958
    LBFGS Accumulation History: 3 points
    Removed measurement 540952ca to history. Total: 3
    th(0)=6.176423326134682;dx=-1.0384266281210636E-6
    New Minimum: 6.176423326134682 > 4.621336758136749
    END: th(2520000.000000001)=4.621336758136749; dx=-1.2937194148326448E-7 evalInputDelta=1.5550865679979324
    Fitness changed from 6.176423326134682 to 4.621336758136749
    Iteration 9 complete. Error: 4.621336758136749 Total: 9.5986; Orientation: 0.2637; Line Search: 6.2248
    Adding measurement 30ef5e0b to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.626e+02, gradient 9.658e-04, dot -0.175; [57e40a79-1afd-4e3c-b351-c6fbe463945a = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 4.621336758136749, 21.86539688706398, 25.946674779057503, 30.109142376109958
    LBFGS Accumulation History: 3 points
    Removed measurement 30ef5e0b to history. Total: 3
    th(0)=4.621336758136749;dx=-7.317570839427498E-7
    Armijo: th(5429175.418880349)=6.471770823001862; dx=9.408247925658331E-8 evalInputDelta=-1.8504340648651123
    Armijo: th(2714587.7094401745)=4.744180127978325; dx=-1.672021259812875E-8 evalInputDelta=-0.12284336984157562
    New Minimum: 4.621336758136749 > 3.9662994742393494
    END: th(904862.5698133915)=3.9662994742393494; dx=-1.2669673883309477E-7 evalInputDelta=0.6550372838973999
    Fitness changed from 4.621336758136749 to 3.9662994742393494
    Iteration 10 complete. Error: 3.9662994742393494 Total: 15.6632; Orientation: 0.1261; Line Search: 12.4264
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/16f417cf-9335-4f26-8a5e-ed2b02960e14.jpg)

    Final threshold in iteration 11: 3.9662994742393494 (> -Infinity) after 133.984s (< 5400.000s)

Returns

    3.9662994742393494