Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

BasicOptimizer.scala:88 executed in 129.36 seconds (2.132 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 7848298348930
Reset training subject: 7852301328529
Adding measurement 5a900e15 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=29.983749708160758;dx=-8.785648131329459E-5
New Minimum: 29.983749708160758 > 29.9832694362849
WOLFE (weak): th(2.154434690031884)=29.9832694362849; dx=-8.485453559319055E-5 evalInputDelta=4.802718758583069E-4
New Minimum: 29.9832694362849 > 29.9827863201499
WOLFE (weak): th(4.308869380063768)=29.9827863201499; dx=-8.709885423050806E-5 evalInputDelta=9.633880108594894E-4
New Minimum: 29.9827863201499 > 29.980860196053982
WOLFE (weak): th(12.926608140191302)=29.980860196053982; dx=-8.62039327431386E-5 evalInputDelta=0.0028895121067762375
New Minimum: 29.980860196053982 > 29.972196089103818
WOLFE (weak): th(51.70643256076521)=29.972196089103818; dx=-8.613760479727477E-5 evalInputDelta=0.011553619056940079
New Minimum: 29.972196089103818 > 29.92608840763569
WOLFE (weak): th(258.53216280382605)=29.92608840763569; dx=-8.575936272220443E-5 evalInputDelta=0.05766130052506924
New Minimum: 29.92608840763569 > 29.641542330384254
WOLFE (weak): th(1551.1929768229563)=29.641542330384254; dx=-8.009477100285329E-5 evalInputDelta=0.34220737777650356
New Minimum: 29.641542330384254 > 27.446797143667936
END: th(10858.350837760694)=27.446797143667936; dx=-6.606074662548228E-5 evalInputDelta=2.5369525644928217
Fitness changed from 29.983749708160758 to 27.446797143667936
Iteration 1 complete. Error: 27.446797143667936 Total: 35.4505; Orientation: 0.0027; Line Search: 25.1025
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/2c8b53d3-b9d1-461a-8e5a-f425ada00dbd.jpg)

    Adding measurement 117b8a9b to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=27.446797143667936;dx=-7.4663683308196E-5
    New Minimum: 27.446797143667936 > 23.426844656467438
    END: th(23393.607721408407)=23.426844656467438; dx=-3.293714533489546E-5 evalInputDelta=4.019952487200499
    Fitness changed from 27.446797143667936 to 23.426844656467438
    Iteration 2 complete. Error: 23.426844656467438 Total: 9.4841; Orientation: 0.0025; Line Search: 6.3466
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/421f2f9f-6b55-4e22-99a7-84cee6734deb.jpg)

    Adding measurement 2c50873d to history. Total: 2
    LBFGS Accumulation History: 3 points
    th(0)=23.426844656467438;dx=-3.937736972996671E-5
    New Minimum: 23.426844656467438 > 20.494123619049788
    END: th(50400.00000000001)=20.494123619049788; dx=-1.5331589910969654E-5 evalInputDelta=2.9327210374176502
    Fitness changed from 23.426844656467438 to 20.494123619049788
    Iteration 3 complete. Error: 20.494123619049788 Total: 9.4965; Orientation: 0.0030; Line Search: 6.2985
    <a id="p-7"></a>Iteration 3
    <a id="p-6"></a>![Iteration 3](etc/c3559a32-a4e1-4bd6-ba4b-a32697cbf9e1.jpg)

    Adding measurement 18559636 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 7.355e+02, gradient 4.435e-03, dot -0.533; [1c91549c-30c8-4126-a5ef-65963c9e8d77 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 20.494123619049788, 23.426844656467438, 27.446797143667936, 29.983749708160758
    LBFGS Accumulation History: 3 points
    Removed measurement 18559636 to history. Total: 3
    th(0)=20.494123619049788;dx=-1.8132417293641697E-5
    New Minimum: 20.494123619049788 > 17.972991913557053
    END: th(108583.50837760697)=17.972991913557053; dx=-7.128397175745621E-6 evalInputDelta=2.521131705492735
    Fitness changed from 20.494123619049788 to 17.972991913557053
    Iteration 4 complete. Error: 17.972991913557053 Total: 9.5687; Orientation: 0.0615; Line Search: 6.3333
    <a id="p-9"></a>Iteration 4
    <a id="p-8"></a>![Iteration 4](etc/124b9861-97e6-45ea-8a79-1ff62a3a45cf.jpg)

    Adding measurement ccd3e8a to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.855e+02, gradient 3.566e-03, dot -0.473; [1c91549c-30c8-4126-a5ef-65963c9e8d77 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 17.972991913557053, 23.426844656467438, 27.446797143667936, 29.983749708160758
    LBFGS Accumulation History: 3 points
    Removed measurement ccd3e8a to history. Total: 3
    th(0)=17.972991913557053;dx=-1.192578576729564E-5
    New Minimum: 17.972991913557053 > 15.273651689291
    END: th(233936.07721408413)=15.273651689291; dx=-3.909127963769534E-6 evalInputDelta=2.6993402242660522
    Fitness changed from 17.972991913557053 to 15.273651689291
    Iteration 5 complete. Error: 15.273651689291 Total: 9.8863; Orientation: 0.0757; Line Search: 6.4713
    <a id="p-11"></a>Iteration 5
    <a id="p-10"></a>![Iteration 5](etc/a8f0a322-3c37-4d18-948a-12ed071fd502.jpg)

    Adding measurement 4add47eb to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 9.067e+02, gradient 2.961e-03, dot -0.474; [1c91549c-30c8-4126-a5ef-65963c9e8d77 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 15.273651689291, 23.426844656467438, 27.446797143667936, 29.983749708160758
    LBFGS Accumulation History: 3 points
    Removed measurement 4add47eb to history. Total: 3
    th(0)=15.273651689291;dx=-8.088722038179916E-6
    New Minimum: 15.273651689291 > 11.184514731168747
    END: th(504000.0000000002)=11.184514731168747; dx=-2.0292315284684478E-6 evalInputDelta=4.089136958122253
    Fitness changed from 15.273651689291 to 11.184514731168747
    Iteration 6 complete. Error: 11.184514731168747 Total: 9.5722; Orientation: 0.0652; Line Search: 6.3553
    Adding measurement 3a57d785 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 9.168e+02, gradient 1.709e-03, dot -0.510; [1c91549c-30c8-4126-a5ef-65963c9e8d77 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 11.184514731168747, 23.426844656467438, 27.446797143667936, 29.983749708160758
    LBFGS Accumulation History: 3 points
    Removed measurement 3a57d785 to history. Total: 3
    th(0)=11.184514731168747;dx=-2.629180791923229E-6
    New Minimum: 11.184514731168747 > 8.027053028345108
    END: th(1085835.0837760698)=8.027053028345108; dx=-7.019567554310889E-7 evalInputDelta=3.157461702823639
    Fitness changed from 11.184514731168747 to 8.027053028345108
    Iteration 7 complete. Error: 8.027053028345108 Total: 10.9695; Orientation: 0.1647; Line Search: 7.6453
    Adding measurement 4d38be3 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 7.694e+02, gradient 1.335e-03, dot -0.394; [1c91549c-30c8-4126-a5ef-65963c9e8d77 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 8.027053028345108, 23.426844656467438, 27.446797143667936, 29.983749708160758
    LBFGS Accumulation History: 3 points
    Removed measurement 4d38be3 to history. Total: 3
    th(0)=8.027053028345108;dx=-1.5176462860709069E-6
    New Minimum: 8.027053028345108 > 5.295333743095398
    END: th(2339360.7721408415)=5.295333743095398; dx=-1.880205427088188E-7 evalInputDelta=2.73171928524971
    Fitness changed from 8.027053028345108 to 5.295333743095398
    Iteration 8 complete. Error: 5.295333743095398 Total: 9.6142; Orientation: 0.2887; Line Search: 6.1943
    Adding measurement 227b6f3 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.845e+02, gradient 9.350e-04, dot -0.244; [1c91549c-30c8-4126-a5ef-65963c9e8d77 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 5.295333743095398, 23.426844656467438, 27.446797143667936, 29.983749708160758
    LBFGS Accumulation History: 3 points
    Removed measurement 227b6f3 to history. Total: 3
    th(0)=5.295333743095398;dx=-8.056634803425567E-7
    Armijo: th(5040000.000000002)=6.747839376330376; dx=2.560691875039564E-8 evalInputDelta=-1.4525056332349777
    New Minimum: 5.295333743095398 > 4.974128872156143
    END: th(2520000.000000001)=4.974128872156143; dx=-1.1208336433302607E-7 evalInputDelta=0.32120487093925476
    Fitness changed from 5.295333743095398 to 4.974128872156143
    Iteration 9 complete. Error: 4.974128872156143 Total: 12.6245; Orientation: 0.2162; Line Search: 9.3351
    Adding measurement 240a13ec to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.317e+02, gradient 8.452e-04, dot -0.240; [1c91549c-30c8-4126-a5ef-65963c9e8d77 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 4.974128872156143, 23.426844656467438, 27.446797143667936, 29.983749708160758
    LBFGS Accumulation History: 3 points
    Removed measurement 240a13ec to history. Total: 3
    th(0)=4.974128872156143;dx=-6.61552369915303E-7
    Armijo: th(5429175.418880349)=5.369165733456612; dx=7.173542242588223E-8 evalInputDelta=-0.39503686130046844
    New Minimum: 4.974128872156143 > 4.206704214215279
    END: th(2714587.7094401745)=4.206704214215279; dx=-7.342946618425932E-8 evalInputDelta=0.7674246579408646
    Fitness changed from 4.974128872156143 to 4.206704214215279
    Iteration 10 complete. Error: 4.206704214215279 Total: 12.6741; Orientation: 0.0827; Line Search: 9.4268
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/e14a8d67-9ffc-4540-907d-1fcbb3d025eb.jpg)

    Final threshold in iteration 11: 4.206704214215279 (> -Infinity) after 129.361s (< 5400.000s)

Returns

    4.206704214215279