Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

BasicOptimizer.scala:88 executed in 826.23 seconds (2.832 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 8062645631760
Reset training subject: 8080319967852
Adding measurement 140a5c28 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=6.317411541938782;dx=-4.3466506819417443E-7
New Minimum: 6.317411541938782 > 6.317407876253128
WOLFE (weak): th(2.154434690031884)=6.317407876253128; dx=-4.372131254502337E-7 evalInputDelta=3.6656856536865234E-6
New Minimum: 6.317407876253128 > 6.317403584718704
WOLFE (weak): th(4.308869380063768)=6.317403584718704; dx=-4.318476338968676E-7 evalInputDelta=7.957220077514648E-6
New Minimum: 6.317403584718704 > 6.317387491464615
WOLFE (weak): th(12.926608140191302)=6.317387491464615; dx=-4.3859181104276364E-7 evalInputDelta=2.4050474166870117E-5
New Minimum: 6.317387491464615 > 6.317316234111786
WOLFE (weak): th(51.70643256076521)=6.317316234111786; dx=-4.359004891307276E-7 evalInputDelta=9.530782699584961E-5
New Minimum: 6.317316234111786 > 6.316935315728188
WOLFE (weak): th(258.53216280382605)=6.316935315728188; dx=-4.362610288569508E-7 evalInputDelta=4.7622621059417725E-4
New Minimum: 6.316935315728188 > 6.314549654722214
WOLFE (weak): th(1551.1929768229563)=6.314549654722214; dx=-4.335563705547709E-7 evalInputDelta=0.002861887216567993
New Minimum: 6.314549654722214 > 6.294028356671333
WOLFE (weak): th(10858.350837760694)=6.294028356671333; dx=-4.31253869640586E-7 evalInputDelta=0.023383185267448425
New Minimum: 6.294028356671333 > 6.159163609147072
WOLFE (weak): th(86866.80670208555)=6.159163609147072; dx=-4.1333941486251836E-7 evalInputDelta=0.1582479327917099
New Minimum: 6.159163609147072 > 5.154031157493591
END: th(781801.26031877)=5.154031157493591; dx=-2.4606007725912197E-7 evalInputDelta=1.1633803844451904
Fitness changed from 6.317411541938782 to 5.154031157493591
Iteration 1 complete. Error: 5.154031157493591 Total: 240.7351; Orientation: 0.0103; Line Search: 185.8381
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/89168e89-146b-45a2-9098-cb029917303f.jpg)

    Adding measurement 544c902f to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=5.154031157493591;dx=-2.7226201816191447E-7
    New Minimum: 5.154031157493591 > 3.7994706630706787
    END: th(1684339.7559414052)=3.7994706630706787; dx=-1.514995747785048E-7 evalInputDelta=1.3545604944229126
    Fitness changed from 5.154031157493591 to 3.7994706630706787
    Iteration 2 complete. Error: 3.7994706630706787 Total: 55.1786; Orientation: 0.0110; Line Search: 36.3682
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/847cbe7c-9946-46b8-9e6b-0d25de0530da.jpg)

    Adding measurement 12f0171b to history. Total: 2
    LBFGS Accumulation History: 3 points
    th(0)=3.7994706630706787;dx=-1.8772752660568674E-7
    New Minimum: 3.7994706630706787 > 2.3209671676158905
    END: th(3628800.0)=2.3209671676158905; dx=-9.586783164540146E-8 evalInputDelta=1.4785034954547882
    Fitness changed from 3.7994706630706787 to 2.3209671676158905
    Iteration 3 complete. Error: 2.3209671676158905 Total: 55.7050; Orientation: 0.0107; Line Search: 37.6013
    <a id="p-7"></a>Iteration 3
    <a id="p-6"></a>![Iteration 3](etc/6771c809-6c85-42c5-acf1-6f91ab78fb67.jpg)

    Adding measurement 33769df2 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.690e+03, gradient 3.728e-04, dot -0.656; [5429384f-2f79-472f-bb4c-e605e234ece1 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 2.3209671676158905, 3.7994706630706787, 5.154031157493591, 6.317411541938782
    LBFGS Accumulation History: 3 points
    Removed measurement 33769df2 to history. Total: 3
    th(0)=2.3209671676158905;dx=-1.3442782251247654E-7
    New Minimum: 2.3209671676158905 > 0.7640352249145508
    END: th(7818012.6031877)=0.7640352249145508; dx=-4.154324196026552E-8 evalInputDelta=1.5569319427013397
    Fitness changed from 2.3209671676158905 to 0.7640352249145508
    Iteration 4 complete. Error: 0.7640352249145508 Total: 54.9157; Orientation: 0.2835; Line Search: 36.5127
    <a id="p-9"></a>Iteration 4
    <a id="p-8"></a>![Iteration 4](etc/c853f32c-e348-4265-ba26-bf9b74d655a8.jpg)

    Adding measurement 21df324b to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.723e+03, gradient 3.628e-04, dot -0.472; [5429384f-2f79-472f-bb4c-e605e234ece1 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.7640352249145508, 3.7994706630706787, 5.154031157493591, 6.317411541938782
    LBFGS Accumulation History: 3 points
    Removed measurement 21df324b to history. Total: 3
    th(0)=0.7640352249145508;dx=-1.2808083264251601E-7
    New Minimum: 0.7640352249145508 > 0.1704356074333191
    END: th(1.684339755941405E7)=0.1704356074333191; dx=-1.710905513147573E-10 evalInputDelta=0.5935996174812317
    Fitness changed from 0.7640352249145508 to 0.1704356074333191
    Iteration 5 complete. Error: 0.1704356074333191 Total: 55.5851; Orientation: 0.2824; Line Search: 36.4399
    <a id="p-11"></a>Iteration 5
    <a id="p-10"></a>![Iteration 5](etc/ed510bf3-8830-4588-84cc-39b19bae7246.jpg)

    Adding measurement 8369d4b to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.905e+03, gradient 3.794e-04, dot -0.245; [5429384f-2f79-472f-bb4c-e605e234ece1 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 0.1704356074333191, 3.7994706630706787, 5.154031157493591, 6.317411541938782
    LBFGS Accumulation History: 3 points
    Removed measurement 8369d4b to history. Total: 3
    th(0)=0.1704356074333191;dx=-1.3688588580082635E-7
    Armijo: th(3.6288E7)=1.5023167133331299; dx=2.5239511178723925E-8 evalInputDelta=-1.3318811058998108
    New Minimum: 0.1704356074333191 > -0.34619686007499695
    WOLF (strong): th(1.8144E7)=-0.34619686007499695; dx=1.1525103463400968E-8 evalInputDelta=0.516632467508316
    New Minimum: -0.34619686007499695 > -0.8617061525583267
    END: th(6048000.0)=-0.8617061525583267; dx=-6.9190228213519234E-9 evalInputDelta=1.0321417599916458
    Fitness changed from 0.1704356074333191 to -0.8617061525583267
    Iteration 6 complete. Error: -0.8617061525583267 Total: 90.8444; Orientation: 0.3851; Line Search: 72.7769
    Adding measurement 4b6b25fc to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.129e+03, gradient 2.535e-04, dot -0.261; [5429384f-2f79-472f-bb4c-e605e234ece1 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -0.8617061525583267, 3.7994706630706787, 5.154031157493591, 6.317411541938782
    LBFGS Accumulation History: 3 points
    Removed measurement 4b6b25fc to history. Total: 3
    th(0)=-0.8617061525583267;dx=-6.283971065737805E-8
    New Minimum: -0.8617061525583267 > -1.2222920283675194
    WOLF (strong): th(1.3030021005312834E7)=-1.2222920283675194; dx=4.870208318960978E-9 evalInputDelta=0.36058587580919266
    New Minimum: -1.2222920283675194 > -1.4586405232548714
    END: th(6515010.502656417)=-1.4586405232548714; dx=-8.476361597518134E-9 evalInputDelta=0.5969343706965446
    Fitness changed from -0.8617061525583267 to -1.4586405232548714
    Iteration 7 complete. Error: -1.4586405232548714 Total: 72.8838; Orientation: 0.2814; Line Search: 54.7937
    Adding measurement 790dc3d to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.254e+03, gradient 2.206e-04, dot -0.280; [5429384f-2f79-472f-bb4c-e605e234ece1 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -1.4586405232548714, 3.7994706630706787, 5.154031157493591, 6.317411541938782
    LBFGS Accumulation History: 3 points
    Removed measurement 790dc3d to history. Total: 3
    th(0)=-1.4586405232548714;dx=-4.746235652885963E-8
    New Minimum: -1.4586405232548714 > -1.746029607951641
    END: th(1.4036164632845046E7)=-1.746029607951641; dx=-4.063965627029119E-9 evalInputDelta=0.2873890846967697
    Fitness changed from -1.4586405232548714 to -1.746029607951641
    Iteration 8 complete. Error: -1.746029607951641 Total: 54.1770; Orientation: 0.2758; Line Search: 36.3292
    Adding measurement 96817ae to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.379e+03, gradient 2.178e-04, dot -0.256; [5429384f-2f79-472f-bb4c-e605e234ece1 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -1.746029607951641, 3.7994706630706787, 5.154031157493591, 6.317411541938782
    LBFGS Accumulation History: 3 points
    Removed measurement 96817ae to history. Total: 3
    th(0)=-1.746029607951641;dx=-4.504196788868687E-8
    Armijo: th(3.0240000000000007E7)=-1.2539303600788116; dx=9.358184481684884E-9 evalInputDelta=-0.49209924787282944
    New Minimum: -1.746029607951641 > -2.076585426926613
    WOLF (strong): th(1.5120000000000004E7)=-2.076585426926613; dx=4.417654243220191E-9 evalInputDelta=0.33055581897497177
    New Minimum: -2.076585426926613 > -2.2449972331523895
    END: th(5040000.000000001)=-2.2449972331523895; dx=-1.2153315747598675E-8 evalInputDelta=0.49896762520074844
    Fitness changed from -1.746029607951641 to -2.2449972331523895
    Iteration 9 complete. Error: -2.2449972331523895 Total: 91.5518; Orientation: 0.2823; Line Search: 73.5525
    Adding measurement 75aff9b3 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.221e+03, gradient 1.487e-04, dot -0.311; [5429384f-2f79-472f-bb4c-e605e234ece1 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from -2.2449972331523895, 3.7994706630706787, 5.154031157493591, 6.317411541938782
    LBFGS Accumulation History: 3 points
    Removed measurement 75aff9b3 to history. Total: 3
    th(0)=-2.2449972331523895;dx=-2.1711386182294618E-8
    New Minimum: -2.2449972331523895 > -2.493000879883766
    END: th(1.0858350837760696E7)=-2.493000879883766; dx=-7.3707489236821144E-9 evalInputDelta=0.24800364673137665
    Fitness changed from -2.2449972331523895 to -2.493000879883766
    Iteration 10 complete. Error: -2.493000879883766 Total: 54.4747; Orientation: 0.2800; Line Search: 35.8999
    <a id="p-13"></a>Iteration 10
    <a id="p-12"></a>![Iteration 10](etc/7cdb84e4-6f31-49ac-9dae-8ee1b2909ed5.jpg)

    Final threshold in iteration 11: -2.493000879883766 (> -Infinity) after 826.224s (< 5400.000s)

Returns

    -2.493000879883766