Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

BasicOptimizer.scala:89 executed in 54.49 seconds (1.474 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 208892495393300
Reset training subject: 208893294631700
Adding measurement 3c83b072 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=10.97895762453004;dx=-4.545045482613796E-6
New Minimum: 10.97895762453004 > 10.97894885293302
WOLFE (weak): th(2.154434690031884)=10.97894885293302; dx=-4.570042740988529E-6 evalInputDelta=8.771597020285071E-6
New Minimum: 10.97894885293302 > 10.978940105123039
WOLFE (weak): th(4.308869380063768)=10.978940105123039; dx=-4.532021312144601E-6 evalInputDelta=1.751940700067678E-5
New Minimum: 10.978940105123039 > 10.978907332659572
WOLFE (weak): th(12.926608140191302)=10.978907332659572; dx=-4.505116825527359E-6 evalInputDelta=5.0291870467233935E-5
New Minimum: 10.978907332659572 > 10.978761357983748
WOLFE (weak): th(51.70643256076521)=10.978761357983748; dx=-4.537120565496144E-6 evalInputDelta=1.9626654629156803E-4
New Minimum: 10.978761357983748 > 10.977982892209315
WOLFE (weak): th(258.53216280382605)=10.977982892209315; dx=-4.52689492862776E-6 evalInputDelta=9.747323207243141E-4
New Minimum: 10.977982892209315 > 10.973120842072186
WOLFE (weak): th(1551.1929768229563)=10.973120842072186; dx=-4.524967350835842E-6 evalInputDelta=0.005836782457853573
New Minimum: 10.973120842072186 > 10.938181887288385
WOLFE (weak): th(10858.350837760694)=10.938181887288385; dx=-4.451232581543187E-6 evalInputDelta=0.0407757372416544
New Minimum: 10.938181887288385 > 10.688547445794356
END: th(86866.80670208555)=10.688547445794356; dx=-3.3714758907596245E-6 evalInputDelta=0.29041017873568364
Fitness changed from 10.97895762453004 to 10.688547445794356
Iteration 1 complete. Error: 10.688547445794356 Total: 9.0660; Orientation: 0.0154; Line Search: 6.6773
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/564051ac-c0ab-49cf-9f3b-4b9f4c34ae0e.jpg)
Adding measurement 6bde0e8c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=10.688547445794356;dx=-3.7507731196599777E-6
New Minimum: 10.688547445794356 > 10.20706345725415
END: th(187148.86177126726)=10.20706345725415; dx=-2.862110754043271E-6 evalInputDelta=0.48148398854020513
Fitness changed from 10.688547445794356 to 10.20706345725415
Iteration 2 complete. Error: 10.20706345725415 Total: 2.4862; Orientation: 0.0152; Line Search: 1.5990
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>![Iteration 2](etc/222f42c4-17e4-434d-8fe8-f1fbe7ea10e7.jpg)
Adding measurement 55658319 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=10.20706345725415;dx=-4.676246644878425E-6
New Minimum: 10.20706345725415 > 9.380936475665893
END: th(403200.00000000006)=9.380936475665893; dx=-2.135392032712526E-6 evalInputDelta=0.8261269815882581
Fitness changed from 10.20706345725415 to 9.380936475665893
Iteration 3 complete. Error: 9.380936475665893 Total: 2.1541; Orientation: 0.0158; Line Search: 1.4233
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>![Iteration 3](etc/4a1360b3-9c56-4da2-b638-5612e67eed6e.jpg)
Adding measurement 5cf48a53 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.638e+04, gradient 2.996e-03, dot -0.418; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 9.380936475665893, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 5cf48a53 to history. Total: 3
th(0)=9.380936475665893;dx=-8.690429216203511E-6
New Minimum: 9.380936475665893 > 8.717627773832959
END: th(868668.0670208557)=8.717627773832959; dx=-6.953392502826206E-7 evalInputDelta=0.6633087018329338
Fitness changed from 9.380936475665893 to 8.717627773832959
Iteration 4 complete. Error: 8.717627773832959 Total: 2.5187; Orientation: 0.3170; Line Search: 1.4965
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>![Iteration 4](etc/fbe37ff6-5f08-4330-b99c-f631ffa5f4e5.jpg)
Adding measurement 37f1abe9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.776e+03, gradient 3.362e-03, dot -0.239; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.717627773832959, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 37f1abe9 to history. Total: 3
th(0)=8.717627773832959;dx=-1.115480196214425E-5
Armijo: th(1871488.617712673)=9.96581236637308; dx=4.7499620101869326E-7 evalInputDelta=-1.248184592540122
New Minimum: 8.717627773832959 > 8.31798308998458
END: th(935744.3088563365)=8.31798308998458; dx=-4.6783735997041694E-7 evalInputDelta=0.39964468384837915
Fitness changed from 8.717627773832959 to 8.31798308998458
Iteration 5 complete. Error: 8.31798308998458 Total: 3.3943; Orientation: 0.2990; Line Search: 2.3805
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>![Iteration 5](etc/32ed7a42-1883-42e3-b38b-609713413999.jpg)
Adding measurement 7a537404 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.632e+03, gradient 2.792e-03, dot -0.200; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 8.31798308998458, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 7a537404 to history. Total: 3
th(0)=8.31798308998458;dx=-7.673408722512177E-6
Armijo: th(2016000.0000000007)=9.286521013700012; dx=1.6446461568338405E-7 evalInputDelta=-0.9685379237154326
New Minimum: 8.31798308998458 > 7.343285095949389
END: th(1008000.0000000003)=7.343285095949389; dx=-1.0703511962697278E-6 evalInputDelta=0.974697994035191
Fitness changed from 8.31798308998458 to 7.343285095949389
Iteration 6 complete. Error: 7.343285095949389 Total: 3.3018; Orientation: 0.3020; Line Search: 2.3060
Adding measurement 369be006 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.607e+03, gradient 2.962e-03, dot -0.163; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 7.343285095949389, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 369be006 to history. Total: 3
th(0)=7.343285095949389;dx=-8.608801476440585E-6
Armijo: th(2171670.1675521396)=9.772136220492868; dx=6.892369621457387E-7 evalInputDelta=-2.4288511245434794
New Minimum: 7.343285095949389 > 6.48266484071266
END: th(1085835.0837760698)=6.48266484071266; dx=-1.0086634486281668E-6 evalInputDelta=0.8606202552367286
Fitness changed from 7.343285095949389 to 6.48266484071266
Iteration 7 complete. Error: 6.48266484071266 Total: 3.4932; Orientation: 0.3013; Line Search: 2.4999
Adding measurement 21cad7b3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.746e+03, gradient 3.060e-03, dot -0.129; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.48266484071266, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 21cad7b3 to history. Total: 3
th(0)=6.48266484071266;dx=-9.256166325241241E-6
Armijo: th(2339360.7721408415)=10.657628556951492; dx=1.2216158417891828E-6 evalInputDelta=-4.174963716238832
New Minimum: 6.48266484071266 > 6.312784209885094
END: th(1169680.3860704207)=6.312784209885094; dx=-6.000184901094606E-7 evalInputDelta=0.16988063082756621
Fitness changed from 6.48266484071266 to 6.312784209885094
Iteration 8 complete. Error: 6.312784209885094 Total: 3.2885; Orientation: 0.2830; Line Search: 2.2733
Adding measurement d80c573 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.161e+03, gradient 3.013e-03, dot -0.120; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.312784209885094, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement d80c573 to history. Total: 3
th(0)=6.312784209885094;dx=-8.900717873178321E-6
Armijo: th(2520000.000000001)=11.110973251578075; dx=1.4868722220022569E-6 evalInputDelta=-4.798189041692981
New Minimum: 6.312784209885094 > 5.949776670615901
END: th(1260000.0000000005)=5.949776670615901; dx=-4.5300454015999203E-7 evalInputDelta=0.3630075392691925
Fitness changed from 6.312784209885094 to 5.949776670615901
Iteration 9 complete. Error: 5.949776670615901 Total: 3.2529; Orientation: 0.3116; Line Search: 2.2800
Adding measurement 690eceb0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.832e+03, gradient 2.929e-03, dot -0.110; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.949776670615901, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 690eceb0 to history. Total: 3
th(0)=5.949776670615901;dx=-8.49625098969425E-6
Armijo: th(2714587.7094401745)=11.596818262875944; dx=1.6884327711991952E-6 evalInputDelta=-5.6470415922600425
New Minimum: 5.949776670615901 > 5.562189229527056
END: th(1357293.8547200873)=5.562189229527056; dx=-5.742218220741019E-7 evalInputDelta=0.38758744108884535
Fitness changed from 5.949776670615901 to 5.562189229527056
Iteration 10 complete. Error: 5.562189229527056 Total: 3.6122; Orientation: 0.4693; Line Search: 2.4428
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>![Iteration 10](etc/9b171104-ab21-4e3e-8c7b-433383043d81.jpg)
Adding measurement 5ab240b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.675e+03, gradient 3.011e-03, dot -0.105; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 5.562189229527056, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 5ab240b0 to history. Total: 3
th(0)=5.562189229527056;dx=-8.945348729376712E-6
Armijo: th(2924200.965176052)=12.661015810329088; dx=1.9916314334523204E-6 evalInputDelta=-7.098826580802032
Armijo: th(1462100.482588026)=5.564219489173981; dx=-3.1991866090540117E-7 evalInputDelta=-0.0020302596469248257
New Minimum: 5.562189229527056 > 3.6576893476499146
END: th(487366.827529342)=3.6576893476499146; dx=-4.965475287907819E-6 evalInputDelta=1.9044998818771415
Fitness changed from 5.562189229527056 to 3.6576893476499146
Iteration 11 complete. Error: 3.6576893476499146 Total: 4.0582; Orientation: 0.3115; Line Search: 2.9726
Adding measurement 3fc9eba0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.624e+03, gradient 2.764e-03, dot -0.116; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.6576893476499146, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 3fc9eba0 to history. Total: 3
th(0)=3.6576893476499146;dx=-7.542637262399273E-6
New Minimum: 3.6576893476499146 > 3.1175412484460363
END: th(1050000.0000000005)=3.1175412484460363; dx=-1.688754876146491E-6 evalInputDelta=0.5401480992038783
Fitness changed from 3.6576893476499146 to 3.1175412484460363
Iteration 12 complete. Error: 3.1175412484460363 Total: 2.7810; Orientation: 0.3040; Line Search: 1.5620
Adding measurement 17530672 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.932e+03, gradient 3.178e-03, dot -0.108; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.1175412484460363, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 17530672 to history. Total: 3
th(0)=3.1175412484460363;dx=-1.0037222759465178E-5
Armijo: th(2262156.424533479)=10.009744967455934; dx=1.8776983737357586E-6 evalInputDelta=-6.892203719009897
Armijo: th(1131078.2122667395)=3.548477269493876; dx=-6.944851988528021E-7 evalInputDelta=-0.4309360210478399
New Minimum: 3.1175412484460363 > 1.65726599145637
END: th(377026.0707555798)=1.65726599145637; dx=-5.654002710688113E-6 evalInputDelta=1.4602752569896662
Fitness changed from 3.1175412484460363 to 1.65726599145637
Iteration 13 complete. Error: 1.65726599145637 Total: 3.9792; Orientation: 0.3023; Line Search: 3.0114
Adding measurement 52d8a822 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.136e+04, gradient 2.890e-03, dot -0.127; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.65726599145637, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement 52d8a822 to history. Total: 3
th(0)=1.65726599145637;dx=-8.284609465214147E-6
New Minimum: 1.65726599145637 > 1.2326647304309475
END: th(812278.0458822367)=1.2326647304309475; dx=-2.485162993695284E-6 evalInputDelta=0.4246012610254226
Fitness changed from 1.65726599145637 to 1.2326647304309475
Iteration 14 complete. Error: 1.2326647304309475 Total: 3.0361; Orientation: 0.3006; Line Search: 2.0203
Adding measurement e20f553 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.291e+04, gradient 3.250e-03, dot -0.117; [af0a39e5-f333-4246-ac79-dbeca817869e = 1.000/1.000e+00]
Orientation rejected. Popping history element from 1.2326647304309475, 10.20706345725415, 10.688547445794356, 10.97895762453004
LBFGS Accumulation History: 3 points
Removed measurement e20f553 to history. Total: 3
th(0)=1.2326647304309475;dx=-1.0447771333274687E-5
Armijo: th(1750000.000000001)=6.976168979381975; dx=1.4958203231722796E-6 evalInputDelta=-5.7435042489510275
Armijo: th(875000.0000000005)=1.6466258931772622; dx=-1.3643813815615485E-6 evalInputDelta=-0.41396116274631467
New Minimum: 1.2326647304309475 > 0.15297232681454354
END: th(291666.6666666668)=0.15297232681454354; dx=-6.683076694924507E-6 evalInputDelta=1.079692403616404
Fitness changed from 1.2326647304309475 to 0.15297232681454354
Iteration 15 complete. Error: 0.15297232681454354 Total: 4.0425; Orientation: 0.2816; Line Search: 3.0343
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>![Iteration 15](etc/41d9a247-aa75-4792-8d1b-6e4bb33498ae.jpg)
Final threshold in iteration 16: 0.15297232681454354 (> -Infinity) after 54.486s (< 5400.000s)

Returns

    0.15297232681454354