BasicOptimizer.scala:89 executed in 221.96 seconds (3.795 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 694964483718600
Reset training subject: 694968984142600
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -875.1283834386816 > -876.745615900074
th(0)=-876.745615900074;dx=-0.00567816807494659
New Minimum: -876.745615900074 > -876.7578414248715
WOLFE (weak): th(2.154434690031884)=-876.7578414248715; dx=-0.005184172200042515 evalInputDelta=0.012225524797486287
New Minimum: -876.7578414248715 > -876.7700667791984
WOLFE (weak): th(4.308869380063768)=-876.7700667791984; dx=-0.005186228062236582 evalInputDelta=0.02445087912440158
New Minimum: -876.7700667791984 > -876.8189308517466
WOLFE (weak): th(12.926608140191302)=-876.8189308517466; dx=-0.0051847927014711335 evalInputDelta=0.07331495167261437
New Minimum: -876.8189308517466 > -877.0388745660024
WOLFE (weak): th(51.70643256076521)=-877.0388745660024; dx=-0.0051817038940047844 evalInputDelta=0.29325866592841976
New Minimum: -877.0388745660024 > -878.2100403661985
WOLFE (weak): th(258.53216280382605)=-878.2100403661985; dx=-0.005166465452253917 evalInputDelta=1.4644244661245693
New Minimum: -878.2100403661985 > -885.482460300598
END: th(1551.1929768229563)=-885.482460300598; dx=-0.005110328964823745 evalInputDelta=8.736844400523978
Fitness changed from -875.1283834386816 to -885.482460300598
Iteration 1 complete. Error: -885.482460300598 Total: 45.5813; Orientation: 0.0608; Line Search: 31.8216
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 18b6dd7c to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-885.482460300598;dx=-0.005079732296149142
New Minimum: -885.482460300598 > -903.7540110561059
WOLFE (weak): th(3341.943960201201)=-903.7540110561059; dx=-0.00492419494924182 evalInputDelta=18.271550755507974
New Minimum: -903.7540110561059 > -921.4682916411177
WOLFE (weak): th(6683.887920402402)=-921.4682916411177; dx=-0.004765766808757845 evalInputDelta=35.98583134051978
New Minimum: -921.4682916411177 > -986.0886750186314
END: th(20051.663761207208)=-986.0886750186314; dx=-0.004059178781527736 evalInputDelta=100.60621471803347
Fitness changed from -885.482460300598 to -986.0886750186314
Iteration 2 complete. Error: -986.0886750186314 Total: 23.6304; Orientation: 0.0966; Line Search: 18.2259
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5f67e294 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-986.0886750186314;dx=-0.003786116214105474
New Minimum: -986.0886750186314 > -1136.7272100200646
END: th(43200.00000000001)=-1136.7272100200646; dx=-0.002610405196450403 evalInputDelta=150.63853500143318
Fitness changed from -986.0886750186314 to -1136.7272100200646
Iteration 3 complete. Error: -1136.7272100200646 Total: 14.9094; Orientation: 0.0992; Line Search: 9.1841
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7dbeac7d to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1136.7272100200646;dx=-0.0026265489754099657
New Minimum: -1136.7272100200646 > -1337.5252334068318
END: th(93071.5786093774)=-1337.5252334068318; dx=-0.0014632779604029588 evalInputDelta=200.79802338676723
Fitness changed from -1136.7272100200646 to -1337.5252334068318
Iteration 4 complete. Error: -1337.5252334068318 Total: 14.7275; Orientation: 0.0991; Line Search: 9.2220
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5ec6ebd6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.681e+04, gradient 4.873e-02, dot -0.705; [c17308fb-25cc-4713-b882-302fbda0dd8e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1337.5252334068318, -1136.7272100200646, -986.0886750186314, -885.482460300598
LBFGS Accumulation History: 3 points
Removed measurement 5ec6ebd6 to history. Total: 3
th(0)=-1337.5252334068318;dx=-0.0023730172149061843
New Minimum: -1337.5252334068318 > -1550.305937549102
END: th(200516.63761207208)=-1550.305937549102; dx=-4.982960578479351E-4 evalInputDelta=212.78070414227022
Fitness changed from -1337.5252334068318 to -1550.305937549102
Iteration 5 complete. Error: -1550.305937549102 Total: 16.1657; Orientation: 1.7176; Line Search: 9.0428
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6ad3ccfe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.472e+04, gradient 5.483e-02, dot -0.436; [c17308fb-25cc-4713-b882-302fbda0dd8e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1550.305937549102, -1136.7272100200646, -986.0886750186314, -885.482460300598
LBFGS Accumulation History: 3 points
Removed measurement 6ad3ccfe to history. Total: 3
th(0)=-1550.305937549102;dx=-0.0030030408684783624
New Minimum: -1550.305937549102 > -1649.1482650143982
WOLF (strong): th(432000.0000000001)=-1649.1482650143982; dx=2.0567130254354785E-4 evalInputDelta=98.8423274652962
New Minimum: -1649.1482650143982 > -1707.36426986159
WOLF (strong): th(216000.00000000006)=-1707.36426986159; dx=1.3788893167967624E-4 evalInputDelta=157.0583323124879
END: th(72000.00000000001)=-1690.7649895831178; dx=-8.228367411917917E-4 evalInputDelta=140.4590520340157
Fitness changed from -1550.305937549102 to -1707.36426986159
Iteration 6 complete. Error: -1707.36426986159 Total: 25.2823; Orientation: 1.6828; Line Search: 18.2095
Adding measurement fd04deb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.844e+04, gradient 5.801e-02, dot -0.346; [c17308fb-25cc-4713-b882-302fbda0dd8e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1707.36426986159, -1136.7272100200646, -986.0886750186314, -885.482460300598
LBFGS Accumulation History: 3 points
Removed measurement fd04deb to history. Total: 3
th(0)=-1707.36426986159;dx=-0.0033592681114552768
New Minimum: -1707.36426986159 > -1926.0577613353137
WOLF (strong): th(155119.29768229567)=-1926.0577613353137; dx=4.516693271852663E-5 evalInputDelta=218.69349147372372
END: th(77559.64884114783)=-1892.9883121667708; dx=-0.0010745433555077195 evalInputDelta=185.62404230518086
Fitness changed from -1707.36426986159 to -1926.0577613353137
Iteration 7 complete. Error: -1926.0577613353137 Total: 19.9440; Orientation: 1.6153; Line Search: 13.6638
Adding measurement 3bf2be97 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.467e+04, gradient 5.247e-02, dot -0.370; [c17308fb-25cc-4713-b882-302fbda0dd8e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1926.0577613353137, -1136.7272100200646, -986.0886750186314, -885.482460300598
LBFGS Accumulation History: 3 points
Removed measurement 3bf2be97 to history. Total: 3
th(0)=-1926.0577613353137;dx=-0.0027470794431236284
New Minimum: -1926.0577613353137 > -2081.921108015383
WOLF (strong): th(167097.1980100601)=-2081.921108015383; dx=2.861214024931524E-4 evalInputDelta=155.86334668006953
END: th(83548.59900503005)=-2074.3572498487333; dx=-6.906001104380072E-4 evalInputDelta=148.29948851341965
Fitness changed from -1926.0577613353137 to -2081.921108015383
Iteration 8 complete. Error: -2081.921108015383 Total: 20.0575; Orientation: 1.6753; Line Search: 13.9881
Adding measurement 7fe123a7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.482e+04, gradient 5.576e-02, dot -0.358; [c17308fb-25cc-4713-b882-302fbda0dd8e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2081.921108015383, -1136.7272100200646, -986.0886750186314, -885.482460300598
LBFGS Accumulation History: 3 points
Removed measurement 7fe123a7 to history. Total: 3
th(0)=-2081.921108015383;dx=-0.0031069917297152985
New Minimum: -2081.921108015383 > -2208.2108142113607
WOLF (strong): th(180000.00000000006)=-2208.2108142113607; dx=6.178267686468817E-4 evalInputDelta=126.28970619597749
New Minimum: -2208.2108142113607 > -2233.2365526293893
END: th(90000.00000000003)=-2233.2365526293893; dx=-3.799476501297074E-4 evalInputDelta=151.31544461400608
Fitness changed from -2081.921108015383 to -2233.2365526293893
Iteration 9 complete. Error: -2233.2365526293893 Total: 20.0153; Orientation: 1.7662; Line Search: 13.6325
Adding measurement d189e1b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.479e+04, gradient 4.235e-02, dot -0.475; [c17308fb-25cc-4713-b882-302fbda0dd8e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2233.2365526293893, -1136.7272100200646, -986.0886750186314, -885.482460300598
LBFGS Accumulation History: 3 points
Removed measurement d189e1b to history. Total: 3
th(0)=-2233.2365526293893;dx=-0.0017903791020195963
New Minimum: -2233.2365526293893 > -2348.531668936998
WOLF (strong): th(193899.12210286962)=-2348.531668936998; dx=3.6997698201061404E-5 evalInputDelta=115.29511630760862
END: th(96949.56105143481)=-2336.5698149210093; dx=-4.496711942712034E-4 evalInputDelta=103.33326229162003
Fitness changed from -2233.2365526293893 to -2348.531668936998
Iteration 10 complete. Error: -2348.531668936998 Total: 20.7093; Orientation: 1.6522; Line Search: 14.2484
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -2348.531668936998 (> -Infinity) after 221.959s (< 5400.000s)
Returns
-2348.531668936998