BasicOptimizer.scala:89 executed in 293.10 seconds (3.590 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 581864449800700
Reset training subject: 581870647101300
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -37.382410407275394 > -37.42014245291966
th(0)=-37.42014245291966;dx=-2.025831820148893E-6
New Minimum: -37.42014245291966 > -37.42014447127778
END: th(2.154434690031884)=-37.42014447127778; dx=-1.7731486339448578E-6 evalInputDelta=2.0183581170840625E-6
Fitness changed from -37.382410407275394 to -37.42014447127778
Iteration 1 complete. Error: -37.42014447127778 Total: 30.5464; Orientation: 0.0932; Line Search: 11.8960
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4b7b93fc to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-37.42014447127778;dx=-1.7804278729351252E-6
New Minimum: -37.42014447127778 > -37.42015456306671
WOLFE (weak): th(4.641588833612779)=-37.42015456306671; dx=-1.7779536740471343E-6 evalInputDelta=1.0091788929855738E-5
New Minimum: -37.42015456306671 > -37.420164654852925
WOLFE (weak): th(9.283177667225559)=-37.420164654852925; dx=-1.7779776526783764E-6 evalInputDelta=2.0183575145438226E-5
New Minimum: -37.420164654852925 > -37.42020300361572
WOLFE (weak): th(27.849533001676676)=-37.42020300361572; dx=-1.7766805482398116E-6 evalInputDelta=5.853233793828849E-5
New Minimum: -37.42020300361572 > -37.42036447165952
WOLFE (weak): th(111.3981320067067)=-37.42036447165952; dx=-1.7758368586052693E-6 evalInputDelta=2.2000038173786152E-4
New Minimum: -37.42036447165952 > -37.4212363870568
WOLFE (weak): th(556.9906600335335)=-37.4212363870568; dx=-1.7758323239441297E-6 evalInputDelta=0.0010919157790212353
New Minimum: -37.4212363870568 > -37.426689434091
WOLFE (weak): th(3341.9439602012008)=-37.426689434091; dx=-1.7753978454677118E-6 evalInputDelta=0.00654496281322281
New Minimum: -37.426689434091 > -37.46591705768039
WOLFE (weak): th(23393.607721408407)=-37.46591705768039; dx=-1.7730437714140846E-6 evalInputDelta=0.04577258640261306
New Minimum: -37.46591705768039 > -37.78423092052022
WOLFE (weak): th(187148.86177126726)=-37.78423092052022; dx=-1.7518901903310322E-6 evalInputDelta=0.3640864492424427
New Minimum: -37.78423092052022 > -40.483189235216614
END: th(1684339.7559414054)=-40.483189235216614; dx=-1.5107067874976562E-6 evalInputDelta=3.0630447639388336
Fitness changed from -37.42014447127778 to -40.483189235216614
Iteration 2 complete. Error: -40.483189235216614 Total: 67.0399; Orientation: 0.1512; Line Search: 59.9780
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2a51cf84 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-40.483189235216614;dx=-1.4487617202269062E-6
New Minimum: -40.483189235216614 > -45.21769432840057
END: th(3628800.0000000005)=-45.21769432840057; dx=-9.592880154376027E-7 evalInputDelta=4.734505093183955
Fitness changed from -40.483189235216614 to -45.21769432840057
Iteration 3 complete. Error: -45.21769432840057 Total: 19.1321; Orientation: 0.1351; Line Search: 11.9417
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 68806e35 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-45.21769432840057;dx=-9.339707294468233E-7
New Minimum: -45.21769432840057 > -51.29868901877223
END: th(7818012.603187701)=-51.29868901877223; dx=-5.149959635050044E-7 evalInputDelta=6.080994690371661
Fitness changed from -45.21769432840057 to -51.29868901877223
Iteration 4 complete. Error: -51.29868901877223 Total: 19.7959; Orientation: 0.1340; Line Search: 12.2899
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 16109edb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.375e+04, gradient 8.557e-04, dot -0.710; [d9c3fc32-5b01-4eb0-8648-4993a256de49 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -51.29868901877223, -45.21769432840057, -40.483189235216614, -37.42014447127778
LBFGS Accumulation History: 3 points
Removed measurement 16109edb to history. Total: 3
th(0)=-51.29868901877223;dx=-7.319772893734079E-7
New Minimum: -51.29868901877223 > -57.72858967390094
END: th(1.6843397559414055E7)=-57.72858967390094; dx=-1.5479461929216363E-7 evalInputDelta=6.429900655128712
Fitness changed from -51.29868901877223 to -57.72858967390094
Iteration 5 complete. Error: -57.72858967390094 Total: 21.9801; Orientation: 2.5505; Line Search: 12.5618
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2bd4b2c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.450e+04, gradient 1.020e-03, dot -0.462; [d9c3fc32-5b01-4eb0-8648-4993a256de49 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.72858967390094, -45.21769432840057, -40.483189235216614, -37.42014447127778
LBFGS Accumulation History: 3 points
Removed measurement 2bd4b2c to history. Total: 3
th(0)=-57.72858967390094;dx=-1.038715013260152E-6
Armijo: th(3.628800000000001E7)=-56.237457606950656; dx=2.076167123712837E-7 evalInputDelta=-1.4911320669502857
New Minimum: -57.72858967390094 > -61.55038742115107
WOLF (strong): th(1.8144000000000004E7)=-61.55038742115107; dx=2.4211832301276834E-7 evalInputDelta=3.8217977472501303
New Minimum: -61.55038742115107 > -62.500548977797685
END: th(6048000.000000001)=-62.500548977797685; dx=-3.7057544818310436E-7 evalInputDelta=4.771959303896743
Fitness changed from -57.72858967390094 to -62.500548977797685
Iteration 6 complete. Error: -62.500548977797685 Total: 33.9122; Orientation: 2.5312; Line Search: 24.4528
Adding measurement 24c39e7f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.967e+04, gradient 7.244e-04, dot -0.672; [d9c3fc32-5b01-4eb0-8648-4993a256de49 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -62.500548977797685, -45.21769432840057, -40.483189235216614, -37.42014447127778
LBFGS Accumulation History: 3 points
Removed measurement 24c39e7f to history. Total: 3
th(0)=-62.500548977797685;dx=-5.234658476777971E-7
New Minimum: -62.500548977797685 > -67.00744092451123
END: th(1.3030021005312836E7)=-67.00744092451123; dx=-1.7706462173381943E-7 evalInputDelta=4.506891946713544
Fitness changed from -62.500548977797685 to -67.00744092451123
Iteration 7 complete. Error: -67.00744092451123 Total: 20.5067; Orientation: 2.4702; Line Search: 12.0190
Adding measurement 505d8970 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.917e+04, gradient 8.257e-04, dot -0.558; [d9c3fc32-5b01-4eb0-8648-4993a256de49 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -67.00744092451123, -45.21769432840057, -40.483189235216614, -37.42014447127778
LBFGS Accumulation History: 3 points
Removed measurement 505d8970 to history. Total: 3
th(0)=-67.00744092451123;dx=-6.811696752011651E-7
New Minimum: -67.00744092451123 > -68.39867040338771
WOLF (strong): th(2.8072329265690096E7)=-68.39867040338771; dx=1.3210756953968265E-7 evalInputDelta=1.3912294788764825
New Minimum: -68.39867040338771 > -70.40333061458942
WOLF (strong): th(1.4036164632845048E7)=-70.40333061458942; dx=5.199985983570339E-8 evalInputDelta=3.3958896900781923
END: th(4678721.544281683)=-69.62442143140308; dx=-3.3619825707629383E-7 evalInputDelta=2.6169805068918492
Fitness changed from -67.00744092451123 to -70.40333061458942
Iteration 8 complete. Error: -70.40333061458942 Total: 32.3216; Orientation: 2.4237; Line Search: 24.2743
Adding measurement 388fffab to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.152e+04, gradient 9.379e-04, dot -0.449; [d9c3fc32-5b01-4eb0-8648-4993a256de49 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.40333061458942, -45.21769432840057, -40.483189235216614, -37.42014447127778
LBFGS Accumulation History: 3 points
Removed measurement 388fffab to history. Total: 3
th(0)=-70.40333061458942;dx=-8.796411052009541E-7
New Minimum: -70.40333061458942 > -75.34693383799916
WOLF (strong): th(1.0080000000000004E7)=-75.34693383799916; dx=8.215978517252526E-9 evalInputDelta=4.943603223409738
END: th(5040000.000000002)=-74.21460231166158; dx=-4.6573331663895585E-7 evalInputDelta=3.811271697072158
Fitness changed from -70.40333061458942 to -75.34693383799916
Iteration 9 complete. Error: -75.34693383799916 Total: 26.7658; Orientation: 2.4495; Line Search: 18.2989
Adding measurement 29651084 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.664e+04, gradient 7.417e-04, dot -0.566; [d9c3fc32-5b01-4eb0-8648-4993a256de49 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -75.34693383799916, -45.21769432840057, -40.483189235216614, -37.42014447127778
LBFGS Accumulation History: 3 points
Removed measurement 29651084 to history. Total: 3
th(0)=-75.34693383799916;dx=-5.4966583359246E-7
New Minimum: -75.34693383799916 > -78.6491361847928
END: th(1.0858350837760698E7)=-78.6491361847928; dx=-5.4097268801870174E-8 evalInputDelta=3.302202346793635
Fitness changed from -75.34693383799916 to -78.6491361847928
Iteration 10 complete. Error: -78.6491361847928 Total: 20.0458; Orientation: 2.4395; Line Search: 11.8035
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -78.6491361847928 (> -Infinity) after 293.101s (< 5400.000s)
Returns
-78.6491361847928