BasicOptimizer.scala:89 executed in 138.04 seconds (1.691 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 70445793588300
Reset training subject: 70448497669000
Adding measurement 18e7afe0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-13.519568354508586;dx=-9.096113128933702E-7
New Minimum: -13.519568354508586 > -13.51957026386808
END: th(2.154434690031884)=-13.51957026386808; dx=-8.159417713198861E-7 evalInputDelta=1.909359493623697E-6
Fitness changed from -13.519568354508586 to -13.51957026386808
Iteration 1 complete. Error: -13.51957026386808 Total: 13.1339; Orientation: 0.0625; Line Search: 5.1484
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2de35662 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-13.51957026386808;dx=-8.135072347934116E-7
New Minimum: -13.51957026386808 > -13.519574082586258
WOLFE (weak): th(4.641588833612779)=-13.519574082586258; dx=-8.135528961240132E-7 evalInputDelta=3.818718179005032E-6
New Minimum: -13.519574082586258 > -13.519578855982465
WOLFE (weak): th(9.283177667225559)=-13.519578855982465; dx=-8.133161980552471E-7 evalInputDelta=8.592114385308491E-6
New Minimum: -13.519578855982465 > -13.519596040194854
WOLFE (weak): th(27.849533001676676)=-13.519596040194854; dx=-8.131149630387391E-7 evalInputDelta=2.57763267743627E-5
New Minimum: -13.519596040194854 > -13.51967336888031
WOLFE (weak): th(111.3981320067067)=-13.51967336888031; dx=-8.126023701712675E-7 evalInputDelta=1.0310501223109725E-4
New Minimum: -13.51967336888031 > -13.520082917136246
WOLFE (weak): th(556.9906600335335)=-13.520082917136246; dx=-8.127101522961158E-7 evalInputDelta=5.126532681671847E-4
New Minimum: -13.520082917136246 > -13.52264636918251
WOLFE (weak): th(3341.9439602012008)=-13.52264636918251; dx=-8.121418633805327E-7 evalInputDelta=0.0030761053144310324
New Minimum: -13.52264636918251 > -13.541033996037076
WOLFE (weak): th(23393.607721408407)=-13.541033996037076; dx=-8.06933092221188E-7 evalInputDelta=0.02146373216899633
New Minimum: -13.541033996037076 > -13.685899584839222
WOLFE (weak): th(187148.86177126726)=-13.685899584839222; dx=-7.542485688861933E-7 evalInputDelta=0.16632932097114228
New Minimum: -13.685899584839222 > -14.500323898462227
END: th(1684339.7559414054)=-14.500323898462227; dx=-2.945316230051663E-7 evalInputDelta=0.9807536345941479
Fitness changed from -13.51957026386808 to -14.500323898462227
Iteration 2 complete. Error: -14.500323898462227 Total: 29.2532; Orientation: 0.0631; Line Search: 25.8879
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5c9340c to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-14.500323898462227;dx=-8.949895146553464E-7
New Minimum: -14.500323898462227 > -15.305768975536143
END: th(3628800.0000000005)=-15.305768975536143; dx=-8.762065942351169E-8 evalInputDelta=0.8054450770739159
Fitness changed from -14.500323898462227 to -15.305768975536143
Iteration 3 complete. Error: -15.305768975536143 Total: 8.3563; Orientation: 0.0634; Line Search: 5.2924
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 71afd5ae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.995e+03, gradient 8.688e-04, dot -0.481; [b8e5851b-ff8c-420f-9b65-b20f97abf514 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.305768975536143, -14.500323898462227, -13.51957026386808, -13.519568354508586
LBFGS Accumulation History: 3 points
Removed measurement 71afd5ae to history. Total: 3
th(0)=-15.305768975536143;dx=-7.548367992457454E-7
New Minimum: -15.305768975536143 > -16.70094487871767
WOLF (strong): th(7818012.603187701)=-16.70094487871767; dx=1.911266473649967E-8 evalInputDelta=1.3951759031815278
New Minimum: -16.70094487871767 > -16.850196182003202
END: th(3909006.3015938504)=-16.850196182003202; dx=-1.5663433714541385E-8 evalInputDelta=1.5444272064670592
Fitness changed from -15.305768975536143 to -16.850196182003202
Iteration 4 complete. Error: -16.850196182003202 Total: 11.8817; Orientation: 1.1059; Line Search: 7.8302
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 602a0a80 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.027e+03, gradient 9.206e-04, dot -0.314; [b8e5851b-ff8c-420f-9b65-b20f97abf514 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.850196182003202, -14.500323898462227, -13.51957026386808, -13.519568354508586
LBFGS Accumulation History: 3 points
Removed measurement 602a0a80 to history. Total: 3
th(0)=-16.850196182003202;dx=-8.463858754321491E-7
New Minimum: -16.850196182003202 > -17.10260236383172
WOLF (strong): th(8421698.779707028)=-17.10260236383172; dx=8.079553875573944E-8 evalInputDelta=0.25240618182851904
New Minimum: -17.10260236383172 > -17.891089522172038
WOLF (strong): th(4210849.389853514)=-17.891089522172038; dx=2.224050184699129E-7 evalInputDelta=1.0408933401688358
New Minimum: -17.891089522172038 > -17.973831074634692
END: th(1403616.4632845046)=-17.973831074634692; dx=-4.5786907808770137E-7 evalInputDelta=1.1236348926314896
Fitness changed from -16.850196182003202 to -17.973831074634692
Iteration 5 complete. Error: -17.973831074634692 Total: 14.7059; Orientation: 1.1054; Line Search: 10.5188
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 748d435e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.864e+03, gradient 7.813e-04, dot -0.499; [b8e5851b-ff8c-420f-9b65-b20f97abf514 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.973831074634692, -14.500323898462227, -13.51957026386808, -13.519568354508586
LBFGS Accumulation History: 3 points
Removed measurement 748d435e to history. Total: 3
th(0)=-17.973831074634692;dx=-6.101768761672555E-7
New Minimum: -17.973831074634692 > -18.612133133386404
WOLF (strong): th(3024000.0000000005)=-18.612133133386404; dx=6.96507106791006E-8 evalInputDelta=0.6383020587517123
New Minimum: -18.612133133386404 > -18.63897236671824
END: th(1512000.0000000002)=-18.63897236671824; dx=-1.349020868258031E-7 evalInputDelta=0.6651412920835469
Fitness changed from -17.973831074634692 to -18.63897236671824
Iteration 6 complete. Error: -18.63897236671824 Total: 11.8822; Orientation: 1.0968; Line Search: 7.7836
Adding measurement 37182d6e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.041e+04, gradient 7.536e-04, dot -0.550; [b8e5851b-ff8c-420f-9b65-b20f97abf514 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.63897236671824, -14.500323898462227, -13.51957026386808, -13.519568354508586
LBFGS Accumulation History: 3 points
Removed measurement 37182d6e to history. Total: 3
th(0)=-18.63897236671824;dx=-5.678873860624278E-7
New Minimum: -18.63897236671824 > -18.935491204621698
WOLF (strong): th(3257505.251328209)=-18.935491204621698; dx=3.324333771243816E-8 evalInputDelta=0.29651883790345934
New Minimum: -18.935491204621698 > -19.041233258820583
WOLF (strong): th(1628752.6256641045)=-19.041233258820583; dx=2.0681959496343066E-8 evalInputDelta=0.40226089210234406
END: th(542917.5418880348)=-18.926512105881162; dx=-3.3481636663173404E-7 evalInputDelta=0.2875397391629235
Fitness changed from -18.63897236671824 to -19.041233258820583
Iteration 7 complete. Error: -19.041233258820583 Total: 13.9229; Orientation: 1.0326; Line Search: 10.2836
Adding measurement cbc1822 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.068e+04, gradient 8.320e-04, dot -0.476; [b8e5851b-ff8c-420f-9b65-b20f97abf514 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.041233258820583, -14.500323898462227, -13.51957026386808, -13.519568354508586
LBFGS Accumulation History: 3 points
Removed measurement cbc1822 to history. Total: 3
th(0)=-19.041233258820583;dx=-6.916891065045245E-7
New Minimum: -19.041233258820583 > -19.558450116758113
WOLF (strong): th(1169680.3860704205)=-19.558450116758113; dx=2.3761255179861926E-8 evalInputDelta=0.5172168579375302
END: th(584840.1930352102)=-19.43551211599841; dx=-4.3506225820119316E-7 evalInputDelta=0.3942788571778273
Fitness changed from -19.041233258820583 to -19.558450116758113
Iteration 8 complete. Error: -19.558450116758113 Total: 11.2937; Orientation: 1.0327; Line Search: 7.7488
Adding measurement 5fc63af5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.331e+04, gradient 7.386e-04, dot -0.561; [b8e5851b-ff8c-420f-9b65-b20f97abf514 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.558450116758113, -14.500323898462227, -13.51957026386808, -13.519568354508586
LBFGS Accumulation History: 3 points
Removed measurement 5fc63af5 to history. Total: 3
th(0)=-19.558450116758113;dx=-5.451260493475677E-7
New Minimum: -19.558450116758113 > -19.849128736373757
WOLF (strong): th(1260000.0000000002)=-19.849128736373757; dx=5.493292698457077E-8 evalInputDelta=0.2906786196156439
END: th(630000.0000000001)=-19.830188375476656; dx=-1.795911770560141E-7 evalInputDelta=0.2717382587185426
Fitness changed from -19.558450116758113 to -19.849128736373757
Iteration 9 complete. Error: -19.849128736373757 Total: 11.4393; Orientation: 1.0927; Line Search: 7.9120
Adding measurement 46e33567 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.329e+04, gradient 8.137e-04, dot -0.490; [b8e5851b-ff8c-420f-9b65-b20f97abf514 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.849128736373757, -14.500323898462227, -13.51957026386808, -13.519568354508586
LBFGS Accumulation History: 3 points
Removed measurement 46e33567 to history. Total: 3
th(0)=-19.849128736373757;dx=-6.620992545637557E-7
New Minimum: -19.849128736373757 > -20.161808059167942
WOLF (strong): th(1357293.854720087)=-20.161808059167942; dx=1.7775087764334155E-7 evalInputDelta=0.3126793227941853
New Minimum: -20.161808059167942 > -20.212696392925626
END: th(678646.9273600435)=-20.212696392925626; dx=-1.8053181770708359E-7 evalInputDelta=0.3635676565518686
Fitness changed from -19.849128736373757 to -20.212696392925626
Iteration 10 complete. Error: -20.212696392925626 Total: 11.6091; Orientation: 1.0678; Line Search: 7.9096
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -20.212696392925626 (> -Infinity) after 138.042s (< 5400.000s)
Returns
-20.212696392925626