BasicOptimizer.scala:89 executed in 249.78 seconds (3.368 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 6065669970628
Reset training subject: 6071664929969
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
th(0)=-13.09066504240036;dx=-2.7401856996405894E-6
WOLFE (weak): th(2.154434690031884)=-13.090671986341476; dx=-2.7404408217924796E-6 evalInputDelta=6.943941116333008E-6
WOLFE (weak): th(4.308869380063768)=-13.09067952632904; dx=-2.740478441520664E-6 evalInputDelta=1.4483928680419922E-5
WOLFE (weak): th(12.926608140191302)=-13.090707510709763; dx=-2.740402025276177E-6 evalInputDelta=4.246830940246582E-5
WOLFE (weak): th(51.70643256076521)=-13.090836077928543; dx=-2.740150657280276E-6 evalInputDelta=1.710355281829834E-4
WOLFE (weak): th(258.53216280382605)=-13.091518133878708; dx=-2.739730757134475E-6 evalInputDelta=8.530914783477783E-4
WOLFE (weak): th(1551.1929768229563)=-13.095779806375504; dx=-2.7380009249461606E-6 evalInputDelta=0.005114763975143433
WOLFE (weak): th(10858.350837760694)=-13.126350224018097; dx=-2.7193435877735996E-6 evalInputDelta=0.035685181617736816
New Minimum: -13.289176404476166 > -13.366921216249466
WOLFE (weak): th(86866.80670208555)=-13.366921216249466; dx=-2.5380379735077443E-6 evalInputDelta=0.27625617384910583
New Minimum: -13.366921216249466 > -14.622052431106567
END: th(781801.26031877)=-14.622052431106567; dx=-7.869162265814523E-7 evalInputDelta=1.5313873887062073
Fitness changed from -13.289176404476166 to -14.622052431106567
Iteration 1 complete. Error: -14.622052431106567 Total: 67.2381; Orientation: 0.0211; Line Search: 51.2787
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 13bdf000 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-14.622052431106567;dx=-1.7865595153032225E-6
New Minimum: -14.622052431106567 > -15.723776251077652
END: th(1684339.7559414052)=-15.723776251077652; dx=-1.0048661645246106E-7 evalInputDelta=1.1017238199710846
Fitness changed from -14.622052431106567 to -15.723776251077652
Iteration 2 complete. Error: -15.723776251077652 Total: 15.2791; Orientation: 0.0290; Line Search: 9.8290
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1fc6ecc6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-15.723776251077652;dx=-1.302720031602462E-6
New Minimum: -15.723776251077652 > -16.093281507492065
WOLF (strong): th(3628800.0)=-16.093281507492065; dx=3.0948798124380815E-7 evalInputDelta=0.36950525641441345
New Minimum: -16.093281507492065 > -16.70912003517151
WOLF (strong): th(1814400.0)=-16.70912003517151; dx=8.920112960883058E-8 evalInputDelta=0.9853437840938568
END: th(604800.0)=-16.448246985673904; dx=-6.422825115848693E-7 evalInputDelta=0.7244707345962524
Fitness changed from -15.723776251077652 to -16.70912003517151
Iteration 3 complete. Error: -16.70912003517151 Total: 25.7427; Orientation: 0.0299; Line Search: 20.3708
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 67742ed4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-16.70912003517151;dx=-8.96409962470381E-7
New Minimum: -16.70912003517151 > -17.45534771680832
END: th(1303002.1005312833)=-17.45534771680832; dx=-1.2438968982277035E-7 evalInputDelta=0.7462276816368103
Fitness changed from -16.70912003517151 to -17.45534771680832
Iteration 4 complete. Error: -17.45534771680832 Total: 15.6095; Orientation: 0.0294; Line Search: 10.1313
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 53bcb75a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.487e+03, gradient 6.218e-04, dot -0.705; [e166994e-a78e-4dad-9d12-d17c9b5e6c7d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.45534771680832, -16.70912003517151, -15.723776251077652, -14.622052431106567
LBFGS Accumulation History: 3 points
Removed measurement 53bcb75a to history. Total: 3
th(0)=-17.45534771680832;dx=-3.862020608221848E-7
New Minimum: -17.45534771680832 > -17.920230269432068
WOLF (strong): th(2807232.9265690087)=-17.920230269432068; dx=1.2585739700583919E-8 evalInputDelta=0.4648825526237488
END: th(1403616.4632845044)=-17.863306939601898; dx=-1.1966540410343052E-7 evalInputDelta=0.4079592227935791
Fitness changed from -17.45534771680832 to -17.920230269432068
Iteration 5 complete. Error: -17.920230269432068 Total: 20.8478; Orientation: 0.6207; Line Search: 14.8262
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2fc2e23d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.130e+03, gradient 7.003e-04, dot -0.565; [e166994e-a78e-4dad-9d12-d17c9b5e6c7d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.920230269432068, -16.70912003517151, -15.723776251077652, -14.622052431106567
LBFGS Accumulation History: 3 points
Removed measurement 2fc2e23d to history. Total: 3
th(0)=-17.920230269432068;dx=-4.896348641388252E-7
New Minimum: -17.920230269432068 > -18.139648377895355
WOLF (strong): th(3024000.0)=-18.139648377895355; dx=2.1266962687276515E-7 evalInputDelta=0.21941810846328735
New Minimum: -18.139648377895355 > -18.367403864860535
END: th(1512000.0)=-18.367403864860535; dx=-3.231282051786018E-8 evalInputDelta=0.4471735954284668
Fitness changed from -17.920230269432068 to -18.367403864860535
Iteration 6 complete. Error: -18.367403864860535 Total: 20.8580; Orientation: 0.6227; Line Search: 14.9934
Adding measurement 367198b5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.589e+03, gradient 4.809e-04, dot -0.667; [e166994e-a78e-4dad-9d12-d17c9b5e6c7d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.367403864860535, -16.70912003517151, -15.723776251077652, -14.622052431106567
LBFGS Accumulation History: 3 points
Removed measurement 367198b5 to history. Total: 3
th(0)=-18.367403864860535;dx=-2.3101941952954782E-7
New Minimum: -18.367403864860535 > -18.62668687105179
WOLF (strong): th(3257505.2513282085)=-18.62668687105179; dx=4.9396773531421125E-8 evalInputDelta=0.25928300619125366
New Minimum: -18.62668687105179 > -18.639514207839966
END: th(1628752.6256641042)=-18.639514207839966; dx=-6.15184271483783E-8 evalInputDelta=0.27211034297943115
Fitness changed from -18.367403864860535 to -18.639514207839966
Iteration 7 complete. Error: -18.639514207839966 Total: 21.1948; Orientation: 0.6374; Line Search: 15.5105
Adding measurement 5e35b91a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.023e+03, gradient 4.143e-04, dot -0.709; [e166994e-a78e-4dad-9d12-d17c9b5e6c7d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.639514207839966, -16.70912003517151, -15.723776251077652, -14.622052431106567
LBFGS Accumulation History: 3 points
Removed measurement 5e35b91a to history. Total: 3
th(0)=-18.639514207839966;dx=-1.7134786030769226E-7
New Minimum: -18.639514207839966 > -18.86084794998169
WOLF (strong): th(3509041.1582112615)=-18.86084794998169; dx=4.349926000922308E-9 evalInputDelta=0.22133374214172363
END: th(1754520.5791056307)=-18.843112468719482; dx=-4.339524189787237E-8 evalInputDelta=0.2035982608795166
Fitness changed from -18.639514207839966 to -18.86084794998169
Iteration 8 complete. Error: -18.86084794998169 Total: 21.0193; Orientation: 0.6117; Line Search: 15.3412
Adding measurement 2d713322 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.991e+03, gradient 5.533e-04, dot -0.512; [e166994e-a78e-4dad-9d12-d17c9b5e6c7d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.86084794998169, -16.70912003517151, -15.723776251077652, -14.622052431106567
LBFGS Accumulation History: 3 points
Removed measurement 2d713322 to history. Total: 3
th(0)=-18.86084794998169;dx=-3.0576528341246233E-7
Armijo: th(3780000.000000001)=-18.834987580776215; dx=1.8131507426812792E-7 evalInputDelta=-0.025860369205474854
New Minimum: -18.86084794998169 > -19.149667501449585
WOLF (strong): th(1890000.0000000005)=-19.149667501449585; dx=3.3407122221570725E-8 evalInputDelta=0.2888195514678955
END: th(630000.0000000001)=-19.047780334949493; dx=-1.9019398076092672E-7 evalInputDelta=0.18693238496780396
Fitness changed from -18.86084794998169 to -19.149667501449585
Iteration 9 complete. Error: -19.149667501449585 Total: 25.8000; Orientation: 0.6306; Line Search: 19.8972
Adding measurement b0c54b6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.147e+03, gradient 4.391e-04, dot -0.559; [e166994e-a78e-4dad-9d12-d17c9b5e6c7d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.149667501449585, -16.70912003517151, -15.723776251077652, -14.622052431106567
LBFGS Accumulation History: 3 points
Removed measurement b0c54b6 to history. Total: 3
th(0)=-19.149667501449585;dx=-1.9255337788824296E-7
New Minimum: -19.149667501449585 > -19.330060839653015
END: th(1357293.854720087)=-19.330060839653015; dx=-4.028581815442173E-8 evalInputDelta=0.18039333820343018
Fitness changed from -19.149667501449585 to -19.330060839653015
Iteration 10 complete. Error: -19.330060839653015 Total: 15.9241; Orientation: 0.6321; Line Search: 9.8609
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -19.330060839653015 (> -Infinity) after 249.784s (< 5400.000s)
Returns
-19.330060839653015