BasicOptimizer.scala:89 executed in 133.49 seconds (1.474 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 329015643831200
Reset training subject: 329018370746400
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -114.06487353158583 > -114.2148035357162
th(0)=-114.2148035357162;dx=-2.7621217125075046E-5
New Minimum: -114.2148035357162 > -114.21485420298696
END: th(2.154434690031884)=-114.21485420298696; dx=-2.078295447028958E-5 evalInputDelta=5.066727075586641E-5
Fitness changed from -114.06487353158583 to -114.21485420298696
Iteration 1 complete. Error: -114.21485420298696 Total: 13.1094; Orientation: 0.0375; Line Search: 5.3402
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 75315ea9 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-114.21485420298696;dx=-2.075945864376188E-5
New Minimum: -114.21485420298696 > -114.21495975972886
WOLFE (weak): th(4.641588833612779)=-114.21495975972886; dx=-2.0753934752835292E-5 evalInputDelta=1.0555674190015907E-4
New Minimum: -114.21495975972886 > -114.21506953863694
WOLFE (weak): th(9.283177667225559)=-114.21506953863694; dx=-2.073945871047806E-5 evalInputDelta=2.1533564998321708E-4
New Minimum: -114.21506953863694 > -114.21551287546149
WOLFE (weak): th(27.849533001676676)=-114.21551287546149; dx=-2.0735368127275893E-5 evalInputDelta=6.586724745289985E-4
New Minimum: -114.21551287546149 > -114.21748464792007
WOLFE (weak): th(111.3981320067067)=-114.21748464792007; dx=-2.0720380153970208E-5 evalInputDelta=0.002630444933117815
New Minimum: -114.21748464792007 > -114.2280184872689
WOLFE (weak): th(556.9906600335335)=-114.2280184872689; dx=-2.0716283580362727E-5 evalInputDelta=0.013164284281941718
New Minimum: -114.2280184872689 > -114.2937876293952
WOLFE (weak): th(3341.9439602012008)=-114.2937876293952; dx=-2.0702251775284472E-5 evalInputDelta=0.0789334264082413
New Minimum: -114.2937876293952 > -114.76541193710148
WOLFE (weak): th(23393.607721408407)=-114.76541193710148; dx=-2.055754029481239E-5 evalInputDelta=0.5505577341145198
New Minimum: -114.76541193710148 > -118.47661154416701
WOLFE (weak): th(187148.86177126726)=-118.47661154416701; dx=-1.9149546439160945E-5 evalInputDelta=4.261757341180058
New Minimum: -118.47661154416701 > -138.97286868620452
END: th(1684339.7559414054)=-138.97286868620452; dx=-6.048755013027415E-6 evalInputDelta=24.758014483217565
Fitness changed from -114.21485420298696 to -138.97286868620452
Iteration 2 complete. Error: -138.97286868620452 Total: 29.2140; Orientation: 0.0533; Line Search: 26.1824
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1a85a260 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-138.97286868620452;dx=-1.0441095535073618E-5
New Minimum: -138.97286868620452 > -157.63834064603776
END: th(3628800.0000000005)=-157.63834064603776; dx=-1.0844954255351825E-6 evalInputDelta=18.66547195983324
Fitness changed from -138.97286868620452 to -157.63834064603776
Iteration 3 complete. Error: -157.63834064603776 Total: 8.0228; Orientation: 0.0587; Line Search: 4.9441
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement fadb431 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-157.63834064603776;dx=-1.0248279274458398E-5
Armijo: th(7818012.603187701)=-156.81069976781163; dx=2.9923716946067767E-6 evalInputDelta=-0.8276408782261342
New Minimum: -157.63834064603776 > -169.4175105030541
WOLF (strong): th(3909006.3015938504)=-169.4175105030541; dx=1.5316235399832789E-6 evalInputDelta=11.779169857016342
END: th(1303002.1005312835)=-167.92689850874655; dx=-4.181703072603996E-6 evalInputDelta=10.288557862708785
Fitness changed from -157.63834064603776 to -169.4175105030541
Iteration 4 complete. Error: -169.4175105030541 Total: 13.3909; Orientation: 0.0602; Line Search: 10.1837
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5eb820ba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.048e+04, gradient 3.312e-03, dot -0.520; [4df84128-9884-4833-a670-d2c01d405c99 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -169.4175105030541, -157.63834064603776, -138.97286868620452, -114.21485420298696
LBFGS Accumulation History: 3 points
Removed measurement 5eb820ba to history. Total: 3
th(0)=-169.4175105030541;dx=-1.0964461781232594E-5
New Minimum: -169.4175105030541 > -184.74883405597134
WOLF (strong): th(2807232.926569009)=-184.74883405597134; dx=4.055966331139699E-7 evalInputDelta=15.33132355291724
END: th(1403616.4632845046)=-181.9062557998672; dx=-4.728528507537427E-6 evalInputDelta=12.488745296813107
Fitness changed from -169.4175105030541 to -184.74883405597134
Iteration 5 complete. Error: -184.74883405597134 Total: 11.8978; Orientation: 1.0566; Line Search: 7.6319
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 50d19d55 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.979e+04, gradient 2.593e-03, dot -0.535; [4df84128-9884-4833-a670-d2c01d405c99 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -184.74883405597134, -157.63834064603776, -138.97286868620452, -114.21485420298696
LBFGS Accumulation History: 3 points
Removed measurement 50d19d55 to history. Total: 3
th(0)=-184.74883405597134;dx=-6.722859617473393E-6
New Minimum: -184.74883405597134 > -194.42845991801906
WOLF (strong): th(3024000.0000000005)=-194.42845991801906; dx=1.5264479862877499E-7 evalInputDelta=9.679625862047715
END: th(1512000.0000000002)=-192.65667613162293; dx=-2.8291418345455575E-6 evalInputDelta=7.907842075651587
Fitness changed from -184.74883405597134 to -194.42845991801906
Iteration 6 complete. Error: -194.42845991801906 Total: 11.9135; Orientation: 1.0581; Line Search: 7.8414
Adding measurement 300f684c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.517e+04, gradient 2.486e-03, dot -0.539; [4df84128-9884-4833-a670-d2c01d405c99 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -194.42845991801906, -157.63834064603776, -138.97286868620452, -114.21485420298696
LBFGS Accumulation History: 3 points
Removed measurement 300f684c to history. Total: 3
th(0)=-194.42845991801906;dx=-6.178648581012216E-6
New Minimum: -194.42845991801906 > -201.93473646012544
WOLF (strong): th(3257505.251328209)=-201.93473646012544; dx=1.0845373016130368E-6 evalInputDelta=7.506276542106377
END: th(1628752.6256641045)=-201.70693333037252; dx=-2.006973537529397E-6 evalInputDelta=7.278473412353463
Fitness changed from -194.42845991801906 to -201.93473646012544
Iteration 7 complete. Error: -201.93473646012544 Total: 11.5082; Orientation: 1.0609; Line Search: 7.8710
Adding measurement 47419e24 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.126e+04, gradient 2.617e-03, dot -0.471; [4df84128-9884-4833-a670-d2c01d405c99 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -201.93473646012544, -157.63834064603776, -138.97286868620452, -114.21485420298696
LBFGS Accumulation History: 3 points
Removed measurement 47419e24 to history. Total: 3
th(0)=-201.93473646012544;dx=-6.838213125613704E-6
New Minimum: -201.93473646012544 > -208.1415432903231
WOLF (strong): th(3509041.158211262)=-208.1415432903231; dx=2.3342761376350553E-6 evalInputDelta=6.206806830197678
New Minimum: -208.1415432903231 > -209.90155560990368
END: th(1754520.579105631)=-209.90155560990368; dx=-1.4372712410258777E-6 evalInputDelta=7.966819149778246
Fitness changed from -201.93473646012544 to -209.90155560990368
Iteration 8 complete. Error: -209.90155560990368 Total: 11.2802; Orientation: 1.0461; Line Search: 7.8237
Adding measurement 492bd80c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.763e+04, gradient 1.697e-03, dot -0.688; [4df84128-9884-4833-a670-d2c01d405c99 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -209.90155560990368, -157.63834064603776, -138.97286868620452, -114.21485420298696
LBFGS Accumulation History: 3 points
Removed measurement 492bd80c to history. Total: 3
th(0)=-209.90155560990368;dx=-2.8669112340508765E-6
New Minimum: -209.90155560990368 > -217.06630733643493
END: th(3780000.0000000014)=-217.06630733643493; dx=-9.514878795965254E-7 evalInputDelta=7.16475172653125
Fitness changed from -209.90155560990368 to -217.06630733643493
Iteration 9 complete. Error: -217.06630733643493 Total: 8.5820; Orientation: 0.9713; Line Search: 5.0785
Adding measurement 1cd70235 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.908e+04, gradient 1.997e-03, dot -0.548; [4df84128-9884-4833-a670-d2c01d405c99 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -217.06630733643493, -157.63834064603776, -138.97286868620452, -114.21485420298696
LBFGS Accumulation History: 3 points
Removed measurement 1cd70235 to history. Total: 3
th(0)=-217.06630733643493;dx=-3.975805067826428E-6
Armijo: th(8143763.128320524)=-213.47929581503527; dx=1.8526403708833283E-6 evalInputDelta=-3.58701152139966
New Minimum: -217.06630733643493 > -221.16639216288678
WOLF (strong): th(4071881.564160262)=-221.16639216288678; dx=7.915335935740324E-7 evalInputDelta=4.100084826451848
END: th(1357293.8547200873)=-221.14215275745536; dx=-1.6464286091181263E-6 evalInputDelta=4.075845421020432
Fitness changed from -217.06630733643493 to -221.16639216288678
Iteration 10 complete. Error: -221.16639216288678 Total: 14.1077; Orientation: 1.0553; Line Search: 10.5847
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -221.16639216288678 (> -Infinity) after 133.490s (< 5400.000s)
Returns
-221.16639216288678