BasicOptimizer.scala:89 executed in 2191.92 seconds (10.623 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 22347290228766
Reset training subject: 22396985378899
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
th(0)=-2.9753184616565704;dx=-2.5909687306337265E-7
New Minimum: -2.9753184616565704 > -2.975319027900696
WOLFE (weak): th(2.154434690031884)=-2.975319027900696; dx=-2.394490126228027E-7 evalInputDelta=5.662441253662109E-7
New Minimum: -2.975319027900696 > -2.9753195345401764
WOLFE (weak): th(4.308869380063768)=-2.9753195345401764; dx=-2.381970402115179E-7 evalInputDelta=1.0728836059570312E-6
New Minimum: -2.9753195345401764 > -2.9753219187259674
WOLFE (weak): th(12.926608140191302)=-2.9753219187259674; dx=-2.381896922902587E-7 evalInputDelta=3.4570693969726562E-6
New Minimum: -2.9753219187259674 > -2.9753326177597046
WOLFE (weak): th(51.70643256076521)=-2.9753326177597046; dx=-2.3819362178619898E-7 evalInputDelta=1.4156103134155273E-5
New Minimum: -2.9753326177597046 > -2.9753882586956024
WOLFE (weak): th(258.53216280382605)=-2.9753882586956024; dx=-2.3898581474502404E-7 evalInputDelta=6.979703903198242E-5
New Minimum: -2.9753882586956024 > -2.9757361710071564
WOLFE (weak): th(1551.1929768229563)=-2.9757361710071564; dx=-2.3815791898787404E-7 evalInputDelta=4.177093505859375E-4
New Minimum: -2.9757361710071564 > -2.9782398641109467
WOLFE (weak): th(10858.350837760694)=-2.9782398641109467; dx=-2.387273741932265E-7 evalInputDelta=0.0029214024543762207
New Minimum: -2.9782398641109467 > -2.9985977709293365
WOLFE (weak): th(86866.80670208555)=-2.9985977709293365; dx=-2.3725116075784248E-7 evalInputDelta=0.023279309272766113
New Minimum: -2.9985977709293365 > -3.175896853208542
END: th(781801.26031877)=-3.175896853208542; dx=-2.1482071634943303E-7 evalInputDelta=0.20057839155197144
Fitness changed from -2.9753184616565704 to -3.175896853208542
Iteration 1 complete. Error: -3.175896853208542 Total: 637.7271; Orientation: 0.0700; Line Search: 489.8458
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 56837c8e to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-3.175896853208542;dx=-1.9834345102941805E-7
New Minimum: -3.175896853208542 > -3.5185486972332
END: th(1684339.7559414052)=-3.5185486972332; dx=-1.5488333678545476E-7 evalInputDelta=0.3426518440246582
Fitness changed from -3.175896853208542 to -3.5185486972332
Iteration 2 complete. Error: -3.5185486972332 Total: 149.9710; Orientation: 0.0869; Line Search: 99.1903
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4b023ab0 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.5185486972332;dx=-1.4349449561634026E-7
New Minimum: -3.5185486972332 > -4.021243214607239
END: th(3628800.0)=-4.021243214607239; dx=-9.696475246438292E-8 evalInputDelta=0.5026945173740387
Fitness changed from -3.5185486972332 to -4.021243214607239
Iteration 3 complete. Error: -4.021243214607239 Total: 149.7003; Orientation: 0.0978; Line Search: 99.3113
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 63514626 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.021243214607239;dx=-8.139104783808898E-8
New Minimum: -4.021243214607239 > -4.6528109312057495
END: th(7818012.6031877)=-4.6528109312057495; dx=-5.3579688420417055E-8 evalInputDelta=0.6315677165985107
Fitness changed from -4.021243214607239 to -4.6528109312057495
Iteration 4 complete. Error: -4.6528109312057495 Total: 149.7971; Orientation: 0.0910; Line Search: 99.7972
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5d54aed1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.509e+03, gradient 2.247e-04, dot -0.904; [4612ccf7-77b1-4252-bd8a-4376c51dbe5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.6528109312057495, -4.021243214607239, -3.5185486972332, -3.175896853208542
LBFGS Accumulation History: 3 points
Removed measurement 5d54aed1 to history. Total: 3
th(0)=-4.6528109312057495;dx=-4.993215624720263E-8
New Minimum: -4.6528109312057495 > -5.357013955712318
END: th(1.684339755941405E7)=-5.357013955712318; dx=-2.7217146727574082E-8 evalInputDelta=0.7042030245065689
Fitness changed from -4.6528109312057495 to -5.357013955712318
Iteration 5 complete. Error: -5.357013955712318 Total: 151.6866; Orientation: 2.2322; Line Search: 98.9650
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4b529ca1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.909e+03, gradient 1.698e-04, dot -0.791; [4612ccf7-77b1-4252-bd8a-4376c51dbe5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.357013955712318, -4.021243214607239, -3.5185486972332, -3.175896853208542
LBFGS Accumulation History: 3 points
Removed measurement 4b529ca1 to history. Total: 3
th(0)=-5.357013955712318;dx=-2.8569054118621846E-8
New Minimum: -5.357013955712318 > -6.011703431606293
END: th(3.6288E7)=-6.011703431606293; dx=-9.157689387060258E-9 evalInputDelta=0.6546894758939743
Fitness changed from -5.357013955712318 to -6.011703431606293
Iteration 6 complete. Error: -6.011703431606293 Total: 150.7343; Orientation: 2.2998; Line Search: 98.1096
Adding measurement 44a9f326 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.425e+03, gradient 1.625e-04, dot -0.543; [4612ccf7-77b1-4252-bd8a-4376c51dbe5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.011703431606293, -4.021243214607239, -3.5185486972332, -3.175896853208542
LBFGS Accumulation History: 3 points
Removed measurement 44a9f326 to history. Total: 3
th(0)=-6.011703431606293;dx=-2.626990943843409E-8
New Minimum: -6.011703431606293 > -6.072715610265732
WOLF (strong): th(7.8180126031877E7)=-6.072715610265732; dx=2.7423608143623675E-9 evalInputDelta=0.06101217865943909
New Minimum: -6.072715610265732 > -6.142998158931732
END: th(3.90900630159385E7)=-6.142998158931732; dx=-3.9837305605339693E-11 evalInputDelta=0.13129472732543945
Fitness changed from -6.011703431606293 to -6.142998158931732
Iteration 7 complete. Error: -6.142998158931732 Total: 206.7004; Orientation: 3.5606; Line Search: 146.4255
Adding measurement 616d105d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.398e+03, gradient 2.921e-04, dot -0.380; [4612ccf7-77b1-4252-bd8a-4376c51dbe5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.142998158931732, -4.021243214607239, -3.5185486972332, -3.175896853208542
LBFGS Accumulation History: 3 points
Removed measurement 616d105d to history. Total: 3
th(0)=-6.142998158931732;dx=-7.434758976427364E-8
Armijo: th(8.421698779707026E7)=-4.812336087226868; dx=2.0011336597951482E-8 evalInputDelta=-1.3306620717048645
Armijo: th(4.210849389853513E7)=-5.928455024957657; dx=1.9205703303047452E-8 evalInputDelta=-0.21454313397407532
New Minimum: -6.142998158931732 > -6.420821696519852
WOLF (strong): th(1.4036164632845044E7)=-6.420821696519852; dx=5.926003391974886E-9 evalInputDelta=0.2778235375881195
END: th(3509041.158211261)=-6.363094598054886; dx=-3.083954355598782E-8 evalInputDelta=0.2200964391231537
Fitness changed from -6.142998158931732 to -6.420821696519852
Iteration 8 complete. Error: -6.420821696519852 Total: 296.5837; Orientation: 2.2374; Line Search: 245.3715
Adding measurement 57c67e42 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.645e+03, gradient 1.387e-04, dot -0.423; [4612ccf7-77b1-4252-bd8a-4376c51dbe5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.420821696519852, -4.021243214607239, -3.5185486972332, -3.175896853208542
LBFGS Accumulation History: 3 points
Removed measurement 57c67e42 to history. Total: 3
th(0)=-6.420821696519852;dx=-1.9228766976735457E-8
New Minimum: -6.420821696519852 > -6.537517100572586
END: th(7560000.000000001)=-6.537517100572586; dx=-8.632791769182786E-9 evalInputDelta=0.11669540405273438
Fitness changed from -6.420821696519852 to -6.537517100572586
Iteration 9 complete. Error: -6.537517100572586 Total: 148.8781; Orientation: 2.2040; Line Search: 97.4582
Adding measurement 42a8dad3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.122e+03, gradient 9.617e-05, dot -0.551; [4612ccf7-77b1-4252-bd8a-4376c51dbe5b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.537517100572586, -4.021243214607239, -3.5185486972332, -3.175896853208542
LBFGS Accumulation History: 3 points
Removed measurement 42a8dad3 to history. Total: 3
th(0)=-6.537517100572586;dx=-9.243210582812773E-9
New Minimum: -6.537517100572586 > -6.663195192813873
END: th(1.6287526256641043E7)=-6.663195192813873; dx=-5.40450455601556E-9 evalInputDelta=0.12567809224128723
Fitness changed from -6.537517100572586 to -6.663195192813873
Iteration 10 complete. Error: -6.663195192813873 Total: 148.8360; Orientation: 2.2129; Line Search: 97.7982
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -6.663195192813873 (> -Infinity) after 2191.917s (< 5400.000s)
Returns
-6.663195192813873