Code from BasicOptimizer.scala:88 executed in 372.27 seconds (5.039 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 5391152998690
Reset training subject: 5394492343653
Adding measurement 316c7fbc to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=1.5968517661094666;dx=-1.4159597723694494E-7
New Minimum: 1.5968517661094666 > 1.596851333975792
WOLFE (weak): th(2.154434690031884)=1.596851333975792; dx=-1.4083880058962035E-7 evalInputDelta=4.3213367462158203E-7
New Minimum: 1.596851333975792 > 1.5968507826328278
WOLFE (weak): th(4.308869380063768)=1.5968507826328278; dx=-1.408426663320406E-7 evalInputDelta=9.834766387939453E-7
New Minimum: 1.5968507826328278 > 1.5968486815690994
WOLFE (weak): th(12.926608140191302)=1.5968486815690994; dx=-1.4084626134633458E-7 evalInputDelta=3.084540367126465E-6
New Minimum: 1.5968486815690994 > 1.5968392193317413
WOLFE (weak): th(51.70643256076521)=1.5968392193317413; dx=-1.4086703560844474E-7 evalInputDelta=1.2546777725219727E-5
New Minimum: 1.5968392193317413 > 1.596788465976715
WOLFE (weak): th(258.53216280382605)=1.596788465976715; dx=-1.4069578991417172E-7 evalInputDelta=6.330013275146484E-5
New Minimum: 1.596788465976715 > 1.5964718163013458
WOLFE (weak): th(1551.1929768229563)=1.5964718163013458; dx=-1.4083286087427027E-7 evalInputDelta=3.7994980812072754E-4
New Minimum: 1.5964718163013458 > 1.5941919833421707
WOLFE (weak): th(10858.350837760694)=1.5941919833421707; dx=-1.4076668988460676E-7 evalInputDelta=0.0026597827672958374
New Minimum: 1.5941919833421707 > 1.5756125450134277
WOLFE (weak): th(86866.80670208555)=1.5756125450134277; dx=-1.4020881029037426E-7 evalInputDelta=0.02123922109603882
New Minimum: 1.5756125450134277 > 1.410905361175537
WOLFE (weak): th(781801.26031877)=1.410905361175537; dx=-1.3251880299198042E-7 evalInputDelta=0.18594640493392944
New Minimum: 1.410905361175537 > 0.3773341476917267
END: th(7818012.6031877)=0.3773341476917267; dx=-5.740086134683483E-8 evalInputDelta=1.2195176184177399
Fitness changed from 1.5968517661094666 to 0.3773341476917267
Iteration 1 complete. Error: 0.377334
...skipping 25349 bytes...
3
Rejected: LBFGS Orientation magnitude: 2.585e+03, gradient 3.437e-05, dot -0.527; [ccdaeaeb-0178-4aa3-83e7-dc66157433f4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.6963167861104012, -0.42782942950725555, 0.3773341476917267, 1.5968517661094666
LBFGS Accumulation History: 3 points
Removed measurement 3970bacd to history. Total: 3
th(0)=-1.6963167861104012;dx=-1.1793556799949465E-9
Armijo: th(1.7409937540342882E7)=-1.6888352110981941; dx=2.6385202851011373E-10 evalInputDelta=-0.007481575012207031
New Minimum: -1.6963167861104012 > -1.6964056119322777
WOLF (strong): th(8704968.770171441)=-1.6964056119322777; dx=3.0280405571300084E-10 evalInputDelta=8.882582187652588E-5
New Minimum: -1.6964056119322777 > -1.6996904164552689
END: th(2901656.2567238137)=-1.6996904164552689; dx=-3.515468595781091E-10 evalInputDelta=0.0033736303448677063
Fitness changed from -1.6963167861104012 to -1.6996904164552689
Iteration 29 complete. Error: -1.6996904164552689 Total: 14.0894; Orientation: 0.3279; Line Search: 11.0306
Adding measurement 5df1e30f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.522e+03, gradient 2.660e-05, dot -0.622; [ccdaeaeb-0178-4aa3-83e7-dc66157433f4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.6996904164552689, -0.42782942950725555, 0.3773341476917267, 1.5968517661094666
LBFGS Accumulation History: 3 points
Removed measurement 5df1e30f to history. Total: 3
th(0)=-1.6996904164552689;dx=-7.003417895161655E-10
New Minimum: -1.6996904164552689 > -1.7027868032455444
END: th(6251428.898033846)=-1.7027868032455444; dx=-2.5324089235534497E-10 evalInputDelta=0.0030963867902755737
Fitness changed from -1.6996904164552689 to -1.7027868032455444
Iteration 30 complete. Error: -1.7027868032455444 Total: 8.6798; Orientation: 0.3668; Line Search: 5.3924
<a id="p-21"></a>Iteration 30
<a id="p-20"></a>
Final threshold in iteration 31: -1.7027868032455444 (> -Infinity) after 372.266s (< 3600.000s)
-1.7027868032455444