BasicOptimizer.scala:89 executed in 230.57 seconds (2.596 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1969020787387
Reset training subject: 1974233544029
Adding measurement f4e508b to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.376030653715134;dx=-1.9442516959312873E-6
New Minimum: -15.376030653715134 > -15.376034587621689
END: th(2.154434690031884)=-15.376034587621689; dx=-1.6583991052391133E-6 evalInputDelta=3.933906555175781E-6
Fitness changed from -15.376030653715134 to -15.376034587621689
Iteration 1 complete. Error: -15.376034587621689 Total: 25.8000; Orientation: 0.0278; Line Search: 10.6536
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 227a95b9 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-15.376034587621689;dx=-1.6441047415680463E-6
New Minimum: -15.376034587621689 > -15.376044034957886
WOLFE (weak): th(4.641588833612779)=-15.376044034957886; dx=-1.6438730261730808E-6 evalInputDelta=9.447336196899414E-6
New Minimum: -15.376044034957886 > -15.376052588224411
WOLFE (weak): th(9.283177667225559)=-15.376052588224411; dx=-1.6433337041651582E-6 evalInputDelta=1.800060272216797E-5
New Minimum: -15.376052588224411 > -15.376089990139008
WOLFE (weak): th(27.849533001676676)=-15.376089990139008; dx=-1.6424131976768361E-6 evalInputDelta=5.5402517318725586E-5
New Minimum: -15.376089990139008 > -15.376256108283997
WOLFE (weak): th(111.3981320067067)=-15.376256108283997; dx=-1.641663561156871E-6 evalInputDelta=2.2152066230773926E-4
New Minimum: -15.376256108283997 > -15.377141922712326
WOLFE (weak): th(556.9906600335335)=-15.377141922712326; dx=-1.641160372082982E-6 evalInputDelta=0.001107335090637207
New Minimum: -15.377141922712326 > -15.382670044898987
WOLFE (weak): th(3341.9439602012008)=-15.382670044898987; dx=-1.6374075011002309E-6 evalInputDelta=0.006635457277297974
New Minimum: -15.382670044898987 > -15.422057569026947
WOLFE (weak): th(23393.607721408407)=-15.422057569026947; dx=-1.6062907461154064E-6 evalInputDelta=0.04602298140525818
New Minimum: -15.422057569026947 > -15.71341273188591
END: th(187148.86177126726)=-15.71341273188591; dx=-1.3276442365654514E-6 evalInputDelta=0.3373781442642212
Fitness changed from -15.376034587621689 to -15.71341273188591
Iteration 2 complete. Error: -15.71341273188591 Total: 50.2513; Orientation: 0.0280; Line Search: 44.9317
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 13655793 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-15.71341273188591;dx=-1.2261873163364884E-6
New Minimum: -15.71341273188591 > -16.217809796333313
END: th(403200.00000000006)=-16.217809796333313; dx=-8.713555080663552E-7 evalInputDelta=0.504397064447403
Fitness changed from -15.71341273188591 to -16.217809796333313
Iteration 3 complete. Error: -16.217809796333313 Total: 16.4254; Orientation: 0.0281; Line Search: 10.9412
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 57d1eea7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.636e+02, gradient 8.887e-04, dot -0.866; [bd707490-5d28-44af-aebf-53be9f4cb6b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.217809796333313, -15.71341273188591, -15.376034587621689, -15.376030653715134
LBFGS Accumulation History: 3 points
Removed measurement 57d1eea7 to history. Total: 3
th(0)=-16.217809796333313;dx=-7.884500644954763E-7
New Minimum: -16.217809796333313 > -16.890207529067993
END: th(868668.0670208557)=-16.890207529067993; dx=-5.092786963258764E-7 evalInputDelta=0.6723977327346802
Fitness changed from -16.217809796333313 to -16.890207529067993
Iteration 4 complete. Error: -16.890207529067993 Total: 15.7977; Orientation: 0.5736; Line Search: 9.8884
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 415b5563 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.862e+02, gradient 7.132e-04, dot -0.775; [bd707490-5d28-44af-aebf-53be9f4cb6b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.890207529067993, -15.71341273188591, -15.376034587621689, -15.376030653715134
LBFGS Accumulation History: 3 points
Removed measurement 415b5563 to history. Total: 3
th(0)=-16.890207529067993;dx=-5.078499681683397E-7
New Minimum: -16.890207529067993 > -17.68804082274437
END: th(1871488.617712673)=-17.68804082274437; dx=-2.467601086357385E-7 evalInputDelta=0.7978332936763763
Fitness changed from -16.890207529067993 to -17.68804082274437
Iteration 5 complete. Error: -17.68804082274437 Total: 16.4509; Orientation: 0.5914; Line Search: 10.5171
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 734ce621 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.302e+03, gradient 6.338e-04, dot -0.579; [bd707490-5d28-44af-aebf-53be9f4cb6b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.68804082274437, -15.71341273188591, -15.376034587621689, -15.376030653715134
LBFGS Accumulation History: 3 points
Removed measurement 734ce621 to history. Total: 3
th(0)=-17.68804082274437;dx=-4.012320430917284E-7
New Minimum: -17.68804082274437 > -18.21531030535698
WOLF (strong): th(4032000.0000000014)=-18.21531030535698; dx=2.127977764181902E-8 evalInputDelta=0.5272694826126099
END: th(2016000.0000000007)=-18.18056669831276; dx=-8.475927183523599E-8 evalInputDelta=0.4925258755683899
Fitness changed from -17.68804082274437 to -18.21531030535698
Iteration 6 complete. Error: -18.21531030535698 Total: 21.3771; Orientation: 0.5901; Line Search: 15.5644
Adding measurement 251a64f6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.747e+02, gradient 9.320e-04, dot -0.136; [bd707490-5d28-44af-aebf-53be9f4cb6b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.21531030535698, -15.71341273188591, -15.376034587621689, -15.376030653715134
LBFGS Accumulation History: 3 points
Removed measurement 251a64f6 to history. Total: 3
th(0)=-18.21531030535698;dx=-8.678618152997582E-7
Armijo: th(4343340.335104279)=-17.419823169708252; dx=4.616467185836106E-7 evalInputDelta=-0.7954871356487274
New Minimum: -18.21531030535698 > -18.613044679164886
WOLF (strong): th(2171670.1675521396)=-18.613044679164886; dx=2.6641975392420874E-7 evalInputDelta=0.3977343738079071
New Minimum: -18.613044679164886 > -18.68392026424408
END: th(723890.0558507132)=-18.68392026424408; dx=-2.52725981547899E-7 evalInputDelta=0.4686099588871002
Fitness changed from -18.21531030535698 to -18.68392026424408
Iteration 7 complete. Error: -18.68392026424408 Total: 25.8214; Orientation: 0.5963; Line Search: 20.1751
Adding measurement 11666cbc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.400e+03, gradient 5.199e-04, dot -0.423; [bd707490-5d28-44af-aebf-53be9f4cb6b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.68392026424408, -15.71341273188591, -15.376034587621689, -15.376030653715134
LBFGS Accumulation History: 3 points
Removed measurement 11666cbc to history. Total: 3
th(0)=-18.68392026424408;dx=-2.697307081637488E-7
New Minimum: -18.68392026424408 > -19.02966558933258
END: th(1559573.8480938943)=-19.02966558933258; dx=-1.0958606070055357E-7 evalInputDelta=0.345745325088501
Fitness changed from -18.68392026424408 to -19.02966558933258
Iteration 8 complete. Error: -19.02966558933258 Total: 16.2285; Orientation: 0.6364; Line Search: 10.3618
Adding measurement 3d97374d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.800e+03, gradient 4.439e-04, dot -0.525; [bd707490-5d28-44af-aebf-53be9f4cb6b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.02966558933258, -15.71341273188591, -15.376034587621689, -15.376030653715134
LBFGS Accumulation History: 3 points
Removed measurement 3d97374d to history. Total: 3
th(0)=-19.02966558933258;dx=-1.9675310500567118E-7
New Minimum: -19.02966558933258 > -19.322887063026428
END: th(3360000.0000000014)=-19.322887063026428; dx=-3.4456321335603057E-9 evalInputDelta=0.29322147369384766
Fitness changed from -19.02966558933258 to -19.322887063026428
Iteration 9 complete. Error: -19.322887063026428 Total: 15.7656; Orientation: 0.5928; Line Search: 10.0765
Adding measurement 3ca7de99 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.629e+03, gradient 5.956e-04, dot -0.297; [bd707490-5d28-44af-aebf-53be9f4cb6b4 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.322887063026428, -15.71341273188591, -15.376034587621689, -15.376030653715134
LBFGS Accumulation History: 3 points
Removed measurement 3ca7de99 to history. Total: 3
th(0)=-19.322887063026428;dx=-3.54169958810221E-7
Armijo: th(7238900.558507132)=-18.246912598609924; dx=2.1008111515855417E-7 evalInputDelta=-1.075974464416504
Armijo: th(3619450.279253566)=-19.18180823326111; dx=1.9099330936524453E-7 evalInputDelta=-0.14107882976531982
New Minimum: -19.322887063026428 > -19.561588943004608
END: th(1206483.4264178553)=-19.561588943004608; dx=-1.7365770845266603E-8 evalInputDelta=0.23870187997817993
Fitness changed from -19.322887063026428 to -19.561588943004608
Iteration 10 complete. Error: -19.561588943004608 Total: 26.3812; Orientation: 0.6100; Line Search: 20.3210
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -19.561588943004608 (> -Infinity) after 230.564s (< 5400.000s)
Returns
-19.561588943004608