Code from BasicOptimizer.scala:75 executed in 3753.29 seconds (40.826 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1187318733744200
Reset training subject: 1187372703958200
Adding measurement a09fe7e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=1.267790514197229;dx=-2.638604003746722E-7
New Minimum: 1.267790514197229 > 1.2677903208667143
WOLFE (weak): th(2.154434690031884)=1.2677903208667143; dx=-2.6036998361808776E-7 evalInputDelta=1.933305147083786E-7
New Minimum: 1.2677903208667143 > 1.267790071689169
WOLFE (weak): th(4.308869380063768)=1.267790071689169; dx=-2.6036996515694655E-7 evalInputDelta=4.4250805997236853E-7
New Minimum: 1.267790071689169 > 1.2677894000509315
WOLFE (weak): th(12.926608140191302)=1.2677894000509315; dx=-2.603697747962987E-7 evalInputDelta=1.1141462974428862E-6
New Minimum: 1.2677894000509315 > 1.2677859197605306
WOLFE (weak): th(51.70643256076521)=1.2677859197605306; dx=-2.603703256573426E-7 evalInputDelta=4.594436698424076E-6
New Minimum: 1.2677859197605306 > 1.2677675827744221
WOLFE (weak): th(258.53216280382605)=1.2677675827744221; dx=-2.6036501223353694E-7 evalInputDelta=2.29314228068489E-5
New Minimum: 1.2677675827744221 > 1.2676526775688264
WOLFE (weak): th(1551.1929768229563)=1.2676526775688264; dx=-2.603413338379829E-7 evalInputDelta=1.3783662840261002E-4
New Minimum: 1.2676526775688264 > 1.2668257780936842
WOLFE (weak): th(10858.350837760694)=1.2668257780936842; dx=-2.6016495756247186E-7 evalInputDelta=9.647361035447499E-4
New Minimum: 1.2668257780936842 > 1.2601102100841732
WOLFE (weak): th(86866.80670208555)=1.2601102100841732; dx=-2.5869393427124803E-7 evalInputDelta=0.007680304113055758
New Minimum: 1.2601102100841732 > 1.2020853565648304
WOLFE (weak): th(781801.26031877)=1.2020853565648304; dx=-2.4474495218901595E-7 evalInputDelta=0.06570515763239859
New Minimum: 1.2020853565648304 > 0.890924745920601
END: th(7818012.6031877)=0.890924745920601; dx=-1.3306970027760307E-7 evalInputDelta=0.376865768276628
Fitness changed from 1.267790514197229 to 0.890924745920601
Iteration
...skipping 10990 bytes...
easurement 21d1f97f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.067e+04, gradient 1.400e-04, dot -0.744; [720c5a8e-10c8-4a28-b228-7616b7dd1bc0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.2045452964156549, 0.16146093700171527, 0.1263330167051701, 0.10288565754773729
LBFGS Accumulation History: 3 points
Removed measurement 68812a5c to history. Total: 4
th(0)=0.10288565754773729;dx=-1.9603630638727868E-8
Armijo: th(5.482876809705099E7)=0.24940199641503233; dx=-2.54632713845967E-9 evalInputDelta=-0.14651633886729504
New Minimum: 0.10288565754773729 > 0.08792990649217541
END: th(2.7414384048525494E7)=0.08792990649217541; dx=-7.2013616416018434E-9 evalInputDelta=0.014955751055561878
Fitness changed from 0.10288565754773729 to 0.08792990649217541
Iteration 14 complete. Error: 0.08792990649217541 Total: 219.0442; Orientation: 1.5082; Line Search: 164.1269
Adding measurement 8d902f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.467e+03, gradient 1.414e-04, dot -0.698; [720c5a8e-10c8-4a28-b228-7616b7dd1bc0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.16146093700171527, 0.1263330167051701, 0.10288565754773729, 0.08792990649217541
LBFGS Accumulation History: 3 points
Removed measurement 3657201c to history. Total: 4
th(0)=0.08792990649217541;dx=-1.9995528701692146E-8
Armijo: th(5.9062500000000045E7)=0.30297197127655134; dx=-2.423468491627841E-10 evalInputDelta=-0.21504206478437593
New Minimum: 0.08792990649217541 > 0.0875170307479206
END: th(2.9531250000000022E7)=0.0875170307479206; dx=-4.859166247289371E-9 evalInputDelta=4.1287574425481754E-4
Fitness changed from 0.08792990649217541 to 0.0875170307479206
Iteration 15 complete. Error: 0.0875170307479206 Total: 213.9987; Orientation: 1.4830; Line Search: 159.3337
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 15: 0.0875170307479206 (> -Infinity) after 3753.284s (< 3600.000s)
0.0875170307479206