BasicOptimizer.scala:89 executed in 94.10 seconds (1.198 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 285593406784900
Reset training subject: 285595355999200
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -55.86984871927633 > -55.91411836921493
th(0)=-55.91411836921493;dx=-4.894626423626967E-6
New Minimum: -55.91411836921493 > -55.91412507306619
END: th(2.154434690031884)=-55.91412507306619; dx=-3.5388422177369547E-6 evalInputDelta=6.703851255451809E-6
Fitness changed from -55.86984871927633 to -55.91412507306619
Iteration 1 complete. Error: -55.91412507306619 Total: 9.6515; Orientation: 0.0358; Line Search: 3.6709
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 273ab805 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-55.91412507306619;dx=-3.5463239264718905E-6
New Minimum: -55.91412507306619 > -55.91414015672859
WOLFE (weak): th(4.641588833612779)=-55.91414015672859; dx=-3.546295771521509E-6 evalInputDelta=1.5083662404435927E-5
New Minimum: -55.91414015672859 > -55.91415691634871
WOLFE (weak): th(9.283177667225559)=-55.91415691634871; dx=-3.5462943602171396E-6 evalInputDelta=3.184328252103796E-5
New Minimum: -55.91415691634871 > -55.91422227881916
WOLFE (weak): th(27.849533001676676)=-55.91422227881916; dx=-3.5462821679253645E-6 evalInputDelta=9.720575297222922E-5
New Minimum: -55.91422227881916 > -55.9145189229171
WOLFE (weak): th(111.3981320067067)=-55.9145189229171; dx=-3.5462696628555635E-6 evalInputDelta=3.93849850908623E-4
New Minimum: -55.9145189229171 > -55.91610099819392
WOLFE (weak): th(556.9906600335335)=-55.91610099819392; dx=-3.5460540679732495E-6 evalInputDelta=0.0019759251277307044
New Minimum: -55.91610099819392 > -55.925974549836475
WOLFE (weak): th(3341.9439602012008)=-55.925974549836475; dx=-3.5447693268259218E-6 evalInputDelta=0.011849476770287026
New Minimum: -55.925974549836475 > -55.996980263798854
WOLFE (weak): th(23393.607721408407)=-55.996980263798854; dx=-3.5378925296648734E-6 evalInputDelta=0.08285519073266556
New Minimum: -55.996980263798854 > -56.56980548242515
WOLFE (weak): th(187148.86177126726)=-56.56980548242515; dx=-3.4570289502377447E-6 evalInputDelta=0.6556804093589648
New Minimum: -56.56980548242515 > -61.19742602902078
END: th(1684339.7559414054)=-61.19742602902078; dx=-2.7230796077669515E-6 evalInputDelta=5.283300955954594
Fitness changed from -55.91412507306619 to -61.19742602902078
Iteration 2 complete. Error: -61.19742602902078 Total: 21.4860; Orientation: 0.0578; Line Search: 19.2728
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7565921f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-61.19742602902078;dx=-2.4398367524273597E-6
New Minimum: -61.19742602902078 > -68.35776563102938
END: th(3628800.0000000005)=-68.35776563102938; dx=-1.5145113678921801E-6 evalInputDelta=7.160339602008598
Fitness changed from -61.19742602902078 to -68.35776563102938
Iteration 3 complete. Error: -68.35776563102938 Total: 5.9938; Orientation: 0.0580; Line Search: 3.7784
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5b4fe0f7 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-68.35776563102938;dx=-1.5085990460504768E-6
New Minimum: -68.35776563102938 > -76.94194044604643
END: th(7818012.603187701)=-76.94194044604643; dx=-7.449696970439319E-7 evalInputDelta=8.58417481501705
Fitness changed from -68.35776563102938 to -76.94194044604643
Iteration 4 complete. Error: -76.94194044604643 Total: 6.1146; Orientation: 0.0582; Line Search: 3.7151
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6ec07ee6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.392e+04, gradient 1.110e-03, dot -0.711; [79922f99-ce4e-4d63-a468-40111972ea96 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -76.94194044604643, -68.35776563102938, -61.19742602902078, -55.91412507306619
LBFGS Accumulation History: 3 points
Removed measurement 6ec07ee6 to history. Total: 3
th(0)=-76.94194044604643;dx=-1.2317176995140164E-6
New Minimum: -76.94194044604643 > -84.25421682699178
END: th(1.6843397559414055E7)=-84.25421682699178; dx=-5.514578614832759E-8 evalInputDelta=7.312276380945349
Fitness changed from -76.94194044604643 to -84.25421682699178
Iteration 5 complete. Error: -84.25421682699178 Total: 7.1751; Orientation: 1.0256; Line Search: 3.9589
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement d3981b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.018e+04, gradient 1.433e-03, dot -0.439; [79922f99-ce4e-4d63-a468-40111972ea96 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.25421682699178, -68.35776563102938, -61.19742602902078, -55.91412507306619
LBFGS Accumulation History: 3 points
Removed measurement d3981b0 to history. Total: 3
th(0)=-84.25421682699178;dx=-2.0525203447981637E-6
Armijo: th(3.628800000000001E7)=-76.40092111005319; dx=4.4646550162827995E-7 evalInputDelta=-7.8532957169385895
New Minimum: -84.25421682699178 > -86.86259253368927
WOLF (strong): th(1.8144000000000004E7)=-86.86259253368927; dx=7.001694858786125E-7 evalInputDelta=2.608375706697487
New Minimum: -86.86259253368927 > -92.02408925144785
END: th(6048000.000000001)=-92.02408925144785; dx=-4.1527218094088494E-7 evalInputDelta=7.769872424456068
Fitness changed from -84.25421682699178 to -92.02408925144785
Iteration 6 complete. Error: -92.02408925144785 Total: 10.7573; Orientation: 1.0140; Line Search: 7.6499
Adding measurement 44ff3821 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.792e+04, gradient 9.941e-04, dot -0.595; [79922f99-ce4e-4d63-a468-40111972ea96 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.02408925144785, -68.35776563102938, -61.19742602902078, -55.91412507306619
LBFGS Accumulation History: 3 points
Removed measurement 44ff3821 to history. Total: 3
th(0)=-92.02408925144785;dx=-9.882621479037479E-7
New Minimum: -92.02408925144785 > -97.00290289659591
WOLF (strong): th(1.3030021005312836E7)=-97.00290289659591; dx=3.979342735024191E-8 evalInputDelta=4.978813645148065
END: th(6515010.502656418)=-96.24526707038825; dx=-3.26721763790438E-7 evalInputDelta=4.221177818940404
Fitness changed from -92.02408925144785 to -97.00290289659591
Iteration 7 complete. Error: -97.00290289659591 Total: 8.4080; Orientation: 0.9954; Line Search: 5.5750
Adding measurement 640a2835 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.918e+04, gradient 1.242e-03, dot -0.437; [79922f99-ce4e-4d63-a468-40111972ea96 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -97.00290289659591, -68.35776563102938, -61.19742602902078, -55.91412507306619
LBFGS Accumulation History: 3 points
Removed measurement 640a2835 to history. Total: 3
th(0)=-97.00290289659591;dx=-1.543556155526575E-6
New Minimum: -97.00290289659591 > -99.69873249170624
WOLF (strong): th(1.4036164632845048E7)=-99.69873249170624; dx=6.248642614462542E-7 evalInputDelta=2.6958295951103253
New Minimum: -99.69873249170624 > -102.60682879359831
WOLF (strong): th(7018082.316422524)=-102.60682879359831; dx=6.1591873057745885E-9 evalInputDelta=5.603925897002398
END: th(2339360.7721408415)=-100.11539823102243; dx=-1.0946726994472982E-6 evalInputDelta=3.1124953344265123
Fitness changed from -97.00290289659591 to -102.60682879359831
Iteration 8 complete. Error: -102.60682879359831 Total: 10.6718; Orientation: 0.9439; Line Search: 7.5944
Adding measurement 1ca7841e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.040e+04, gradient 9.258e-04, dot -0.566; [79922f99-ce4e-4d63-a468-40111972ea96 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -102.60682879359831, -68.35776563102938, -61.19742602902078, -55.91412507306619
LBFGS Accumulation History: 3 points
Removed measurement 1ca7841e to history. Total: 3
th(0)=-102.60682879359831;dx=-8.57164918910357E-7
New Minimum: -102.60682879359831 > -105.55058353452894
END: th(5040000.000000002)=-105.55058353452894; dx=-3.094513893587453E-7 evalInputDelta=2.943754740930629
Fitness changed from -102.60682879359831 to -105.55058353452894
Iteration 9 complete. Error: -105.55058353452894 Total: 6.5730; Orientation: 0.9680; Line Search: 3.7419
Adding measurement 5a232c0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.429e+04, gradient 7.410e-04, dot -0.681; [79922f99-ce4e-4d63-a468-40111972ea96 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -105.55058353452894, -68.35776563102938, -61.19742602902078, -55.91412507306619
LBFGS Accumulation History: 3 points
Removed measurement 5a232c0b to history. Total: 3
th(0)=-105.55058353452894;dx=-5.490996882783998E-7
New Minimum: -105.55058353452894 > -109.3298824432993
END: th(1.0858350837760698E7)=-109.3298824432993; dx=-1.9556231569930137E-7 evalInputDelta=3.7792989087703575
Fitness changed from -105.55058353452894 to -109.3298824432993
Iteration 10 complete. Error: -109.3298824432993 Total: 6.9137; Orientation: 1.0219; Line Search: 4.0088
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -109.3298824432993 (> -Infinity) after 94.097s (< 5400.000s)
Returns
-109.3298824432993