Code from BasicOptimizer.scala:75 executed in 294.36 seconds (3.715 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1762590294014
Reset training subject: 1763541837797
Adding measurement 5b81bb04 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=0.9305623571834871;dx=-9.94513782647717E-9
New Minimum: 0.9305623571834871 > 0.9305623398812042
WOLFE (weak): th(2.154434690031884)=0.9305623398812042; dx=-9.945137068325808E-9 evalInputDelta=1.7302282939724023E-8
New Minimum: 0.9305623398812042 > 0.930562322578923
WOLFE (weak): th(4.308869380063768)=0.930562322578923; dx=-9.945136310174448E-9 evalInputDelta=3.4604564103091207E-8
New Minimum: 0.930562322578923 > 0.9305622533698105
WOLFE (weak): th(12.926608140191302)=0.9305622533698105; dx=-9.945131028042428E-9 evalInputDelta=1.0381367665512897E-7
New Minimum: 0.9305622533698105 > 0.9305619419268306
WOLFE (weak): th(51.70643256076521)=0.9305619419268306; dx=-9.945155076424137E-9 evalInputDelta=4.1525665650521404E-7
New Minimum: 0.9305619419268306 > 0.9305602808190174
WOLFE (weak): th(258.53216280382605)=0.9305602808190174; dx=-9.945595213498527E-9 evalInputDelta=2.0763644696941697E-6
New Minimum: 0.9305602808190174 > 0.9305498971702386
WOLFE (weak): th(1551.1929768229563)=0.9305498971702386; dx=-9.947297592442254E-9 evalInputDelta=1.246001324850532E-5
New Minimum: 0.9305498971702386 > 0.9304750723934052
WOLFE (weak): th(10858.350837760694)=0.9304750723934052; dx=-9.956467810985013E-9 evalInputDelta=8.728479008190959E-5
New Minimum: 0.9304750723934052 > 0.9298591128921492
WOLFE (weak): th(86866.80670208555)=0.9298591128921492; dx=-1.0048276524245235E-8 evalInputDelta=7.032442913379455E-4
New Minimum: 0.9298591128921492 > 0.9238576365226459
WOLFE (weak): th(781801.26031877)=0.9238576365226459; dx=-1.0719328878248866E-8 evalInputDelta=0.006704720660841268
New Minimum: 0.9238576365226459 > 0.8502610342067672
WOLFE (weak): th(7818012.6031877)=0.8502610342067672; dx=-1.092857606335615E-8 evalInputDelta=0.08030132297671988
New Minimum: 0.8502610342067672 > 0.7541555366107379
WOLF (strong): th(8.599813863506469
...skipping 43816 bytes...
+03, gradient 2.166e-05, dot -0.822; [0ebad2b1-6e9d-489b-9eaa-5ea76a976592 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.3552111039173597, 0.35461922747671965, 0.3542191206590147, 0.3539523527689336
LBFGS Accumulation History: 3 points
Removed measurement 7aa2ca31 to history. Total: 4
th(0)=0.3539523527689336;dx=-4.693039611684737E-10
Armijo: th(3.853310188348365E7)=0.3583624885067259; dx=-1.9044736126985474E-10 evalInputDelta=-0.004410135737792309
New Minimum: 0.3539523527689336 > 0.35383974059799567
END: th(1.9266550941741824E7)=0.35383974059799567; dx=-2.86543584933459E-10 evalInputDelta=1.126121709379424E-4
Fitness changed from 0.3539523527689336 to 0.35383974059799567
Iteration 49 complete. Error: 0.35383974059799567 Total: 5.6084; Orientation: 1.6156; Line Search: 3.1180
Adding measurement 5f95c7c0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.631e+03, gradient 2.231e-05, dot -0.790; [0ebad2b1-6e9d-489b-9eaa-5ea76a976592 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 0.35461922747671965, 0.3542191206590147, 0.3539523527689336, 0.35383974059799567
LBFGS Accumulation History: 3 points
Removed measurement 1cd07175 to history. Total: 4
th(0)=0.35383974059799567;dx=-4.977589217104601E-10
Armijo: th(4.150852570615505E7)=0.3599316635567249; dx=-1.5946347019432012E-10 evalInputDelta=-0.0060919229587292345
Armijo: th(2.0754262853077523E7)=0.3539657094999698; dx=-2.662865577625684E-10 evalInputDelta=-1.2596890197413835E-4
New Minimum: 0.35383974059799567 > 0.35297414014460077
END: th(6918087.617692508)=0.35297414014460077; dx=-4.093615843175677E-10 evalInputDelta=8.656004533948991E-4
Fitness changed from 0.35383974059799567 to 0.35297414014460077
Iteration 50 complete. Error: 0.35297414014460077 Total: 6.6679; Orientation: 1.6179; Line Search: 4.1718
<a id="p-29"></a>Iteration 50
<a id="p-28"></a>
Final threshold in iteration 51: 0.35297414014460077 (> -Infinity) after 294.365s (< 3600.000s)
0.35297414014460077