Code from BasicOptimizer.scala:75 executed in 3790.68 seconds (61.555 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1089081816060900
Reset training subject: 1089134242850000
Adding measurement 39b5d60 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=1.6322704207634136;dx=-1.1538925758939153E-7
New Minimum: 1.6322704207634136 > 1.6322703909610912
WOLFE (weak): th(2.154434690031884)=1.6322703909610912; dx=-1.1434208563749825E-7 evalInputDelta=2.9802322387695312E-8
New Minimum: 1.6322703909610912 > 1.6322701823448345
WOLFE (weak): th(4.308869380063768)=1.6322701823448345; dx=-1.1434203355936921E-7 evalInputDelta=2.384185791015625E-7
New Minimum: 1.6322701823448345 > 1.6322687244097254
WOLFE (weak): th(12.926608140191302)=1.6322687244097254; dx=-1.1434189680952973E-7 evalInputDelta=1.6963536881853258E-6
New Minimum: 1.6322687244097254 > 1.632264407830355
WOLFE (weak): th(51.70643256076521)=1.632264407830355; dx=-1.1434137013952224E-7 evalInputDelta=6.012933058574532E-6
New Minimum: 1.632264407830355 > 1.632240932486031
WOLFE (weak): th(258.53216280382605)=1.632240932486031; dx=-1.1434009174293087E-7 evalInputDelta=2.9488277382583306E-5
New Minimum: 1.632240932486031 > 1.6320928551065643
WOLFE (weak): th(1551.1929768229563)=1.6320928551065643; dx=-1.1432605179652403E-7 evalInputDelta=1.775656568492323E-4
New Minimum: 1.6320928551065643 > 1.6310296945899947
WOLFE (weak): th(10858.350837760694)=1.6310296945899947; dx=-1.142040084266207E-7 evalInputDelta=0.0012407261734188602
New Minimum: 1.6310296945899947 > 1.622383385455556
WOLFE (weak): th(86866.80670208555)=1.622383385455556; dx=-1.132909223343937E-7 evalInputDelta=0.009887035307857506
New Minimum: 1.622383385455556 > 1.5468392020006165
WOLFE (weak): th(781801.26031877)=1.5468392020006165; dx=-1.040075558097817E-7 evalInputDelta=0.08543121876279702
New Minimum: 1.5468392020006165 > 1.0834979772701043
END: th(7818012.6031877)=1.0834979772701043; dx=-4.004488135488733E-8 evalInputDelta=0.5487724434933092
Fitness changed from 1.6322704207634136 to 1.0834979772701043
Iterat
...skipping 12638 bytes...
Line Search: 208.7522
Adding measurement 66da2007 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.650e+03, gradient 5.300e-05, dot -0.707; [df41c0e4-8240-424a-bf88-301e8ae85fbe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.17079664650744686, -0.19803567954774204, -0.2208101025466629, -0.24589866899825719
LBFGS Accumulation History: 3 points
Removed measurement 36545439 to history. Total: 4
th(0)=-0.24589866899825719;dx=-2.8092659043632025E-9
New Minimum: -0.24589866899825719 > -0.2670170922190098
END: th(1.3125000000000011E7)=-0.2670170922190098; dx=-6.376309031025481E-10 evalInputDelta=0.021118423220752636
Fitness changed from -0.24589866899825719 to -0.2670170922190098
Iteration 15 complete. Error: -0.2670170922190098 Total: 153.7463; Orientation: 2.8066; Line Search: 99.3393
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 1b38563a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.192e+03, gradient 5.116e-05, dot -0.650; [df41c0e4-8240-424a-bf88-301e8ae85fbe = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.19803567954774204, -0.2208101025466629, -0.24589866899825719, -0.2670170922190098
LBFGS Accumulation History: 3 points
Removed measurement 61267fa2 to history. Total: 4
th(0)=-0.2670170922190098;dx=-2.6176022900522797E-9
New Minimum: -0.2670170922190098 > -0.27684104584579616
WOLF (strong): th(2.8276955306668498E7)=-0.27684104584579616; dx=7.926329654226953E-10 evalInputDelta=0.009823953626786341
New Minimum: -0.27684104584579616 > -0.2828147979737383
END: th(1.4138477653334249E7)=-0.2828147979737383; dx=-5.850923675956411E-11 evalInputDelta=0.015797705754728497
Fitness changed from -0.2670170922190098 to -0.2828147979737383
Iteration 16 complete. Error: -0.2828147979737383 Total: 207.4757; Orientation: 2.7914; Line Search: 153.2830
Final threshold in iteration 16: -0.2828147979737383 (> -Infinity) after 3790.674s (< 3600.000s)
-0.2828147979737383