Code from BasicOptimizer.scala:75 executed in 3403.26 seconds (36.167 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
IterativeTrainer.wrap(trainable)
.setOrientation(orientation())
.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = trainingMonitor.log(msg)
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable, currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable, currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
.setTimeout(trainingMinutes, TimeUnit.MINUTES)
.setMaxIterations(trainingIterations)
.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
.runAndFree
.asInstanceOf[lang.Double]
Reset training subject: 1430966476633700
Reset training subject: 1431003210421800
Adding measurement 451cef67 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-30.200962781906128;dx=-2.817553202408519E-6
New Minimum: -30.200962781906128 > -30.200963206589222
WOLFE (weak): th(2.154434690031884)=-30.200963206589222; dx=-2.8156941483050593E-6 evalInputDelta=4.246830940246582E-7
New Minimum: -30.200963206589222 > -30.200964327901602
WOLFE (weak): th(4.308869380063768)=-30.200964327901602; dx=-2.8158239276326077E-6 evalInputDelta=1.5459954738616943E-6
New Minimum: -30.200964327901602 > -30.200968738024432
WOLFE (weak): th(12.926608140191302)=-30.200968738024432; dx=-2.816220705917446E-6 evalInputDelta=5.956118304339952E-6
New Minimum: -30.200968738024432 > -30.200987804681063
WOLFE (weak): th(51.70643256076521)=-30.200987804681063; dx=-2.818235992742415E-6 evalInputDelta=2.5022774934768677E-5
New Minimum: -30.200987804681063 > -30.201103587324422
WOLFE (weak): th(258.53216280382605)=-30.201103587324422; dx=-2.8275123287335283E-6 evalInputDelta=1.4080541829386561E-4
New Minimum: -30.201103587324422 > -30.201830902447302
WOLFE (weak): th(1551.1929768229563)=-30.201830902447302; dx=-2.847042936487693E-6 evalInputDelta=8.681205411740223E-4
New Minimum: -30.201830902447302 > -30.20671426442762
END: th(10858.350837760694)=-30.20671426442762; dx=-2.5118945590447076E-6 evalInputDelta=0.005751482521493045
Fitness changed from -30.200962781906128 to -30.20671426442762
Iteration 1 complete. Error: -30.20671426442762 Total: 436.3155; Orientation: 0.0786; Line Search: 321.7239
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 78141c58 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-30.20671426442762;dx=-2.4361296403066134E-6
New Minimum: -30.20671426442762 > -30.217785951991875
WOLFE (weak): th(23393.607721408407)=-30.217785951991875; dx=-2.3312378561151
...skipping 15410 bytes...
rement 67062a93 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.190e+03, gradient 5.312e-04, dot -0.535; [98b4face-a43a-41a1-b125-d4eb3f2d634f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.513917901242774, -31.548664196083944, -31.581898473203182, -31.614508757988613
LBFGS Accumulation History: 3 points
Removed measurement 66d73e2d to history. Total: 4
th(0)=-31.614508757988613;dx=-2.821703545901398E-7
New Minimum: -31.614508757988613 > -31.646175297598045
WOLF (strong): th(2094589.2819754444)=-31.646175297598045; dx=3.509158133971691E-8 evalInputDelta=0.03166653960943222
END: th(1047294.6409877222)=-31.64473340349893; dx=-7.400397117244349E-8 evalInputDelta=0.030224645510315895
Fitness changed from -31.614508757988613 to -31.646175297598045
Iteration 19 complete. Error: -31.646175297598045 Total: 162.3158; Orientation: 2.3478; Line Search: 123.4207
Adding measurement 65871cfd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.735e+03, gradient 4.585e-04, dot -0.629; [98b4face-a43a-41a1-b125-d4eb3f2d634f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.548664196083944, -31.581898473203182, -31.614508757988613, -31.646175297598045
LBFGS Accumulation History: 3 points
Removed measurement 4718d320 to history. Total: 4
th(0)=-31.646175297598045;dx=-2.102147418486338E-7
New Minimum: -31.646175297598045 > -31.67729593006273
WOLF (strong): th(2256327.9052284365)=-31.67729593006273; dx=1.2527663806182277E-8 evalInputDelta=0.03112063246468466
END: th(1128163.9526142182)=-31.673265855759382; dx=-7.578186672186606E-8 evalInputDelta=0.027090558161336986
Fitness changed from -31.646175297598045 to -31.67729593006273
Iteration 20 complete. Error: -31.67729593006273 Total: 160.3592; Orientation: 2.3772; Line Search: 117.6804
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: -31.67729593006273 (> -Infinity) after 3403.263s (< 5400.000s)
-31.67729593006273