BasicOptimizer.scala:89 executed in 226.86 seconds (2.912 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3096191649232
Reset training subject: 3101429143944
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
th(0)=-12.675865948200226;dx=-3.4522637007370406E-6
WOLFE (weak): th(2.154434690031884)=-12.67587360739708; dx=-3.4444825820024025E-6 evalInputDelta=7.659196853637695E-6
WOLFE (weak): th(4.308869380063768)=-12.675883173942566; dx=-3.4435438231264124E-6 evalInputDelta=1.722574234008789E-5
WOLFE (weak): th(12.926608140191302)=-12.67591866850853; dx=-3.444211289180549E-6 evalInputDelta=5.272030830383301E-5
WOLFE (weak): th(51.70643256076521)=-12.676078706979752; dx=-3.4439964090181608E-6 evalInputDelta=2.1275877952575684E-4
WOLFE (weak): th(258.53216280382605)=-12.676931500434875; dx=-3.4432719745449555E-6 evalInputDelta=0.0010655522346496582
WOLFE (weak): th(1551.1929768229563)=-12.68225610256195; dx=-3.43815997943543E-6 evalInputDelta=0.0063901543617248535
WOLFE (weak): th(10858.350837760694)=-12.720327734947205; dx=-3.3905865399411533E-6 evalInputDelta=0.04446178674697876
New Minimum: -12.930862993001938 > -13.011330395936966
END: th(86866.80670208555)=-13.011330395936966; dx=-2.9941930099561338E-6 evalInputDelta=0.3354644477367401
Fitness changed from -12.930862993001938 to -13.011330395936966
Iteration 1 complete. Error: -13.011330395936966 Total: 60.9786; Orientation: 0.0205; Line Search: 45.6741
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 79af83ff to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-13.011330395936966;dx=-2.845030748171443E-6
New Minimum: -13.011330395936966 > -13.575046569108963
END: th(187148.86177126726)=-13.575046569108963; dx=-2.1668300285435493E-6 evalInputDelta=0.5637161731719971
Fitness changed from -13.011330395936966 to -13.575046569108963
Iteration 2 complete. Error: -13.575046569108963 Total: 14.8714; Orientation: 0.0276; Line Search: 9.5233
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1e781ecc to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-13.575046569108963;dx=-1.9572813311629645E-6
New Minimum: -13.575046569108963 > -14.366790741682053
END: th(403200.00000000006)=-14.366790741682053; dx=-1.337591664762339E-6 evalInputDelta=0.7917441725730896
Fitness changed from -13.575046569108963 to -14.366790741682053
Iteration 3 complete. Error: -14.366790741682053 Total: 15.3852; Orientation: 0.0292; Line Search: 9.9121
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement b9bf5a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-14.366790741682053;dx=-1.2730704162035075E-6
New Minimum: -14.366790741682053 > -15.330069571733475
END: th(868668.0670208557)=-15.330069571733475; dx=-6.873628472999695E-7 evalInputDelta=0.9632788300514221
Fitness changed from -14.366790741682053 to -15.330069571733475
Iteration 4 complete. Error: -15.330069571733475 Total: 15.7615; Orientation: 0.0287; Line Search: 10.4759
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 42c70b23 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.194e+03, gradient 9.560e-04, dot -0.778; [a0397d77-363d-4eba-abc4-5082c4bf57e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.330069571733475, -14.366790741682053, -13.575046569108963, -13.011330395936966
LBFGS Accumulation History: 3 points
Removed measurement 42c70b23 to history. Total: 3
th(0)=-15.330069571733475;dx=-9.131464644681835E-7
New Minimum: -15.330069571733475 > -16.252404898405075
END: th(1871488.617712673)=-16.252404898405075; dx=-2.2408992154687305E-7 evalInputDelta=0.9223353266716003
Fitness changed from -15.330069571733475 to -16.252404898405075
Iteration 5 complete. Error: -16.252404898405075 Total: 15.6475; Orientation: 0.5915; Line Search: 9.7671
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 630f8b3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.904e+03, gradient 9.592e-04, dot -0.532; [a0397d77-363d-4eba-abc4-5082c4bf57e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.252404898405075, -14.366790741682053, -13.575046569108963, -13.011330395936966
LBFGS Accumulation History: 3 points
Removed measurement 630f8b3 to history. Total: 3
th(0)=-16.252404898405075;dx=-9.183695831247223E-7
Armijo: th(4032000.0000000014)=-16.20831537246704; dx=1.677124232547243E-7 evalInputDelta=-0.04408952593803406
New Minimum: -16.252404898405075 > -16.678050756454468
WOLF (strong): th(2016000.0000000007)=-16.678050756454468; dx=1.4615101222287267E-7 evalInputDelta=0.4256458580493927
New Minimum: -16.678050756454468 > -16.722013860940933
END: th(672000.0000000002)=-16.722013860940933; dx=-2.7062757223101454E-7 evalInputDelta=0.46960896253585815
Fitness changed from -16.252404898405075 to -16.722013860940933
Iteration 6 complete. Error: -16.722013860940933 Total: 25.8745; Orientation: 0.5908; Line Search: 20.3636
Adding measurement 21032611 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.230e+03, gradient 6.730e-04, dot -0.676; [a0397d77-363d-4eba-abc4-5082c4bf57e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.722013860940933, -14.366790741682053, -13.575046569108963, -13.011330395936966
LBFGS Accumulation History: 3 points
Removed measurement 21032611 to history. Total: 3
th(0)=-16.722013860940933;dx=-4.5255320088203243E-7
New Minimum: -16.722013860940933 > -17.19341990351677
END: th(1447780.1117014263)=-17.19341990351677; dx=-1.5929060150186644E-7 evalInputDelta=0.4714060425758362
Fitness changed from -16.722013860940933 to -17.19341990351677
Iteration 7 complete. Error: -17.19341990351677 Total: 15.6075; Orientation: 0.5946; Line Search: 10.1214
Adding measurement 727a096f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.473e+03, gradient 7.030e-04, dot -0.603; [a0397d77-363d-4eba-abc4-5082c4bf57e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.19341990351677, -14.366790741682053, -13.575046569108963, -13.011330395936966
LBFGS Accumulation History: 3 points
Removed measurement 727a096f to history. Total: 3
th(0)=-17.19341990351677;dx=-4.937640868698339E-7
New Minimum: -17.19341990351677 > -17.39375901222229
WOLF (strong): th(3119147.6961877886)=-17.39375901222229; dx=9.86565363525688E-8 evalInputDelta=0.20033910870552063
New Minimum: -17.39375901222229 > -17.549523204565048
WOLF (strong): th(1559573.8480938943)=-17.549523204565048; dx=3.751776956339215E-9 evalInputDelta=0.3561033010482788
END: th(519857.94936463144)=-17.429995357990265; dx=-2.7486858252632527E-7 evalInputDelta=0.23657545447349548
Fitness changed from -17.19341990351677 to -17.549523204565048
Iteration 8 complete. Error: -17.549523204565048 Total: 25.2800; Orientation: 0.5987; Line Search: 19.5513
Adding measurement 2a10064 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.513e+03, gradient 7.189e-04, dot -0.525; [a0397d77-363d-4eba-abc4-5082c4bf57e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.549523204565048, -14.366790741682053, -13.575046569108963, -13.011330395936966
LBFGS Accumulation History: 3 points
Removed measurement 2a10064 to history. Total: 3
th(0)=-17.549523204565048;dx=-5.155749876023109E-7
New Minimum: -17.549523204565048 > -17.907769083976746
END: th(1120000.0000000005)=-17.907769083976746; dx=-4.687453552534346E-8 evalInputDelta=0.3582458794116974
Fitness changed from -17.549523204565048 to -17.907769083976746
Iteration 9 complete. Error: -17.907769083976746 Total: 16.6146; Orientation: 0.6126; Line Search: 9.8710
Adding measurement f14deda to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.648e+03, gradient 5.273e-04, dot -0.648; [a0397d77-363d-4eba-abc4-5082c4bf57e0 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.907769083976746, -14.366790741682053, -13.575046569108963, -13.011330395936966
LBFGS Accumulation History: 3 points
Removed measurement f14deda to history. Total: 3
th(0)=-17.907769083976746;dx=-2.7779555720618004E-7
New Minimum: -17.907769083976746 > -18.187688022851944
WOLF (strong): th(2412966.852835711)=-18.187688022851944; dx=1.1455443865791269E-8 evalInputDelta=0.27991893887519836
END: th(1206483.4264178555)=-18.157729238271713; dx=-8.42015863369233E-8 evalInputDelta=0.24996015429496765
Fitness changed from -17.907769083976746 to -18.187688022851944
Iteration 10 complete. Error: -18.187688022851944 Total: 20.5742; Orientation: 0.5861; Line Search: 15.0197
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -18.187688022851944 (> -Infinity) after 226.856s (< 5400.000s)
Returns
-18.187688022851944