BasicOptimizer.scala:89 executed in 235.04 seconds (3.814 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 1318452619175
Reset training subject: 1324405569672
Adding measurement 3b9a5894 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.667650640010834;dx=-2.2228063334232786E-6
New Minimum: -15.667650640010834 > -15.667656272649765
WOLFE (weak): th(2.154434690031884)=-15.667656272649765; dx=-2.0421727862556353E-6 evalInputDelta=5.632638931274414E-6
New Minimum: -15.667656272649765 > -15.667661398649216
WOLFE (weak): th(4.308869380063768)=-15.667661398649216; dx=-2.0425091543413444E-6 evalInputDelta=1.0758638381958008E-5
New Minimum: -15.667661398649216 > -15.66768205165863
WOLFE (weak): th(12.926608140191302)=-15.66768205165863; dx=-2.043009435176718E-6 evalInputDelta=3.141164779663086E-5
New Minimum: -15.66768205165863 > -15.667776554822922
WOLFE (weak): th(51.70643256076521)=-15.667776554822922; dx=-2.042789416490123E-6 evalInputDelta=1.259148120880127E-4
New Minimum: -15.667776554822922 > -15.668280571699142
WOLFE (weak): th(258.53216280382605)=-15.668280571699142; dx=-2.0422641324487536E-6 evalInputDelta=6.299316883087158E-4
New Minimum: -15.668280571699142 > -15.671429067850113
WOLFE (weak): th(1551.1929768229563)=-15.671429067850113; dx=-2.039365185730915E-6 evalInputDelta=0.003778427839279175
New Minimum: -15.671429067850113 > -15.693949282169342
WOLFE (weak): th(10858.350837760694)=-15.693949282169342; dx=-2.0158564146354125E-6 evalInputDelta=0.0262986421585083
New Minimum: -15.693949282169342 > -15.867470473051071
END: th(86866.80670208555)=-15.867470473051071; dx=-1.8069713246334074E-6 evalInputDelta=0.19981983304023743
Fitness changed from -15.667650640010834 to -15.867470473051071
Iteration 1 complete. Error: -15.867470473051071 Total: 61.7955; Orientation: 0.0275; Line Search: 45.8656
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement e4cf921 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-15.867470473051071;dx=-1.6501627045402966E-6
New Minimum: -15.867470473051071 > -16.191319346427917
END: th(187148.86177126726)=-16.191319346427917; dx=-1.2493110687266437E-6 evalInputDelta=0.3238488733768463
Fitness changed from -15.867470473051071 to -16.191319346427917
Iteration 2 complete. Error: -16.191319346427917 Total: 15.5664; Orientation: 0.0350; Line Search: 10.1407
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 298fd514 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-16.191319346427917;dx=-1.0720828353706095E-6
New Minimum: -16.191319346427917 > -16.61191812157631
END: th(403200.00000000006)=-16.61191812157631; dx=-6.99477047796448E-7 evalInputDelta=0.4205987751483917
Fitness changed from -16.191319346427917 to -16.61191812157631
Iteration 3 complete. Error: -16.61191812157631 Total: 15.4493; Orientation: 0.0271; Line Search: 10.1556
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement f83393a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.028e+03, gradient 8.063e-04, dot -0.880; [9f079dac-5fdc-43a9-b3da-ba31e4512e7a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.61191812157631, -16.191319346427917, -15.867470473051071, -15.667650640010834
LBFGS Accumulation History: 3 points
Removed measurement f83393a to history. Total: 3
th(0)=-16.61191812157631;dx=-6.492845919541869E-7
New Minimum: -16.61191812157631 > -17.12127074599266
END: th(868668.0670208557)=-17.12127074599266; dx=-3.826899288200334E-7 evalInputDelta=0.5093526244163513
Fitness changed from -16.61191812157631 to -17.12127074599266
Iteration 4 complete. Error: -17.12127074599266 Total: 15.9421; Orientation: 0.6011; Line Search: 9.9148
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 13c547f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.115e+03, gradient 6.810e-04, dot -0.773; [9f079dac-5fdc-43a9-b3da-ba31e4512e7a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.12127074599266, -16.191319346427917, -15.867470473051071, -15.667650640010834
LBFGS Accumulation History: 3 points
Removed measurement 13c547f0 to history. Total: 3
th(0)=-17.12127074599266;dx=-4.634616349951493E-7
New Minimum: -17.12127074599266 > -17.700931012630463
END: th(1871488.617712673)=-17.700931012630463; dx=-1.5899482248337252E-7 evalInputDelta=0.5796602666378021
Fitness changed from -17.12127074599266 to -17.700931012630463
Iteration 5 complete. Error: -17.700931012630463 Total: 15.7782; Orientation: 0.5909; Line Search: 9.9223
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1d6642a1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.164e+03, gradient 6.595e-04, dot -0.543; [9f079dac-5fdc-43a9-b3da-ba31e4512e7a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.700931012630463, -16.191319346427917, -15.867470473051071, -15.667650640010834
LBFGS Accumulation History: 3 points
Removed measurement 1d6642a1 to history. Total: 3
th(0)=-17.700931012630463;dx=-4.344838804912985E-7
New Minimum: -17.700931012630463 > -17.900214970111847
WOLF (strong): th(4032000.0000000014)=-17.900214970111847; dx=7.476422240578854E-8 evalInputDelta=0.19928395748138428
New Minimum: -17.900214970111847 > -18.06492191553116
END: th(2016000.0000000007)=-18.06492191553116; dx=-1.4272156679227823E-9 evalInputDelta=0.3639909029006958
Fitness changed from -17.700931012630463 to -18.06492191553116
Iteration 6 complete. Error: -18.06492191553116 Total: 21.2204; Orientation: 0.6212; Line Search: 14.9015
Adding measurement 72eb6f8a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.374e+03, gradient 7.238e-04, dot -0.419; [9f079dac-5fdc-43a9-b3da-ba31e4512e7a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.06492191553116, -16.191319346427917, -15.867470473051071, -15.667650640010834
LBFGS Accumulation History: 3 points
Removed measurement 72eb6f8a to history. Total: 3
th(0)=-18.06492191553116;dx=-5.233736276050185E-7
Armijo: th(4343340.335104279)=-17.636815071105957; dx=2.4775844947496804E-7 evalInputDelta=-0.4281068444252014
New Minimum: -18.06492191553116 > -18.340416848659515
WOLF (strong): th(2171670.1675521396)=-18.340416848659515; dx=1.8503894024321052E-7 evalInputDelta=0.27549493312835693
New Minimum: -18.340416848659515 > -18.370934545993805
END: th(723890.0558507132)=-18.370934545993805; dx=-2.0157202373569365E-7 evalInputDelta=0.3060126304626465
Fitness changed from -18.06492191553116 to -18.370934545993805
Iteration 7 complete. Error: -18.370934545993805 Total: 25.4917; Orientation: 0.5963; Line Search: 19.8914
Adding measurement f69fca to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.823e+03, gradient 4.713e-04, dot -0.654; [9f079dac-5fdc-43a9-b3da-ba31e4512e7a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.370934545993805, -16.191319346427917, -15.867470473051071, -15.667650640010834
LBFGS Accumulation History: 3 points
Removed measurement f69fca to history. Total: 3
th(0)=-18.370934545993805;dx=-2.2185002224371686E-7
New Minimum: -18.370934545993805 > -18.66735416650772
END: th(1559573.8480938943)=-18.66735416650772; dx=-1.0392644932240042E-7 evalInputDelta=0.296419620513916
Fitness changed from -18.370934545993805 to -18.66735416650772
Iteration 8 complete. Error: -18.66735416650772 Total: 15.3239; Orientation: 0.6036; Line Search: 9.8098
Adding measurement 6c2ad318 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.111e+03, gradient 4.458e-04, dot -0.662; [9f079dac-5fdc-43a9-b3da-ba31e4512e7a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.66735416650772, -16.191319346427917, -15.867470473051071, -15.667650640010834
LBFGS Accumulation History: 3 points
Removed measurement 6c2ad318 to history. Total: 3
th(0)=-18.66735416650772;dx=-1.9772498562418536E-7
New Minimum: -18.66735416650772 > -18.918651223182678
WOLF (strong): th(3360000.0000000014)=-18.918651223182678; dx=2.166642248593159E-9 evalInputDelta=0.2512970566749573
END: th(1680000.0000000007)=-18.88998693227768; dx=-5.193711784897156E-8 evalInputDelta=0.2226327657699585
Fitness changed from -18.66735416650772 to -18.918651223182678
Iteration 9 complete. Error: -18.918651223182678 Total: 21.8141; Orientation: 0.6170; Line Search: 14.9058
Adding measurement 3dc39bbc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.892e+03, gradient 6.610e-04, dot -0.345; [9f079dac-5fdc-43a9-b3da-ba31e4512e7a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.918651223182678, -16.191319346427917, -15.867470473051071, -15.667650640010834
LBFGS Accumulation History: 3 points
Removed measurement 3dc39bbc to history. Total: 3
th(0)=-18.918651223182678;dx=-4.3654089636495895E-7
Armijo: th(3619450.279253566)=-18.65681004524231; dx=2.592331563072759E-7 evalInputDelta=-0.26184117794036865
New Minimum: -18.918651223182678 > -19.16784131526947
WOLF (strong): th(1809725.139626783)=-19.16784131526947; dx=1.2277701550779733E-7 evalInputDelta=0.249190092086792
END: th(603241.7132089277)=-19.14461576938629; dx=-1.9488976365118994E-7 evalInputDelta=0.22596454620361328
Fitness changed from -18.918651223182678 to -19.16784131526947
Iteration 10 complete. Error: -19.16784131526947 Total: 26.3999; Orientation: 0.5957; Line Search: 20.8573
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -19.16784131526947 (> -Infinity) after 235.035s (< 5400.000s)
Returns
-19.16784131526947