BasicOptimizer.scala:89 executed in 208.33 seconds (2.709 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3591324871638
Reset training subject: 3595619393825
Adding measurement 27b4f61b to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.5563511997461319;dx=-5.936726003183952E-7
New Minimum: -0.5563511997461319 > -0.5563513338565826
WOLFE (weak): th(2.154434690031884)=-0.5563513338565826; dx=-5.626772195617994E-7 evalInputDelta=1.341104507446289E-7
New Minimum: -0.5563513338565826 > -0.5563521087169647
WOLFE (weak): th(4.308869380063768)=-0.5563521087169647; dx=-5.626768387491538E-7 evalInputDelta=9.08970832824707E-7
New Minimum: -0.5563521087169647 > -0.5563547611236572
WOLFE (weak): th(12.926608140191302)=-0.5563547611236572; dx=-5.626758533173144E-7 evalInputDelta=3.56137752532959E-6
New Minimum: -0.5563547611236572 > -0.5563656538724899
WOLFE (weak): th(51.70643256076521)=-0.5563656538724899; dx=-5.626746200168708E-7 evalInputDelta=1.4454126358032227E-5
New Minimum: -0.5563656538724899 > -0.5564238429069519
WOLFE (weak): th(258.53216280382605)=-0.5564238429069519; dx=-5.626318907132165E-7 evalInputDelta=7.264316082000732E-5
New Minimum: -0.5564238429069519 > -0.5567874908447266
WOLFE (weak): th(1551.1929768229563)=-0.5567874908447266; dx=-5.623529062190418E-7 evalInputDelta=4.3629109859466553E-4
New Minimum: -0.5567874908447266 > -0.5594007223844528
WOLFE (weak): th(10858.350837760694)=-0.5594007223844528; dx=-5.606332146400205E-7 evalInputDelta=0.003049522638320923
New Minimum: -0.5594007223844528 > -0.5804778337478638
WOLFE (weak): th(86866.80670208555)=-0.5804778337478638; dx=-5.483211936401697E-7 evalInputDelta=0.024126634001731873
New Minimum: -0.5804778337478638 > -0.7496712356805801
END: th(781801.26031877)=-0.7496712356805801; dx=-4.2388872482034966E-7 evalInputDelta=0.19332003593444824
Fitness changed from -0.5563511997461319 to -0.7496712356805801
Iteration 1 complete. Error: -0.7496712356805801 Total: 56.8453; Orientation: 0.0433; Line Search: 43.6002
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 70dd14c4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.7496712356805801;dx=-3.7295498173511086E-7
New Minimum: -0.7496712356805801 > -1.0037526339292526
END: th(1684339.7559414052)=-1.0037526339292526; dx=-2.3578979733213904E-7 evalInputDelta=0.2540813982486725
Fitness changed from -0.7496712356805801 to -1.0037526339292526
Iteration 2 complete. Error: -1.0037526339292526 Total: 12.6332; Orientation: 0.0466; Line Search: 8.7052
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7c43c4f1 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1.0037526339292526;dx=-2.1279414580647864E-7
New Minimum: -1.0037526339292526 > -1.2869074046611786
END: th(3628800.0)=-1.2869074046611786; dx=-1.1140940236523128E-7 evalInputDelta=0.28315477073192596
Fitness changed from -1.0037526339292526 to -1.2869074046611786
Iteration 3 complete. Error: -1.2869074046611786 Total: 12.3901; Orientation: 0.0468; Line Search: 8.0537
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1de02083 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.849e+03, gradient 3.574e-04, dot -0.808; [da3b4cce-0e9d-4734-938c-8c7fc81829bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.2869074046611786, -1.0037526339292526, -0.7496712356805801, -0.5563511997461319
LBFGS Accumulation History: 3 points
Removed measurement 1de02083 to history. Total: 3
th(0)=-1.2869074046611786;dx=-1.2772306545606912E-7
New Minimum: -1.2869074046611786 > -1.561873897910118
END: th(7818012.6031877)=-1.561873897910118; dx=-3.740734629531635E-8 evalInputDelta=0.2749664932489395
Fitness changed from -1.2869074046611786 to -1.561873897910118
Iteration 4 complete. Error: -1.561873897910118 Total: 14.6372; Orientation: 1.0814; Line Search: 8.7392
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 55c9223e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.456e+03, gradient 3.175e-04, dot -0.590; [da3b4cce-0e9d-4734-938c-8c7fc81829bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.561873897910118, -1.0037526339292526, -0.7496712356805801, -0.5563511997461319
LBFGS Accumulation History: 3 points
Removed measurement 55c9223e to history. Total: 3
th(0)=-1.561873897910118;dx=-1.0078420819525912E-7
New Minimum: -1.561873897910118 > -1.7085851579904556
WOLF (strong): th(1.684339755941405E7)=-1.7085851579904556; dx=1.144595817642243E-8 evalInputDelta=0.14671126008033752
New Minimum: -1.7085851579904556 > -1.7309936359524727
END: th(8421698.779707026)=-1.7309936359524727; dx=-4.501425604701749E-9 evalInputDelta=0.16911973804235458
Fitness changed from -1.561873897910118 to -1.7309936359524727
Iteration 5 complete. Error: -1.7309936359524727 Total: 19.0101; Orientation: 1.0625; Line Search: 13.3705
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 48a213d2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.499e+03, gradient 2.910e-04, dot -0.506; [da3b4cce-0e9d-4734-938c-8c7fc81829bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.7309936359524727, -1.0037526339292526, -0.7496712356805801, -0.5563511997461319
LBFGS Accumulation History: 3 points
Removed measurement 48a213d2 to history. Total: 3
th(0)=-1.7309936359524727;dx=-8.468694869016444E-8
New Minimum: -1.7309936359524727 > -1.7599697187542915
WOLF (strong): th(1.8144E7)=-1.7599697187542915; dx=2.894998458516291E-8 evalInputDelta=0.028976082801818848
New Minimum: -1.7599697187542915 > -1.8683386594057083
WOLF (strong): th(9072000.0)=-1.8683386594057083; dx=9.919436212291802E-9 evalInputDelta=0.13734502345323563
END: th(3024000.0)=-1.82527956366539; dx=-4.141760364325795E-8 evalInputDelta=0.09428592771291733
Fitness changed from -1.7309936359524727 to -1.8683386594057083
Iteration 6 complete. Error: -1.8683386594057083 Total: 23.2022; Orientation: 1.1847; Line Search: 17.2612
Adding measurement 2bbea0cb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.349e+03, gradient 2.677e-04, dot -0.453; [da3b4cce-0e9d-4734-938c-8c7fc81829bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8683386594057083, -1.0037526339292526, -0.7496712356805801, -0.5563511997461319
LBFGS Accumulation History: 3 points
Removed measurement 2bbea0cb to history. Total: 3
th(0)=-1.8683386594057083;dx=-7.168876185056053E-8
New Minimum: -1.8683386594057083 > -1.9781645759940147
END: th(6515010.502656417)=-1.9781645759940147; dx=-7.649983691062698E-9 evalInputDelta=0.10982591658830643
Fitness changed from -1.8683386594057083 to -1.9781645759940147
Iteration 7 complete. Error: -1.9781645759940147 Total: 14.2554; Orientation: 1.0392; Line Search: 9.0156
Adding measurement 7668ad0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.681e+03, gradient 1.812e-04, dot -0.571; [da3b4cce-0e9d-4734-938c-8c7fc81829bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.9781645759940147, -1.0037526339292526, -0.7496712356805801, -0.5563511997461319
LBFGS Accumulation History: 3 points
Removed measurement 7668ad0 to history. Total: 3
th(0)=-1.9781645759940147;dx=-3.283360235656427E-8
New Minimum: -1.9781645759940147 > -2.059281140565872
WOLF (strong): th(1.4036164632845046E7)=-2.059281140565872; dx=6.332368722019835E-9 evalInputDelta=0.08111656457185745
END: th(7018082.316422523)=-2.053126700222492; dx=-1.0877013409560414E-8 evalInputDelta=0.07496212422847748
Fitness changed from -1.9781645759940147 to -2.059281140565872
Iteration 8 complete. Error: -2.059281140565872 Total: 17.7728; Orientation: 1.0128; Line Search: 12.5657
Adding measurement 229512e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.308e+03, gradient 2.571e-04, dot -0.366; [da3b4cce-0e9d-4734-938c-8c7fc81829bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.059281140565872, -1.0037526339292526, -0.7496712356805801, -0.5563511997461319
LBFGS Accumulation History: 3 points
Removed measurement 229512e3 to history. Total: 3
th(0)=-2.059281140565872;dx=-6.608857442117896E-8
Armijo: th(1.5120000000000004E7)=-2.0434882789850235; dx=2.8385952644467058E-8 evalInputDelta=-0.015792861580848694
New Minimum: -2.059281140565872 > -2.1303555965423584
WOLF (strong): th(7560000.000000002)=-2.1303555965423584; dx=1.1769060562226783E-8 evalInputDelta=0.0710744559764862
END: th(2520000.0000000005)=-2.117382764816284; dx=-2.7358806555695513E-8 evalInputDelta=0.05810162425041199
Fitness changed from -2.059281140565872 to -2.1303555965423584
Iteration 9 complete. Error: -2.1303555965423584 Total: 23.7571; Orientation: 0.9789; Line Search: 18.4421
Adding measurement 4dff683f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.428e+03, gradient 1.787e-04, dot -0.434; [da3b4cce-0e9d-4734-938c-8c7fc81829bf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.1303555965423584, -1.0037526339292526, -0.7496712356805801, -0.5563511997461319
LBFGS Accumulation History: 3 points
Removed measurement 4dff683f to history. Total: 3
th(0)=-2.1303555965423584;dx=-3.192271167992684E-8
New Minimum: -2.1303555965423584 > -2.185507297515869
END: th(5429175.418880348)=-2.185507297515869; dx=-9.968335594039912E-9 evalInputDelta=0.05515170097351074
Fitness changed from -2.1303555965423584 to -2.185507297515869
Iteration 10 complete. Error: -2.185507297515869 Total: 13.7557; Orientation: 0.9876; Line Search: 8.6553
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -2.185507297515869 (> -Infinity) after 208.332s (< 5400.000s)
Returns
-2.185507297515869