BasicOptimizer.scala:89 executed in 9.06 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 144738534119100
Reset training subject: 144738665111300
Adding measurement 16f9e4be to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.084663592041197;dx=-6.281396410735135E-5
New Minimum: -1.084663592041197 > -1.0847890947400485
WOLFE (weak): th(2.154434690031884)=-1.0847890947400485; dx=-6.261597688464404E-5 evalInputDelta=1.2550269885136522E-4
New Minimum: -1.0847890947400485 > -1.0849149608752957
WOLFE (weak): th(4.308869380063768)=-1.0849149608752957; dx=-6.26615959611061E-5 evalInputDelta=2.5136883409859756E-4
New Minimum: -1.0849149608752957 > -1.0854145488847595
WOLFE (weak): th(12.926608140191302)=-1.0854145488847595; dx=-6.288161799048705E-5 evalInputDelta=7.509568435624114E-4
New Minimum: -1.0854145488847595 > -1.087663413144699
WOLFE (weak): th(51.70643256076521)=-1.087663413144699; dx=-6.289187213289072E-5 evalInputDelta=0.002999821103501832
New Minimum: -1.087663413144699 > -1.0996785133582891
WOLFE (weak): th(258.53216280382605)=-1.0996785133582891; dx=-6.254882713233083E-5 evalInputDelta=0.015014921317092034
New Minimum: -1.0996785133582891 > -1.1767797081585
WOLFE (weak): th(1551.1929768229563)=-1.1767797081585; dx=-6.299544594292532E-5 evalInputDelta=0.09211611611730297
New Minimum: -1.1767797081585 > -1.7518825484936515
END: th(10858.350837760694)=-1.7518825484936515; dx=-5.3883199887157606E-5 evalInputDelta=0.6672189564524544
Fitness changed from -1.084663592041197 to -1.7518825484936515
Iteration 1 complete. Error: -1.7518825484936515 Total: 1.2747; Orientation: 0.0015; Line Search: 0.9216
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 574ff216 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.7518825484936515;dx=-8.863268999402756E-5
New Minimum: -1.7518825484936515 > -3.020144331010518
END: th(23393.607721408407)=-3.020144331010518; dx=-1.8067300071433747E-5 evalInputDelta=1.2682617825168663
Fitness changed from -1.7518825484936515 to -3.020144331010518
Iteration 2 complete. Error: -3.020144331010518 Total: 0.3534; Orientation: 0.0016; Line Search: 0.2309
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 15e35b39 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.020144331010518;dx=-6.542854374242765E-5
Armijo: th(50400.00000000001)=-2.91600078916723; dx=-7.648100605604503E-7 evalInputDelta=-0.10414354184328767
New Minimum: -3.020144331010518 > -3.335940020573545
END: th(25200.000000000004)=-3.335940020573545; dx=-1.3850300196239233E-5 evalInputDelta=0.31579568956302717
Fitness changed from -3.020144331010518 to -3.335940020573545
Iteration 3 complete. Error: -3.335940020573545 Total: 0.5040; Orientation: 0.0015; Line Search: 0.3740
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6036d5ee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.370e+03, gradient 8.238e-03, dot -0.265; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.335940020573545, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 6036d5ee to history. Total: 3
th(0)=-3.335940020573545;dx=-6.463526586274064E-5
New Minimum: -3.335940020573545 > -4.1639067458719
END: th(54291.75418880348)=-4.1639067458719; dx=-1.9966942360783117E-6 evalInputDelta=0.8279667252983551
Fitness changed from -3.335940020573545 to -4.1639067458719
Iteration 4 complete. Error: -4.1639067458719 Total: 0.3850; Orientation: 0.0296; Line Search: 0.2332
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 424a6a0e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.416e+03, gradient 8.988e-03, dot -0.131; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.1639067458719, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 424a6a0e to history. Total: 3
th(0)=-4.1639067458719;dx=-7.82963338737222E-5
Armijo: th(116968.03860704206)=-3.6105147110412696; dx=3.7867253894502173E-6 evalInputDelta=-0.5533920348306305
New Minimum: -4.1639067458719 > -4.714834849024234
WOLF (strong): th(58484.01930352103)=-4.714834849024234; dx=5.127539636249409E-6 evalInputDelta=0.5509281031523336
New Minimum: -4.714834849024234 > -5.146391990062931
END: th(19494.673101173677)=-5.146391990062931; dx=-2.8742768045651743E-5 evalInputDelta=0.9824852441910306
Fitness changed from -4.1639067458719 to -5.146391990062931
Iteration 5 complete. Error: -5.146391990062931 Total: 0.6236; Orientation: 0.0289; Line Search: 0.4692
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4a963864 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.760e+03, gradient 7.164e-03, dot -0.253; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.146391990062931, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 4a963864 to history. Total: 3
th(0)=-5.146391990062931;dx=-4.882200832970913E-5
New Minimum: -5.146391990062931 > -5.947130500206563
END: th(42000.000000000015)=-5.947130500206563; dx=-1.4649423353058581E-5 evalInputDelta=0.8007385101436322
Fitness changed from -5.146391990062931 to -5.947130500206563
Iteration 6 complete. Error: -5.947130500206563 Total: 0.3798; Orientation: 0.0296; Line Search: 0.2327
Adding measurement 64946d7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.656e+03, gradient 7.607e-03, dot -0.235; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.947130500206563, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 64946d7 to history. Total: 3
th(0)=-5.947130500206563;dx=-5.540896706641295E-5
New Minimum: -5.947130500206563 > -6.263795790213083
END: th(90486.25698133915)=-6.263795790213083; dx=-3.020961329947902E-6 evalInputDelta=0.31666529000652055
Fitness changed from -5.947130500206563 to -6.263795790213083
Iteration 7 complete. Error: -6.263795790213083 Total: 0.3646; Orientation: 0.0284; Line Search: 0.2251
Adding measurement 20999d79 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.466e+03, gradient 9.565e-03, dot -0.223; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.263795790213083, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 20999d79 to history. Total: 3
th(0)=-6.263795790213083;dx=-8.93812658359109E-5
New Minimum: -6.263795790213083 > -6.450115059874471
END: th(194946.7310117368)=-6.450115059874471; dx=-2.4665381468150965E-7 evalInputDelta=0.18631926966138757
Fitness changed from -6.263795790213083 to -6.450115059874471
Iteration 8 complete. Error: -6.450115059874471 Total: 0.3641; Orientation: 0.0282; Line Search: 0.2274
Adding measurement 33d50679 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.186e+03, gradient 8.223e-03, dot -0.188; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.450115059874471, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 33d50679 to history. Total: 3
th(0)=-6.450115059874471;dx=-6.606447262218187E-5
New Minimum: -6.450115059874471 > -8.966940701238368
END: th(420000.0000000002)=-8.966940701238368; dx=-3.5948451850026726E-8 evalInputDelta=2.516825641363897
Fitness changed from -6.450115059874471 to -8.966940701238368
Iteration 9 complete. Error: -8.966940701238368 Total: 0.3712; Orientation: 0.0279; Line Search: 0.2309
Adding measurement 265dbcf3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.091e+04, gradient 7.776e-03, dot -0.185; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.966940701238368, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 265dbcf3 to history. Total: 3
th(0)=-8.966940701238368;dx=-5.93355721673994E-5
New Minimum: -8.966940701238368 > -12.924337592873908
WOLF (strong): th(904862.5698133915)=-12.924337592873908; dx=3.979361422711581E-6 evalInputDelta=3.9573968916355398
New Minimum: -12.924337592873908 > -19.23834039879642
END: th(452431.2849066958)=-19.23834039879642; dx=-1.4065240755524287E-6 evalInputDelta=10.271399697558053
Fitness changed from -8.966940701238368 to -19.23834039879642
Iteration 10 complete. Error: -19.23834039879642 Total: 0.4823; Orientation: 0.0285; Line Search: 0.3429
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7b4516bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.941e+05, gradient 7.956e-03, dot -0.247; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.23834039879642, -3.020144331010518, -1.7518825484936515, -1.084663592041197
LBFGS Accumulation History: 3 points
Removed measurement 7b4516bb to history. Total: 3
th(0)=-19.23834039879642;dx=-6.248909024355685E-5
Armijo: th(974733.6550586841)=-10.873889719740673; dx=6.924648227974224E-6 evalInputDelta=-8.364450679055748
New Minimum: -19.23834039879642 > -21.692596911858164
WOLF (strong): th(487366.82752934203)=-21.692596911858164; dx=5.649112524397179E-6 evalInputDelta=2.4542565130617433
New Minimum: -21.692596911858164 > -26.13590830988711
END: th(162455.60917644735)=-26.13590830988711; dx=-2.9453326736254268E-5 evalInputDelta=6.89756791109069
Fitness changed from -19.23834039879642 to -26.13590830988711
Iteration 11 complete. Error: -26.13590830988711 Total: 0.6096; Orientation: 0.0287; Line Search: 0.4602
Adding measurement 3ce2ac67 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.636e+04, gradient 7.374e-03, dot 0.398; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-26.13590830988711;dx=-77.50784825374043
Armijo: th(350000.00000000023)=-16.588977635724184; dx=2.4562504443168765E-5 evalInputDelta=-9.546930674162926
Armijo: th(175000.00000000012)=-16.588977635724184; dx=4.9463761765514915E-5 evalInputDelta=-9.546930674162926
Armijo: th(58333.33333333337)=-16.588977635724184; dx=1.4679857099805075E-4 evalInputDelta=-9.546930674162926
Armijo: th(14583.333333333343)=-16.588977635724184; dx=5.822764664275888E-4 evalInputDelta=-9.546930674162926
Armijo: th(2916.666666666669)=-16.598725973428426; dx=0.002895131380500823 evalInputDelta=-9.537182336458685
Armijo: th(486.1111111111115)=-16.662531537304574; dx=0.017810381257188673 evalInputDelta=-9.473376772582537
Armijo: th(69.4444444444445)=-16.922327988243055; dx=0.1217243277674174 evalInputDelta=-9.213580321644056
Armijo: th(8.680555555555562)=-17.80832023379177; dx=0.8982598516647087 evalInputDelta=-8.327588076095342
Armijo: th(0.9645061728395069)=-23.703184427585136; dx=5.86974080952351 evalInputDelta=-2.432723882301975
New Minimum: -26.13590830988711 > -30.25548899952383
END: th(0.09645061728395068)=-30.25548899952383; dx=-18.519361856709963 evalInputDelta=4.11958068963672
Fitness changed from -26.13590830988711 to -30.25548899952383
Iteration 12 complete. Error: -30.25548899952383 Total: 1.4027; Orientation: 0.0280; Line Search: 1.2615
Adding measurement 4111d138 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.171e+04, gradient 7.611e-03, dot 0.297; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
th(0)=-30.25548899952383;dx=-49.00114410321362
Armijo: th(0.20779655575153214)=-28.79359815601673; dx=2.550717403202193 evalInputDelta=-1.4618908435070992
Armijo: th(0.10389827787576607)=-30.12859014598308; dx=-1.685986147708745 evalInputDelta=-0.1268988535407516
New Minimum: -30.25548899952383 > -30.441743090479395
END: th(0.03463275929192202)=-30.441743090479395; dx=-6.162126259373851 evalInputDelta=0.18625409095556478
Fitness changed from -30.25548899952383 to -30.441743090479395
Iteration 13 complete. Error: -30.441743090479395 Total: 0.5931; Orientation: 0.0326; Line Search: 0.4506
Adding measurement 33cdd0dd to history. Total: 5
Accepted: LBFGS Orientation magnitude: 2.066e+04, gradient 7.764e-03, dot 0.274; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
th(0)=-30.441743090479395;dx=-43.80338319283458
Armijo: th(0.07461401803004086)=-30.08651583794134; dx=-1.199992538386502 evalInputDelta=-0.35522725253805376
Armijo: th(0.03730700901502043)=-30.35858052900149; dx=-3.9919945826873513 evalInputDelta=-0.08316256147790568
Armijo: th(0.012435669671673477)=-30.436652967377615; dx=-4.714536153303041 evalInputDelta=-0.005090123101780364
New Minimum: -30.441743090479395 > -30.44379303852453
END: th(0.0031089174179183693)=-30.44379303852453; dx=-4.9465518040842795 evalInputDelta=0.002049948045133476
Fitness changed from -30.441743090479395 to -30.44379303852453
Iteration 14 complete. Error: -30.44379303852453 Total: 0.7225; Orientation: 0.0401; Line Search: 0.5705
Adding measurement 22374975 to history. Total: 6
Accepted: LBFGS Orientation magnitude: 2.030e+04, gradient 7.765e-03, dot 0.273; [8a9899db-fa32-43c6-a42c-c78d45c09924 = 1.000/1.000e+00]
th(0)=-30.44379303852453;dx=-42.420234828176135
Armijo: th(0.0066979595336076865)=-30.440017165115528; dx=-5.087143852675486 evalInputDelta=-0.003775873409001207
Armijo: th(0.0033489797668038433)=-30.443175827312476; dx=-5.535610963689112 evalInputDelta=-6.17211212052382E-4
New Minimum: -30.44379303852453 > -30.443903376640634
END: th(0.0011163265889346144)=-30.443903376640634; dx=-5.631722153889324 evalInputDelta=1.1033811610516864E-4
Fitness changed from -30.44379303852453 to -30.443903376640634
Iteration 15 complete. Error: -30.443903376640634 Total: 0.6151; Orientation: 0.0475; Line Search: 0.4539
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -30.443903376640634 (> -Infinity) after 9.057s (< 5400.000s)
Returns
-30.443903376640634