BasicOptimizer.scala:89 executed in 48.03 seconds (0.959 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 220419647357200
Reset training subject: 220420370122800
Adding measurement 38c2f881 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-93.70813127571961;dx=-0.09715261338721676
New Minimum: -93.70813127571961 > -93.84428055846303
WOLFE (weak): th(2.154434690031884)=-93.84428055846303; dx=-0.09732008241315697 evalInputDelta=0.13614928274341764
New Minimum: -93.84428055846303 > -93.98066351430118
WOLFE (weak): th(4.308869380063768)=-93.98066351430118; dx=-0.09747874227251198 evalInputDelta=0.2725322385815758
New Minimum: -93.98066351430118 > -94.52601169773101
WOLFE (weak): th(12.926608140191302)=-94.52601169773101; dx=-0.0977022746620789 evalInputDelta=0.8178804220113989
New Minimum: -94.52601169773101 > -97.01624254459988
WOLFE (weak): th(51.70643256076521)=-97.01624254459988; dx=-0.0988151355138272 evalInputDelta=3.3081112688802676
New Minimum: -97.01624254459988 > -110.21122171825483
WOLFE (weak): th(258.53216280382605)=-110.21122171825483; dx=-0.09584300351312969 evalInputDelta=16.503090442535225
New Minimum: -110.21122171825483 > -165.10806872055073
END: th(1551.1929768229563)=-165.10806872055073; dx=-0.06205852843110179 evalInputDelta=71.39993744483112
Fitness changed from -93.70813127571961 to -165.10806872055073
Iteration 1 complete. Error: -165.10806872055073 Total: 6.1166; Orientation: 0.0163; Line Search: 4.2770
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1f8c15f7 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-165.10806872055073;dx=-0.13219980977911747
New Minimum: -165.10806872055073 > -282.3476248302147
END: th(3341.943960201201)=-282.3476248302147; dx=-0.04162003987882232 evalInputDelta=117.239556109664
Fitness changed from -165.10806872055073 to -282.3476248302147
Iteration 2 complete. Error: -282.3476248302147 Total: 1.7591; Orientation: 0.0159; Line Search: 1.1854
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 248d696b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-282.3476248302147;dx=-0.12795766270336661
New Minimum: -282.3476248302147 > -443.56318227047564
END: th(7200.000000000001)=-443.56318227047564; dx=-0.018802534374182327 evalInputDelta=161.2155574402609
Fitness changed from -282.3476248302147 to -443.56318227047564
Iteration 3 complete. Error: -443.56318227047564 Total: 1.7929; Orientation: 0.0159; Line Search: 1.1777
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3b7d6fa0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.002e+04, gradient 3.737e-01, dot -0.232; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -443.56318227047564, -282.3476248302147, -165.10806872055073, -93.70813127571961
LBFGS Accumulation History: 3 points
Removed measurement 3b7d6fa0 to history. Total: 3
th(0)=-443.56318227047564;dx=-0.13951085108315137
New Minimum: -443.56318227047564 > -469.898443120898
END: th(15511.929768229566)=-469.898443120898; dx=-0.0036476188318295836 evalInputDelta=26.335260850422344
Fitness changed from -443.56318227047564 to -469.898443120898
Iteration 4 complete. Error: -469.898443120898 Total: 2.0773; Orientation: 0.2911; Line Search: 1.2293
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5eebc17f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.976e+03, gradient 3.413e-01, dot -0.198; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -469.898443120898, -282.3476248302147, -165.10806872055073, -93.70813127571961
LBFGS Accumulation History: 3 points
Removed measurement 5eebc17f to history. Total: 3
th(0)=-469.898443120898;dx=-0.11644632691043562
Armijo: th(33419.43960201202)=-443.23477651911736; dx=5.458451231613547E-4 evalInputDelta=-26.66366660178062
New Minimum: -469.898443120898 > -682.5626227953283
END: th(16709.71980100601)=-682.5626227953283; dx=-0.011386216787752318 evalInputDelta=212.66417967443027
Fitness changed from -469.898443120898 to -682.5626227953283
Iteration 5 complete. Error: -682.5626227953283 Total: 3.0094; Orientation: 0.2784; Line Search: 1.7961
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4a1baaec to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.483e+04, gradient 3.331e-01, dot -0.207; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -682.5626227953283, -282.3476248302147, -165.10806872055073, -93.70813127571961
LBFGS Accumulation History: 3 points
Removed measurement 4a1baaec to history. Total: 3
th(0)=-682.5626227953283;dx=-0.11089671027863623
Armijo: th(36000.000000000015)=-617.2812490130503; dx=0.002351283753946248 evalInputDelta=-65.28137378227791
New Minimum: -682.5626227953283 > -978.8034087675696
END: th(18000.000000000007)=-978.8034087675696; dx=-0.014175136330491882 evalInputDelta=296.2407859722414
Fitness changed from -682.5626227953283 to -978.8034087675696
Iteration 6 complete. Error: -978.8034087675696 Total: 2.6934; Orientation: 0.2940; Line Search: 1.8169
Adding measurement 204fe57b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.881e+04, gradient 3.239e-01, dot -0.222; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -978.8034087675696, -282.3476248302147, -165.10806872055073, -93.70813127571961
LBFGS Accumulation History: 3 points
Removed measurement 204fe57b to history. Total: 3
th(0)=-978.8034087675696;dx=-0.10488127998472363
Armijo: th(38779.824420573925)=-830.8866581221563; dx=0.0026764398246918023 evalInputDelta=-147.91675064541334
New Minimum: -978.8034087675696 > -1256.033548394175
END: th(19389.912210286962)=-1256.033548394175; dx=-0.013846322398081867 evalInputDelta=277.2301396266055
Fitness changed from -978.8034087675696 to -1256.033548394175
Iteration 7 complete. Error: -1256.033548394175 Total: 2.6718; Orientation: 0.2873; Line Search: 1.8251
Adding measurement 6e6fe275 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.214e+04, gradient 3.328e-01, dot -0.228; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1256.033548394175, -282.3476248302147, -165.10806872055073, -93.70813127571961
LBFGS Accumulation History: 3 points
Removed measurement 6e6fe275 to history. Total: 3
th(0)=-1256.033548394175;dx=-0.11072286632590855
Armijo: th(41774.29950251503)=-934.0201190246116; dx=0.0037620497277542912 evalInputDelta=-322.0134293695635
New Minimum: -1256.033548394175 > -1463.7421843266382
END: th(20887.149751257515)=-1463.7421843266382; dx=-0.010641829288586245 evalInputDelta=207.70863593246304
Fitness changed from -1256.033548394175 to -1463.7421843266382
Iteration 8 complete. Error: -1463.7421843266382 Total: 2.9021; Orientation: 0.2986; Line Search: 2.0277
Adding measurement 5760d51f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.730e+05, gradient 3.331e-01, dot -0.228; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1463.7421843266382, -282.3476248302147, -165.10806872055073, -93.70813127571961
LBFGS Accumulation History: 3 points
Removed measurement 5760d51f to history. Total: 3
th(0)=-1463.7421843266382;dx=-0.11094580931060366
Armijo: th(45000.00000000002)=-1004.1242038372666; dx=0.005289952207790173 evalInputDelta=-459.6179804893716
New Minimum: -1463.7421843266382 > -1654.1856165039212
END: th(22500.00000000001)=-1654.1856165039212; dx=-0.009283464198225165 evalInputDelta=190.44343217728306
Fitness changed from -1463.7421843266382 to -1654.1856165039212
Iteration 9 complete. Error: -1654.1856165039212 Total: 2.6708; Orientation: 0.2722; Line Search: 1.8219
Adding measurement f89c1cd to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.472e+04, gradient 3.383e-01, dot 0.264; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1654.1856165039212;dx=-3986.9433008718893
Armijo: th(48474.78052571741)=90.88583875199811; dx=0.027688177580328946 evalInputDelta=-1745.0714552559193
Armijo: th(24237.390262858706)=80.54300653486473; dx=0.05497875344897585 evalInputDelta=-1734.7286230387858
Armijo: th(8079.130087619568)=58.31764261738965; dx=0.16234452154747603 evalInputDelta=-1712.5032591213107
Armijo: th(2019.782521904892)=19.595285550199378; dx=0.6319373515441968 evalInputDelta=-1673.7809020541206
Armijo: th(403.9565043809784)=-36.16715075293371; dx=3.0128101526196267 evalInputDelta=-1618.0184657509876
Armijo: th(67.3260840634964)=-123.17055646388462; dx=16.548899817378103 evalInputDelta=-1531.0150600400366
Armijo: th(9.618012009070913)=-297.6843749373637; dx=92.85530051765159 evalInputDelta=-1356.5012415665574
Armijo: th(1.2022515011338641)=-825.1646921092241; dx=334.7465343457373 evalInputDelta=-829.0209243946971
Armijo: th(0.1335835001259849)=-1585.6763971986834; dx=-241.56108750552156 evalInputDelta=-68.50921930523782
New Minimum: -1654.1856165039212 > -1655.2670302802767
END: th(0.013358350012598491)=-1655.2670302802767; dx=-1101.282305027446 evalInputDelta=1.0814137763554754
Fitness changed from -1654.1856165039212 to -1655.2670302802767
Iteration 10 complete. Error: -1655.2670302802767 Total: 7.7377; Orientation: 0.2928; Line Search: 6.8843
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 58354146 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 5.694e+05, gradient 3.366e-01, dot -0.209; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1655.2670302802767, -1654.1856165039212, -282.3476248302147, -165.10806872055073, -93.70813127571961
Rejected: LBFGS Orientation magnitude: 1.393e+06, gradient 3.366e-01, dot -0.224; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1655.2670302802767, -1654.1856165039212, -282.3476248302147, -165.10806872055073
LBFGS Accumulation History: 3 points
Removed measurement 58354146 to history. Total: 4
Removed measurement f89c1cd to history. Total: 3
th(0)=-1655.2670302802767;dx=-0.11327974664859942
New Minimum: -1655.2670302802767 > -1655.26872478849
END: th(0.028779692668730043)=-1655.26872478849; dx=-0.07784933285025601 evalInputDelta=0.0016945082134043332
Fitness changed from -1655.2670302802767 to -1655.26872478849
Iteration 11 complete. Error: -1655.26872478849 Total: 2.3537; Orientation: 0.6135; Line Search: 1.1622
Adding measurement 5f2f5406 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.791e+04, gradient 3.366e-01, dot 0.271; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
th(0)=-1655.26872478849;dx=-3458.7805700001672
Armijo: th(0.06200396825396829)=-1641.4071766667403; dx=-658.3352970045291 evalInputDelta=-13.86154812174982
Armijo: th(0.031001984126984145)=-1652.1277713008824; dx=-841.2629480985859 evalInputDelta=-3.140953487607703
Armijo: th(0.010333994708994714)=-1655.169401390616; dx=-990.8888688527176 evalInputDelta=-0.09932339787405908
New Minimum: -1655.26872478849 > -1655.3419005876303
END: th(0.0025834986772486786)=-1655.3419005876303; dx=-1058.5154732637861 evalInputDelta=0.07317579914024464
Fitness changed from -1655.26872478849 to -1655.3419005876303
Iteration 12 complete. Error: -1655.3419005876303 Total: 3.8759; Orientation: 0.2962; Line Search: 3.0207
Adding measurement 278869ea to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.264e+06, gradient 3.369e-01, dot -0.210; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1655.3419005876303, -1655.26872478849, -282.3476248302147, -165.10806872055073, -93.70813127571961
Rejected: LBFGS Orientation magnitude: 1.269e+07, gradient 3.369e-01, dot -0.221; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1655.3419005876303, -1655.26872478849, -282.3476248302147, -165.10806872055073
LBFGS Accumulation History: 3 points
Removed measurement 278869ea to history. Total: 4
Removed measurement 5f2f5406 to history. Total: 3
th(0)=-1655.3419005876303;dx=-0.11315843241629804
New Minimum: -1655.3419005876303 > -1655.3421224985798
END: th(0.005565979171916039)=-1655.3421224985798; dx=-0.07767498887282138 evalInputDelta=2.2191094944901124E-4
Fitness changed from -1655.3419005876303 to -1655.3421224985798
Iteration 13 complete. Error: -1655.3421224985798 Total: 2.4578; Orientation: 0.6668; Line Search: 1.2534
Adding measurement 45a10ba3 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.725e+04, gradient 3.368e-01, dot 0.272; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
th(0)=-1655.3421224985798;dx=-3406.7202854994994
Armijo: th(0.011991538611970854)=-1654.901963917973; dx=-951.2963459259785 evalInputDelta=-0.44015858060674873
Armijo: th(0.005995769305985427)=-1655.3140075396866; dx=-1010.5330112327822 evalInputDelta=-0.02811495889318394
New Minimum: -1655.3421224985798 > -1655.375893168472
END: th(0.001998589768661809)=-1655.375893168472; dx=-1047.2568763858405 evalInputDelta=0.03377066989219202
Fitness changed from -1655.3421224985798 to -1655.375893168472
Iteration 14 complete. Error: -1655.375893168472 Total: 3.5333; Orientation: 0.2789; Line Search: 2.6381
Adding measurement 51eb2d6e to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.438e+06, gradient 3.359e-01, dot -0.209; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1655.375893168472, -1655.3421224985798, -282.3476248302147, -165.10806872055073, -93.70813127571961
Rejected: LBFGS Orientation magnitude: 1.036e+07, gradient 3.359e-01, dot -0.220; [e3c4ee4a-80e4-4f5b-a911-f77fc685e2e9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1655.375893168472, -1655.3421224985798, -282.3476248302147, -165.10806872055073
LBFGS Accumulation History: 3 points
Removed measurement 51eb2d6e to history. Total: 4
Removed measurement 45a10ba3 to history. Total: 3
th(0)=-1655.375893168472;dx=-0.1128040715952416
New Minimum: -1655.375893168472 > -1655.376262968097
END: th(0.004305831128747798)=-1655.376262968097; dx=-0.07726080579292567 evalInputDelta=3.6979962510486075E-4
Fitness changed from -1655.375893168472 to -1655.376262968097
Iteration 15 complete. Error: -1655.376262968097 Total: 2.3560; Orientation: 0.6155; Line Search: 1.1658
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1655.376262968097 (> -Infinity) after 48.030s (< 5400.000s)
Returns
-1655.376262968097