BasicOptimizer.scala:89 executed in 54.70 seconds (1.257 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 209629146000700
Reset training subject: 209629893265200
Adding measurement 40f43cbd to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=10.09482674311014;dx=-7.53957755093432E-5
New Minimum: 10.09482674311014 > 10.094700976243972
WOLFE (weak): th(2.154434690031884)=10.094700976243972; dx=-7.53935737270883E-5 evalInputDelta=1.2576686616760924E-4
New Minimum: 10.094700976243972 > 10.094572633098059
WOLFE (weak): th(4.308869380063768)=10.094572633098059; dx=-7.539395484636058E-5 evalInputDelta=2.54110012081199E-4
New Minimum: 10.094572633098059 > 10.094067088245918
WOLFE (weak): th(12.926608140191302)=10.094067088245918; dx=-7.529151407683207E-5 evalInputDelta=7.596548642219147E-4
New Minimum: 10.094067088245918 > 10.091785504039391
WOLFE (weak): th(51.70643256076521)=10.091785504039391; dx=-7.544083779117694E-5 evalInputDelta=0.0030412390707486026
New Minimum: 10.091785504039391 > 10.079601541340434
WOLFE (weak): th(258.53216280382605)=10.079601541340434; dx=-7.55643569802041E-5 evalInputDelta=0.015225201769705876
New Minimum: 10.079601541340434 > 10.00240628221896
WOLFE (weak): th(1551.1929768229563)=10.00240628221896; dx=-7.679390557695081E-5 evalInputDelta=0.09242046089118006
New Minimum: 10.00240628221896 > 9.431107366845897
WOLFE (weak): th(10858.350837760694)=9.431107366845897; dx=-7.701551778191903E-5 evalInputDelta=0.6637193762642433
New Minimum: 9.431107366845897 > 6.4177571207686945
END: th(86866.80670208555)=6.4177571207686945; dx=-3.3586859750189004E-5 evalInputDelta=3.6770696223414454
Fitness changed from 10.09482674311014 to 6.4177571207686945
Iteration 1 complete. Error: 6.4177571207686945 Total: 9.1757; Orientation: 0.0156; Line Search: 6.9726
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4702e036 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=6.4177571207686945;dx=-9.54050861956145E-5
New Minimum: 6.4177571207686945 > 1.1582678183971122
END: th(187148.86177126726)=1.1582678183971122; dx=-2.342546346899467E-5 evalInputDelta=5.259489302371582
Fitness changed from 6.4177571207686945 to 1.1582678183971122
Iteration 2 complete. Error: 1.1582678183971122 Total: 2.2865; Orientation: 0.0185; Line Search: 1.5251
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 37434757 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=1.1582678183971122;dx=-1.3943822785031512E-4
New Minimum: 1.1582678183971122 > -1.4704216613291408
END: th(403200.00000000006)=-1.4704216613291408; dx=-6.336372713787761E-8 evalInputDelta=2.628689479726253
Fitness changed from 1.1582678183971122 to -1.4704216613291408
Iteration 3 complete. Error: -1.4704216613291408 Total: 2.2879; Orientation: 0.0161; Line Search: 1.4986
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5e10ab9b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.306e+03, gradient 1.255e-02, dot -0.159; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.4704216613291408, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 5e10ab9b to history. Total: 3
th(0)=-1.4704216613291408;dx=-1.5721911647597324E-4
Armijo: th(868668.0670208557)=4.133759518590253; dx=1.1801449436304505E-5 evalInputDelta=-5.604181179919394
New Minimum: -1.4704216613291408 > -6.286907624951819
WOLF (strong): th(434334.03351042786)=-6.286907624951819; dx=9.761622776701264E-7 evalInputDelta=4.816485963622679
New Minimum: -6.286907624951819 > -10.715038434302244
END: th(144778.01117014262)=-10.715038434302244; dx=-5.739226986355111E-5 evalInputDelta=9.244616772973103
Fitness changed from -1.4704216613291408 to -10.715038434302244
Iteration 4 complete. Error: -10.715038434302244 Total: 4.3542; Orientation: 0.3003; Line Search: 3.3606
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 14e7426b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.977e+03, gradient 1.178e-02, dot -0.156; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.715038434302244, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 14e7426b to history. Total: 3
th(0)=-10.715038434302244;dx=-1.3853861318773264E-4
New Minimum: -10.715038434302244 > -15.908847816752676
END: th(311914.76961877884)=-15.908847816752676; dx=-1.6423041102033196E-5 evalInputDelta=5.193809382450432
Fitness changed from -10.715038434302244 to -15.908847816752676
Iteration 5 complete. Error: -15.908847816752676 Total: 2.6624; Orientation: 0.3274; Line Search: 1.5656
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 31c4b635 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.718e+03, gradient 1.251e-02, dot -0.153; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.908847816752676, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 31c4b635 to history. Total: 3
th(0)=-15.908847816752676;dx=-1.5646586888280978E-4
Armijo: th(672000.0000000002)=-8.40009508974667; dx=1.1202051253959592E-5 evalInputDelta=-7.508752727006005
New Minimum: -15.908847816752676 > -20.96886508885685
END: th(336000.0000000001)=-20.96886508885685; dx=-9.24052812108956E-6 evalInputDelta=5.060017272104176
Fitness changed from -15.908847816752676 to -20.96886508885685
Iteration 6 complete. Error: -20.96886508885685 Total: 3.4493; Orientation: 0.2980; Line Search: 2.4628
Adding measurement 2646f3fe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.641e+03, gradient 1.233e-02, dot -0.149; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.96886508885685, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 2646f3fe to history. Total: 3
th(0)=-20.96886508885685;dx=-1.5177887429251234E-4
Armijo: th(723890.0558507132)=-10.536226213647126; dx=1.4621330903239859E-5 evalInputDelta=-10.432638875209726
New Minimum: -20.96886508885685 > -26.513216663527388
END: th(361945.0279253566)=-26.513216663527388; dx=-8.769691539119885E-6 evalInputDelta=5.544351574670536
Fitness changed from -20.96886508885685 to -26.513216663527388
Iteration 7 complete. Error: -26.513216663527388 Total: 3.2063; Orientation: 0.2976; Line Search: 2.2329
Adding measurement 3df17cfa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.678e+03, gradient 1.241e-02, dot -0.134; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.513216663527388, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 3df17cfa to history. Total: 3
th(0)=-26.513216663527388;dx=-1.5377922790943597E-4
Armijo: th(779786.9240469472)=-12.289664895074573; dx=1.7426296960530123E-5 evalInputDelta=-14.223551768452815
New Minimum: -26.513216663527388 > -31.601046997598928
END: th(389893.4620234736)=-31.601046997598928; dx=-6.556176250984482E-6 evalInputDelta=5.0878303340715405
Fitness changed from -26.513216663527388 to -31.601046997598928
Iteration 8 complete. Error: -31.601046997598928 Total: 3.5464; Orientation: 0.2984; Line Search: 2.5628
Adding measurement 42dc3b9b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.684e+03, gradient 1.262e-02, dot -0.124; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -31.601046997598928, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 42dc3b9b to history. Total: 3
th(0)=-31.601046997598928;dx=-1.5909850789626453E-4
Armijo: th(840000.0000000003)=-12.741330612071721; dx=2.102644634617703E-5 evalInputDelta=-18.859716385527207
New Minimum: -31.601046997598928 > -36.2299064759756
END: th(420000.0000000002)=-36.2299064759756; dx=-2.287398662911516E-6 evalInputDelta=4.628859478376675
Fitness changed from -31.601046997598928 to -36.2299064759756
Iteration 9 complete. Error: -36.2299064759756 Total: 3.3968; Orientation: 0.3121; Line Search: 2.3251
Adding measurement 23d168da to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.195e+04, gradient 1.274e-02, dot -0.117; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.2299064759756, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 23d168da to history. Total: 3
th(0)=-36.2299064759756;dx=-1.622443717745311E-4
Armijo: th(904862.5698133915)=-13.731892527433885; dx=2.3952650073401936E-5 evalInputDelta=-22.49801394854172
New Minimum: -36.2299064759756 > -40.32200708873353
END: th(452431.2849066958)=-40.32200708873353; dx=-1.7474066017982287E-8 evalInputDelta=4.092100612757925
Fitness changed from -36.2299064759756 to -40.32200708873353
Iteration 10 complete. Error: -40.32200708873353 Total: 3.5568; Orientation: 0.2987; Line Search: 2.5277
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 5576c35b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.375e+04, gradient 1.283e-02, dot -0.106; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -40.32200708873353, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 5576c35b to history. Total: 3
th(0)=-40.32200708873353;dx=-1.6454467780606795E-4
Armijo: th(974733.6550586841)=-14.15418789045102; dx=2.4016389832466067E-5 evalInputDelta=-26.16781919828251
New Minimum: -40.32200708873353 > -43.884363802470986
WOLF (strong): th(487366.82752934203)=-43.884363802470986; dx=8.071377847271506E-7 evalInputDelta=3.562356713737458
New Minimum: -43.884363802470986 > -50.41267359769844
END: th(162455.60917644735)=-50.41267359769844; dx=-7.722960039927233E-5 evalInputDelta=10.090666508964915
Fitness changed from -40.32200708873353 to -50.41267359769844
Iteration 11 complete. Error: -50.41267359769844 Total: 3.9788; Orientation: 0.2890; Line Search: 3.0166
Adding measurement 2f5bc423 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.755e+04, gradient 1.058e-02, dot -0.136; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -50.41267359769844, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 2f5bc423 to history. Total: 3
th(0)=-50.41267359769844;dx=-1.1175441946284381E-4
New Minimum: -50.41267359769844 > -56.45306416613482
END: th(350000.00000000023)=-56.45306416613482; dx=-3.6335209426820555E-5 evalInputDelta=6.040390568436379
Fitness changed from -50.41267359769844 to -56.45306416613482
Iteration 12 complete. Error: -56.45306416613482 Total: 2.8862; Orientation: 0.3063; Line Search: 1.8261
Adding measurement 3e26ab09 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.360e+04, gradient 1.099e-02, dot -0.135; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -56.45306416613482, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 3e26ab09 to history. Total: 3
th(0)=-56.45306416613482;dx=-1.2079816082142006E-4
Armijo: th(754052.1415111598)=-45.37914277357007; dx=3.832904777438813E-6 evalInputDelta=-11.073921392564756
New Minimum: -56.45306416613482 > -59.595251616399835
END: th(377026.0707555799)=-59.595251616399835; dx=-2.3451750538751377E-5 evalInputDelta=3.1421874502650127
Fitness changed from -56.45306416613482 to -59.595251616399835
Iteration 13 complete. Error: -59.595251616399835 Total: 3.1551; Orientation: 0.2850; Line Search: 2.1946
Adding measurement 521e1616 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.122e+04, gradient 1.156e-02, dot -0.125; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -59.595251616399835, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 521e1616 to history. Total: 3
th(0)=-59.595251616399835;dx=-1.3357948433641143E-4
Armijo: th(812278.0458822369)=-41.30149971589837; dx=1.3801403842644874E-5 evalInputDelta=-18.293751900501462
New Minimum: -59.595251616399835 > -61.63488812471479
END: th(406139.02294111846)=-61.63488812471479; dx=-1.193622443035731E-5 evalInputDelta=2.039636508314956
Fitness changed from -59.595251616399835 to -61.63488812471479
Iteration 14 complete. Error: -61.63488812471479 Total: 3.4253; Orientation: 0.3045; Line Search: 2.4318
Adding measurement 27ce7974 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.314e+04, gradient 1.236e-02, dot -0.112; [abf02e14-5620-4928-a0a5-a7aab95ddf40 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -61.63488812471479, 1.1582678183971122, 6.4177571207686945, 10.09482674311014
LBFGS Accumulation History: 3 points
Removed measurement 27ce7974 to history. Total: 3
th(0)=-61.63488812471479;dx=-1.527140907665027E-4
Armijo: th(875000.0000000007)=-36.17999833081312; dx=2.2267081896419004E-5 evalInputDelta=-25.45488979390167
New Minimum: -61.63488812471479 > -61.992343947811406
END: th(437500.00000000035)=-61.992343947811406; dx=-1.6192717659826644E-6 evalInputDelta=0.3574558230966147
Fitness changed from -61.63488812471479 to -61.992343947811406
Iteration 15 complete. Error: -61.992343947811406 Total: 3.3094; Orientation: 0.2909; Line Search: 2.3105
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -61.992343947811406 (> -Infinity) after 54.699s (< 5400.000s)
Returns
-61.992343947811406