BasicOptimizer.scala:89 executed in 74.78 seconds (1.648 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139107030871700
Reset training subject: 139107933576200
Adding measurement 5126363 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-16.261477458428722;dx=-4.3316410620883826E-4
New Minimum: -16.261477458428722 > -16.26239003492179
WOLFE (weak): th(2.154434690031884)=-16.26239003492179; dx=-4.099654198664899E-4 evalInputDelta=9.125764930679736E-4
New Minimum: -16.26239003492179 > -16.263299731628248
WOLFE (weak): th(4.308869380063768)=-16.263299731628248; dx=-4.1022850473573135E-4 evalInputDelta=0.0018222731995258812
New Minimum: -16.263299731628248 > -16.266948755919643
WOLFE (weak): th(12.926608140191302)=-16.266948755919643; dx=-4.0885135701508054E-4 evalInputDelta=0.00547129749092079
New Minimum: -16.266948755919643 > -16.283386649236274
WOLFE (weak): th(51.70643256076521)=-16.283386649236274; dx=-4.113587992701207E-4 evalInputDelta=0.021909190807551937
New Minimum: -16.283386649236274 > -16.37125510403083
WOLFE (weak): th(258.53216280382605)=-16.37125510403083; dx=-4.1012839674215795E-4 evalInputDelta=0.10977764560210801
New Minimum: -16.37125510403083 > -16.928400564594615
WOLFE (weak): th(1551.1929768229563)=-16.928400564594615; dx=-4.17247296688625E-4 evalInputDelta=0.6669231061658927
New Minimum: -16.928400564594615 > -21.168847982244742
WOLFE (weak): th(10858.350837760694)=-21.168847982244742; dx=-4.411197787257548E-4 evalInputDelta=4.90737052381602
New Minimum: -21.168847982244742 > -44.840178522058004
END: th(86866.80670208555)=-44.840178522058004; dx=-2.0819300448707902E-4 evalInputDelta=28.578701063629282
Fitness changed from -16.261477458428722 to -44.840178522058004
Iteration 1 complete. Error: -44.840178522058004 Total: 12.3136; Orientation: 0.0212; Line Search: 8.8007
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6c55e918 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-44.840178522058004;dx=-8.922474065628833E-4
New Minimum: -44.840178522058004 > -84.05582204109297
END: th(187148.86177126726)=-84.05582204109297; dx=-1.1041057081442075E-4 evalInputDelta=39.21564351903496
Fitness changed from -44.840178522058004 to -84.05582204109297
Iteration 2 complete. Error: -84.05582204109297 Total: 2.7101; Orientation: 0.0150; Line Search: 1.7611
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3bc694b4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-84.05582204109297;dx=-0.0010729011664531649
New Minimum: -84.05582204109297 > -120.7674834812747
WOLF (strong): th(403200.00000000006)=-120.7674834812747; dx=2.38961332482532E-5 evalInputDelta=36.71166144018173
New Minimum: -120.7674834812747 > -147.413911735229
END: th(201600.00000000003)=-147.413911735229; dx=-6.450911067687774E-5 evalInputDelta=63.35808969413603
Fitness changed from -84.05582204109297 to -147.413911735229
Iteration 3 complete. Error: -147.413911735229 Total: 3.8116; Orientation: 0.0145; Line Search: 2.8306
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5bc2dffc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.564e+04, gradient 3.509e-02, dot -0.237; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -147.413911735229, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 5bc2dffc to history. Total: 3
th(0)=-147.413911735229;dx=-0.0012293650862220723
New Minimum: -147.413911735229 > -163.2387317913245
WOLF (strong): th(434334.03351042786)=-163.2387317913245; dx=1.0926244740178214E-4 evalInputDelta=15.824820056095518
New Minimum: -163.2387317913245 > -223.76768992476707
WOLF (strong): th(217167.01675521393)=-223.76768992476707; dx=2.0849173225417656E-5 evalInputDelta=76.35377818953808
END: th(72389.00558507131)=-213.29231343200289; dx=-5.147012369092832E-4 evalInputDelta=65.87840169677389
Fitness changed from -147.413911735229 to -223.76768992476707
Iteration 4 complete. Error: -223.76768992476707 Total: 5.4297; Orientation: 0.2841; Line Search: 3.8330
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3ef3cc5d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.324e+04, gradient 3.612e-02, dot -0.219; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -223.76768992476707, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 3ef3cc5d to history. Total: 3
th(0)=-223.76768992476707;dx=-0.0013028280418415806
New Minimum: -223.76768992476707 > -309.6885849934296
END: th(155957.38480938942)=-309.6885849934296; dx=-8.985749939612205E-5 evalInputDelta=85.9208950686625
Fitness changed from -223.76768992476707 to -309.6885849934296
Iteration 5 complete. Error: -309.6885849934296 Total: 3.0681; Orientation: 0.2905; Line Search: 1.8327
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 15817003 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.002e+04, gradient 3.427e-02, dot -0.224; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -309.6885849934296, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 15817003 to history. Total: 3
th(0)=-309.6885849934296;dx=-0.0011659803967660207
New Minimum: -309.6885849934296 > -310.83177168097194
WOLF (strong): th(336000.0000000001)=-310.83177168097194; dx=1.7628026776757747E-4 evalInputDelta=1.143186687542368
New Minimum: -310.83177168097194 > -371.29700416623297
END: th(168000.00000000006)=-371.29700416623297; dx=-9.49722209175848E-6 evalInputDelta=61.608419172803394
Fitness changed from -309.6885849934296 to -371.29700416623297
Iteration 6 complete. Error: -371.29700416623297 Total: 4.2164; Orientation: 0.2848; Line Search: 2.9920
Adding measurement 8207375 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.033e+05, gradient 3.584e-02, dot -0.254; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -371.29700416623297, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 8207375 to history. Total: 3
th(0)=-371.29700416623297;dx=-0.0012779443383463492
Armijo: th(361945.0279253566)=-331.79478675424144; dx=2.0416521679244591E-4 evalInputDelta=-39.50221741199152
New Minimum: -371.29700416623297 > -412.5845554782028
WOLF (strong): th(180972.5139626783)=-412.5845554782028; dx=1.2752072393380738E-4 evalInputDelta=41.28755131196982
New Minimum: -412.5845554782028 > -421.0768594926052
END: th(60324.17132089276)=-421.0768594926052; dx=-4.8340579918204835E-4 evalInputDelta=49.77985532637223
Fitness changed from -371.29700416623297 to -421.0768594926052
Iteration 7 complete. Error: -421.0768594926052 Total: 6.1994; Orientation: 0.6007; Line Search: 4.7228
Adding measurement 60f34e26 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.795e+05, gradient 2.911e-02, dot -0.318; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -421.0768594926052, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 60f34e26 to history. Total: 3
th(0)=-421.0768594926052;dx=-8.451506231606846E-4
New Minimum: -421.0768594926052 > -465.88808875859644
END: th(129964.48734115784)=-465.88808875859644; dx=-2.698508289547603E-4 evalInputDelta=44.811229265991244
Fitness changed from -421.0768594926052 to -465.88808875859644
Iteration 8 complete. Error: -465.88808875859644 Total: 2.8848; Orientation: 0.2940; Line Search: 1.7729
Adding measurement b7d7d00 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.774e+05, gradient 3.048e-02, dot -0.327; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -465.88808875859644, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement b7d7d00 to history. Total: 3
th(0)=-465.88808875859644;dx=-9.269958811591921E-4
New Minimum: -465.88808875859644 > -480.2846660933071
WOLF (strong): th(280000.0000000001)=-480.2846660933071; dx=1.784161488185468E-5 evalInputDelta=14.396577334710685
New Minimum: -480.2846660933071 > -499.4681437766196
END: th(140000.00000000006)=-499.4681437766196; dx=-1.1826571064596686E-4 evalInputDelta=33.58005501802319
Fitness changed from -465.88808875859644 to -499.4681437766196
Iteration 9 complete. Error: -499.4681437766196 Total: 3.9911; Orientation: 0.2834; Line Search: 2.8623
Adding measurement 1399e007 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.535e+05, gradient 3.311e-02, dot -0.304; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -499.4681437766196, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 1399e007 to history. Total: 3
th(0)=-499.4681437766196;dx=-0.0010943998348177616
Armijo: th(301620.8566044639)=-476.0398972220213; dx=1.5554653261153653E-4 evalInputDelta=-23.42824655459833
New Minimum: -499.4681437766196 > -528.4256899672105
WOLF (strong): th(150810.42830223194)=-528.4256899672105; dx=6.0714167414101233E-5 evalInputDelta=28.957546190590847
New Minimum: -528.4256899672105 > -529.7570218820018
END: th(50270.142767410645)=-529.7570218820018; dx=-3.9782803015569297E-4 evalInputDelta=30.288878105382196
Fitness changed from -499.4681437766196 to -529.7570218820018
Iteration 10 complete. Error: -529.7570218820018 Total: 4.7268; Orientation: 0.2794; Line Search: 3.6033
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 3f442acd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.654e+06, gradient 2.758e-02, dot -0.367; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -529.7570218820018, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 3f442acd to history. Total: 3
th(0)=-529.7570218820018;dx=-7.562415050472234E-4
New Minimum: -529.7570218820018 > -558.2096767753444
END: th(108303.7394509649)=-558.2096767753444; dx=-2.5599818337935647E-4 evalInputDelta=28.452654893342583
Fitness changed from -529.7570218820018 to -558.2096767753444
Iteration 11 complete. Error: -558.2096767753444 Total: 2.9739; Orientation: 0.2857; Line Search: 1.7282
Adding measurement 605edf3e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.349e+07, gradient 2.898e-02, dot -0.359; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -558.2096767753444, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 605edf3e to history. Total: 3
th(0)=-558.2096767753444;dx=-8.357439080466809E-4
New Minimum: -558.2096767753444 > -577.2209398067818
END: th(233333.3333333335)=-577.2209398067818; dx=-6.0015603550613744E-5 evalInputDelta=19.011263031437352
Fitness changed from -558.2096767753444 to -577.2209398067818
Iteration 12 complete. Error: -577.2209398067818 Total: 3.0170; Orientation: 0.3186; Line Search: 1.8548
Adding measurement 5c49a79e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.371e+07, gradient 3.538e-02, dot -0.277; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -577.2209398067818, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 5c49a79e to history. Total: 3
th(0)=-577.2209398067818;dx=-0.0012492921867435151
Armijo: th(502701.42767410656)=-439.76783511534677; dx=2.0473158611197491E-4 evalInputDelta=-137.453104691435
Armijo: th(251350.71383705328)=-563.1775565508393; dx=2.8727177279659177E-4 evalInputDelta=-14.043383255942445
New Minimum: -577.2209398067818 > -623.8335978903804
END: th(83783.57127901776)=-623.8335978903804; dx=-2.2714279179551425E-4 evalInputDelta=46.6126580835986
Fitness changed from -577.2209398067818 to -623.8335978903804
Iteration 13 complete. Error: -623.8335978903804 Total: 4.8716; Orientation: 0.2709; Line Search: 3.4993
Adding measurement 43f38cae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.317e+04, gradient 2.907e-02, dot -0.123; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Orientation rejected. Popping history element from -623.8335978903804, -84.05582204109297, -44.840178522058004, -16.261477458428722
LBFGS Accumulation History: 3 points
Removed measurement 43f38cae to history. Total: 3
th(0)=-623.8335978903804;dx=-8.397940408434714E-4
New Minimum: -623.8335978903804 > -650.7120466716324
END: th(180506.23241827489)=-650.7120466716324; dx=-7.692454724664852E-5 evalInputDelta=26.87844878125202
Fitness changed from -623.8335978903804 to -650.7120466716324
Iteration 14 complete. Error: -650.7120466716324 Total: 2.8265; Orientation: 0.2787; Line Search: 1.7156
Adding measurement 5d1c5d5a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.034e+04, gradient 3.351e-02, dot 0.567; [a444bc20-9623-47d3-b516-d4a171a8bdff = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-650.7120466716324;dx=-386.10496688521005
Armijo: th(388888.8888888892)=-79.80827321678079; dx=2.2557411840833758E-4 evalInputDelta=-570.9037734548516
Armijo: th(194444.4444444446)=-79.80982258354828; dx=4.519455339779422E-4 evalInputDelta=-570.902224088084
Armijo: th(64814.81481481487)=-79.80305679421986; dx=0.001352308274027333 evalInputDelta=-570.9089898774125
Armijo: th(16203.703703703717)=-79.80307524044491; dx=0.005426104372780039 evalInputDelta=-570.9089714311874
Armijo: th(3240.7407407407436)=-80.04609206606274; dx=0.027030044675866245 evalInputDelta=-570.6659546055696
Armijo: th(540.1234567901239)=-81.35549858930497; dx=0.1645819469344534 evalInputDelta=-569.3565480823274
Armijo: th(77.16049382716055)=-88.39975026690288; dx=1.252560853024658 evalInputDelta=-562.3122964047295
Armijo: th(9.645061728395069)=-154.3121106000763; dx=15.137267676801638 evalInputDelta=-496.3999360715561
Armijo: th(1.0716735253772298)=-589.5756262343145; dx=52.611004041220674 evalInputDelta=-61.13642043731784
New Minimum: -650.7120466716324 > -667.7352849380803
END: th(0.10716735253772298)=-667.7352849380803; dx=-173.52094211766013 evalInputDelta=17.023238266447947
Fitness changed from -650.7120466716324 to -667.7352849380803
Iteration 15 complete. Error: -667.7352849380803 Total: 11.5975; Orientation: 0.2908; Line Search: 10.4841
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -667.7352849380803 (> -Infinity) after 74.774s (< 5400.000s)
Returns
-667.7352849380803