BasicOptimizer.scala:89 executed in 11.21 seconds (0.799 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 145116105869300
Reset training subject: 145116234680300
Adding measurement 4e7955b1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.4130107540123882;dx=-5.227118168413279E-4
New Minimum: -1.4130107540123882 > -1.4145793040233134
WOLFE (weak): th(2.154434690031884)=-1.4145793040233134; dx=-5.036154824044497E-4 evalInputDelta=0.0015685500109252626
New Minimum: -1.4145793040233134 > -1.4161422972944808
WOLFE (weak): th(4.308869380063768)=-1.4161422972944808; dx=-6.147527570409205E-4 evalInputDelta=0.0031315432820926414
New Minimum: -1.4161422972944808 > -1.4224134102073918
WOLFE (weak): th(12.926608140191302)=-1.4224134102073918; dx=-4.7842898786720637E-4 evalInputDelta=0.009402656195003578
New Minimum: -1.4224134102073918 > -1.45124551277755
WOLFE (weak): th(51.70643256076521)=-1.45124551277755; dx=-5.162669518863402E-4 evalInputDelta=0.038234758765161914
New Minimum: -1.45124551277755 > -1.6086139545317266
WOLFE (weak): th(258.53216280382605)=-1.6086139545317266; dx=-4.924667432951581E-4 evalInputDelta=0.19560320051933844
New Minimum: -1.6086139545317266 > -2.6265917429172463
WOLFE (weak): th(1551.1929768229563)=-2.6265917429172463; dx=-4.837719113547277E-4 evalInputDelta=1.213580988904858
New Minimum: -2.6265917429172463 > -3.336527470529685
END: th(10858.350837760694)=-3.336527470529685; dx=-3.888063836811899E-5 evalInputDelta=1.9235167165172966
Fitness changed from -1.4130107540123882 to -3.336527470529685
Iteration 1 complete. Error: -3.336527470529685 Total: 1.2554; Orientation: 0.0015; Line Search: 0.9051
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 34a51b53 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.336527470529685;dx=-8.115443135527478E-4
New Minimum: -3.336527470529685 > -14.422025938354318
END: th(23393.607721408407)=-14.422025938354318; dx=-1.8113306091949762E-4 evalInputDelta=11.085498467824634
Fitness changed from -3.336527470529685 to -14.422025938354318
Iteration 2 complete. Error: -14.422025938354318 Total: 0.3500; Orientation: 0.0015; Line Search: 0.2289
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2e65b03b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-14.422025938354318;dx=-0.002633085332369268
New Minimum: -14.422025938354318 > -93.45666621767568
END: th(50400.00000000001)=-93.45666621767568; dx=-3.413540160310212E-4 evalInputDelta=79.03464027932135
Fitness changed from -14.422025938354318 to -93.45666621767568
Iteration 3 complete. Error: -93.45666621767568 Total: 0.3563; Orientation: 0.0015; Line Search: 0.2328
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3636abbd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.120e+04, gradient 9.636e-02, dot -0.130; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -93.45666621767568, -14.422025938354318, -3.336527470529685, -1.4130107540123882
LBFGS Accumulation History: 3 points
Removed measurement 3636abbd to history. Total: 3
th(0)=-93.45666621767568;dx=-0.008951787781963556
Armijo: th(108583.50837760697)=-61.7575779431215; dx=2.5001803607120697E-4 evalInputDelta=-31.699088274554178
New Minimum: -93.45666621767568 > -138.7939098837019
WOLF (strong): th(54291.75418880348)=-138.7939098837019; dx=6.095663833363336E-4 evalInputDelta=45.337243666026225
New Minimum: -138.7939098837019 > -211.54901105601098
END: th(18097.251396267828)=-211.54901105601098; dx=-7.848691869425503E-4 evalInputDelta=118.09234483833531
Fitness changed from -93.45666621767568 to -211.54901105601098
Iteration 4 complete. Error: -211.54901105601098 Total: 0.6070; Orientation: 0.0274; Line Search: 0.4591
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 78afac98 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.780e+03, gradient 8.796e-02, dot -0.110; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -211.54901105601098, -14.422025938354318, -3.336527470529685, -1.4130107540123882
LBFGS Accumulation History: 3 points
Removed measurement 78afac98 to history. Total: 3
th(0)=-211.54901105601098;dx=-0.0073868488383426936
Armijo: th(38989.346202347355)=-209.03157520089363; dx=3.5686472573926537E-4 evalInputDelta=-2.5174358551173555
New Minimum: -211.54901105601098 > -270.9949947581552
END: th(19494.673101173677)=-270.9949947581552; dx=-9.235799685903234E-4 evalInputDelta=59.44598370214422
Fitness changed from -211.54901105601098 to -270.9949947581552
Iteration 5 complete. Error: -270.9949947581552 Total: 0.5134; Orientation: 0.0318; Line Search: 0.3523
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 32cfe458 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.828e+04, gradient 9.820e-02, dot -0.157; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -270.9949947581552, -14.422025938354318, -3.336527470529685, -1.4130107540123882
LBFGS Accumulation History: 3 points
Removed measurement 32cfe458 to history. Total: 3
th(0)=-270.9949947581552;dx=-0.009335035852855951
New Minimum: -270.9949947581552 > -313.164144216459
WOLF (strong): th(42000.000000000015)=-313.164144216459; dx=4.129871810258761E-4 evalInputDelta=42.16914945830382
New Minimum: -313.164144216459 > -335.2947499816882
END: th(21000.000000000007)=-335.2947499816882; dx=-6.100069475400014E-4 evalInputDelta=64.299755223533
Fitness changed from -270.9949947581552 to -335.2947499816882
Iteration 6 complete. Error: -335.2947499816882 Total: 0.5107; Orientation: 0.0323; Line Search: 0.3511
Adding measurement 2cbf7bb6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.638e+04, gradient 9.559e-02, dot -0.166; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -335.2947499816882, -14.422025938354318, -3.336527470529685, -1.4130107540123882
LBFGS Accumulation History: 3 points
Removed measurement 2cbf7bb6 to history. Total: 3
th(0)=-335.2947499816882;dx=-0.00862613532520179
Armijo: th(45243.12849066957)=-255.33433344307056; dx=0.001273036156184812 evalInputDelta=-79.96041653861764
New Minimum: -335.2947499816882 > -361.67383089417734
END: th(22621.564245334786)=-361.67383089417734; dx=-2.1500684691207702E-4 evalInputDelta=26.379080912489144
Fitness changed from -335.2947499816882 to -361.67383089417734
Iteration 7 complete. Error: -361.67383089417734 Total: 0.5156; Orientation: 0.0293; Line Search: 0.3537
Adding measurement 5ab7c993 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.614e+06, gradient 1.022e-01, dot -0.224; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -361.67383089417734, -14.422025938354318, -3.336527470529685, -1.4130107540123882
LBFGS Accumulation History: 3 points
Removed measurement 5ab7c993 to history. Total: 3
th(0)=-361.67383089417734;dx=-0.009660191660913885
Armijo: th(48736.6827529342)=-359.3084521737741; dx=0.001149042252829251 evalInputDelta=-2.365378720403214
New Minimum: -361.67383089417734 > -440.9705783132776
WOLF (strong): th(24368.3413764671)=-440.9705783132776; dx=1.1061668596273379E-6 evalInputDelta=79.29674741910026
END: th(8122.780458822366)=-423.84848535097535; dx=-0.0034843700936586877 evalInputDelta=62.17465445679801
Fitness changed from -361.67383089417734 to -440.9705783132776
Iteration 8 complete. Error: -440.9705783132776 Total: 0.6025; Orientation: 0.0313; Line Search: 0.4582
Adding measurement 20a96f38 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.009e+06, gradient 9.077e-02, dot -0.164; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -440.9705783132776, -14.422025938354318, -3.336527470529685, -1.4130107540123882
LBFGS Accumulation History: 3 points
Removed measurement 20a96f38 to history. Total: 3
th(0)=-440.9705783132776;dx=-0.007782571429251293
New Minimum: -440.9705783132776 > -483.7020430601886
END: th(17500.000000000007)=-483.7020430601886; dx=-0.0012216574710225063 evalInputDelta=42.731464746911
Fitness changed from -440.9705783132776 to -483.7020430601886
Iteration 9 complete. Error: -483.7020430601886 Total: 0.3703; Orientation: 0.0265; Line Search: 0.2308
Adding measurement 3378c466 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.110e+03, gradient 8.298e-02, dot 0.652; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-483.7020430601886;dx=-378.80110862860454
Armijo: th(37702.607075557986)=-175.7708853610791; dx=0.0028688990034773662 evalInputDelta=-307.93115769910946
Armijo: th(18851.303537778993)=-175.84273120579485; dx=0.0054471409806900704 evalInputDelta=-307.85931185439375
Armijo: th(6283.767845926331)=-176.31627838743495; dx=0.016101583734723403 evalInputDelta=-307.3857646727537
Armijo: th(1570.9419614815827)=-176.22260107294093; dx=0.0631193340713884 evalInputDelta=-307.4794419872477
Armijo: th(314.1883922963165)=-176.69905044115686; dx=0.3556338861869579 evalInputDelta=-307.0029926190317
Armijo: th(52.364732049386085)=-181.5647728520715; dx=1.954656031720329 evalInputDelta=-302.1372702081171
Armijo: th(7.480676007055155)=-250.74800590960163; dx=15.344810232392367 evalInputDelta=-232.95403715058697
New Minimum: -483.7020430601886 > -500.70890722372343
END: th(0.9350845008818943)=-500.70890722372343; dx=-11.24510570044329 evalInputDelta=17.00686416353483
Fitness changed from -483.7020430601886 to -500.70890722372343
Iteration 10 complete. Error: -500.70890722372343 Total: 1.1712; Orientation: 0.0273; Line Search: 1.0329
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7fb62862 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.215e+05, gradient 1.012e-01, dot -0.063; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -500.70890722372343, -483.7020430601886, -14.422025938354318, -3.336527470529685, -1.4130107540123882
Rejected: LBFGS Orientation magnitude: 1.956e+05, gradient 1.012e-01, dot -0.184; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -500.70890722372343, -483.7020430601886, -14.422025938354318, -3.336527470529685
LBFGS Accumulation History: 3 points
Removed measurement 7fb62862 to history. Total: 4
Removed measurement 3378c466 to history. Total: 3
th(0)=-500.70890722372343;dx=-0.009744212285518972
New Minimum: -500.70890722372343 > -500.72811558860934
END: th(2.0145784868111027)=-500.72811558860934; dx=-0.0064769481989115215 evalInputDelta=0.019208364885912488
Fitness changed from -500.70890722372343 to -500.72811558860934
Iteration 11 complete. Error: -500.72811558860934 Total: 0.4098; Orientation: 0.0612; Line Search: 0.2256
Adding measurement 30795d03 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.306e+03, gradient 9.947e-02, dot 0.162; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
th(0)=-500.72811558860934;dx=-80.60202765641549
Armijo: th(4.3402777777777795)=-93.79862970908843; dx=-12.077491924880793 evalInputDelta=-406.92948587952094
Armijo: th(2.1701388888888897)=-113.80450139483136; dx=21.97876457321067 evalInputDelta=-386.923614193778
Armijo: th(0.7233796296296299)=-411.258074042196; dx=113.79934013756474 evalInputDelta=-89.47004154641337
New Minimum: -500.72811558860934 > -502.1673313076441
END: th(0.18084490740740747)=-502.1673313076441; dx=-19.51205277475369 evalInputDelta=1.4392157190347348
Fitness changed from -500.72811558860934 to -502.1673313076441
Iteration 12 complete. Error: -502.1673313076441 Total: 0.7344; Orientation: 0.0267; Line Search: 0.5975
Adding measurement a4de299 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.381e+05, gradient 9.033e-02, dot -0.126; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -502.1673313076441, -500.72811558860934, -14.422025938354318, -3.336527470529685, -1.4130107540123882
Rejected: LBFGS Orientation magnitude: 4.715e+05, gradient 9.033e-02, dot -0.195; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -502.1673313076441, -500.72811558860934, -14.422025938354318, -3.336527470529685
LBFGS Accumulation History: 3 points
Removed measurement a4de299 to history. Total: 4
Removed measurement 30795d03 to history. Total: 3
th(0)=-502.1673313076441;dx=-0.007865473430151898
New Minimum: -502.1673313076441 > -502.17180187714865
END: th(0.38961854203412266)=-502.17180187714865; dx=-0.006939640359851696 evalInputDelta=0.0044705695045763605
Fitness changed from -502.1673313076441 to -502.17180187714865
Iteration 13 complete. Error: -502.17180187714865 Total: 0.4145; Orientation: 0.0672; Line Search: 0.2303
Adding measurement 2a323078 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.063e+04, gradient 9.100e-02, dot 0.212; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
th(0)=-502.17180187714865;dx=-363.50562969886414
Armijo: th(0.8394077028379595)=-364.65673569233235; dx=92.77896850319891 evalInputDelta=-137.5150661848163
Armijo: th(0.41970385141897976)=-440.77517435490506; dx=110.29835301541621 evalInputDelta=-61.39662752224359
Armijo: th(0.13990128380632658)=-501.8020925061861; dx=100.43791582314518 evalInputDelta=-0.3697093709625392
New Minimum: -502.17180187714865 > -519.5123949147612
END: th(0.034975320951581644)=-519.5123949147612; dx=-161.8885531962663 evalInputDelta=17.340593037612564
Fitness changed from -502.17180187714865 to -519.5123949147612
Iteration 14 complete. Error: -519.5123949147612 Total: 0.7519; Orientation: 0.0260; Line Search: 0.5935
Adding measurement 7d54d22e to history. Total: 4
Accepted: LBFGS Orientation magnitude: 3.327e+03, gradient 9.425e-02, dot 0.147; [029a204b-89fc-44a7-98ce-53ef58990188 = 1.000/1.000e+00]
th(0)=-519.5123949147612;dx=-57.74645034265395
Armijo: th(0.07535204475308645)=-515.1307624341996; dx=38.93539075236906 evalInputDelta=-4.3816324805616205
Armijo: th(0.037676022376543224)=-517.3278278349819; dx=37.88229203646755 evalInputDelta=-2.184567079779299
Armijo: th(0.012558674125514408)=-518.7852887415249; dx=41.04195273981039 evalInputDelta=-0.7271061732362796
Armijo: th(0.003139668531378602)=-519.3307081218647; dx=38.65778612853206 evalInputDelta=-0.18168679289647116
Armijo: th(6.279337062757204E-4)=-519.4759807562349; dx=38.214691493953474 evalInputDelta=-0.0364141585263269
Armijo: th(1.0465561771262008E-4)=-519.5063677631097; dx=42.24643313530433 evalInputDelta=-0.006027151651551321
Armijo: th(1.4950802530374296E-5)=-519.5115306482529; dx=39.00542015299499 evalInputDelta=-8.642665083016254E-4
Armijo: th(1.868850316296787E-6)=-519.512303939407; dx=37.244635345725484 evalInputDelta=-9.09753541691316E-5
Armijo: th(2.0765003514408746E-7)=-519.512326683247; dx=40.009713044030335 evalInputDelta=-6.823151420576323E-5
Armijo: th(2.0765003514408746E-8)=-519.5123949147612; dx=36.80413664710003 evalInputDelta=0.0
Armijo: th(1.887727592218977E-9)=-519.512326683247; dx=41.278678247923125 evalInputDelta=-6.823151420576323E-5
WOLF (strong): th(1.5731063268491472E-10)=-519.5123949147612; dx=39.95797242977906 evalInputDelta=0.0
MIN ALPHA (1.2100817898839594E-11): th(0.0)=-519.5123949147612
Fitness changed from -519.5123949147612 to -519.5123949147612
Static Iteration Total: 2.6454; Orientation: 0.0367; Line Search: 2.4955
Iteration 15 failed. Error: -519.5123949147612
Previous Error: 0.0 -> -519.5123949147612
Retrying iteration 15
Final threshold in iteration 15: -519.5123949147612 (> -Infinity) after 11.208s (< 5400.000s)
Returns
-519.5123949147612