BasicOptimizer.scala:89 executed in 62.87 seconds (0.805 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 153133421651700
Reset training subject: 153134338277300
Adding measurement 56996b22 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-78.28538225121437;dx=-0.010170705387282084
New Minimum: -78.28538225121437 > -78.31201054815124
WOLFE (weak): th(2.154434690031884)=-78.31201054815124; dx=-0.01018164838978324 evalInputDelta=0.026628296936863194
New Minimum: -78.31201054815124 > -78.33862356083525
WOLFE (weak): th(4.308869380063768)=-78.33862356083525; dx=-0.010168918168649718 evalInputDelta=0.05324130962087281
New Minimum: -78.33862356083525 > -78.44519073352267
WOLFE (weak): th(12.926608140191302)=-78.44519073352267; dx=-0.01020285362665594 evalInputDelta=0.1598084823082928
New Minimum: -78.44519073352267 > -78.92734557114062
WOLFE (weak): th(51.70643256076521)=-78.92734557114062; dx=-0.010283278150682557 evalInputDelta=0.6419633199262478
New Minimum: -78.92734557114062 > -81.56720831716781
WOLFE (weak): th(258.53216280382605)=-81.56720831716781; dx=-0.010624877686763044 evalInputDelta=3.281826065953439
New Minimum: -81.56720831716781 > -100.11548507380114
WOLFE (weak): th(1551.1929768229563)=-100.11548507380114; dx=-0.012273341783503323 evalInputDelta=21.830102822586767
New Minimum: -100.11548507380114 > -246.91128689015008
WOLFE (weak): th(10858.350837760694)=-246.91128689015008; dx=-0.009807986211058826 evalInputDelta=168.6259046389357
New Minimum: -246.91128689015008 > -352.5919095604719
END: th(86866.80670208555)=-352.5919095604719; dx=-2.922820270599335E-4 evalInputDelta=274.3065273092575
Fitness changed from -78.28538225121437 to -352.5919095604719
Iteration 1 complete. Error: -352.5919095604719 Total: 10.4665; Orientation: 0.0148; Line Search: 7.8970
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 70dd3c20 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-352.5919095604719;dx=-0.03234291313348191
New Minimum: -352.5919095604719 > -417.1079933602328
WOLF (strong): th(187148.86177126726)=-417.1079933602328; dx=2.1523040158241992E-4 evalInputDelta=64.5160837997609
New Minimum: -417.1079933602328 > -680.6209940194323
WOLF (strong): th(93574.43088563363)=-680.6209940194323; dx=5.749270490597977E-4 evalInputDelta=328.02908445896037
New Minimum: -680.6209940194323 > -920.423238244233
END: th(31191.476961877877)=-920.423238244233; dx=-0.0025561511825203187 evalInputDelta=567.8313286837611
Fitness changed from -352.5919095604719 to -920.423238244233
Iteration 2 complete. Error: -920.423238244233 Total: 4.2734; Orientation: 0.0143; Line Search: 3.3389
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5ef72f82 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-920.423238244233;dx=-0.037106143770434405
New Minimum: -920.423238244233 > -1258.717452409396
WOLF (strong): th(67200.00000000001)=-1258.717452409396; dx=0.003371942089325291 evalInputDelta=338.294214165163
New Minimum: -1258.717452409396 > -1574.6562323250114
END: th(33600.00000000001)=-1574.6562323250114; dx=-0.0022837519872735395 evalInputDelta=654.2329940807784
Fitness changed from -920.423238244233 to -1574.6562323250114
Iteration 3 complete. Error: -1574.6562323250114 Total: 3.4363; Orientation: 0.0144; Line Search: 2.5248
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 20fa1d89 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.278e+04, gradient 1.784e-01, dot -0.338; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1574.6562323250114, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 20fa1d89 to history. Total: 3
th(0)=-1574.6562323250114;dx=-0.03175045585822296
New Minimum: -1574.6562323250114 > -1823.6871305133454
WOLF (strong): th(72389.00558507131)=-1823.6871305133454; dx=0.004696942847083605 evalInputDelta=249.03089818833405
New Minimum: -1823.6871305133454 > -2125.7205837080282
END: th(36194.502792535655)=-2125.7205837080282; dx=-0.0027818382315111948 evalInputDelta=551.0643513830169
Fitness changed from -1574.6562323250114 to -2125.7205837080282
Iteration 4 complete. Error: -2125.7205837080282 Total: 4.0033; Orientation: 0.2767; Line Search: 2.6093
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 61d8fd21 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.057e+04, gradient 1.650e-01, dot -0.321; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2125.7205837080282, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 61d8fd21 to history. Total: 3
th(0)=-2125.7205837080282;dx=-0.02713298279219189
New Minimum: -2125.7205837080282 > -2239.868634540874
WOLF (strong): th(77978.69240469471)=-2239.868634540874; dx=0.0037526452242211145 evalInputDelta=114.14805083284591
New Minimum: -2239.868634540874 > -2580.534537649128
END: th(38989.346202347355)=-2580.534537649128; dx=-0.0019885859576520516 evalInputDelta=454.81395394109995
Fitness changed from -2125.7205837080282 to -2580.534537649128
Iteration 5 complete. Error: -2580.534537649128 Total: 3.7522; Orientation: 0.2793; Line Search: 2.5854
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7f24d649 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.282e+04, gradient 1.749e-01, dot -0.330; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2580.534537649128, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 7f24d649 to history. Total: 3
th(0)=-2580.534537649128;dx=-0.030526406614522883
Armijo: th(84000.00000000003)=-2513.889516267571; dx=0.005242231300595864 evalInputDelta=-66.6450213815574
New Minimum: -2580.534537649128 > -2921.0554599322486
WOLF (strong): th(42000.000000000015)=-2921.0554599322486; dx=5.431462291897311E-4 evalInputDelta=340.5209222831204
END: th(14000.000000000005)=-2870.5538664167234; dx=-0.009938558518756795 evalInputDelta=290.0193287675952
Fitness changed from -2580.534537649128 to -2921.0554599322486
Iteration 6 complete. Error: -2921.0554599322486 Total: 4.8576; Orientation: 0.2768; Line Search: 3.6241
Adding measurement 24bb0c87 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.442e+04, gradient 1.787e-01, dot -0.281; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2921.0554599322486, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 24bb0c87 to history. Total: 3
th(0)=-2921.0554599322486;dx=-0.03173202799335958
New Minimum: -2921.0554599322486 > -3301.444835219877
END: th(30162.085660446384)=-3301.444835219877; dx=-0.0012076704821504817 evalInputDelta=380.38937528762835
Fitness changed from -2921.0554599322486 to -3301.444835219877
Iteration 7 complete. Error: -3301.444835219877 Total: 2.7356; Orientation: 0.2766; Line Search: 1.6662
Adding measurement 701a8f28 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.006e+05, gradient 1.695e-01, dot -0.334; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3301.444835219877, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 701a8f28 to history. Total: 3
th(0)=-3301.444835219877;dx=-0.028642768265442355
New Minimum: -3301.444835219877 > -3308.7905645416727
WOLF (strong): th(64982.24367057894)=-3308.7905645416727; dx=0.004666680721135559 evalInputDelta=7.345729321795716
New Minimum: -3308.7905645416727 > -3555.9786276073146
END: th(32491.12183528947)=-3555.9786276073146; dx=-5.628460987374382E-4 evalInputDelta=254.53379238743764
Fitness changed from -3301.444835219877 to -3555.9786276073146
Iteration 8 complete. Error: -3555.9786276073146 Total: 3.7445; Orientation: 0.2768; Line Search: 2.6360
Adding measurement 2953d0bd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.162e+05, gradient 1.734e-01, dot -0.295; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3555.9786276073146, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 2953d0bd to history. Total: 3
th(0)=-3555.9786276073146;dx=-0.030020484356036048
Armijo: th(70000.00000000004)=-3190.3383833066987; dx=0.005835436281522093 evalInputDelta=-365.6402443006159
New Minimum: -3555.9786276073146 > -3689.141092449569
WOLF (strong): th(35000.00000000002)=-3689.141092449569; dx=0.003905995940554054 evalInputDelta=133.1624648422544
New Minimum: -3689.141092449569 > -3776.104606601888
END: th(11666.666666666673)=-3776.104606601888; dx=-0.010011182945693465 evalInputDelta=220.12597899457342
Fitness changed from -3555.9786276073146 to -3776.104606601888
Iteration 9 complete. Error: -3776.104606601888 Total: 4.7597; Orientation: 0.2725; Line Search: 3.7061
Adding measurement 3cfa101e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.686e+05, gradient 1.377e-01, dot -0.387; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3776.104606601888, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 3cfa101e to history. Total: 3
th(0)=-3776.104606601888;dx=-0.018866878680719776
New Minimum: -3776.104606601888 > -3966.2020876400134
END: th(25135.071383705326)=-3966.2020876400134; dx=-0.00711942450284123 evalInputDelta=190.09748103812535
Fitness changed from -3776.104606601888 to -3966.2020876400134
Iteration 10 complete. Error: -3966.2020876400134 Total: 2.7739; Orientation: 0.2693; Line Search: 1.6968
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 78467789 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.952e+05, gradient 1.370e-01, dot -0.409; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3966.2020876400134, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 78467789 to history. Total: 3
th(0)=-3966.2020876400134;dx=-0.018662683066494726
New Minimum: -3966.2020876400134 > -4200.710892218125
END: th(54151.86972548246)=-4200.710892218125; dx=-0.0036034132214338784 evalInputDelta=234.50880457811172
Fitness changed from -3966.2020876400134 to -4200.710892218125
Iteration 11 complete. Error: -4200.710892218125 Total: 2.9525; Orientation: 0.2710; Line Search: 1.7613
Adding measurement 53f8e5e8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.435e+05, gradient 1.571e-01, dot -0.346; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4200.710892218125, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 53f8e5e8 to history. Total: 3
th(0)=-4200.710892218125;dx=-0.024563539620316578
Armijo: th(116666.66666666676)=-3698.4868527547856; dx=0.0036963016174083553 evalInputDelta=-502.2240394633395
New Minimum: -4200.710892218125 > -4244.531776297593
WOLF (strong): th(58333.33333333338)=-4244.531776297593; dx=0.0027855952011286533 evalInputDelta=43.82088407946776
New Minimum: -4244.531776297593 > -4409.550090428727
END: th(19444.44444444446)=-4409.550090428727; dx=-0.005552044508329712 evalInputDelta=208.83919821060226
Fitness changed from -4200.710892218125 to -4409.550090428727
Iteration 12 complete. Error: -4409.550090428727 Total: 4.6737; Orientation: 0.2823; Line Search: 3.5727
Adding measurement 15f119b8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.098e+05, gradient 1.411e-01, dot -0.388; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4409.550090428727, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 15f119b8 to history. Total: 3
th(0)=-4409.550090428727;dx=-0.01982820251749259
New Minimum: -4409.550090428727 > -4563.81923393116
END: th(41891.78563950888)=-4563.81923393116; dx=-0.0029455035722057606 evalInputDelta=154.2691435024326
Fitness changed from -4409.550090428727 to -4563.81923393116
Iteration 13 complete. Error: -4563.81923393116 Total: 2.7557; Orientation: 0.2688; Line Search: 1.6673
Adding measurement 6aabe3c0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.663e+06, gradient 1.590e-01, dot -0.359; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4563.81923393116, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 6aabe3c0 to history. Total: 3
th(0)=-4563.81923393116;dx=-0.02516053447060989
Armijo: th(90253.11620913744)=-4239.938442949378; dx=0.004064523924120026 evalInputDelta=-323.8807909817815
New Minimum: -4563.81923393116 > -4620.098050907578
WOLF (strong): th(45126.55810456872)=-4620.098050907578; dx=0.002823786015342684 evalInputDelta=56.27881697641806
New Minimum: -4620.098050907578 > -4729.164619676502
END: th(15042.18603485624)=-4729.164619676502; dx=-0.005068903420780935 evalInputDelta=165.34538574534236
Fitness changed from -4563.81923393116 to -4729.164619676502
Iteration 14 complete. Error: -4729.164619676502 Total: 4.7164; Orientation: 0.2747; Line Search: 3.6252
Adding measurement 109e6838 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.010e+06, gradient 1.374e-01, dot -0.397; [a55819eb-422a-45f4-ae64-9d6797dffb7b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4729.164619676502, -920.423238244233, -352.5919095604719, -78.28538225121437
LBFGS Accumulation History: 3 points
Removed measurement 109e6838 to history. Total: 3
th(0)=-4729.164619676502;dx=-0.018758047142285666
New Minimum: -4729.164619676502 > -4855.57761754459
END: th(32407.407407407434)=-4855.57761754459; dx=-0.0040958707882146565 evalInputDelta=126.41299786808759
Fitness changed from -4729.164619676502 to -4855.57761754459
Iteration 15 complete. Error: -4855.57761754459 Total: 2.8708; Orientation: 0.2680; Line Search: 1.7447
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -4855.57761754459 (> -Infinity) after 62.871s (< 5400.000s)
Returns
-4855.57761754459