BasicOptimizer.scala:89 executed in 47.42 seconds (0.888 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 217949840488700
Reset training subject: 217950437355700
Adding measurement 153f28db to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-109.63202389841437;dx=-0.05970498055410458
New Minimum: -109.63202389841437 > -109.72725401457006
WOLFE (weak): th(2.154434690031884)=-109.72725401457006; dx=-0.059642726517556165 evalInputDelta=0.09523011615569033
New Minimum: -109.72725401457006 > -109.82257904387242
WOLFE (weak): th(4.308869380063768)=-109.82257904387242; dx=-0.05976768709682973 evalInputDelta=0.19055514545804897
New Minimum: -109.82257904387242 > -110.20464613784661
WOLFE (weak): th(12.926608140191302)=-110.20464613784661; dx=-0.05985303125532941 evalInputDelta=0.5726222394322349
New Minimum: -110.20464613784661 > -111.9266508443622
WOLFE (weak): th(51.70643256076521)=-111.9266508443622; dx=-0.06066381487797094 evalInputDelta=2.294626945947826
New Minimum: -111.9266508443622 > -120.68079759583739
WOLFE (weak): th(258.53216280382605)=-120.68079759583739; dx=-0.06017565085117091 evalInputDelta=11.048773697423016
New Minimum: -120.68079759583739 > -154.14182582749012
END: th(1551.1929768229563)=-154.14182582749012; dx=-0.03560569145781203 evalInputDelta=44.50980192907575
Fitness changed from -109.63202389841437 to -154.14182582749012
Iteration 1 complete. Error: -154.14182582749012 Total: 5.9767; Orientation: 0.0160; Line Search: 4.2861
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 1c3f30b0 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-154.14182582749012;dx=-0.08277260891041487
New Minimum: -154.14182582749012 > -224.0438771392947
END: th(3341.943960201201)=-224.0438771392947; dx=-0.02501194584147768 evalInputDelta=69.90205131180457
Fitness changed from -154.14182582749012 to -224.0438771392947
Iteration 2 complete. Error: -224.0438771392947 Total: 1.7972; Orientation: 0.0166; Line Search: 1.2163
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6e4ac2df to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-224.0438771392947;dx=-0.09202146580935058
New Minimum: -224.0438771392947 > -278.54037990890185
END: th(7200.000000000001)=-278.54037990890185; dx=-0.011003920859414389 evalInputDelta=54.49650276960716
Fitness changed from -224.0438771392947 to -278.54037990890185
Iteration 3 complete. Error: -278.54037990890185 Total: 1.7671; Orientation: 0.0159; Line Search: 1.1745
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 63b190fc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.933e+03, gradient 2.696e-01, dot -0.205; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -278.54037990890185, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 63b190fc to history. Total: 3
th(0)=-278.54037990890185;dx=-0.0725680061167255
New Minimum: -278.54037990890185 > -313.10711514444625
END: th(15511.929768229566)=-313.10711514444625; dx=-0.0031938556355602886 evalInputDelta=34.5667352355444
Fitness changed from -278.54037990890185 to -313.10711514444625
Iteration 4 complete. Error: -313.10711514444625 Total: 2.2848; Orientation: 0.2914; Line Search: 1.2010
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2da7b13f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.569e+03, gradient 2.797e-01, dot -0.128; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -313.10711514444625, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 2da7b13f to history. Total: 3
th(0)=-313.10711514444625;dx=-0.07816792819177286
Armijo: th(33419.43960201202)=-305.0075863287945; dx=-0.001440308656979728 evalInputDelta=-8.099528815651752
New Minimum: -313.10711514444625 > -453.8884343619103
END: th(16709.71980100601)=-453.8884343619103; dx=-0.007929757838045734 evalInputDelta=140.78131921746404
Fitness changed from -313.10711514444625 to -453.8884343619103
Iteration 5 complete. Error: -453.8884343619103 Total: 2.6732; Orientation: 0.2879; Line Search: 1.8192
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2fe72a85 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.908e+03, gradient 2.487e-01, dot -0.131; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -453.8884343619103, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 2fe72a85 to history. Total: 3
th(0)=-453.8884343619103;dx=-0.0618186244625837
Armijo: th(36000.000000000015)=-418.9677969279725; dx=-4.0993777101596264E-4 evalInputDelta=-34.92063743393783
New Minimum: -453.8884343619103 > -633.7106065412158
END: th(18000.000000000007)=-633.7106065412158; dx=-0.009569868222221731 evalInputDelta=179.82217217930554
Fitness changed from -453.8884343619103 to -633.7106065412158
Iteration 6 complete. Error: -633.7106065412158 Total: 2.7218; Orientation: 0.2937; Line Search: 1.8441
Adding measurement 1440f71c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.249e+03, gradient 2.552e-01, dot -0.133; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -633.7106065412158, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 1440f71c to history. Total: 3
th(0)=-633.7106065412158;dx=-0.06509265286770005
Armijo: th(38779.824420573925)=-523.7470368592536; dx=0.0010403466892267594 evalInputDelta=-109.96356968196221
New Minimum: -633.7106065412158 > -828.5880248048949
END: th(19389.912210286962)=-828.5880248048949; dx=-0.009392526150008244 evalInputDelta=194.87741826367903
Fitness changed from -633.7106065412158 to -828.5880248048949
Iteration 7 complete. Error: -828.5880248048949 Total: 2.6507; Orientation: 0.2979; Line Search: 1.8113
Adding measurement 3d7407e9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.266e+04, gradient 2.740e-01, dot -0.135; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -828.5880248048949, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 3d7407e9 to history. Total: 3
th(0)=-828.5880248048949;dx=-0.07479989282982347
Armijo: th(41774.29950251503)=-600.3550660030196; dx=0.0022413286787633193 evalInputDelta=-228.23295880187527
New Minimum: -828.5880248048949 > -995.796761960986
END: th(20887.149751257515)=-995.796761960986; dx=-0.006849574816239042 evalInputDelta=167.20873715609116
Fitness changed from -828.5880248048949 to -995.796761960986
Iteration 8 complete. Error: -995.796761960986 Total: 3.0862; Orientation: 0.3020; Line Search: 2.2473
Adding measurement 97cae80 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.516e+04, gradient 2.849e-01, dot -0.139; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -995.796761960986, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 97cae80 to history. Total: 3
th(0)=-995.796761960986;dx=-0.08106812111107319
Armijo: th(45000.00000000002)=-603.3976836421404; dx=0.0035248913571578637 evalInputDelta=-392.3990783188457
New Minimum: -995.796761960986 > -1135.2840975838585
END: th(22500.00000000001)=-1135.2840975838585; dx=-0.005231122344374366 evalInputDelta=139.48733562287248
Fitness changed from -995.796761960986 to -1135.2840975838585
Iteration 9 complete. Error: -1135.2840975838585 Total: 2.8842; Orientation: 0.3056; Line Search: 1.9898
Adding measurement 204363ae to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.020e+05, gradient 2.999e-01, dot -0.149; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1135.2840975838585, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 204363ae to history. Total: 3
th(0)=-1135.2840975838585;dx=-0.08989544757823115
Armijo: th(48474.78052571741)=-648.8081358450133; dx=0.004589114220765788 evalInputDelta=-486.47596173884517
New Minimum: -1135.2840975838585 > -1265.4076863202965
END: th(24237.390262858706)=-1265.4076863202965; dx=-0.0029709641028367465 evalInputDelta=130.12358873643802
Fitness changed from -1135.2840975838585 to -1265.4076863202965
Iteration 10 complete. Error: -1265.4076863202965 Total: 2.7094; Orientation: 0.2997; Line Search: 1.8463
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 767f9519 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.854e+06, gradient 2.969e-01, dot -0.149; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1265.4076863202965, -224.0438771392947, -154.14182582749012, -109.63202389841437
LBFGS Accumulation History: 3 points
Removed measurement 767f9519 to history. Total: 3
th(0)=-1265.4076863202965;dx=-0.08804615252770931
Armijo: th(52217.874378143795)=-698.2187478311696; dx=0.00552838172149939 evalInputDelta=-567.1889384891269
New Minimum: -1265.4076863202965 > -1441.3745236245434
END: th(26108.937189071898)=-1441.3745236245434; dx=-0.002100827799537229 evalInputDelta=175.96683730424684
Fitness changed from -1265.4076863202965 to -1441.3745236245434
Iteration 11 complete. Error: -1441.3745236245434 Total: 2.7994; Orientation: 0.4356; Line Search: 1.8041
Adding measurement 3bd9e940 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.057e+04, gradient 3.070e-01, dot 0.173; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1441.3745236245434;dx=-2157.0601747374426
Armijo: th(56250.00000000004)=115.37836058296773; dx=0.014355842966629137 evalInputDelta=-1556.7528842075112
Armijo: th(28125.00000000002)=112.51771067523873; dx=0.028611609209048115 evalInputDelta=-1553.892234299782
Armijo: th(9375.000000000005)=106.1667081990845; dx=0.08526971226307106 evalInputDelta=-1547.5412318236279
Armijo: th(2343.7500000000014)=95.85122175486742; dx=0.33692117131863214 evalInputDelta=-1537.225745379411
Armijo: th(468.7500000000003)=73.34710859085203; dx=1.6413680302598572 evalInputDelta=-1514.7216322153954
Armijo: th(78.12500000000004)=13.390104575189639; dx=9.069887753470095 evalInputDelta=-1454.7646281997331
Armijo: th(11.160714285714292)=-169.08249522607593; dx=48.69152046802711 evalInputDelta=-1272.2920283984674
Armijo: th(1.3950892857142865)=-753.8853574850046; dx=165.77654908521203 evalInputDelta=-687.4891661395387
Armijo: th(0.15500992063492072)=-1437.0992177200842; dx=-365.4434644886426 evalInputDelta=-4.275305904459174
New Minimum: -1441.3745236245434 > -1450.5572969259883
END: th(0.015500992063492073)=-1450.5572969259883; dx=-972.1615284548025 evalInputDelta=9.182773301444968
Fitness changed from -1441.3745236245434 to -1450.5572969259883
Iteration 12 complete. Error: -1450.5572969259883 Total: 7.7880; Orientation: 0.2882; Line Search: 6.9304
Adding measurement 113ff91b to history. Total: 4
Rejected: LBFGS Orientation magnitude: 5.062e+05, gradient 3.033e-01, dot -0.139; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1450.5572969259883, -1441.3745236245434, -224.0438771392947, -154.14182582749012, -109.63202389841437
Rejected: LBFGS Orientation magnitude: 1.184e+06, gradient 3.033e-01, dot -0.151; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1450.5572969259883, -1441.3745236245434, -224.0438771392947, -154.14182582749012
LBFGS Accumulation History: 3 points
Removed measurement 113ff91b to history. Total: 4
Removed measurement 3bd9e940 to history. Total: 3
th(0)=-1450.5572969259883;dx=-0.09188480749625705
New Minimum: -1450.5572969259883 > -1450.5589559306331
END: th(0.033395875031496235)=-1450.5589559306331; dx=-0.0767250736728851 evalInputDelta=0.0016590046448072826
Fitness changed from -1450.5572969259883 to -1450.5589559306331
Iteration 13 complete. Error: -1450.5589559306331 Total: 2.5091; Orientation: 0.6862; Line Search: 1.2522
Adding measurement 70635265 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.767e+04, gradient 3.034e-01, dot 0.192; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
th(0)=-1450.5589559306331;dx=-1608.1621270775693
New Minimum: -1450.5589559306331 > -1469.507442713027
END: th(0.07194923167182511)=-1469.507442713027; dx=-608.5465073197558 evalInputDelta=18.948486782393957
Fitness changed from -1450.5589559306331 to -1469.507442713027
Iteration 14 complete. Error: -1469.507442713027 Total: 2.3986; Orientation: 0.2998; Line Search: 1.5253
Adding measurement 606c1e1 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.381e+05, gradient 2.940e-01, dot -0.123; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1469.507442713027, -1450.5589559306331, -224.0438771392947, -154.14182582749012, -109.63202389841437
Rejected: LBFGS Orientation magnitude: 4.474e+05, gradient 2.940e-01, dot -0.143; [cf485add-4f06-4fd6-901b-6ac283077614 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1469.507442713027, -1450.5589559306331, -224.0438771392947, -154.14182582749012
LBFGS Accumulation History: 3 points
Removed measurement 606c1e1 to history. Total: 4
Removed measurement 70635265 to history. Total: 3
th(0)=-1469.507442713027;dx=-0.08635038395359759
New Minimum: -1469.507442713027 > -1469.5146217083554
END: th(0.15500992063492072)=-1469.5146217083554; dx=-0.06992712138766824 evalInputDelta=0.007178995328331439
Fitness changed from -1469.507442713027 to -1469.5146217083554
Iteration 15 complete. Error: -1469.5146217083554 Total: 3.3498; Orientation: 0.9136; Line Search: 1.8591
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1469.5146217083554 (> -Infinity) after 47.423s (< 5400.000s)
Returns
-1469.5146217083554