BasicOptimizer.scala:89 executed in 334.28 seconds (3.496 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 10292244240462
Reset training subject: 10293666901554
Adding measurement 3aac9d6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.8663300385331085;dx=-7.160845711997602E-4
New Minimum: -1.8663300385331085 > -1.8682898132218742
WOLFE (weak): th(2.154434690031884)=-1.8682898132218742; dx=-7.187135806498884E-4 evalInputDelta=0.0019597746887656253
New Minimum: -1.8682898132218742 > -1.8702461898040605
WOLFE (weak): th(4.308869380063768)=-1.8702461898040605; dx=-7.212454118061186E-4 evalInputDelta=0.003916151270952017
New Minimum: -1.8702461898040605 > -1.8780842038119798
WOLFE (weak): th(12.926608140191302)=-1.8780842038119798; dx=-7.163205941980972E-4 evalInputDelta=0.011754165278871298
New Minimum: -1.8780842038119798 > -1.9134583579753837
WOLFE (weak): th(51.70643256076521)=-1.9134583579753837; dx=-7.108766772059767E-4 evalInputDelta=0.047128319442275135
New Minimum: -1.9134583579753837 > -2.107982101113575
WOLFE (weak): th(258.53216280382605)=-2.107982101113575; dx=-7.398786655661231E-4 evalInputDelta=0.24165206258046656
New Minimum: -2.107982101113575 > -3.364510424619097
WOLFE (weak): th(1551.1929768229563)=-3.364510424619097; dx=-7.108105076938046E-4 evalInputDelta=1.4981803860859884
New Minimum: -3.364510424619097 > -8.736349359284665
END: th(10858.350837760694)=-8.736349359284665; dx=-3.2858156967176895E-4 evalInputDelta=6.870019320751556
Fitness changed from -1.8663300385331085 to -8.736349359284665
Iteration 1 complete. Error: -8.736349359284665 Total: 16.0133; Orientation: 0.0198; Line Search: 11.8320
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2f4a67a5 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-8.736349359284665;dx=-0.0024072495659944997
New Minimum: -8.736349359284665 > -17.70723004563921
END: th(23393.607721408407)=-17.70723004563921; dx=-6.412971434129993E-5 evalInputDelta=8.970880686354546
Fitness changed from -8.736349359284665 to -17.70723004563921
Iteration 2 complete. Error: -17.70723004563921 Total: 4.3577; Orientation: 0.0174; Line Search: 2.8151
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5cf31ecd to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-17.70723004563921;dx=-0.005881711516215083
Armijo: th(50400.00000000001)=-17.665260406843373; dx=5.3745766411582325E-5 evalInputDelta=-0.04196963879583748
New Minimum: -17.70723004563921 > -29.870482891732827
WOLF (strong): th(25200.000000000004)=-29.870482891732827; dx=6.9912281407766505E-6 evalInputDelta=12.163252846093616
New Minimum: -29.870482891732827 > -37.25302482264084
END: th(8400.000000000002)=-37.25302482264084; dx=-5.072396861658076E-4 evalInputDelta=19.54579477700163
Fitness changed from -17.70723004563921 to -37.25302482264084
Iteration 3 complete. Error: -37.25302482264084 Total: 7.2825; Orientation: 0.0168; Line Search: 5.7497
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6c014641 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.416e+03, gradient 8.520e-02, dot -0.104; [52a36548-c384-4626-8063-6235d2d29572 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.25302482264084, -17.70723004563921, -8.736349359284665, -1.8663300385331085
LBFGS Accumulation History: 3 points
Removed measurement 6c014641 to history. Total: 3
th(0)=-37.25302482264084;dx=-0.007012810547106561
New Minimum: -37.25302482264084 > -47.68344507831244
WOLF (strong): th(18097.251396267828)=-47.68344507831244; dx=2.1962456558015462E-4 evalInputDelta=10.4304202556716
New Minimum: -47.68344507831244 > -59.480465177894644
END: th(9048.625698133914)=-59.480465177894644; dx=-4.6655483229664756E-4 evalInputDelta=22.227440355253805
Fitness changed from -37.25302482264084 to -59.480465177894644
Iteration 4 complete. Error: -59.480465177894644 Total: 6.2206; Orientation: 0.4585; Line Search: 4.2573
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 721c56cc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.519e+03, gradient 1.098e-01, dot -0.102; [52a36548-c384-4626-8063-6235d2d29572 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -59.480465177894644, -17.70723004563921, -8.736349359284665, -1.8663300385331085
LBFGS Accumulation History: 3 points
Removed measurement 721c56cc to history. Total: 3
th(0)=-59.480465177894644;dx=-0.011943325590958714
Armijo: th(19494.673101173677)=-57.97448220970219; dx=5.095158405257223E-4 evalInputDelta=-1.5059829681924555
New Minimum: -59.480465177894644 > -80.4555164283348
WOLF (strong): th(9747.336550586839)=-80.4555164283348; dx=6.787073484399823E-4 evalInputDelta=20.97505125044016
New Minimum: -80.4555164283348 > -85.9540106058969
END: th(3249.1121835289464)=-85.9540106058969; dx=-0.0027883629673479244 evalInputDelta=26.473545428002254
Fitness changed from -59.480465177894644 to -85.9540106058969
Iteration 5 complete. Error: -85.9540106058969 Total: 7.6587; Orientation: 0.4678; Line Search: 5.6908
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement c08fa01 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.030e+04, gradient 1.187e-01, dot -0.095; [52a36548-c384-4626-8063-6235d2d29572 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -85.9540106058969, -17.70723004563921, -8.736349359284665, -1.8663300385331085
LBFGS Accumulation History: 3 points
Removed measurement c08fa01 to history. Total: 3
th(0)=-85.9540106058969;dx=-0.013993968087853512
New Minimum: -85.9540106058969 > -96.2800894439231
END: th(7000.000000000003)=-96.2800894439231; dx=-0.00131803215041076 evalInputDelta=10.326078838026206
Fitness changed from -85.9540106058969 to -96.2800894439231
Iteration 6 complete. Error: -96.2800894439231 Total: 5.3421; Orientation: 0.4755; Line Search: 3.3561
Adding measurement 30598ba0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.675e+04, gradient 1.777e-01, dot -0.113; [52a36548-c384-4626-8063-6235d2d29572 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -96.2800894439231, -17.70723004563921, -8.736349359284665, -1.8663300385331085
LBFGS Accumulation History: 3 points
Removed measurement 30598ba0 to history. Total: 3
th(0)=-96.2800894439231;dx=-0.030990681682794518
Armijo: th(15081.042830223192)=-77.07246086670084; dx=3.5876020391307524E-4 evalInputDelta=-19.207628577222266
New Minimum: -96.2800894439231 > -109.65037686307132
WOLF (strong): th(7540.521415111596)=-109.65037686307132; dx=3.632906293094707E-4 evalInputDelta=13.370287419148212
New Minimum: -109.65037686307132 > -131.55103083743205
END: th(2513.507138370532)=-131.55103083743205; dx=-0.0031093466503917715 evalInputDelta=35.27094139350895
Fitness changed from -96.2800894439231 to -131.55103083743205
Iteration 7 complete. Error: -131.55103083743205 Total: 7.5136; Orientation: 0.4825; Line Search: 5.6785
Adding measurement cd941ee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.121e+05, gradient 1.578e-01, dot -0.091; [52a36548-c384-4626-8063-6235d2d29572 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -131.55103083743205, -17.70723004563921, -8.736349359284665, -1.8663300385331085
LBFGS Accumulation History: 3 points
Removed measurement cd941ee to history. Total: 3
th(0)=-131.55103083743205;dx=-0.024225439843659772
New Minimum: -131.55103083743205 > -149.0871880603086
WOLF (strong): th(5415.186972548245)=-149.0871880603086; dx=3.899045656427852E-4 evalInputDelta=17.53615722287654
New Minimum: -149.0871880603086 > -163.86863533105242
END: th(2707.5934862741224)=-163.86863533105242; dx=-0.003388863454280636 evalInputDelta=32.317604493620365
Fitness changed from -131.55103083743205 to -163.86863533105242
Iteration 8 complete. Error: -163.86863533105242 Total: 6.0779; Orientation: 0.4726; Line Search: 4.2344
Adding measurement 27aa2691 to histo
...skipping 19917 bytes...
66; dx=-0.027296751237185817 evalInputDelta=1.4460259876614145E-5
Armijo: th(3.195133872807975E-6)=-356.1394923583688; dx=-0.027565038128492293 evalInputDelta=0.0
WOLFE (weak): th(3.1656584680772745E-6)=-356.13950681862866; dx=-0.02757698020080153 evalInputDelta=1.4460259876614145E-5
WOLFE (weak): th(3.180396170442625E-6)=-356.13950681862866; dx=-0.026939038313626366 evalInputDelta=1.4460259876614145E-5
Armijo: th(3.1877650216253003E-6)=-356.1394923583688; dx=-0.027267679870577877 evalInputDelta=0.0
Armijo: th(3.1840805960339627E-6)=-356.1394923583688; dx=-0.027466748813837857 evalInputDelta=0.0
Armijo: th(3.182238383238294E-6)=-356.1394923583688; dx=-0.02748471522331923 evalInputDelta=0.0
Armijo: th(3.1813172768404595E-6)=-356.1394923583688; dx=-0.027426204494169965 evalInputDelta=0.0
Armijo: th(3.180856723641542E-6)=-356.1394923583688; dx=-0.027509414274549386 evalInputDelta=0.0
Armijo: th(3.1806264470420836E-6)=-356.1394923583688; dx=-0.027447377097572404 evalInputDelta=0.0
WOLFE (weak): th(3.1805113087423543E-6)=-356.13950681862866; dx=-0.02698311256767779 evalInputDelta=1.4460259876614145E-5
WOLFE (weak): th(3.180568877892219E-6)=-356.13950681862866; dx=-0.02754093068458514 evalInputDelta=1.4460259876614145E-5
Armijo: th(3.1805976624671512E-6)=-356.1394923583688; dx=-0.027491520183810063 evalInputDelta=0.0
mu ~= nu (3.180568877892219E-6): th(2.515234537019808E-8)=-356.13950681862866
Fitness changed from -356.1394923583688 to -356.13950681862866
Iteration 20 complete. Error: -356.13950681862866 Total: 41.4138; Orientation: 1.0381; Line Search: 39.0061
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Adding measurement 796b1973 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.618e+03, gradient 1.671e-01, dot 0.063; [52a36548-c384-4626-8063-6235d2d29572 = 1.000/1.000e+00]
th(0)=-356.13950681862866;dx=-99.14244698415851
Armijo: th(6.852358931810165E-6)=-356.13921761331926; dx=-90.89403985487135 evalInputDelta=-2.8920530940013123E-4
Armijo: th(3.4261794659050825E-6)=-356.13934775573756; dx=-89.96192279882817 evalInputDelta=-1.5906289110034777E-4
Armijo: th(1.1420598219683608E-6)=-356.1394345173233; dx=-88.69215234339512 evalInputDelta=-7.230130535162971E-5
Armijo: th(2.855149554920902E-7)=-356.1394345173233; dx=-89.53118347191564 evalInputDelta=-7.230130535162971E-5
Armijo: th(5.710299109841804E-8)=-356.1394923583688; dx=-89.96265507828117 evalInputDelta=-1.4460259876614145E-5
Armijo: th(9.517165183069674E-9)=-356.1394923583688; dx=-89.00466439649091 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(1.3595950261528106E-9)=-356.13950681862866; dx=-90.80743210525792 evalInputDelta=0.0
Armijo: th(5.4383801046112424E-9)=-356.1394778981083; dx=-90.54800940684078 evalInputDelta=-2.89205203785059E-5
Armijo: th(3.3989875653820266E-9)=-356.1394923583688; dx=-90.89103572069462 evalInputDelta=-1.4460259876614145E-5
Armijo: th(2.3792912957674185E-9)=-356.13950681862866; dx=-89.57373290797408 evalInputDelta=0.0
Armijo: th(1.8694431609601145E-9)=-356.1394778981083; dx=-91.00954846781337 evalInputDelta=-2.89205203785059E-5
Armijo: th(1.6145190935564624E-9)=-356.13950681862866; dx=-89.98395133332036 evalInputDelta=0.0
Armijo: th(1.4870570598546364E-9)=-356.1394923583688; dx=-86.5933168847587 evalInputDelta=-1.4460259876614145E-5
Armijo: th(1.4233260430037236E-9)=-356.1394778981083; dx=-88.76725120980328 evalInputDelta=-2.89205203785059E-5
WOLFE (weak): th(1.391460534578267E-9)=-356.13950681862866; dx=-90.65332613628584 evalInputDelta=0.0
Armijo: th(1.4073932887909953E-9)=-356.1394923583688; dx=-89.69864671121488 evalInputDelta=-1.4460259876614145E-5
Armijo: th(1.3994269116846312E-9)=-356.1394923583688; dx=-90.54697544026338 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(1.395443723131449E-9)=-356.13950681862866; dx=-90.0580022462255 evalInputDelta=0.0
Armijo: th(1.3974353174080402E-9)=-356.1394923583688; dx=-88.32814606247763 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(1.3964395202697445E-9)=-356.13950681862866; dx=-90.64488330888953 evalInputDelta=0.0
Armijo: th(1.3969374188388924E-9)=-356.1394923583688; dx=-88.44744659740125 evalInputDelta=-1.4460259876614145E-5
Armijo: th(1.3966884695543185E-9)=-356.1394923583688; dx=-83.65904160284822 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(1.3965639949120315E-9)=-356.13950681862866; dx=-90.99569232617314 evalInputDelta=0.0
WOLFE (weak): th(1.396626232233175E-9)=-356.13950681862866; dx=-91.18643215614611 evalInputDelta=0.0
Armijo: th(1.3966573508937467E-9)=-356.1394923583688; dx=-90.37321792670272 evalInputDelta=-1.4460259876614145E-5
Armijo: th(1.3966417915634609E-9)=-356.1394923583688; dx=-91.10275544750745 evalInputDelta=-1.4460259876614145E-5
Armijo: th(1.396634011898318E-9)=-356.1394923583688; dx=-89.97099324748632 evalInputDelta=-1.4460259876614145E-5
mu ~= nu (1.396626232233175E-9): th(0.0)=-356.13950681862866
Fitness changed from -356.13950681862866 to -356.13950681862866
Static Iteration Total: 44.5441; Orientation: 0.4713; Line Search: 42.5544
Iteration 21 failed. Error: -356.13950681862866
Previous Error: 0.0 -> -356.13950681862866
Retrying iteration 21
Reset training subject: 10589075883657
Adding measurement 1dcf335 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-356.13950681862866;dx=-0.027526211155961287
WOLFE (weak): th(3.0089483841219083E-9)=-356.13950681862866; dx=-0.02734918687702813 evalInputDelta=0.0
WOLFE (weak): th(6.0178967682438165E-9)=-356.13950681862866; dx=-0.027381712258486235 evalInputDelta=0.0
WOLFE (weak): th(1.805369030473145E-8)=-356.13950681862866; dx=-0.027385105126609292 evalInputDelta=0.0
Armijo: th(7.22147612189258E-8)=-356.1394923583688; dx=-0.027414867419856284 evalInputDelta=-1.4460259876614145E-5
Armijo: th(4.513422576182863E-8)=-356.1394923583688; dx=-0.027100933213588807 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(3.1593958033280035E-8)=-356.13950681862866; dx=-0.027309627860653406 evalInputDelta=0.0
Armijo: th(3.836409189755433E-8)=-356.1394923583688; dx=-0.027100532596552568 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(3.497902496541718E-8)=-356.13950681862866; dx=-0.027173188662913964 evalInputDelta=0.0
WOLFE (weak): th(3.6671558431485754E-8)=-356.13950681862866; dx=-0.0274035301520272 evalInputDelta=0.0
Armijo: th(3.751782516452004E-8)=-356.1394923583688; dx=-0.027236968208659142 evalInputDelta=-1.4460259876614145E-5
Armijo: th(3.70946917980029E-8)=-356.1394923583688; dx=-0.02740131854070636 evalInputDelta=-1.4460259876614145E-5
Armijo: th(3.688312511474433E-8)=-356.1394923583688; dx=-0.02726282059098478 evalInputDelta=-1.4460259876614145E-5
Armijo: th(3.677734177311504E-8)=-356.1394923583688; dx=-0.02707629550857215 evalInputDelta=-1.4460259876614145E-5
Armijo: th(3.67244501023004E-8)=-356.1394923583688; dx=-0.02731548123081028 evalInputDelta=-1.4460259876614145E-5
Armijo: th(3.669800426689308E-8)=-356.1394923583688; dx=-0.0273004253357592 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(3.668478134918941E-8)=-356.13950681862866; dx=-0.02735376547910573 evalInputDelta=0.0
WOLFE (weak): th(3.669139280804124E-8)=-356.13950681862866; dx=-0.027371676995994618 evalInputDelta=0.0
WOLFE (weak): th(3.6694698537467156E-8)=-356.13950681862866; dx=-0.02738501879994075 evalInputDelta=0.0
Armijo: th(3.669635140218011E-8)=-356.1394923583688; dx=-0.027108990960215656 evalInputDelta=-1.4460259876614145E-5
Armijo: th(3.6695524969823635E-8)=-356.1394923583688; dx=-0.027322398179647128 evalInputDelta=-1.4460259876614145E-5
WOLFE (weak): th(3.66951117536454E-8)=-356.13950681862866; dx=-0.027248082272844722 evalInputDelta=0.0
WOLFE (weak): th(3.669531836173451E-8)=-356.13950681862866; dx=-0.02695226143616603 evalInputDelta=0.0
mu ~= nu (3.669531836173451E-8): th(0.0)=-356.13950681862866
Fitness changed from -356.13950681862866 to -356.13950681862866
Static Iteration Total: 37.4477; Orientation: 0.0165; Line Search: 34.6697
Iteration 22 failed. Error: -356.13950681862866
Previous Error: 0.0 -> -356.13950681862866
Optimization terminated 22
Final threshold in iteration 22: -356.13950681862866 (> -Infinity) after 334.280s (< 3600.000s)
Returns
-356.13950681862866