BasicOptimizer.scala:89 executed in 9.39 seconds (0.448 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139993976488900
Reset training subject: 139994114061800
Adding measurement 10afe96c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-16.702993451306117;dx=-0.06362000672354767
New Minimum: -16.702993451306117 > -16.943703279110707
WOLFE (weak): th(2.154434690031884)=-16.943703279110707; dx=-0.06446624490890557 evalInputDelta=0.24070982780459005
New Minimum: -16.943703279110707 > -17.1853318945029
WOLFE (weak): th(4.308869380063768)=-17.1853318945029; dx=-0.0656875629614459 evalInputDelta=0.48233844319678454
New Minimum: -17.1853318945029 > -18.159660084773922
WOLFE (weak): th(12.926608140191302)=-18.159660084773922; dx=-0.06723569749632888 evalInputDelta=1.4566666334678047
New Minimum: -18.159660084773922 > -22.954772266877725
WOLFE (weak): th(51.70643256076521)=-22.954772266877725; dx=-0.07418689115669712 evalInputDelta=6.251778815571608
New Minimum: -22.954772266877725 > -47.90801071046861
END: th(258.53216280382605)=-47.90801071046861; dx=-0.0401650096975022 evalInputDelta=31.205017259162492
Fitness changed from -16.702993451306117 to -47.90801071046861
Iteration 1 complete. Error: -47.90801071046861 Total: 1.0506; Orientation: 0.0016; Line Search: 0.6852
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 739dfc48 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-47.90801071046861;dx=-0.09537842771442624
New Minimum: -47.90801071046861 > -96.57576264098306
END: th(556.9906600335335)=-96.57576264098306; dx=-0.02455477624325063 evalInputDelta=48.66775193051445
Fitness changed from -47.90801071046861 to -96.57576264098306
Iteration 2 complete. Error: -96.57576264098306 Total: 0.3571; Orientation: 0.0016; Line Search: 0.2322
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5dd6816b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-96.57576264098306;dx=-0.11388194407886813
New Minimum: -96.57576264098306 > -174.57727962853585
END: th(1200.0)=-174.57727962853585; dx=-0.007689229929962722 evalInputDelta=78.00151698755279
Fitness changed from -96.57576264098306 to -174.57727962853585
Iteration 3 complete. Error: -174.57727962853585 Total: 0.3555; Orientation: 0.0016; Line Search: 0.2320
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6d99797c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.752e+03, gradient 4.396e-01, dot -0.177; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -174.57727962853585, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 6d99797c to history. Total: 3
th(0)=-174.57727962853585;dx=-0.18538464600341792
New Minimum: -174.57727962853585 > -235.7674966996087
WOLF (strong): th(2585.3216280382608)=-235.7674966996087; dx=0.005366515606209463 evalInputDelta=61.19021707107285
New Minimum: -235.7674966996087 > -288.41497068026825
WOLF (strong): th(1292.6608140191304)=-288.41497068026825; dx=4.8372187263709225E-4 evalInputDelta=113.8376910517324
END: th(430.8869380063768)=-258.13141638514287; dx=-0.03893639782516526 evalInputDelta=83.55413675660702
Fitness changed from -174.57727962853585 to -288.41497068026825
Iteration 4 complete. Error: -288.41497068026825 Total: 0.6122; Orientation: 0.0282; Line Search: 0.4608
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2ee8e73c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.037e+03, gradient 3.749e-01, dot -0.184; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -288.41497068026825, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 2ee8e73c to history. Total: 3
th(0)=-288.41497068026825;dx=-0.13581440891297775
New Minimum: -288.41497068026825 > -353.8912924543355
END: th(928.3177667225559)=-353.8912924543355; dx=-0.026367864619686612 evalInputDelta=65.47632177406723
Fitness changed from -288.41497068026825 to -353.8912924543355
Iteration 5 complete. Error: -353.8912924543355 Total: 0.3879; Orientation: 0.0305; Line Search: 0.2338
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7049b299 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.362e+03, gradient 3.539e-01, dot -0.125; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -353.8912924543355, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 7049b299 to history. Total: 3
th(0)=-353.8912924543355;dx=-0.11867387418937975
Armijo: th(2000.0000000000005)=-319.63569350512154; dx=0.013801871444686743 evalInputDelta=-34.25559894921395
New Minimum: -353.8912924543355 > -399.26121227211644
WOLF (strong): th(1000.0000000000002)=-399.26121227211644; dx=0.010664561759146849 evalInputDelta=45.36991981778095
New Minimum: -399.26121227211644 > -399.57272100958045
END: th(333.3333333333334)=-399.57272100958045; dx=-0.060142817132010695 evalInputDelta=45.68142855524496
Fitness changed from -353.8912924543355 to -399.57272100958045
Iteration 6 complete. Error: -399.57272100958045 Total: 0.6043; Orientation: 0.0287; Line Search: 0.4559
Adding measurement 44982f2e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.856e+03, gradient 2.704e-01, dot -0.224; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -399.57272100958045, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 44982f2e to history. Total: 3
th(0)=-399.57272100958045;dx=-0.06728947063740502
New Minimum: -399.57272100958045 > -447.99991280691114
END: th(718.1448966772948)=-447.99991280691114; dx=-0.04503132101379784 evalInputDelta=48.42719179733069
Fitness changed from -399.57272100958045 to -447.99991280691114
Iteration 7 complete. Error: -447.99991280691114 Total: 0.3730; Orientation: 0.0306; Line Search: 0.2339
Adding measurement 4edac6d9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.859e+03, gradient 2.540e-01, dot -0.294; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -447.99991280691114, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 4edac6d9 to history. Total: 3
th(0)=-447.99991280691114;dx=-0.06045325148795999
New Minimum: -447.99991280691114 > -534.7819356990286
END: th(1547.196277870927)=-534.7819356990286; dx=-0.03215496239095928 evalInputDelta=86.78202289211748
Fitness changed from -447.99991280691114 to -534.7819356990286
Iteration 8 complete. Error: -534.7819356990286 Total: 0.3724; Orientation: 0.0275; Line Search: 0.2318
Adding measurement 2974141c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.572e+03, gradient 3.012e-01, dot -0.282; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -534.7819356990286, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 2974141c to history. Total: 3
th(0)=-534.7819356990286;dx=-0.08307954552110236
New Minimum: -534.7819356990286 > -638.6934906510321
END: th(3333.333333333335)=-638.6934906510321; dx=-0.012237953005208035 evalInputDelta=103.91155495200348
Fitness changed from -534.7819356990286 to -638.6934906510321
Iteration 9 complete. Error: -638.6934906510321 Total: 0.3651; Orientation: 0.0276; Line Search: 0.2288
Adding measurement 4b5d5581 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.120e+04, gradient 3.340e-01, dot -0.247; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -638.6934906510321, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 4b5d5581 to history. Total: 3
th(0)=-638.6934906510321;dx=-0.1051657475785674
Armijo: th(7181.4489667729495)=-516.6707425914497; dx=0.012353764604480576 evalInputDelta=-122.02274805958245
New Minimum: -638.6934906510321 > -745.8837459684987
WOLF (strong): th(3590.7244833864747)=-745.8837459684987; dx=0.013512425752453008 evalInputDelta=107.19025531746661
New Minimum: -745.8837459684987 > -786.8713760837917
END: th(1196.908161128825)=-786.8713760837917; dx=-0.04017348898268144 evalInputDelta=148.17788543275958
Fitness changed from -638.6934906510321 to -786.8713760837917
Iteration 10 complete. Error: -786.8713760837917 Total: 1.0869; Orientation: 0.0283; Line Search: 0.9446
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 3f2e87d6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.262e+04, gradient 2.637e-01, dot -0.313; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -786.8713760837917, -96.57576264098306, -47.90801071046861, -16.702993451306117
LBFGS Accumulation History: 3 points
Removed measurement 3f2e87d6 to history. Total: 3
th(0)=-786.8713760837917;dx=-0.06263443290246676
New Minimum: -786.8713760837917 > -933.7980576655747
END: th(2578.660463118212)=-933.7980576655747; dx=-0.01857045944907542 evalInputDelta=146.926681581783
Fitness changed from -786.8713760837917 to -933.7980576655747
Iteration 11 complete. Error: -933.7980576655747 Total: 0.3794; Orientation: 0.0280; Line Search: 0.2285
Adding measurement 38c1bd6f to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.604e+04, gradient 3.054e-01, dot 0.372; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-933.7980576655747;dx=-2919.2755679427673
Armijo: th(5555.555555555559)=-681.7336127476773; dx=0.061212375794602286 evalInputDelta=-252.0644449178974
Armijo: th(2777.7777777777796)=-681.1880935174953; dx=0.11827351539684247 evalInputDelta=-252.60996414807937
Armijo: th(925.9259259259265)=-680.4119156804942; dx=0.36754466746697034 evalInputDelta=-253.38614198508049
Armijo: th(231.48148148148164)=-676.9310803176347; dx=1.4412424831872819 evalInputDelta=-256.86697734793995
Armijo: th(46.296296296296326)=-670.0044542762383; dx=7.177915030333535 evalInputDelta=-263.79360338933634
Armijo: th(7.716049382716054)=-684.9923585340205; dx=48.04363809729661 evalInputDelta=-248.8056991315542
Armijo: th(1.1022927689594364)=-857.0085690936818; dx=303.6035194096137 evalInputDelta=-76.78948857189289
New Minimum: -933.7980576655747 > -1290.4187304902234
END: th(0.13778659611992955)=-1290.4187304902234; dx=-545.9653848373381 evalInputDelta=356.62067282464875
Fitness changed from -933.7980576655747 to -1290.4187304902234
Iteration 12 complete. Error: -1290.4187304902234 Total: 1.2227; Orientation: 0.0268; Line Search: 1.0828
Adding measurement 7511af70 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 7.184e+03, gradient 3.542e-01, dot 0.031; [49f62eaa-6375-4973-b53b-25dccbadee06 = 1.000/1.000e+00]
th(0)=-1290.4187304902234;dx=-85.18781012681696
Armijo: th(0.2968522225021888)=-1147.9560096101243; dx=159.55158611114163 evalInputDelta=-142.46272088009914
Armijo: th(0.1484261112510944)=-1216.0091488142677; dx=193.19183962365315 evalInputDelta=-74.40958167595568
Armijo: th(0.04947537041703146)=-1265.4322384070986; dx=198.42491266885307 evalInputDelta=-24.98649208312486
Armijo: th(0.012368842604257866)=-1284.3299809628365; dx=217.12422975348966 evalInputDelta=-6.088749527386881
Armijo: th(0.002473768520851573)=-1289.232087329508; dx=225.20342728645142 evalInputDelta=-1.1866431607154482
Armijo: th(4.1229475347526217E-4)=-1290.22086287581; dx=226.92999386494222 evalInputDelta=-0.197867614413326
Armijo: th(5.8899250496466024E-5)=-1290.3905416578348; dx=226.70854946846066 evalInputDelta=-0.028188832388650553
Armijo: th(7.362406312058253E-6)=-1290.4150979432936; dx=218.930120543758 evalInputDelta=-0.0036325469297935342
Armijo: th(8.180451457842503E-7)=-1290.4181977173137; dx=216.22528440585592 evalInputDelta=-5.327729097643896E-4
Armijo: th(8.180451457842503E-8)=-1290.418633622438; dx=209.52252018212616 evalInputDelta=-9.686778548712027E-5
Armijo: th(7.436774052584094E-9)=-1290.4187304902234; dx=206.7955024296635 evalInputDelta=0.0
WOLF (strong): th(6.197311710486745E-10)=-1290.4187304902234; dx=201.69503704946106 evalInputDelta=0.0
MIN ALPHA (4.767162854220573E-11): th(0.0)=-1290.4187304902234
Fitness changed from -1290.4187304902234 to -1290.4187304902234
Static Iteration Total: 1.7784; Orientation: 0.0419; Line Search: 1.6235
Iteration 13 failed. Error: -1290.4187304902234
Previous Error: 0.0 -> -1290.4187304902234
Retrying iteration 13
Reset training subject: 140002922126099
Adding measurement 6f4e06f9 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-1290.4187304902234;dx=-0.12015647783651226
END: th(2.1544346900318838E-10)=-1290.4187304902234; dx=-0.10712329758887751 evalInputDelta=0.0
Fitness changed from -1290.4187304902234 to -1290.4187304902234
Static Iteration Total: 0.4469; Orientation: 0.0016; Line Search: 0.2237
Iteration 14 failed. Error: -1290.4187304902234
Previous Error: 0.0 -> -1290.4187304902234
Optimization terminated 14
Final threshold in iteration 14: -1290.4187304902234 (> -Infinity) after 9.393s (< 5400.000s)
Returns
-1290.4187304902234