BasicOptimizer.scala:89 executed in 55.44 seconds (1.655 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 220844597370600
Reset training subject: 220845307202300
Adding measurement 381fc45d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-6.610192430078826;dx=-9.445997087875708E-5
New Minimum: -6.610192430078826 > -6.610406829439133
WOLFE (weak): th(2.154434690031884)=-6.610406829439133; dx=-9.462564228065857E-5 evalInputDelta=2.1439936030631657E-4
New Minimum: -6.610406829439133 > -6.610620559275869
WOLFE (weak): th(4.308869380063768)=-6.610620559275869; dx=-9.433447072807973E-5 evalInputDelta=4.281291970427148E-4
New Minimum: -6.610620559275869 > -6.6114751802267655
WOLFE (weak): th(12.926608140191302)=-6.6114751802267655; dx=-9.483161915165712E-5 evalInputDelta=0.0012827501479391046
New Minimum: -6.6114751802267655 > -6.6153250553508425
WOLFE (weak): th(51.70643256076521)=-6.6153250553508425; dx=-9.47406884833483E-5 evalInputDelta=0.005132625272016078
New Minimum: -6.6153250553508425 > -6.635920224051779
WOLFE (weak): th(258.53216280382605)=-6.635920224051779; dx=-9.430147276792065E-5 evalInputDelta=0.025727793972952995
New Minimum: -6.635920224051779 > -6.773985804907188
WOLFE (weak): th(1551.1929768229563)=-6.773985804907188; dx=-9.14420068830136E-5 evalInputDelta=0.16379337482836132
New Minimum: -6.773985804907188 > -7.767250086166193
END: th(10858.350837760694)=-7.767250086166193; dx=-7.27124307194711E-5 evalInputDelta=1.1570576560873667
Fitness changed from -6.610192430078826 to -7.767250086166193
Iteration 1 complete. Error: -7.767250086166193 Total: 8.2154; Orientation: 0.0200; Line Search: 6.1553
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 32595739 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-7.767250086166193;dx=-9.276266669418791E-5
New Minimum: -7.767250086166193 > -9.63259914530326
END: th(23393.607721408407)=-9.63259914530326; dx=-6.684947119589786E-5 evalInputDelta=1.865349059137067
Fitness changed from -7.767250086166193 to -9.63259914530326
Iteration 2 complete. Error: -9.63259914530326 Total: 1.9047; Orientation: 0.0160; Line Search: 1.2744
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement ddb1315 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-9.63259914530326;dx=-1.078170241533471E-4
New Minimum: -9.63259914530326 > -12.655858298674639
END: th(50400.00000000001)=-12.655858298674639; dx=-5.2064700351279176E-5 evalInputDelta=3.023259153371379
Fitness changed from -9.63259914530326 to -12.655858298674639
Iteration 3 complete. Error: -12.655858298674639 Total: 1.8708; Orientation: 0.0159; Line Search: 1.2523
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 50490608 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.035e+04, gradient 1.132e-02, dot -0.430; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.655858298674639, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 50490608 to history. Total: 3
th(0)=-12.655858298674639;dx=-1.2798145352034145E-4
New Minimum: -12.655858298674639 > -16.02006614285941
END: th(108583.50837760697)=-16.02006614285941; dx=-2.426175324799575E-5 evalInputDelta=3.3642078441847705
Fitness changed from -12.655858298674639 to -16.02006614285941
Iteration 4 complete. Error: -16.02006614285941 Total: 2.1380; Orientation: 0.3130; Line Search: 1.2174
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7992bb2e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.068e+03, gradient 1.207e-02, dot -0.200; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.02006614285941, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 7992bb2e to history. Total: 3
th(0)=-16.02006614285941;dx=-1.4557267979740067E-4
New Minimum: -16.02006614285941 > -16.413836125223163
END: th(233936.07721408413)=-16.413836125223163; dx=-5.387855514847567E-6 evalInputDelta=0.3937699823637537
Fitness changed from -16.02006614285941 to -16.413836125223163
Iteration 5 complete. Error: -16.413836125223163 Total: 2.1669; Orientation: 0.3099; Line Search: 1.2504
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 61c2afbf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.320e+03, gradient 1.201e-02, dot -0.136; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.413836125223163, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 61c2afbf to history. Total: 3
th(0)=-16.413836125223163;dx=-1.4381922026335608E-4
Armijo: th(504000.0000000002)=-10.937018985169374; dx=6.511335972440138E-6 evalInputDelta=-5.4768171400537895
New Minimum: -16.413836125223163 > -19.136023089535104
END: th(252000.0000000001)=-19.136023089535104; dx=-8.016390360692119E-6 evalInputDelta=2.722186964311941
Fitness changed from -16.413836125223163 to -19.136023089535104
Iteration 6 complete. Error: -19.136023089535104 Total: 2.7219; Orientation: 0.2973; Line Search: 1.8479
Adding measurement 9f5d176 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.076e+03, gradient 1.154e-02, dot -0.103; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.136023089535104, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 9f5d176 to history. Total: 3
th(0)=-19.136023089535104;dx=-1.3295331534909753E-4
Armijo: th(542917.5418880349)=-13.086497646362911; dx=9.151469839149846E-6 evalInputDelta=-6.049525443172193
New Minimum: -19.136023089535104 > -22.74248812462351
END: th(271458.77094401745)=-22.74248812462351; dx=-1.1025776159443947E-5 evalInputDelta=3.606465035088405
Fitness changed from -19.136023089535104 to -22.74248812462351
Iteration 7 complete. Error: -22.74248812462351 Total: 3.0808; Orientation: 0.2798; Line Search: 1.8762
Adding measurement 5183b36c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.310e+03, gradient 1.142e-02, dot -0.117; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.74248812462351, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 5183b36c to history. Total: 3
th(0)=-22.74248812462351;dx=-1.303881435116314E-4
Armijo: th(584840.1930352104)=-13.635059130526102; dx=8.791795227510243E-6 evalInputDelta=-9.107428994097408
New Minimum: -22.74248812462351 > -25.193038832265117
END: th(292420.0965176052)=-25.193038832265117; dx=-8.186757107644377E-6 evalInputDelta=2.4505507076416073
Fitness changed from -22.74248812462351 to -25.193038832265117
Iteration 8 complete. Error: -25.193038832265117 Total: 2.6804; Orientation: 0.2984; Line Search: 1.8306
Adding measurement 634bd000 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.557e+03, gradient 1.149e-02, dot -0.099; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.193038832265117, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 634bd000 to history. Total: 3
th(0)=-25.193038832265117;dx=-1.3182305908389754E-4
Armijo: th(630000.0000000002)=-13.849272639569215; dx=1.045296321403861E-5 evalInputDelta=-11.343766192695902
New Minimum: -25.193038832265117 > -27.147117284871662
END: th(315000.0000000001)=-27.147117284871662; dx=-6.56933457621687E-6 evalInputDelta=1.9540784526065451
Fitness changed from -25.193038832265117 to -27.147117284871662
Iteration 9 complete. Error: -27.147117284871662 Total: 2.7203; Orientation: 0.3056; Line Search: 1.8524
Adding measurement 55306b68 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.779e+04, gradient 1.150e-02, dot -0.108; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.147117284871662, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 55306b68 to history. Total: 3
th(0)=-27.147117284871662;dx=-1.3204113050420865E-4
Armijo: th(678646.9273600436)=-12.02710483064936; dx=1.2687662044789323E-5 evalInputDelta=-15.120012454222302
New Minimum: -27.147117284871662 > -28.413213777523673
END: th(339323.4636800218)=-28.413213777523673; dx=-6.078874674236645E-6 evalInputDelta=1.2660964926520109
Fitness changed from -27.147117284871662 to -28.413213777523673
Iteration 10 complete. Error: -28.413213777523673 Total: 3.0386; Orientation: 0.3020; Line Search: 2.1470
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7b0d6b6d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.574e+04, gradient 1.157e-02, dot -0.104; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.413213777523673, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 7b0d6b6d to history. Total: 3
th(0)=-28.413213777523673;dx=-1.335716604985365E-4
Armijo: th(731050.241294013)=-11.590480125820093; dx=1.3685654964053406E-5 evalInputDelta=-16.82273365170358
New Minimum: -28.413213777523673 > -29.29520766790133
END: th(365525.1206470065)=-29.29520766790133; dx=-4.336068912076267E-6 evalInputDelta=0.8819938903776574
Fitness changed from -28.413213777523673 to -29.29520766790133
Iteration 11 complete. Error: -29.29520766790133 Total: 2.6750; Orientation: 0.2840; Line Search: 1.8044
Adding measurement 6c3f2160 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.209e+04, gradient 1.111e-02, dot -0.103; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.29520766790133, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 6c3f2160 to history. Total: 3
th(0)=-29.29520766790133;dx=-1.2329457357441835E-4
Armijo: th(787500.0000000003)=-10.469788062571006; dx=1.533240573074959E-5 evalInputDelta=-18.825419605330325
New Minimum: -29.29520766790133 > -30.97883226813932
END: th(393750.0000000002)=-30.97883226813932; dx=-4.117947111626254E-6 evalInputDelta=1.6836246002379909
Fitness changed from -29.29520766790133 to -30.97883226813932
Iteration 12 complete. Error: -30.97883226813932 Total: 2.7931; Orientation: 0.3002; Line Search: 1.9195
Adding measurement 5d2976a6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.239e+05, gradient 1.133e-02, dot -0.104; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.97883226813932, -9.63259914530326, -7.767250086166193, -6.610192430078826
LBFGS Accumulation History: 3 points
Removed measurement 5d2976a6 to history. Total: 3
th(0)=-30.97883226813932;dx=-1.2814503929644904E-4
Armijo: th(848308.6592000546)=-7.128700674286051; dx=1.639654153443223E-5 evalInputDelta=-23.85013159385327
Armijo: th(424154.3296000273)=-29.98433320822705; dx=-5.847337974183019E-7 evalInputDelta=-0.9944990599122718
New Minimum: -30.97883226813932 > -36.465505616058785
END: th(141384.77653334243)=-36.465505616058785; dx=-5.451772555252753E-5 evalInputDelta=5.486673347919464
Fitness changed from -30.97883226813932 to -36.465505616058785
Iteration 13 complete. Error: -36.465505616058785 Total: 3.3924; Orientation: 0.3227; Line Search: 2.4864
Adding measurement 78577673 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.434e+04, gradient 9.435e-03, dot 0.167; [e6870c59-e39b-49ae-98f9-d7a8caf3b80c = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-36.465505616058785;dx=-54.127360505222335
Armijo: th(304604.26720583875)=724.8975156962443; dx=0.0012028619732755697 evalInputDelta=-761.3630213123031
Armijo: th(152302.13360291938)=722.1465096175301; dx=0.0023885727130101993 evalInputDelta=-758.6120152335889
Armijo: th(50767.37786763979)=716.1429795220467; dx=0.00707338441150567 evalInputDelta=-752.6084851381055
Armijo: th(12691.844466909948)=704.9583721366673; dx=0.027693760102260285 evalInputDelta=-741.4238777527261
Armijo: th(2538.3688933819894)=681.6061147455687; dx=0.13153085865125683 evalInputDelta=-718.0716203616274
Armijo: th(423.0614822303316)=633.997146863698; dx=0.7119555905035964 evalInputDelta=-670.4626524797568
Armijo: th(60.43735460433309)=541.9020845040877; dx=4.037730878944971 evalInputDelta=-578.3675901201465
Armijo: th(7.554669325541636)=356.74988937151954; dx=20.994221821332214 evalInputDelta=-393.21539498757835
Armijo: th(0.8394077028379595)=63.66056852635346; dx=47.697105289455784 evalInputDelta=-100.12607414241225
Armijo: th(0.08394077028379596)=-31.25629750381625; dx=1.8530198552600137 evalInputDelta=-5.2092081122425355
Armijo: th(0.007630979116708723)=-36.21536962662964; dx=-23.47907864639635 evalInputDelta=-0.25013598942914683
Armijo: th(6.359149263923936E-4)=-36.44687564919151; dx=-27.161333567863288 evalInputDelta=-0.01862996686727314
Armijo: th(4.891653279941489E-5)=-36.46409029660489; dx=-27.450141093312947 evalInputDelta=-0.0014153194538977232
Armijo: th(3.4940380571010635E-6)=-36.46540248699136; dx=-27.505342262543003 evalInputDelta=-1.0312906742626637E-4
Armijo: th(2.3293587047340425E-7)=-36.46550036992484; dx=-27.459997429157077 evalInputDelta=-5.246133945036036E-6
Armijo: th(1.4558491904587765E-8)=-36.465503231872994; dx=-27.687083929382865 evalInputDelta=-2.384185791015625E-6
Armijo: th(8.563818767404567E-10)=-36.465505616058785; dx=-27.540184679069608 evalInputDelta=0.0
MIN ALPHA (4.7576770930025376E-11): th(0.0)=-36.465505616058785
Fitness changed from -36.465505616058785 to -36.465505616058785
Static Iteration Total: 13.5873; Orientation: 0.2900; Line Search: 12.7354
Iteration 14 failed. Error: -36.465505616058785
Previous Error: 0.0 -> -36.465505616058785
Retrying iteration 14
Reset training subject: 220897583115300
Adding measurement 2dbad544 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-36.465505616058785;dx=-8.880576347876393E-5
END: th(2.1544346900318838E-10)=-36.465505616058785; dx=-7.358359786898504E-5 evalInputDelta=0.0
Fitness changed from -36.465505616058785 to -36.465505616058785
Static Iteration Total: 2.4491; Orientation: 0.0156; Line Search: 1.2882
Iteration 15 failed. Error: -36.465505616058785
Previous Error: 0.0 -> -36.465505616058785
Optimization terminated 15
Final threshold in iteration 15: -36.465505616058785 (> -Infinity) after 55.435s (< 5400.000s)
Returns
-36.465505616058785