BasicOptimizer.scala:89 executed in 68.03 seconds (1.557 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 141713925357600
Reset training subject: 141714828897700
Adding measurement 55c75b30 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.5257366207522143;dx=-1.740303342602562E-5
New Minimum: -3.5257366207522143 > -3.525767077926312
WOLFE (weak): th(2.154434690031884)=-3.525767077926312; dx=-1.687574894564867E-5 evalInputDelta=3.0457174097886508E-5
New Minimum: -3.525767077926312 > -3.5257977150549347
WOLFE (weak): th(4.308869380063768)=-3.5257977150549347; dx=-1.685630347961238E-5 evalInputDelta=6.109430272038807E-5
New Minimum: -3.5257977150549347 > -3.52591917963902
WOLFE (weak): th(12.926608140191302)=-3.52591917963902; dx=-1.6726319370177167E-5 evalInputDelta=1.825588868058503E-4
New Minimum: -3.52591917963902 > -3.5264687816121
WOLFE (weak): th(51.70643256076521)=-3.5264687816121; dx=-1.6747260641615398E-5 evalInputDelta=7.321608598855889E-4
New Minimum: -3.5264687816121 > -3.529400227020544
WOLFE (weak): th(258.53216280382605)=-3.529400227020544; dx=-1.6778566424462358E-5 evalInputDelta=0.0036636062683297332
New Minimum: -3.529400227020544 > -3.5477501655494543
WOLFE (weak): th(1551.1929768229563)=-3.5477501655494543; dx=-1.6911218381542634E-5 evalInputDelta=0.022013544797240048
New Minimum: -3.5477501655494543 > -3.6787092883283363
WOLFE (weak): th(10858.350837760694)=-3.6787092883283363; dx=-1.6418200144103105E-5 evalInputDelta=0.15297266757612205
New Minimum: -3.6787092883283363 > -4.676995973159813
END: th(86866.80670208555)=-4.676995973159813; dx=-1.3624491179702065E-5 evalInputDelta=1.1512593524075987
Fitness changed from -3.5257366207522143 to -4.676995973159813
Iteration 1 complete. Error: -4.676995973159813 Total: 11.2509; Orientation: 0.0144; Line Search: 8.7306
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 756a0b55 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.676995973159813;dx=-1.8317646119181372E-5
New Minimum: -4.676995973159813 > -6.936093076720105
END: th(187148.86177126726)=-6.936093076720105; dx=-1.1350011922901544E-5 evalInputDelta=2.2590971035602925
Fitness changed from -4.676995973159813 to -6.936093076720105
Iteration 2 complete. Error: -6.936093076720105 Total: 2.7477; Orientation: 0.0143; Line Search: 1.8223
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement e897852 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-6.936093076720105;dx=-1.9434520633564503E-5
New Minimum: -6.936093076720105 > -11.041601134312357
END: th(403200.00000000006)=-11.041601134312357; dx=-8.703170004787389E-6 evalInputDelta=4.105508057592251
Fitness changed from -6.936093076720105 to -11.041601134312357
Iteration 3 complete. Error: -11.041601134312357 Total: 2.7271; Orientation: 0.0160; Line Search: 1.7422
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 78b8d2f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.379e+04, gradient 5.076e-03, dot -0.381; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.041601134312357, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 78b8d2f9 to history. Total: 3
th(0)=-11.041601134312357;dx=-2.560986013489322E-5
New Minimum: -11.041601134312357 > -17.661673972387394
END: th(868668.0670208557)=-17.661673972387394; dx=-5.322825701162189E-6 evalInputDelta=6.620072838075037
Fitness changed from -11.041601134312357 to -17.661673972387394
Iteration 4 complete. Error: -17.661673972387394 Total: 3.0267; Orientation: 0.2839; Line Search: 1.7911
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1f700b05 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.160e+04, gradient 5.262e-03, dot -0.249; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.661673972387394, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 1f700b05 to history. Total: 3
th(0)=-17.661673972387394;dx=-2.7581931218489512E-5
New Minimum: -17.661673972387394 > -27.158615449978328
END: th(1871488.617712673)=-27.158615449978328; dx=-8.112202397893696E-7 evalInputDelta=9.496941477590934
Fitness changed from -17.661673972387394 to -27.158615449978328
Iteration 5 complete. Error: -27.158615449978328 Total: 2.8855; Orientation: 0.2803; Line Search: 1.6931
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 5ea047c8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.974e+04, gradient 5.783e-03, dot -0.128; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.158615449978328, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 5ea047c8 to history. Total: 3
th(0)=-27.158615449978328;dx=-3.336098939911355E-5
New Minimum: -27.158615449978328 > -30.46940860619916
WOLF (strong): th(4032000.0000000014)=-30.46940860619916; dx=2.0143219247316885E-6 evalInputDelta=3.310793156220832
New Minimum: -30.46940860619916 > -38.46858080644049
WOLF (strong): th(2016000.0000000007)=-38.46858080644049; dx=9.6064653583826E-7 evalInputDelta=11.309965356462165
New Minimum: -38.46858080644049 > -38.910880568917975
END: th(672000.0000000002)=-38.910880568917975; dx=-1.0014722254511369E-5 evalInputDelta=11.752265118939647
Fitness changed from -27.158615449978328 to -38.910880568917975
Iteration 6 complete. Error: -38.910880568917975 Total: 4.7867; Orientation: 0.2767; Line Search: 3.4853
Adding measurement 2e617100 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.382e+04, gradient 4.827e-03, dot -0.234; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -38.910880568917975, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 2e617100 to history. Total: 3
th(0)=-38.910880568917975;dx=-2.3174229087291135E-5
New Minimum: -38.910880568917975 > -49.30033972926706
END: th(1447780.1117014263)=-49.30033972926706; dx=-3.167782565576234E-6 evalInputDelta=10.389459160349084
Fitness changed from -38.910880568917975 to -49.30033972926706
Iteration 7 complete. Error: -49.30033972926706 Total: 2.8326; Orientation: 0.2763; Line Search: 1.7287
Adding measurement 517036f3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.589e+04, gradient 5.854e-03, dot -0.175; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -49.30033972926706, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 517036f3 to history. Total: 3
th(0)=-49.30033972926706;dx=-3.4001781685133216E-5
Armijo: th(3119147.6961877886)=-44.8636064680594; dx=3.43634600841596E-6 evalInputDelta=-4.436733261207657
New Minimum: -49.30033972926706 > -56.0045918468458
WOLF (strong): th(1559573.8480938943)=-56.0045918468458; dx=1.6714880016034792E-6 evalInputDelta=6.70425211757874
New Minimum: -56.0045918468458 > -57.999991079026614
END: th(519857.94936463144)=-57.999991079026614; dx=-9.858754528738615E-6 evalInputDelta=8.699651349759556
Fitness changed from -49.30033972926706 to -57.999991079026614
Iteration 8 complete. Error: -57.999991079026614 Total: 4.6321; Orientation: 0.2948; Line Search: 3.5347
Adding measurement 124cbceb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.718e+07, gradient 4.610e-03, dot -0.279; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.999991079026614, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 124cbceb to history. Total: 3
th(0)=-57.999991079026614;dx=-2.1138230781925315E-5
New Minimum: -57.999991079026614 > -65.84717830801243
END: th(1120000.0000000005)=-65.84717830801243; dx=-4.767499673369745E-6 evalInputDelta=7.847187228985817
Fitness changed from -57.999991079026614 to -65.84717830801243
Iteration 9 complete. Error: -65.84717830801243 Total: 2.8273; Orientation: 0.2763; Line Search: 1.7393
Adding measurement 282a1bc2 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.269e+05, gradient 5.543e-03, dot 0.248; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-65.84717830801243;dx=-175.5677042978656
Armijo: th(2412966.852835711)=-15.28567311867529; dx=6.356835185753193E-6 evalInputDelta=-50.56150518933714
Armijo: th(1206483.4264178555)=-15.28575891694897; dx=1.264574139606193E-5 evalInputDelta=-50.56141939106346
Armijo: th(402161.14213928516)=-15.286561672813374; dx=3.820480544335829E-5 evalInputDelta=-50.560616635199054
Armijo: th(100540.28553482129)=-15.28690816832617; dx=1.5206504034691432E-4 evalInputDelta=-50.56027013968626
Armijo: th(20108.057106964257)=-15.287595150602542; dx=7.606825826212422E-4 evalInputDelta=-50.55958315740989
Armijo: th(3351.3428511607094)=-15.287458820660332; dx=0.004555017944867529 evalInputDelta=-50.5597194873521
Armijo: th(478.7632644515299)=-15.33794809505374; dx=0.03201904280667939 evalInputDelta=-50.50923021295869
Armijo: th(59.84540805644124)=-15.571133016134047; dx=0.25695285759241954 evalInputDelta=-50.27604529187838
Armijo: th(6.649489784049027)=-17.60794360829378; dx=2.4540806765826293 evalInputDelta=-48.23923469971865
Armijo: th(0.6649489784049026)=-36.169246736480474; dx=31.58103951627156 evalInputDelta=-29.677931571531957
New Minimum: -65.84717830801243 > -68.3311495386705
WOLF (strong): th(0.06044990712771842)=-68.3311495386705; dx=3.5733833601170346 evalInputDelta=2.483971230658071
END: th(0.005037492260643202)=-66.5624036922007; dx=-107.78111332903433 evalInputDelta=0.7152253841882725
Fitness changed from -65.84717830801243 to -68.3311495386705
Iteration 10 complete. Error: -68.3311495386705 Total: 12.4430; Orientation: 0.2787; Line Search: 11.3737
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 3d6f09d3 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.398e+05, gradient 6.139e-03, dot -0.066; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -68.3311495386705, -65.84717830801243, -6.936093076720105, -4.676995973159813, -3.5257366207522143
Rejected: LBFGS Orientation magnitude: 4.112e+05, gradient 6.139e-03, dot -0.123; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -68.3311495386705, -65.84717830801243, -6.936093076720105, -4.676995973159813
LBFGS Accumulation History: 3 points
Removed measurement 3d6f09d3 to history. Total: 4
Removed measurement 282a1bc2 to history. Total: 3
th(0)=-68.3311495386705;dx=-3.75995635255169E-5
New Minimum: -68.3311495386705 > -68.33115191921264
END: th(0.01085294807709685)=-68.33115191921264; dx=-3.0951401373921284E-5 evalInputDelta=2.380542142077502E-6
Fitness changed from -68.3311495386705 to -68.33115191921264
Iteration 11 complete. Error: -68.33115191921264 Total: 3.2627; Orientation: 0.6009; Line Search: 1.6955
Adding measurement 5af389ad to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.798e+04, gradient 6.141e-03, dot -0.116; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -68.33115191921264, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 5af389ad to history. Total: 3
th(0)=-68.33115191921264;dx=-3.7661360542395626E-5
Armijo: th(0.02338196782641228)=-68.33115191921264; dx=-3.094281187257993E-5 evalInputDelta=0.0
New Minimum: -68.33115191921264 > -68.33115668029669
END: th(0.01169098391320614)=-68.33115668029669; dx=-3.0932784200637364E-5 evalInputDelta=4.761084042570474E-6
Fitness changed from -68.33115191921264 to -68.33115668029669
Iteration 12 complete. Error: -68.33115668029669 Total: 3.8985; Orientation: 0.3921; Line Search: 2.6681
Adding measurement 54ced4d9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.783e+04, gradient 6.140e-03, dot -0.116; [2c6bfebd-5853-4603-9bc3-2f4ff4f6d7df = 1.000/1.000e+00]
Orientation rejected. Popping history element from -68.33115668029669, -6.936093076720105, -4.676995973159813, -3.5257366207522143
LBFGS Accumulation History: 3 points
Removed measurement 54ced4d9 to history. Total: 3
th(0)=-68.33115668029669;dx=-3.741129128485627E-5
Armijo: th(0.02518746130321601)=-68.33115191921264; dx=-3.090263080972169E-5 evalInputDelta=-4.761084042570474E-6
Armijo: th(0.012593730651608005)=-68.33115668029669; dx=-3.077694891661647E-5 evalInputDelta=0.0
Armijo: th(0.004197910217202668)=-68.33115668029669; dx=-3.091264716570528E-5 evalInputDelta=0.0
Armijo: th(0.001049477554300667)=-68.33115668029669; dx=-3.0651872812014065E-5 evalInputDelta=0.0
Armijo: th(2.0989551086013342E-4)=-68.33115668029669; dx=-3.0918954761946354E-5 evalInputDelta=0.0
END: th(3.498258514335557E-5)=-68.33115668029669; dx=-3.0831874913429035E-5 evalInputDelta=0.0
Fitness changed from -68.33115668029669 to -68.33115668029669
Static Iteration Total: 7.1060; Orientation: 0.2858; Line Search: 6.0124
Iteration 13 failed. Error: -68.33115668029669
Previous Error: 0.0 -> -68.33115668029669
Retrying iteration 13
Reset training subject: 141778352195500
Adding measurement 2c80f2cf to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-68.33115668029669;dx=-3.7205844047700256E-5
END: th(7.536769497983923E-5)=-68.33115668029669; dx=-3.094482728773492E-5 evalInputDelta=0.0
Fitness changed from -68.33115668029669 to -68.33115668029669
Static Iteration Total: 3.6006; Orientation: 0.0140; Line Search: 1.7192
Iteration 14 failed. Error: -68.33115668029669
Previous Error: 0.0 -> -68.33115668029669
Optimization terminated 14
Final threshold in iteration 14: -68.33115668029669 (> -Infinity) after 68.027s (< 5400.000s)
Returns
-68.33115668029669