BasicOptimizer.scala:89 executed in 8.66 seconds (0.109 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140706469401999
Reset training subject: 140706598919200
Adding measurement 8adedfb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.9253050206428974;dx=-0.0038888497674294223
New Minimum: -2.9253050206428974 > -2.9339423022438726
WOLFE (weak): th(2.154434690031884)=-2.9339423022438726; dx=-0.00392674315720502 evalInputDelta=0.00863728160097521
New Minimum: -2.9339423022438726 > -2.9426039412964777
WOLFE (weak): th(4.308869380063768)=-2.9426039412964777; dx=-0.003916622001743085 evalInputDelta=0.017298920653580296
New Minimum: -2.9426039412964777 > -2.977278881986737
WOLFE (weak): th(12.926608140191302)=-2.977278881986737; dx=-0.003860836303514194 evalInputDelta=0.051973861343839634
New Minimum: -2.977278881986737 > -3.139911531291756
WOLFE (weak): th(51.70643256076521)=-3.139911531291756; dx=-0.004211066563663657 evalInputDelta=0.2146065106488586
New Minimum: -3.139911531291756 > -4.1379711145294555
WOLFE (weak): th(258.53216280382605)=-4.1379711145294555; dx=-0.004777545717293477 evalInputDelta=1.212666093886558
New Minimum: -4.1379711145294555 > -10.158274296017543
END: th(1551.1929768229563)=-10.158274296017543; dx=-0.002655244563228174 evalInputDelta=7.232969275374646
Fitness changed from -2.9253050206428974 to -10.158274296017543
Iteration 1 complete. Error: -10.158274296017543 Total: 1.3013; Orientation: 0.0017; Line Search: 0.8062
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 505b2054 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-10.158274296017543;dx=-0.007839826156372845
New Minimum: -10.158274296017543 > -24.509666072552914
END: th(3341.943960201201)=-24.509666072552914; dx=-0.0010840836735479557 evalInputDelta=14.35139177653537
Fitness changed from -10.158274296017543 to -24.509666072552914
Iteration 2 complete. Error: -24.509666072552914 Total: 0.3534; Orientation: 0.0015; Line Search: 0.2306
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 57d2dc6d to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-24.509666072552914;dx=-0.01117790212743951
Armijo: th(7200.000000000001)=-17.38410550109499; dx=3.392250706098495E-4 evalInputDelta=-7.125560571457925
New Minimum: -24.509666072552914 > -24.8170402419462
WOLF (strong): th(3600.0000000000005)=-24.8170402419462; dx=5.924638480243243E-4 evalInputDelta=0.30737416939328455
New Minimum: -24.8170402419462 > -33.17066590381631
END: th(1200.0000000000002)=-33.17066590381631; dx=-0.003727715024152895 evalInputDelta=8.660999831263393
Fitness changed from -24.509666072552914 to -33.17066590381631
Iteration 3 complete. Error: -33.17066590381631 Total: 0.5783; Orientation: 0.0015; Line Search: 0.4557
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 53bed9c2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.648e+03, gradient 9.718e-02, dot -0.224; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.17066590381631, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 53bed9c2 to history. Total: 3
th(0)=-33.17066590381631;dx=-0.009279556449133046
New Minimum: -33.17066590381631 > -41.40732719666079
END: th(2585.321628038261)=-41.40732719666079; dx=-0.0021455962826997755 evalInputDelta=8.236661292844481
Fitness changed from -33.17066590381631 to -41.40732719666079
Iteration 4 complete. Error: -41.40732719666079 Total: 0.3841; Orientation: 0.0293; Line Search: 0.2333
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 228ab924 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.638e+03, gradient 1.114e-01, dot -0.192; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.40732719666079, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 228ab924 to history. Total: 3
th(0)=-41.40732719666079;dx=-0.011904541831511496
Armijo: th(5569.906600335336)=-30.265270646056116; dx=-5.652391691839834E-4 evalInputDelta=-11.142056550604671
Armijo: th(2784.953300167668)=-41.324479585052146; dx=6.501740199746323E-4 evalInputDelta=-0.08284761160864207
New Minimum: -41.40732719666079 > -49.84693052364989
END: th(928.317766722556)=-49.84693052364989; dx=-0.005447057251875034 evalInputDelta=8.4396033269891
Fitness changed from -41.40732719666079 to -49.84693052364989
Iteration 5 complete. Error: -49.84693052364989 Total: 0.6019; Orientation: 0.0291; Line Search: 0.4530
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4ac88933 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.800e+03, gradient 1.023e-01, dot -0.221; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -49.84693052364989, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 4ac88933 to history. Total: 3
th(0)=-49.84693052364989;dx=-0.010093714761721889
New Minimum: -49.84693052364989 > -52.24112415587047
END: th(2000.0000000000007)=-52.24112415587047; dx=-0.0017992226185117844 evalInputDelta=2.394193632220585
Fitness changed from -49.84693052364989 to -52.24112415587047
Iteration 6 complete. Error: -52.24112415587047 Total: 0.3778; Orientation: 0.0288; Line Search: 0.2298
Adding measurement 7355808f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.701e+03, gradient 1.734e-01, dot -0.135; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -52.24112415587047, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 7355808f to history. Total: 3
th(0)=-52.24112415587047;dx=-0.02914272499436492
Armijo: th(4308.869380063769)=-42.05898217843209; dx=4.093976831215631E-5 evalInputDelta=-10.182141977438384
Armijo: th(2154.4346900318847)=-49.00475988986854; dx=0.002261319172870275 evalInputDelta=-3.236364266001935
New Minimum: -52.24112415587047 > -61.232142146649714
WOLF (strong): th(718.1448966772949)=-61.232142146649714; dx=0.0011072567022555498 evalInputDelta=8.991017990779241
END: th(179.53622416932373)=-59.10156110428998; dx=-0.015442894921046144 evalInputDelta=6.860436948419505
Fitness changed from -52.24112415587047 to -61.232142146649714
Iteration 7 complete. Error: -61.232142146649714 Total: 0.7030; Orientation: 0.0279; Line Search: 0.5638
Adding measurement e01e883 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.744e+03, gradient 1.276e-01, dot -0.204; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -61.232142146649714, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement e01e883 to history. Total: 3
th(0)=-61.232142146649714;dx=-0.01617712436224095
New Minimum: -61.232142146649714 > -67.16974764849745
END: th(386.7990694677318)=-67.16974764849745; dx=-0.010783244393773131 evalInputDelta=5.9376055018477345
Fitness changed from -61.232142146649714 to -67.16974764849745
Iteration 8 complete. Error: -67.16974764849745 Total: 0.3742; Orientation: 0.0270; Line Search: 0.2327
Adding measurement 210f0032 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.110e+04, gradient 1.039e-01, dot -0.260; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -67.16974764849745, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 210f0032 to history. Total: 3
th(0)=-67.16974764849745;dx=-0.010599763096537396
New Minimum: -67.16974764849745 > -71.57204043533412
END: th(833.3333333333338)=-71.57204043533412; dx=-0.006146422413333669 evalInputDelta=4.402292786836668
Fitness changed from -67.16974764849745 to -71.57204043533412
Iteration 9 complete. Error: -71.57204043533412 Total: 0.3677; Orientation: 0.0299; Line Search: 0.2298
Adding measurement 1f7c789d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.975e+05, gradient 1.062e-01, dot -0.237; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -71.57204043533412, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 1f7c789d to history. Total: 3
th(0)=-71.57204043533412;dx=-0.011010419968583436
New Minimum: -71.57204043533412 > -79.81979117788399
END: th(1795.3622416932376)=-79.81979117788399; dx=-0.003381513790950968 evalInputDelta=8.247750742549869
Fitness changed from -71.57204043533412 to -79.81979117788399
Iteration 10 complete. Error: -79.81979117788399 Total: 0.3740; Orientation: 0.0278; Line Search: 0.2338
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 2f3e8f9d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.522e+05, gradient 1.489e-01, dot -0.200; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.81979117788399, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 2f3e8f9d to history. Total: 3
th(0)=-79.81979117788399;dx=-0.02175365131184247
Armijo: th(3867.9906946773185)=-58.06407521506147; dx=5.642884928961434E-4 evalInputDelta=-21.755715962822514
Armijo: th(1933.9953473386593)=-71.51944744288467; dx=0.0037661544630572122 evalInputDelta=-8.300343734999316
New Minimum: -79.81979117788399 > -88.4300836949202
END: th(644.665115779553)=-88.4300836949202; dx=-0.002031053010093097 evalInputDelta=8.610292517036214
Fitness changed from -79.81979117788399 to -88.4300836949202
Iteration 11 complete. Error: -88.4300836949202 Total: 0.6199; Orientation: 0.0304; Line Search: 0.4638
Adding measurement 4fd40f9d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.870e+05, gradient 1.235e-01, dot -0.198; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -88.4300836949202, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 4fd40f9d to history. Total: 3
th(0)=-88.4300836949202;dx=-0.014934347550578892
Armijo: th(1388.8888888888898)=-88.33722500997428; dx=-3.108447714074969E-4 evalInputDelta=-0.09285868494592364
New Minimum: -88.4300836949202 > -92.33251114770951
END: th(694.4444444444449)=-92.33251114770951; dx=-0.0019682705027856133 evalInputDelta=3.9024274527893112
Fitness changed from -88.4300836949202 to -92.33251114770951
Iteration 12 complete. Error: -92.33251114770951 Total: 0.4825; Orientation: 0.0288; Line Search: 0.3434
Adding measurement 7cf50336 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.228e+05, gradient 1.222e-01, dot -0.234; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.33251114770951, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 7cf50336 to history. Total: 3
th(0)=-92.33251114770951;dx=-0.014701186050529627
Armijo: th(1496.1352014110314)=-86.42831724555427; dx=0.003950430857501356 evalInputDelta=-5.904193902155242
New Minimum: -92.33251114770951 > -96.23971759377221
WOLF (strong): th(748.0676007055157)=-96.23971759377221; dx=0.001472859713192537 evalInputDelta=3.907206446062702
END: th(249.35586690183857)=-96.17185204011844; dx=-0.010708990795483379 evalInputDelta=3.839340892408927
Fitness changed from -92.33251114770951 to -96.23971759377221
Iteration 13 complete. Error: -96.23971759377221 Total: 0.5955; Orientation: 0.0274; Line Search: 0.4535
Adding measurement 563c2a2d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.932e+04, gradient 1.506e-01, dot -0.134; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -96.23971759377221, -24.509666072552914, -10.158274296017543, -2.9253050206428974
LBFGS Accumulation History: 3 points
Removed measurement 563c2a2d to history. Total: 3
th(0)=-96.23971759377221;dx=-0.022452303553375477
New Minimum: -96.23971759377221 > -100.52184036213364
WOLF (strong): th(537.2209298162943)=-100.52184036213364; dx=0.0011091872719215278 evalInputDelta=4.2821227683614325
New Minimum: -100.52184036213364 > -101.3827515340318
END: th(268.61046490814715)=-101.3827515340318; dx=-0.007646031799854547 evalInputDelta=5.143033940259585
Fitness changed from -96.23971759377221 to -101.3827515340318
Iteration 14 complete. Error: -101.3827515340318 Total: 0.4824; Orientation: 0.0283; Line Search: 0.3473
Adding measurement 3a0b731 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.360e+02, gradient 9.899e-02, dot 0.564; [addf2aca-cbc9-4f88-84fb-784a568ea9a3 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-101.3827515340318;dx=-45.9199071526354
Armijo: th(578.7037037037043)=-67.78733285810073; dx=0.0014923073705474027 evalInputDelta=-33.59541867593107
Armijo: th(289.35185185185213)=-68.30061771866605; dx=0.0034231203212924926 evalInputDelta=-33.08213381536575
Armijo: th(96.4506172839507)=-74.08749083243946; dx=-0.06153278184730657 evalInputDelta=-27.29526070159234
Armijo: th(24.112654320987676)=-79.88043715820407; dx=-1.100489496092278 evalInputDelta=-21.502314375827723
Armijo: th(4.822530864197535)=-73.29156342107059; dx=-2.109213912617286 evalInputDelta=-28.091188112961206
Armijo: th(0.8037551440329226)=-88.01360185860199; dx=4.671359736021135 evalInputDelta=-13.36914967542981
New Minimum: -101.3827515340318 > -104.16955318545233
END: th(0.11482216343327466)=-104.16955318545233; dx=-31.44314059817124 evalInputDelta=2.7868016514205323
Fitness changed from -101.3827515340318 to -104.16955318545233
Iteration 15 complete. Error: -104.16955318545233 Total: 1.0517; Orientation: 0.0295; Line Search: 0.9104
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -104.16955318545233 (> -Infinity) after 8.661s (< 5400.000s)
Returns
-104.16955318545233