BasicOptimizer.scala:89 executed in 59.50 seconds (1.591 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 210472397433400
Reset training subject: 210473220436000
Adding measurement 66235b20 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-5.646259598058094;dx=-0.021932608590755246
New Minimum: -5.646259598058094 > -5.676986847303709
WOLFE (weak): th(2.154434690031884)=-5.676986847303709; dx=-0.02192566961557159 evalInputDelta=0.030727249245614985
New Minimum: -5.676986847303709 > -5.707726163074614
WOLFE (weak): th(4.308869380063768)=-5.707726163074614; dx=-0.021932177648231754 evalInputDelta=0.061466565016520036
New Minimum: -5.707726163074614 > -5.830517419746357
WOLFE (weak): th(12.926608140191302)=-5.830517419746357; dx=-0.02188626051269147 evalInputDelta=0.18425782168826288
New Minimum: -5.830517419746357 > -6.381188424555354
WOLFE (weak): th(51.70643256076521)=-6.381188424555354; dx=-0.021749812144348103 evalInputDelta=0.73492882649726
New Minimum: -6.381188424555354 > -9.31037171743522
WOLFE (weak): th(258.53216280382605)=-9.31037171743522; dx=-0.02135470769021447 evalInputDelta=3.664112119377126
New Minimum: -9.31037171743522 > -25.74364442670906
END: th(1551.1929768229563)=-25.74364442670906; dx=-0.017553711746009212 evalInputDelta=20.097384828650966
Fitness changed from -5.646259598058094 to -25.74364442670906
Iteration 1 complete. Error: -25.74364442670906 Total: 7.6546; Orientation: 0.0156; Line Search: 5.4210
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5d5f5b44 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-25.74364442670906;dx=-0.02846951226755476
New Minimum: -25.74364442670906 > -78.48095369883934
END: th(3341.943960201201)=-78.48095369883934; dx=-0.01781888442264562 evalInputDelta=52.73730927213028
Fitness changed from -25.74364442670906 to -78.48095369883934
Iteration 2 complete. Error: -78.48095369883934 Total: 2.2021; Orientation: 0.0158; Line Search: 1.4461
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 703bb54b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-78.48095369883934;dx=-0.06649244166897494
New Minimum: -78.48095369883934 > -182.28588237243818
END: th(7200.000000000001)=-182.28588237243818; dx=-0.0077248760625776795 evalInputDelta=103.80492867359884
Fitness changed from -78.48095369883934 to -182.28588237243818
Iteration 3 complete. Error: -182.28588237243818 Total: 2.5210; Orientation: 0.0184; Line Search: 1.7540
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 49d39746 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.295e+04, gradient 3.766e-01, dot -0.086; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -182.28588237243818, -78.48095369883934, -25.74364442670906, -5.646259598058094
LBFGS Accumulation History: 3 points
Removed measurement 49d39746 to history. Total: 3
th(0)=-182.28588237243818;dx=-0.14165533463369806
Armijo: th(15511.929768229566)=-172.19414386625033; dx=5.119468423030816E-4 evalInputDelta=-10.091738506187852
New Minimum: -182.28588237243818 > -309.94051237492914
END: th(7755.964884114783)=-309.94051237492914; dx=-0.0030145582744036275 evalInputDelta=127.65463000249096
Fitness changed from -182.28588237243818 to -309.94051237492914
Iteration 4 complete. Error: -309.94051237492914 Total: 3.2941; Orientation: 0.2898; Line Search: 2.3067
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 63885f54 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.124e+04, gradient 3.495e-01, dot -0.076; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -309.94051237492914, -78.48095369883934, -25.74364442670906, -5.646259598058094
LBFGS Accumulation History: 3 points
Removed measurement 63885f54 to history. Total: 3
th(0)=-309.94051237492914;dx=-0.12214249995596596
New Minimum: -309.94051237492914 > -379.4107844447063
WOLF (strong): th(16709.71980100601)=-379.4107844447063; dx=0.005347756045538936 evalInputDelta=69.47027206977714
New Minimum: -379.4107844447063 > -621.3911563498101
END: th(8354.859900503005)=-621.3911563498101; dx=-0.01088303387619859 evalInputDelta=311.45064397488096
Fitness changed from -309.94051237492914 to -621.3911563498101
Iteration 5 complete. Error: -621.3911563498101 Total: 3.3573; Orientation: 0.3049; Line Search: 2.3074
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 19a08809 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.062e+04, gradient 3.487e-01, dot -0.074; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -621.3911563498101, -78.48095369883934, -25.74364442670906, -5.646259598058094
LBFGS Accumulation History: 3 points
Removed measurement 19a08809 to history. Total: 3
th(0)=-621.3911563498101;dx=-0.12125245363509426
Armijo: th(18000.000000000007)=-523.4411648070387; dx=0.011639007525402936 evalInputDelta=-97.94999154277139
New Minimum: -621.3911563498101 > -891.7753792848287
END: th(9000.000000000004)=-891.7753792848287; dx=-0.0068839614688425135 evalInputDelta=270.38422293501856
Fitness changed from -621.3911563498101 to -891.7753792848287
Iteration 6 complete. Error: -891.7753792848287 Total: 3.1827; Orientation: 0.2810; Line Search: 2.1706
Adding measurement 352956bc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.946e+04, gradient 3.558e-01, dot -0.052; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -891.7753792848287, -78.48095369883934, -25.74364442670906, -5.646259598058094
LBFGS Accumulation History: 3 points
Removed measurement 352956bc to history. Total: 3
th(0)=-891.7753792848287;dx=-0.12647862279239946
Armijo: th(19389.912210286962)=-560.6306577905011; dx=0.013698538491732404 evalInputDelta=-331.14472149432754
New Minimum: -891.7753792848287 > -1080.2235059518484
END: th(9694.956105143481)=-1080.2235059518484; dx=-4.066036254389829E-4 evalInputDelta=188.44812666701978
Fitness changed from -891.7753792848287 to -1080.2235059518484
Iteration 7 complete. Error: -1080.2235059518484 Total: 3.3406; Orientation: 0.2919; Line Search: 2.3009
Adding measurement 1d4c1bac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.611e+05, gradient 3.614e-01, dot -0.058; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1080.2235059518484, -78.48095369883934, -25.74364442670906, -5.646259598058094
LBFGS Accumulation History: 3 points
Removed measurement 1d4c1bac to history. Total: 3
th(0)=-1080.2235059518484;dx=-0.1305804036624668
Armijo: th(20887.149751257515)=-680.1486164562573; dx=0.01677866634072924 evalInputDelta=-400.07488949559115
New Minimum: -1080.2235059518484 > -1249.5355738846354
WOLF (strong): th(10443.574875628758)=-1249.5355738846354; dx=2.2641063198145656E-4 evalInputDelta=169.31206793278693
New Minimum: -1249.5355738846354 > -1323.9697393217552
END: th(3481.1916252095857)=-1323.9697393217552; dx=-0.06285395244059502 evalInputDelta=243.7462333699068
Fitness changed from -1080.2235059518484 to -1323.9697393217552
Iteration 8 complete. Error: -1323.9697393217552 Total: 4.5350; Orientation: 0.2764; Line Search: 3.0344
Adding measurement 42b6e1ee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.902e+05, gradient 3.081e-01, dot -0.065; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1323.9697393217552, -78.48095369883934, -25.74364442670906, -5.646259598058094
LBFGS Accumulation History: 3 points
Removed measurement 42b6e1ee to history. Total: 3
th(0)=-1323.9697393217552;dx=-0.09486794036807111
New Minimum: -1323.9697393217552 > -1485.1817010835412
END: th(7500.000000000004)=-1485.1817010835412; dx=-0.029149549707856513 evalInputDelta=161.21196176178591
Fitness changed from -1323.9697393217552 to -1485.1817010835412
Iteration 9 complete. Error: -1485.1817010835412 Total: 2.5123; Orientation: 0.2922; Line Search: 1.5153
Adding measurement 2088c3f9 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.367e+04, gradient 3.149e-01, dot 0.081; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1485.1817010835412;dx=-1359.8272596930615
Armijo: th(16158.260175239137)=389.74186264043675; dx=0.03813075059033581 evalInputDelta=-1874.9235637239778
Armijo: th(8079.130087619568)=389.08172545933695; dx=0.07589516146296232 evalInputDelta=-1874.263426542878
Armijo: th(2693.043362539856)=386.04561359611347; dx=0.2284005144603229 evalInputDelta=-1871.2273146796547
Armijo: th(673.260840634964)=365.93824325660376; dx=0.861025891663341 evalInputDelta=-1851.119944340145
Armijo: th(134.6521681269928)=211.45654680859315; dx=2.5466407338960124 evalInputDelta=-1696.6382478921344
Armijo: th(22.442028021165466)=23.675097155928427; dx=9.198238100123348 evalInputDelta=-1508.8567982394695
Armijo: th(3.206004003023638)=-83.3176788828911; dx=37.714383811405355 evalInputDelta=-1401.86402220065
Armijo: th(0.40075050037795473)=-273.29637823638285; dx=467.44130569362494 evalInputDelta=-1211.8853228471582
Armijo: th(0.044527833375328306)=-1303.5500426609485; dx=2977.632887293341 evalInputDelta=-181.63165842259264
Armijo: th(0.004452783337532831)=-1485.1385443206493; dx=-50.87770816640034 evalInputDelta=-0.04315676289184012
New Minimum: -1485.1817010835412 > -1485.4055110186978
END: th(4.0479848523025734E-4)=-1485.4055110186978; dx=-532.723165821182 evalInputDelta=0.2238099351566234
Fitness changed from -1485.1817010835412 to -1485.4055110186978
Iteration 10 complete. Error: -1485.4055110186978 Total: 10.5682; Orientation: 0.2957; Line Search: 9.5332
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7c7c2e40 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 3.511e+06, gradient 3.144e-01, dot -0.068; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1485.4055110186978, -1485.1817010835412, -78.48095369883934, -25.74364442670906, -5.646259598058094
Rejected: LBFGS Orientation magnitude: 2.797e+06, gradient 3.144e-01, dot -0.195; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1485.4055110186978, -1485.1817010835412, -78.48095369883934, -25.74364442670906
LBFGS Accumulation History: 3 points
Removed measurement 7c7c2e40 to history. Total: 4
Removed measurement 2088c3f9 to history. Total: 3
th(0)=-1485.4055110186978;dx=-0.0987835305077609
Armijo: th(8.721118990524256E-4)=-1485.4055098266049; dx=-0.08487790960228297 evalInputDelta=-1.1920928955078125E-6
New Minimum: -1485.4055110186978 > -1485.4055929280325
END: th(4.360559495262128E-4)=-1485.4055929280325; dx=-0.08505096239192442 evalInputDelta=8.190933476726059E-5
Fitness changed from -1485.4055110186978 to -1485.4055929280325
Iteration 11 complete. Error: -1485.4055929280325 Total: 3.6360; Orientation: 0.6250; Line Search: 2.2508
Adding measurement 5e57abe4 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.232e+04, gradient 3.144e-01, dot 0.078; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
th(0)=-1485.4055929280325;dx=-1285.7225294276661
New Minimum: -1485.4055929280325 > -1485.750446715534
END: th(9.39454064454065E-4)=-1485.750446715534; dx=-395.1803459590833 evalInputDelta=0.3448537875015063
Fitness changed from -1485.4055929280325 to -1485.750446715534
Iteration 12 complete. Error: -1485.750446715534 Total: 2.7412; Orientation: 0.2722; Line Search: 1.5118
Adding measurement 301da879 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 9.533e+05, gradient 3.143e-01, dot -0.062; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1485.750446715534, -1485.4055929280325, -78.48095369883934, -25.74364442670906, -5.646259598058094
Rejected: LBFGS Orientation magnitude: 8.225e+05, gradient 3.143e-01, dot -0.190; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1485.750446715534, -1485.4055929280325, -78.48095369883934, -25.74364442670906
LBFGS Accumulation History: 3 points
Removed measurement 301da879 to history. Total: 4
Removed measurement 5e57abe4 to history. Total: 3
th(0)=-1485.750446715534;dx=-0.09866514663621695
New Minimum: -1485.750446715534 > -1485.7506128769794
END: th(0.0020239924261512866)=-1485.7506128769794; dx=-0.0849446276742385 evalInputDelta=1.6616144534964405E-4
Fitness changed from -1485.750446715534 to -1485.7506128769794
Iteration 13 complete. Error: -1485.7506128769794 Total: 2.8668; Orientation: 0.6609; Line Search: 1.5310
Adding measurement 597ffe1b to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.781e+04, gradient 3.143e-01, dot 0.071; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
th(0)=-1485.7506128769794;dx=-1061.440317254795
Armijo: th(0.004360559495262127)=-1484.5083725742193; dx=39.235927247363165 evalInputDelta=-1.2422403027601376
Armijo: th(0.0021802797476310636)=-1485.71765119689; dx=-158.16135570965506 evalInputDelta=-0.03296168008932909
New Minimum: -1485.7506128769794 > -1485.8473135503789
END: th(7.267599158770212E-4)=-1485.8473135503789; dx=-296.04555768680996 evalInputDelta=0.09670067339948218
Fitness changed from -1485.7506128769794 to -1485.8473135503789
Iteration 14 complete. Error: -1485.8473135503789 Total: 4.1216; Orientation: 0.2899; Line Search: 3.1429
Adding measurement 1811b7d1 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.384e+06, gradient 3.143e-01, dot -0.058; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1485.8473135503789, -1485.7506128769794, -78.48095369883934, -25.74364442670906, -5.646259598058094
Rejected: LBFGS Orientation magnitude: 2.157e+06, gradient 3.143e-01, dot -0.189; [e87a8ca1-68db-429b-98cd-5c915bc10a9b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1485.8473135503789, -1485.7506128769794, -78.48095369883934, -25.74364442670906
LBFGS Accumulation History: 3 points
Removed measurement 1811b7d1 to history. Total: 4
Removed measurement 597ffe1b to history. Total: 3
th(0)=-1485.8473135503789;dx=-0.09870231724163882
New Minimum: -1485.8473135503789 > -1485.8474796987784
END: th(0.0015657567740901082)=-1485.8474796987784; dx=-0.08492067825369384 evalInputDelta=1.6614839955764182E-4
Fitness changed from -1485.8473135503789 to -1485.8474796987784
Iteration 15 complete. Error: -1485.8474796987784 Total: 2.9422; Orientation: 0.6533; Line Search: 1.5715
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1485.8474796987784 (> -Infinity) after 59.497s (< 5400.000s)
Returns
-1485.8474796987784