BasicOptimizer.scala:89 executed in 8.37 seconds (0.570 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 166260900582900
Reset training subject: 166261028318300
Adding measurement 3cf90397 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-9.49003500421865;dx=-0.05731597662405999
New Minimum: -9.49003500421865 > -9.554601981452997
WOLFE (weak): th(2.154434690031884)=-9.554601981452997; dx=-0.057114650254735916 evalInputDelta=0.06456697723434779
New Minimum: -9.554601981452997 > -9.619113120101849
WOLFE (weak): th(4.308869380063768)=-9.619113120101849; dx=-0.05681866349084909 evalInputDelta=0.12907811588319973
New Minimum: -9.619113120101849 > -9.881609004491946
WOLFE (weak): th(12.926608140191302)=-9.881609004491946; dx=-0.05658997644039308 evalInputDelta=0.39157400027329636
New Minimum: -9.881609004491946 > -11.051696644423208
WOLFE (weak): th(51.70643256076521)=-11.051696644423208; dx=-0.05926642585675186 evalInputDelta=1.5616616402045587
New Minimum: -11.051696644423208 > -16.31412811018025
END: th(258.53216280382605)=-16.31412811018025; dx=-0.03191399068184998 evalInputDelta=6.8240931059616
Fitness changed from -9.49003500421865 to -16.31412811018025
Iteration 1 complete. Error: -16.31412811018025 Total: 1.0233; Orientation: 0.0015; Line Search: 0.6734
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4ee8792c to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-16.31412811018025;dx=-0.04975482326682372
New Minimum: -16.31412811018025 > -27.810673639934876
END: th(556.9906600335335)=-27.810673639934876; dx=-0.02033808905281075 evalInputDelta=11.496545529754627
Fitness changed from -16.31412811018025 to -27.810673639934876
Iteration 2 complete. Error: -27.810673639934876 Total: 0.3421; Orientation: 0.0014; Line Search: 0.2182
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6713e12c to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-27.810673639934876;dx=-0.051251301383263344
New Minimum: -27.810673639934876 > -45.847142732774394
END: th(1200.0)=-45.847142732774394; dx=-0.0027790520597399736 evalInputDelta=18.036469092839518
Fitness changed from -27.810673639934876 to -45.847142732774394
Iteration 3 complete. Error: -45.847142732774394 Total: 0.3442; Orientation: 0.0016; Line Search: 0.2279
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5e6ba570 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.577e+02, gradient 2.274e-01, dot -0.246; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.847142732774394, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 5e6ba570 to history. Total: 3
th(0)=-45.847142732774394;dx=-0.050875395489270656
Armijo: th(2585.3216280382608)=-36.67558065889892; dx=1.6210574384277558E-4 evalInputDelta=-9.17156207387547
New Minimum: -45.847142732774394 > -62.88301543674504
END: th(1292.6608140191304)=-62.88301543674504; dx=-0.004528477736985604 evalInputDelta=17.035872703970647
Fitness changed from -45.847142732774394 to -62.88301543674504
Iteration 4 complete. Error: -62.88301543674504 Total: 0.4906; Orientation: 0.0281; Line Search: 0.3435
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 138c93e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.216e+02, gradient 2.505e-01, dot -0.210; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -62.88301543674504, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 138c93e3 to history. Total: 3
th(0)=-62.88301543674504;dx=-0.06229462578331077
Armijo: th(2784.9533001676677)=-46.682652159563084; dx=0.0014181292729412462 evalInputDelta=-16.200363277181957
New Minimum: -62.88301543674504 > -78.80754642371987
END: th(1392.4766500838339)=-78.80754642371987; dx=-0.0038982785216024034 evalInputDelta=15.92453098697483
Fitness changed from -62.88301543674504 to -78.80754642371987
Iteration 5 complete. Error: -78.80754642371987 Total: 0.4891; Orientation: 0.0291; Line Search: 0.3409
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4ea49902 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.010e+03, gradient 2.395e-01, dot -0.205; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -78.80754642371987, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 4ea49902 to history. Total: 3
th(0)=-78.80754642371987;dx=-0.055471327646346456
Armijo: th(3000.0000000000005)=-71.67073606736076; dx=0.0016253525664042786 evalInputDelta=-7.136810356359106
New Minimum: -78.80754642371987 > -97.24606581658679
END: th(1500.0000000000002)=-97.24606581658679; dx=-0.0035202551255062832 evalInputDelta=18.438519392866922
Fitness changed from -78.80754642371987 to -97.24606581658679
Iteration 6 complete. Error: -97.24606581658679 Total: 0.4819; Orientation: 0.0282; Line Search: 0.3349
Adding measurement 6e26001b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.440e+03, gradient 2.478e-01, dot -0.208; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -97.24606581658679, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 6e26001b to history. Total: 3
th(0)=-97.24606581658679;dx=-0.060004631749605276
Armijo: th(3231.6520350478263)=-83.65714947982052; dx=6.805114820844541E-4 evalInputDelta=-13.588916336766275
New Minimum: -97.24606581658679 > -105.03217805016196
END: th(1615.8260175239132)=-105.03217805016196; dx=-0.004242589868141681 evalInputDelta=7.786112233575167
Fitness changed from -97.24606581658679 to -105.03217805016196
Iteration 7 complete. Error: -105.03217805016196 Total: 0.4723; Orientation: 0.0288; Line Search: 0.3355
Adding measurement 28aa915d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.653e+03, gradient 2.363e-01, dot -0.192; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -105.03217805016196, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 28aa915d to history. Total: 3
th(0)=-105.03217805016196;dx=-0.0551852080285026
New Minimum: -105.03217805016196 > -107.7510216792772
END: th(3481.1916252095853)=-107.7510216792772; dx=-0.0017958278216919062 evalInputDelta=2.7188436291152414
Fitness changed from -105.03217805016196 to -107.7510216792772
Iteration 8 complete. Error: -107.7510216792772 Total: 0.3601; Orientation: 0.0283; Line Search: 0.2238
Adding measurement 3a1af3c6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.264e+03, gradient 2.627e-01, dot -0.149; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -107.7510216792772, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 3a1af3c6 to history. Total: 3
th(0)=-107.7510216792772;dx=-0.06798345785093322
Armijo: th(7500.000000000003)=-64.88444086812922; dx=0.0036686903722410306 evalInputDelta=-42.86658081114798
New Minimum: -107.7510216792772 > -138.0772283526031
WOLF (strong): th(3750.0000000000014)=-138.0772283526031; dx=0.0031974401066826984 evalInputDelta=30.3262066733259
New Minimum: -138.0772283526031 > -153.7413866686655
END: th(1250.0000000000005)=-153.7413866686655; dx=-0.018048550969082894 evalInputDelta=45.99036498938831
Fitness changed from -107.7510216792772 to -153.7413866686655
Iteration 9 complete. Error: -153.7413866686655 Total: 0.5811; Orientation: 0.0272; Line Search: 0.4446
Adding measurement 3c02c3a7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.169e+03, gradient 2.181e-01, dot -0.224; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -153.7413866686655, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 3c02c3a7 to history. Total: 3
th(0)=-153.7413866686655;dx=-0.04656835229666019
New Minimum: -153.7413866686655 > -182.26643692956839
END: th(2693.043362539856)=-182.26643692956839; dx=-0.00866660159635669 evalInputDelta=28.525050260902873
Fitness changed from -153.7413866686655 to -182.26643692956839
Iteration 10 complete. Error: -182.26643692956839 Total: 0.3697; Orientation: 0.0294; Line Search: 0.2326
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6e22aa5c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.122e+04, gradient 2.336e-01, dot -0.234; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -182.26643692956839, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 6e22aa5c to history. Total: 3
th(0)=-182.26643692956839;dx=-0.05303946142678354
New Minimum: -182.26643692956839 > -189.39171537713685
WOLF (strong): th(5801.986042015977)=-189.39171537713685; dx=0.0015058081823539058 evalInputDelta=7.125278447568462
New Minimum: -189.39171537713685 > -228.22549501868616
END: th(2900.9930210079883)=-228.22549501868616; dx=-0.008151594721701138 evalInputDelta=45.959058089117775
Fitness changed from -182.26643692956839 to -228.22549501868616
Iteration 11 complete. Error: -228.22549501868616 Total: 0.4945; Orientation: 0.0278; Line Search: 0.3478
Adding measurement 30e4cd7a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.952e+05, gradient 2.442e-01, dot -0.221; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -228.22549501868616, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 30e4cd7a to history. Total: 3
th(0)=-228.22549501868616;dx=-0.058283870680119185
Armijo: th(6250.000000000004)=-156.3649875343358; dx=0.006687906333378995 evalInputDelta=-71.86050748435036
New Minimum: -228.22549501868616 > -241.6110544296122
END: th(3125.000000000002)=-241.6110544296122; dx=-7.361625014380133E-4 evalInputDelta=13.38555941092605
Fitness changed from -228.22549501868616 to -241.6110544296122
Iteration 12 complete. Error: -241.6110544296122 Total: 0.4657; Orientation: 0.0271; Line Search: 0.3309
Adding measurement 14b1a9fd to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.150e+03, gradient 2.764e-01, dot 0.292; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-241.6110544296122;dx=-336.76670357603723
Armijo: th(6732.608406349641)=-149.73157472908568; dx=0.01198075652458764 evalInputDelta=-91.87947970052653
Armijo: th(3366.3042031748205)=-149.21943335900838; dx=0.02248755416903027 evalInputDelta=-92.39162107060383
Armijo: th(1122.1014010582735)=-150.02956168289677; dx=0.06875993530003592 evalInputDelta=-91.58149274671544
Armijo: th(280.5253502645684)=-150.0602038128816; dx=0.25964011392780956 evalInputDelta=-91.5508506167306
Armijo: th(56.105070052913675)=-163.15047444980763; dx=1.1235166357145778 evalInputDelta=-78.46057997980458
Armijo: th(9.350845008818945)=-234.05806843415138; dx=6.855512040170611 evalInputDelta=-7.55298599546083
New Minimum: -241.6110544296122 > -389.26750950825505
END: th(1.3358350012598492)=-389.26750950825505; dx=-42.92471750118125 evalInputDelta=147.65645507864284
Fitness changed from -241.6110544296122 to -389.26750950825505
Iteration 13 complete. Error: -389.26750950825505 Total: 1.6730; Orientation: 0.0289; Line Search: 1.5354
Adding measurement 6ca46d8c to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.381e+03, gradient 2.299e-01, dot -0.030; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -389.26750950825505, -241.6110544296122, -27.810673639934876, -16.31412811018025, -9.49003500421865
Rejected: LBFGS Orientation magnitude: 7.825e+03, gradient 2.299e-01, dot -0.169; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -389.26750950825505, -241.6110544296122, -27.810673639934876, -16.31412811018025
LBFGS Accumulation History: 3 points
Removed measurement 6ca46d8c to history. Total: 4
Removed measurement 14b1a9fd to history. Total: 3
th(0)=-389.26750950825505;dx=-0.0518554612187589
New Minimum: -389.26750950825505 > -389.4005772846681
END: th(2.8779692668730044)=-389.4005772846681; dx=-0.04473861434884582 evalInputDelta=0.1330677764130428
Fitness changed from -389.26750950825505 to -389.4005772846681
Iteration 14 complete. Error: -389.4005772846681 Total: 0.3957; Orientation: 0.0622; Line Search: 0.2245
Adding measurement 1fbdb8f5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.643e+03, gradient 2.288e-01, dot -0.141; [aebd9bd5-db48-40e1-8451-fde3bd790d38 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -389.4005772846681, -27.810673639934876, -16.31412811018025, -9.49003500421865
LBFGS Accumulation History: 3 points
Removed measurement 1fbdb8f5 to history. Total: 3
th(0)=-389.4005772846681;dx=-0.051552123526280594
New Minimum: -389.4005772846681 > -389.68608743935573
END: th(6.200396825396829)=-389.68608743935573; dx=-0.04479647126162363 evalInputDelta=0.2855101546876426
Fitness changed from -389.4005772846681 to -389.68608743935573
Iteration 15 complete. Error: -389.68608743935573 Total: 0.3725; Orientation: 0.0298; Line Search: 0.2265
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -389.68608743935573 (> -Infinity) after 8.369s (< 5400.000s)
Returns
-389.68608743935573