BasicOptimizer.scala:89 executed in 47.92 seconds (1.676 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 213050779712300
Reset training subject: 213052100508900
Adding measurement 74ba6ff5 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.459529975283067;dx=-5.441902084927909E-5
New Minimum: -3.459529975283067 > -3.459601166500831
WOLFE (weak): th(2.154434690031884)=-3.459601166500831; dx=-5.441447955859963E-5 evalInputDelta=7.11912177639995E-5
New Minimum: -3.459601166500831 > -3.459672609171478
WOLFE (weak): th(4.308869380063768)=-3.459672609171478; dx=-5.428056066441906E-5 evalInputDelta=1.4263388841095903E-4
New Minimum: -3.459672609171478 > -3.4599578378782527
WOLFE (weak): th(12.926608140191302)=-3.4599578378782527; dx=-5.426384892951653E-5 evalInputDelta=4.278625951856796E-4
New Minimum: -3.4599578378782527 > -3.461241995271637
WOLFE (weak): th(51.70643256076521)=-3.461241995271637; dx=-5.4219141961764166E-5 evalInputDelta=0.0017120199885698462
New Minimum: -3.461241995271637 > -3.4680601072168584
WOLFE (weak): th(258.53216280382605)=-3.4680601072168584; dx=-5.381759006993369E-5 evalInputDelta=0.008530131933791374
New Minimum: -3.4680601072168584 > -3.508920411549508
WOLFE (weak): th(1551.1929768229563)=-3.508920411549508; dx=-5.143297150745477E-5 evalInputDelta=0.04939043626644102
New Minimum: -3.508920411549508 > -3.7424528705237297
END: th(10858.350837760694)=-3.7424528705237297; dx=-3.987144557231027E-5 evalInputDelta=0.2829228952406626
Fitness changed from -3.459529975283067 to -3.7424528705237297
Iteration 1 complete. Error: -3.7424528705237297 Total: 8.6933; Orientation: 0.0769; Line Search: 5.8328
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 20b54cfe to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.7424528705237297;dx=-7.140702909728115E-5
New Minimum: -3.7424528705237297 > -4.226899907294741
END: th(23393.607721408407)=-4.226899907294741; dx=-3.635684015589474E-5 evalInputDelta=0.4844470367710114
Fitness changed from -3.7424528705237297 to -4.226899907294741
Iteration 2 complete. Error: -4.226899907294741 Total: 1.8813; Orientation: 0.0145; Line Search: 1.2490
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 74707df8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.226899907294741;dx=-7.45855428940184E-5
New Minimum: -4.226899907294741 > -4.740649797335828
END: th(50400.00000000001)=-4.740649797335828; dx=-2.2321387479763528E-5 evalInputDelta=0.5137498900410868
Fitness changed from -4.226899907294741 to -4.740649797335828
Iteration 3 complete. Error: -4.740649797335828 Total: 1.8108; Orientation: 0.0133; Line Search: 1.2121
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6a1d1467 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.815e+03, gradient 8.360e-03, dot -0.302; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.740649797335828, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 6a1d1467 to history. Total: 3
th(0)=-4.740649797335828;dx=-6.977092345348241E-5
Armijo: th(108583.50837760697)=-3.9672103256608766; dx=-7.292422063765567E-6 evalInputDelta=-0.7734394716749513
New Minimum: -4.740649797335828 > -5.082164700990325
END: th(54291.75418880348)=-5.082164700990325; dx=-2.1556226254353145E-5 evalInputDelta=0.3415149036544971
Fitness changed from -4.740649797335828 to -5.082164700990325
Iteration 4 complete. Error: -5.082164700990325 Total: 3.2385; Orientation: 0.4383; Line Search: 2.1929
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 37c87fcc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.965e+03, gradient 8.305e-03, dot -0.278; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.082164700990325, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 37c87fcc to history. Total: 3
th(0)=-5.082164700990325;dx=-6.8755593337526E-5
Armijo: th(116968.03860704206)=-3.952059316944527; dx=-7.674982375214756E-6 evalInputDelta=-1.1301053840457982
New Minimum: -5.082164700990325 > -5.34041103295415
END: th(58484.01930352103)=-5.34041103295415; dx=-2.0410784493299633E-5 evalInputDelta=0.25824633196382507
Fitness changed from -5.082164700990325 to -5.34041103295415
Iteration 5 complete. Error: -5.34041103295415 Total: 2.9799; Orientation: 0.3242; Line Search: 2.0217
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1c2cdc67 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.035e+03, gradient 7.965e-03, dot -0.237; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.34041103295415, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 1c2cdc67 to history. Total: 3
th(0)=-5.34041103295415;dx=-6.309230843976751E-5
Armijo: th(126000.00000000004)=-4.149874586446204; dx=-9.072315971292512E-6 evalInputDelta=-1.190536446507946
New Minimum: -5.34041103295415 > -5.5813536125566525
END: th(63000.00000000002)=-5.5813536125566525; dx=-2.1019316844578447E-5 evalInputDelta=0.24094257960250243
Fitness changed from -5.34041103295415 to -5.5813536125566525
Iteration 6 complete. Error: -5.5813536125566525 Total: 2.9901; Orientation: 0.3103; Line Search: 2.1061
Adding measurement 1f547af8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.674e+03, gradient 7.303e-03, dot -0.229; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.5813536125566525, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 1f547af8 to history. Total: 3
th(0)=-5.5813536125566525;dx=-5.3195206383402625E-5
Armijo: th(135729.38547200873)=-4.294577963902874; dx=-9.493883465317052E-6 evalInputDelta=-1.2867756486537782
New Minimum: -5.5813536125566525 > -5.747552657604977
END: th(67864.69273600436)=-5.747552657604977; dx=-2.0681879206032604E-5 evalInputDelta=0.16619904504832483
Fitness changed from -5.5813536125566525 to -5.747552657604977
Iteration 7 complete. Error: -5.747552657604977 Total: 2.7774; Orientation: 0.2956; Line Search: 1.8871
Adding measurement 5c9e6d95 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.912e+03, gradient 6.957e-03, dot -0.235; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.747552657604977, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 5c9e6d95 to history. Total: 3
th(0)=-5.747552657604977;dx=-4.818282040293052E-5
Armijo: th(146210.0482588026)=-4.3927450027912425; dx=-9.531706534048968E-6 evalInputDelta=-1.3548076548137349
New Minimum: -5.747552657604977 > -5.88291545465456
END: th(73105.0241294013)=-5.88291545465456; dx=-1.995818244802512E-5 evalInputDelta=0.13536279704958254
Fitness changed from -5.747552657604977 to -5.88291545465456
Iteration 8 complete. Error: -5.88291545465456 Total: 2.7428; Orientation: 0.3103; Line Search: 1.8614
Adding measurement 74787609 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.989e+03, gradient 6.791e-03, dot -0.225; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.88291545465456, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 74787609 to history. Total: 3
th(0)=-5.88291545465456;dx=-4.594317376283346E-5
Armijo: th(157500.00000000006)=-4.290586906214655; dx=-9.612281299238632E-6 evalInputDelta=-1.5923285484399052
New Minimum: -5.88291545465456 > -5.9196342882835715
END: th(78750.00000000003)=-5.9196342882835715; dx=-1.8674501817847585E-5 evalInputDelta=0.03671883362901163
Fitness changed from -5.88291545465456 to -5.9196342882835715
Iteration 9 complete. Error: -5.9196342882835715 Total: 3.1567; Orientation: 0.3618; Line Search: 2.2146
Adding measurement 6e1d9b32 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.306e+03, gradient 6.501e-03, dot -0.228; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.9196342882835715, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 6e1d9b32 to history. Total: 3
th(0)=-5.9196342882835715;dx=-4.2034791825956757E-5
Armijo: th(169661.7318400109)=-4.259194917910309; dx=-8.565356428421949E-6 evalInputDelta=-1.660439370373263
New Minimum: -5.9196342882835715 > -5.978938103496615
END: th(84830.86592000545)=-5.978938103496615; dx=-1.794042959559149E-5 evalInputDelta=0.05930381521304362
Fitness changed from -5.9196342882835715 to -5.978938103496615
Iteration 10 complete. Error: -5.978938103496615 Total: 2.8280; Orientation: 0.2968; Line Search: 1.9094
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 3d213a2b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.128e+03, gradient 6.367e-03, dot -0.240; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.978938103496615, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 3d213a2b to history. Total: 3
th(0)=-5.978938103496615;dx=-4.0289240567487256E-5
Armijo: th(182762.56032350325)=-3.8891657202870054; dx=-7.5801535192028275E-6 evalInputDelta=-2.08977238320961
Armijo: th(91381.28016175162)=-5.8681156525559235; dx=-1.632523092622497E-5 evalInputDelta=-0.11082245094069165
New Minimum: -5.978938103496615 > -6.25413302384165
END: th(30460.426720583873)=-6.25413302384165; dx=-3.0497176153051753E-5 evalInputDelta=0.2751949203450348
Fitness changed from -5.978938103496615 to -6.25413302384165
Iteration 11 complete. Error: -6.25413302384165 Total: 3.9750; Orientation: 0.3096; Line Search: 3.1033
Adding measurement 4f72078d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.578e+03, gradient 5.831e-03, dot -0.265; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.25413302384165, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 4f72078d to history. Total: 3
th(0)=-6.25413302384165;dx=-3.370117162815835E-5
New Minimum: -6.25413302384165 > -6.354085415694517
END: th(65625.00000000003)=-6.354085415694517; dx=-2.1462578476913464E-5 evalInputDelta=0.09995239185286664
Fitness changed from -6.25413302384165 to -6.354085415694517
Iteration 12 complete. Error: -6.354085415694517 Total: 2.0729; Orientation: 0.3112; Line Search: 1.1990
Adding measurement 63d14dbf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.240e+03, gradient 5.823e-03, dot -0.264; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.354085415694517, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 63d14dbf to history. Total: 3
th(0)=-6.354085415694517;dx=-3.3492452334278795E-5
Armijo: th(141384.77653334243)=-5.332064842781829; dx=-1.176356890841408E-5 evalInputDelta=-1.0220205729126874
Armijo: th(70692.38826667122)=-6.341435182937772; dx=-1.8597411551080604E-5 evalInputDelta=-0.012650232756744373
New Minimum: -6.354085415694517 > -6.494121816708127
END: th(23564.129422223737)=-6.494121816708127; dx=-2.7047146942247074E-5 evalInputDelta=0.14003640101361015
Fitness changed from -6.354085415694517 to -6.494121816708127
Iteration 13 complete. Error: -6.494121816708127 Total: 3.3885; Orientation: 0.3144; Line Search: 2.5088
Adding measurement 7a25f8ee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.761e+03, gradient 5.374e-03, dot -0.292; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.494121816708127, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 7a25f8ee to history. Total: 3
th(0)=-6.494121816708127;dx=-2.8740227965689023E-5
New Minimum: -6.494121816708127 > -6.587090987992573
END: th(50767.37786763979)=-6.587090987992573; dx=-2.2137172565049762E-5 evalInputDelta=0.09296917128444626
Fitness changed from -6.494121816708127 to -6.587090987992573
Iteration 14 complete. Error: -6.587090987992573 Total: 2.1169; Orientation: 0.3274; Line Search: 1.2094
Adding measurement 2bd8c8f1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.311e+03, gradient 5.363e-03, dot -0.294; [59bdc241-bbfd-4472-b077-9dd25d73abc1 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.587090987992573, -4.226899907294741, -3.7424528705237297, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 2bd8c8f1 to history. Total: 3
th(0)=-6.587090987992573;dx=-2.8610233324285405E-5
Armijo: th(109375.00000000006)=-6.2413964129296815; dx=-1.486200539464793E-5 evalInputDelta=-0.3456945750628915
New Minimum: -6.587090987992573 > -6.661773096740722
END: th(54687.50000000003)=-6.661773096740722; dx=-2.022846913580639E-5 evalInputDelta=0.07468210874814929
Fitness changed from -6.587090987992573 to -6.661773096740722
Iteration 15 complete. Error: -6.661773096740722 Total: 3.2235; Orientation: 0.6298; Line Search: 1.9880
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -6.661773096740722 (> -Infinity) after 47.898s (< 5400.000s)
Returns
-6.661773096740722