BasicOptimizer.scala:89 executed in 56.97 seconds (0.984 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2865784492290
Reset training subject: 2866218350548
Adding measurement 4367fd9 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.8795892659763558;dx=-5.227299415646435E-4
New Minimum: -1.8795892659763558 > -1.879628128881574
WOLFE (weak): th(2.154434690031884)=-1.879628128881574; dx=-5.227179919957562E-4 evalInputDelta=3.8862905218151056E-5
New Minimum: -1.879628128881574 > -1.8796658629219891
WOLFE (weak): th(4.308869380063768)=-1.8796658629219891; dx=-5.225624731761062E-4 evalInputDelta=7.659694563333375E-5
New Minimum: -1.8796658629219891 > -1.879821472583285
WOLFE (weak): th(12.926608140191302)=-1.879821472583285; dx=-5.225184751187725E-4 evalInputDelta=2.3220660692913064E-4
New Minimum: -1.879821472583285 > -1.8805150451106554
WOLFE (weak): th(51.70643256076521)=-1.8805150451106554; dx=-5.202112003376479E-4 evalInputDelta=9.257791342995869E-4
New Minimum: -1.8805150451106554 > -1.8839941591085037
WOLFE (weak): th(258.53216280382605)=-1.8839941591085037; dx=-5.168789452802788E-4 evalInputDelta=0.004404893132147869
New Minimum: -1.8839941591085037 > -1.9037033343221295
WOLFE (weak): th(1551.1929768229563)=-1.9037033343221295; dx=-4.961825237391278E-4 evalInputDelta=0.024114068345773676
New Minimum: -1.9037033343221295 > -2.033849609329213
END: th(10858.350837760694)=-2.033849609329213; dx=-4.1049673271829375E-4 evalInputDelta=0.1542603433528571
Fitness changed from -1.8795892659763558 to -2.033849609329213
Iteration 1 complete. Error: -2.033849609329213 Total: 5.1707; Orientation: 0.0159; Line Search: 3.8207
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3c199e24 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.033849609329213;dx=-5.327356687851185E-4
New Minimum: -2.033849609329213 > -2.784766843927866
END: th(23393.607721408407)=-2.784766843927866; dx=-3.019187026308625E-4 evalInputDelta=0.7509172345986532
Fitness changed from -2.033849609329213 to -2.784766843927866
Iteration 2 complete. Error: -2.784766843927866 Total: 1.4551; Orientation: 0.0158; Line Search: 0.9755
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 709a8ac4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.784766843927866;dx=-6.303193249525749E-4
New Minimum: -2.784766843927866 > -3.5454294979349164
END: th(50400.00000000001)=-3.5454294979349164; dx=-1.6884628598292192E-4 evalInputDelta=0.7606626540070502
Fitness changed from -2.784766843927866 to -3.5454294979349164
Iteration 3 complete. Error: -3.5454294979349164 Total: 1.4140; Orientation: 0.0153; Line Search: 0.9420
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3553dc40 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.446e+03, gradient 2.177e-02, dot -0.275; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.5454294979349164, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 3553dc40 to history. Total: 3
th(0)=-3.5454294979349164;dx=-4.739129310862097E-4
New Minimum: -3.5454294979349164 > -3.8195139998878926
END: th(108583.50837760697)=-3.8195139998878926; dx=-6.608698741634604E-5 evalInputDelta=0.2740845019529763
Fitness changed from -3.5454294979349164 to -3.8195139998878926
Iteration 4 complete. Error: -3.8195139998878926 Total: 1.9919; Orientation: 0.5721; Line Search: 0.9575
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 226c0001 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.200e+03, gradient 1.989e-02, dot -0.156; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.8195139998878926, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 226c0001 to history. Total: 3
th(0)=-3.8195139998878926;dx=-3.95481444159075E-4
New Minimum: -3.8195139998878926 > -4.552456140718113
END: th(233936.07721408413)=-4.552456140718113; dx=-2.9620093366480262E-5 evalInputDelta=0.7329421408302204
Fitness changed from -3.8195139998878926 to -4.552456140718113
Iteration 5 complete. Error: -4.552456140718113 Total: 1.8309; Orientation: 0.4143; Line Search: 0.9587
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7940d5ac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.727e+03, gradient 1.768e-02, dot -0.111; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.552456140718113, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 7940d5ac to history. Total: 3
th(0)=-4.552456140718113;dx=-3.1250297155533074E-4
Armijo: th(504000.0000000002)=-4.100546038157795; dx=-1.2833504995496436E-5 evalInputDelta=-0.45191010256031827
New Minimum: -4.552456140718113 > -4.780437484908979
END: th(252000.0000000001)=-4.780437484908979; dx=-4.14192098009038E-5 evalInputDelta=0.2279813441908658
Fitness changed from -4.552456140718113 to -4.780437484908979
Iteration 6 complete. Error: -4.780437484908979 Total: 2.8081; Orientation: 0.4051; Line Search: 1.9385
Adding measurement 27a7bf1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.901e+03, gradient 1.582e-02, dot -0.116; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.780437484908979, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 27a7bf1a to history. Total: 3
th(0)=-4.780437484908979;dx=-2.502360217159713E-4
New Minimum: -4.780437484908979 > -5.098785983192726
END: th(542917.5418880349)=-5.098785983192726; dx=-1.462771529016691E-5 evalInputDelta=0.318348498283747
Fitness changed from -4.780437484908979 to -5.098785983192726
Iteration 7 complete. Error: -5.098785983192726 Total: 1.8307; Orientation: 0.4241; Line Search: 0.9433
Adding measurement 472af1eb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.350e+03, gradient 1.286e-02, dot -0.096; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.098785983192726, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 472af1eb to history. Total: 3
th(0)=-5.098785983192726;dx=-1.6518307008485164E-4
New Minimum: -5.098785983192726 > -5.106196704593966
END: th(1169680.3860704207)=-5.106196704593966; dx=-3.6388736055032416E-6 evalInputDelta=0.007410721401240217
Fitness changed from -5.098785983192726 to -5.106196704593966
Iteration 8 complete. Error: -5.106196704593966 Total: 1.7687; Orientation: 0.4047; Line Search: 0.9392
Adding measurement 142d6430 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.553e+03, gradient 8.502e-03, dot -0.065; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.106196704593966, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 142d6430 to history. Total: 3
th(0)=-5.106196704593966;dx=-7.221350640320474E-5
New Minimum: -5.106196704593966 > -5.195741531735578
END: th(2520000.000000001)=-5.195741531735578; dx=-1.7693600442240324E-6 evalInputDelta=0.08954482714161216
Fitness changed from -5.106196704593966 to -5.195741531735578
Iteration 9 complete. Error: -5.195741531735578 Total: 1.8282; Orientation: 0.4093; Line Search: 0.9767
Adding measurement 49a7a77e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.991e+03, gradient 6.675e-03, dot -0.054; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.195741531735578, -2.784766843927866, -2.0338496093292
...skipping 4079 bytes...
Error: -8.526173521736524 Total: 1.7637; Orientation: 0.4123; Line Search: 0.9411
Adding measurement 4ad0c3f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.702e+04, gradient 4.907e-03, dot -0.048; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.526173521736524, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 4ad0c3f to history. Total: 3
th(0)=-8.526173521736524;dx=-2.3930019261230102E-5
Armijo: th(1.575000000000001E7)=-5.4888199738686145; dx=-6.180549137545304E-8 evalInputDelta=-3.0373535478679097
Armijo: th(7875000.000000005)=-7.175254932988053; dx=-8.417602789543507E-7 evalInputDelta=-1.3509185887484714
New Minimum: -8.526173521736524 > -8.835897168503278
END: th(2625000.0000000014)=-8.835897168503278; dx=-5.619804442140257E-6 evalInputDelta=0.3097236467667539
Fitness changed from -8.526173521736524 to -8.835897168503278
Iteration 15 complete. Error: -8.835897168503278 Total: 3.0048; Orientation: 0.4092; Line Search: 2.1511
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 7707da12 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.598e+05, gradient 4.905e-03, dot -0.088; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.835897168503278, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 7707da12 to history. Total: 3
th(0)=-8.835897168503278;dx=-2.394434152306015E-5
Armijo: th(5655391.061333698)=-8.428753778469796; dx=-1.6554706292749284E-6 evalInputDelta=-0.40714339003348243
New Minimum: -8.835897168503278 > -9.105820293406456
END: th(2827695.530666849)=-9.105820293406456; dx=-4.873803639148319E-6 evalInputDelta=0.26992312490317794
Fitness changed from -8.835897168503278 to -9.105820293406456
Iteration 16 complete. Error: -9.105820293406456 Total: 2.4596; Orientation: 0.4901; Line Search: 1.4938
Adding measurement 2af2f81c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.843e+07, gradient 4.916e-03, dot -0.101; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.105820293406456, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 2af2f81c to history. Total: 3
th(0)=-9.105820293406456;dx=-2.4044746191354913E-5
Armijo: th(6092085.344116776)=-9.065891379443505; dx=-1.3737446381310576E-6 evalInputDelta=-0.039928913962951285
New Minimum: -9.105820293406456 > -9.580448635589159
END: th(3046042.672058388)=-9.580448635589159; dx=-4.439186078853806E-6 evalInputDelta=0.47462834218270267
Fitness changed from -9.105820293406456 to -9.580448635589159
Iteration 17 complete. Error: -9.580448635589159 Total: 2.3272; Orientation: 0.4021; Line Search: 1.4891
Adding measurement 42a25b6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.627e+05, gradient 4.777e-03, dot -0.106; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.580448635589159, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 42a25b6 to history. Total: 3
th(0)=-9.580448635589159;dx=-2.2754406501894618E-5
New Minimum: -9.580448635589159 > -10.311690378735767
END: th(6562500.000000005)=-10.311690378735767; dx=-1.1741779427618976E-6 evalInputDelta=0.7312417431466081
Fitness changed from -9.580448635589159 to -10.311690378735767
Iteration 18 complete. Error: -10.311690378735767 Total: 1.8223; Orientation: 0.4039; Line Search: 1.0016
Adding measurement 32aa4944 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.966e+05, gradient 4.583e-03, dot -0.064; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.311690378735767, -2.784766843927866, -2.033849609329213, -1.8795892659763558
LBFGS Accumulation History: 3 points
Removed measurement 32aa4944 to history. Total: 3
th(0)=-10.311690378735767;dx=-2.086147225313219E-5
Armijo: th(1.4138477653334247E7)=-8.483251312417538; dx=-1.8111972952281234E-7 evalInputDelta=-1.8284390663182286
Armijo: th(7069238.8266671235)=-9.594024387933793; dx=-1.2031029212117695E-6 evalInputDelta=-0.717665990801974
New Minimum: -10.311690378735767 > -11.16566037983433
END: th(2356412.9422223745)=-11.16566037983433; dx=-5.899120106727001E-6 evalInputDelta=0.8539700010985634
Fitness changed from -10.311690378735767 to -11.16566037983433
Iteration 19 complete. Error: -11.16566037983433 Total: 2.7485; Orientation: 0.4043; Line Search: 1.9242
Adding measurement 16a498be to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.099e+05, gradient 4.557e-03, dot 0.102; [64ae2332-2fec-4fa9-b891-ddc6945fae1f = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-11.16566037983433;dx=-50.80320492278604
Armijo: th(5076737.786763981)=-8.337784830681993; dx=2.8620488951154687E-7 evalInputDelta=-2.8278755491523366
Armijo: th(2538368.8933819905)=-8.337784830681993; dx=5.709683987062815E-7 evalInputDelta=-2.8278755491523366
Armijo: th(846122.9644606635)=-8.33775634795519; dx=1.7210845300558054E-6 evalInputDelta=-2.8279040318791395
Armijo: th(211530.74111516587)=-8.337568359517947; dx=6.874101120197062E-6 evalInputDelta=-2.828092020316383
Armijo: th(42306.148223033175)=-8.335886040526663; dx=3.419678356207429E-5 evalInputDelta=-2.829774339307667
Armijo: th(7051.0247038388625)=-8.32654449955402; dx=2.054850960975502E-4 evalInputDelta=-2.8391158802803105
Armijo: th(1007.2892434055518)=-8.305198409606422; dx=0.0014366385951841384 evalInputDelta=-2.8604619702279077
Armijo: th(125.91115542569398)=-8.366617228320282; dx=0.012187401535138719 evalInputDelta=-2.7990431515140486
Armijo: th(13.990128380632664)=-8.864898240738226; dx=0.09903087342675779 evalInputDelta=-2.300762139096104
Armijo: th(1.3990128380632663)=-9.57081036981402; dx=0.34028724124316984 evalInputDelta=-1.594850010020311
Armijo: th(0.12718298527847874)=-10.883334039644637; dx=-13.685033665482536 evalInputDelta=-0.2823263401896927
Armijo: th(0.010598582106539895)=-11.140518244389714; dx=-26.200498043679065 evalInputDelta=-0.025142135444616187
Armijo: th(8.15275546656915E-4)=-11.164035917673964; dx=-27.96583831599076 evalInputDelta=-0.001624462160366491
Armijo: th(5.8233967618351075E-5)=-11.165535195869522; dx=-28.27987926564186 evalInputDelta=-1.2518396480842853E-4
Armijo: th(3.882264507890072E-6)=-11.165651264544465; dx=-28.2182117547273 evalInputDelta=-9.115289865135878E-6
Armijo: th(2.426415317431295E-7)=-11.165657949091095; dx=-27.20127814734625 evalInputDelta=-2.430743235137811E-6
Armijo: th(1.4273031279007617E-8)=-11.16565977214857; dx=-28.28854369269976 evalInputDelta=-6.076857594905505E-7
Armijo: th(7.929461821670898E-10)=-11.165657949091095; dx=-28.252140200907647 evalInputDelta=-2.430743235137811E-6
MIN ALPHA (4.173400958774157E-11): th(0.0)=-11.16566037983433
Fitness changed from -11.16566037983433 to -11.16566037983433
Static Iteration Total: 10.8576; Orientation: 0.4225; Line Search: 9.4604
Iteration 20 failed. Error: -11.16566037983433
Previous Error: 0.0 -> -11.16566037983433
Retrying iteration 20
Reset training subject: 2920067767752
Adding measurement 787d72d8 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-11.16566037983433;dx=-2.0649218069823475E-5
Armijo: th(2.1544346900318838E-10)=-11.16565977214857; dx=-1.6903882488768368E-5 evalInputDelta=-6.076857594905505E-7
Armijo: th(1.0772173450159419E-10)=-11.16565977214857; dx=-1.691355441248418E-5 evalInputDelta=-6.076857594905505E-7
MIN ALPHA (3.590724483386473E-11): th(0.0)=-11.16566037983433
Fitness changed from -11.16566037983433 to -11.16566037983433
Static Iteration Total: 2.6892; Orientation: 0.0171; Line Search: 1.8752
Iteration 21 failed. Error: -11.16566037983433
Previous Error: 0.0 -> -11.16566037983433
Optimization terminated 21
Final threshold in iteration 21: -11.16566037983433 (> -Infinity) after 56.973s (< 3600.000s)
Returns
-11.16566037983433