BasicOptimizer.scala:89 executed in 68.30 seconds (1.323 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 156313483337200
Reset training subject: 156314399436900
Adding measurement 50a33f3c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-58.33706804719413;dx=-0.004500404049440785
New Minimum: -58.33706804719413 > -58.345244520107435
WOLFE (weak): th(2.154434690031884)=-58.345244520107435; dx=-0.004450300527478881 evalInputDelta=0.008176472913305588
New Minimum: -58.345244520107435 > -58.35342710732923
WOLFE (weak): th(4.308869380063768)=-58.35342710732923; dx=-0.004464356595395949 evalInputDelta=0.0163590601351018
New Minimum: -58.35342710732923 > -58.38616412845728
WOLFE (weak): th(12.926608140191302)=-58.38616412845728; dx=-0.004462595786331407 evalInputDelta=0.04909608126315135
New Minimum: -58.38616412845728 > -58.533527366503996
WOLFE (weak): th(51.70643256076521)=-58.533527366503996; dx=-0.004467733515525004 evalInputDelta=0.1964593193098665
New Minimum: -58.533527366503996 > -59.31957496244697
WOLFE (weak): th(258.53216280382605)=-59.31957496244697; dx=-0.004456196246484176 evalInputDelta=0.9825069152528414
New Minimum: -59.31957496244697 > -64.04547616803711
WOLFE (weak): th(1551.1929768229563)=-64.04547616803711; dx=-0.00420673418613876 evalInputDelta=5.708408120842982
New Minimum: -64.04547616803711 > -85.04141397104365
END: th(10858.350837760694)=-85.04141397104365; dx=-0.0020133838146613476 evalInputDelta=26.704345923849523
Fitness changed from -58.33706804719413 to -85.04141397104365
Iteration 1 complete. Error: -85.04141397104365 Total: 10.3474; Orientation: 0.0162; Line Search: 7.7171
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 47209f34 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-85.04141397104365;dx=-0.004810483408514782
New Minimum: -85.04141397104365 > -113.56988637798341
END: th(23393.607721408407)=-113.56988637798341; dx=-5.600656616267289E-4 evalInputDelta=28.528472406939756
Fitness changed from -85.04141397104365 to -113.56988637798341
Iteration 2 complete. Error: -113.56988637798341 Total: 2.6779; Orientation: 0.0149; Line Search: 1.7282
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement d11405f to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-113.56988637798341;dx=-0.003400917989780733
New Minimum: -113.56988637798341 > -162.3287062796027
END: th(50400.00000000001)=-162.3287062796027; dx=-1.47229182513218E-4 evalInputDelta=48.75881990161929
Fitness changed from -113.56988637798341 to -162.3287062796027
Iteration 3 complete. Error: -162.3287062796027 Total: 2.8157; Orientation: 0.0146; Line Search: 1.6980
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4437d606 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.272e+03, gradient 5.540e-02, dot -0.286; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -162.3287062796027, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 4437d606 to history. Total: 3
th(0)=-162.3287062796027;dx=-0.0030618181891715066
New Minimum: -162.3287062796027 > -196.96787557805945
WOLF (strong): th(108583.50837760697)=-196.96787557805945; dx=4.2027546585268856E-5 evalInputDelta=34.63916929845675
New Minimum: -196.96787557805945 > -207.2288181248497
END: th(54291.75418880348)=-207.2288181248497; dx=-1.897823625817994E-4 evalInputDelta=44.90011184524701
Fitness changed from -162.3287062796027 to -207.2288181248497
Iteration 4 complete. Error: -207.2288181248497 Total: 3.7920; Orientation: 0.2746; Line Search: 2.5990
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 38526c40 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.037e+03, gradient 5.040e-02, dot -0.253; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -207.2288181248497, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 38526c40 to history. Total: 3
th(0)=-207.2288181248497;dx=-0.002525266762043561
New Minimum: -207.2288181248497 > -241.35733626723854
WOLF (strong): th(116968.03860704206)=-241.35733626723854; dx=1.8134351473642052E-4 evalInputDelta=34.12851814238883
New Minimum: -241.35733626723854 > -269.70628250065135
END: th(58484.01930352103)=-269.70628250065135; dx=-2.6269938371128836E-4 evalInputDelta=62.477464375801645
Fitness changed from -207.2288181248497 to -269.70628250065135
Iteration 5 complete. Error: -269.70628250065135 Total: 3.8990; Orientation: 0.2730; Line Search: 2.7420
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2cfe812f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.993e+03, gradient 5.208e-02, dot -0.292; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -269.70628250065135, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 2cfe812f to history. Total: 3
th(0)=-269.70628250065135;dx=-0.0027037530567688038
New Minimum: -269.70628250065135 > -288.2356193054394
WOLF (strong): th(126000.00000000004)=-288.2356193054394; dx=1.3817806777384549E-4 evalInputDelta=18.529336804788045
New Minimum: -288.2356193054394 > -322.6060171047358
END: th(63000.00000000002)=-322.6060171047358; dx=-8.465811004825655E-6 evalInputDelta=52.89973460408447
Fitness changed from -269.70628250065135 to -322.6060171047358
Iteration 6 complete. Error: -322.6060171047358 Total: 3.7624; Orientation: 0.2731; Line Search: 2.5735
Adding measurement 30cfb831 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.962e+03, gradient 5.320e-02, dot -0.274; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -322.6060171047358, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 30cfb831 to history. Total: 3
th(0)=-322.6060171047358;dx=-0.002803402593373596
Armijo: th(135729.38547200873)=-318.38830480749266; dx=3.057772042818526E-4 evalInputDelta=-4.2177122972431675
New Minimum: -322.6060171047358 > -379.7050429290398
WOLF (strong): th(67864.69273600436)=-379.7050429290398; dx=1.2428116929191665E-4 evalInputDelta=57.099025824303965
END: th(22621.564245334786)=-372.9962106754116; dx=-0.0014280778767585666 evalInputDelta=50.39019357067576
Fitness changed from -322.6060171047358 to -379.7050429290398
Iteration 7 complete. Error: -379.7050429290398 Total: 4.8423; Orientation: 0.3078; Line Search: 3.6907
Adding measurement 25dd1dd4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.422e+04, gradient 5.611e-02, dot -0.247; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -379.7050429290398, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 25dd1dd4 to history. Total: 3
th(0)=-379.7050429290398;dx=-0.003140285216648395
New Minimum: -379.7050429290398 > -448.6796573513278
END: th(48736.6827529342)=-448.6796573513278; dx=-3.0026575619264797E-4 evalInputDelta=68.97461442228803
Fitness changed from -379.7050429290398 to -448.6796573513278
Iteration 8 complete. Error: -448.6796573513278 Total: 3.1323; Orientation: 0.3057; Line Search: 2.0283
Adding measurement 4752be6f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.298e+04, gradient 5.070e-02, dot -0.347; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -448.6796573513278, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 4752be6f to history. Total: 3
th(0)=-448.6796573513278;dx=-0.00256141625693694
New Minimum: -448.6796573513278 > -466.3497843327529
WOLF (strong): th(105000.00000000004)=-466.3497843327529; dx=2.177831664845353E-4 evalInputDelta=17.670126981425085
New Minimum: -466.3497843327529 > -502.15109930689994
END: th(52500.00000000002)=-502.15109930689994; dx=-2.0879791569379576E-4 evalInputDelta=53.47144195557212
Fitness changed from -448.6796573513278 to -502.15109930689994
Iteration 9 complete. Error: -502.15109930689994 Total: 3.6161; Orientation: 0.2711; Line Search: 2.5152
Adding measurement 4afa437a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.574e+05, gradient 5.367e-02, dot -0.359; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -502.15109930689994, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 4afa437a to history. Total: 3
th(0)=-502.15109930689994;dx=-0.002869532375663278
Armijo: th(113107.82122667394)=-487.84140497147075; dx=3.1412814225476255E-4 evalInputDelta=-14.309694335429185
New Minimum: -502.15109930689994 > -540.8131770579679
WOLF (strong): th(56553.91061333697)=-540.8131770579679; dx=1.5901049887852677E-4 evalInputDelta=38.662077751067955
END: th(18851.30353777899)=-540.6634050278409; dx=-0.0013602936095610005 evalInputDelta=38.51230572094096
Fitness changed from -502.15109930689994 to -540.8131770579679
Iteration 10 complete. Error: -540.8131770579679 Total: 4.4128; Orientation: 0.2703; Line Search: 3.3496
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 2d6912ec to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.969e+05, gradient 5.737e-02, dot -0.326; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -540.8131770579679, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 2d6912ec to history. Total: 3
th(0)=-540.8131770579679;dx=-0.0032802799089227683
New Minimum: -540.8131770579679 > -601.4164172739229
END: th(40613.90229411183)=-601.4164172739229; dx=-3.1662505834152384E-4 evalInputDelta=60.603240215955
Fitness changed from -540.8131770579679 to -601.4164172739229
Iteration 11 complete. Error: -601.4164172739229 Total: 3.1844; Orientation: 0.3163; Line Search: 1.7134
Adding measurement 20d02f2a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.311e+04, gradient 5.065e-02, dot -0.320; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Orientation rejected. Popping history element from -601.4164172739229, -113.56988637798341, -85.04141397104365, -58.33706804719413
LBFGS Accumulation History: 3 points
Removed measurement 20d02f2a to history. Total: 3
th(0)=-601.4164172739229;dx=-0.002549214646303432
New Minimum: -601.4164172739229 > -611.4151975187899
WOLF (strong): th(87500.00000000004)=-611.4151975187899; dx=2.0734430935916148E-4 evalInputDelta=9.99878024486702
New Minimum: -611.4151975187899 > -640.7387140992232
END: th(43750.00000000002)=-640.7387140992232; dx=-2.1640244339797175E-4 evalInputDelta=39.32229682530033
Fitness changed from -601.4164172739229 to -640.7387140992232
Iteration 12 complete. Error: -640.7387140992232 Total: 3.7254; Orientation: 0.2819; Line Search: 2.6303
Adding measurement d8e57a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.687e+04, gradient 5.374e-02, dot 0.428; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-640.7387140992232;dx=-849.9261107548984
Armijo: th(94256.51768889496)=-309.70918185770154; dx=0.001810970098854661 evalInputDelta=-331.0295322415217
Armijo: th(47128.25884444748)=-309.71177535068773; dx=0.003615974979689339 evalInputDelta=-331.0269387485355
Armijo: th(15709.419614815828)=-309.71163309214177; dx=0.010807432780278103 evalInputDelta=-331.02708100708145
Armijo: th(3927.354903703957)=-309.8578185249434; dx=0.04331238210778636 evalInputDelta=-330.8808955742798
Armijo: th(785.4709807407914)=-310.2394987830509; dx=0.21600480440566516 evalInputDelta=-330.4992153161723
Armijo: th(130.91183012346525)=-312.9106155790727; dx=1.3066984967662068 evalInputDelta=-327.8280985201505
Armijo: th(18.701690017637894)=-334.7467786911039; dx=9.279240234775767 evalInputDelta=-305.99193540811933
Armijo: th(2.3377112522047367)=-530.9270677612141; dx=39.89221225123287 evalInputDelta=-109.81164633800915
New Minimum: -640.7387140992232 > -725.1925066488694
END: th(0.2597456946894152)=-725.1925066488694; dx=-187.93522160901105 evalInputDelta=84.45379254964621
Fitness changed from -640.7387140992232 to -725.1925066488694
Iteration 13 complete. Error: -725.1925066488694 Total: 10.3165; Orientation: 0.2795; Line Search: 9.2082
Adding measurement 67dc4d53 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 2.215e+04, gradient 5.534e-02, dot 0.227; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
th(0)=-725.1925066488694;dx=-278.24626179304244
Armijo: th(0.5596051352253066)=-707.6710198990365; dx=-17.03361518534335 evalInputDelta=-17.52148674983289
Armijo: th(0.2798025676126533)=-721.5166864282323; dx=-37.055560272100664 evalInputDelta=-3.6758202206370925
New Minimum: -725.1925066488694 > -725.6464014187958
END: th(0.0932675225375511)=-725.6464014187958; dx=-57.703743229983694 evalInputDelta=0.45389476992636446
Fitness changed from -725.1925066488694 to -725.6464014187958
Iteration 14 complete. Error: -725.6464014187958 Total: 4.5466; Orientation: 0.3444; Line Search: 3.4063
Adding measurement 1bff2ea3 to history. Total: 5
Accepted: LBFGS Orientation magnitude: 1.270e+04, gradient 5.577e-02, dot 0.225; [2be0159c-802a-4860-a3ea-9c5ed7e8715d = 1.000/1.000e+00]
th(0)=-725.6464014187958;dx=-158.69827367137793
New Minimum: -725.6464014187958 > -728.8362850039404
END: th(0.20093878600823065)=-728.8362850039404; dx=-27.457838743418968 evalInputDelta=3.189883585144571
Fitness changed from -725.6464014187958 to -728.8362850039404
Iteration 15 complete. Error: -728.8362850039404 Total: 3.1279; Orientation: 0.4030; Line Search: 1.7734
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -728.8362850039404 (> -Infinity) after 68.302s (< 5400.000s)
Returns
-728.8362850039404