BasicOptimizer.scala:89 executed in 518.32 seconds (5.894 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 3605434835022
Reset training subject: 3606575935328
Adding measurement fa0acb7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.1395512355959886;dx=-8.845544214117864E-5
New Minimum: -0.1395512355959886 > -0.1397257370853759
WOLFE (weak): th(2.154434690031884)=-0.1397257370853759; dx=-8.846070441023631E-5 evalInputDelta=1.7450148938730203E-4
New Minimum: -0.1397257370853759 > -0.13990111932427735
WOLFE (weak): th(4.308869380063768)=-0.13990111932427735; dx=-8.896153280230627E-5 evalInputDelta=3.498837282887579E-4
New Minimum: -0.13990111932427735 > -0.14060532246775595
WOLFE (weak): th(12.926608140191302)=-0.14060532246775595; dx=-8.738798987216616E-5 evalInputDelta=0.0010540868717673624
New Minimum: -0.14060532246775595 > -0.14376483508973997
WOLFE (weak): th(51.70643256076521)=-0.14376483508973997; dx=-8.811808647547648E-5 evalInputDelta=0.004213599493751385
New Minimum: -0.14376483508973997 > -0.16011096151196344
WOLFE (weak): th(258.53216280382605)=-0.16011096151196344; dx=-8.329135364401668E-5 evalInputDelta=0.020559725915974852
New Minimum: -0.16011096151196344 > -0.2642804927636907
END: th(1551.1929768229563)=-0.2642804927636907; dx=-7.644082028600948E-5 evalInputDelta=0.12472925716770211
Fitness changed from -0.1395512355959886 to -0.2642804927636907
Iteration 1 complete. Error: -0.2642804927636907 Total: 11.7994; Orientation: 0.0173; Line Search: 8.3666
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6110339f to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.2642804927636907;dx=-1.0421466967674094E-4
New Minimum: -0.2642804927636907 > -0.6170001409536662
WOLFE (weak): th(3341.943960201201)=-0.6170001409536662; dx=-9.452250644394174E-5 evalInputDelta=0.3527196481899755
New Minimum: -0.6170001409536662 > -0.9155782537928472
END: th(6683.887920402402)=-0.9155782537928472; dx=-7.463170458924936E-5 evalInputDelta=0.6512977610291565
Fitness changed from -0.2642804927636907 to -0.9155782537928472
Iteration 2 complete. Error: -0.9155782537928472 Total: 5.3254; Orientation: 0.0180; Line Search: 4.0227
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 27acd073 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.9155782537928472;dx=-1.503602031591715E-4
New Minimum: -0.9155782537928472 > -2.0831051002099024
END: th(14400.000000000002)=-2.0831051002099024; dx=-3.986193954723357E-5 evalInputDelta=1.1675268464170552
Fitness changed from -0.9155782537928472 to -2.0831051002099024
Iteration 3 complete. Error: -2.0831051002099024 Total: 3.7812; Orientation: 0.0187; Line Search: 2.4743
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 100a4719 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.674e+03, gradient 1.910e-02, dot -0.135; [e3668570-8d2a-4a87-879d-0b63a857dc54 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.0831051002099024, -0.9155782537928472, -0.2642804927636907, -0.1395512355959886
LBFGS Accumulation History: 3 points
Removed measurement 100a4719 to history. Total: 3
th(0)=-2.0831051002099024;dx=-3.5122867465400126E-4
New Minimum: -2.0831051002099024 > -3.0491504486589585
END: th(31023.85953645913)=-3.0491504486589585; dx=-1.277378396053145E-5 evalInputDelta=0.9660453484490561
Fitness changed from -2.0831051002099024 to -3.0491504486589585
Iteration 4 complete. Error: -3.0491504486589585 Total: 4.0919; Orientation: 0.4579; Line Search: 2.3636
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 36a9bc07 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.850e+03, gradient 3.204e-02, dot -0.114; [e3668570-8d2a-4a87-879d-0b63a857dc54 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.0491504486589585, -0.9155782537928472, -0.2642804927636907, -0.1395512355959886
LBFGS Accumulation History: 3 points
Removed measurement 36a9bc07 to history. Total: 3
th(0)=-3.0491504486589585;dx=-9.597891322497907E-4
New Minimum: -3.0491504486589585 > -3.3347112827491427
WOLF (strong): th(66838.87920402404)=-3.3347112827491427; dx=1.9131054584612384E-5 evalInputDelta=0.2855608340901843
New Minimum: -3.3347112827491427 > -5.014143965770494
WOLF (strong): th(33419.43960201202)=-5.014143965770494; dx=2.063560598967991E-6 evalInputDelta=1.9649935171115351
New Minimum: -5.014143965770494 > -5.863747066772742
END: th(11139.813200670673)=-5.863747066772742; dx=-5.1296078485816104E-5 evalInputDelta=2.8145966181137836
Fitness changed from -3.0491504486589585 to -5.863747066772742
Iteration 5 complete. Error: -5.863747066772742 Total: 6.6378; Orientation: 0.6253; Line Search: 4.7088
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 2a62e0cd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.173e+04, gradient 3.077e-02, dot -0.094; [e3668570-8d2a-4a87-879d-0b63a857dc54 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.863747066772742, -0.9155782537928472, -0.2642804927636907, -0.1395512355959886
LBFGS Accumulation History: 3 points
Removed measurement 2a62e0cd to history. Total: 3
th(0)=-5.863747066772742;dx=-9.428254738011921E-4
New Minimum: -5.863747066772742 > -6.575108504759844
WOLF (strong): th(24000.000000000007)=-6.575108504759844; dx=6.3197964300884E-5 evalInputDelta=0.7113614379871018
New Minimum: -6.575108504759844 > -8.82262515352485
WOLF (strong): th(12000.000000000004)=-8.82262515352485; dx=9.249102803968218E-6 evalInputDelta=2.958878086752108
END: th(4000.0000000000014)=-8.373845718321888; dx=-3.7338958109772697E-4 evalInputDelta=2.5100986515491464
Fitness changed from -5.863747066772742 to -8.82262515352485
Iteration 6 complete. Error: -8.82262515352485 Total: 6.4415; Orientation: 0.4547; Line Search: 4.7164
Adding measurement 5df88927 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.320e+05, gradient 3.387e-02, dot -0.082; [e3668570-8d2a-4a87-879d-0b63a857dc54 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.82262515352485, -0.9155782537928472, -0.2642804927636907, -0.1395512355959886
LBFGS Accumulation History: 3 points
Removed measurement 5df88927 to history. Total: 3
th(0)=-8.82262515352485;dx=-0.001144889135718069
New Minimum: -8.82262515352485 > -12.387112325227882
END: th(8617.738760127539)=-12.387112325227882; dx=-8.780456287232959E-5 evalInputDelta=3.564487171703032
Fitness changed from -8.82262515352485 to -12.387112325227882
Iteration 7 complete. Error: -12.387112325227882 Total: 3.9113; Orientation: 0.4475; Line Search: 2.3379
Adding measurement e1be184 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.343e+04, gradient 2.922e-02, dot 0.108; [e3668570-8d2a-4a87-879d-0b63a857dc54 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-12.387112325227882;dx=-42.610948334212864
Armijo: th(18566.355334451124)=-4.277890188768652; dx=6.320191999107854E-5 evalInputDelta=-8.10922213645923
Armijo: th(9283.177667225562)=-4.260928968051024; dx=1.2355412231181514E-4 evalInputDelta=-8.126183357176858
Armijo: th(3094.392555741854)=-4.232635703888424; dx=4.027242281740282E-4 evalInputDelta=-8.154476621339459
Armijo: th(773.5981389354635)=-4.273579206331396; dx=0.0018165513899612556 evalInputDelta=-8.113533118896486
Armijo: th(154.7196277870927)=-4.467293882220751; dx=0.008082905770183176 evalInputDelta=-7.919818443007131
Armijo: th(25.786604631182115)=-5.414859509393547; dx=0.001083114510038733 evalInputDelta=-6.972252815834335
Armijo: th(3.683800661597445)=-3.7598421004803426; dx=-0.2231826497897058 evalInputDelta=-8.627270224747539
Armijo: th(0.4604750826996806)=-3.836832287878903; dx=-0.46408104270304495 evalInputDelta=-8.55028003734898
Armijo: th(0.05116389807774229)=-11.761254946642467; dx=8.0153342956781 evalInputDelta=-0.6258573785854153
New Minimum: -12.387112325227882 > -12.586933420986064
END:
...skipping 53045 bytes...
8.84934659618324
WOLFE (weak): th(1.5511929768229565E-7)=-228.84934659618324; dx=-0.001579075968653273 evalInputDelta=8.534544974736491E-6
Armijo: th(1.0858350837760695E-6)=-228.8493209925474; dx=-0.0015721900976681748 evalInputDelta=-1.7069090858967684E-5
Armijo: th(6.204771907291826E-7)=-228.8493209925474; dx=-0.0015664784908478984 evalInputDelta=-1.7069090858967684E-5
WOLFE (weak): th(3.877982442057391E-7)=-228.84934659618324; dx=-0.0015722221423777017 evalInputDelta=8.534544974736491E-6
WOLFE (weak): th(5.041377174674609E-7)=-228.84934659618324; dx=-0.0015747683264898905 evalInputDelta=8.534544974736491E-6
WOLFE (weak): th(5.623074540983217E-7)=-228.84934659618324; dx=-0.0015670501583046614 evalInputDelta=8.534544974736491E-6
Armijo: th(5.913923224137522E-7)=-228.8493209925474; dx=-0.001562958897834561 evalInputDelta=-1.7069090858967684E-5
WOLFE (weak): th(5.768498882560369E-7)=-228.84934659618324; dx=-0.0015611014704833609 evalInputDelta=8.534544974736491E-6
WOLFE (weak): th(5.841211053348945E-7)=-228.84934659618324; dx=-0.001569399279028913 evalInputDelta=8.534544974736491E-6
Armijo: th(5.877567138743234E-7)=-228.8493209925474; dx=-0.0015788251822386554 evalInputDelta=-1.7069090858967684E-5
Armijo: th(5.859389096046089E-7)=-228.8493209925474; dx=-0.001547412537466465 evalInputDelta=-1.7069090858967684E-5
WOLFE (weak): th(5.850300074697518E-7)=-228.84934659618324; dx=-0.0015785498640474172 evalInputDelta=8.534544974736491E-6
WOLFE (weak): th(5.854844585371804E-7)=-228.84934659618324; dx=-0.0015526700979940872 evalInputDelta=8.534544974736491E-6
Armijo: th(5.857116840708946E-7)=-228.8493209925474; dx=-0.00157752034636389 evalInputDelta=-1.7069090858967684E-5
Armijo: th(5.855980713040374E-7)=-228.8493209925474; dx=-0.001567748659589322 evalInputDelta=-1.7069090858967684E-5
WOLFE (weak): th(5.855412649206089E-7)=-228.84934659618324; dx=-0.0015767589446387086 evalInputDelta=8.534544974736491E-6
Armijo: th(5.855696681123232E-7)=-228.8493209925474; dx=-0.0015778735269397516 evalInputDelta=-1.7069090858967684E-5
Armijo: th(5.85555466516466E-7)=-228.8493209925474; dx=-0.0015798482084663696 evalInputDelta=-1.7069090858967684E-5
WOLFE (weak): th(5.855483657185375E-7)=-228.84934659618324; dx=-0.0015694065075880434 evalInputDelta=8.534544974736491E-6
Armijo: th(5.855519161175017E-7)=-228.8493209925474; dx=-0.0015663897463646789 evalInputDelta=-1.7069090858967684E-5
mu ~= nu (5.855483657185375E-7): th(1.5511929768229565E-7)=-228.84934659618324
Fitness changed from -228.84933806163826 to -228.84934659618324
Iteration 41 complete. Error: -228.84934659618324 Total: 34.8628; Orientation: 0.0164; Line Search: 32.5950
Adding measurement 44141aca to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-228.84934659618324;dx=-0.0018137545133642837
Armijo: th(1.2615295363468394E-6)=-228.8493209925474; dx=-0.0015695612309518014 evalInputDelta=-2.5603635833704175E-5
Armijo: th(6.307647681734197E-7)=-228.8493209925474; dx=-0.0015545590779641855 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(2.1025492272447322E-7)=-228.84934659618324; dx=-0.0015562345639248706 evalInputDelta=0.0
Armijo: th(4.2050984544894643E-7)=-228.8493209925474; dx=-0.0015418458170929844 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(3.1538238408670984E-7)=-228.84934659618324; dx=-0.0015325643836866562 evalInputDelta=0.0
WOLFE (weak): th(3.6794611476782816E-7)=-228.84934659618324; dx=-0.001562780695040619 evalInputDelta=0.0
WOLFE (weak): th(3.942279801083873E-7)=-228.84934659618324; dx=-0.001551576260317912 evalInputDelta=0.0
WOLFE (weak): th(4.073689127786669E-7)=-228.84934659618324; dx=-0.0015387747261417796 evalInputDelta=0.0
WOLFE (weak): th(4.139393791138067E-7)=-228.84934659618324; dx=-0.0015571489480746457 evalInputDelta=0.0
Armijo: th(4.1722461228137654E-7)=-228.8493209925474; dx=-0.0015629361140607587 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.155819956975916E-7)=-228.8493209925474; dx=-0.0015613881913502663 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.147606874056992E-7)=-228.8493209925474; dx=-0.0015681344963464593 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(4.1435003325975294E-7)=-228.84934659618324; dx=-0.0015670038960302359 evalInputDelta=0.0
Armijo: th(4.1455536033272606E-7)=-228.8493209925474; dx=-0.0015194317976188639 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.144526967962395E-7)=-228.8493209925474; dx=-0.0015692781862139877 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.144013650279962E-7)=-228.8493209925474; dx=-0.001526465869185015 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.143756991438746E-7)=-228.8493209925474; dx=-0.0015568339472944348 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.1436286620181373E-7)=-228.8493209925474; dx=-0.001552593880979324 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.1435644973078334E-7)=-228.8493209925474; dx=-0.001564409058482761 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.143532414952681E-7)=-228.8493209925474; dx=-0.0015679249223346835 evalInputDelta=-2.5603635833704175E-5
mu ~= nu (4.1435003325975294E-7): th(0.0)=-228.84934659618324
Fitness changed from -228.84934659618324 to -228.84934659618324
Static Iteration Total: 27.7775; Orientation: 0.0170; Line Search: 26.6482
Iteration 42 failed. Error: -228.84934659618324
Previous Error: 0.0 -> -228.84934659618324
Retrying iteration 42
Reset training subject: 4095277850331
Adding measurement 4f357a02 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-228.84934659618324;dx=-0.0017869751809758568
Armijo: th(8.926935414376204E-7)=-228.8493209925474; dx=-0.0015478549356288797 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.463467707188102E-7)=-228.8493209925474; dx=-0.0015452116830857413 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(1.4878225690627008E-7)=-228.84934659618324; dx=-0.001534311475035789 evalInputDelta=0.0
WOLFE (weak): th(2.9756451381254016E-7)=-228.84934659618324; dx=-0.0015095254876475548 evalInputDelta=0.0
WOLFE (weak): th(3.719556422656752E-7)=-228.84934659618324; dx=-0.0015384539743697944 evalInputDelta=0.0
WOLFE (weak): th(4.091512064922427E-7)=-228.84934659618324; dx=-0.001542359064579554 evalInputDelta=0.0
WOLFE (weak): th(4.277489886055265E-7)=-228.84934659618324; dx=-0.0015219838355499393 evalInputDelta=0.0
Armijo: th(4.3704787966216837E-7)=-228.8493209925474; dx=-0.0015383268398000442 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.323984341338474E-7)=-228.8493209925474; dx=-0.001547568949977428 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(4.3007371136968695E-7)=-228.84934659618324; dx=-0.001503512205764678 evalInputDelta=0.0
Armijo: th(4.312360727517672E-7)=-228.8493209925474; dx=-0.0015302085017479927 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.306548920607271E-7)=-228.8493209925474; dx=-0.0015389124688917656 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(4.30364301715207E-7)=-228.84934659618324; dx=-0.0015443357491606215 evalInputDelta=0.0
Armijo: th(4.3050959688796706E-7)=-228.8493209925474; dx=-0.0015497970712171016 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.3043694930158707E-7)=-228.8493209925474; dx=-0.0015441438573896326 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(4.3040062550839704E-7)=-228.84934659618324; dx=-0.001485985799298985 evalInputDelta=0.0
Armijo: th(4.304187874049921E-7)=-228.8493209925474; dx=-0.0015182735955768113 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.3040970645669454E-7)=-228.8493209925474; dx=-0.0015415333588506739 evalInputDelta=-2.5603635833704175E-5
Armijo: th(4.3040516598254576E-7)=-228.8493209925474; dx=-0.0015429052785609443 evalInputDelta=-2.5603635833704175E-5
WOLFE (weak): th(4.3040289574547143E-7)=-228.84934659618324; dx=-0.0015464808744906889 evalInputDelta=0.0
mu ~= nu (4.3040289574547143E-7): th(0.0)=-228.84934659618324
Fitness changed from -228.84934659618324 to -228.84934659618324
Static Iteration Total: 28.4799; Orientation: 0.0169; Line Search: 26.1877
Iteration 43 failed. Error: -228.84934659618324
Previous Error: 0.0 -> -228.84934659618324
Optimization terminated 43
Final threshold in iteration 43: -228.84934659618324 (> -Infinity) after 518.323s (< 3600.000s)
Returns
-228.84934659618324