BasicOptimizer.scala:89 executed in 122.25 seconds (1.510 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 4778987875788
Reset training subject: 4780123551177
Adding measurement 2cf0571c to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.15616769411025722;dx=-3.34596108892634E-6
New Minimum: -0.15616769411025722 > -0.1561747310066227
WOLFE (weak): th(2.154434690031884)=-0.1561747310066227; dx=-3.2504134343001157E-6 evalInputDelta=7.036896365497647E-6
New Minimum: -0.1561747310066227 > -0.15618166621373814
WOLFE (weak): th(4.308869380063768)=-0.15618166621373814; dx=-3.088753938708169E-6 evalInputDelta=1.3972103480919262E-5
New Minimum: -0.15618166621373814 > -0.15620880179866198
WOLFE (weak): th(12.926608140191302)=-0.15620880179866198; dx=-3.301048055616555E-6 evalInputDelta=4.110768840476231E-5
New Minimum: -0.15620880179866198 > -0.15633193490391503
WOLFE (weak): th(51.70643256076521)=-0.15633193490391503; dx=-3.0926461547119682E-6 evalInputDelta=1.6424079365781608E-4
New Minimum: -0.15633193490391503 > -0.15699809516750432
WOLFE (weak): th(258.53216280382605)=-0.15699809516750432; dx=-3.3724200741459394E-6 evalInputDelta=8.304010572471043E-4
New Minimum: -0.15699809516750432 > -0.1613188103579996
WOLFE (weak): th(1551.1929768229563)=-0.1613188103579996; dx=-3.3345408295879194E-6 evalInputDelta=0.005151116247742393
New Minimum: -0.1613188103579996 > -0.19440328686578706
WOLFE (weak): th(10858.350837760694)=-0.19440328686578706; dx=-3.734519083116757E-6 evalInputDelta=0.038235592755529846
New Minimum: -0.19440328686578706 > -0.43397827968214076
END: th(86866.80670208555)=-0.43397827968214076; dx=-1.6195266280508558E-6 evalInputDelta=0.27781058557188354
Fitness changed from -0.15616769411025722 to -0.43397827968214076
Iteration 1 complete. Error: -0.43397827968214076 Total: 14.5950; Orientation: 0.1961; Line Search: 11.0006
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4e4de91a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.43397827968214076;dx=-1.0492480709509061E-5
New Minimum: -0.43397827968214076 > -0.8068722445487466
END: th(187148.86177126726)=-0.8068722445487466; dx=-1.5581619003809806E-6 evalInputDelta=0.37289396486660586
Fitness changed from -0.43397827968214076 to -0.8068722445487466
Iteration 2 complete. Error: -0.8068722445487466 Total: 3.6117; Orientation: 0.0166; Line Search: 2.3388
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5d1da7f to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.8068722445487466;dx=-4.94098911154241E-5
Armijo: th(403200.00000000006)=0.0; dx=0.0 evalInputDelta=-0.8068722445487466
Armijo: th(201600.00000000003)=-0.2102383931660903; dx=-2.2024070514606901E-7 evalInputDelta=-0.5966338513826563
New Minimum: -0.8068722445487466 > -1.599453984911871
END: th(67200.00000000001)=-1.599453984911871; dx=-9.421950427377782E-6 evalInputDelta=0.7925817403631243
Fitness changed from -0.8068722445487466 to -1.599453984911871
Iteration 3 complete. Error: -1.599453984911871 Total: 6.0243; Orientation: 0.0173; Line Search: 4.7543
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2a8b4817 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.901e+04, gradient 1.711e-02, dot -0.102; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.599453984911871, -0.8068722445487466, -0.43397827968214076, -0.15616769411025722
LBFGS Accumulation History: 3 points
Removed measurement 2a8b4817 to history. Total: 3
th(0)=-1.599453984911871;dx=-2.8406720237525603E-4
Armijo: th(144778.01117014262)=-0.27968875362605444; dx=5.55552708284102E-7 evalInputDelta=-1.3197652312858166
New Minimum: -1.599453984911871 > -3.387557369182756
WOLF (strong): th(72389.00558507131)=-3.387557369182756; dx=2.3085077952807458E-5 evalInputDelta=1.788103384270885
New Minimum: -3.387557369182756 > -4.499458704287861
END: th(24129.668528357102)=-4.499458704287861; dx=-1.0162626967575734E-4 evalInputDelta=2.90000471937599
Fitness changed from -1.599453984911871 to -4.499458704287861
Iteration 4 complete. Error: -4.499458704287861 Total: 6.4958; Orientation: 0.4588; Line Search: 4.7640
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5d397a20 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.061e+05, gradient 2.621e-02, dot -0.057; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.499458704287861, -0.8068722445487466, -0.43397827968214076, -0.15616769411025722
LBFGS Accumulation History: 3 points
Removed measurement 5d397a20 to history. Total: 3
th(0)=-4.499458704287861;dx=-6.741610900492868E-4
Armijo: th(51985.794936463135)=-1.7085757742565422; dx=3.9266374269324306E-5 evalInputDelta=-2.7908829300313185
New Minimum: -4.499458704287861 > -5.153150887132236
WOLF (strong): th(25992.897468231567)=-5.153150887132236; dx=8.342794051875892E-5 evalInputDelta=0.6536921828443756
New Minimum: -5.153150887132236 > -6.996074393206751
END: th(8664.29915607719)=-6.996074393206751; dx=-2.9750255015133947E-6 evalInputDelta=2.49661568891889
Fitness changed from -4.499458704287861 to -6.996074393206751
Iteration 5 complete. Error: -6.996074393206751 Total: 6.4725; Orientation: 0.4483; Line Search: 4.7729
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 127c5294 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.710e+05, gradient 3.823e-02, dot -0.036; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.996074393206751, -0.8068722445487466, -0.43397827968214076, -0.15616769411025722
LBFGS Accumulation History: 3 points
Removed measurement 127c5294 to history. Total: 3
th(0)=-6.996074393206751;dx=-0.0014148249946267155
New Minimum: -6.996074393206751 > -8.855851268205125
WOLF (strong): th(18666.66666666667)=-8.855851268205125; dx=6.90964672710429E-5 evalInputDelta=1.8597768749983743
New Minimum: -8.855851268205125 > -9.192088946835913
WOLF (strong): th(9333.333333333336)=-9.192088946835913; dx=1.407647402970334E-4 evalInputDelta=2.196014553629162
New Minimum: -9.192088946835913 > -9.294741909964555
END: th(3111.1111111111118)=-9.294741909964555; dx=-4.2699699636898056E-4 evalInputDelta=2.2986675167578046
Fitness changed from -6.996074393206751 to -9.294741909964555
Iteration 6 complete. Error: -9.294741909964555 Total: 6.5162; Orientation: 0.4472; Line Search: 4.8150
Adding measurement 19fe3899 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.777e+05, gradient 3.067e-02, dot -0.031; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.294741909964555, -0.8068722445487466, -0.43397827968214076, -0.15616769411025722
LBFGS Accumulation History: 3 points
Removed measurement 19fe3899 to history. Total: 3
th(0)=-9.294741909964555;dx=-9.108397585297114E-4
New Minimum: -9.294741909964555 > -11.966272640559513
END: th(6702.685702321418)=-11.966272640559513; dx=-2.7632444681738116E-4 evalInputDelta=2.6715307305949576
Fitness changed from -9.294741909964555 to -11.966272640559513
Iteration 7 complete. Error: -11.966272640559513 Total: 4.4805; Orientation: 0.4637; Line Search: 2.3768
Adding measurement 451a4c88 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.868e+04, gradient 2.971e-02, dot -0.020; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.966272640559513, -0.8068722445487466, -0.43397827968214076, -0.15616769411025722
LBFGS Accumulation History: 3 points
Removed measurement 451a4c88 to history. Total: 3
th(0)=-11.966272640559513;dx=-8.384702726028147E-4
New Minimum: -11.966272640559513 > -13.242660505010939
WOLF (strong): th(14440.498593461984)=-13.242660505010939; dx=2.129815586712461E-5 evalInputDelta=1.2763878644514257
New Minimum: -13.242660505010939 > -13.802672329217062
END: th(7220.249296730992)=-13.802672329217062; dx=-9.692575683501408E-5 evalInputDelta=1.8363996886575489
Fitness changed from -11.966272640559513 to -13.802672329217062
Iteration 8 complete. Error: -13.802672329217062 Total: 5.1820; Orientation: 0.4552; Line Search: 3.5744
Adding measurement 48246be0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.154e+04, gradient 3.101e-02, dot -0.056; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.802672329217062, -0.8068722445487466, -0.43397827968214076, -0.15616769411025722
LBFGS Accumulation History: 3 points
Removed measurement 48246be0 to history. Total: 3
th(0)=-13.802672329217062;dx=-9.402942925937089E-4
Armijo: th(15555.555555555562)=-13.396316796453325; dx=1.1854891428248259E-4 evalInputDelta=-0.4063555327637367
New Minimum: -13.802672329217062 > -15.489092526829566
WOLF (strong): th(7777.777777777781)=-15.489092526829566; dx=2.9761780658166767E-5 evalInputDelta=1.6864201976125042
END: th(2592.5925925925935)=-15.249755357124535; dx=-4.807133235188083E-4 evalInputDelta=1.447083027907473
Fitness changed from -13.802672329217062 to -15.489092526829566
Iteration 9 complete. Error: -15.489092526829566 Total: 6.3402; Orientation: 0.4492; Line Search: 4.7749
Adding measurement 39017c0f to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.550e+02, gradient 3.706e-02, dot 0.204; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-15.489092526829566;dx=-2.585312470807593
Armijo: th(5585.571418601182)=-1.38837954506474; dx=-5.8379472596924165E-5 evalInputDelta=-14.100712981764826
Armijo: th(2792.785709300591)=-1.8554386819109767; dx=-1.056817745947164E-4 evalInputDelta=-13.63365384491859
Armijo: th(930.9285697668637)=-2.895845884300495; dx=-6.487200210838213E-4 evalInputDelta=-12.59324664252907
Armijo: th(232.73214244171592)=-0.4138704610337776; dx=-0.0022190836293039798 evalInputDelta=-15.075222065795788
Armijo: th(46.54642848834318)=0.0; dx=0.0 evalInputDelta=-15.489092526829566
Armijo: th(7.757738081390531)=-0.1896638092034296; dx=0.002095565066122117 evalInputDelta=-15.299428717626137
Armijo: th(1.1082482973415044)=-12.373612143017104; dx=5.395671386877616 evalInputDelta=-3.1154803838124625
New Minimum: -15.489092526829566 > -15.646016627362728
END: th(0.13853103716768805)=-15.646016627362728; dx=-1.4616517056068834 evalInputDelta=0.1569241005331623
Fitness changed from -15.489092526829566 to -15.646016627362728
Iteration 10 complete. Error: -15.646016627362728 Total: 12.3132; Orientation: 0.4389; Line Search: 10.7313
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 10727f7f to history. Total: 4
Accepted: LBFGS Orientation magnitude: 9.866e+03, gradient 3.305e-02, dot 0.001; [adbdf6f3-775a-403d-a6fb-59efa37a908f = 1.000/1.000e+00]
th(0)=-15.646016627362728;dx=-0.39931667509614854
Armijo: th(0.2984560721201634)=0.0; dx=0.0 evalInputDelta=-15.646016627362728
Armijo: th(0.1492280360600817)=-0.4583312561970663; dx=10.022600431186191 evalInputDelta=-15.187685371165662
Armijo: th(0.0497426786866939)=-11.681111430759168; dx=77.52136238191363 evalInputDelta=-3.96490519660356
Armijo: th(0.012435669671673475)=-15.215429926930364; dx=20.638550764365718 evalInputDelta=-0.43058670043236447
Armijo: th(0.002487133934334695)=-15.593486776022365; dx=1.9862929819316584 evalInputDelta=-0.052529851340363365
Armijo: th(4.1452232238911585E-4)=-15.63805753230845; dx=1.1071644948436954 evalInputDelta=-0.00795909505427872
Armijo: th(5.921747462701655E-5)=-15.644951129665102; dx=2.2239172920421306 evalInputDelta=-0.0010654976976258723
Armijo: th(7.4021843283770685E-6)=-15.645883992701913; dx=-0.8611582850657881 evalInputDelta=-1.326346608152562E-4
Armijo: th(8.224649253752299E-7)=-15.646000535717487; dx=1.6503009972407585 evalInputDelta=-1.6091645241189667E-5
Armijo: th(8.224649253752298E-8)=-15.646016139737357; dx=1.0282479033825531 evalInputDelta=-4.876253711927347E-7
Armijo: th(7.476953867047545E-9)=-15.646016627362728; dx=0.7319744223817578 evalInputDelta=0.0
END: th(6.230794889206288E-10)=-15.646016627362728; dx=-0.2144248566834157 evalInputDelta=0.0
Fitness changed from -15.646016627362728 to -15.646016627362728
Static Iteration Total: 17.6188; Orientation: 0.5505; Line Search: 15.7913
Iteration 11 failed. Error: -15.646016627362728
Previous Error: 0.0 -> -15.646016627362728
Retrying iteration 11
Reset training subject: 4868638522860
Adding measurement 72fef79e to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-15.646016627362728;dx=-0.0010827235102605247
WOLFE (weak): th(1.3423840655779394E-9)=-15.646016627362728; dx=-0.001073121414455352 evalInputDelta=0.0
WOLFE (weak): th(2.6847681311558788E-9)=-15.646016627362728; dx=-0.0010780094555144222 evalInputDelta=0.0
WOLFE (weak): th(8.054304393467637E-9)=-15.646016627362728; dx=-0.0010427494461923308 evalInputDelta=0.0
WOLFE (weak): th(3.221721757387055E-8)=-15.646016627362728; dx=-0.001067595647471707 evalInputDelta=0.0
WOLFE (weak): th(1.6108608786935276E-7)=-15.646016627362728; dx=-0.001073299750663721 evalInputDelta=0.0
Armijo: th(9.665165272161165E-7)=-15.646016627362728; dx=-0.001080862294892943 evalInputDelta=0.0
WOLFE (weak): th(5.638013075427346E-7)=-15.646016627362728; dx=-0.0010760322341172262 evalInputDelta=0.0
WOLFE (weak): th(7.651589173794256E-7)=-15.646016627362728; dx=-0.0010000646377609102 evalInputDelta=0.0
Armijo: th(8.658377222977711E-7)=-15.646016627362728; dx=-0.0010287972576719295 evalInputDelta=0.0
WOLFE (weak): th(8.154983198385984E-7)=-15.646016627362728; dx=-0.0010722783990010634 evalInputDelta=0.0
Armijo: th(8.406680210681848E-7)=-15.646016627362728; dx=-0.0010682286562931872 evalInputDelta=0.0
Armijo: th(8.280831704533916E-7)=-15.646016627362728; dx=-0.0010600837578091937 evalInputDelta=0.0
Armijo: th(8.21790745145995E-7)=-15.646016627362728; dx=-0.0010734044020253631 evalInputDelta=0.0
WOLFE (weak): th(8.186445324922967E-7)=-15.646016627362728; dx=-0.0010848478938916906 evalInputDelta=0.0
WOLFE (weak): th(8.202176388191459E-7)=-15.646016627362728; dx=-0.0010764347600325116 evalInputDelta=0.0
Armijo: th(8.210041919825704E-7)=-15.646016627362728; dx=-0.001081753539856768 evalInputDelta=0.0
Armijo: th(8.206109154008582E-7)=-15.646016627362728; dx=-0.0010678437830851845 evalInputDelta=0.0
Armijo: th(8.204142771100021E-7)=-15.646016627362728; dx=-0.001069155743358376 evalInputDelta=0.0
WOLFE (weak): th(8.20315957964574E-7)=-15.646016627362728; dx=-0.001071955248643007 evalInputDelta=0.0
Armijo: th(8.20365117537288E-7)=-15.646016627362728; dx=-0.0010486880943223033 evalInputDelta=0.0
Armijo: th(8.20340537750931E-7)=-15.646016627362728; dx=-0.001064091783648665 evalInputDelta=0.0
Armijo: th(8.203282478577524E-7)=-15.646016627362728; dx=-0.001063979861464222 evalInputDelta=0.0
Armijo: th(8.203221029111632E-7)=-15.646016627362728; dx=-0.0010675154398872596 evalInputDelta=0.0
mu ~= nu (8.20315957964574E-7): th(0.0)=-15.646016627362728
Fitness changed from -15.646016627362728 to -15.646016627362728
Static Iteration Total: 32.5966; Orientation: 0.0384; Line Search: 30.3347
Iteration 12 failed. Error: -15.646016627362728
Previous Error: 0.0 -> -15.646016627362728
Optimization terminated 12
Final threshold in iteration 12: -15.646016627362728 (> -Infinity) after 122.247s (< 3600.000s)
Returns
-15.646016627362728