BasicOptimizer.scala:89 executed in 124.09 seconds (1.367 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2496117230186
Reset training subject: 2496971674460
Adding measurement 782873b3 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.4999513590438985;dx=-2.15639892344066E-5
New Minimum: -0.4999513590438985 > -0.49998927541292776
WOLFE (weak): th(2.154434690031884)=-0.49998927541292776; dx=-2.157617075441786E-5 evalInputDelta=3.791636902927831E-5
New Minimum: -0.49998927541292776 > -0.5000277689464389
WOLFE (weak): th(4.308869380063768)=-0.5000277689464389; dx=-2.1578257555450687E-5 evalInputDelta=7.640990254037527E-5
New Minimum: -0.5000277689464389 > -0.5001802638021864
WOLFE (weak): th(12.926608140191302)=-0.5001802638021864; dx=-2.1573520351645952E-5 evalInputDelta=2.2890475828790668E-4
New Minimum: -0.5001802638021864 > -0.5008679750016065
WOLFE (weak): th(51.70643256076521)=-0.5008679750016065; dx=-2.159875702964325E-5 evalInputDelta=9.166159577080335E-4
New Minimum: -0.5008679750016065 > -0.5045121648299594
WOLFE (weak): th(258.53216280382605)=-0.5045121648299594; dx=-2.1505688457245692E-5 evalInputDelta=0.00456080578606094
New Minimum: -0.5045121648299594 > -0.5271643958433074
WOLFE (weak): th(1551.1929768229563)=-0.5271643958433074; dx=-2.1178319774443083E-5 evalInputDelta=0.02721303679940895
New Minimum: -0.5271643958433074 > -0.6742609193413601
END: th(10858.350837760694)=-0.6742609193413601; dx=-1.6978102637344413E-5 evalInputDelta=0.17430956029746164
Fitness changed from -0.4999513590438985 to -0.6742609193413601
Iteration 1 complete. Error: -0.6742609193413601 Total: 9.7974; Orientation: 0.0218; Line Search: 7.2216
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 56226b79 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.6742609193413601;dx=-2.0548030284968754E-5
New Minimum: -0.6742609193413601 > -0.9776197183707028
END: th(23393.607721408407)=-0.9776197183707028; dx=-1.2681453022242465E-5 evalInputDelta=0.3033587990293427
Fitness changed from -0.6742609193413601 to -0.9776197183707028
Iteration 2 complete. Error: -0.9776197183707028 Total: 2.6813; Orientation: 0.0209; Line Search: 1.7984
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4a26de40 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.9776197183707028;dx=-2.5473663770515283E-5
New Minimum: -0.9776197183707028 > -1.3688880460677226
END: th(50400.00000000001)=-1.3688880460677226; dx=-4.374799384130988E-6 evalInputDelta=0.3912683276970198
Fitness changed from -0.9776197183707028 to -1.3688880460677226
Iteration 3 complete. Error: -1.3688880460677226 Total: 2.7061; Orientation: 0.0208; Line Search: 1.7920
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4e2aad50 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.778e+03, gradient 6.975e-03, dot -0.215; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.3688880460677226, -0.9776197183707028, -0.6742609193413601, -0.4999513590438985
LBFGS Accumulation History: 3 points
Removed measurement 4e2aad50 to history. Total: 3
th(0)=-1.3688880460677226;dx=-4.859689278533073E-5
New Minimum: -1.3688880460677226 > -1.880415169566315
END: th(108583.50837760697)=-1.880415169566315; dx=-2.348727986193053E-6 evalInputDelta=0.5115271234985923
Fitness changed from -1.3688880460677226 to -1.880415169566315
Iteration 4 complete. Error: -1.880415169566315 Total: 3.1718; Orientation: 0.4893; Line Search: 1.8092
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 17d6ca98 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.817e+03, gradient 9.401e-03, dot -0.151; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.880415169566315, -0.9776197183707028, -0.6742609193413601, -0.4999513590438985
LBFGS Accumulation History: 3 points
Removed measurement 17d6ca98 to history. Total: 3
th(0)=-1.880415169566315;dx=-8.831623862816792E-5
Armijo: th(233936.07721408413)=-1.84982300793093; dx=7.477453578344876E-7 evalInputDelta=-0.03059216163538503
New Minimum: -1.880415169566315 > -2.5864460656515464
WOLF (strong): th(116968.03860704206)=-2.5864460656515464; dx=9.671740524297501E-7 evalInputDelta=0.7060308960852315
New Minimum: -2.5864460656515464 > -2.936228584292796
END: th(38989.346202347355)=-2.936228584292796; dx=-1.3225430430878566E-5 evalInputDelta=1.0558134147264813
Fitness changed from -1.880415169566315 to -2.936228584292796
Iteration 5 complete. Error: -2.936228584292796 Total: 4.9755; Orientation: 0.4911; Line Search: 3.6145
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 698da472 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.872e+03, gradient 9.054e-03, dot -0.124; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.936228584292796, -0.9776197183707028, -0.6742609193413601, -0.4999513590438985
LBFGS Accumulation History: 3 points
Removed measurement 698da472 to history. Total: 3
th(0)=-2.936228584292796;dx=-8.19073185710194E-5
New Minimum: -2.936228584292796 > -3.7998146387132286
END: th(84000.00000000003)=-3.7998146387132286; dx=-1.4277487322153224E-6 evalInputDelta=0.8635860544204323
Fitness changed from -2.936228584292796 to -3.7998146387132286
Iteration 6 complete. Error: -3.7998146387132286 Total: 3.1611; Orientation: 0.4898; Line Search: 1.7934
Adding measurement 5012af92 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.242e+03, gradient 1.044e-02, dot -0.120; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.7998146387132286, -0.9776197183707028, -0.6742609193413601, -0.4999513590438985
LBFGS Accumulation History: 3 points
Removed measurement 5012af92 to history. Total: 3
th(0)=-3.7998146387132286;dx=-1.0895804562344909E-4
Armijo: th(180972.5139626783)=-3.715598460239834; dx=2.001977376470153E-6 evalInputDelta=-0.08421617847339435
New Minimum: -3.7998146387132286 > -4.7409184236034765
WOLF (strong): th(90486.25698133915)=-4.7409184236034765; dx=4.3500924313737946E-7 evalInputDelta=0.9411037848902479
New Minimum: -4.7409184236034765 > -5.047866705108117
END: th(30162.08566044638)=-5.047866705108117; dx=-2.2430535646870927E-5 evalInputDelta=1.2480520663948882
Fitness changed from -3.7998146387132286 to -5.047866705108117
Iteration 7 complete. Error: -5.047866705108117 Total: 4.9208; Orientation: 0.4881; Line Search: 3.5879
Adding measurement 60ae3f8e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.958e+04, gradient 8.792e-03, dot -0.138; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.047866705108117, -0.9776197183707028, -0.6742609193413601, -0.4999513590438985
LBFGS Accumulation History: 3 points
Removed measurement 60ae3f8e to history. Total: 3
th(0)=-5.047866705108117;dx=-7.701010240293579E-5
New Minimum: -5.047866705108117 > -6.029053433406357
END: th(64982.24367057892)=-6.029053433406357; dx=-2.751937189396514E-6 evalInputDelta=0.9811867282982405
Fitness changed from -5.047866705108117 to -6.029053433406357
Iteration 8 complete. Error: -6.029053433406357 Total: 3.1433; Orientation: 0.4899; Line Search: 1.8033
Adding measurement 4683288 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.453e+05, gradient 1.125e-02, dot -0.112; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.029053433406357, -0.9776197183707028, -0.6742609193413601, -0.4999513590438985
LBFGS Accumulation History: 3 points
Removed measurement 4683288 to history. Total: 3
th(0)=-6.029053433406357;dx=-1.2643998690678373E-4
Armijo: th(140000.00000000006)=-5.654674159121294; dx=3.3193908757642994E-6 evalInputDelta=-0.37437927428506335
New Minimum: -6.02905
...skipping 5474 bytes...
evalInputDelta=2.385362816060624
Fitness changed from -10.997538236350268 to -13.382901052410892
Iteration 13 complete. Error: -13.382901052410892 Total: 3.1443; Orientation: 0.4872; Line Search: 1.8202
Adding measurement 6a69d358 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.056e+05, gradient 8.331e-03, dot -0.187; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.382901052410892, -0.9776197183707028, -0.6742609193413601, -0.4999513590438985
LBFGS Accumulation History: 3 points
Removed measurement 6a69d358 to history. Total: 3
th(0)=-13.382901052410892;dx=-6.932038664847448E-5
New Minimum: -13.382901052410892 > -16.362386047837138
END: th(135379.67431370614)=-16.362386047837138; dx=-1.9680100238539272E-5 evalInputDelta=2.9794849954262457
Fitness changed from -13.382901052410892 to -16.362386047837138
Iteration 14 complete. Error: -16.362386047837138 Total: 3.1673; Orientation: 0.4828; Line Search: 1.8312
Adding measurement 41814ac to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.427e+04, gradient 1.070e-02, dot 0.193; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
th(0)=-16.362386047837138;dx=-29.422127844387006
Armijo: th(291666.66666666686)=-7.332538257206708; dx=1.1621374873721887E-5 evalInputDelta=-9.02984779063043
Armijo: th(145833.33333333343)=-7.331897479550224; dx=2.325026844424695E-5 evalInputDelta=-9.030488568286913
Armijo: th(48611.111111111146)=-7.333336759944713; dx=7.005167924232169E-5 evalInputDelta=-9.029049287892425
Armijo: th(12152.777777777786)=-7.334882609898383; dx=2.773853575649964E-4 evalInputDelta=-9.027503437938755
Armijo: th(2430.5555555555575)=-7.33124792476214; dx=0.0013934111682510097 evalInputDelta=-9.031138123074998
Armijo: th(405.0925925925929)=-7.3361447514804246; dx=0.008075023678665458 evalInputDelta=-9.026241296356712
Armijo: th(57.870370370370416)=-7.582043372412597; dx=0.05236095077414255 evalInputDelta=-8.78034267542454
Armijo: th(7.233796296296302)=-10.492585875165206; dx=0.7877065986485484 evalInputDelta=-5.8698001726719315
New Minimum: -16.362386047837138 > -20.259607397600604
END: th(0.8037551440329225)=-20.259607397600604; dx=-3.709641436277169 evalInputDelta=3.8972213497634662
Fitness changed from -16.362386047837138 to -20.259607397600604
Iteration 15 complete. Error: -20.259607397600604 Total: 10.4821; Orientation: 0.5029; Line Search: 9.1261
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 438a50e3 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 3.775e+03, gradient 1.058e-02, dot 0.079; [b8680cb7-a9f2-4d30-9b1e-0a3f0f34ecf3 = 1.000/1.000e+00]
th(0)=-20.259607397600604;dx=-3.138520889174916
Armijo: th(1.7316379645961015)=-18.274736380138116; dx=0.4283730111248446 evalInputDelta=-1.9848710174624884
Armijo: th(0.8658189822980508)=-19.22610731826438; dx=0.39242298316660607 evalInputDelta=-1.0335000793362248
Armijo: th(0.2886063274326836)=-19.91152664858481; dx=0.3306313724590704 evalInputDelta=-0.3480807490157929
Armijo: th(0.0721515818581709)=-20.174139519258834; dx=0.30423920515317837 evalInputDelta=-0.08546787834177039
Armijo: th(0.01443031637163418)=-20.24263349345836; dx=0.29841136688870756 evalInputDelta=-0.016973904142243867
Armijo: th(0.0024050527286056966)=-20.256771391715194; dx=0.30450967780497096 evalInputDelta=-0.002836005885409776
Armijo: th(3.4357896122938524E-4)=-20.25920099838178; dx=0.300720264840107 evalInputDelta=-4.063992188250154E-4
Armijo: th(4.2947370153673155E-5)=-20.25955771797315; dx=0.3020132805083431 evalInputDelta=-4.967962745539012E-5
Armijo: th(4.771930017074795E-6)=-20.25960169666594; dx=0.30183379551339595 evalInputDelta=-5.700934664787383E-6
Armijo: th(4.771930017074795E-7)=-20.259606583181466; dx=0.30102581255115207 evalInputDelta=-8.144191383507859E-7
Armijo: th(4.3381181973407224E-8)=-20.259607397600604; dx=0.3019397100726947 evalInputDelta=0.0
Armijo: th(3.615098497783935E-9)=-20.259607397600604; dx=0.2996524554336579 evalInputDelta=0.0
WOLF (strong): th(2.7808449982953345E-10)=-20.259607397600604; dx=0.30143754177708176 evalInputDelta=0.0
MIN ALPHA (1.986317855925239E-11): th(0.0)=-20.259607397600604
Fitness changed from -20.259607397600604 to -20.259607397600604
Static Iteration Total: 15.1861; Orientation: 0.6389; Line Search: 13.6726
Iteration 16 failed. Error: -20.259607397600604
Previous Error: 0.0 -> -20.259607397600604
Retrying iteration 16
Reset training subject: 2590280086243
Adding measurement 7be292b9 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-20.259607397600604;dx=-1.119226932674824E-4
WOLFE (weak): th(2.1544346900318838E-10)=-20.259607397600604; dx=-1.0468213233931076E-4 evalInputDelta=0.0
WOLFE (weak): th(4.3088693800637676E-10)=-20.259607397600604; dx=-1.0469795384560434E-4 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191302E-9)=-20.259607397600604; dx=-1.0471064656439781E-4 evalInputDelta=0.0
WOLFE (weak): th(5.170643256076521E-9)=-20.259607397600604; dx=-1.0468209257215735E-4 evalInputDelta=0.0
WOLFE (weak): th(2.5853216280382606E-8)=-20.259607397600604; dx=-1.0467913086335374E-4 evalInputDelta=0.0
WOLFE (weak): th(1.5511929768229565E-7)=-20.259607397600604; dx=-1.046920092208734E-4 evalInputDelta=0.0
WOLFE (weak): th(1.0858350837760695E-6)=-20.259607397600604; dx=-1.0471078714588254E-4 evalInputDelta=0.0
WOLFE (weak): th(8.686680670208556E-6)=-20.259607397600604; dx=-1.0471296811973612E-4 evalInputDelta=0.0
Armijo: th(7.8180126031877E-5)=-20.259606583181466; dx=-1.0468809641117025E-4 evalInputDelta=-8.144191383507859E-7
Armijo: th(4.343340335104278E-5)=-20.259606583181466; dx=-1.0459138332394889E-4 evalInputDelta=-8.144191383507859E-7
Armijo: th(2.606004201062567E-5)=-20.259606583181466; dx=-1.0467563380927108E-4 evalInputDelta=-8.144191383507859E-7
Armijo: th(1.7373361340417112E-5)=-20.259606583181466; dx=-1.0470323081814122E-4 evalInputDelta=-8.144191383507859E-7
WOLFE (weak): th(1.3030021005312834E-5)=-20.259607397600604; dx=-1.0471194554381507E-4 evalInputDelta=0.0
Armijo: th(1.5201691172864973E-5)=-20.259606583181466; dx=-1.0469695740627084E-4 evalInputDelta=-8.144191383507859E-7
Armijo: th(1.4115856089088904E-5)=-20.259606583181466; dx=-1.0469938211570747E-4 evalInputDelta=-8.144191383507859E-7
WOLFE (weak): th(1.3572938547200869E-5)=-20.259607397600604; dx=-1.0469509760442335E-4 evalInputDelta=0.0
WOLFE (weak): th(1.3844397318144886E-5)=-20.259607397600604; dx=-1.0467429794841092E-4 evalInputDelta=0.0
WOLFE (weak): th(1.3980126703616896E-5)=-20.259607397600604; dx=-1.0465005722420507E-4 evalInputDelta=0.0
Armijo: th(1.40479913963529E-5)=-20.259606583181466; dx=-1.0468115524426597E-4 evalInputDelta=-8.144191383507859E-7
WOLFE (weak): th(1.4014059049984898E-5)=-20.259607397600604; dx=-1.045869796337435E-4 evalInputDelta=0.0
Armijo: th(1.4031025223168899E-5)=-20.259606583181466; dx=-1.0463438697962653E-4 evalInputDelta=-8.144191383507859E-7
Armijo: th(1.4022542136576898E-5)=-20.259606583181466; dx=-1.0470462689096362E-4 evalInputDelta=-8.144191383507859E-7
WOLFE (weak): th(1.4018300593280898E-5)=-20.259607397600604; dx=-1.0470952558480001E-4 evalInputDelta=0.0
WOLFE (weak): th(1.4020421364928898E-5)=-20.259607397600604; dx=-1.047116338885353E-4 evalInputDelta=0.0
Armijo: th(1.4021481750752898E-5)=-20.259606583181466; dx=-1.046164548991237E-4 evalInputDelta=-8.144191383507859E-7
Armijo: th(1.40209515578409E-5)=-20.259606583181466; dx=-1.0466871351510326E-4 evalInputDelta=-8.144191383507859E-7
Armijo: th(1.4020686461384899E-5)=-20.259606583181466; dx=-1.047000412927732E-4 evalInputDelta=-8.144191383507859E-7
WOLFE (weak): th(1.4020553913156899E-5)=-20.259607397600604; dx=-1.0467238794913117E-4 evalInputDelta=0.0
mu ~= nu (1.4020553913156899E-5): th(0.0)=-20.259607397600604
Fitness changed from -20.259607397600604 to -20.259607397600604
Static Iteration Total: 29.9284; Orientation: 0.0200; Line Search: 28.2091
Iteration 17 failed. Error: -20.259607397600604
Previous Error: 0.0 -> -20.259607397600604
Optimization terminated 17
Final threshold in iteration 17: -20.259607397600604 (> -Infinity) after 124.091s (< 3600.000s)
Returns
-20.259607397600604