BasicOptimizer.scala:89 executed in 11.07 seconds (0.200 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139464231638800
Reset training subject: 139464404438400
Adding measurement 1fb4be0a to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.6265888444704836;dx=-3.8598934097579837E-4
New Minimum: -0.6265888444704836 > -0.627387848588893
WOLFE (weak): th(2.154434690031884)=-0.627387848588893; dx=-3.862067375053678E-4 evalInputDelta=7.990041184093855E-4
New Minimum: -0.627387848588893 > -0.6281820712307954
WOLFE (weak): th(4.308869380063768)=-0.6281820712307954; dx=-3.8419709981036567E-4 evalInputDelta=0.0015932267603118166
New Minimum: -0.6281820712307954 > -0.6313648844347851
WOLFE (weak): th(12.926608140191302)=-0.6313648844347851; dx=-3.806447378445789E-4 evalInputDelta=0.004776039964301537
New Minimum: -0.6313648844347851 > -0.6457684180208745
WOLFE (weak): th(51.70643256076521)=-0.6457684180208745; dx=-3.861254387642686E-4 evalInputDelta=0.019179573550390905
New Minimum: -0.6457684180208745 > -0.7238854560636211
WOLFE (weak): th(258.53216280382605)=-0.7238854560636211; dx=-3.8978626896422674E-4 evalInputDelta=0.09729661159313752
New Minimum: -0.7238854560636211 > -1.192663604045423
END: th(1551.1929768229563)=-1.192663604045423; dx=-3.3218663151025874E-4 evalInputDelta=0.5660747595749395
Fitness changed from -0.6265888444704836 to -1.192663604045423
Iteration 1 complete. Error: -1.192663604045423 Total: 1.5799; Orientation: 0.0021; Line Search: 1.1298
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6307f0ea to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.192663604045423;dx=-3.938243602354788E-4
New Minimum: -1.192663604045423 > -2.4527687241370497
END: th(3341.943960201201)=-2.4527687241370497; dx=-2.994896458675113E-4 evalInputDelta=1.2601051200916267
Fitness changed from -1.192663604045423 to -2.4527687241370497
Iteration 2 complete. Error: -2.4527687241370497 Total: 0.3949; Orientation: 0.0019; Line Search: 0.2599
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1442c3d7 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.4527687241370497;dx=-5.991306203120103E-4
New Minimum: -2.4527687241370497 > -6.427941383538052
END: th(7200.000000000001)=-6.427941383538052; dx=-1.6032296960121964E-4 evalInputDelta=3.975172659401002
Fitness changed from -2.4527687241370497 to -6.427941383538052
Iteration 3 complete. Error: -6.427941383538052 Total: 0.3832; Orientation: 0.0018; Line Search: 0.2472
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 20a561e0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.099e+03, gradient 3.529e-02, dot -0.149; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.427941383538052, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 20a561e0 to history. Total: 3
th(0)=-6.427941383538052;dx=-0.001223655919951034
Armijo: th(15511.929768229566)=-6.199245063722466; dx=8.817529031177264E-5 evalInputDelta=-0.2286963198155858
New Minimum: -6.427941383538052 > -8.509448010664272
WOLF (strong): th(7755.964884114783)=-8.509448010664272; dx=8.795378519991398E-5 evalInputDelta=2.0815066271262204
New Minimum: -8.509448010664272 > -9.018999656146722
END: th(2585.3216280382608)=-9.018999656146722; dx=-4.4111188911193154E-4 evalInputDelta=2.59105827260867
Fitness changed from -6.427941383538052 to -9.018999656146722
Iteration 4 complete. Error: -9.018999656146722 Total: 0.6639; Orientation: 0.0327; Line Search: 0.4993
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1d5c523 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.410e+03, gradient 3.148e-02, dot -0.205; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.018999656146722, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 1d5c523 to history. Total: 3
th(0)=-9.018999656146722;dx=-9.661449892316108E-4
New Minimum: -9.018999656146722 > -10.967842327944163
END: th(5569.9066003353355)=-10.967842327944163; dx=-1.7551680316598364E-4 evalInputDelta=1.9488426717974416
Fitness changed from -9.018999656146722 to -10.967842327944163
Iteration 5 complete. Error: -10.967842327944163 Total: 0.4157; Orientation: 0.0327; Line Search: 0.2531
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 53cb4c50 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.549e+03, gradient 3.394e-02, dot -0.147; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.967842327944163, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 53cb4c50 to history. Total: 3
th(0)=-10.967842327944163;dx=-0.0011389031641368145
Armijo: th(12000.000000000002)=-10.47851448172687; dx=1.20680538742707E-5 evalInputDelta=-0.48932784621729297
New Minimum: -10.967842327944163 > -13.23632340250627
END: th(6000.000000000001)=-13.23632340250627; dx=-4.416457714276114E-5 evalInputDelta=2.268481074562107
Fitness changed from -10.967842327944163 to -13.23632340250627
Iteration 6 complete. Error: -13.23632340250627 Total: 0.5388; Orientation: 0.0327; Line Search: 0.3756
Adding measurement 2b231e1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.305e+04, gradient 3.564e-02, dot -0.227; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.23632340250627, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 2b231e1a to history. Total: 3
th(0)=-13.23632340250627;dx=-0.0012507054983581626
Armijo: th(12926.608140191305)=-11.91535073641207; dx=6.925412696110061E-5 evalInputDelta=-1.3209726660942014
New Minimum: -13.23632340250627 > -15.56933176731361
WOLF (strong): th(6463.304070095653)=-15.56933176731361; dx=8.800296843398119E-5 evalInputDelta=2.3330083648073394
END: th(2154.434690031884)=-15.358499580344875; dx=-5.926145421907554E-4 evalInputDelta=2.122176177838604
Fitness changed from -13.23632340250627 to -15.56933176731361
Iteration 7 complete. Error: -15.56933176731361 Total: 0.6224; Orientation: 0.0302; Line Search: 0.4763
Adding measurement 539ed4f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.021e+03, gradient 3.713e-02, dot -0.096; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.56933176731361, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 539ed4f7 to history. Total: 3
th(0)=-15.56933176731361;dx=-0.0013570444093014909
New Minimum: -15.56933176731361 > -18.301309861593825
END: th(4641.58883361278)=-18.301309861593825; dx=-5.7826341062055285E-6 evalInputDelta=2.7319780942802154
Fitness changed from -15.56933176731361 to -18.301309861593825
Iteration 8 complete. Error: -18.301309861593825 Total: 0.4029; Orientation: 0.0340; Line Search: 0.2488
Adding measurement 7890b48c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.534e+05, gradient 3.336e-02, dot -0.217; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.301309861593825, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 7890b48c to history. Total: 3
th(0)=-18.301309861593825;dx=-0.0010828033999292796
Armijo: th(10000.000000000004)=-17.92080207192483; dx=8.672458844968731E-5 evalInputDelta=-0.38050778966899657
New Minimum: -18.301309861593825 > -20.536810767765964
WOLF (strong): th(5000.000000000002)=-20.536810767765964; dx=1.5250067915919102E-5 evalInputDelta=2.23550090617213
...skipping 467 bytes...
Orientation rejected. Popping history element from -20.536810767765964, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 3e0864b1 to history. Total: 3
th(0)=-20.536810767765964;dx=-0.0011978442357787867
New Minimum: -20.536810767765964 > -22.489649647851458
END: th(3590.7244833864743)=-22.489649647851458; dx=-1.8214386467811327E-4 evalInputDelta=1.952838880085494
Fitness changed from -20.536810767765964 to -22.489649647851458
Iteration 10 complete. Error: -22.489649647851458 Total: 0.3799; Orientation: 0.0322; Line Search: 0.2340
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1a692058 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.549e+04, gradient 2.965e-02, dot 0.260; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-22.489649647851458;dx=-118.7483236557521
Armijo: th(7735.981389354635)=-10.237819723884765; dx=1.1332922079400668E-4 evalInputDelta=-12.251829923966692
Armijo: th(3867.9906946773176)=-10.23009325626177; dx=2.2282552139650367E-4 evalInputDelta=-12.259556391589689
Armijo: th(1289.3302315591059)=-10.167939350142342; dx=7.458371103250864E-4 evalInputDelta=-12.321710297709116
Armijo: th(322.33255788977647)=-10.13203289505833; dx=0.0033225982047701682 evalInputDelta=-12.357616752793128
Armijo: th(64.46651157795529)=-10.02978821907803; dx=0.023774603656161124 evalInputDelta=-12.459861428773428
Armijo: th(10.74441859632588)=-10.559668794483446; dx=0.17629591115797869 evalInputDelta=-11.929980853368011
Armijo: th(1.5349169423322686)=-8.545363984341844; dx=1.8445063889555402 evalInputDelta=-13.944285663509614
Armijo: th(0.19186461779153358)=-12.389188874194014; dx=-12.161685443948482 evalInputDelta=-10.100460773657444
New Minimum: -22.489649647851458 > -23.781300488493162
WOLF (strong): th(0.021318290865725953)=-23.781300488493162; dx=35.176304542693664 evalInputDelta=1.2916508406417044
END: th(0.0021318290865725955)=-22.924935843171045; dx=-89.56689964004259 evalInputDelta=0.4352861953195877
Fitness changed from -22.489649647851458 to -23.781300488493162
Iteration 11 complete. Error: -23.781300488493162 Total: 1.4592; Orientation: 0.0304; Line Search: 1.2989
Adding measurement 26802c79 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.079e+04, gradient 3.771e-02, dot -0.077; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.781300488493162, -22.489649647851458, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
Rejected: LBFGS Orientation magnitude: 8.338e+03, gradient 3.771e-02, dot -0.090; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.781300488493162, -22.489649647851458, -2.4527687241370497, -1.192663604045423
LBFGS Accumulation History: 3 points
Removed measurement 26802c79 to history. Total: 4
Removed measurement 1a692058 to history. Total: 3
th(0)=-23.781300488493162;dx=-0.0014036598127184622
New Minimum: -23.781300488493162 > -23.78130745225546
WOLFE (weak): th(0.0045928865373309835)=-23.78130745225546; dx=-0.0013880757381061987 evalInputDelta=6.96376229925022E-6
New Minimum: -23.78130745225546 > -23.781315286485615
WOLFE (weak): th(0.009185773074661967)=-23.781315286485615; dx=-0.0013943785223750455 evalInputDelta=1.4797992452741937E-5
New Minimum: -23.781315286485615 > -23.78135271666072
WOLFE (weak): th(0.027557319223985903)=-23.78135271666072; dx=-0.0014013405263248499 evalInputDelta=5.2228167557899496E-5
New Minimum: -23.78135271666072 > -23.78149895491967
WOLFE (weak): th(0.11022927689594361)=-23.78149895491967; dx=-0.0013905044805566946 evalInputDelta=1.984664265073377E-4
New Minimum: -23.78149895491967 > -23.782301508402764
WOLFE (weak): th(0.5511463844797181)=-23.782301508402764; dx=-0.0014022875736860022 evalInputDelta=0.0010010199096015526
New Minimum: -23.782301508402764 > -23.787336434828685
WOLFE (weak): th(3.3068783068783087)=-23.787336434828685; dx=-0.0013956270228815159 evalInputDelta=0.006035946335522624
New Minimum: -23.787336434828685 > -23.823633827414262
WOLFE (weak): th(23.14814814814816)=-23.823633827414262; dx=-0.0014024098751786868 evalInputDelta=0.04233333892110025
New Minimum: -23.823633827414262 > -24.116612070225134
WOLFE (weak): th(185.18518518518528)=-24.116612070225134; dx=-0.00135644108999575 evalInputDelta=0.33531158173197184
New Minimum: -24.116612070225134 > -26.211026341966814
END: th(1666.6666666666674)=-26.211026341966814; dx=-8.494438818301178E-4 evalInputDelta=2.429725853473652
Fitness changed from -23.781300488493162 to -26.211026341966814
Iteration 12 complete. Error: -26.211026341966814 Total: 1.3342; Orientation: 0.0640; Line Search: 1.1576
Adding measurement 6900caba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.381e+02, gradient 2.940e-02, dot -0.147; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.211026341966814, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 6900caba to history. Total: 3
th(0)=-26.211026341966814;dx=-8.436057211256649E-4
New Minimum: -26.211026341966814 > -28.846891343308293
END: th(3590.7244833864747)=-28.846891343308293; dx=-5.32437787266504E-4 evalInputDelta=2.635865001341479
Fitness changed from -26.211026341966814 to -28.846891343308293
Iteration 13 complete. Error: -28.846891343308293 Total: 0.3833; Orientation: 0.0293; Line Search: 0.2360
Adding measurement 69b8f836 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.226e+05, gradient 2.614e-02, dot -0.165; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.846891343308293, -2.4527687241370497, -1.192663604045423, -0.6265888444704836
LBFGS Accumulation History: 3 points
Removed measurement 69b8f836 to history. Total: 3
th(0)=-28.846891343308293;dx=-6.622291562383737E-4
New Minimum: -28.846891343308293 > -31.939540162990635
END: th(7735.981389354636)=-31.939540162990635; dx=-3.7690919067641523E-4 evalInputDelta=3.092648819682342
Fitness changed from -28.846891343308293 to -31.939540162990635
Iteration 14 complete. Error: -31.939540162990635 Total: 0.3714; Orientation: 0.0284; Line Search: 0.2294
Adding measurement 5c8a0778 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.942e+04, gradient 2.926e-02, dot 0.225; [5d45b6ec-4526-4c8a-9fe8-a0e19c9771a5 = 1.000/1.000e+00]
th(0)=-31.939540162990635;dx=-128.6247100311718
Armijo: th(16666.66666666668)=-11.825641641038754; dx=1.6558551828315125E-4 evalInputDelta=-20.11389852195188
Armijo: th(8333.33333333334)=-11.810738901225292; dx=3.176407418432263E-4 evalInputDelta=-20.128801261765343
Armijo: th(2777.7777777777796)=-11.80018061648926; dx=0.0010007721747199577 evalInputDelta=-20.139359546501375
Armijo: th(694.4444444444449)=-11.785884594320063; dx=0.00380141574115619 evalInputDelta=-20.153655568670573
Armijo: th(138.88888888888897)=-11.763871950566383; dx=0.018860942692198398 evalInputDelta=-20.175668212424252
Armijo: th(23.148148148148163)=-11.782897430037362; dx=0.10708266475223932 evalInputDelta=-20.156642732953273
Armijo: th(3.306878306878309)=-11.203283016320343; dx=1.1473944617008456 evalInputDelta=-20.73625714667029
Armijo: th(0.41335978835978865)=-10.654443254270932; dx=-1.3585843989422897 evalInputDelta=-21.2850969087197
Armijo: th(0.04592886537330985)=-27.985564918028217; dx=18.877673836986002 evalInputDelta=-3.953975244962418
New Minimum: -31.939540162990635 > -32.65988724044517
END: th(0.004592886537330984)=-32.65988724044517; dx=-71.95770502661684 evalInputDelta=0.7203470774545373
Fitness changed from -31.939540162990635 to -32.65988724044517
Iteration 15 complete. Error: -32.65988724044517 Total: 1.4807; Orientation: 0.0291; Line Search: 1.3406
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -32.65988724044517 (> -Infinity) after 11.064s (< 5400.000s)
Returns
-32.65988724044517