BasicOptimizer.scala:89 executed in 70.06 seconds (1.399 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 160451138899400
Reset training subject: 160452032923900
Adding measurement 3db71312 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-28.626632177471013;dx=-0.004369237079605983
New Minimum: -28.626632177471013 > -28.63670995813874
WOLFE (weak): th(2.154434690031884)=-28.63670995813874; dx=-0.004366119356629059 evalInputDelta=0.010077780667728575
New Minimum: -28.63670995813874 > -28.6467873894155
WOLFE (weak): th(4.308869380063768)=-28.6467873894155; dx=-0.004334115337694855 evalInputDelta=0.02015521194448766
New Minimum: -28.6467873894155 > -28.687093627976893
WOLFE (weak): th(12.926608140191302)=-28.687093627976893; dx=-0.004369787735304179 evalInputDelta=0.06046145050587981
New Minimum: -28.687093627976893 > -28.868790128043663
WOLFE (weak): th(51.70643256076521)=-28.868790128043663; dx=-0.004390241475297813 evalInputDelta=0.24215795057265055
New Minimum: -28.868790128043663 > -29.84562739513738
WOLFE (weak): th(258.53216280382605)=-29.84562739513738; dx=-0.0044341221746719995 evalInputDelta=1.218995217666368
New Minimum: -29.84562739513738 > -36.073836159652295
WOLFE (weak): th(1551.1929768229563)=-36.073836159652295; dx=-0.004557632962673762 evalInputDelta=7.447203982181282
New Minimum: -36.073836159652295 > -67.8902240350645
END: th(10858.350837760694)=-67.8902240350645; dx=-0.002318435082154028 evalInputDelta=39.2635918575935
Fitness changed from -28.626632177471013 to -67.8902240350645
Iteration 1 complete. Error: -67.8902240350645 Total: 10.2224; Orientation: 0.0145; Line Search: 6.9205
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 648bbc53 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-67.8902240350645;dx=-0.006137967294708415
New Minimum: -67.8902240350645 > -129.79023792017256
END: th(23393.607721408407)=-129.79023792017256; dx=-0.001332015885205197 evalInputDelta=61.90001388510805
Fitness changed from -67.8902240350645 to -129.79023792017256
Iteration 2 complete. Error: -129.79023792017256 Total: 2.8324; Orientation: 0.0179; Line Search: 1.8582
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 70c606d5 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-129.79023792017256;dx=-0.008494254549464637
New Minimum: -129.79023792017256 > -187.78506455187005
END: th(50400.00000000001)=-187.78506455187005; dx=-1.3482687078671248E-4 evalInputDelta=57.99482663169749
Fitness changed from -129.79023792017256 to -187.78506455187005
Iteration 3 complete. Error: -187.78506455187005 Total: 2.5771; Orientation: 0.0144; Line Search: 1.6624
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 18dd8e01 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.381e+03, gradient 9.625e-02, dot -0.148; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -187.78506455187005, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 18dd8e01 to history. Total: 3
th(0)=-187.78506455187005;dx=-0.00923799938138171
New Minimum: -187.78506455187005 > -210.8511675181098
WOLF (strong): th(108583.50837760697)=-210.8511675181098; dx=7.44310411967453E-4 evalInputDelta=23.066102966239754
New Minimum: -210.8511675181098 > -287.31549786341145
WOLF (strong): th(54291.75418880348)=-287.31549786341145; dx=2.103350591973157E-4 evalInputDelta=99.5304333115414
New Minimum: -287.31549786341145 > -297.3604452091771
END: th(18097.251396267828)=-297.3604452091771; dx=-0.0032479750612255873 evalInputDelta=109.57538065730705
Fitness changed from -187.78506455187005 to -297.3604452091771
Iteration 4 complete. Error: -297.3604452091771 Total: 4.5330; Orientation: 0.2738; Line Search: 3.3793
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1f5a03ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.448e+03, gradient 8.180e-02, dot -0.170; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -297.3604452091771, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 1f5a03ea to history. Total: 3
th(0)=-297.3604452091771;dx=-0.006681722445946324
New Minimum: -297.3604452091771 > -406.2625382680515
END: th(38989.346202347355)=-406.2625382680515; dx=-0.001319044281430777 evalInputDelta=108.90209305887441
Fitness changed from -297.3604452091771 to -406.2625382680515
Iteration 5 complete. Error: -406.2625382680515 Total: 2.9968; Orientation: 0.2713; Line Search: 1.6833
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4e73ab0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.418e+04, gradient 9.970e-02, dot -0.168; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -406.2625382680515, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 4e73ab0 to history. Total: 3
th(0)=-406.2625382680515;dx=-0.009927169343755083
Armijo: th(84000.00000000003)=-343.6786516864264; dx=8.659729885664472E-4 evalInputDelta=-62.58388658162511
New Minimum: -406.2625382680515 > -458.8559652821351
WOLF (strong): th(42000.000000000015)=-458.8559652821351; dx=3.631416873674232E-4 evalInputDelta=52.5934270140836
New Minimum: -458.8559652821351 > -495.70286273633724
END: th(14000.000000000005)=-495.70286273633724; dx=-0.003734422685221987 evalInputDelta=89.44032446828572
Fitness changed from -406.2625382680515 to -495.70286273633724
Iteration 6 complete. Error: -495.70286273633724 Total: 4.5849; Orientation: 0.2733; Line Search: 3.4080
Adding measurement 36bf4df9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.938e+04, gradient 8.319e-02, dot -0.210; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -495.70286273633724, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 36bf4df9 to history. Total: 3
th(0)=-495.70286273633724;dx=-0.00689456565554342
New Minimum: -495.70286273633724 > -584.4522542517909
END: th(30162.085660446384)=-584.4522542517909; dx=-0.0022369546135085874 evalInputDelta=88.74939151545368
Fitness changed from -495.70286273633724 to -584.4522542517909
Iteration 7 complete. Error: -584.4522542517909 Total: 2.8431; Orientation: 0.2750; Line Search: 1.7337
Adding measurement 6e6256aa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.283e+04, gradient 1.010e-01, dot -0.207; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -584.4522542517909, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 6e6256aa to history. Total: 3
th(0)=-584.4522542517909;dx=-0.010150072986391473
Armijo: th(64982.24367057894)=-563.4607905613309; dx=5.122936809194736E-4 evalInputDelta=-20.991463690460023
New Minimum: -584.4522542517909 > -644.8462752858235
END: th(32491.12183528947)=-644.8462752858235; dx=-5.488615895922571E-4 evalInputDelta=60.39402103403256
Fitness changed from -584.4522542517909 to -644.8462752858235
Iteration 8 complete. Error: -644.8462752858235 Total: 3.7841; Orientation: 0.2634; Line Search: 2.4801
Adding measurement 29df61c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.151e+04, gradient 1.083e-01, dot -0.199; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -644.8462752858235, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 29df61c to history. Total: 3
th(0)=-644.8462752858235;dx=-0.011689960544872962
Armijo: th(70000.00000000004)=-550.8361541783546; dx=0.0018189893168056254 evalInputDelta=-94.01012110746888
New Minimum: -644.8462752858235 > -721.5204648864784
WOLF (strong): th(35000.00000000002)=-721.5204648864784; dx=0.0010686718056029827 evalInputDelta=76.67418960065493
New Minimum: -721.5204648864784 > -753.4224412638636
END: th(11666.666666666673)=-753.4224412638636; dx=-0.005667354158941602 evalInputDelta=108.57616597804008
Fitness changed from -644.8462752858235 to -753.4224412638636
Iteration 9 complete. Error: -753.4224412638636 Total: 4.4116; Orientation: 0.2728; Line Search: 3.3598
Adding measurement 7e8c782b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.548e+04, gradient 8.864e-02, dot -0.270; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -753.4224412638636, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 7e8c782b to history. Total: 3
th(0)=-753.4224412638636;dx=-0.007827649576051959
New Minimum: -753.4224412638636 > -844.3993427283089
END: th(25135.071383705326)=-844.3993427283089; dx=-0.0035147273710885666 evalInputDelta=90.97690146444529
Fitness changed from -753.4224412638636 to -844.3993427283089
Iteration 10 complete. Error: -844.3993427283089 Total: 2.9992; Orientation: 0.2726; Line Search: 1.9434
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 17c37deb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.622e+05, gradient 9.659e-02, dot -0.282; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -844.3993427283089, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 17c37deb to history. Total: 3
th(0)=-844.3993427283089;dx=-0.009285099186699569
New Minimum: -844.3993427283089 > -858.4956319050202
END: th(54151.86972548246)=-858.4956319050202; dx=-7.468340496889639E-6 evalInputDelta=14.096289176711366
Fitness changed from -844.3993427283089 to -858.4956319050202
Iteration 11 complete. Error: -858.4956319050202 Total: 2.8186; Orientation: 0.2709; Line Search: 1.6643
Adding measurement 73d602a3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.579e+05, gradient 1.240e-01, dot -0.195; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -858.4956319050202, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 73d602a3 to history. Total: 3
th(0)=-858.4956319050202;dx=-0.015320543140912135
Armijo: th(116666.66666666676)=-536.7139601314652; dx=0.0023854140384814504 evalInputDelta=-321.781671773555
Armijo: th(58333.33333333338)=-849.0137292765058; dx=0.003002191288929402 evalInputDelta=-9.481902628514376
New Minimum: -858.4956319050202 > -1042.0041386674047
END: th(19444.44444444446)=-1042.0041386674047; dx=-0.003204991044192843 evalInputDelta=183.5085067623845
Fitness changed from -858.4956319050202 to -1042.0041386674047
Iteration 12 complete. Error: -1042.0041386674047 Total: 4.4519; Orientation: 0.2707; Line Search: 3.3879
Adding measurement 447e1b04 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.036e+05, gradient 1.024e-01, dot -0.254; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1042.0041386674047, -129.79023792017256, -67.8902240350645, -28.626632177471013
LBFGS Accumulation History: 3 points
Removed measurement 447e1b04 to history. Total: 3
th(0)=-1042.0041386674047;dx=-0.010473104176342838
New Minimum: -1042.0041386674047 > -1090.953424762029
END: th(41891.78563950888)=-1090.953424762029; dx=-1.653760362978199E-4 evalInputDelta=48.94928609462431
Fitness changed from -1042.0041386674047 to -1090.953424762029
Iteration 13 complete. Error: -1090.953424762029 Total: 3.0402; Orientation: 0.2716; Line Search: 1.9603
Adding measurement 103078db to history. Total: 3
Accepted: LBFGS Orientation magnitude: 8.505e+03, gradient 1.323e-01, dot 0.370; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1090.953424762029;dx=-415.2518644829638
Armijo: th(90253.11620913744)=-45.15124999351139; dx=2.109031041947497E-4 evalInputDelta=-1045.8021747685177
Armijo: th(45126.55810456872)=-45.16345503721386; dx=4.2110002218543886E-4 evalInputDelta=-1045.7899697248151
Armijo: th(15042.18603485624)=-45.16243875194781; dx=0.0012430978042225877 evalInputDelta=-1045.7909860100813
Armijo: th(3760.54650871406)=-44.64728440658863; dx=0.005000043415116846 evalInputDelta=-1046.3061403554404
Armijo: th(752.109301742812)=-44.297686555616956; dx=0.02734487377093489 evalInputDelta=-1046.655738206412
Armijo: th(125.35155029046867)=-44.32645892621584; dx=0.1703852138639441 evalInputDelta=-1046.6269658358133
Armijo: th(17.90736432720981)=-46.013956357691086; dx=1.8007321365445548 evalInputDelta=-1044.939468404338
Armijo: th(2.238420540901226)=-339.5407169066767; dx=118.71103908470124 evalInputDelta=-751.4127078553523
New Minimum: -1090.953424762029 > -1128.644098022047
END: th(0.24871339343346957)=-1128.644098022047; dx=-57.57254148647935 evalInputDelta=37.690673260018
Fitness changed from -1090.953424762029 to -1128.644098022047
Iteration 14 complete. Error: -1128.644098022047 Total: 9.6987; Orientation: 0.2665; Line Search: 8.6237
Adding measurement 79b86459 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.194e+06, gradient 1.198e-01, dot -0.363; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1128.644098022047, -1090.953424762029, -129.79023792017256, -67.8902240350645, -28.626632177471013
Rejected: LBFGS Orientation magnitude: 9.605e+05, gradient 1.198e-01, dot -0.338; [52163ed3-69e1-42a9-a38a-bd4a975b6500 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1128.644098022047, -1090.953424762029, -129.79023792017256, -67.8902240350645
LBFGS Accumulation History: 3 points
Removed measurement 79b86459 to history. Total: 4
Removed measurement 103078db to history. Total: 3
th(0)=-1128.644098022047;dx=-0.014256486896831157
New Minimum: -1128.644098022047 > -1128.6527366732425
WOLFE (weak): th(0.535836762688615)=-1128.6527366732425; dx=-0.013220277517886498 evalInputDelta=0.008638651195497005
New Minimum: -1128.6527366732425 > -1128.6614306336512
WOLFE (weak): th(1.07167352537723)=-1128.6614306336512; dx=-0.013273888466004715 evalInputDelta=0.017332611604160775
New Minimum: -1128.6614306336512 > -1128.696150431993
WOLFE (weak): th(3.21502057613169)=-1128.696150431993; dx=-0.013291911461052283 evalInputDelta=0.05205240994587257
New Minimum: -1128.696150431993 > -1128.851905698883
WOLFE (weak): th(12.86008230452676)=-1128.851905698883; dx=-0.013234322573992943 evalInputDelta=0.20780767683595514
New Minimum: -1128.851905698883 > -1129.6822008866034
WOLFE (weak): th(64.3004115226338)=-1129.6822008866034; dx=-0.013255129807015105 evalInputDelta=1.038102864556322
New Minimum: -1129.6822008866034 > -1134.8209109811116
WOLFE (weak): th(385.80246913580277)=-1134.8209109811116; dx=-0.013059996305390616 evalInputDelta=6.176812959064591
New Minimum: -1134.8209109811116 > -1169.07591712429
END: th(2700.6172839506194)=-1169.07591712429; dx=-0.01170726907560582 evalInputDelta=40.431819102243026
Fitness changed from -1128.644098022047 to -1169.07591712429
Iteration 15 complete. Error: -1169.07591712429 Total: 8.1696; Orientation: 0.6131; Line Search: 6.7415
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1169.07591712429 (> -Infinity) after 70.064s (< 5400.000s)
Returns
-1169.07591712429