BasicOptimizer.scala:89 executed in 11.03 seconds (0.656 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 148631317109700
Reset training subject: 148631448799900
Adding measurement 3c75cb72 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-43.599740571816334;dx=-0.07490279064346261
New Minimum: -43.599740571816334 > -43.84439175626186
WOLFE (weak): th(2.154434690031884)=-43.84439175626186; dx=-0.07537663573947084 evalInputDelta=0.24465118444552303
New Minimum: -43.84439175626186 > -44.08907526380189
WOLFE (weak): th(4.308869380063768)=-44.08907526380189; dx=-0.07656071667949799 evalInputDelta=0.48933469198555457
New Minimum: -44.08907526380189 > -45.080960157809415
WOLFE (weak): th(12.926608140191302)=-45.080960157809415; dx=-0.07587829231950238 evalInputDelta=1.481219585993081
New Minimum: -45.080960157809415 > -49.54234152233446
WOLFE (weak): th(51.70643256076521)=-49.54234152233446; dx=-0.07605285866281188 evalInputDelta=5.942600950518127
New Minimum: -49.54234152233446 > -70.74236997836763
END: th(258.53216280382605)=-70.74236997836763; dx=-0.06306597156620034 evalInputDelta=27.1426294065513
Fitness changed from -43.599740571816334 to -70.74236997836763
Iteration 1 complete. Error: -70.74236997836763 Total: 1.0417; Orientation: 0.0016; Line Search: 0.6900
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6cde5cfa to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-70.74236997836763;dx=-0.11281204350014488
New Minimum: -70.74236997836763 > -121.95803661633988
END: th(556.9906600335335)=-121.95803661633988; dx=-0.04717212063136292 evalInputDelta=51.21566663797225
Fitness changed from -70.74236997836763 to -121.95803661633988
Iteration 2 complete. Error: -121.95803661633988 Total: 0.3458; Orientation: 0.0016; Line Search: 0.2247
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement d780465 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-121.95803661633988;dx=-0.1648599828336676
New Minimum: -121.95803661633988 > -173.30302782445838
END: th(1200.0)=-173.30302782445838; dx=-0.016977417472136474 evalInputDelta=51.3449912081185
Fitness changed from -121.95803661633988 to -173.30302782445838
Iteration 3 complete. Error: -173.30302782445838 Total: 0.3489; Orientation: 0.0015; Line Search: 0.2271
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2795907 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.244e+03, gradient 4.257e-01, dot -0.147; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -173.30302782445838, -121.95803661633988, -70.74236997836763, -43.599740571816334
LBFGS Accumulation History: 3 points
Removed measurement 2795907 to history. Total: 3
th(0)=-173.30302782445838;dx=-0.17464915538623194
New Minimum: -173.30302782445838 > -320.5157464930545
END: th(2585.3216280382608)=-320.5157464930545; dx=-0.019367938113055205 evalInputDelta=147.21271866859612
Fitness changed from -173.30302782445838 to -320.5157464930545
Iteration 4 complete. Error: -320.5157464930545 Total: 0.3747; Orientation: 0.0281; Line Search: 0.2271
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 5985cd35 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.590e+03, gradient 4.392e-01, dot -0.140; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -320.5157464930545, -121.95803661633988, -70.74236997836763, -43.599740571816334
LBFGS Accumulation History: 3 points
Removed measurement 5985cd35 to history. Total: 3
th(0)=-320.5157464930545;dx=-0.18892418295738722
New Minimum: -320.5157464930545 > -521.3795702269125
WOLF (strong): th(5569.9066003353355)=-521.3795702269125; dx=1.0160225109661714E-4 evalInputDelta=200.863823733858
New Minimum: -521.3795702269125 > -581.1381343966337
END: th(2784.9533001676677)=-581.1381343966337; dx=-0.024130195780581943 evalInputDelta=260.62238790357924
Fitness changed from -320.5157464930545 to -581.1381343966337
Iteration 5 complete. Error: -581.1381343966337 Total: 0.4931; Orientation: 0.0278; Line Search: 0.3419
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 27a4e04 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.978e+04, gradient 4.633e-01, dot -0.211; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -581.1381343966337, -121.95803661633988, -70.74236997836763, -43.599740571816334
LBFGS Accumulation History: 3 points
Removed measurement 27a4e04 to history. Total: 3
th(0)=-581.1381343966337;dx=-0.20794981788841066
New Minimum: -581.1381343966337 > -684.9455726552293
WOLF (strong): th(6000.000000000001)=-684.9455726552293; dx=0.0017502959417594974 evalInputDelta=103.80743825859554
New Minimum: -684.9455726552293 > -797.4485093095353
END: th(3000.0000000000005)=-797.4485093095353; dx=-0.0138176080164933 evalInputDelta=216.31037491290158
Fitness changed from -581.1381343966337 to -797.4485093095353
Iteration 6 complete. Error: -797.4485093095353 Total: 0.4876; Orientation: 0.0277; Line Search: 0.3388
Adding measurement 2b930e4c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.410e+04, gradient 5.596e-01, dot -0.159; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -797.4485093095353, -121.95803661633988, -70.74236997836763, -43.599740571816334
LBFGS Accumulation History: 3 points
Removed measurement 2b930e4c to history. Total: 3
th(0)=-797.4485093095353;dx=-0.3000242018315581
Armijo: th(6463.304070095653)=-768.0509830082897; dx=0.021934664007830625 evalInputDelta=-29.397526301245648
New Minimum: -797.4485093095353 > -923.7716167971389
WOLF (strong): th(3231.6520350478263)=-923.7716167971389; dx=0.017258342269461262 evalInputDelta=126.32310748760358
New Minimum: -923.7716167971389 > -1010.8333195932947
END: th(1077.217345015942)=-1010.8333195932947; dx=-0.03537954233858947 evalInputDelta=213.3848102837594
Fitness changed from -797.4485093095353 to -1010.8333195932947
Iteration 7 complete. Error: -1010.8333195932947 Total: 0.5947; Orientation: 0.0268; Line Search: 0.4582
Adding measurement 763e3342 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.439e+05, gradient 4.889e-01, dot -0.194; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1010.8333195932947, -121.95803661633988, -70.74236997836763, -43.599740571816334
LBFGS Accumulation History: 3 points
Removed measurement 763e3342 to history. Total: 3
th(0)=-1010.8333195932947;dx=-0.2335651388603467
New Minimum: -1010.8333195932947 > -1067.0834667447527
WOLF (strong): th(2320.79441680639)=-1067.0834667447527; dx=0.007551549289045434 evalInputDelta=56.25014715145801
New Minimum: -1067.0834667447527 > -1145.384215012587
WOLF (strong): th(1160.397208403195)=-1145.384215012587; dx=8.100199267078948E-4 evalInputDelta=134.55089541929226
END: th(386.79906946773167)=-1110.181516689951; dx=-0.13472541543564026 evalInputDelta=99.34819709665624
Fitness changed from -1010.8333195932947 to -1145.384215012587
Iteration 8 complete. Error: -1145.384215012587 Total: 0.5972; Orientation: 0.0275; Line Search: 0.4570
Adding measurement 44a3d762 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.350e+04, gradient 5.661e-01, dot 0.195; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1145.384215012587;dx=-3774.025905586411
Armijo: th(833.3333333333336)=-177.97484350849982; dx=0.30414262770674017 evalInputDelta=-967.4093715040872
Armijo: th(416.6666666666668)=-178.33700650909503; dx=0.5897217237304941 evalInputDelta=-967.0472085034919
Armijo: th(138.88888888888894)=-177.52709410291433; dx=1.7842418103814786 evalInputDelta=-967.8571209096726
Armijo: th(34.722222222222236)=-172.3323687311818; dx=7.1660872772477475 evalI
...skipping 746 bytes...
tion magnitude: 1.902e+04, gradient 5.770e-01, dot -0.220; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1198.6406363043095, -1145.384215012587, -121.95803661633988, -70.74236997836763, -43.599740571816334
Rejected: LBFGS Orientation magnitude: 4.531e+04, gradient 5.770e-01, dot -0.274; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1198.6406363043095, -1145.384215012587, -121.95803661633988, -70.74236997836763
LBFGS Accumulation History: 3 points
Removed measurement 6b848bfd to history. Total: 4
Removed measurement 44a3d762 to history. Total: 3
th(0)=-1198.6406363043095;dx=-0.3305123119375619
New Minimum: -1198.6406363043095 > -1198.6618059319317
WOLFE (weak): th(0.044527833375328306)=-1198.6618059319317; dx=-0.306551381619099 evalInputDelta=0.02116962762215735
New Minimum: -1198.6618059319317 > -1198.6830273262403
WOLFE (weak): th(0.08905566675065661)=-1198.6830273262403; dx=-0.30653380689004406 evalInputDelta=0.042391021930825445
New Minimum: -1198.6830273262403 > -1198.7679612835839
WOLFE (weak): th(0.2671670002519698)=-1198.7679612835839; dx=-0.30434096094368435 evalInputDelta=0.1273249792743627
New Minimum: -1198.7679612835839 > -1199.1500635867055
WOLFE (weak): th(1.0686680010078793)=-1199.1500635867055; dx=-0.3084044510585521 evalInputDelta=0.5094272823960182
New Minimum: -1199.1500635867055 > -1201.1823862344968
WOLFE (weak): th(5.343340005039396)=-1201.1823862344968; dx=-0.30326262404062165 evalInputDelta=2.5417499301872795
New Minimum: -1201.1823862344968 > -1213.7294282499704
END: th(32.06004003023638)=-1213.7294282499704; dx=-0.29431001844779936 evalInputDelta=15.088791945660887
Fitness changed from -1198.6406363043095 to -1213.7294282499704
Iteration 10 complete. Error: -1213.7294282499704 Total: 0.9855; Orientation: 0.0607; Line Search: 0.8129
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 298c735e to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.793e+04, gradient 5.601e-01, dot 0.163; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
th(0)=-1213.7294282499704;dx=-2609.0092135123245
Armijo: th(69.07126240495211)=-243.11512672754446; dx=4.700561378218346 evalInputDelta=-970.6143015224259
Armijo: th(34.535631202476054)=-244.19437058417216; dx=9.497959512041106 evalInputDelta=-969.5350576657983
Armijo: th(11.511877067492017)=-253.98149077491848; dx=27.608964512900247 evalInputDelta=-959.7479374750519
Armijo: th(2.8779692668730044)=-306.00883361432557; dx=104.31568009598308 evalInputDelta=-907.7205946356448
Armijo: th(0.5755938533746009)=-647.8073112430887; dx=352.7546052272925 evalInputDelta=-565.9221170068816
Armijo: th(0.09593230889576682)=-1196.7589042910856; dx=-290.5225379970202 evalInputDelta=-16.97052395888477
New Minimum: -1213.7294282499704 > -1231.1621034616035
END: th(0.013704615556538116)=-1231.1621034616035; dx=-1633.95959091489 evalInputDelta=17.432675211633068
Fitness changed from -1213.7294282499704 to -1231.1621034616035
Iteration 11 complete. Error: -1231.1621034616035 Total: 1.0607; Orientation: 0.0268; Line Search: 0.9129
Adding measurement 529a50cb to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.822e+06, gradient 5.839e-01, dot -0.144; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1231.1621034616035, -1213.7294282499704, -121.95803661633988, -70.74236997836763, -43.599740571816334
Rejected: LBFGS Orientation magnitude: 3.114e+06, gradient 5.839e-01, dot -0.285; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1231.1621034616035, -1213.7294282499704, -121.95803661633988, -70.74236997836763
LBFGS Accumulation History: 3 points
Removed measurement 529a50cb to history. Total: 4
Removed measurement 298c735e to history. Total: 3
th(0)=-1231.1621034616035;dx=-0.33273105228576455
New Minimum: -1231.1621034616035 > -1231.1763175922447
WOLFE (weak): th(0.029525699168556328)=-1231.1763175922447; dx=-0.3019891278489973 evalInputDelta=0.014214130641221345
New Minimum: -1231.1763175922447 > -1231.190430031033
WOLFE (weak): th(0.059051398337112655)=-1231.190430031033; dx=-0.30595014509135676 evalInputDelta=0.028326569429509618
New Minimum: -1231.190430031033 > -1231.24692893018
WOLFE (weak): th(0.17715419501133797)=-1231.24692893018; dx=-0.30565530812209807 evalInputDelta=0.08482546857658235
New Minimum: -1231.24692893018 > -1231.5019792919538
WOLFE (weak): th(0.7086167800453519)=-1231.5019792919538; dx=-0.302285876831234 evalInputDelta=0.3398758303503655
New Minimum: -1231.5019792919538 > -1232.853042742727
WOLFE (weak): th(3.5430839002267596)=-1232.853042742727; dx=-0.3040166501650541 evalInputDelta=1.6909392811235193
New Minimum: -1232.853042742727 > -1241.1867305123753
END: th(21.258503401360556)=-1241.1867305123753; dx=-0.2960744369324406 evalInputDelta=10.024627050771869
Fitness changed from -1231.1621034616035 to -1241.1867305123753
Iteration 12 complete. Error: -1241.1867305123753 Total: 1.7119; Orientation: 0.0647; Line Search: 0.8499
Adding measurement b47264c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.205e+04, gradient 5.697e-01, dot -0.104; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1241.1867305123753, -121.95803661633988, -70.74236997836763, -43.599740571816334
LBFGS Accumulation History: 3 points
Removed measurement b47264c to history. Total: 3
th(0)=-1241.1867305123753;dx=-0.31245113659057944
New Minimum: -1241.1867305123753 > -1261.1819852820606
END: th(45.80005718605198)=-1261.1819852820606; dx=-0.26311815094377716 evalInputDelta=19.99525476968529
Fitness changed from -1241.1867305123753 to -1261.1819852820606
Iteration 13 complete. Error: -1261.1819852820606 Total: 0.3735; Orientation: 0.0295; Line Search: 0.2330
Adding measurement 7a0fa617 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.201e+04, gradient 5.273e-01, dot -0.135; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1261.1819852820606, -121.95803661633988, -70.74236997836763, -43.599740571816334
LBFGS Accumulation History: 3 points
Removed measurement 7a0fa617 to history. Total: 3
th(0)=-1261.1819852820606;dx=-0.2712614442833069
New Minimum: -1261.1819852820606 > -1296.0069926508884
END: th(98.67323200707445)=-1296.0069926508884; dx=-0.2106143026322364 evalInputDelta=34.825007368827755
Fitness changed from -1261.1819852820606 to -1296.0069926508884
Iteration 14 complete. Error: -1296.0069926508884 Total: 0.3791; Orientation: 0.0297; Line Search: 0.2329
Adding measurement 7b184c6a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.704e+04, gradient 4.761e-01, dot 0.201; [ce227475-2c5a-44d8-9e65-0603a91bb4b3 = 1.000/1.000e+00]
th(0)=-1296.0069926508884;dx=-3479.6967040770796
Armijo: th(212.5850340136056)=-272.72083941275923; dx=1.6180521924947993 evalInputDelta=-1023.2861532381291
Armijo: th(106.2925170068028)=-275.3712125250205; dx=3.1713255361703823 evalInputDelta=-1020.6357801258679
Armijo: th(35.4308390022676)=-283.1726229263698; dx=9.740020583224666 evalInputDelta=-1012.8343697245186
Armijo: th(8.8577097505669)=-297.03051190071363; dx=39.13134022980327 evalInputDelta=-998.9764807501747
Armijo: th(1.7715419501133798)=-371.3692990137984; dx=186.82291809239277 evalInputDelta=-924.63769363709
Armijo: th(0.2952569916855633)=-881.4450138834526; dx=562.3956732339419 evalInputDelta=-414.5619787674358
New Minimum: -1296.0069926508884 > -1311.2729025645272
END: th(0.04217957024079476)=-1311.2729025645272; dx=-845.818531454524 evalInputDelta=15.26590991363878
Fitness changed from -1296.0069926508884 to -1311.2729025645272
Iteration 15 complete. Error: -1311.2729025645272 Total: 1.0733; Orientation: 0.0295; Line Search: 0.9202
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1311.2729025645272 (> -Infinity) after 11.036s (< 5400.000s)
Returns
-1311.2729025645272