BasicOptimizer.scala:89 executed in 59.17 seconds (1.364 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 207904778323600
Reset training subject: 207905525639300
Adding measurement 15d4cc5 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=10.880682933739754;dx=-7.041217677832479E-6
New Minimum: 10.880682933739754 > 10.880670214564743
WOLFE (weak): th(2.154434690031884)=10.880670214564743; dx=-7.029881496219393E-6 evalInputDelta=1.2719175011000061E-5
New Minimum: 10.880670214564743 > 10.880658952226234
WOLFE (weak): th(4.308869380063768)=10.880658952226234; dx=-7.040925909257394E-6 evalInputDelta=2.3981513519899522E-5
New Minimum: 10.880658952226234 > 10.880611373267206
WOLFE (weak): th(12.926608140191302)=10.880611373267206; dx=-7.050822615895375E-6 evalInputDelta=7.156047254852638E-5
New Minimum: 10.880611373267206 > 10.880396121652405
WOLFE (weak): th(51.70643256076521)=10.880396121652405; dx=-7.025126501438918E-6 evalInputDelta=2.8681208734937513E-4
New Minimum: 10.880396121652405 > 10.87925253261009
WOLFE (weak): th(258.53216280382605)=10.87925253261009; dx=-7.034639879007129E-6 evalInputDelta=0.0014304011296637498
New Minimum: 10.87925253261009 > 10.872098239740321
WOLFE (weak): th(1551.1929768229563)=10.872098239740321; dx=-7.053825185924833E-6 evalInputDelta=0.008584693999432957
New Minimum: 10.872098239740321 > 10.819999479402359
WOLFE (weak): th(10858.350837760694)=10.819999479402359; dx=-7.131809912784369E-6 evalInputDelta=0.060683454337395304
New Minimum: 10.819999479402359 > 10.400216146285107
WOLFE (weak): th(86866.80670208555)=10.400216146285107; dx=-6.825660079937531E-6 evalInputDelta=0.4804667874546471
New Minimum: 10.400216146285107 > 9.046755796052285
END: th(781801.26031877)=9.046755796052285; dx=-1.7155302042256854E-6 evalInputDelta=1.8339271376874695
Fitness changed from 10.880682933739754 to 9.046755796052285
Iteration 1 complete. Error: 9.046755796052285 Total: 9.7806; Orientation: 0.0168; Line Search: 7.6586
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4de22ca8 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=9.046755796052285;dx=-1.0538952452924554E-5
Armijo: th(1684339.7559414052)=9.772970838421696; dx=9.488870994911866E-7 evalInputDelta=-0.7262150423694109
New Minimum: 9.046755796052285 > 7.662844077294257
END: th(842169.8779707026)=7.662844077294257; dx=-1.0075430329987634E-6 evalInputDelta=1.3839117187580277
Fitness changed from 9.046755796052285 to 7.662844077294257
Iteration 2 complete. Error: 7.662844077294257 Total: 3.2704; Orientation: 0.0166; Line Search: 2.4802
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6efc96df to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=7.662844077294257;dx=-1.2976146734902594E-5
Armijo: th(1814400.0)=10.238158177860022; dx=1.7364682821544528E-6 evalInputDelta=-2.575314100565765
New Minimum: 7.662844077294257 > 6.56506743164395
END: th(907200.0)=6.56506743164395; dx=-5.385021654843715E-7 evalInputDelta=1.097776645650307
Fitness changed from 7.662844077294257 to 6.56506743164395
Iteration 3 complete. Error: 6.56506743164395 Total: 3.0150; Orientation: 0.0167; Line Search: 2.2502
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7ffabd53 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.275e+03, gradient 3.884e-03, dot -0.291; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 6.56506743164395, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 7ffabd53 to history. Total: 3
th(0)=6.56506743164395;dx=-1.506711847947096E-5
Armijo: th(1954503.150796925)=11.129504233185846; dx=2.5895503681443523E-6 evalInputDelta=-4.564436801541897
New Minimum: 6.56506743164395 > 5.868722360551524
WOLF (strong): th(977251.5753984625)=5.868722360551524; dx=1.4268253309792226E-8 evalInputDelta=0.6963450710924262
New Minimum: 5.868722360551524 > 4.4337335023308135
END: th(325750.5251328208)=4.4337335023308135; dx=-7.010467551162929E-6 evalInputDelta=2.1313339293131364
Fitness changed from 6.56506743164395 to 4.4337335023308135
Iteration 4 complete. Error: 4.4337335023308135 Total: 4.4351; Orientation: 0.3124; Line Search: 3.3425
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 519a39d8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.858e+03, gradient 3.549e-03, dot -0.289; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 4.4337335023308135, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 519a39d8 to history. Total: 3
th(0)=4.4337335023308135;dx=-1.2573476005532408E-5
New Minimum: 4.4337335023308135 > 3.0944796777735517
END: th(701808.2316422522)=3.0944796777735517; dx=-2.1053380117552407E-6 evalInputDelta=1.3392538245572618
Fitness changed from 4.4337335023308135 to 3.0944796777735517
Iteration 5 complete. Error: 3.0944796777735517 Total: 2.5624; Orientation: 0.3215; Line Search: 1.5583
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6eb06d55 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.305e+03, gradient 4.015e-03, dot -0.257; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 3.0944796777735517, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 6eb06d55 to history. Total: 3
th(0)=3.0944796777735517;dx=-1.60992617844844E-5
Armijo: th(1512000.0)=7.210723456683984; dx=2.265093412093724E-6 evalInputDelta=-4.116243778910432
New Minimum: 3.0944796777735517 > 2.6080214412073666
END: th(756000.0)=2.6080214412073666; dx=-5.224384320895555E-7 evalInputDelta=0.48645823656618514
Fitness changed from 3.0944796777735517 to 2.6080214412073666
Iteration 6 complete. Error: 2.6080214412073666 Total: 3.3157; Orientation: 0.3113; Line Search: 2.3007
Adding measurement 5088d842 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.493e+03, gradient 4.127e-03, dot -0.241; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.6080214412073666, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 5088d842 to history. Total: 3
th(0)=2.6080214412073666;dx=-1.7004223175015884E-5
Armijo: th(1628752.6256641042)=8.128870047761174; dx=2.9496348230066025E-6 evalInputDelta=-5.520848606553807
New Minimum: 2.6080214412073666 > 2.064565344812527
END: th(814376.3128320521)=2.064565344812527; dx=-2.5659112053111E-7 evalInputDelta=0.5434560963948396
Fitness changed from 2.6080214412073666 to 2.064565344812527
Iteration 7 complete. Error: 2.064565344812527 Total: 3.6326; Orientation: 0.3417; Line Search: 2.5680
Adding measurement ce13131 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.960e+03, gradient 4.227e-03, dot -0.226; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from 2.064565344812527, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement ce13131 to history. Total: 3
th(0)=2.064565344812527;dx=-1.784292051519972E-5
Armijo: th(1754520.5791056307)=9.197374328134455; dx=3.42290214119178E-6 evalInputDelta=-7.132808983321928
New Minimum: 2.064565344812527 > 1.6922137181378112
WOLF (strong): th(877260.2895528154)=1.6922137181378112; dx=5.659001617923263E-8 evalInputDelta=0.3723516266747158
New Minimum: 1.6922137181378112 > -0.10170486687374058
END: th(292420.0965176051)=-0.10170486687374058; dx=-8.798708227040493E-6 evalInputDelta=2.1662702116862675
Fitness changed from 2.064565344812527 to -0.10170486687374058
Iteration 8 complete. Error: -0.10170486687374058 Total: 4.2545; Orientation: 0.3072; Line Search: 3.2648
Adding measurement 557e98b2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.585e+03, gradient 3.512e-03, dot -0.231; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.10170486687374058, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 557e98b2 to history. Total: 3
th(0)=-0.10170486687374058;dx=-1.2309223488860404E-5
New Minimum: -0.10170486687374058 > -1.389749522431762
END: th(630000.0000000001)=-1.389749522431762; dx=-3.243762060058836E-6 evalInputDelta=1.2880446555580214
Fitness changed from -0.10170486687374058 to -1.389749522431762
Iteration 9 complete. Error: -1.389749522431762 Total: 2.9336; Orientation: 0.3580; Line Search: 1.8706
Adding measurement 2e197618 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.252e+04, gradient 3.747e-03, dot -0.214; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.389749522431762, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 2e197618 to history. Total: 3
th(0)=-1.389749522431762;dx=-1.402156917967035E-5
Armijo: th(1357293.854720087)=2.77576566432732; dx=2.053164446043955E-6 evalInputDelta=-4.165515186759082
New Minimum: -1.389749522431762 > -1.8283571223766568
END: th(678646.9273600435)=-1.8283571223766568; dx=-1.1303758532004935E-6 evalInputDelta=0.43860759994489484
Fitness changed from -1.389749522431762 to -1.8283571223766568
Iteration 10 complete. Error: -1.8283571223766568 Total: 3.6360; Orientation: 0.4325; Line Search: 2.4230
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 29d95a74 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.233e+04, gradient 4.025e-03, dot -0.198; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.8283571223766568, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 29d95a74 to history. Total: 3
th(0)=-1.8283571223766568;dx=-1.6179761330702836E-5
Armijo: th(1462100.4825880257)=4.036932351942042; dx=3.1727450749298744E-6 evalInputDelta=-5.865289474318699
New Minimum: -1.8283571223766568 > -1.965693330299949
WOLF (strong): th(731050.2412940129)=-1.965693330299949; dx=7.593407012581675E-8 evalInputDelta=0.13733620792329226
New Minimum: -1.965693330299949 > -3.307284918114588
END: th(243683.41376467096)=-3.307284918114588; dx=-7.865076284583614E-6 evalInputDelta=1.4789277957379312
Fitness changed from -1.8283571223766568 to -3.307284918114588
Iteration 11 complete. Error: -3.307284918114588 Total: 4.4627; Orientation: 0.3161; Line Search: 3.4355
Adding measurement 34f862e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.454e+04, gradient 3.183e-03, dot -0.223; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.307284918114588, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 34f862e3 to history. Total: 3
th(0)=-3.307284918114588;dx=-1.0108457239313122E-5
New Minimum: -3.307284918114588 > -4.237220504025856
END: th(525000.0000000001)=-4.237220504025856; dx=-3.5207760207323913E-6 evalInputDelta=0.9299355859112683
Fitness changed from -3.307284918114588 to -4.237220504025856
Iteration 12 complete. Error: -4.237220504025856 Total: 2.7497; Orientation: 0.3995; Line Search: 1.5974
Adding measurement 7a2f4fde to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.803e+04, gradient 3.369e-03, dot -0.213; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.237220504025856, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 7a2f4fde to history. Total: 3
th(0)=-4.237220504025856;dx=-1.1328337905486815E-5
Armijo: th(1131078.2122667392)=-1.7202828273737047; dx=1.213032306067641E-6 evalInputDelta=-2.5169376766521516
New Minimum: -4.237220504025856 > -4.6990124847370005
END: th(565539.1061333696)=-4.6990124847370005; dx=-1.8175070252153299E-6 evalInputDelta=0.46179198071114413
Fitness changed from -4.237220504025856 to -4.6990124847370005
Iteration 13 complete. Error: -4.6990124847370005 Total: 3.5128; Orientation: 0.3102; Line Search: 2.5290
Adding measurement e72ceb8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.976e+04, gradient 3.658e-03, dot -0.194; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.6990124847370005, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement e72ceb8 to history. Total: 3
th(0)=-4.6990124847370005;dx=-1.3371587658433659E-5
Armijo: th(1218417.0688233548)=-0.7776533045800083; dx=2.681875221450281E-6 evalInputDelta=-3.9213591801569923
New Minimum: -4.6990124847370005 > -4.925805981285844
END: th(609208.5344116774)=-4.925805981285844; dx=-1.2857916289780424E-7 evalInputDelta=0.22679349654884362
Fitness changed from -4.6990124847370005 to -4.925805981285844
Iteration 14 complete. Error: -4.925805981285844 Total: 3.2463; Orientation: 0.3221; Line Search: 2.2015
Adding measurement 700d49a3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.948e+04, gradient 3.885e-03, dot -0.180; [0c709c21-736f-45fb-9e0b-a46b245f82bd = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.925805981285844, 7.662844077294257, 9.046755796052285, 10.880682933739754
LBFGS Accumulation History: 3 points
Removed measurement 700d49a3 to history. Total: 3
th(0)=-4.925805981285844;dx=-1.506820830783013E-5
Armijo: th(1312500.0000000005)=0.48682052400617515; dx=3.5334179769969814E-6 evalInputDelta=-5.412626505292019
Armijo: th(656250.0000000002)=-4.916561171933768; dx=6.478483775406374E-7 evalInputDelta=-0.009244809352075833
New Minimum: -4.925805981285844 > -6.124113101063724
END: th(218750.0000000001)=-6.124113101063724; dx=-7.375207751388046E-6 evalInputDelta=1.1983071197778798
Fitness changed from -4.925805981285844 to -6.124113101063724
Iteration 15 complete. Error: -6.124113101063724 Total: 4.3378; Orientation: 0.3142; Line Search: 3.2843
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -6.124113101063724 (> -Infinity) after 59.168s (< 5400.000s)
Returns
-6.124113101063724