BasicOptimizer.scala:89 executed in 43.81 seconds (1.526 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 214157172846600
Reset training subject: 214157997935900
Adding measurement 5bda157e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.459529975283067;dx=-5.448350592302893E-5
New Minimum: -3.459529975283067 > -3.459601421932049
WOLFE (weak): th(2.154434690031884)=-3.459601421932049; dx=-5.436594569152167E-5 evalInputDelta=7.14466489819543E-5
New Minimum: -3.459601421932049 > -3.4596729940985655
WOLFE (weak): th(4.308869380063768)=-3.4596729940985655; dx=-5.437747432483656E-5 evalInputDelta=1.4301881549849682E-4
New Minimum: -3.4596729940985655 > -3.4599587550013378
WOLFE (weak): th(12.926608140191302)=-3.4599587550013378; dx=-5.445294184320735E-5 evalInputDelta=4.2877971827071093E-4
New Minimum: -3.4599587550013378 > -3.4612458434807003
WOLFE (weak): th(51.70643256076521)=-3.4612458434807003; dx=-5.435296243077411E-5 evalInputDelta=0.0017158681976332524
New Minimum: -3.4612458434807003 > -3.4680886102021553
WOLFE (weak): th(258.53216280382605)=-3.4680886102021553; dx=-5.3922288138113795E-5 evalInputDelta=0.00855863491908826
New Minimum: -3.4680886102021553 > -3.5093792766112455
WOLFE (weak): th(1551.1929768229563)=-3.5093792766112455; dx=-5.174956316120503E-5 evalInputDelta=0.04984930132817844
New Minimum: -3.5093792766112455 > -3.7606040245004047
END: th(10858.350837760694)=-3.7606040245004047; dx=-4.1837121763002566E-5 evalInputDelta=0.3010740492173376
Fitness changed from -3.459529975283067 to -3.7606040245004047
Iteration 1 complete. Error: -3.7606040245004047 Total: 7.6512; Orientation: 0.0788; Line Search: 5.5422
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7e747037 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.7606040245004047;dx=-7.290192775042667E-5
New Minimum: -3.7606040245004047 > -4.4550350758292385
END: th(23393.607721408407)=-4.4550350758292385; dx=-4.2622032039814506E-5 evalInputDelta=0.6944310513288339
Fitness changed from -3.7606040245004047 to -4.4550350758292385
Iteration 2 complete. Error: -4.4550350758292385 Total: 1.8230; Orientation: 0.0135; Line Search: 1.2180
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4f8659d0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.4550350758292385;dx=-8.110000711576565E-5
New Minimum: -4.4550350758292385 > -5.770546765600421
END: th(50400.00000000001)=-5.770546765600421; dx=-3.1122751650316713E-5 evalInputDelta=1.3155116897711823
Fitness changed from -4.4550350758292385 to -5.770546765600421
Iteration 3 complete. Error: -5.770546765600421 Total: 1.8929; Orientation: 0.0125; Line Search: 1.3029
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7bc2bb58 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.793e+03, gradient 8.641e-03, dot -0.358; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.770546765600421, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 7bc2bb58 to history. Total: 3
th(0)=-5.770546765600421;dx=-7.452222335840964E-5
New Minimum: -5.770546765600421 > -7.140185956376888
END: th(108583.50837760697)=-7.140185956376888; dx=-1.6517969568668704E-5 evalInputDelta=1.369639190776467
Fitness changed from -5.770546765600421 to -7.140185956376888
Iteration 4 complete. Error: -7.140185956376888 Total: 2.6209; Orientation: 0.3645; Line Search: 1.1848
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1d585fb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.540e+03, gradient 7.683e-03, dot -0.271; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.140185956376888, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 1d585fb to history. Total: 3
th(0)=-7.140185956376888;dx=-5.886809769060691E-5
New Minimum: -7.140185956376888 > -8.532386286231878
END: th(233936.07721408413)=-8.532386286231878; dx=-9.577287975690213E-6 evalInputDelta=1.3922003298549903
Fitness changed from -7.140185956376888 to -8.532386286231878
Iteration 5 complete. Error: -8.532386286231878 Total: 2.0996; Orientation: 0.3164; Line Search: 1.1883
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6b3b2c34 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.122e+03, gradient 6.710e-03, dot -0.198; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.532386286231878, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 6b3b2c34 to history. Total: 3
th(0)=-8.532386286231878;dx=-4.4836221402986706E-5
New Minimum: -8.532386286231878 > -9.066910099038148
END: th(504000.0000000002)=-9.066910099038148; dx=-2.7982309307311143E-6 evalInputDelta=0.5345238128062704
Fitness changed from -8.532386286231878 to -9.066910099038148
Iteration 6 complete. Error: -9.066910099038148 Total: 2.0967; Orientation: 0.3092; Line Search: 1.2021
Adding measurement 4b9fa2f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.942e+03, gradient 7.246e-03, dot -0.083; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.066910099038148, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 4b9fa2f to history. Total: 3
th(0)=-9.066910099038148;dx=-5.221329998219361E-5
Armijo: th(1085835.0837760698)=1.739685622651301; dx=4.801649809634314E-6 evalInputDelta=-10.80659572168945
Armijo: th(542917.5418880349)=-8.987730705864443; dx=-3.2251012008864157E-6 evalInputDelta=-0.07917939317370504
New Minimum: -9.066910099038148 > -12.05617101721031
END: th(180972.5139626783)=-12.05617101721031; dx=-2.4095136328147443E-5 evalInputDelta=2.989260918172162
Fitness changed from -9.066910099038148 to -12.05617101721031
Iteration 7 complete. Error: -12.05617101721031 Total: 3.5427; Orientation: 0.2828; Line Search: 2.4568
Adding measurement 3e12c5de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.783e+03, gradient 6.449e-03, dot -0.150; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.05617101721031, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 3e12c5de to history. Total: 3
th(0)=-12.05617101721031;dx=-4.149462018790019E-5
New Minimum: -12.05617101721031 > -13.402270960906996
END: th(389893.4620234736)=-13.402270960906996; dx=-1.039529295849031E-5 evalInputDelta=1.3460999436966858
Fitness changed from -12.05617101721031 to -13.402270960906996
Iteration 8 complete. Error: -13.402270960906996 Total: 2.0823; Orientation: 0.3158; Line Search: 1.2017
Adding measurement 56dde9f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.086e+03, gradient 6.945e-03, dot -0.123; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.402270960906996, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 56dde9f7 to history. Total: 3
th(0)=-13.402270960906996;dx=-4.8008506541913915E-5
Armijo: th(840000.0000000003)=-5.891953716504844; dx=1.1771115403849793E-6 evalInputDelta=-7.510317244402152
New Minimum: -13.402270960906996 > -13.781158711639254
END: th(420000.0000000002)=-13.781158711639254; dx=-7.738066058335962E-6 evalInputDelta=0.37888775073225744
Fitness changed from -13.402270960906996 to -13.781158711639254
Iteration 9 complete. Error: -13.781158711639254 Total: 3.1041; Orientation: 0.3059; Line Search: 2.2420
Adding measurement 726d8fa0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.365e+03, gradient 7.148e-03, dot -0.115; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.781158711639254, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 726d8fa0 to history. Total: 3
th(0)=-13.781158711639254;dx=-5.0834557190131566E-5
Armijo: th(904862.5698133915)=-4.405660665841058; dx=2.8699386462342144E-6 evalInputDelta=-9.375498045798196
New Minimum: -13.781158711639254 > -14.315815387699526
END: th(452431.2849066958)=-14.315815387699526; dx=-6.621299987029278E-6 evalInputDelta=0.5346566760602727
Fitness changed from -13.781158711639254 to -14.315815387699526
Iteration 10 complete. Error: -14.315815387699526 Total: 2.6682; Orientation: 0.2973; Line Search: 1.7993
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 570b85dd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.400e+03, gradient 7.394e-03, dot -0.118; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.315815387699526, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 570b85dd to history. Total: 3
th(0)=-14.315815387699526;dx=-5.447910746162636E-5
Armijo: th(974733.6550586841)=-2.2460652375534096; dx=3.6401781625853392E-6 evalInputDelta=-12.069750150146117
New Minimum: -14.315815387699526 > -14.447024425118776
END: th(487366.82752934203)=-14.447024425118776; dx=-6.046278388342877E-6 evalInputDelta=0.13120903741925005
Fitness changed from -14.315815387699526 to -14.447024425118776
Iteration 11 complete. Error: -14.447024425118776 Total: 2.6974; Orientation: 0.3084; Line Search: 1.7915
Adding measurement 1e09c0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.095e+04, gradient 7.465e-03, dot -0.109; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.447024425118776, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 1e09c0b to history. Total: 3
th(0)=-14.447024425118776;dx=-5.560430632216085E-5
Armijo: th(1050000.0000000005)=-0.7063688517730604; dx=4.2050921898104895E-6 evalInputDelta=-13.740655573345716
New Minimum: -14.447024425118776 > -14.56448377366991
END: th(525000.0000000002)=-14.56448377366991; dx=-5.515223216543323E-6 evalInputDelta=0.11745934855113305
Fitness changed from -14.447024425118776 to -14.56448377366991
Iteration 12 complete. Error: -14.56448377366991 Total: 3.0296; Orientation: 0.3244; Line Search: 2.1401
Adding measurement 64c37088 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.500e+04, gradient 7.351e-03, dot -0.111; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.56448377366991, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 64c37088 to history. Total: 3
th(0)=-14.56448377366991;dx=-5.383953211298785E-5
Armijo: th(1131078.2122667395)=1.4569876874013659; dx=4.151963582676981E-6 evalInputDelta=-16.021471461071275
Armijo: th(565539.1061333697)=-14.244867569884228; dx=-6.390804553029144E-6 evalInputDelta=-0.31961620378568156
New Minimum: -14.56448377366991 > -17.730263905052396
END: th(188513.0353777899)=-17.730263905052396; dx=-3.028451658296262E-5 evalInputDelta=3.165780131382487
Fitness changed from -14.56448377366991 to -17.730263905052396
Iteration 13 complete. Error: -17.730263905052396 Total: 3.2319; Orientation: 0.2919; Line Search: 2.3696
Adding measurement 48224381 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.530e+04, gradient 6.319e-03, dot -0.163; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.730263905052396, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 48224381 to history. Total: 3
th(0)=-17.730263905052396;dx=-3.971491356526122E-5
New Minimum: -17.730263905052396 > -18.991239827584835
END: th(406139.02294111834)=-18.991239827584835; dx=-1.6151957010570066E-5 evalInputDelta=1.2609759225324382
Fitness changed from -17.730263905052396 to -18.991239827584835
Iteration 14 complete. Error: -18.991239827584835 Total: 2.0304; Orientation: 0.3035; Line Search: 1.1731
Adding measurement 762d2e4c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.412e+04, gradient 6.458e-03, dot -0.167; [08bf1d92-cb47-43c9-bc39-d7450d524f08 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.991239827584835, -4.4550350758292385, -3.7606040245004047, -3.459529975283067
LBFGS Accumulation History: 3 points
Removed measurement 762d2e4c to history. Total: 3
th(0)=-18.991239827584835;dx=-4.153031862966156E-5
Armijo: th(875000.0000000005)=-11.63127208546372; dx=-2.5645548131008336E-6 evalInputDelta=-7.359967742121114
New Minimum: -18.991239827584835 > -19.570212437069905
END: th(437500.00000000023)=-19.570212437069905; dx=-1.2979350874784367E-5 evalInputDelta=0.5789726094850707
Fitness changed from -18.991239827584835 to -19.570212437069905
Iteration 15 complete. Error: -19.570212437069905 Total: 3.2003; Orientation: 0.3324; Line Search: 2.2942
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -19.570212437069905 (> -Infinity) after 43.793s (< 5400.000s)
Returns
-19.570212437069905