BasicOptimizer.scala:89 executed in 198.04 seconds (1.793 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 7965213227002
Reset training subject: 7974637272023
Adding measurement 477edcba to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-13.653117888394432;dx=-4.174016445622057E-4
New Minimum: -13.653117888394432 > -13.653166571487583
WOLFE (weak): th(2.154434690031884)=-13.653166571487583; dx=-4.218918300004962E-4 evalInputDelta=4.868309315142483E-5
New Minimum: -13.653166571487583 > -13.653229059979608
WOLFE (weak): th(4.308869380063768)=-13.653229059979608; dx=-4.202872994259274E-4 evalInputDelta=1.1117158517670589E-4
New Minimum: -13.653229059979608 > -13.653449947240526
WOLFE (weak): th(12.926608140191302)=-13.653449947240526; dx=-4.214553662321154E-4 evalInputDelta=3.320588460944407E-4
New Minimum: -13.653449947240526 > -13.654467145085519
WOLFE (weak): th(51.70643256076521)=-13.654467145085519; dx=-4.1761332759608047E-4 evalInputDelta=0.0013492566910873194
New Minimum: -13.654467145085519 > -13.660078535399201
WOLFE (weak): th(258.53216280382605)=-13.660078535399201; dx=-4.221963588429321E-4 evalInputDelta=0.0069606470047691715
New Minimum: -13.660078535399201 > -13.696801830066619
WOLFE (weak): th(1551.1929768229563)=-13.696801830066619; dx=-4.0189951148172454E-4 evalInputDelta=0.04368394167218703
New Minimum: -13.696801830066619 > -14.03475926794875
END: th(10858.350837760694)=-14.03475926794875; dx=-3.2186665474682406E-4 evalInputDelta=0.38164137955431876
Fitness changed from -13.653117888394432 to -14.03475926794875
Iteration 1 complete. Error: -14.03475926794875 Total: 36.6063; Orientation: 0.0158; Line Search: 24.5464
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 461c05f6 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-14.03475926794875;dx=-4.4394487601685437E-4
New Minimum: -14.03475926794875 > -15.047696808599152
END: th(23393.607721408407)=-15.047696808599152; dx=-2.3794452508276318E-4 evalInputDelta=1.0129375406504018
Fitness changed from -14.03475926794875 to -15.047696808599152
Iteration 2 complete. Error: -15.047696808599152 Total: 4.0910; Orientation: 0.0173; Line Search: 2.5737
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 799daa67 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-15.047696808599152;dx=-5.092844172448323E-4
New Minimum: -15.047696808599152 > -16.63993021345801
END: th(50400.00000000001)=-16.63993021345801; dx=-1.1470464302940482E-4 evalInputDelta=1.592233404858856
Fitness changed from -15.047696808599152 to -16.63993021345801
Iteration 3 complete. Error: -16.63993021345801 Total: 3.9589; Orientation: 0.0172; Line Search: 2.4638
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 29a0f7ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.111e+03, gradient 2.142e-02, dot -0.233; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.63993021345801, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 29a0f7ea to history. Total: 3
th(0)=-16.63993021345801;dx=-4.5015481830231103E-4
New Minimum: -16.63993021345801 > -19.58245012745893
END: th(108583.50837760697)=-19.58245012745893; dx=-3.2180506941988706E-5 evalInputDelta=2.942519914000922
Fitness changed from -16.63993021345801 to -19.58245012745893
Iteration 4 complete. Error: -19.58245012745893 Total: 4.5739; Orientation: 0.5746; Line Search: 2.4975
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7cca621 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.123e+03, gradient 2.054e-02, dot -0.124; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.58245012745893, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 7cca621 to history. Total: 3
th(0)=-19.58245012745893;dx=-4.12001323837125E-4
New Minimum: -19.58245012745893 > -22.169286821504514
END: th(233936.07721408413)=-22.169286821504514; dx=-9.722235627079906E-6 evalInputDelta=2.5868366940455836
Fitness changed from -19.58245012745893 to -22.169286821504514
Iteration 5 complete. Error: -22.169286821504514 Total: 4.5766; Orientation: 0.4825; Line Search: 2.5888
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 41f9661b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.396e+03, gradient 1.825e-02, dot -0.103; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.169286821504514, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 41f9661b to history. Total: 3
th(0)=-22.169286821504514;dx=-3.2247189613448016E-4
New Minimum: -22.169286821504514 > -22.251145568128745
END: th(504000.0000000002)=-22.251145568128745; dx=-3.094626146754891E-6 evalInputDelta=0.08185874662423132
Fitness changed from -22.169286821504514 to -22.251145568128745
Iteration 6 complete. Error: -22.251145568128745 Total: 4.4300; Orientation: 0.4953; Line Search: 2.4341
Adding measurement 30d5e496 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.010e+02, gradient 1.082e-02, dot -0.094; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.251145568128745, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 30d5e496 to history. Total: 3
th(0)=-22.251145568128745;dx=-1.1553472236732097E-4
Armijo: th(1085835.0837760698)=-19.70379901471901; dx=-6.909284381668996E-7 evalInputDelta=-2.547346553409735
Armijo: th(542917.5418880349)=-21.443416521056143; dx=-9.618235190787588E-6 evalInputDelta=-0.807729047072602
New Minimum: -22.251145568128745 > -22.812080438483093
END: th(180972.5139626783)=-22.812080438483093; dx=-5.0608189819879196E-5 evalInputDelta=0.5609348703543482
Fitness changed from -22.251145568128745 to -22.812080438483093
Iteration 7 complete. Error: -22.812080438483093 Total: 6.7633; Orientation: 0.4755; Line Search: 5.0964
Adding measurement 4296bc67 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.050e+03, gradient 1.102e-02, dot -0.096; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.812080438483093, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 4296bc67 to history. Total: 3
th(0)=-22.812080438483093;dx=-1.1927372266436736E-4
New Minimum: -22.812080438483093 > -22.83308251237651
END: th(389893.4620234736)=-22.83308251237651; dx=-1.942820233903672E-5 evalInputDelta=0.021002073893416906
Fitness changed from -22.812080438483093 to -22.83308251237651
Iteration 8 complete. Error: -22.83308251237651 Total: 4.2882; Orientation: 0.6625; Line Search: 2.4794
Adding measurement 26f8496f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.935e+03, gradient 1.029e-02, dot -0.089; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.83308251237651, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 26f8496f to history. Total: 3
th(0)=-22.83308251237651;dx=-1.0396272731867561E-4
Armijo: th(840000.0000000003)=-21.92577086466051; dx=-6.419551315632382E-6 evalInputDelta=-0.9073116477159999
New Minimum: -22.83308251237651 > -23.181514690233072
END: th(420000.0000000002)=-23.181514690233072; dx=-2.0568600255465033E-5 evalInputDelta=0.3484321778565622
Fitness changed from -22.83308251237651 to -23.181514690233072
Iteration 9 complete. Error: -23.181514690233072 Total: 5.4894; Orientation: 0.4765; Line Search: 3.8088
Adding measurement 4cb34923 to history
...skipping 10519 bytes...
emoved measurement 33f38710 to history. Total: 3
th(0)=-35.50429695024438;dx=-2.8410421412485603E-5
New Minimum: -35.50429695024438 > -35.537140249853636
END: th(405092.592592593)=-35.537140249853636; dx=-1.8463439167647793E-5 evalInputDelta=0.03284329960925447
Fitness changed from -35.50429695024438 to -35.537140249853636
Iteration 21 complete. Error: -35.537140249853636 Total: 4.5286; Orientation: 0.4836; Line Search: 2.4583
Adding measurement 347fcd58 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.972e+06, gradient 5.560e-03, dot -0.153; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.537140249853636, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 347fcd58 to history. Total: 3
th(0)=-35.537140249853636;dx=-2.9970335084552634E-5
Armijo: th(872745.5341564353)=-35.282526223816; dx=-1.3963247447754263E-5 evalInputDelta=-0.25461402603763617
New Minimum: -35.537140249853636 > -35.740284039816935
END: th(436372.76707821764)=-35.740284039816935; dx=-1.790579355608093E-5 evalInputDelta=0.20314378996329907
Fitness changed from -35.537140249853636 to -35.740284039816935
Iteration 22 complete. Error: -35.740284039816935 Total: 5.4860; Orientation: 0.4703; Line Search: 3.8363
Adding measurement 516a0983 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.029e+08, gradient 5.487e-03, dot -0.159; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.740284039816935, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 516a0983 to history. Total: 3
th(0)=-35.740284039816935;dx=-2.890970730022201E-5
Armijo: th(940136.6271785153)=-35.08823487997325; dx=-1.2974194942366579E-5 evalInputDelta=-0.6520491598436848
Armijo: th(470068.31358925765)=-35.46560099832695; dx=-1.7794575916334214E-5 evalInputDelta=-0.2746830414899861
Armijo: th(156689.43786308588)=-35.70930784928829; dx=-2.19178725179802E-5 evalInputDelta=-0.030976190528647862
New Minimum: -35.740284039816935 > -35.742924221277
END: th(39172.35946577147)=-35.742924221277; dx=-2.3817109780584728E-5 evalInputDelta=0.00264018146006606
Fitness changed from -35.740284039816935 to -35.742924221277
Iteration 23 complete. Error: -35.742924221277 Total: 8.0890; Orientation: 0.5974; Line Search: 6.3056
Adding measurement 2950e805 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.754e+07, gradient 5.520e-03, dot -0.160; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.742924221277, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 2950e805 to history. Total: 3
th(0)=-35.742924221277;dx=-2.983292210665693E-5
Armijo: th(84394.29012345689)=-35.73355391461535; dx=-2.3485223287549525E-5 evalInputDelta=-0.009370306661651284
New Minimum: -35.742924221277 > -35.74411656899344
END: th(42197.14506172844)=-35.74411656899344; dx=-2.4277369674863625E-5 evalInputDelta=0.0011923477164401675
Fitness changed from -35.742924221277 to -35.74411656899344
Iteration 24 complete. Error: -35.74411656899344 Total: 5.3930; Orientation: 0.4747; Line Search: 3.7099
Adding measurement e1b0171 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.844e+06, gradient 5.559e-03, dot -0.161; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.74411656899344, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement e1b0171 to history. Total: 3
th(0)=-35.74411656899344;dx=-2.9950386840146937E-5
Armijo: th(90910.99314129536)=-35.70098467514793; dx=-2.3098012885274067E-5 evalInputDelta=-0.04313189384551208
Armijo: th(45455.49657064768)=-35.72746450754182; dx=-2.4013542179046446E-5 evalInputDelta=-0.016652061451623013
Armijo: th(15151.832190215893)=-35.740080855449314; dx=-2.443250380969836E-5 evalInputDelta=-0.0040357135441269065
Armijo: th(3787.9580475539733)=-35.743410490791064; dx=-2.4955999527940193E-5 evalInputDelta=-7.060782023771139E-4
Armijo: th(757.5916095107947)=-35.74398667829558; dx=-2.4775340086123568E-5 evalInputDelta=-1.2989069786328855E-4
Armijo: th(126.26526825179911)=-35.74410435706759; dx=-2.4916107336067612E-5 evalInputDelta=-1.2211925849214822E-5
Armijo: th(18.037895464542732)=-35.74411656899344; dx=-2.490684358993647E-5 evalInputDelta=0.0
Armijo: th(2.2547369330678415)=-35.74411656899344; dx=-2.47389158798326E-5 evalInputDelta=0.0
Armijo: th(0.2505263258964268)=-35.74411323846863; dx=-2.5018799361826294E-5 evalInputDelta=-3.3305248123838282E-6
New Minimum: -35.74411656899344 > -35.74411767916831
END: th(0.02505263258964268)=-35.74411767916831; dx=-2.494747358778857E-5 evalInputDelta=1.110174871143954E-6
Fitness changed from -35.74411656899344 to -35.74411767916831
Iteration 25 complete. Error: -35.74411767916831 Total: 15.8203; Orientation: 0.5005; Line Search: 14.0672
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 49305f5c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.444e+05, gradient 5.553e-03, dot -0.163; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.74411767916831, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 49305f5c to history. Total: 3
th(0)=-35.74411767916831;dx=-3.0434976257450216E-5
Armijo: th(0.053974260727749496)=-35.74411767916831; dx=-2.5265911539909665E-5 evalInputDelta=0.0
Armijo: th(0.026987130363874748)=-35.74411656899344; dx=-2.4867596296477864E-5 evalInputDelta=-1.110174871143954E-6
Armijo: th(0.008995710121291583)=-35.74411656899344; dx=-2.526778969399396E-5 evalInputDelta=-1.110174871143954E-6
Armijo: th(0.002248927530322896)=-35.74411767916831; dx=-2.5354676679852446E-5 evalInputDelta=0.0
Armijo: th(4.4978550606457915E-4)=-35.74411767916831; dx=-2.5353117948073274E-5 evalInputDelta=0.0
New Minimum: -35.74411767916831 > -35.74411878934315
END: th(7.49642510107632E-5)=-35.74411878934315; dx=-2.505164499567188E-5 evalInputDelta=1.1101748356168173E-6
Fitness changed from -35.74411767916831 to -35.74411878934315
Iteration 26 complete. Error: -35.74411878934315 Total: 10.8902; Orientation: 0.4881; Line Search: 8.8560
Adding measurement 613326f9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.440e+05, gradient 5.551e-03, dot -0.163; [1ad9740c-12d2-4650-b09d-18626a1e2990 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.74411878934315, -15.047696808599152, -14.03475926794875, -13.653117888394432
LBFGS Accumulation History: 3 points
Removed measurement 613326f9 to history. Total: 3
th(0)=-35.74411878934315;dx=-2.994925610669051E-5
Armijo: th(1.6150558288984595E-4)=-35.74411767916831; dx=-2.5016454381934272E-5 evalInputDelta=-1.1101748356168173E-6
Armijo: th(8.075279144492297E-5)=-35.74411656899344; dx=-2.5055378679537905E-5 evalInputDelta=-2.2203497067607714E-6
END: th(2.691759714830766E-5)=-35.74411878934315; dx=-2.4953825064514617E-5 evalInputDelta=0.0
Fitness changed from -35.74411878934315 to -35.74411878934315
Static Iteration Total: 6.6905; Orientation: 0.4774; Line Search: 5.0108
Iteration 27 failed. Error: -35.74411878934315
Previous Error: 0.0 -> -35.74411878934315
Retrying iteration 27
Reset training subject: 8158219339857
Adding measurement 5b88906c to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-35.74411878934315;dx=-3.001074697143848E-5
END: th(5.799220506861733E-5)=-35.74411878934315; dx=-2.4791044924695346E-5 evalInputDelta=0.0
Fitness changed from -35.74411878934315 to -35.74411878934315
Static Iteration Total: 5.0370; Orientation: 0.0189; Line Search: 2.5175
Iteration 28 failed. Error: -35.74411878934315
Previous Error: 0.0 -> -35.74411878934315
Optimization terminated 28
Final threshold in iteration 28: -35.74411878934315 (> -Infinity) after 198.044s (< 3600.000s)
Returns
-35.74411878934315