BasicOptimizer.scala:89 executed in 12.83 seconds (0.622 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 634972743142400
Reset training subject: 634973147346800
Adding measurement 6b8b5020 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.465451475622352;dx=-1.5274079840585247E-5
New Minimum: -1.465451475622352 > -1.4655193149494241
WOLFE (weak): th(2.154434690031884)=-1.4655193149494241; dx=-1.5288960744190682E-5 evalInputDelta=6.78393270721056E-5
New Minimum: -1.4655193149494241 > -1.465586962313454
WOLFE (weak): th(4.308869380063768)=-1.465586962313454; dx=-1.5366136673876482E-5 evalInputDelta=1.3548669110208245E-4
New Minimum: -1.465586962313454 > -1.4658578981272625
WOLFE (weak): th(12.926608140191302)=-1.4658578981272625; dx=-1.529387949999155E-5 evalInputDelta=4.064225049105197E-4
New Minimum: -1.4658578981272625 > -1.4670774804109916
WOLFE (weak): th(51.70643256076521)=-1.4670774804109916; dx=-1.5299475118258654E-5 evalInputDelta=0.0016260047886396123
New Minimum: -1.4670774804109916 > -1.4735936522796527
WOLFE (weak): th(258.53216280382605)=-1.4735936522796527; dx=-1.538033060302413E-5 evalInputDelta=0.00814217665730066
New Minimum: -1.4735936522796527 > -1.514717918716646
WOLFE (weak): th(1551.1929768229563)=-1.514717918716646; dx=-1.5633911630304243E-5 evalInputDelta=0.049266443094293955
New Minimum: -1.514717918716646 > -1.8274658999595585
WOLFE (weak): th(10858.350837760694)=-1.8274658999595585; dx=-1.7066177373360674E-5 evalInputDelta=0.36201442433720654
New Minimum: -1.8274658999595585 > -3.5471097511972425
END: th(86866.80670208555)=-3.5471097511972425; dx=-4.1449038013038574E-6 evalInputDelta=2.0816582755748905
Fitness changed from -1.465451475622352 to -3.5471097511972425
Iteration 1 complete. Error: -3.5471097511972425 Total: 2.6663; Orientation: 0.0278; Line Search: 1.8420
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 8ee03f5 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.5471097511972425;dx=-2.356791468484745E-5
New Minimum: -3.5471097511972425 > -4.62404193072116
WOLF (strong): th(187148.86177126726)=-4.62404193072116; dx=2.5515786565627428E-6 evalInputDelta=1.0769321795239173
New Minimum: -4.62404193072116 > -5.195446955863651
WOLF (strong): th(93574.43088563363)=-5.195446955863651; dx=1.4018397626345809E-6 evalInputDelta=1.6483372046664084
END: th(31191.476961877877)=-4.712042120333556; dx=-1.2496100342934877E-5 evalInputDelta=1.1649323691363134
Fitness changed from -3.5471097511972425 to -5.195446955863651
Iteration 2 complete. Error: -5.195446955863651 Total: 0.9063; Orientation: 0.0026; Line Search: 0.7122
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 342394b3 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-5.195446955863651;dx=-2.4768750431781817E-5
New Minimum: -5.195446955863651 > -6.933460966071418
END: th(67200.00000000001)=-6.933460966071418; dx=-1.1278351589242702E-6 evalInputDelta=1.7380140102077668
Fitness changed from -5.195446955863651 to -6.933460966071418
Iteration 3 complete. Error: -6.933460966071418 Total: 0.6288; Orientation: 0.0047; Line Search: 0.4119
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 29174dfe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.184e+03, gradient 3.567e-03, dot -0.556; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.933460966071418, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 29174dfe to history. Total: 3
th(0)=-6.933460966071418;dx=-1.2686021969020543E-5
New Minimum: -6.933460966071418 > -8.175034454308987
WOLF (strong): th(144778.01117014262)=-8.175034454308987; dx=7.323619646189546E-7 evalInputDelta=1.2415734882375693
END: th(72389.00558507131)=-8.08557783673297; dx=-2.8256193324104933E-6 evalInputDelta=1.152116870661552
Fitness changed from -6.933460966071418 to -8.175034454308987
Iteration 4 complete. Error: -8.175034454308987 Total: 1.0317; Orientation: 0.1064; Line Search: 0.6880
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 227fc322 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.276e+03, gradient 4.314e-03, dot -0.446; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.175034454308987, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 227fc322 to history. Total: 3
th(0)=-8.175034454308987;dx=-1.857928137715481E-5
New Minimum: -8.175034454308987 > -9.608767437137459
WOLF (strong): th(155957.38480938942)=-9.608767437137459; dx=5.016005809428077E-6 evalInputDelta=1.433732982828472
New Minimum: -9.608767437137459 > -9.974573240350312
END: th(77978.69240469471)=-9.974573240350312; dx=-3.200032198072743E-6 evalInputDelta=1.7995387860413246
Fitness changed from -8.175034454308987 to -9.974573240350312
Iteration 5 complete. Error: -9.974573240350312 Total: 0.7344; Orientation: 0.0492; Line Search: 0.4905
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 535aa7c7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.575e+03, gradient 2.813e-03, dot -0.640; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.974573240350312, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 535aa7c7 to history. Total: 3
th(0)=-9.974573240350312;dx=-7.875296500730958E-6
New Minimum: -9.974573240350312 > -11.472398470048333
END: th(168000.00000000006)=-11.472398470048333; dx=-2.0882936887732864E-6 evalInputDelta=1.4978252296980212
Fitness changed from -9.974573240350312 to -11.472398470048333
Iteration 6 complete. Error: -11.472398470048333 Total: 0.5438; Orientation: 0.0422; Line Search: 0.3369
Adding measurement 6c502018 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.430e+03, gradient 3.506e-03, dot -0.532; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.472398470048333, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 6c502018 to history. Total: 3
th(0)=-11.472398470048333;dx=-1.2228226397064672E-5
New Minimum: -11.472398470048333 > -11.522283770440982
WOLF (strong): th(361945.0279253566)=-11.522283770440982; dx=3.0239134952918574E-6 evalInputDelta=0.04988530039264916
New Minimum: -11.522283770440982 > -12.77513236436038
WOLF (strong): th(180972.5139626783)=-12.77513236436038; dx=1.725604602159728E-6 evalInputDelta=1.3027338943120466
END: th(60324.17132089276)=-12.60436242742387; dx=-5.811564278239223E-6 evalInputDelta=1.1319639573755378
Fitness changed from -11.472398470048333 to -12.77513236436038
Iteration 7 complete. Error: -12.77513236436038 Total: 0.8700; Orientation: 0.0499; Line Search: 0.6469
Adding measurement 41acf2f1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.576e+03, gradient 4.031e-03, dot -0.459; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.77513236436038, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 41acf2f1 to history. Total: 3
th(0)=-12.77513236436038;dx=-1.6217184797317162E-5
New Minimum: -12.77513236436038 > -15.116536728462544
END: th(129964.48734115784)=-15.116536728462544; dx=-8.129577943726245E-7 evalInputDelta=2.3414043641021642
Fitness changed from -12.77513236436038 to -15.116536728462544
Iteration 8 complete. Error: -15.116536728462544 Total: 0.5033; Orientation: 0.0410; Line Search: 0.3113
Adding measurement ca90087 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.308e+03, gradient 3.060e-03, dot -0.631; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.116536728462544, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement ca90087 to history. Total: 3
th(0)=-15.116536728462544;dx=-9.320228655205146E-6
New Minimum: -15.116536728462544 > -17.084268126151727
END: th(280000.0000000001)=-17.084268126151727; dx=-6.450800684817384E-8 evalInputDelta=1.9677313976891835
Fitness changed from -15.116536728462544 to -17.084268126151727
Iteration 9 complete. Error: -17.084268126151727 Total: 0.7420; Orientation: 0.0430; Line Search: 0.5384
Adding measurement 7734ddb8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.936e+03, gradient 4.069e-03, dot -0.472; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.084268126151727, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 7734ddb8 to history. Total: 3
th(0)=-17.084268126151727;dx=-1.651048285631452E-5
Armijo: th(603241.7132089278)=-16.435880983989385; dx=3.504029431589357E-6 evalInputDelta=-0.648387142162342
New Minimum: -17.084268126151727 > -19.29447744970762
WOLF (strong): th(301620.8566044639)=-19.29447744970762; dx=4.437705336857216E-6 evalInputDelta=2.210209323555894
New Minimum: -19.29447744970762 > -19.52880750772638
END: th(100540.28553482129)=-19.52880750772638; dx=-7.250454824979828E-6 evalInputDelta=2.4445393815746534
Fitness changed from -17.084268126151727 to -19.52880750772638
Iteration 10 complete. Error: -19.52880750772638 Total: 0.8945; Orientation: 0.0414; Line Search: 0.6839
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 64a009e7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.854e+04, gradient 2.724e-03, dot -0.736; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.52880750772638, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 64a009e7 to history. Total: 3
th(0)=-19.52880750772638;dx=-7.390860560432297E-6
New Minimum: -19.52880750772638 > -21.891199631026005
END: th(216607.4789019298)=-21.891199631026005; dx=-3.7930820655880723E-6 evalInputDelta=2.3623921232996246
Fitness changed from -19.52880750772638 to -21.891199631026005
Iteration 11 complete. Error: -21.891199631026005 Total: 0.5429; Orientation: 0.0403; Line Search: 0.3242
Adding measurement 3a837f7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.774e+04, gradient 3.034e-03, dot -0.659; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.891199631026005, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 3a837f7 to history. Total: 3
th(0)=-21.891199631026005;dx=-9.178274780695015E-6
New Minimum: -21.891199631026005 > -24.99951952250697
END: th(466666.666666667)=-24.99951952250697; dx=-1.1175965035760995E-6 evalInputDelta=3.1083198914809635
Fitness changed from -21.891199631026005 to -24.99951952250697
Iteration 12 complete. Error: -24.99951952250697 Total: 0.5435; Orientation: 0.0461; Line Search: 0.3438
Adding measurement 6ccb3dd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.410e+04, gradient 3.944e-03, dot -0.485; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.99951952250697, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 6ccb3dd to history. Total: 3
th(0)=-24.99951952250697;dx=-1.551585982531837E-5
New Minimum: -24.99951952250697 > -25.197399542965123
WOLF (strong): th(1005402.8553482131)=-25.197399542965123; dx=2.767437544560069E-6 evalInputDelta=0.1978800204581539
New Minimum: -25.197399542965123 > -28.905869927329938
WOLF (strong): th(502701.42767410656)=-28.905869927329938; dx=3.6697034996238274E-6 evalInputDelta=3.9063504048229696
END: th(167567.14255803553)=-28.879383501268975; dx=-7.662985295960833E-6 evalInputDelta=3.879863978762007
Fitness changed from -24.99951952250697 to -28.905869927329938
Iteration 13 complete. Error: -28.905869927329938 Total: 0.8125; Orientation: 0.0405; Line Search: 0.6220
Adding measurement 6f83db89 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.131e+04, gradient 4.262e-03, dot -0.392; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.905869927329938, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 6f83db89 to history. Total: 3
th(0)=-28.905869927329938;dx=-1.812487554341688E-5
New Minimum: -28.905869927329938 > -35.556910122485796
WOLF (strong): th(361012.46483654977)=-35.556910122485796; dx=8.020016824976182E-7 evalInputDelta=6.651040195155858
END: th(180506.23241827489)=-34.25660276488714; dx=-9.305124645264464E-6 evalInputDelta=5.350732837557203
Fitness changed from -28.905869927329938 to -35.556910122485796
Iteration 14 complete. Error: -35.556910122485796 Total: 0.8733; Orientation: 0.0416; Line Search: 0.6833
Adding measurement 645b2ac7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.866e+04, gradient 3.174e-03, dot -0.539; [aaeb3f4c-6d2a-4f40-b05e-52b24ac8ba3a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.556910122485796, -5.195446955863651, -3.5471097511972425, -1.465451475622352
LBFGS Accumulation History: 3 points
Removed measurement 645b2ac7 to history. Total: 3
th(0)=-35.556910122485796;dx=-1.0049627044685908E-5
New Minimum: -35.556910122485796 > -39.70310016623587
END: th(388888.8888888892)=-39.70310016623587; dx=-5.813556912851892E-7 evalInputDelta=4.1461900437500745
Fitness changed from -35.556910122485796 to -39.70310016623587
Iteration 15 complete. Error: -39.70310016623587 Total: 0.5060; Orientation: 0.0405; Line Search: 0.3126
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -39.70310016623587 (> -Infinity) after 12.819s (< 5400.000s)
Returns
-39.70310016623587