BasicOptimizer.scala:89 executed in 8.06 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 142426699475400
Reset training subject: 142426828182800
Adding measurement 236d4f84 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.2507924125596435;dx=-1.3713546006435996E-4
New Minimum: -1.2507924125596435 > -1.2511873190417073
WOLFE (weak): th(2.154434690031884)=-1.2511873190417073; dx=-1.4157156387755126E-4 evalInputDelta=3.949064820638615E-4
New Minimum: -1.2511873190417073 > -1.2515824128532254
WOLFE (weak): th(4.308869380063768)=-1.2515824128532254; dx=-1.4080341305352303E-4 evalInputDelta=7.900002935818939E-4
New Minimum: -1.2515824128532254 > -1.2531677732326574
WOLFE (weak): th(12.926608140191302)=-1.2531677732326574; dx=-1.4766011757699776E-4 evalInputDelta=0.0023753606730139154
New Minimum: -1.2531677732326574 > -1.2602990241134773
WOLFE (weak): th(51.70643256076521)=-1.2602990241134773; dx=-1.3973270212059273E-4 evalInputDelta=0.00950661155383381
New Minimum: -1.2602990241134773 > -1.298652985497596
WOLFE (weak): th(258.53216280382605)=-1.298652985497596; dx=-1.4306000259403976E-4 evalInputDelta=0.04786057293795243
New Minimum: -1.298652985497596 > -1.5422792920437687
WOLFE (weak): th(1551.1929768229563)=-1.5422792920437687; dx=-1.3671639533631273E-4 evalInputDelta=0.2914868794841252
New Minimum: -1.5422792920437687 > -2.769938452873265
END: th(10858.350837760694)=-2.769938452873265; dx=-5.301123165120769E-5 evalInputDelta=1.5191460403136214
Fitness changed from -1.2507924125596435 to -2.769938452873265
Iteration 1 complete. Error: -2.769938452873265 Total: 1.2603; Orientation: 0.0016; Line Search: 0.9083
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2ad8fd16 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.769938452873265;dx=-1.251978910240769E-4
New Minimum: -2.769938452873265 > -4.720699752661243
END: th(23393.607721408407)=-4.720699752661243; dx=-1.9612581431887094E-5 evalInputDelta=1.9507612997879784
Fitness changed from -2.769938452873265 to -4.720699752661243
Iteration 2 complete. Error: -4.720699752661243 Total: 0.3586; Orientation: 0.0015; Line Search: 0.2321
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 391bdb5d to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.720699752661243;dx=-1.0238046968501363E-4
New Minimum: -4.720699752661243 > -6.2144729386135875
END: th(50400.00000000001)=-6.2144729386135875; dx=-2.2912953627518844E-6 evalInputDelta=1.4937731859523442
Fitness changed from -4.720699752661243 to -6.2144729386135875
Iteration 3 complete. Error: -6.2144729386135875 Total: 0.3592; Orientation: 0.0015; Line Search: 0.2321
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6718e78f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.675e+02, gradient 1.086e-02, dot -0.346; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.2144729386135875, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 6718e78f to history. Total: 3
th(0)=-6.2144729386135875;dx=-1.1379127635306627E-4
New Minimum: -6.2144729386135875 > -8.031498707182193
WOLF (strong): th(108583.50837760697)=-8.031498707182193; dx=3.3631576758140937E-6 evalInputDelta=1.8170257685686053
New Minimum: -8.031498707182193 > -9.091573603988907
WOLF (strong): th(54291.75418880348)=-9.091573603988907; dx=6.574509686090392E-8 evalInputDelta=2.8771006653753197
END: th(18097.251396267828)=-8.69735557415648; dx=-4.7636088334708454E-5 evalInputDelta=2.482882635542893
Fitness changed from -6.2144729386135875 to -9.091573603988907
Iteration 4 complete. Error: -9.091573603988907 Total: 0.5992; Orientation: 0.0286; Line Search: 0.4522
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3c1150af to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.106e+02, gradient 1.077e-02, dot -0.277; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.091573603988907, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 3c1150af to history. Total: 3
th(0)=-9.091573603988907;dx=-1.1338075015149016E-4
New Minimum: -9.091573603988907 > -12.392280995982356
END: th(38989.346202347355)=-12.392280995982356; dx=-2.3964021379346803E-5 evalInputDelta=3.3007073919934484
Fitness changed from -9.091573603988907 to -12.392280995982356
Iteration 5 complete. Error: -12.392280995982356 Total: 0.3728; Orientation: 0.0268; Line Search: 0.2259
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 256f1f57 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.911e+03, gradient 8.892e-03, dot -0.380; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.392280995982356, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 256f1f57 to history. Total: 3
th(0)=-12.392280995982356;dx=-7.694969350944224E-5
New Minimum: -12.392280995982356 > -14.955618956352836
END: th(84000.00000000003)=-14.955618956352836; dx=-5.268111667090679E-6 evalInputDelta=2.5633379603704807
Fitness changed from -12.392280995982356 to -14.955618956352836
Iteration 6 complete. Error: -14.955618956352836 Total: 0.3747; Orientation: 0.0286; Line Search: 0.2278
Adding measurement 751a921b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.270e+03, gradient 1.255e-02, dot -0.298; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.955618956352836, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 751a921b to history. Total: 3
th(0)=-14.955618956352836;dx=-1.5420590187289785E-4
New Minimum: -14.955618956352836 > -16.107398826901136
WOLF (strong): th(180972.5139626783)=-16.107398826901136; dx=4.630943162944474E-6 evalInputDelta=1.1517798705482996
New Minimum: -16.107398826901136 > -18.245342719778197
WOLF (strong): th(90486.25698133915)=-18.245342719778197; dx=5.805085038514442E-6 evalInputDelta=3.2897237634253607
New Minimum: -18.245342719778197 > -18.74886669403446
END: th(30162.08566044638)=-18.74886669403446; dx=-2.2773734125598877E-5 evalInputDelta=3.793247737681625
Fitness changed from -14.955618956352836 to -18.74886669403446
Iteration 7 complete. Error: -18.74886669403446 Total: 0.5935; Orientation: 0.0280; Line Search: 0.4552
Adding measurement 3de46dd0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.897e+04, gradient 9.784e-03, dot -0.456; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.74886669403446, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 3de46dd0 to history. Total: 3
th(0)=-18.74886669403446;dx=-9.301556908737393E-5
New Minimum: -18.74886669403446 > -21.543590360811272
END: th(64982.24367057892)=-21.543590360811272; dx=-1.1863796375414742E-5 evalInputDelta=2.794723666776811
Fitness changed from -18.74886669403446 to -21.543590360811272
Iteration 8 complete. Error: -21.543590360811272 Total: 0.3644; Orientation: 0.0279; Line Search: 0.2287
Adding measurement 573173f4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.101e+04, gradient 1.141e-02, dot -0.378; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.543590360811272, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 573173f4 to history. Total: 3
th(0)=-21.543590360811272;dx=-1.2746713469494541E-4
New Minimum: -21.543590360811272 > -24.434075411534756
WOLF (strong): th(140000.00000000006)=-24.434075411534756; dx=4.179092136685519E-6 evalInputDelta=2.890485050723484
New Minimum: -24.434075411534756 > -25.591088918526914
END: th(70000.00000000003)=-25.591088918526914; dx=-7.761177739971406E-6 evalInputDelta=4.047498557715642
Fitness changed from -21.543590360811272 to -25.591088918526914
Iteration 9 complete. Error: -25.591088918526914 Total: 0.4810; Orientation: 0.0278; Line Search: 0.3418
Adding measurement 7a80969c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.061e+05, gradient 9.873e-03, dot -0.431; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.591088918526914, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 7a80969c to history. Total: 3
th(0)=-25.591088918526914;dx=-9.308629070172221E-5
New Minimum: -25.591088918526914 > -26.965881460208102
WOLF (strong): th(150810.42830223194)=-26.965881460208102; dx=1.0381293866170211E-5 evalInputDelta=1.3747925416811881
New Minimum: -26.965881460208102 > -29.77950329818291
END: th(75405.21415111597)=-29.77950329818291; dx=-1.7806182444489013E-5 evalInputDelta=4.188414379655995
Fitness changed from -25.591088918526914 to -29.77950329818291
Iteration 10 complete. Error: -29.77950329818291 Total: 0.4823; Orientation: 0.0265; Line Search: 0.3424
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 149b5023 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.164e+03, gradient 9.876e-03, dot -0.349; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.77950329818291, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 149b5023 to history. Total: 3
th(0)=-29.77950329818291;dx=-9.471148618818591E-5
New Minimum: -29.77950329818291 > -32.147309483895455
END: th(162455.60917644735)=-32.147309483895455; dx=-5.361616012371344E-6 evalInputDelta=2.3678061857125456
Fitness changed from -29.77950329818291 to -32.147309483895455
Iteration 11 complete. Error: -32.147309483895455 Total: 0.3797; Orientation: 0.0272; Line Search: 0.2298
Adding measurement 1ac9c3e0 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 9.037e+03, gradient 1.144e-02, dot 0.451; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-32.147309483895455;dx=-46.59816728804993
New Minimum: -32.147309483895455 > -36.04942334810833
Armijo: th(350000.00000000023)=-36.04942334810833; dx=3.772963534917833E-5 evalInputDelta=3.902113864212872
Armijo: th(175000.00000000012)=-36.0492802900157; dx=7.331913403367426E-5 evalInputDelta=3.9019708061202465
WOLF (strong): th(58333.33333333337)=-36.043340105532; dx=2.2576847705011534E-4 evalInputDelta=3.896030621636548
WOLF (strong): th(14583.333333333343)=-36.03937130385142; dx=9.032509053198915E-4 evalInputDelta=3.8920618199559627
New Minimum: -36.04942334810833 > -36.071623741867796
WOLF (strong): th(2916.666666666669)=-36.071623741867796; dx=0.004532610886485351 evalInputDelta=3.924314257972341
New Minimum: -36.071623741867796 > -36.27798716580705
WOLF (strong): th(486.1111111111115)=-36.27798716580705; dx=0.027221094225513186 evalInputDelta=4.1306776819115925
New Minimum: -36.27798716580705 > -37.239015683642656
WOLF (strong): th(69.4444444444445)=-37.239015683642656; dx=0.19024944477172007 evalInputDelta=5.0917061997472
New Minimum: -37.239015683642656 > -39.397469505646185
WOLF (strong): th(8.680555555555562)=-39.397469505646185; dx=1.296700409321249 evalInputDelta=7.25016002175073
New Minimum: -39.397469505646185 > -45.418957638995494
END: th(0.9645061728395069)=-45.418957638995494; dx=-2.8424470833247195 evalInputDelta=13.271648155100038
Fitness changed from -32.147309483895455 to -45.418957638995494
Iteration 12 complete. Error: -45.418957638995494 Total: 1.2876; Orientation: 0.0286; Line Search: 1.1511
Adding measurement 5f88769e to history. Total: 4
Rejected: LBFGS Orientation magnitude: 4.583e+04, gradient 1.061e-02, dot -0.306; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.418957638995494, -32.147309483895455, -4.720699752661243, -2.769938452873265, -1.2507924125596435
Rejected: LBFGS Orientation magnitude: 6.377e+04, gradient 1.061e-02, dot -0.338; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.418957638995494, -32.147309483895455, -4.720699752661243, -2.769938452873265
LBFGS Accumulation History: 3 points
Removed measurement 5f88769e to history. Total: 4
Removed measurement 1ac9c3e0 to history. Total: 3
th(0)=-45.418957638995494;dx=-1.0960942153024654E-4
New Minimum: -45.418957638995494 > -45.41919705367585
END: th(2.0779655575153217)=-45.41919705367585; dx=-7.147003086993713E-5 evalInputDelta=2.3941468035815205E-4
Fitness changed from -45.418957638995494 to -45.41919705367585
Iteration 13 complete. Error: -45.41919705367585 Total: 0.4020; Orientation: 0.0595; Line Search: 0.2339
Adding measurement 5e6a1f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.865e+04, gradient 1.050e-02, dot -0.200; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.41919705367585, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 5e6a1f0 to history. Total: 3
th(0)=-45.41919705367585;dx=-1.0816447540323932E-4
New Minimum: -45.41919705367585 > -45.41971284719167
END: th(4.476841081802453)=-45.41971284719167; dx=-7.061313172946979E-5 evalInputDelta=5.157935158166538E-4
Fitness changed from -45.41919705367585 to -45.41971284719167
Iteration 14 complete. Error: -45.41971284719167 Total: 0.3669; Orientation: 0.0285; Line Search: 0.2285
Adding measurement 9d26faf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.018e+04, gradient 1.062e-02, dot -0.202; [5cbf83eb-bd0f-4f25-baed-b12c87fb6f28 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.41971284719167, -4.720699752661243, -2.769938452873265, -1.2507924125596435
LBFGS Accumulation History: 3 points
Removed measurement 9d26faf to history. Total: 3
th(0)=-45.41971284719167;dx=-1.1033444913946857E-4
New Minimum: -45.41971284719167 > -45.42082715243618
END: th(9.64506172839507)=-45.42082715243618; dx=-7.210690677357394E-5 evalInputDelta=0.0011143052445135027
Fitness changed from -45.41971284719167 to -45.42082715243618
Iteration 15 complete. Error: -45.42082715243618 Total: 0.3678; Orientation: 0.0271; Line Search: 0.2286
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -45.42082715243618 (> -Infinity) after 8.061s (< 5400.000s)
Returns
-45.42082715243618