BasicOptimizer.scala:89 executed in 221.51 seconds (3.418 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 685350298480300
Reset training subject: 685354958670800
Corrupt weights measurement
LBFGS Accumulation History: 0 points
Constructing line search parameters: GD+Trust
New Minimum: -847.5911455412922 > -847.871673662943
th(0)=-847.871673662943;dx=-0.006386386649489945
New Minimum: -847.871673662943 > -847.8853843533335
WOLFE (weak): th(2.154434690031884)=-847.8853843533335; dx=-0.0058238753487786146 evalInputDelta=0.013710690390553282
New Minimum: -847.8853843533335 > -847.8991316778194
WOLFE (weak): th(4.308869380063768)=-847.8991316778194; dx=-0.0058258323784906 evalInputDelta=0.02745801487640165
New Minimum: -847.8991316778194 > -847.9540818935893
WOLFE (weak): th(12.926608140191302)=-847.9540818935893; dx=-0.005826318763184855 evalInputDelta=0.08240823064636515
New Minimum: -847.9540818935893 > -848.2012585465786
WOLFE (weak): th(51.70643256076521)=-848.2012585465786; dx=-0.005824148967211992 evalInputDelta=0.3295848836356754
New Minimum: -848.2012585465786 > -849.5179883322071
WOLFE (weak): th(258.53216280382605)=-849.5179883322071; dx=-0.005808811923756221 evalInputDelta=1.646314669264143
New Minimum: -849.5179883322071 > -857.6878802921259
END: th(1551.1929768229563)=-857.6878802921259; dx=-0.005740339743892663 evalInputDelta=9.816206629182943
Fitness changed from -847.5911455412922 to -857.6878802921259
Iteration 1 complete. Error: -857.6878802921259 Total: 46.0495; Orientation: 0.0692; Line Search: 32.3228
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement c68fedb to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-857.6878802921259;dx=-0.005688038781160727
New Minimum: -857.6878802921259 > -878.0871255177359
WOLFE (weak): th(3341.943960201201)=-878.0871255177359; dx=-0.00547895243481724 evalInputDelta=20.399245225610002
New Minimum: -878.0871255177359 > -897.7632970332436
WOLFE (weak): th(6683.887920402402)=-897.7632970332436; dx=-0.005273700612358533 evalInputDelta=40.07541674111769
New Minimum: -897.7632970332436 > -968.3786384983923
END: th(20051.663761207208)=-968.3786384983923; dx=-0.004372285596060533 evalInputDelta=110.69075820626642
Fitness changed from -857.6878802921259 to -968.3786384983923
Iteration 2 complete. Error: -968.3786384983923 Total: 23.5647; Orientation: 0.0861; Line Search: 18.0955
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4d436e1a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-968.3786384983923;dx=-0.004086969863598335
New Minimum: -968.3786384983923 > -1128.8506876465108
END: th(43200.00000000001)=-1128.8506876465108; dx=-0.0027257050938539256 evalInputDelta=160.47204914811846
Fitness changed from -968.3786384983923 to -1128.8506876465108
Iteration 3 complete. Error: -1128.8506876465108 Total: 14.4768; Orientation: 0.0978; Line Search: 8.9587
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 2d5f674b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1128.8506876465108;dx=-0.002848383126779721
New Minimum: -1128.8506876465108 > -1333.6027332005585
END: th(93071.5786093774)=-1333.6027332005585; dx=-0.001423343630625554 evalInputDelta=204.75204555404775
Fitness changed from -1128.8506876465108 to -1333.6027332005585
Iteration 4 complete. Error: -1333.6027332005585 Total: 14.8759; Orientation: 0.1032; Line Search: 9.2215
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 23561b08 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.492e+04, gradient 5.084e-02, dot -0.665; [e2593374-b553-4dea-a3f1-40b00dd6c82f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1333.6027332005585, -1128.8506876465108, -968.3786384983923, -857.6878802921259
LBFGS Accumulation History: 3 points
Removed measurement 23561b08 to history. Total: 3
th(0)=-1333.6027332005585;dx=-0.0025810954621673693
New Minimum: -1333.6027332005585 > -1534.9905537168625
END: th(200516.63761207208)=-1534.9905537168625; dx=-3.5800136015314737E-4 evalInputDelta=201.38782051630392
Fitness changed from -1333.6027332005585 to -1534.9905537168625
Iteration 5 complete. Error: -1534.9905537168625 Total: 16.3701; Orientation: 1.7219; Line Search: 9.1362
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 76291fb7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.264e+04, gradient 5.836e-02, dot -0.399; [e2593374-b553-4dea-a3f1-40b00dd6c82f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1534.9905537168625, -1128.8506876465108, -968.3786384983923, -857.6878802921259
LBFGS Accumulation History: 3 points
Removed measurement 76291fb7 to history. Total: 3
th(0)=-1534.9905537168625;dx=-0.0034028941269839408
New Minimum: -1534.9905537168625 > -1582.7351326106336
WOLF (strong): th(432000.0000000001)=-1582.7351326106336; dx=3.7446997640063086E-4 evalInputDelta=47.74457889377118
New Minimum: -1582.7351326106336 > -1688.4130863032306
WOLF (strong): th(216000.00000000006)=-1688.4130863032306; dx=3.0917289840681826E-4 evalInputDelta=153.4225325863681
New Minimum: -1688.4130863032306 > -1691.731952763203
END: th(72000.00000000001)=-1691.731952763203; dx=-8.477613764125662E-4 evalInputDelta=156.74139904634058
Fitness changed from -1534.9905537168625 to -1691.731952763203
Iteration 6 complete. Error: -1691.731952763203 Total: 25.2792; Orientation: 1.6924; Line Search: 18.1464
Adding measurement 77edee8d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.128e+04, gradient 4.732e-02, dot -0.504; [e2593374-b553-4dea-a3f1-40b00dd6c82f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1691.731952763203, -1128.8506876465108, -968.3786384983923, -857.6878802921259
LBFGS Accumulation History: 3 points
Removed measurement 77edee8d to history. Total: 3
th(0)=-1691.731952763203;dx=-0.0022371756610589976
New Minimum: -1691.731952763203 > -1833.1618177345938
END: th(155119.29768229567)=-1833.1618177345938; dx=-1.6154984529517775E-4 evalInputDelta=141.42986497139077
Fitness changed from -1691.731952763203 to -1833.1618177345938
Iteration 7 complete. Error: -1833.1618177345938 Total: 15.4776; Orientation: 1.6692; Line Search: 9.2966
Adding measurement 3be98cbc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.799e+04, gradient 6.003e-02, dot -0.390; [e2593374-b553-4dea-a3f1-40b00dd6c82f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1833.1618177345938, -1128.8506876465108, -968.3786384983923, -857.6878802921259
LBFGS Accumulation History: 3 points
Removed measurement 3be98cbc to history. Total: 3
th(0)=-1833.1618177345938;dx=-0.0036003002621443876
Armijo: th(334194.3960201202)=-1801.0140893396697; dx=5.338997759074014E-4 evalInputDelta=-32.147728394924115
New Minimum: -1833.1618177345938 > -1928.34871068487
WOLF (strong): th(167097.1980100601)=-1928.34871068487; dx=6.198595447170893E-4 evalInputDelta=95.18689295027616
New Minimum: -1928.34871068487 > -1962.324641847011
END: th(55699.066003353364)=-1962.324641847011; dx=-8.166339173724134E-4 evalInputDelta=129.16282411241718
Fitness changed from -1833.1618177345938 to -1962.324641847011
Iteration 8 complete. Error: -1962.324641847011 Total: 24.0741; Orientation: 1.6191; Line Search: 17.8545
Adding measurement 784f41d9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.731e+04, gradient 4.220e-02, dot -0.558; [e2593374-b553-4dea-a3f1-40b00dd6c82f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1962.324641847011, -1128.8506876465108, -968.3786384983923, -857.6878802921259
LBFGS Accumulation History: 3 points
Removed measurement 784f41d9 to history. Total: 3
th(0)=-1962.324641847011;dx=-0.001776747462156186
New Minimum: -1962.324641847011 > -2079.0336697610264
END: th(120000.00000000004)=-2079.0336697610264; dx=-4.200042915882317E-4 evalInputDelta=116.70902791401545
Fitness changed from -1962.324641847011 to -2079.0336697610264
Iteration 9 complete. Error: -2079.0336697610264 Total: 15.4552; Orientation: 1.6185; Line Search: 9.3338
Adding measurement 252ac6ff to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.437e+04, gradient 5.127e-02, dot -0.453; [e2593374-b553-4dea-a3f1-40b00dd6c82f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2079.0336697610264, -1128.8506876465108, -968.3786384983923, -857.6878802921259
LBFGS Accumulation History: 3 points
Removed measurement 252ac6ff to history. Total: 3
th(0)=-2079.0336697610264;dx=-0.0026214136989531093
New Minimum: -2079.0336697610264 > -2095.1381816004405
WOLF (strong): th(258532.16280382616)=-2095.1381816004405; dx=4.1585399517801177E-4 evalInputDelta=16.104511839414045
New Minimum: -2095.1381816004405 > -2159.199157095056
WOLF (strong): th(129266.08140191308)=-2159.199157095056; dx=2.524854109794234E-4 evalInputDelta=80.16548733402942
END: th(43088.693800637695)=-2155.022969715172; dx=-8.140329750812598E-4 evalInputDelta=75.98929995414574
Fitness changed from -2079.0336697610264 to -2159.199157095056
Iteration 10 complete. Error: -2159.199157095056 Total: 25.0211; Orientation: 1.6357; Line Search: 18.9880
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Final threshold in iteration 11: -2159.199157095056 (> -Infinity) after 221.511s (< 5400.000s)
Returns
-2159.199157095056