BasicOptimizer.scala:89 executed in 53.80 seconds (1.070 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 218462161717400
Reset training subject: 218462793589200
Adding measurement 6aca004 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-64.21251110205277;dx=-0.1117049164515864
New Minimum: -64.21251110205277 > -64.39949064410554
WOLFE (weak): th(2.154434690031884)=-64.39949064410554; dx=-0.11185291289309354 evalInputDelta=0.186979542052768
New Minimum: -64.39949064410554 > -64.58741761478322
WOLFE (weak): th(4.308869380063768)=-64.58741761478322; dx=-0.11262256866307874 evalInputDelta=0.3749065127304476
New Minimum: -64.58741761478322 > -65.35105820036098
WOLFE (weak): th(12.926608140191302)=-65.35105820036098; dx=-0.11511984845520365 evalInputDelta=1.1385470983082087
New Minimum: -65.35105820036098 > -68.99195672901504
WOLFE (weak): th(51.70643256076521)=-68.99195672901504; dx=-0.12323017731300578 evalInputDelta=4.779445626962271
New Minimum: -68.99195672901504 > -92.17328785383594
WOLFE (weak): th(258.53216280382605)=-92.17328785383594; dx=-0.1340538323643676 evalInputDelta=27.960776751783172
New Minimum: -92.17328785383594 > -167.41877067162827
END: th(1551.1929768229563)=-167.41877067162827; dx=-0.04527132632917201 evalInputDelta=103.2062595695755
Fitness changed from -64.21251110205277 to -167.41877067162827
Iteration 1 complete. Error: -167.41877067162827 Total: 6.2752; Orientation: 0.0159; Line Search: 4.2897
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4f7e9187 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-167.41877067162827;dx=-0.17551321588334737
New Minimum: -167.41877067162827 > -338.20963619743895
END: th(3341.943960201201)=-338.20963619743895; dx=-0.02597251156711991 evalInputDelta=170.7908655258107
Fitness changed from -167.41877067162827 to -338.20963619743895
Iteration 2 complete. Error: -338.20963619743895 Total: 1.7773; Orientation: 0.0202; Line Search: 1.2006
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 7af8f2b7 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-338.20963619743895;dx=-0.12503988039886343
New Minimum: -338.20963619743895 > -555.4369077086728
END: th(7200.000000000001)=-555.4369077086728; dx=-0.013059424625047741 evalInputDelta=217.22727151123388
Fitness changed from -338.20963619743895 to -555.4369077086728
Iteration 3 complete. Error: -555.4369077086728 Total: 1.7742; Orientation: 0.0165; Line Search: 1.1882
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7cb4a4ce to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.008e+03, gradient 3.429e-01, dot -0.213; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -555.4369077086728, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 7cb4a4ce to history. Total: 3
th(0)=-555.4369077086728;dx=-0.11745808096206971
New Minimum: -555.4369077086728 > -760.136820888423
WOLF (strong): th(15511.929768229566)=-760.136820888423; dx=7.72446012778265E-4 evalInputDelta=204.69991317975018
New Minimum: -760.136820888423 > -821.9626787913086
END: th(7755.964884114783)=-821.9626787913086; dx=-0.015417433871460287 evalInputDelta=266.52577108263574
Fitness changed from -555.4369077086728 to -821.9626787913086
Iteration 4 complete. Error: -821.9626787913086 Total: 2.7210; Orientation: 0.3034; Line Search: 1.8179
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 658f9bf8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.814e+03, gradient 3.414e-01, dot -0.205; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -821.9626787913086, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 658f9bf8 to history. Total: 3
th(0)=-821.9626787913086;dx=-0.11644092297421269
New Minimum: -821.9626787913086 > -937.1667189538844
WOLF (strong): th(16709.71980100601)=-937.1667189538844; dx=0.002512495089332749 evalInputDelta=115.20404016257578
New Minimum: -937.1667189538844 > -1076.5433075604094
END: th(8354.859900503005)=-1076.5433075604094; dx=-0.014516597754977208 evalInputDelta=254.58062876910083
Fitness changed from -821.9626787913086 to -1076.5433075604094
Iteration 5 complete. Error: -1076.5433075604094 Total: 2.9985; Orientation: 0.2855; Line Search: 1.8835
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6c62d64 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.916e+03, gradient 3.573e-01, dot -0.220; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1076.5433075604094, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 6c62d64 to history. Total: 3
th(0)=-1076.5433075604094;dx=-0.12759192090945154
New Minimum: -1076.5433075604094 > -1127.191658993246
WOLF (strong): th(18000.000000000007)=-1127.191658993246; dx=0.0038976986580865035 evalInputDelta=50.648351432836535
New Minimum: -1127.191658993246 > -1322.0229077319138
END: th(9000.000000000004)=-1322.0229077319138; dx=-0.015405020929195514 evalInputDelta=245.47960017150444
Fitness changed from -1076.5433075604094 to -1322.0229077319138
Iteration 6 complete. Error: -1322.0229077319138 Total: 2.7181; Orientation: 0.3012; Line Search: 1.8170
Adding measurement 1b40f41c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.126e+04, gradient 3.548e-01, dot -0.224; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1322.0229077319138, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 1b40f41c to history. Total: 3
th(0)=-1322.0229077319138;dx=-0.12566937511165288
New Minimum: -1322.0229077319138 > -1346.138177467767
WOLF (strong): th(19389.912210286962)=-1346.138177467767; dx=0.004343549099081732 evalInputDelta=24.11526973585319
New Minimum: -1346.138177467767 > -1579.48535404982
END: th(9694.956105143481)=-1579.48535404982; dx=-0.016477988225441317 evalInputDelta=257.46244631790614
Fitness changed from -1322.0229077319138 to -1579.48535404982
Iteration 7 complete. Error: -1579.48535404982 Total: 2.6957; Orientation: 0.3025; Line Search: 1.8007
Adding measurement 62786a04 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.934e+04, gradient 3.606e-01, dot -0.228; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1579.48535404982, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 62786a04 to history. Total: 3
th(0)=-1579.48535404982;dx=-0.12993989743391243
Armijo: th(20887.149751257515)=-1523.4577171293377; dx=0.005878777026477371 evalInputDelta=-56.02763692048234
New Minimum: -1579.48535404982 > -1808.300425461396
END: th(10443.574875628758)=-1808.300425461396; dx=-0.015108944286752978 evalInputDelta=228.8150714115759
Fitness changed from -1579.48535404982 to -1808.300425461396
Iteration 8 complete. Error: -1808.300425461396 Total: 2.6474; Orientation: 0.2940; Line Search: 1.7914
Adding measurement 798c88f8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.315e+04, gradient 3.665e-01, dot -0.229; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1808.300425461396, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 798c88f8 to history. Total: 3
th(0)=-1808.300425461396;dx=-0.13415907485436512
Armijo: th(22500.00000000001)=-1670.9506501001488; dx=0.008192290864654624 evalInputDelta=-137.3497753612471
New Minimum: -1808.300425461396 > -2038.0244667983561
END: th(11250.000000000005)=-2038.0244667983561; dx=-0.014071858009585823 evalInputDelta=229.72404133696023
Fitness changed from -1808.300425461396 to -2038.0244667983561
Iteration 9 complete. Error: -2038.0244667983561 Total: 2.9867; Orientation: 0.5208; Line Search: 1.9127
Adding measurement 7eebb70 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.214e+04, gradient 3.689e-01, dot -0.232; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2038.0244667983561, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 7eebb70 to history. Total: 3
th(0)=-2038.0244667983561;dx=-0.13595577307302853
Armijo: th(24237.390262858706)=-1849.8243401508698; dx=0.009532442039409424 evalInputDelta=-188.2001266474863
New Minimum: -2038.0244667983561 > -2272.4011123214536
END: th(12118.695131429353)=-2272.4011123214536; dx=-0.012391687800799852 evalInputDelta=234.37664552309752
Fitness changed from -2038.0244667983561 to -2272.4011123214536
Iteration 10 complete. Error: -2272.4011123214536 Total: 2.6449; Orientation: 0.2944; Line Search: 1.8114
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 76e12243 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.018e+05, gradient 3.781e-01, dot -0.226; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2272.4011123214536, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 76e12243 to history. Total: 3
th(0)=-2272.4011123214536;dx=-0.1427881441139028
Armijo: th(26108.937189071898)=-2006.338913542128; dx=0.011377518941794111 evalInputDelta=-266.0621987793256
New Minimum: -2272.4011123214536 > -2490.741604725248
END: th(13054.468594535949)=-2490.741604725248; dx=-0.010930821751395103 evalInputDelta=218.34049240379454
Fitness changed from -2272.4011123214536 to -2490.741604725248
Iteration 11 complete. Error: -2490.741604725248 Total: 2.6971; Orientation: 0.2983; Line Search: 1.8174
Adding measurement 642ae3f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.461e+06, gradient 3.780e-01, dot -0.220; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2490.741604725248, -338.20963619743895, -167.41877067162827, -64.21251110205277
LBFGS Accumulation History: 3 points
Removed measurement 642ae3f to history. Total: 3
th(0)=-2490.741604725248;dx=-0.14279124826941775
Armijo: th(28125.00000000002)=-2166.185697926125; dx=0.013483010908924445 evalInputDelta=-324.555906799123
New Minimum: -2490.741604725248 > -2723.8363413605166
END: th(14062.50000000001)=-2723.8363413605166; dx=-0.01010027297752281 evalInputDelta=233.09473663526842
Fitness changed from -2490.741604725248 to -2723.8363413605166
Iteration 12 complete. Error: -2723.8363413605166 Total: 2.6854; Orientation: 0.2949; Line Search: 1.8183
Adding measurement 669d4387 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.526e+04, gradient 3.821e-01, dot 0.285; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-2723.8363413605166;dx=-2753.0016025548757
Armijo: th(30296.737828573387)=-1100.345361426216; dx=0.03937733685606944 evalInputDelta=-1623.4909799343006
Armijo: th(15148.368914286693)=-1103.227610685654; dx=0.07872231385039118 evalInputDelta=-1620.6087306748625
Armijo: th(5049.456304762231)=-1110.1459273941177; dx=0.23462952955142505 evalInputDelta=-1613.6904139663989
Armijo: th(1262.3640761905579)=-1124.3315676891211; dx=0.924254039134208 evalInputDelta=-1599.5047736713955
Armijo: th(252.47281523811156)=-1158.0879387350537; dx=4.472062529435661 evalInputDelta=-1565.7484026254629
Armijo: th(42.07880253968526)=-1270.2421014988943; dx=23.03304477763245 evalInputDelta=-1453.5942398616223
Armijo: th(6.011257505669323)=-1762.7225581370967; dx=90.4254414790349 evalInputDelta=-961.1137832234199
New Minimum: -2723.8363413605166 > -2897.8518650477918
END: th(0.7514071882086654)=-2897.8518650477918; dx=-174.34799273681585 evalInputDelta=174.01552368727516
Fitness changed from -2723.8363413605166 to -2897.8518650477918
Iteration 13 complete. Error: -2897.8518650477918 Total: 6.7139; Orientation: 0.5235; Line Search: 5.6345
Adding measurement 4c2e7e9f to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.269e+04, gradient 3.567e-01, dot 0.079; [beec8f9d-c095-406f-9d09-56e15389eb60 = 1.000/1.000e+00]
th(0)=-2897.8518650477918;dx=-354.56607610453557
Armijo: th(1.6188577126160653)=-2356.212621094983; dx=43.46083248466328 evalInputDelta=-541.6392439528086
Armijo: th(0.8094288563080326)=-2606.4173812376293; dx=1.5934407143844567 evalInputDelta=-291.4344838101624
Armijo: th(0.26980961876934423)=-2803.8238721870152; dx=-85.02908024290444 evalInputDelta=-94.02799286077652
Armijo: th(0.06745240469233606)=-2875.891911066761; dx=-152.2150989905644 evalInputDelta=-21.959953981030594
Armijo: th(0.013490480938467211)=-2893.5720245745474; dx=-174.54984630477762 evalInputDelta=-4.279840473244349
Armijo: th(0.002248413489744535)=-2897.147648163589; dx=-180.49992989536213 evalInputDelta=-0.7042168842026513
Armijo: th(3.212019271063622E-4)=-2897.7510636983607; dx=-180.19639484175718 evalInputDelta=-0.10080134943109442
Armijo: th(4.015024088829527E-5)=-2897.838870857843; dx=-181.23606586454716 evalInputDelta=-0.0129941899485857
Armijo: th(4.461137876477252E-6)=-2897.8502949620843; dx=-179.98175649200815 evalInputDelta=-0.0015700857074989472
Armijo: th(4.4611378764772524E-7)=-2897.851513057579; dx=-180.95197156001575 evalInputDelta=-3.5199021294829436E-4
Armijo: th(4.055579887706593E-8)=-2897.851698589433; dx=-180.37932659603155 evalInputDelta=-1.6645835876261117E-4
Armijo: th(3.3796499064221608E-9)=-2897.851698589433; dx=-181.10055225625453 evalInputDelta=-1.6645835876261117E-4
END: th(2.599730697247816E-10)=-2897.8518650477918; dx=-180.92361608721035 evalInputDelta=0.0
Fitness changed from -2897.8518650477918 to -2897.8518650477918
Static Iteration Total: 9.9734; Orientation: 0.3638; Line Search: 9.0588
Iteration 14 failed. Error: -2897.8518650477918
Previous Error: 0.0 -> -2897.8518650477918
Retrying iteration 14
Reset training subject: 218513470696200
Adding measurement 472a6cab to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-2897.8518650477918;dx=-0.12707102241608143
END: th(5.600949998891472E-10)=-2897.8518650477918; dx=-0.09473658585022082 evalInputDelta=0.0
Fitness changed from -2897.8518650477918 to -2897.8518650477918
Static Iteration Total: 2.4933; Orientation: 0.0155; Line Search: 1.3196
Iteration 15 failed. Error: -2897.8518650477918
Previous Error: 0.0 -> -2897.8518650477918
Optimization terminated 15
Final threshold in iteration 15: -2897.8518650477918 (> -Infinity) after 53.802s (< 5400.000s)
Returns
-2897.8518650477918