BasicOptimizer.scala:89 executed in 55.71 seconds (1.154 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 219594689689700
Reset training subject: 219596282982700
Adding measurement 186b675 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-79.54310447015504;dx=-0.07844666578375306
New Minimum: -79.54310447015504 > -79.65727963699777
WOLFE (weak): th(2.154434690031884)=-79.65727963699777; dx=-0.07855015340798885 evalInputDelta=0.11417516684272755
New Minimum: -79.65727963699777 > -79.77183887177613
WOLFE (weak): th(4.308869380063768)=-79.77183887177613; dx=-0.0785617880898995 evalInputDelta=0.22873440162108238
New Minimum: -79.77183887177613 > -80.23075535907432
WOLFE (weak): th(12.926608140191302)=-80.23075535907432; dx=-0.07858585373064642 evalInputDelta=0.6876508889192792
New Minimum: -80.23075535907432 > -82.30923077215775
WOLFE (weak): th(51.70643256076521)=-82.30923077215775; dx=-0.0785322828612929 evalInputDelta=2.7661263020027036
New Minimum: -82.30923077215775 > -93.36295753915867
WOLFE (weak): th(258.53216280382605)=-93.36295753915867; dx=-0.07497432056871563 evalInputDelta=13.819853069003628
New Minimum: -93.36295753915867 > -153.12477436894216
END: th(1551.1929768229563)=-153.12477436894216; dx=-0.05603775668644058 evalInputDelta=73.58166989878711
Fitness changed from -79.54310447015504 to -153.12477436894216
Iteration 1 complete. Error: -153.12477436894216 Total: 7.7074; Orientation: 0.0253; Line Search: 4.8037
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 202a4fee to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-153.12477436894216;dx=-0.10002298938744503
New Minimum: -153.12477436894216 > -278.7353194845501
END: th(3341.943960201201)=-278.7353194845501; dx=-0.038630629994975865 evalInputDelta=125.61054511560792
Fitness changed from -153.12477436894216 to -278.7353194845501
Iteration 2 complete. Error: -278.7353194845501 Total: 1.9831; Orientation: 0.0201; Line Search: 1.3294
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 626747fc to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-278.7353194845501;dx=-0.1165538709855247
New Minimum: -278.7353194845501 > -416.985872401009
END: th(7200.000000000001)=-416.985872401009; dx=-0.021471941383850134 evalInputDelta=138.2505529164589
Fitness changed from -278.7353194845501 to -416.985872401009
Iteration 3 complete. Error: -416.985872401009 Total: 2.1618; Orientation: 0.0210; Line Search: 1.4905
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 43a43d5c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.287e+04, gradient 3.459e-01, dot -0.222; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -416.985872401009, -278.7353194845501, -153.12477436894216, -79.54310447015504
LBFGS Accumulation History: 3 points
Removed measurement 43a43d5c to history. Total: 3
th(0)=-416.985872401009;dx=-0.11962127181088614
New Minimum: -416.985872401009 > -589.3247673207792
END: th(15511.929768229566)=-589.3247673207792; dx=-0.008267703051287656 evalInputDelta=172.3388949197702
Fitness changed from -416.985872401009 to -589.3247673207792
Iteration 4 complete. Error: -589.3247673207792 Total: 2.0926; Orientation: 0.2992; Line Search: 1.1961
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 29c50bc7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.300e+04, gradient 3.434e-01, dot -0.175; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -589.3247673207792, -278.7353194845501, -153.12477436894216, -79.54310447015504
LBFGS Accumulation History: 3 points
Removed measurement 29c50bc7 to history. Total: 3
th(0)=-589.3247673207792;dx=-0.11792706136577893
New Minimum: -589.3247673207792 > -632.032324582094
END: th(33419.43960201202)=-632.032324582094; dx=-0.001991282037706501 evalInputDelta=42.70755726131483
Fitness changed from -589.3247673207792 to -632.032324582094
Iteration 5 complete. Error: -632.032324582094 Total: 2.1280; Orientation: 0.2974; Line Search: 1.2245
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4b504351 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.958e+03, gradient 3.075e-01, dot -0.112; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -632.032324582094, -278.7353194845501, -153.12477436894216, -79.54310447015504
LBFGS Accumulation History: 3 points
Removed measurement 4b504351 to history. Total: 3
th(0)=-632.032324582094;dx=-0.09454449114894385
Armijo: th(72000.00000000003)=-629.1919604189081; dx=-3.197361708007562E-4 evalInputDelta=-2.840364163185882
New Minimum: -632.032324582094 > -979.2315880786599
END: th(36000.000000000015)=-979.2315880786599; dx=-0.007677087635581959 evalInputDelta=347.1992634965659
Fitness changed from -632.032324582094 to -979.2315880786599
Iteration 6 complete. Error: -979.2315880786599 Total: 2.7093; Orientation: 0.2964; Line Search: 1.8195
Adding measurement 59adf78c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.522e+04, gradient 3.193e-01, dot -0.126; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -979.2315880786599, -278.7353194845501, -153.12477436894216, -79.54310447015504
LBFGS Accumulation History: 3 points
Removed measurement 59adf78c to history. Total: 3
th(0)=-979.2315880786599;dx=-0.10194080272617126
Armijo: th(77559.64884114785)=-799.5505077773533; dx=6.22850488410169E-4 evalInputDelta=-179.68108030130657
New Minimum: -979.2315880786599 > -1327.9660138589256
END: th(38779.824420573925)=-1327.9660138589256; dx=-0.007835921982086158 evalInputDelta=348.7344257802657
Fitness changed from -979.2315880786599 to -1327.9660138589256
Iteration 7 complete. Error: -1327.9660138589256 Total: 2.6887; Orientation: 0.3019; Line Search: 1.8436
Adding measurement 354377ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.032e+04, gradient 3.287e-01, dot -0.138; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1327.9660138589256, -278.7353194845501, -153.12477436894216, -79.54310447015504
LBFGS Accumulation History: 3 points
Removed measurement 354377ea to history. Total: 3
th(0)=-1327.9660138589256;dx=-0.10792086619924163
Armijo: th(83548.59900503006)=-990.7697987378888; dx=9.636731773992347E-4 evalInputDelta=-337.19621512103686
New Minimum: -1327.9660138589256 > -1604.4585037325915
END: th(41774.29950251503)=-1604.4585037325915; dx=-0.006747137101315718 evalInputDelta=276.4924898736658
Fitness changed from -1327.9660138589256 to -1604.4585037325915
Iteration 8 complete. Error: -1604.4585037325915 Total: 2.9692; Orientation: 0.3010; Line Search: 2.0782
Adding measurement 3eb71f18 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.309e+05, gradient 3.298e-01, dot -0.127; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1604.4585037325915, -278.7353194845501, -153.12477436894216, -79.54310447015504
LBFGS Accumulation History: 3 points
Removed measurement 3eb71f18 to history. Total: 3
th(0)=-1604.4585037325915;dx=-0.1087263124578419
Armijo: th(90000.00000000004)=-1082.1359224946198; dx=0.0017762496642256959 evalInputDelta=-522.3225812379717
New Minimum: -1604.4585037325915 > -1814.6330225624406
END: th(45000.00000000002)=-1814.6330225624406; dx=-0.006131374954887414 evalInputDelta=210.17451882984915
Fitness changed from -1604.4585037325915 to -1814.6330225624406
Iteration 9 complete. Error: -1814.6330225624406 Total: 2.9675; Orientation: 0.3243; Line Search: 2.0615
Adding measurement 6082a524 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.495e+04, gradient 3.307e-01, dot 0.167; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1814.6330225624406;dx=-3582.5380307260616
Armijo: th(96949.56105143482)=-251.81765951132115; dx=0.014268695899705076 evalInputDelta=-1562.8153630511194
Armijo: th(48474.78052571741)=-252.48349573741666; dx=0.028545434528286232 evalInputDelta=-1562.149526825024
Armijo: th(16158.260175239137)=-254.39230019537172; dx=0.08550881207872985 evalInputDelta=-1560.2407223670689
Armijo: th(4039.565043809784)=-259.5522604048647; dx=0.341388332617505 evalInputDelta=-1555.080762157576
Armijo: th(807.9130087619568)=-271.1066905097898; dx=1.6925820806674843 evalInputDelta=-1543.5263320526508
Armijo: th(134.6521681269928)=-303.7097334659063; dx=10.061694115039366 evalInputDelta=-1510.9232890965343
Armijo: th(19.236024018141826)=-415.8023248343475; dx=63.51142839959567 evalInputDelta=-1398.8306977280931
Armijo: th(2.4045030022677283)=-815.9823137006551; dx=301.97441119017026 evalInputDelta=-998.6507088617855
Armijo: th(0.2671670002519698)=-1645.275338805815; dx=262.66577485797495 evalInputDelta=-169.35768375662565
Armijo: th(0.026716700025196983)=-1813.3594810910606; dx=-635.6590109793052 evalInputDelta=-1.2735414713799855
New Minimum: -1814.6330225624406 > -1814.904056224014
END: th(0.0024287909113815437)=-1814.904056224014; dx=-872.006677099596 evalInputDelta=0.2710336615734832
Fitness changed from -1814.6330225624406 to -1814.904056224014
Iteration 10 complete. Error: -1814.904056224014 Total: 8.4330; Orientation: 0.2980; Line Search: 7.5467
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 11cc9e15 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 6.095e+06, gradient 3.302e-01, dot -0.144; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1814.904056224014, -1814.6330225624406, -278.7353194845501, -153.12477436894216, -79.54310447015504
Rejected: LBFGS Orientation magnitude: 1.542e+07, gradient 3.302e-01, dot -0.153; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1814.904056224014, -1814.6330225624406, -278.7353194845501, -153.12477436894216
LBFGS Accumulation History: 3 points
Removed measurement 11cc9e15 to history. Total: 4
Removed measurement 6082a524 to history. Total: 3
th(0)=-1814.904056224014;dx=-0.10901820165215312
New Minimum: -1814.904056224014 > -1814.904254595764
END: th(0.005232671394314553)=-1814.904254595764; dx=-0.07742915718179702 evalInputDelta=1.9837175000247953E-4
Fitness changed from -1814.904056224014 to -1814.904254595764
Iteration 11 complete. Error: -1814.904254595764 Total: 2.4721; Orientation: 0.6418; Line Search: 1.2154
Adding measurement 1fdd1d93 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 6.601e+04, gradient 3.302e-01, dot 0.167; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
th(0)=-1814.904254595764;dx=-3635.006870907087
New Minimum: -1814.904254595764 > -1815.1787818507612
END: th(0.011273448773448779)=-1815.1787818507612; dx=-783.180243802744 evalInputDelta=0.2745272549971105
Fitness changed from -1814.904254595764 to -1815.1787818507612
Iteration 12 complete. Error: -1815.1787818507612 Total: 2.0363; Orientation: 0.2895; Line Search: 1.2120
Adding measurement 38fdc9b to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.532e+06, gradient 3.283e-01, dot -0.142; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1815.1787818507612, -1814.904254595764, -278.7353194845501, -153.12477436894216, -79.54310447015504
Rejected: LBFGS Orientation magnitude: 3.190e+06, gradient 3.283e-01, dot -0.153; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1815.1787818507612, -1814.904254595764, -278.7353194845501, -153.12477436894216
LBFGS Accumulation History: 3 points
Removed measurement 38fdc9b to history. Total: 4
Removed measurement 1fdd1d93 to history. Total: 3
th(0)=-1815.1787818507612;dx=-0.10776774243799882
New Minimum: -1815.1787818507612 > -1815.179842881195
END: th(0.02428790911381544)=-1815.179842881195; dx=-0.07557961356310068 evalInputDelta=0.0010610304339024879
Fitness changed from -1815.1787818507612 to -1815.179842881195
Iteration 13 complete. Error: -1815.179842881195 Total: 2.4494; Orientation: 0.6477; Line Search: 1.2370
Adding measurement 18861e28 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.059e+04, gradient 3.283e-01, dot 0.165; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
th(0)=-1815.179842881195;dx=-3832.908597391765
Armijo: th(0.05232671394314553)=-1798.1271129023455; dx=-419.4771902152151 evalInputDelta=-17.052729978849584
Armijo: th(0.026163356971572765)=-1809.3584206899352; dx=-611.2978848438522 evalInputDelta=-5.821422191259899
Armijo: th(0.008721118990524255)=-1814.0965014857134; dx=-764.838919711478 evalInputDelta=-1.0833413954817388
Armijo: th(0.0021802797476310636)=-1815.0100248095487; dx=-834.9169793516573 evalInputDelta=-0.16981807164643214
Armijo: th(4.3605594952621273E-4)=-1815.1566992360956; dx=-867.991506679833 evalInputDelta=-0.023143645099480636
Armijo: th(7.267599158770212E-5)=-1815.1761697965399; dx=-871.6595970560267 evalInputDelta=-0.003673084655247294
Armijo: th(1.0382284512528874E-5)=-1815.179354418284; dx=-857.8986681943339 evalInputDelta=-4.884629111074901E-4
Armijo: th(1.2977855640661093E-6)=-1815.1797540871216; dx=-729.1081477848662 evalInputDelta=-8.879407346285006E-5
Armijo: th(1.4419839600734548E-7)=-1815.1797951974793; dx=-24.42404686543196 evalInputDelta=-4.76837158203125E-5
New Minimum: -1815.179842881195 > -1815.1798945924397
WOLF (strong): th(1.4419839600734548E-8)=-1815.1798945924397; dx=194.545026645998 evalInputDelta=5.171124462322041E-5
Armijo: th(1.3108945091576862E-9)=-1815.179842881195; dx=148.10131447664975 evalInputDelta=0.0
New Minimum: -1815.1798945924397 > -1815.1799088975545
WOLF (strong): th(1.0924120909647385E-10)=-1815.1799088975545; dx=141.24873732549173 evalInputDelta=6.601635936931416E-5
MIN ALPHA (8.403169930497988E-12): th(1.0924120909647385E-10)=-1815.1799088975545
Fitness changed from -1815.179842881195 to -1815.1799088975545
Iteration 14 complete. Error: -1815.1799088975545 Total: 10.1809; Orientation: 0.3276; Line Search: 9.2913
Adding measurement 4515a5eb to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.159e+08, gradient 3.284e-01, dot -0.138; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1815.1799088975545, -1815.179842881195, -278.7353194845501, -153.12477436894216, -79.54310447015504
Rejected: LBFGS Orientation magnitude: 5.857e+08, gradient 3.284e-01, dot -0.150; [bdfc9ac5-2810-4c41-8fa6-6a0b9c601a72 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1815.1799088975545, -1815.179842881195, -278.7353194845501, -153.12477436894216
LBFGS Accumulation History: 3 points
Removed measurement 4515a5eb to history. Total: 4
Removed measurement 18861e28 to history. Total: 3
th(0)=-1815.1799088975545;dx=-0.1077888501955355
END: th(2.1544346900318838E-10)=-1815.1799088975545; dx=-0.07564438619284339 evalInputDelta=0.0
Fitness changed from -1815.1799088975545 to -1815.1799088975545
Static Iteration Total: 2.7280; Orientation: 0.6501; Line Search: 1.5126
Iteration 15 failed. Error: -1815.1799088975545
Previous Error: 0.0 -> -1815.1799088975545
Retrying iteration 15
Final threshold in iteration 15: -1815.1799088975545 (> -Infinity) after 55.707s (< 5400.000s)
Returns
-1815.1799088975545