BasicOptimizer.scala:89 executed in 113.92 seconds (1.141 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 4911057057958
Reset training subject: 4912224026777
Adding measurement 4b6a7bb9 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.11999729034225583;dx=-9.54096464715768E-5
New Minimum: -0.11999729034225583 > -0.12018640198778818
WOLFE (weak): th(2.154434690031884)=-0.12018640198778818; dx=-9.542849505784644E-5 evalInputDelta=1.891116455323555E-4
New Minimum: -0.12018640198778818 > -0.12037591284854382
WOLFE (weak): th(4.308869380063768)=-0.12037591284854382; dx=-9.54859473071559E-5 evalInputDelta=3.7862250628799277E-4
New Minimum: -0.12037591284854382 > -0.12113435126111188
WOLFE (weak): th(12.926608140191302)=-0.12113435126111188; dx=-9.547847782428603E-5 evalInputDelta=0.001137060918856056
New Minimum: -0.12113435126111188 > -0.1245527947681675
WOLFE (weak): th(51.70643256076521)=-0.1245527947681675; dx=-9.566502453677163E-5 evalInputDelta=0.004555504425911669
New Minimum: -0.1245527947681675 > -0.14283554145863658
WOLFE (weak): th(258.53216280382605)=-0.14283554145863658; dx=-9.475925968985318E-5 evalInputDelta=0.022838251116380756
New Minimum: -0.14283554145863658 > -0.25413596692230067
WOLFE (weak): th(1551.1929768229563)=-0.25413596692230067; dx=-8.959303001424015E-5 evalInputDelta=0.13413867658004486
New Minimum: -0.25413596692230067 > -0.6610125118101601
END: th(10858.350837760694)=-0.6610125118101601; dx=-1.991327169780937E-5 evalInputDelta=0.5410152214679043
Fitness changed from -0.11999729034225583 to -0.6610125118101601
Iteration 1 complete. Error: -0.6610125118101601 Total: 12.9086; Orientation: 0.0175; Line Search: 9.4520
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 42261b03 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.6610125118101601;dx=-1.7795805006333274E-4
New Minimum: -0.6610125118101601 > -1.0392491452651997
WOLF (strong): th(23393.607721408407)=-1.0392491452651997; dx=1.754687215999737E-5 evalInputDelta=0.37823663345503966
New Minimum: -1.0392491452651997 > -1.1745279926489842
END: th(11696.803860704204)=-1.1745279926489842; dx=-9.787125926191787E-7 evalInputDelta=0.5135154808388241
Fitness changed from -0.6610125118101601 to -1.1745279926489842
Iteration 2 complete. Error: -1.1745279926489842 Total: 4.8548; Orientation: 0.0170; Line Search: 3.5693
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 65f4ea67 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-1.1745279926489842;dx=-2.155452320710613E-4
New Minimum: -1.1745279926489842 > -2.8980256890842573
WOLF (strong): th(25200.000000000004)=-2.8980256890842573; dx=4.176054506561647E-6 evalInputDelta=1.723497696435273
END: th(12600.000000000002)=-2.468216464177744; dx=-8.164638634321157E-6 evalInputDelta=1.29368847152876
Fitness changed from -1.1745279926489842 to -2.8980256890842573
Iteration 3 complete. Error: -2.8980256890842573 Total: 4.7847; Orientation: 0.0167; Line Search: 3.5156
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 49aa62ea to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.195e+03, gradient 2.617e-02, dot -0.244; [65296433-eddd-4b75-aa9d-a5806d837aaf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.8980256890842573, -1.1745279926489842, -0.6610125118101601, -0.11999729034225583
LBFGS Accumulation History: 3 points
Removed measurement 49aa62ea to history. Total: 3
th(0)=-2.8980256890842573;dx=-6.759196085367193E-4
New Minimum: -2.8980256890842573 > -8.808395403809014
END: th(27145.87709440174)=-8.808395403809014; dx=-4.9731317350983595E-5 evalInputDelta=5.910369714724757
Fitness changed from -2.8980256890842573 to -8.808395403809014
Iteration 4 complete. Error: -8.808395403809014 Total: 4.0677; Orientation: 0.4514; Line Search: 2.3532
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7b2b0662 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.122e+03, gradient 3.345e-02, dot -0.162; [65296433-eddd-4b75-aa9d-a5806d837aaf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.808395403809014, -1.1745279926489842, -0.6610125118101601, -0.11999729034225583
LBFGS Accumulation History: 3 points
Removed measurement 7b2b0662 to history. Total: 3
th(0)=-8.808395403809014;dx=-0.00111536430910207
New Minimum: -8.808395403809014 > -11.133363815844564
WOLF (strong): th(58484.01930352103)=-11.133363815844564; dx=5.286780845336715E-5 evalInputDelta=2.3249684120355507
New Minimum: -11.133363815844564 > -16.066117053834134
END: th(29242.009651760516)=-16.066117053834134; dx=-3.2708837449806874E-5 evalInputDelta=7.25772165002512
Fitness changed from -8.808395403809014 to -16.066117053834134
Iteration 5 complete. Error: -16.066117053834134 Total: 5.4231; Orientation: 0.5569; Line Search: 3.5808
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 34cf98c1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.690e+04, gradient 3.878e-02, dot -0.127; [65296433-eddd-4b75-aa9d-a5806d837aaf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.066117053834134, -1.1745279926489842, -0.6610125118101601, -0.11999729034225583
LBFGS Accumulation History: 3 points
Removed measurement 34cf98c1 to history. Total: 3
th(0)=-16.066117053834134;dx=-0.0014987122765974966
Armijo: th(63000.00000000002)=-16.030038293327525; dx=6.932584787043683E-5 evalInputDelta=-0.03607876050660863
New Minimum: -16.066117053834134 > -24.31726058991136
END: th(31500.00000000001)=-24.31726058991136; dx=-1.130297573415851E-5 evalInputDelta=8.251143536077226
Fitness changed from -16.066117053834134 to -24.31726058991136
Iteration 6 complete. Error: -24.31726058991136 Total: 5.8423; Orientation: 0.5224; Line Search: 4.0458
Adding measurement 766ddfe0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.267e+04, gradient 4.465e-02, dot -0.065; [65296433-eddd-4b75-aa9d-a5806d837aaf = 1.000/1.000e+00]
Orientation rejected. Popping history element from -24.31726058991136, -1.1745279926489842, -0.6610125118101601, -0.11999729034225583
LBFGS Accumulation History: 3 points
Removed measurement 766ddfe0 to history. Total: 3
th(0)=-24.31726058991136;dx=-0.0018467134436606024
Armijo: th(67864.69273600436)=-20.01966730136697; dx=1.3573105210780314E-4 evalInputDelta=-4.297593288544391
New Minimum: -24.31726058991136 > -34.58054315426162
WOLF (strong): th(33932.34636800218)=-34.58054315426162; dx=2.5728352708110363E-5 evalInputDelta=10.26328256435026
New Minimum: -34.58054315426162 > -37.51351644428478
END: th(11310.782122667393)=-37.51351644428478; dx=-5.938393200686472E-4 evalInputDelta=13.196255854373423
Fitness changed from -24.31726058991136 to -37.51351644428478
Iteration 7 complete. Error: -37.51351644428478 Total: 6.3771; Orientation: 0.4513; Line Search: 4.7942
Adding measurement 68424e42 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.833e+03, gradient 3.548e-02, dot 0.278; [65296433-eddd-4b75-aa9d-a5806d837aaf = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-37.51351644428478;dx=-27.969147139131554
Armijo: th(24368.3413764671)=-9.077655195728633; dx=8.143693649682606E-4 evalInputDelta=-28.43586124855615
Armijo: th(12184.17068823355)=-8.998830931286731; dx=0.0016540092184630637 evalInputDelta=-28.51468551299805
Armijo: th(4061.390229411183)=-8.646967679413873; dx=0.005031785308423908 evalInputDelta=-28.86654876487091
Armijo: th(1015.3475573527958)=-8.253100188397482; dx=0.020187836053103834 evalInputDelta=-29.2604162558873
Armijo: th(203.06951147055915)=-8.274806326360125; dx=0.10217866200803191 evalInputDelta=-29.238710117924658
Armijo: th(33.84491857842652)=-12.068440768104312; dx=0.5128852139973908 evalInputDelta=-25.445075676180473
Armijo: th(4.834988368346646)=-28.531204006329304; dx=1.7479468432846437 evalInputDelta=-8.982312437955478
New Minimum: -37.51351644428478 > -42.64727326561673
END: th(0.6043735460433307)=-42.64727326561673; dx=-9.145841234339336 evalInputDelta=5.13375682133195
Fitness changed from -37.51351644428478 to -42.64727326561673
Iteration 8 complete. Error: -42.64727326561673 Total: 12.3085; Orientation: 0.4401; Line Search: 10.7411
Adding measurement 5b91d573 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.561e+03, gradient 3.674e-02, dot 0.069; [65296433-eddd-4b75-aa9d-a5806d837aaf = 1.000/1.000e+00]
th(0)=-42.64727326561673;dx=-4.0646132426965345
Armijo: th(1.3020833333333337)=-35.35516469907102; dx=1.1360212642314904 evalInputDelta=-7.292108566545714
Armijo: th(0.6510416666666669)=-38.94024932609093; dx=0.6641178818363653 evalInputDelta=-3.7070239395258042
Armijo: th(0.21701388888888895)=-41.5551068377507; dx=-0.26612235184060107 evalInputDelta=-1.092166427866033
Armijo: th(0.05425347222222224)=-42.40046507382392; dx=-0.7567011890527547 evalInputDelta=-0.24680819179280888
Armijo: th(0.010850694444444448)=-42.60005742458718; dx=-1.0033638565495742 evalInputDelta=-0.047215841029554895
Armijo: th(0.0018084490740740745)=-42.63951995636648; dx=-1.0862037984548116 evalInputDelta=-0.007753309250254858
Armijo: th(2.5834986772486776E-4)=-42.64615980705722; dx=-1.0842770224131533 evalInputDelta=-0.0011134585595158342
Armijo: th(3.229373346560847E-5)=-42.647130149178125; dx=-1.0685402548485292 evalInputDelta=-1.431164386076489E-4
Armijo: th(3.58819260728983E-6)=-42.64725322934424; dx=-1.0824988085176441 evalInputDelta=-2.0036272495360663E-5
Armijo: th(3.5881926072898297E-7)=-42.64726897213056; dx=-1.071096426967834 evalInputDelta=-4.29348617103642E-6
Armijo: th(3.2619932793543904E-8)=-42.64727326561673; dx=-1.091467547338227 evalInputDelta=0.0
Armijo: th(2.7183277327953254E-9)=-42.64727326561673; dx=-1.0742501047538857 evalInputDelta=0.0
END: th(2.0910213329194811E-10)=-42.64727326561673; dx=-1.0846963364209536 evalInputDelta=0.0
Fitness changed from -42.64727326561673 to -42.64727326561673
Static Iteration Total: 18.8145; Orientation: 0.5498; Line Search: 17.1548
Iteration 9 failed. Error: -42.64727326561673
Previous Error: 0.0 -> -42.64727326561673
Retrying iteration 9
Reset training subject: 4986438825239
Adding measurement 7d060e3f to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-42.64727326561673;dx=-0.0012726266115559751
WOLFE (weak): th(4.504968897238439E-10)=-42.64727326561673; dx=-0.001217798691328942 evalInputDelta=0.0
WOLFE (weak): th(9.009937794476878E-10)=-42.64727326561673; dx=-0.0012178950978557319 evalInputDelta=0.0
WOLFE (weak): th(2.7029813383430632E-9)=-42.64727326561673; dx=-0.0012191003660856246 evalInputDelta=0.0
WOLFE (weak): th(1.0811925353372253E-8)=-42.64727326561673; dx=-0.0012174911905429519 evalInputDelta=0.0
WOLFE (weak): th(5.405962676686127E-8)=-42.64727326561673; dx=-0.0012174435496636335 evalInputDelta=0.0
WOLFE (weak): th(3.243577606011676E-7)=-42.64727326561673; dx=-0.0012175578550438506 evalInputDelta=0.0
WOLFE (weak): th(2.2705043242081733E-6)=-42.64727326561673; dx=-0.0012179301013567987 evalInputDelta=0.0
Armijo: th(1.8164034593665386E-5)=-42.64727326561673; dx=-0.0012175162597381926 evalInputDelta=0.0
Armijo: th(1.0217269458936779E-5)=-42.64727326561673; dx=-0.0012154645952184241 evalInputDelta=0.0
Armijo: th(6.243886891572476E-6)=-42.64727183445472; dx=-0.0012181535306532586 evalInputDelta=-1.4311620120111002E-6
Armijo: th(4.257195607890325E-6)=-42.64727326561673; dx=-0.0012135794766427524 evalInputDelta=0.0
Armijo: th(3.2638499660492492E-6)=-42.64727326561673; dx=-0.0012169490886144124 evalInputDelta=0.0
WOLFE (weak): th(2.7671771451287113E-6)=-42.64727326561673; dx=-0.0012185826343842775 evalInputDelta=0.0
Armijo: th(3.0155135555889805E-6)=-42.64727326561673; dx=-0.0012165935788813116 evalInputDelta=0.0
Armijo: th(2.8913453503588456E-6)=-42.64727326561673; dx=-0.0012173409660042962 evalInputDelta=0.0
Armijo: th(2.8292612477437787E-6)=-42.64727326561673; dx=-0.0012172382440944736 evalInputDelta=0.0
Armijo: th(2.798219196436245E-6)=-42.64727326561673; dx=-0.0012164598751561195 evalInputDelta=0.0
WOLFE (weak): th(2.7826981707824784E-6)=-42.64727326561673; dx=-0.001216654883690178 evalInputDelta=0.0
WOLFE (weak): th(2.790458683609362E-6)=-42.64727326561673; dx=-0.0012177463599214312 evalInputDelta=0.0
Armijo: th(2.7943389400228037E-6)=-42.64727326561673; dx=-0.00121661084464848 evalInputDelta=0.0
Armijo: th(2.792398811816083E-6)=-42.64727326561673; dx=-0.0012184541079115364 evalInputDelta=0.0
WOLFE (weak): th(2.7914287477127226E-6)=-42.64727326561673; dx=-0.0012170013903204276 evalInputDelta=0.0
Armijo: th(2.7919137797644028E-6)=-42.64727326561673; dx=-0.001217250149638693 evalInputDelta=0.0
Armijo: th(2.7916712637385627E-6)=-42.64727326561673; dx=-0.0012168242213805138 evalInputDelta=0.0
WOLFE (weak): th(2.7915500057256424E-6)=-42.64727326561673; dx=-0.0012181820087932404 evalInputDelta=0.0
WOLFE (weak): th(2.7916106347321023E-6)=-42.64727326561673; dx=-0.0012165300955508248 evalInputDelta=0.0
Armijo: th(2.7916409492353327E-6)=-42.64727326561673; dx=-0.0012177650024877747 evalInputDelta=0.0
WOLFE (weak): th(2.7916257919837175E-6)=-42.64727326561673; dx=-0.0012183835830422468 evalInputDelta=0.0
mu ~= nu (2.7916257919837175E-6): th(0.0)=-42.64727326561673
Fitness changed from -42.64727326561673 to -42.64727326561673
Static Iteration Total: 38.5362; Orientation: 0.0187; Line Search: 36.2540
Iteration 10 failed. Error: -42.64727326561673
Previous Error: 0.0 -> -42.64727326561673
Optimization terminated 10
Final threshold in iteration 10: -42.64727326561673 (> -Infinity) after 113.918s (< 3600.000s)
Returns
-42.64727326561673