BasicOptimizer.scala:89 executed in 67.70 seconds (0.891 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 164214570380800
Reset training subject: 164215462919100
Adding measurement 1b8fbbfc to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-24.56522247978934;dx=-0.001627323350794401
New Minimum: -24.56522247978934 > -24.570104735360033
WOLFE (weak): th(2.154434690031884)=-24.570104735360033; dx=-0.0016196480209856715 evalInputDelta=0.004882255570691996
New Minimum: -24.570104735360033 > -24.574990456731445
WOLFE (weak): th(4.308869380063768)=-24.574990456731445; dx=-0.0016204761435436282 evalInputDelta=0.009767976942104184
New Minimum: -24.574990456731445 > -24.5945421042875
WOLFE (weak): th(12.926608140191302)=-24.5945421042875; dx=-0.0016198991927640423 evalInputDelta=0.029319624498157282
New Minimum: -24.5945421042875 > -24.68238328272898
WOLFE (weak): th(51.70643256076521)=-24.68238328272898; dx=-0.0016131394346436724 evalInputDelta=0.11716080293964026
New Minimum: -24.68238328272898 > -25.15160555133324
WOLFE (weak): th(258.53216280382605)=-25.15160555133324; dx=-0.0016123945065139853 evalInputDelta=0.5863830715438993
New Minimum: -25.15160555133324 > -28.08876583122791
WOLFE (weak): th(1551.1929768229563)=-28.08876583122791; dx=-0.0015851835065199041 evalInputDelta=3.523543351438569
New Minimum: -28.08876583122791 > -43.121616911204654
END: th(10858.350837760694)=-43.121616911204654; dx=-5.864632352503726E-4 evalInputDelta=18.556394431415313
Fitness changed from -24.56522247978934 to -43.121616911204654
Iteration 1 complete. Error: -43.121616911204654 Total: 9.5234; Orientation: 0.0145; Line Search: 6.9899
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 45d5900e to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-43.121616911204654;dx=-0.0018620743969883099
New Minimum: -43.121616911204654 > -55.44761311798879
WOLF (strong): th(23393.607721408407)=-55.44761311798879; dx=1.273602677204938E-4 evalInputDelta=12.325996206784133
New Minimum: -55.44761311798879 > -56.70002120748309
END: th(11696.803860704204)=-56.70002120748309; dx=-2.1064829884300535E-4 evalInputDelta=13.578404296278435
Fitness changed from -43.121616911204654 to -56.70002120748309
Iteration 2 complete. Error: -56.70002120748309 Total: 3.4099; Orientation: 0.0148; Line Search: 2.5109
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3e9e0598 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-56.70002120748309;dx=-0.0016739289655583588
New Minimum: -56.70002120748309 > -67.66896237711713
WOLF (strong): th(25200.000000000004)=-67.66896237711713; dx=2.463212828055029E-4 evalInputDelta=10.968941169634036
New Minimum: -67.66896237711713 > -70.60314372571604
END: th(12600.000000000002)=-70.60314372571604; dx=-2.795549418897997E-4 evalInputDelta=13.903122518232948
Fitness changed from -56.70002120748309 to -70.60314372571604
Iteration 3 complete. Error: -70.60314372571604 Total: 3.3653; Orientation: 0.0142; Line Search: 2.4472
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 6138444 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.673e+03, gradient 3.516e-02, dot -0.504; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.60314372571604, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 6138444 to history. Total: 3
th(0)=-70.60314372571604;dx=-0.0012356874994279817
New Minimum: -70.60314372571604 > -84.19166492001172
WOLF (strong): th(27145.87709440174)=-84.19166492001172; dx=2.8586720254785664E-5 evalInputDelta=13.58852119429568
END: th(13572.93854720087)=-83.20829837080417; dx=-4.1041472009824937E-4 evalInputDelta=12.605154645088135
Fitness changed from -70.60314372571604 to -84.19166492001172
Iteration 4 complete. Error: -84.19166492001172 Total: 3.8531; Orientation: 0.2727; Line Search: 2.5220
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2097b65d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.237e+03, gradient 4.399e-02, dot -0.409; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.19166492001172, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 2097b65d to history. Total: 3
th(0)=-84.19166492001172;dx=-0.0019331533393889555
New Minimum: -84.19166492001172 > -96.88289453064027
WOLF (strong): th(29242.009651760516)=-96.88289453064027; dx=3.5866920772763245E-4 evalInputDelta=12.691229610628554
New Minimum: -96.88289453064027 > -102.16114116226007
END: th(14621.004825880258)=-102.16114116226007; dx=-2.4271573467708253E-4 evalInputDelta=17.969476242248348
Fitness changed from -84.19166492001172 to -102.16114116226007
Iteration 5 complete. Error: -102.16114116226007 Total: 3.6380; Orientation: 0.2719; Line Search: 2.4782
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3f9efb1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.199e+03, gradient 3.206e-02, dot -0.474; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -102.16114116226007, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 3f9efb1a to history. Total: 3
th(0)=-102.16114116226007;dx=-0.001026331704649744
New Minimum: -102.16114116226007 > -121.39701131423139
END: th(31500.00000000001)=-121.39701131423139; dx=-1.7612970159383324E-4 evalInputDelta=19.235870151971326
Fitness changed from -102.16114116226007 to -121.39701131423139
Iteration 6 complete. Error: -121.39701131423139 Total: 3.0636; Orientation: 0.2776; Line Search: 1.9074
Adding measurement 15f97cfe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.308e+03, gradient 3.685e-02, dot -0.449; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -121.39701131423139, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 15f97cfe to history. Total: 3
th(0)=-121.39701131423139;dx=-0.0013572174336136831
New Minimum: -121.39701131423139 > -124.97004459058049
WOLF (strong): th(67864.69273600436)=-124.97004459058049; dx=1.2674434346873206E-4 evalInputDelta=3.5730332763490935
New Minimum: -124.97004459058049 > -136.36074048652196
WOLF (strong): th(33932.34636800218)=-136.36074048652196; dx=9.287727228987969E-5 evalInputDelta=14.96372917229057
END: th(11310.782122667393)=-135.13682861324807; dx=-5.274975655006068E-4 evalInputDelta=13.739817299016678
Fitness changed from -121.39701131423139 to -136.36074048652196
Iteration 7 complete. Error: -136.36074048652196 Total: 4.4983; Orientation: 0.2700; Line Search: 3.4052
Adding measurement 1c1ece02 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.639e+03, gradient 4.224e-02, dot -0.323; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -136.36074048652196, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 1c1ece02 to history. Total: 3
th(0)=-136.36074048652196;dx=-0.0017765188573653167
New Minimum: -136.36074048652196 > -161.6890608169532
END: th(24368.3413764671)=-161.6890608169532; dx=-1.405603684430986E-4 evalInputDelta=25.32832033043124
Fitness changed from -136.36074048652196 to -161.6890608169532
Iteration 8 complete. Error: -161.6890608169532 Total: 2.8127; Orientation: 0.2687; Line Search: 1.7238
Adding measurement 78e5316e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.148e+04, gradient 3.150e-02, dot -0.484; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -161.6890608169532, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 78e5316e to history. Total: 3
th(0)=-161.6890608169532;dx=-9.896923825036964E-4
New Minimum: -161.6890608169532 > -186.84457558685412
END: th(52500.00000000002)=-186.84457558685412; dx=-8.0551478667009E-5 evalInputDelta=25.15551476990092
Fitness changed from -161.6890608169532 to -186.84457558685412
Iteration 9 complete. Error: -186.84457558685412 Total: 2.8220; Orientation: 0.2519; Line Search: 1.6467
Adding measurement e973f0b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.700e+04, gradient 3.862e-02, dot -0.387; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -186.84457558685412, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement e973f0b to history. Total: 3
th(0)=-186.84457558685412;dx=-0.0014899767485925149
Armijo: th(113107.82122667394)=-168.3429075994261; dx=1.892846735472334E-4 evalInputDelta=-18.50166798742802
New Minimum: -186.84457558685412 > -201.5964990087509
WOLF (strong): th(56553.91061333697)=-201.5964990087509; dx=2.2064646811975554E-4 evalInputDelta=14.751923421896777
New Minimum: -201.5964990087509 > -209.1432843238063
END: th(18851.30353777899)=-209.1432843238063; dx=-4.2714349201122706E-4 evalInputDelta=22.298708736952193
Fitness changed from -186.84457558685412 to -209.1432843238063
Iteration 10 complete. Error: -209.1432843238063 Total: 4.4011; Orientation: 0.2638; Line Search: 3.3521
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 109018ba to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.043e+05, gradient 2.862e-02, dot -0.555; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -209.1432843238063, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 109018ba to history. Total: 3
th(0)=-209.1432843238063;dx=-8.189025819682095E-4
New Minimum: -209.1432843238063 > -232.24108498778367
END: th(40613.90229411183)=-232.24108498778367; dx=-3.8812334881160805E-4 evalInputDelta=23.097800663977353
Fitness changed from -209.1432843238063 to -232.24108498778367
Iteration 11 complete. Error: -232.24108498778367 Total: 2.8987; Orientation: 0.2777; Line Search: 1.7316
Adding measurement 5ff832a2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.848e+04, gradient 2.909e-02, dot -0.479; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -232.24108498778367, -56.70002120748309, -43.121616911204654, -24.56522247978934
LBFGS Accumulation History: 3 points
Removed measurement 5ff832a2 to history. Total: 3
th(0)=-232.24108498778367;dx=-8.460161308584432E-4
New Minimum: -232.24108498778367 > -265.3378220886344
END: th(87500.00000000004)=-265.3378220886344; dx=-1.9502063037946652E-4 evalInputDelta=33.09673710085076
Fitness changed from -232.24108498778367 to -265.3378220886344
Iteration 12 complete. Error: -265.3378220886344 Total: 2.8201; Orientation: 0.2529; Line Search: 1.6690
Adding measurement 76a3ccb3 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.454e+03, gradient 3.547e-02, dot 0.002; [8a31a030-74ad-463f-9236-3d83d46fa086 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-265.3378220886344;dx=-0.192387954286791
Armijo: th(188513.03537778993)=-8.739927528038635; dx=2.6142426700181246E-5 evalInputDelta=-256.5978945605958
Armijo: th(94256.51768889496)=-8.724813163197373; dx=5.188134006263117E-5 evalInputDelta=-256.61300892543704
Armijo: th(31418.839229631656)=-8.71772145765686; dx=1.5928641917145608E-4 evalInputDelta=-256.62010063097756
Armijo: th(7854.709807407914)=-8.781157787059502; dx=6.444377061185808E-4 evalInputDelta=-256.5566643015749
Armijo: th(1570.941961481583)=-8.795123700347126; dx=0.0029230825010965657 evalInputDelta=-256.5426983882873
Armijo: th(261.8236602469305)=-8.074232140246888; dx=0.014751753906045169 evalInputDelta=-257.26358994838756
Armijo: th(37.40338003527579)=-8.497682134233584; dx=0.02907029660564916 evalInputDelta=-256.84013995440085
Armijo: th(4.6754225044094735)=-7.394162757576986; dx=1.4263845100299994 evalInputDelta=-257.94365933105746
Armijo: th(0.5194913893788304)=-216.98929679156905; dx=95.97727225811107 evalInputDelta=-48.34852529706538
Armijo: th(0.05194913893788304)=-264.71241483846455; dx=8.288480686386254 evalInputDelta=-0.6254072501698715
Armijo: th(0.004722648994353004)=-265.31037949322683; dx=1.0513442620927376 evalInputDelta=-0.027442595407592307
Armijo: th(3.935540828627503E-4)=-265.335788751908; dx=0.39473710920795435 evalInputDelta=-0.0020333367264129265
Armijo: th(3.027339098944233E-5)=-265.3376818590156; dx=0.32520293320898164 evalInputDelta=-1.4022961880755247E-4
Armijo: th(2.162385070674452E-6)=-265.3378220886344; dx=0.3086465850300686 evalInputDelta=0.0
WOLF (strong): th(1.4415900471163015E-7)=-265.3378220886344; dx=0.3284208029882274 evalInputDelta=0.0
WOLF (strong): th(9.009937794476885E-9)=-265.3378220886344; dx=0.297949912974953 evalInputDelta=0.0
WOLF (strong): th(5.299963408515815E-10)=-265.3378220886344; dx=0.32723443784388473 evalInputDelta=0.0
MIN ALPHA (2.9444241158421194E-11): th(0.0)=-265.3378220886344
Fitness changed from -265.3378220886344 to -265.3378220886344
Static Iteration Total: 17.1048; Orientation: 0.2664; Line Search: 16.0355
Iteration 13 failed. Error: -265.3378220886344
Previous Error: 0.0 -> -265.3378220886344
Retrying iteration 13
Reset training subject: 164278781508500
Adding measurement 3a6d0260 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-265.3378220886344;dx=-0.0012579230520826786
END: th(2.1544346900318838E-10)=-265.3378220886344; dx=-9.838199189845921E-4 evalInputDelta=0.0
Fitness changed from -265.3378220886344 to -265.3378220886344
Static Iteration Total: 3.4841; Orientation: 0.0141; Line Search: 1.6895
Iteration 14 failed. Error: -265.3378220886344
Previous Error: 0.0 -> -265.3378220886344
Optimization terminated 14
Final threshold in iteration 14: -265.3378220886344 (> -Infinity) after 67.695s (< 5400.000s)
Returns
-265.3378220886344