BasicOptimizer.scala:89 executed in 11.37 seconds (0.833 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 450043269705800
Reset training subject: 450043643166500
Adding measurement 28d739f1 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.0397091184884035;dx=-8.041564271145116E-7
New Minimum: -1.0397091184884035 > -1.0397134665267145
WOLFE (weak): th(2.154434690031884)=-1.0397134665267145; dx=-8.053410039898562E-7 evalInputDelta=4.348038310952873E-6
New Minimum: -1.0397134665267145 > -1.039717782098998
WOLFE (weak): th(4.308869380063768)=-1.039717782098998; dx=-8.069298660693051E-7 evalInputDelta=8.663610594483728E-6
New Minimum: -1.039717782098998 > -1.0397352713401247
WOLFE (weak): th(12.926608140191302)=-1.0397352713401247; dx=-8.055006551866081E-7 evalInputDelta=2.615285172113957E-5
New Minimum: -1.0397352713401247 > -1.039813304165448
WOLFE (weak): th(51.70643256076521)=-1.039813304165448; dx=-8.074427151986368E-7 evalInputDelta=1.0418567704451043E-4
New Minimum: -1.039813304165448 > -1.0402305911621907
WOLFE (weak): th(258.53216280382605)=-1.0402305911621907; dx=-7.889964947802245E-7 evalInputDelta=5.214726737872155E-4
New Minimum: -1.0402305911621907 > -1.0428414190509647
WOLFE (weak): th(1551.1929768229563)=-1.0428414190509647; dx=-8.093876311862007E-7 evalInputDelta=0.0031323005625611966
New Minimum: -1.0428414190509647 > -1.0617611116822236
WOLFE (weak): th(10858.350837760694)=-1.0617611116822236; dx=-8.194334933430701E-7 evalInputDelta=0.02205199319382012
New Minimum: -1.0617611116822236 > -1.2180617577560504
WOLFE (weak): th(86866.80670208555)=-1.2180617577560504; dx=-8.288173907528687E-7 evalInputDelta=0.17835263926764688
New Minimum: -1.2180617577560504 > -2.0679844699689247
END: th(781801.26031877)=-2.0679844699689247; dx=-2.547066075272047E-7 evalInputDelta=1.0282753514805212
Fitness changed from -1.0397091184884035 to -2.0679844699689247
Iteration 1 complete. Error: -2.0679844699689247 Total: 2.3194; Orientation: 0.0260; Line Search: 1.5557
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 18b3db6e to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.0679844699689247;dx=-1.123586882627894E-6
New Minimum: -2.0679844699689247 > -2.9552212948500522
WOLF (strong): th(1684339.7559414052)=-2.9552212948500522; dx=4.174222167917212E-8 evalInputDelta=0.8872368248811275
New Minimum: -2.9552212948500522 > -3.0599971026514012
END: th(842169.8779707026)=-3.0599971026514012; dx=-8.684345026677938E-8 evalInputDelta=0.9920126326824765
Fitness changed from -2.0679844699689247 to -3.0599971026514012
Iteration 2 complete. Error: -3.0599971026514012 Total: 0.6058; Orientation: 0.0029; Line Search: 0.4455
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4e789704 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.0599971026514012;dx=-1.1081387464245064E-6
New Minimum: -3.0599971026514012 > -3.685232871712419
WOLF (strong): th(1814400.0)=-3.685232871712419; dx=1.3168311111372294E-7 evalInputDelta=0.625235769061018
New Minimum: -3.685232871712419 > -3.9686587486579956
WOLF (strong): th(907200.0)=-3.9686587486579956; dx=2.0247819314785553E-8 evalInputDelta=0.9086616460065944
END: th(302400.0)=-3.6791002345476653; dx=-6.094708101027847E-7 evalInputDelta=0.6191031318962641
Fitness changed from -3.0599971026514012 to -3.9686587486579956
Iteration 3 complete. Error: -3.9686587486579956 Total: 0.7651; Orientation: 0.0025; Line Search: 0.6067
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 15a8bbe5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.170e+03, gradient 9.961e-04, dot -0.455; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.9686587486579956, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 15a8bbe5 to history. Total: 3
th(0)=-3.9686587486579956;dx=-9.809835631573628E-7
New Minimum: -3.9686587486579956 > -4.830588873679461
END: th(651501.0502656417)=-4.830588873679461; dx=-1.70604495635814E-7 evalInputDelta=0.8619301250214653
Fitness changed from -3.9686587486579956 to -4.830588873679461
Iteration 4 complete. Error: -4.830588873679461 Total: 0.5560; Orientation: 0.0722; Line Search: 0.3154
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3599b18 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.665e+03, gradient 7.849e-04, dot -0.578; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.830588873679461, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 3599b18 to history. Total: 3
th(0)=-4.830588873679461;dx=-6.106138908998403E-7
New Minimum: -4.830588873679461 > -5.598154058259948
END: th(1403616.4632845044)=-5.598154058259948; dx=-3.894953086872708E-8 evalInputDelta=0.7675651845804872
Fitness changed from -4.830588873679461 to -5.598154058259948
Iteration 5 complete. Error: -5.598154058259948 Total: 0.5395; Orientation: 0.0533; Line Search: 0.3136
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 32675dd4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.758e+03, gradient 1.013e-03, dot -0.438; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.598154058259948, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 32675dd4 to history. Total: 3
th(0)=-5.598154058259948;dx=-1.016531927181624E-6
Armijo: th(3024000.0)=-5.2564416715422215; dx=2.47078297202391E-7 evalInputDelta=-0.3417123867177265
New Minimum: -5.598154058259948 > -6.350830231273601
WOLF (strong): th(1512000.0)=-6.350830231273601; dx=2.3330512184590515E-7 evalInputDelta=0.7526761730136533
New Minimum: -6.350830231273601 > -6.431020455571659
END: th(504000.0)=-6.431020455571659; dx=-3.9902993243766824E-7 evalInputDelta=0.8328663973117107
Fitness changed from -5.598154058259948 to -6.431020455571659
Iteration 6 complete. Error: -6.431020455571659 Total: 1.2768; Orientation: 0.0475; Line Search: 0.5518
Adding measurement 2b71a12e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.090e+04, gradient 6.874e-04, dot -0.653; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.431020455571659, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 2b71a12e to history. Total: 3
th(0)=-6.431020455571659;dx=-4.6682375122626865E-7
New Minimum: -6.431020455571659 > -7.241117123279172
END: th(1085835.0837760693)=-7.241117123279172; dx=-2.2784703308300306E-7 evalInputDelta=0.810096667707513
Fitness changed from -6.431020455571659 to -7.241117123279172
Iteration 7 complete. Error: -7.241117123279172 Total: 0.4391; Orientation: 0.0403; Line Search: 0.2690
Adding measurement 5a0b925a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.532e+04, gradient 7.390e-04, dot -0.623; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.241117123279172, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 5a0b925a to history. Total: 3
th(0)=-7.241117123279172;dx=-5.420832631583403E-7
New Minimum: -7.241117123279172 > -8.174743472896674
END: th(2339360.7721408405)=-8.174743472896674; dx=-2.407532151610183E-8 evalInputDelta=0.9336263496175023
Fitness changed from -7.241117123279172 to -8.174743472896674
Iteration 8 complete. Error: -8.174743472896674 Total: 0.4372; Orientation: 0.0420; Line Search: 0.2637
Adding measurement dfe2875 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.506e+04, gradient 1.015e-03, dot -0.410; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.174743472896674, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement dfe2875 to history. Total: 3
th(0)=-8.174743472896674;dx=-1.0219752467784998E-6
Armijo: th(5040000.0)=-7.314055664884096; dx=2.534988053753218E-7 evalInputDelta=-0.8606878080125782
New Minimum: -8.174743472896674 > -9.202412737889498
WOLF (strong): th(2520000.0)=-9.202412737889498; dx=2.869643412832564E-7 evalInputDelta=1.0276692649928236
New Minimum: -9.202412737889498 > -9.521003880933518
END: th(840000.0)=-9.521003880933518; dx=-3.7606794159146627E-7 evalInputDelta=1.3462604080368443
Fitness changed from -8.174743472896674 to -9.521003880933518
Iteration 9 complete. Error: -9.521003880933518 Total: 0.6878; Orientation: 0.0385; Line Search: 0.5223
Adding measurement 78b3f926 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.527e+04, gradient 6.956e-04, dot -0.628; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.521003880933518, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 78b3f926 to history. Total: 3
th(0)=-9.521003880933518;dx=-4.794354811285476E-7
New Minimum: -9.521003880933518 > -10.675982627631637
END: th(1809725.1396267824)=-10.675982627631637; dx=-1.4558253503251566E-7 evalInputDelta=1.1549787466981183
Fitness changed from -9.521003880933518 to -10.675982627631637
Iteration 10 complete. Error: -10.675982627631637 Total: 0.4275; Orientation: 0.0400; Line Search: 0.2578
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 135a8812 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.361e+04, gradient 7.994e-04, dot -0.553; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.675982627631637, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 135a8812 to history. Total: 3
th(0)=-10.675982627631637;dx=-6.335271434901935E-7
New Minimum: -10.675982627631637 > -11.086579366911304
WOLF (strong): th(3898934.6202347344)=-11.086579366911304; dx=1.4920409471266405E-7 evalInputDelta=0.4105967392796668
New Minimum: -11.086579366911304 > -11.58520507062296
WOLF (strong): th(1949467.3101173672)=-11.58520507062296; dx=4.773510349613899E-8 evalInputDelta=0.9092224429913234
END: th(649822.4367057891)=-11.338330326110768; dx=-2.9659722085587287E-7 evalInputDelta=0.662347698479131
Fitness changed from -10.675982627631637 to -11.58520507062296
Iteration 11 complete. Error: -11.58520507062296 Total: 0.7012; Orientation: 0.0381; Line Search: 0.5228
Adding measurement 19e3ce9a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.764e+04, gradient 9.165e-04, dot -0.439; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.58520507062296, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 19e3ce9a to history. Total: 3
th(0)=-11.58520507062296;dx=-8.325143734156014E-7
New Minimum: -11.58520507062296 > -12.906361963158966
END: th(1400000.0)=-12.906361963158966; dx=-1.3034286046591992E-8 evalInputDelta=1.3211568925360062
Fitness changed from -11.58520507062296 to -12.906361963158966
Iteration 12 complete. Error: -12.906361963158966 Total: 0.4264; Orientation: 0.0379; Line Search: 0.2577
Adding measurement 35ab67d4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.396e+04, gradient 7.252e-04, dot -0.546; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.906361963158966, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 35ab67d4 to history. Total: 3
th(0)=-12.906361963158966;dx=-5.188531739848373E-7
New Minimum: -12.906361963158966 > -13.77410139325553
WOLF (strong): th(3016208.5660446375)=-13.77410139325553; dx=1.1464844643782091E-7 evalInputDelta=0.8677394300965631
New Minimum: -13.77410139325553 > -13.860698984254272
END: th(1508104.2830223187)=-13.860698984254272; dx=-7.331085911208016E-8 evalInputDelta=0.9543370210953057
Fitness changed from -12.906361963158966 to -13.860698984254272
Iteration 13 complete. Error: -13.860698984254272 Total: 0.5816; Orientation: 0.0393; Line Search: 0.4107
Adding measurement 700e3b73 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.430e+04, gradient 6.975e-04, dot -0.554; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -13.860698984254272, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 700e3b73 to history. Total: 3
th(0)=-13.860698984254272;dx=-4.808579272573577E-7
New Minimum: -13.860698984254272 > -14.582830079584884
WOLF (strong): th(3249112.183528946)=-14.582830079584884; dx=1.093543524656731E-7 evalInputDelta=0.7221310953306119
New Minimum: -14.582830079584884 > -14.734715637132117
END: th(1624556.091764473)=-14.734715637132117; dx=-3.274361232030266E-8 evalInputDelta=0.8740166528778452
Fitness changed from -13.860698984254272 to -14.734715637132117
Iteration 14 complete. Error: -14.734715637132117 Total: 0.5508; Orientation: 0.0379; Line Search: 0.3871
Adding measurement 7e41e1e4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.101e+04, gradient 7.097e-04, dot -0.507; [e308549c-faac-4912-b5f0-0e7402bad036 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.734715637132117, -3.0599971026514012, -2.0679844699689247, -1.0397091184884035
LBFGS Accumulation History: 3 points
Removed measurement 7e41e1e4 to history. Total: 3
th(0)=-14.734715637132117;dx=-4.955851559264266E-7
New Minimum: -14.734715637132117 > -15.156083453172556
WOLF (strong): th(3500000.000000001)=-15.156083453172556; dx=1.6113204061977465E-7 evalInputDelta=0.4213678160404388
New Minimum: -15.156083453172556 > -15.566698299426598
WOLF (strong): th(1750000.0000000005)=-15.566698299426598; dx=2.496993896172033E-8 evalInputDelta=0.8319826622944806
END: th(583333.3333333335)=-15.251836928941936; dx=-2.9359281982424525E-7 evalInputDelta=0.5171212918098185
Fitness changed from -14.734715637132117 to -15.566698299426598
Iteration 15 complete. Error: -15.566698299426598 Total: 1.0240; Orientation: 0.0399; Line Search: 0.5102
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -15.566698299426598 (> -Infinity) after 11.351s (< 5400.000s)
Returns
-15.566698299426598