BasicOptimizer.scala:89 executed in 66.85 seconds (1.295 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 675460719242700
Reset training subject: 675461831368500
Adding measurement 38cd4134 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.154576985078066;dx=-3.667083677618374E-6
New Minimum: -2.154576985078066 > -2.154586608880021
WOLFE (weak): th(2.154434690031884)=-2.154586608880021; dx=-3.67235588544044E-6 evalInputDelta=9.62380195534962E-6
New Minimum: -2.154586608880021 > -2.1545962326389905
WOLFE (weak): th(4.308869380063768)=-2.1545962326389905; dx=-3.6732913363330173E-6 evalInputDelta=1.9247560924640084E-5
New Minimum: -2.1545962326389905 > -2.1546342528438487
WOLFE (weak): th(12.926608140191302)=-2.1546342528438487; dx=-3.6780640975748085E-6 evalInputDelta=5.726776578285353E-5
New Minimum: -2.1546342528438487 > -2.1548063180740296
WOLFE (weak): th(51.70643256076521)=-2.1548063180740296; dx=-3.6759413473981898E-6 evalInputDelta=2.2933299596372692E-4
New Minimum: -2.1548063180740296 > -2.155722660974747
WOLFE (weak): th(258.53216280382605)=-2.155722660974747; dx=-3.66458497566554E-6 evalInputDelta=0.0011456758966810199
New Minimum: -2.155722660974747 > -2.1614559159578377
WOLFE (weak): th(1551.1929768229563)=-2.1614559159578377; dx=-3.657891295640237E-6 evalInputDelta=0.006878930879771872
New Minimum: -2.1614559159578377 > -2.202678244846652
WOLFE (weak): th(10858.350837760694)=-2.202678244846652; dx=-3.634732217913425E-6 evalInputDelta=0.048101259768586146
New Minimum: -2.202678244846652 > -2.519606804321931
END: th(86866.80670208555)=-2.519606804321931; dx=-3.171798589417283E-6 evalInputDelta=0.365029819243865
Fitness changed from -2.154576985078066 to -2.519606804321931
Iteration 1 complete. Error: -2.519606804321931 Total: 12.1435; Orientation: 0.0139; Line Search: 9.0707
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 3ca370ae to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.519606804321931;dx=-3.3710193657073105E-6
New Minimum: -2.519606804321931 > -3.1487434917914823
END: th(187148.86177126726)=-3.1487434917914823; dx=-2.481873548094805E-6 evalInputDelta=0.6291366874695514
Fitness changed from -2.519606804321931 to -3.1487434917914823
Iteration 2 complete. Error: -3.1487434917914823 Total: 3.0610; Orientation: 0.0125; Line Search: 1.9246
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 12ef753b to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.1487434917914823;dx=-2.97465396517228E-6
New Minimum: -3.1487434917914823 > -4.003910801970039
END: th(403200.00000000006)=-4.003910801970039; dx=-1.5978038638042234E-6 evalInputDelta=0.8551673101785568
Fitness changed from -3.1487434917914823 to -4.003910801970039
Iteration 3 complete. Error: -4.003910801970039 Total: 3.0366; Orientation: 0.0125; Line Search: 1.9236
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 9dffb0a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.463e+03, gradient 1.743e-03, dot -0.476; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.003910801970039, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 9dffb0a to history. Total: 3
th(0)=-4.003910801970039;dx=-3.006706764728442E-6
New Minimum: -4.003910801970039 > -4.764796424843495
END: th(868668.0670208557)=-4.764796424843495; dx=-8.39218304477132E-7 evalInputDelta=0.760885622873456
Fitness changed from -4.003910801970039 to -4.764796424843495
Iteration 4 complete. Error: -4.764796424843495 Total: 3.5190; Orientation: 0.2371; Line Search: 2.0269
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2a5f8c53 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.719e+03, gradient 1.891e-03, dot -0.240; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.764796424843495, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 2a5f8c53 to history. Total: 3
th(0)=-4.764796424843495;dx=-3.5600660585333694E-6
New Minimum: -4.764796424843495 > -5.417194249395673
END: th(1871488.617712673)=-5.417194249395673; dx=-3.353220777874382E-7 evalInputDelta=0.6523978245521782
Fitness changed from -4.764796424843495 to -5.417194249395673
Iteration 5 complete. Error: -5.417194249395673 Total: 3.4205; Orientation: 0.2353; Line Search: 2.1124
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 57104caa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.161e+03, gradient 1.740e-03, dot -0.174; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.417194249395673, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 57104caa to history. Total: 3
th(0)=-5.417194249395673;dx=-3.002504805611193E-6
New Minimum: -5.417194249395673 > -6.31201417356217
END: th(4032000.0000000014)=-6.31201417356217; dx=-1.9243972522411576E-7 evalInputDelta=0.8948199241664971
Fitness changed from -5.417194249395673 to -6.31201417356217
Iteration 6 complete. Error: -6.31201417356217 Total: 3.2987; Orientation: 0.2419; Line Search: 2.0201
Adding measurement 8785f98 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.056e+03, gradient 1.492e-03, dot -0.165; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.31201417356217, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 8785f98 to history. Total: 3
th(0)=-6.31201417356217;dx=-2.2211466531714576E-6
New Minimum: -6.31201417356217 > -8.407358263737603
END: th(8686680.670208558)=-8.407358263737603; dx=-8.843155178838832E-8 evalInputDelta=2.0953440901754323
Fitness changed from -6.31201417356217 to -8.407358263737603
Iteration 7 complete. Error: -8.407358263737603 Total: 3.2290; Orientation: 0.2308; Line Search: 2.0522
Adding measurement 39f9c538 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.121e+04, gradient 1.231e-03, dot -0.134; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.407358263737603, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 39f9c538 to history. Total: 3
th(0)=-8.407358263737603;dx=-1.5107113064987698E-6
New Minimum: -8.407358263737603 > -10.977272089306817
WOLF (strong): th(1.871488617712673E7)=-10.977272089306817; dx=3.0111287056243564E-11 evalInputDelta=2.5699138255692144
New Minimum: -10.977272089306817 > -14.434998217158666
END: th(9357443.088563366)=-14.434998217158666; dx=-1.2089551376960437E-7 evalInputDelta=6.027639953421064
Fitness changed from -8.407358263737603 to -14.434998217158666
Iteration 8 complete. Error: -14.434998217158666 Total: 4.2741; Orientation: 0.2674; Line Search: 3.0216
Adding measurement 50a48a40 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.336e+04, gradient 1.120e-03, dot -0.129; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -14.434998217158666, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 50a48a40 to history. Total: 3
th(0)=-14.434998217158666;dx=-1.2424657870054278E-6
New Minimum: -14.434998217158666 > -15.089872957506355
WOLF (strong): th(2.0160000000000007E7)=-15.089872957506355; dx=1.9737417090881504E-8 evalInputDelta=0.6548747403476884
New Minimum: -15.089872957506355 > -19.268406753456446
END: th(1.0080000000000004E7)=-19.268406753456446; dx=-7.835740462782262E-8 evalInputDelta=4.83340853629778
Fitness changed from -14.434998217158666 to -19.268406753456446
Iteration 9 complete. Error: -19.268406753456446 Total: 4.0733; Orientation: 0.2248; Line Search: 2.9073
Adding measurement 4c743b01 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.800e+04, gradient 1.139e-03, dot -0.125; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.268406753456446, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 4c743b01 to history. Total: 3
th(0)=-19.268406753456446;dx=-1.282686683666305E-6
Armijo: th(2.1716701675521396E7)=-17.840609263613526; dx=5.599872936028424E-8 evalInputDelta=-1.42779748984292
New Minimum: -19.268406753456446 > -23.475221570308314
END: th(1.0858350837760698E7)=-23.475221570308314; dx=-1.4340177540154209E-8 evalInputDelta=4.2068148168518675
Fitness changed from -19.268406753456446 to -23.475221570308314
Iteration 10 complete. Error: -23.475221570308314 Total: 4.1925; Orientation: 0.2352; Line Search: 2.9710
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 654a5635 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.409e+04, gradient 1.103e-03, dot -0.125; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.475221570308314, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 654a5635 to history. Total: 3
th(0)=-23.475221570308314;dx=-1.2129745868944105E-6
Armijo: th(2.3393607721408416E7)=-20.691981636005; dx=7.296136711225924E-8 evalInputDelta=-2.783239934303314
New Minimum: -23.475221570308314 > -27.231700936730153
END: th(1.1696803860704208E7)=-27.231700936730153; dx=-8.392301260304928E-10 evalInputDelta=3.756479366421839
Fitness changed from -23.475221570308314 to -27.231700936730153
Iteration 11 complete. Error: -27.231700936730153 Total: 4.3290; Orientation: 0.2213; Line Search: 2.9270
Adding measurement 611f9bcc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.627e+04, gradient 1.084e-03, dot -0.113; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.231700936730153, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 611f9bcc to history. Total: 3
th(0)=-27.231700936730153;dx=-1.1698447282144273E-6
Armijo: th(2.520000000000001E7)=-22.92559170327728; dx=7.094807972294441E-8 evalInputDelta=-4.306109233452872
New Minimum: -27.231700936730153 > -30.558797792437325
WOLF (strong): th(1.2600000000000006E7)=-30.558797792437325; dx=3.2712031656278535E-8 evalInputDelta=3.327096855707172
New Minimum: -30.558797792437325 > -33.005549384231024
END: th(4200000.000000002)=-33.005549384231024; dx=-5.069355969383866E-7 evalInputDelta=5.773848447500871
Fitness changed from -27.231700936730153 to -33.005549384231024
Iteration 12 complete. Error: -33.005549384231024 Total: 5.0682; Orientation: 0.2259; Line Search: 3.9062
Adding measurement 1c21d3de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.540e+04, gradient 8.816e-04, dot -0.191; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.005549384231024, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 1c21d3de to history. Total: 3
th(0)=-33.005549384231024;dx=-7.715751945729443E-7
New Minimum: -33.005549384231024 > -36.11166112177544
END: th(9048625.698133916)=-36.11166112177544; dx=-2.114814635726338E-7 evalInputDelta=3.106111737544417
Fitness changed from -33.005549384231024 to -36.11166112177544
Iteration 13 complete. Error: -36.11166112177544 Total: 3.4557; Orientation: 0.2206; Line Search: 2.1404
Adding measurement 56e3aed9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.542e+04, gradient 9.817e-04, dot -0.177; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.11166112177544, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 56e3aed9 to history. Total: 3
th(0)=-36.11166112177544;dx=-9.60217507061705E-7
Armijo: th(1.949467310117368E7)=-31.09736337806841; dx=5.1792905232719515E-8 evalInputDelta=-5.014297743707033
New Minimum: -36.11166112177544 > -36.54303622102103
END: th(9747336.55058684)=-36.54303622102103; dx=-4.120419987538256E-8 evalInputDelta=0.43137509924559225
Fitness changed from -36.11166112177544 to -36.54303622102103
Iteration 14 complete. Error: -36.54303622102103 Total: 4.3875; Orientation: 0.2591; Line Search: 3.0189
Adding measurement 786f78c3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.297e+04, gradient 1.022e-03, dot -0.135; [744d5b14-4f88-4bcf-afb2-eef0c14dba41 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.54303622102103, -3.1487434917914823, -2.519606804321931, -2.154576985078066
LBFGS Accumulation History: 3 points
Removed measurement 786f78c3 to history. Total: 3
th(0)=-36.54303622102103;dx=-1.0409557786044785E-6
Armijo: th(2.100000000000001E7)=-30.438612970628363; dx=1.1715079651919557E-7 evalInputDelta=-6.104423250392671
New Minimum: -36.54303622102103 > -37.93028382093332
WOLF (strong): th(1.0500000000000006E7)=-37.93028382093332; dx=6.413919930432443E-8 evalInputDelta=1.3872475999122855
New Minimum: -37.93028382093332 > -41.106787831532266
END: th(3500000.000000002)=-41.106787831532266; dx=-4.319713966983355E-7 evalInputDelta=4.563751610511233
Fitness changed from -36.54303622102103 to -41.106787831532266
Iteration 15 complete. Error: -41.106787831532266 Total: 5.2657; Orientation: 0.2272; Line Search: 4.0981
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -41.106787831532266 (> -Infinity) after 66.847s (< 5400.000s)
Returns
-41.106787831532266