BasicOptimizer.scala:89 executed in 71.24 seconds (1.278 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 141975195366700
Reset training subject: 141976194500100
Adding measurement 409765eb to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.9428362708864468;dx=-1.533720841405251E-6
New Minimum: -0.9428362708864468 > -0.9428389214784965
WOLFE (weak): th(2.154434690031884)=-0.9428389214784965; dx=-1.5049742532987478E-6 evalInputDelta=2.6505920496244784E-6
New Minimum: -0.9428389214784965 > -0.9428417824266323
WOLFE (weak): th(4.308869380063768)=-0.9428417824266323; dx=-1.5040975967942961E-6 evalInputDelta=5.511540185465513E-6
New Minimum: -0.9428417824266323 > -0.9428529736991778
WOLFE (weak): th(12.926608140191302)=-0.9428529736991778; dx=-1.5014331278931642E-6 evalInputDelta=1.670281273091767E-5
New Minimum: -0.9428529736991778 > -0.9429026175923461
WOLFE (weak): th(51.70643256076521)=-0.9429026175923461; dx=-1.4957018927656697E-6 evalInputDelta=6.634670589922287E-5
New Minimum: -0.9429026175923461 > -0.9431678736283566
WOLFE (weak): th(258.53216280382605)=-0.9431678736283566; dx=-1.5045053671938086E-6 evalInputDelta=3.3160274190979777E-4
New Minimum: -0.9431678736283566 > -0.94482758432873
WOLFE (weak): th(1551.1929768229563)=-0.94482758432873; dx=-1.504344777210839E-6 evalInputDelta=0.0019913134422832046
New Minimum: -0.94482758432873 > -0.956847220143049
WOLFE (weak): th(10858.350837760694)=-0.956847220143049; dx=-1.5055190265410188E-6 evalInputDelta=0.014010949256602179
New Minimum: -0.956847220143049 > -1.0583064335856254
WOLFE (weak): th(86866.80670208555)=-1.0583064335856254; dx=-1.5701067172925988E-6 evalInputDelta=0.11547016269917854
New Minimum: -1.0583064335856254 > -1.9761691111456385
END: th(781801.26031877)=-1.9761691111456385; dx=-1.2093628791664436E-6 evalInputDelta=1.0333328402591917
Fitness changed from -0.9428362708864468 to -1.9761691111456385
Iteration 1 complete. Error: -1.9761691111456385 Total: 11.5598; Orientation: 0.0145; Line Search: 8.8811
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 512e9908 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.9761691111456385;dx=-2.587053953214031E-6
New Minimum: -1.9761691111456385 > -4.350855879584981
END: th(1684339.7559414052)=-4.350855879584981; dx=-7.652416626177582E-7 evalInputDelta=2.3746867684393425
Fitness changed from -1.9761691111456385 to -4.350855879584981
Iteration 2 complete. Error: -4.350855879584981 Total: 2.6298; Orientation: 0.0145; Line Search: 1.7093
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 4a79dbd8 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-4.350855879584981;dx=-3.155145734087477E-6
New Minimum: -4.350855879584981 > -5.986368460900612
END: th(3628800.0)=-5.986368460900612; dx=-2.145091733545412E-7 evalInputDelta=1.6355125813156306
Fitness changed from -4.350855879584981 to -5.986368460900612
Iteration 3 complete. Error: -5.986368460900612 Total: 2.7262; Orientation: 0.0148; Line Search: 1.7808
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5ce6911a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.283e+04, gradient 1.715e-03, dot -0.159; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.986368460900612, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 5ce6911a to history. Total: 3
th(0)=-5.986368460900612;dx=-2.9299414215035686E-6
New Minimum: -5.986368460900612 > -7.053125949398568
WOLF (strong): th(7818012.6031877)=-7.053125949398568; dx=1.812948491690065E-8 evalInputDelta=1.0667574884979567
New Minimum: -7.053125949398568 > -9.036722670725338
END: th(3909006.30159385)=-9.036722670725338; dx=-1.4493413521177418E-7 evalInputDelta=3.050354209824726
Fitness changed from -5.986368460900612 to -9.036722670725338
Iteration 4 complete. Error: -9.036722670725338 Total: 3.9135; Orientation: 0.2821; Line Search: 2.7031
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1c435b10 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.569e+04, gradient 1.628e-03, dot -0.111; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -9.036722670725338, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 1c435b10 to history. Total: 3
th(0)=-9.036722670725338;dx=-2.6399382354951295E-6
New Minimum: -9.036722670725338 > -9.567371607164947
WOLF (strong): th(8421698.779707026)=-9.567371607164947; dx=9.35636821545993E-8 evalInputDelta=0.5306489364396096
New Minimum: -9.567371607164947 > -12.516025284497738
END: th(4210849.389853513)=-12.516025284497738; dx=-1.4434614215924609E-7 evalInputDelta=3.4793026137724006
Fitness changed from -9.036722670725338 to -12.516025284497738
Iteration 5 complete. Error: -12.516025284497738 Total: 3.7366; Orientation: 0.2781; Line Search: 2.5398
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 55ce4c2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.807e+04, gradient 1.656e-03, dot -0.139; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.516025284497738, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 55ce4c2 to history. Total: 3
th(0)=-12.516025284497738;dx=-2.7289724706929123E-6
Armijo: th(9072000.0)=-11.061199707348267; dx=1.6195510190843708E-7 evalInputDelta=-1.454825577149471
New Minimum: -12.516025284497738 > -15.000301834740252
END: th(4536000.0)=-15.000301834740252; dx=-2.2782611705953434E-8 evalInputDelta=2.484276550242514
Fitness changed from -12.516025284497738 to -15.000301834740252
Iteration 6 complete. Error: -15.000301834740252 Total: 3.7971; Orientation: 0.2759; Line Search: 2.6211
Adding measurement 39705fb8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.857e+04, gradient 1.680e-03, dot -0.107; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.000301834740252, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 39705fb8 to history. Total: 3
th(0)=-15.000301834740252;dx=-2.7953054677701594E-6
Armijo: th(9772515.753984625)=-12.311085496717064; dx=2.9470818057922573E-7 evalInputDelta=-2.689216338023188
New Minimum: -15.000301834740252 > -17.907649261796227
WOLF (strong): th(4886257.876992312)=-17.907649261796227; dx=1.4743085467301137E-7 evalInputDelta=2.9073474270559743
New Minimum: -17.907649261796227 > -18.39925334177124
END: th(1628752.625664104)=-18.39925334177124; dx=-1.5055419338044964E-6 evalInputDelta=3.3989515070309864
Fitness changed from -15.000301834740252 to -18.39925334177124
Iteration 7 complete. Error: -18.39925334177124 Total: 4.5944; Orientation: 0.2648; Line Search: 3.3844
Adding measurement 5cf146b4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.941e+04, gradient 1.522e-03, dot -0.137; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -18.39925334177124, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 5cf146b4 to history. Total: 3
th(0)=-18.39925334177124;dx=-2.310558262292682E-6
New Minimum: -18.39925334177124 > -20.22685189209194
END: th(3509041.158211261)=-20.22685189209194; dx=-3.279397154094378E-7 evalInputDelta=1.827598550320701
Fitness changed from -18.39925334177124 to -20.22685189209194
Iteration 8 complete. Error: -20.22685189209194 Total: 2.8292; Orientation: 0.2752; Line Search: 1.7415
Adding measurement 6ba2fe5d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.276e+05, gradient 1.715e-03, dot -0.156; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.22685189209194, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 6ba2fe5d to history. Total: 3
th(0)=-20.22685189209194;dx=-2.930910382916979E-6
Armijo: th(7560000.000000001)=-17.83178972325016; dx=2.0468033378581207E-7 evalInputDelta=-2.395062168841779
New Minimum: -20.22685189209194 > -21.735569728442577
WOLF (strong): th(3780000.0000000005)=-21.735569728442577; dx=1.3574120638876253E-7 evalInputDelta=1.5087178363506375
New Minimum: -21.735569728442577 > -22.749911993244943
END: th(1260000.0000000002)=-22.749911993244943; dx=-1.361116676828266E-6 evalInputDelta=2.5230601011530034
Fitness changed from -20.22685189209194 to -22.749911993244943
Iteration 9 complete. Error: -22.749911993244943 Total: 4.6690; Orientation: 0.2806; Line Search: 3.5749
Adding measurement 519b83e3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.205e+05, gradient 1.493e-03, dot -0.193; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.749911993244943, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 519b83e3 to history. Total: 3
th(0)=-22.749911993244943;dx=-2.224363644359442E-6
New Minimum: -22.749911993244943 > -23.957319078357607
END: th(2714587.709440174)=-23.957319078357607; dx=-4.2461737438827837E-7 evalInputDelta=1.2074070851126635
Fitness changed from -22.749911993244943 to -23.957319078357607
Iteration 10 complete. Error: -23.957319078357607 Total: 2.8102; Orientation: 0.2727; Line Search: 1.7113
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 6260e9b3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.233e+05, gradient 1.739e-03, dot -0.167; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -23.957319078357607, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 6260e9b3 to history. Total: 3
th(0)=-23.957319078357607;dx=-3.0047967535282195E-6
Armijo: th(5848401.930352103)=-21.722875779370163; dx=1.526194233682622E-7 evalInputDelta=-2.2344432989874434
New Minimum: -23.957319078357607 > -24.713731510797277
WOLF (strong): th(2924200.9651760515)=-24.713731510797277; dx=1.9841344636599594E-7 evalInputDelta=0.7564124324396708
New Minimum: -24.713731510797277 > -25.826998142384024
END: th(974733.6550586838)=-25.826998142384024; dx=-1.1877452549155847E-6 evalInputDelta=1.8696790640264176
Fitness changed from -23.957319078357607 to -25.826998142384024
Iteration 11 complete. Error: -25.826998142384024 Total: 4.6462; Orientation: 0.2899; Line Search: 3.4340
Adding measurement 6ebd48ce to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.564e+05, gradient 1.476e-03, dot -0.204; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.826998142384024, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 6ebd48ce to history. Total: 3
th(0)=-25.826998142384024;dx=-2.160602898857864E-6
New Minimum: -25.826998142384024 > -26.706564542615258
END: th(2100000.0000000005)=-26.706564542615258; dx=-3.8451156439798044E-7 evalInputDelta=0.8795664002312336
Fitness changed from -25.826998142384024 to -26.706564542615258
Iteration 12 complete. Error: -26.706564542615258 Total: 3.2530; Orientation: 0.2812; Line Search: 2.1777
Adding measurement e928647 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.134e+06, gradient 1.748e-03, dot -0.196; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -26.706564542615258, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement e928647 to history. Total: 3
th(0)=-26.706564542615258;dx=-3.0445268478897174E-6
Armijo: th(4524312.849066957)=-24.71930685567802; dx=1.2204151998504006E-7 evalInputDelta=-1.9872576869372374
New Minimum: -26.706564542615258 > -27.072262180657454
WOLF (strong): th(2262156.4245334785)=-27.072262180657454; dx=1.9255243856622288E-7 evalInputDelta=0.3656976380421959
New Minimum: -27.072262180657454 > -28.06520507823496
END: th(754052.1415111595)=-28.06520507823496; dx=-1.0894308955854313E-6 evalInputDelta=1.3586405356197027
Fitness changed from -26.706564542615258 to -28.06520507823496
Iteration 13 complete. Error: -28.06520507823496 Total: 4.5290; Orientation: 0.2740; Line Search: 3.4400
Adding measurement 2e787b0d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.693e+07, gradient 1.398e-03, dot -0.250; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.06520507823496, -4.350855879584981, -1.9761691111456385, -0.9428362708864468
LBFGS Accumulation History: 3 points
Removed measurement 2e787b0d to history. Total: 3
th(0)=-28.06520507823496;dx=-1.9447823483314223E-6
New Minimum: -28.06520507823496 > -28.983736675730984
END: th(1624556.0917644731)=-28.983736675730984; dx=-7.021700363256146E-7 evalInputDelta=0.918531597496024
Fitness changed from -28.06520507823496 to -28.983736675730984
Iteration 14 complete. Error: -28.983736675730984 Total: 2.8592; Orientation: 0.2761; Line Search: 1.7644
Adding measurement 5a93ed7a to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.988e+05, gradient 1.522e-03, dot 0.256; [6ea95667-fb01-4f30-bb13-0a9a2e41d1f8 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-28.983736675730984;dx=-77.39863718168061
Armijo: th(3500000.0000000014)=-2.4532608931518545; dx=3.9112598476262895E-7 evalInputDelta=-26.53047578257913
Armijo: th(1750000.0000000007)=-2.4532608931518545; dx=7.771550571809231E-7 evalInputDelta=-26.53047578257913
Armijo: th(583333.3333333336)=-2.4532608931518545; dx=2.341750620627279E-6 evalInputDelta=-26.53047578257913
Armijo: th(145833.3333333334)=-2.4533006050224833; dx=9.192635099648395E-6 evalInputDelta=-26.5304360707085
Armijo: th(29166.66666666668)=-2.4535074326793342; dx=4.693024569403864E-5 evalInputDelta=-26.53022924305165
Armijo: th(4861.111111111113)=-2.454132854148458; dx=2.863215213944213E-4 evalInputDelta=-26.529603821582526
Armijo: th(694.4444444444447)=-2.4619745977270813; dx=0.0020392880486964646 evalInputDelta=-26.521762078003903
Armijo: th(86.80555555555559)=-2.5102914109431334; dx=0.01536615753778317 evalInputDelta=-26.473445264787852
Armijo: th(9.645061728395065)=-2.719139329075727; dx=0.15027583547383347 evalInputDelta=-26.264597346655258
Armijo: th(0.9645061728395066)=-4.965746604296744; dx=2.8352881299525654 evalInputDelta=-24.01799007143424
Armijo: th(0.08768237934904605)=-22.338361098942613; dx=43.455372655180525 evalInputDelta=-6.645375576788371
New Minimum: -28.983736675730984 > -29.054939357563885
END: th(0.007306864945753837)=-29.054939357563885; dx=-12.768706083907016 evalInputDelta=0.0712026818329008
Fitness changed from -28.983736675730984 to -29.054939357563885
Iteration 15 complete. Error: -29.054939357563885 Total: 12.5781; Orientation: 0.2714; Line Search: 11.4975
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -29.054939357563885 (> -Infinity) after 71.236s (< 5400.000s)
Returns
-29.054939357563885