BasicOptimizer.scala:89 executed in 284.89 seconds (3.270 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 4169794462954
Reset training subject: 4170951257568
Adding measurement 5bb97ec5 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.7065804030478102;dx=-7.679779040014266E-5
New Minimum: -1.7065804030478102 > -1.7067571619221205
WOLFE (weak): th(2.154434690031884)=-1.7067571619221205; dx=-7.682322866995407E-5 evalInputDelta=1.767588743102344E-4
New Minimum: -1.7067571619221205 > -1.7069333020458346
WOLFE (weak): th(4.308869380063768)=-1.7069333020458346; dx=-7.688776580909933E-5 evalInputDelta=3.5289899802437574E-4
New Minimum: -1.7069333020458346 > -1.7076404817814888
WOLFE (weak): th(12.926608140191302)=-1.7076404817814888; dx=-7.7068625778839E-5 evalInputDelta=0.0010600787336785267
New Minimum: -1.7076404817814888 > -1.7108230671952795
WOLFE (weak): th(51.70643256076521)=-1.7108230671952795; dx=-7.760908012520088E-5 evalInputDelta=0.004242664147469322
New Minimum: -1.7108230671952795 > -1.7278805508638049
WOLFE (weak): th(258.53216280382605)=-1.7278805508638049; dx=-7.7599345461025E-5 evalInputDelta=0.021300147815994652
New Minimum: -1.7278805508638049 > -1.8329007340661423
WOLFE (weak): th(1551.1929768229563)=-1.8329007340661423; dx=-7.259588831268993E-5 evalInputDelta=0.12632033101833207
New Minimum: -1.8329007340661423 > -2.340565294249511
END: th(10858.350837760694)=-2.340565294249511; dx=-3.0023832375579298E-5 evalInputDelta=0.633984891201701
Fitness changed from -1.7065804030478102 to -2.340565294249511
Iteration 1 complete. Error: -2.340565294249511 Total: 12.6579; Orientation: 0.0197; Line Search: 9.3177
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 6fb66e9a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-2.340565294249511;dx=-3.460338641950179E-5
New Minimum: -2.340565294249511 > -2.9245749687840763
END: th(23393.607721408407)=-2.9245749687840763; dx=-1.6043322218765245E-5 evalInputDelta=0.5840096745345651
Fitness changed from -2.340565294249511 to -2.9245749687840763
Iteration 2 complete. Error: -2.9245749687840763 Total: 3.4429; Orientation: 0.0201; Line Search: 2.2946
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement b92bc71 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.9245749687840763;dx=-4.301616880285422E-5
New Minimum: -2.9245749687840763 > -3.462372746121591
END: th(50400.00000000001)=-3.462372746121591; dx=-3.8012877241821293E-6 evalInputDelta=0.5377977773375147
Fitness changed from -2.9245749687840763 to -3.462372746121591
Iteration 3 complete. Error: -3.462372746121591 Total: 3.4122; Orientation: 0.0198; Line Search: 2.2881
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 51f7f378 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.217e+03, gradient 1.309e-02, dot -0.248; [cb9d2064-09bc-4eaa-ab79-c84a7511723b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.462372746121591, -2.9245749687840763, -2.340565294249511, -1.7065804030478102
LBFGS Accumulation History: 3 points
Removed measurement 51f7f378 to history. Total: 3
th(0)=-3.462372746121591;dx=-1.713427842849261E-4
New Minimum: -3.462372746121591 > -3.938477285376025
WOLF (strong): th(108583.50837760697)=-3.938477285376025; dx=2.776598063361887E-6 evalInputDelta=0.47610453925443386
New Minimum: -3.938477285376025 > -4.592443868380685
WOLF (strong): th(54291.75418880348)=-4.592443868380685; dx=5.239510330381941E-7 evalInputDelta=1.130071122259094
END: th(18097.251396267828)=-4.586934518331075; dx=-2.492581071939657E-5 evalInputDelta=1.1245617722094843
Fitness changed from -3.462372746121591 to -4.592443868380685
Iteration 4 complete. Error: -4.592443868380685 Total: 6.1261; Orientation: 0.4692; Line Search: 4.5442
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 3f3d31d3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.365e+03, gradient 1.959e-02, dot -0.140; [cb9d2064-09bc-4eaa-ab79-c84a7511723b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.592443868380685, -2.9245749687840763, -2.340565294249511, -1.7065804030478102
LBFGS Accumulation History: 3 points
Removed measurement 3f3d31d3 to history. Total: 3
th(0)=-4.592443868380685;dx=-3.830538112724957E-4
New Minimum: -4.592443868380685 > -7.26538433353962
END: th(38989.346202347355)=-7.26538433353962; dx=-3.882345582322802E-6 evalInputDelta=2.672940465158935
Fitness changed from -4.592443868380685 to -7.26538433353962
Iteration 5 complete. Error: -7.26538433353962 Total: 3.8434; Orientation: 0.4685; Line Search: 2.2618
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 4fe28f7c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.608e+04, gradient 2.593e-02, dot -0.093; [cb9d2064-09bc-4eaa-ab79-c84a7511723b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.26538433353962, -2.9245749687840763, -2.340565294249511, -1.7065804030478102
LBFGS Accumulation History: 3 points
Removed measurement 4fe28f7c to history. Total: 3
th(0)=-7.26538433353962;dx=-6.70985896540563E-4
New Minimum: -7.26538433353962 > -10.364004416116227
WOLF (strong): th(84000.00000000003)=-10.364004416116227; dx=1.232936195756648E-5 evalInputDelta=3.0986200825766073
New Minimum: -10.364004416116227 > -12.10628715569873
WOLF (strong): th(42000.000000000015)=-12.10628715569873; dx=4.406045607090873E-6 evalInputDelta=4.84090282215911
END: th(14000.000000000005)=-11.856353616903696; dx=-1.5675697899643435E-4 evalInputDelta=4.5909692833640765
Fitness changed from -7.26538433353962 to -12.10628715569873
Iteration 6 complete. Error: -12.10628715569873 Total: 6.1986; Orientation: 0.4667; Line Search: 4.6056
Adding measurement 1f3cb117 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.086e+05, gradient 3.097e-02, dot -0.063; [cb9d2064-09bc-4eaa-ab79-c84a7511723b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.10628715569873, -2.9245749687840763, -2.340565294249511, -1.7065804030478102
LBFGS Accumulation History: 3 points
Removed measurement 1f3cb117 to history. Total: 3
th(0)=-12.10628715569873;dx=-9.580050752896069E-4
New Minimum: -12.10628715569873 > -18.57708877681752
END: th(30162.085660446384)=-18.57708877681752; dx=-6.147387160686669E-5 evalInputDelta=6.470801621118788
Fitness changed from -12.10628715569873 to -18.57708877681752
Iteration 7 complete. Error: -18.57708877681752 Total: 3.8920; Orientation: 0.4681; Line Search: 2.3182
Adding measurement 6f454544 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 5.364e+02, gradient 3.290e-02, dot 0.361; [cb9d2064-09bc-4eaa-ab79-c84a7511723b = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-18.57708877681752;dx=-6.35847559772196
Armijo: th(64982.24367057894)=-5.722978048581156; dx=1.34416831154399E-5 evalInputDelta=-12.854110728236362
Armijo: th(32491.12183528947)=-5.77297383897757; dx=2.4638985290625913E-5 evalInputDelta=-12.804114937839948
Armijo: th(10830.37394509649)=-5.850742407289319; dx=8.658642490211729E-5 evalInputDelta=-12.7263463695282
Armijo: th(2707.5934862741224)=-6.0846195072663685; dx=4.104052514661891E-4 evalInputDelta=-12.492469269551151
Armijo: th(541.5186972548245)=-6.243960958565261; dx=0.0011355734323347326 evalInputDelta=-12.333127818252258
Armijo: th(90.25311620913742)=-2.5999488761261547; dx=-0.018673360318467412 evalInputDelta=-15.977139900691364
Armijo: th(12.89330231559106)=-0.48541107857293525; dx=-0.01645155111754415 evalInputDelta=-18.091677698244585
Armijo: th(1.6116627894488824)=-17.84541641592613; dx=2.9367003240726577 evalInputDelta=-0.7316723608913875
New Minimum: -18.57708877681752 > -19.494886955809314
END: th(0.17907364327209804)=-19.494886955809314; dx=-5.696761885770799 evalInputDelta=0.917798178991795
Fitness changed from -18.57708877681752 to -19.494886955809314
Iteration 8 complete. Error: -19.4948
...skipping 21868 bytes...
68229565E-7)=-55.49675670236; dx=-8.615871918537681E-4 evalInputDelta=0.0
WOLFE (weak): th(1.0858350837760695E-6)=-55.49675670236; dx=-8.614053435463471E-4 evalInputDelta=0.0
Armijo: th(8.686680670208556E-6)=-55.496753550462124; dx=-8.61059111658528E-4 evalInputDelta=-3.1518978786948537E-6
New Minimum: -55.49675670236 > -55.496759854257704
WOLFE (weak): th(4.886257876992313E-6)=-55.496759854257704; dx=-8.614180977493516E-4 evalInputDelta=3.1518977010591698E-6
Armijo: th(6.7864692736004345E-6)=-55.496753550462124; dx=-8.612111249205321E-4 evalInputDelta=-3.1518978786948537E-6
WOLFE (weak): th(5.836363575296374E-6)=-55.496759854257704; dx=-8.614886242270427E-4 evalInputDelta=3.1518977010591698E-6
Armijo: th(6.311416424448404E-6)=-55.496753550462124; dx=-8.61546093114346E-4 evalInputDelta=-3.1518978786948537E-6
WOLFE (weak): th(6.073889999872388E-6)=-55.496759854257704; dx=-8.610405796283044E-4 evalInputDelta=3.1518977010591698E-6
WOLFE (weak): th(6.192653212160396E-6)=-55.496759854257704; dx=-8.612757769812785E-4 evalInputDelta=3.1518977010591698E-6
WOLFE (weak): th(6.2520348183044E-6)=-55.496759854257704; dx=-8.613436263449242E-4 evalInputDelta=3.1518977010591698E-6
WOLFE (weak): th(6.281725621376402E-6)=-55.496759854257704; dx=-8.61064766168878E-4 evalInputDelta=3.1518977010591698E-6
Armijo: th(6.296571022912403E-6)=-55.496753550462124; dx=-8.613984565371479E-4 evalInputDelta=-3.1518978786948537E-6
WOLFE (weak): th(6.289148322144402E-6)=-55.496759854257704; dx=-8.61407313358732E-4 evalInputDelta=3.1518977010591698E-6
WOLFE (weak): th(6.292859672528403E-6)=-55.496759854257704; dx=-8.613342878371623E-4 evalInputDelta=3.1518977010591698E-6
WOLFE (weak): th(6.294715347720403E-6)=-55.496759854257704; dx=-8.61197803784317E-4 evalInputDelta=3.1518977010591698E-6
Armijo: th(6.295643185316403E-6)=-55.496753550462124; dx=-8.611927122238443E-4 evalInputDelta=-3.1518978786948537E-6
WOLFE (weak): th(6.295179266518403E-6)=-55.496759854257704; dx=-8.612808090608132E-4 evalInputDelta=3.1518977010591698E-6
WOLFE (weak): th(6.295411225917403E-6)=-55.496759854257704; dx=-8.61591167576012E-4 evalInputDelta=3.1518977010591698E-6
Armijo: th(6.295527205616903E-6)=-55.496753550462124; dx=-8.616712810182947E-4 evalInputDelta=-3.1518978786948537E-6
WOLFE (weak): th(6.2954692157671534E-6)=-55.496759854257704; dx=-8.61438237249117E-4 evalInputDelta=3.1518977010591698E-6
mu ~= nu (6.2954692157671534E-6): th(4.886257876992313E-6)=-55.496759854257704
Fitness changed from -55.49675670236 to -55.496759854257704
Iteration 23 complete. Error: -55.496759854257704 Total: 34.7202; Orientation: 0.0199; Line Search: 32.5036
Adding measurement 718726b4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-55.496759854257704;dx=-9.233134034921939E-4
Armijo: th(1.3563239736148558E-5)=-55.496753550462124; dx=-8.612977958228436E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(6.781619868074279E-6)=-55.49675670236; dx=-8.613333012536226E-4 evalInputDelta=-3.1518977010591698E-6
Armijo: th(2.2605399560247595E-6)=-55.496753550462124; dx=-8.612996353121054E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(5.651349890061899E-7)=-55.496759854257704; dx=-8.610534142437057E-4 evalInputDelta=0.0
Armijo: th(1.4128374725154746E-6)=-55.496753550462124; dx=-8.6120854601147E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(9.889862307608323E-7)=-55.496759854257704; dx=-8.612669844973166E-4 evalInputDelta=0.0
WOLFE (weak): th(1.2009118516381535E-6)=-55.496759854257704; dx=-8.613643921123763E-4 evalInputDelta=0.0
WOLFE (weak): th(1.306874662076814E-6)=-55.496759854257704; dx=-8.614727456083975E-4 evalInputDelta=0.0
WOLFE (weak): th(1.3598560672961442E-6)=-55.496759854257704; dx=-8.612138713510331E-4 evalInputDelta=0.0
WOLFE (weak): th(1.3863467699058093E-6)=-55.496759854257704; dx=-8.609947725239393E-4 evalInputDelta=0.0
WOLFE (weak): th(1.399592121210642E-6)=-55.496759854257704; dx=-8.611965801119813E-4 evalInputDelta=0.0
WOLFE (weak): th(1.4062147968630585E-6)=-55.496759854257704; dx=-8.611659868772151E-4 evalInputDelta=0.0
Armijo: th(1.4095261346892667E-6)=-55.496753550462124; dx=-8.612894079114889E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(1.4078704657761626E-6)=-55.496759854257704; dx=-8.613774570876249E-4 evalInputDelta=0.0
WOLFE (weak): th(1.4086983002327146E-6)=-55.496759854257704; dx=-8.612200162802097E-4 evalInputDelta=0.0
WOLFE (weak): th(1.4091122174609906E-6)=-55.496759854257704; dx=-8.611014862672772E-4 evalInputDelta=0.0
Armijo: th(1.4093191760751288E-6)=-55.496753550462124; dx=-8.613194179986556E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(1.4092156967680596E-6)=-55.496759854257704; dx=-8.611414709963264E-4 evalInputDelta=0.0
Armijo: th(1.4092674364215942E-6)=-55.496753550462124; dx=-8.612080722940757E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(1.409241566594827E-6)=-55.496753550462124; dx=-8.609843534003187E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(1.4092286316814433E-6)=-55.496759854257704; dx=-8.613470638383832E-4 evalInputDelta=0.0
mu ~= nu (1.4092286316814433E-6): th(0.0)=-55.496759854257704
Fitness changed from -55.496759854257704 to -55.496759854257704
Static Iteration Total: 27.7837; Orientation: 0.0204; Line Search: 26.6715
Iteration 24 failed. Error: -55.496759854257704
Previous Error: 0.0 -> -55.496759854257704
Retrying iteration 24
Reset training subject: 4427103603629
Adding measurement 1040c184 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-55.496759854257704;dx=-9.236377705286156E-4
Armijo: th(3.036104983993719E-6)=-55.496753550462124; dx=-8.615373164288689E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(1.5180524919968596E-6)=-55.496753550462124; dx=-8.614473777519976E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(5.060174973322865E-7)=-55.496759854257704; dx=-8.610633228273614E-4 evalInputDelta=0.0
WOLFE (weak): th(1.012034994664573E-6)=-55.496759854257704; dx=-8.610388405565535E-4 evalInputDelta=0.0
WOLFE (weak): th(1.2650437433307163E-6)=-55.496759854257704; dx=-8.612982463034616E-4 evalInputDelta=0.0
WOLFE (weak): th(1.391548117663788E-6)=-55.496759854257704; dx=-8.610708903890695E-4 evalInputDelta=0.0
Armijo: th(1.4548003048303238E-6)=-55.496753550462124; dx=-8.612216553610731E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(1.423174211247056E-6)=-55.496753550462124; dx=-8.610336674801397E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(1.407361164455422E-6)=-55.496759854257704; dx=-8.613994451753434E-4 evalInputDelta=0.0
Armijo: th(1.415267687851239E-6)=-55.496753550462124; dx=-8.610470646194656E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(1.4113144261533305E-6)=-55.496753550462124; dx=-8.615365446131815E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(1.4093377953043764E-6)=-55.496753550462124; dx=-8.610869639158112E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(1.4083494798798993E-6)=-55.496759854257704; dx=-8.614353862471848E-4 evalInputDelta=0.0
WOLFE (weak): th(1.4088436375921378E-6)=-55.496759854257704; dx=-8.612757010993811E-4 evalInputDelta=0.0
WOLFE (weak): th(1.409090716448257E-6)=-55.496759854257704; dx=-8.611622405831926E-4 evalInputDelta=0.0
WOLFE (weak): th(1.4092142558763167E-6)=-55.496759854257704; dx=-8.613810858580804E-4 evalInputDelta=0.0
Armijo: th(1.4092760255903466E-6)=-55.496753550462124; dx=-8.614003137677505E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(1.4092451407333317E-6)=-55.496753550462124; dx=-8.612297668107523E-4 evalInputDelta=-6.3037955797540235E-6
Armijo: th(1.4092296983048242E-6)=-55.496753550462124; dx=-8.614422032416758E-4 evalInputDelta=-6.3037955797540235E-6
WOLFE (weak): th(1.4092219770905705E-6)=-55.496759854257704; dx=-8.612754104965484E-4 evalInputDelta=0.0
mu ~= nu (1.4092219770905705E-6): th(0.0)=-55.496759854257704
Fitness changed from -55.496759854257704 to -55.496759854257704
Static Iteration Total: 27.5767; Orientation: 0.0195; Line Search: 25.3709
Iteration 25 failed. Error: -55.496759854257704
Previous Error: 0.0 -> -55.496759854257704
Optimization terminated 25
Final threshold in iteration 25: -55.496759854257704 (> -Infinity) after 284.886s (< 3600.000s)
Returns
-55.496759854257704