BasicOptimizer.scala:89 executed in 43.44 seconds (1.423 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 221598311967800
Reset training subject: 221599007852400
Adding measurement 430cf852 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-181.91090710839742;dx=-0.13097330756038306
New Minimum: -181.91090710839742 > -182.1249670670095
WOLFE (weak): th(2.154434690031884)=-182.1249670670095; dx=-0.13148043398393044 evalInputDelta=0.2140599586120686
New Minimum: -182.1249670670095 > -182.3409271973142
WOLFE (weak): th(4.308869380063768)=-182.3409271973142; dx=-0.1315716419721099 evalInputDelta=0.4300200889167627
New Minimum: -182.3409271973142 > -183.21251544442563
WOLFE (weak): th(12.926608140191302)=-183.21251544442563; dx=-0.13143515351787532 evalInputDelta=1.3016083360282096
New Minimum: -183.21251544442563 > -187.11231674310386
WOLFE (weak): th(51.70643256076521)=-187.11231674310386; dx=-0.12668460238896787 evalInputDelta=5.201409634706437
New Minimum: -187.11231674310386 > -204.71669816810152
END: th(258.53216280382605)=-204.71669816810152; dx=-0.09070179533530051 evalInputDelta=22.80579105970409
Fitness changed from -181.91090710839742 to -204.71669816810152
Iteration 1 complete. Error: -204.71669816810152 Total: 6.4223; Orientation: 0.0178; Line Search: 4.4573
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 71736e5a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-204.71669816810152;dx=-0.16583176261030041
New Minimum: -204.71669816810152 > -241.69643059819848
END: th(556.9906600335335)=-241.69643059819848; dx=-0.05246437474166253 evalInputDelta=36.97973243009696
Fitness changed from -204.71669816810152 to -241.69643059819848
Iteration 2 complete. Error: -241.69643059819848 Total: 1.9802; Orientation: 0.0174; Line Search: 1.3281
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5f71b355 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-241.69643059819848;dx=-0.1654953512761106
New Minimum: -241.69643059819848 > -269.6463120480031
END: th(1200.0)=-269.6463120480031; dx=-0.024458567844789062 evalInputDelta=27.94988144980462
Fitness changed from -241.69643059819848 to -269.6463120480031
Iteration 3 complete. Error: -269.6463120480031 Total: 1.8814; Orientation: 0.0170; Line Search: 1.2422
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 28be1c0c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.596e+03, gradient 3.948e-01, dot -0.223; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -269.6463120480031, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 28be1c0c to history. Total: 3
th(0)=-269.6463120480031;dx=-0.15582166496434424
New Minimum: -269.6463120480031 > -270.97161780742664
END: th(2585.3216280382608)=-270.97161780742664; dx=-0.005960474199822357 evalInputDelta=1.325305759423543
Fitness changed from -269.6463120480031 to -270.97161780742664
Iteration 4 complete. Error: -270.97161780742664 Total: 2.1312; Orientation: 0.3100; Line Search: 1.2176
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 325aa959 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.890e+02, gradient 3.354e-01, dot -0.160; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -270.97161780742664, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 325aa959 to history. Total: 3
th(0)=-270.97161780742664;dx=-0.11236908002859738
New Minimum: -270.97161780742664 > -280.6070785569211
END: th(5569.9066003353355)=-280.6070785569211; dx=-0.004001214102792267 evalInputDelta=9.635460749494484
Fitness changed from -270.97161780742664 to -280.6070785569211
Iteration 5 complete. Error: -280.6070785569211 Total: 2.4085; Orientation: 0.3177; Line Search: 1.4951
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6f8d5bac to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.358e+02, gradient 2.507e-01, dot -0.133; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -280.6070785569211, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 6f8d5bac to history. Total: 3
th(0)=-280.6070785569211;dx=-0.06277861226612079
New Minimum: -280.6070785569211 > -314.4228749097925
END: th(12000.000000000002)=-314.4228749097925; dx=-0.0025070098725361077 evalInputDelta=33.81579635287136
Fitness changed from -280.6070785569211 to -314.4228749097925
Iteration 6 complete. Error: -314.4228749097925 Total: 3.0799; Orientation: 0.2952; Line Search: 1.3163
Adding measurement 169bf5eb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.338e+02, gradient 2.153e-01, dot -0.098; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -314.4228749097925, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 169bf5eb to history. Total: 3
th(0)=-314.4228749097925;dx=-0.046269609736015974
New Minimum: -314.4228749097925 > -314.7063180909965
END: th(25853.21628038261)=-314.7063180909965; dx=-6.458994090608437E-5 evalInputDelta=0.2834431812040066
Fitness changed from -314.4228749097925 to -314.7063180909965
Iteration 7 complete. Error: -314.7063180909965 Total: 2.5562; Orientation: 0.3973; Line Search: 1.4732
Adding measurement 3c5a424f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.906e+02, gradient 1.972e-01, dot -0.059; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -314.7063180909965, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 3c5a424f to history. Total: 3
th(0)=-314.7063180909965;dx=-0.038670240812427835
Armijo: th(55699.066003353364)=-265.09977868931503; dx=8.607014228869779E-4 evalInputDelta=-49.60653940168146
New Minimum: -314.7063180909965 > -429.03523276803566
END: th(27849.533001676682)=-429.03523276803566; dx=-0.0028215934891830615 evalInputDelta=114.32891467703917
Fitness changed from -314.7063180909965 to -429.03523276803566
Iteration 8 complete. Error: -429.03523276803566 Total: 2.8204; Orientation: 0.3240; Line Search: 1.8897
Adding measurement 333fa8a4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.589e+03, gradient 1.948e-01, dot -0.075; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -429.03523276803566, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 333fa8a4 to history. Total: 3
th(0)=-429.03523276803566;dx=-0.037908473968551744
Armijo: th(60000.00000000002)=-335.8308668767678; dx=0.001010108896222636 evalInputDelta=-93.20436589126786
New Minimum: -429.03523276803566 > -555.6658943603344
END: th(30000.00000000001)=-555.6658943603344; dx=-0.003314466809592306 evalInputDelta=126.63066159229874
Fitness changed from -429.03523276803566 to -555.6658943603344
Iteration 9 complete. Error: -555.6658943603344 Total: 2.8259; Orientation: 0.3297; Line Search: 1.9015
Adding measurement 49350920 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.270e+03, gradient 1.959e-01, dot -0.092; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -555.6658943603344, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 49350920 to history. Total: 3
th(0)=-555.6658943603344;dx=-0.03831634837597146
Armijo: th(64633.04070095654)=-417.216803839047; dx=0.0012092327341632098 evalInputDelta=-138.44909052128742
New Minimum: -555.6658943603344 > -689.9503330170774
END: th(32316.52035047827)=-689.9503330170774; dx=-0.0035905956928152126 evalInputDelta=134.28443865674296
Fitness changed from -555.6658943603344 to -689.9503330170774
Iteration 10 complete. Error: -689.9503330170774 Total: 2.9501; Orientation: 0.2855; Line Search: 1.8511
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 1c36484c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.908e+03, gradient 1.984e-01, dot -0.104; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -689.9503330170774, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 1c36484c to history. Total: 3
th(0)=-689.9503330170774;dx=-0.039280270389522365
Armijo: th(69623.83250419171)=-484.5692508276993; dx=0.0018317313711139341 evalInputDelta=-205.38108218937805
New Minimum: -689.9503330170774 > -817.0757444701818
END: th(34811.916252095856)=-817.0757444701818; dx=-0.0030763562471909536 evalInputDelta=127.12541145310445
Fitness changed from -689.9503330170774 to -817.0757444701818
Iteration 11 complete. Error: -817.0757444701818 Total: 2.7695; Orientation: 0.3105; Line Search: 1.8403
Adding measurement 464c3f3 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.803e+04, gradient 1.999e-01, dot -0.114; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -817.0757444701818, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 464c3f3 to history. Total: 3
th(0)=-817.0757444701818;dx=-0.039902863577238934
Armijo: th(75000.00000000003)=-511.48405427562784; dx=0.002360625592458711 evalInputDelta=-305.59169019455396
New Minimum: -817.0757444701818 > -924.1634335263086
END: th(37500.000000000015)=-924.1634335263086; dx=-0.0026430596463090598 evalInputDelta=107.0876890561268
Fitness changed from -817.0757444701818 to -924.1634335263086
Iteration 12 complete. Error: -924.1634335263086 Total: 2.7392; Orientation: 0.3030; Line Search: 1.8854
Adding measurement 1f79d7a2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.548e+04, gradient 2.048e-01, dot -0.118; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -924.1634335263086, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 1f79d7a2 to history. Total: 3
th(0)=-924.1634335263086;dx=-0.041846129050658015
Armijo: th(80791.30087619567)=-545.5868914577055; dx=0.002649333313730212 evalInputDelta=-378.5765420686031
New Minimum: -924.1634335263086 > -1026.6775239665276
END: th(40395.65043809784)=-1026.6775239665276; dx=-0.002123551812228043 evalInputDelta=102.51409044021898
Fitness changed from -924.1634335263086 to -1026.6775239665276
Iteration 13 complete. Error: -1026.6775239665276 Total: 2.9369; Orientation: 0.3096; Line Search: 2.0512
Adding measurement 2e6ce403 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.451e+04, gradient 2.093e-01, dot -0.125; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1026.6775239665276, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 2e6ce403 to history. Total: 3
th(0)=-1026.6775239665276;dx=-0.043746614964007294
Armijo: th(87029.79063023964)=-566.8786458496182; dx=0.002624931719013656 evalInputDelta=-459.79887811690935
New Minimum: -1026.6775239665276 > -1085.0971304720406
END: th(43514.89531511982)=-1085.0971304720406; dx=-0.001558497664639127 evalInputDelta=58.419606505513
Fitness changed from -1026.6775239665276 to -1085.0971304720406
Iteration 14 complete. Error: -1085.0971304720406 Total: 3.1063; Orientation: 0.2953; Line Search: 1.8568
Adding measurement 8f9e309 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.708e+05, gradient 2.089e-01, dot -0.129; [eb665240-9174-434a-8373-d7fb0e2e48cb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1085.0971304720406, -241.69643059819848, -204.71669816810152, -181.91090710839742
LBFGS Accumulation History: 3 points
Removed measurement 8f9e309 to history. Total: 3
th(0)=-1085.0971304720406;dx=-0.04349856168758001
Armijo: th(93750.00000000004)=-616.5785302229527; dx=0.003211843319517854 evalInputDelta=-468.51860024908785
New Minimum: -1085.0971304720406 > -1192.9739656513416
END: th(46875.00000000002)=-1192.9739656513416; dx=-0.0015510382784296715 evalInputDelta=107.876835179301
Fitness changed from -1085.0971304720406 to -1192.9739656513416
Iteration 15 complete. Error: -1192.9739656513416 Total: 2.8048; Orientation: 0.3077; Line Search: 1.9283
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -1192.9739656513416 (> -Infinity) after 43.438s (< 5400.000s)
Returns
-1192.9739656513416