BasicOptimizer.scala:89 executed in 8.15 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 139836549800200
Reset training subject: 139836680630600
Adding measurement 7002a11e to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.06301701121804003;dx=-4.4422416757313284E-7
New Minimum: -0.06301701121804003 > -0.06301818595396808
WOLFE (weak): th(2.154434690031884)=-0.06301818595396808; dx=-4.326671177636329E-7 evalInputDelta=1.174735928047288E-6
New Minimum: -0.06301818595396808 > -0.0630193499237616
WOLFE (weak): th(4.308869380063768)=-0.0630193499237616; dx=-4.431635594628243E-7 evalInputDelta=2.3387057215690232E-6
New Minimum: -0.0630193499237616 > -0.06302389457228387
WOLFE (weak): th(12.926608140191302)=-0.06302389457228387; dx=-4.4298563555756846E-7 evalInputDelta=6.883354243841788E-6
New Minimum: -0.06302389457228387 > -0.06304434322648164
WOLFE (weak): th(51.70643256076521)=-0.06304434322648164; dx=-4.5223457476503406E-7 evalInputDelta=2.733200844161343E-5
New Minimum: -0.06304434322648164 > -0.0631533206735549
WOLFE (weak): th(258.53216280382605)=-0.0631533206735549; dx=-4.390391747554357E-7 evalInputDelta=1.3630945551487694E-4
New Minimum: -0.0631533206735549 > -0.06383364512431687
WOLFE (weak): th(1551.1929768229563)=-0.06383364512431687; dx=-4.392284358805992E-7 evalInputDelta=8.166339062768424E-4
New Minimum: -0.06383364512431687 > -0.06880019761832548
WOLFE (weak): th(10858.350837760694)=-0.06880019761832548; dx=-4.385800183992951E-7 evalInputDelta=0.005783186400285453
New Minimum: -0.06880019761832548 > -0.11154998888488578
END: th(86866.80670208555)=-0.11154998888488578; dx=-3.95208832969325E-7 evalInputDelta=0.04853297766684575
Fitness changed from -0.06301701121804003 to -0.11154998888488578
Iteration 1 complete. Error: -0.11154998888488578 Total: 1.3806; Orientation: 0.0017; Line Search: 1.0245
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 48dedfeb to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.11154998888488578;dx=-5.79822636043261E-7
New Minimum: -0.11154998888488578 > -0.23089675426844583
END: th(187148.86177126726)=-0.23089675426844583; dx=-2.9815097442141815E-7 evalInputDelta=0.11934676538356005
Fitness changed from -0.11154998888488578 to -0.23089675426844583
Iteration 2 complete. Error: -0.23089675426844583 Total: 0.3523; Orientation: 0.0017; Line Search: 0.2265
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 146990cb to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.23089675426844583;dx=-9.32497842994127E-7
New Minimum: -0.23089675426844583 > -0.4131389116696171
END: th(403200.00000000006)=-0.4131389116696171; dx=-2.324172484142638E-7 evalInputDelta=0.18224215740117125
Fitness changed from -0.23089675426844583 to -0.4131389116696171
Iteration 3 complete. Error: -0.4131389116696171 Total: 0.3536; Orientation: 0.0017; Line Search: 0.2274
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1d5b8822 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.434e+03, gradient 1.085e-03, dot -0.208; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.4131389116696171, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 1d5b8822 to history. Total: 3
th(0)=-0.4131389116696171;dx=-1.128614950637022E-6
New Minimum: -0.4131389116696171 > -0.7371530941534785
END: th(868668.0670208557)=-0.7371530941534785; dx=-9.566738199158808E-8 evalInputDelta=0.32401418248386143
Fitness changed from -0.4131389116696171 to -0.7371530941534785
Iteration 4 complete. Error: -0.7371530941534785 Total: 0.3830; Orientation: 0.0305; Line Search: 0.2336
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 183debb9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.568e+03, gradient 1.289e-03, dot -0.104; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.7371530941534785, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 183debb9 to history. Total: 3
th(0)=-0.7371530941534785;dx=-1.620617587075879E-6
New Minimum: -0.7371530941534785 > -1.0969259868923562
END: th(1871488.617712673)=-1.0969259868923562; dx=-6.964481774355666E-8 evalInputDelta=0.3597728927388777
Fitness changed from -0.7371530941534785 to -1.0969259868923562
Iteration 5 complete. Error: -1.0969259868923562 Total: 0.3824; Orientation: 0.0285; Line Search: 0.2298
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3484ba75 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.206e+03, gradient 9.954e-04, dot -0.083; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.0969259868923562, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 3484ba75 to history. Total: 3
th(0)=-1.0969259868923562;dx=-9.705969952481646E-7
New Minimum: -1.0969259868923562 > -2.0104312228525476
WOLF (strong): th(4032000.0000000014)=-2.0104312228525476; dx=7.823401814454674E-9 evalInputDelta=0.9135052359601914
New Minimum: -2.0104312228525476 > -2.209551555570093
END: th(2016000.0000000007)=-2.209551555570093; dx=-1.2219587414231385E-7 evalInputDelta=1.1126255686777367
Fitness changed from -1.0969259868923562 to -2.209551555570093
Iteration 6 complete. Error: -2.209551555570093 Total: 0.4998; Orientation: 0.0287; Line Search: 0.3474
Adding measurement 4bd2bb25 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.599e+04, gradient 1.159e-03, dot -0.099; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2.209551555570093, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 4bd2bb25 to history. Total: 3
th(0)=-2.209551555570093;dx=-1.3138495705260033E-6
Armijo: th(4343340.335104279)=-2.023573026803393; dx=9.319792806057526E-8 evalInputDelta=-0.18597852876669974
New Minimum: -2.209551555570093 > -3.258958915728459
WOLF (strong): th(2171670.1675521396)=-3.258958915728459; dx=8.010187340371396E-9 evalInputDelta=1.0494073601583662
END: th(723890.0558507132)=-3.135783182877071; dx=-7.054673369734869E-7 evalInputDelta=0.926231627306978
Fitness changed from -2.209551555570093 to -3.258958915728459
Iteration 7 complete. Error: -3.258958915728459 Total: 0.6072; Orientation: 0.0290; Line Search: 0.4650
Adding measurement 77b3cfc9 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.413e+04, gradient 1.534e-03, dot 0.132; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-3.258958915728459;dx=-4.872135741098962
Armijo: th(1559573.8480938943)=-0.790695488718361; dx=1.522686912085099E-7 evalInputDelta=-2.468263427010098
Armijo: th(779786.9240469472)=-0.790695488718361; dx=3.28174389247292E-7 evalInputDelta=-2.468263427010098
Armijo: th(259928.97468231572)=-0.790695488718361; dx=9.580433722342889E-7 evalInputDelta=-2.468263427010098
Armijo: th(64982.24367057893)=-0.790695488718361; dx=4.10861743581815E-6 evalInputDelta=-2.468263427010098
Armijo: th(12996.448734115786)=-0.7913333443848741; dx=1.8794299166162644E-5 evalInputDelta=-2.467625571343585
Armijo: th(2166.0747890192974)=-0.7928240269216653; dx=1.0234339199538321E-4 evalInputDelta=-2.4661348888067938
Armijo: th(309.4392555741853)=-0.8020731372533106; dx=7.71458055652236E-4 evalInputDelta=-2.4568857784751486
Armijo: th(38.679906946773166)=-0.8306023723872104; dx=0.007021504779111219 evalInputDelta=-2.428356543341249
Armijo: th(4.297767438530352)=-1.0262474626139193; dx=0.06391511903835427 evalInputDelta=-2.23271145311454
Armijo: th(0.4297767438530352)=-2.724352988662255; dx=0.3130653953834577 evalInputDelta=-0.5346059270662042
New Minimum: -3.258958915728459 > -3.361546747785712
END: th(0.03907061307754866)=-3.361546747785712; dx=-2.275703930776177 evalInputDelta=0.10258783205725308
Fitness changed from -3.258958915728459 to -3.361546747785712
Iteration 8 complete. Error: -3.361546747785712 Total: 1.5465; Orientation: 0.0275; Line Search: 1.4080
Adding measurement 319ec54f to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.419e+05, gradient 1.476e-03, dot -0.121; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.361546747785712, -3.258958915728459, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
Rejected: LBFGS Orientation magnitude: 1.680e+05, gradient 1.476e-03, dot -0.185; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.361546747785712, -3.258958915728459, -0.23089675426844583, -0.11154998888488578
LBFGS Accumulation History: 3 points
Removed measurement 319ec54f to history. Total: 4
Removed measurement 77b3cfc9 to history. Total: 3
th(0)=-3.361546747785712;dx=-2.08290585204094E-6
New Minimum: -3.361546747785712 > -3.3615470227960578
END: th(0.08417508417508421)=-3.3615470227960578; dx=-1.864956268106301E-6 evalInputDelta=2.75010345607285E-7
Fitness changed from -3.361546747785712 to -3.3615470227960578
Iteration 9 complete. Error: -3.3615470227960578 Total: 0.3997; Orientation: 0.0621; Line Search: 0.2285
Adding measurement 3e158bfe to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.152e+05, gradient 1.467e-03, dot -0.111; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.3615470227960578, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 3e158bfe to history. Total: 3
th(0)=-3.3615470227960578;dx=-2.0710836702106573E-6
New Minimum: -3.3615470227960578 > -3.36154784782696
END: th(0.18134972138315528)=-3.36154784782696; dx=-1.8351097451708543E-6 evalInputDelta=8.250309022628244E-7
Fitness changed from -3.3615470227960578 to -3.36154784782696
Iteration 10 complete. Error: -3.36154784782696 Total: 0.3658; Orientation: 0.0276; Line Search: 0.2281
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 39249a71 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.194e+06, gradient 1.464e-03, dot -0.117; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.36154784782696, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 39249a71 to history. Total: 3
th(0)=-3.36154784782696;dx=-2.093783327112146E-6
New Minimum: -3.36154784782696 > -3.3615486728576602
END: th(0.3907061307754866)=-3.3615486728576602; dx=-1.8739843615791034E-6 evalInputDelta=8.250307002022339E-7
Fitness changed from -3.36154784782696 to -3.3615486728576602
Iteration 11 complete. Error: -3.3615486728576602 Total: 0.3861; Orientation: 0.0284; Line Search: 0.2313
Adding measurement 2cf2bceb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.246e+05, gradient 1.472e-03, dot -0.116; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.3615486728576602, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 2cf2bceb to history. Total: 3
th(0)=-3.3615486728576602;dx=-2.107456866843318E-6
New Minimum: -3.3615486728576602 > -3.3615510104435438
END: th(0.8417508417508421)=-3.3615510104435438; dx=-1.8554564447312739E-6 evalInputDelta=2.3375858835272822E-6
Fitness changed from -3.3615486728576602 to -3.3615510104435438
Iteration 12 complete. Error: -3.3615510104435438 Total: 0.3703; Orientation: 0.0283; Line Search: 0.2301
Adding measurement 4da49b6d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.766e+05, gradient 1.463e-03, dot -0.117; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.3615510104435438, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 4da49b6d to history. Total: 3
th(0)=-3.3615510104435438;dx=-2.090835233925457E-6
New Minimum: -3.3615510104435438 > -3.3615559606200485
END: th(1.8134972138315528)=-3.3615559606200485; dx=-1.855074703195967E-6 evalInputDelta=4.9501765047033075E-6
Fitness changed from -3.3615510104435438 to -3.3615559606200485
Iteration 13 complete. Error: -3.3615559606200485 Total: 0.3692; Orientation: 0.0280; Line Search: 0.2321
Adding measurement 56d0df79 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.980e+05, gradient 1.466e-03, dot -0.116; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.3615559606200485, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 56d0df79 to history. Total: 3
th(0)=-3.3615559606200485;dx=-2.101121880347465E-6
New Minimum: -3.3615559606200485 > -3.361566685977469
END: th(3.9070613077548666)=-3.361566685977469; dx=-1.8661720906662683E-6 evalInputDelta=1.0725357420415804E-5
Fitness changed from -3.3615559606200485 to -3.361566685977469
Iteration 14 complete. Error: -3.361566685977469 Total: 0.3732; Orientation: 0.0274; Line Search: 0.2303
Adding measurement 1696edc9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.290e+05, gradient 1.454e-03, dot -0.116; [316cabcb-a716-40c6-a538-61e38107502e = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.361566685977469, -0.23089675426844583, -0.11154998888488578, -0.06301701121804003
LBFGS Accumulation History: 3 points
Removed measurement 1696edc9 to history. Total: 3
th(0)=-3.361566685977469;dx=-2.077856216079847E-6
New Minimum: -3.361566685977469 > -3.3615885491000808
END: th(8.417508417508422)=-3.3615885491000808; dx=-1.85527526711149E-6 evalInputDelta=2.186312261187595E-5
Fitness changed from -3.361566685977469 to -3.3615885491000808
Iteration 15 complete. Error: -3.3615885491000808 Total: 0.3711; Orientation: 0.0280; Line Search: 0.2316
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -3.3615885491000808 (> -Infinity) after 8.152s (< 5400.000s)
Returns
-3.3615885491000808