BasicOptimizer.scala:89 executed in 9.00 seconds (0.655 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 141875746692700
Reset training subject: 141875876836400
Adding measurement 26ac41f4 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.12461804430424184;dx=-1.6021774019307987E-6
New Minimum: -0.12461804430424184 > -0.1246221578492712
WOLFE (weak): th(2.154434690031884)=-0.1246221578492712; dx=-1.5365849680050605E-6 evalInputDelta=4.1135450293688924E-6
New Minimum: -0.1246221578492712 > -0.12462631366409851
WOLFE (weak): th(4.308869380063768)=-0.12462631366409851; dx=-1.5522858320371521E-6 evalInputDelta=8.269359856677427E-6
New Minimum: -0.12462631366409851 > -0.12464302033758048
WOLFE (weak): th(12.926608140191302)=-0.12464302033758048; dx=-1.6205272181342323E-6 evalInputDelta=2.4976033338647285E-5
New Minimum: -0.12464302033758048 > -0.12471765711817268
WOLFE (weak): th(51.70643256076521)=-0.12471765711817268; dx=-1.598709110849338E-6 evalInputDelta=9.961281393083832E-5
New Minimum: -0.12471765711817268 > -0.12511661063906696
WOLFE (weak): th(258.53216280382605)=-0.12511661063906696; dx=-1.624769172168212E-6 evalInputDelta=4.985663348251179E-4
New Minimum: -0.12511661063906696 > -0.12765278826374382
WOLFE (weak): th(1551.1929768229563)=-0.12765278826374382; dx=-1.5712064429906194E-6 evalInputDelta=0.0030347439595019843
New Minimum: -0.12765278826374382 > -0.14601532243965176
WOLFE (weak): th(10858.350837760694)=-0.14601532243965176; dx=-1.6154281408809863E-6 evalInputDelta=0.021397278135409925
New Minimum: -0.14601532243965176 > -0.2764393200386627
END: th(86866.80670208555)=-0.2764393200386627; dx=-1.0974729451486565E-6 evalInputDelta=0.15182127573442084
Fitness changed from -0.12461804430424184 to -0.2764393200386627
Iteration 1 complete. Error: -0.2764393200386627 Total: 1.3776; Orientation: 0.0016; Line Search: 1.0213
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 46635368 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.2764393200386627;dx=-2.110389752959958E-6
New Minimum: -0.2764393200386627 > -0.46127701092829815
END: th(187148.86177126726)=-0.46127701092829815; dx=-5.473888541362538E-7 evalInputDelta=0.18483769088963548
Fitness changed from -0.2764393200386627 to -0.46127701092829815
Iteration 2 complete. Error: -0.46127701092829815 Total: 0.3538; Orientation: 0.0016; Line Search: 0.2304
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3e3c94b0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.46127701092829815;dx=-3.866890819876565E-6
New Minimum: -0.46127701092829815 > -0.7483486597029213
END: th(403200.00000000006)=-0.7483486597029213; dx=-6.244282215459169E-7 evalInputDelta=0.28707164877462316
Fitness changed from -0.46127701092829815 to -0.7483486597029213
Iteration 3 complete. Error: -0.7483486597029213 Total: 0.3566; Orientation: 0.0015; Line Search: 0.2323
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 7363486f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.792e+03, gradient 2.846e-03, dot -0.266; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.7483486597029213, -0.46127701092829815, -0.2764393200386627, -0.12461804430424184
LBFGS Accumulation History: 3 points
Removed measurement 7363486f to history. Total: 3
th(0)=-0.7483486597029213;dx=-7.193779903639346E-6
New Minimum: -0.7483486597029213 > -1.5082044314782799
END: th(868668.0670208557)=-1.5082044314782799; dx=-4.784664559920768E-7 evalInputDelta=0.7598557717753586
Fitness changed from -0.7483486597029213 to -1.5082044314782799
Iteration 4 complete. Error: -1.5082044314782799 Total: 0.3864; Orientation: 0.0285; Line Search: 0.2331
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4c921e1d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.689e+03, gradient 2.208e-03, dot -0.220; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.5082044314782799, -0.46127701092829815, -0.2764393200386627, -0.12461804430424184
LBFGS Accumulation History: 3 points
Removed measurement 4c921e1d to history. Total: 3
th(0)=-1.5082044314782799;dx=-4.364628437842661E-6
New Minimum: -1.5082044314782799 > -3.8666562350321074
END: th(1871488.617712673)=-3.8666562350321074; dx=-1.2985176805921538E-7 evalInputDelta=2.3584518035538276
Fitness changed from -1.5082044314782799 to -3.8666562350321074
Iteration 5 complete. Error: -3.8666562350321074 Total: 0.3810; Orientation: 0.0273; Line Search: 0.2294
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement d6f9a6c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.469e+03, gradient 1.901e-03, dot -0.206; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.8666562350321074, -0.46127701092829815, -0.2764393200386627, -0.12461804430424184
LBFGS Accumulation History: 3 points
Removed measurement d6f9a6c to history. Total: 3
th(0)=-3.8666562350321074;dx=-3.361176859830269E-6
New Minimum: -3.8666562350321074 > -5.8153202440664025
WOLF (strong): th(4032000.0000000014)=-5.8153202440664025; dx=1.647834309950802E-7 evalInputDelta=1.948664009034295
New Minimum: -5.8153202440664025 > -7.3658233962884205
END: th(2016000.0000000007)=-7.3658233962884205; dx=-4.801601824164135E-7 evalInputDelta=3.499167161256313
Fitness changed from -3.8666562350321074 to -7.3658233962884205
Iteration 6 complete. Error: -7.3658233962884205 Total: 0.4873; Orientation: 0.0290; Line Search: 0.3399
Adding measurement 1936a549 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.957e+04, gradient 2.108e-03, dot -0.287; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -7.3658233962884205, -0.46127701092829815, -0.2764393200386627, -0.12461804430424184
LBFGS Accumulation History: 3 points
Removed measurement 1936a549 to history. Total: 3
th(0)=-7.3658233962884205;dx=-4.310683144284196E-6
New Minimum: -7.3658233962884205 > -8.746135107462454
WOLF (strong): th(4343340.335104279)=-8.746135107462454; dx=2.340882959742189E-7 evalInputDelta=1.3803117111740333
New Minimum: -8.746135107462454 > -10.497804992566136
END: th(2171670.1675521396)=-10.497804992566136; dx=-4.075447797484749E-7 evalInputDelta=3.1319815962777158
Fitness changed from -7.3658233962884205 to -10.497804992566136
Iteration 7 complete. Error: -10.497804992566136 Total: 0.4841; Orientation: 0.0286; Line Search: 0.3436
Adding measurement 4660c834 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.341e+04, gradient 2.383e-03, dot 0.309; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-10.497804992566136;dx=-31.851306635732882
Armijo: th(4678721.544281683)=-6.316538359856742; dx=6.07975590076718E-7 evalInputDelta=-4.181266632709394
Armijo: th(2339360.7721408415)=-6.316538359856742; dx=1.1664964441644796E-6 evalInputDelta=-4.181266632709394
Armijo: th(779786.9240469472)=-6.316538359856742; dx=3.5830357220397377E-6 evalInputDelta=-4.181266632709394
Armijo: th(194946.7310117368)=-6.316538359856742; dx=1.3017837098585913E-5 evalInputDelta=-4.181266632709394
Armijo: th(38989.346202347355)=-6.316538359856742; dx=7.224295991915427E-5 evalInputDelta=-4.181266632709394
Armijo: th(6498.224367057893)=-6.317756253472574; dx=4.453188834372538E-4 evalInputDelta=-4.180048739093563
Armijo: th(928.3177667225561)=-6.322981536262349; dx=0.0028992193530474573 evalInputDelta=-4.174823456303788
Armijo: th(116.03972084031952)=-6.318553111413438; dx=0.02342349349171801 evalInputDelta=-4.179251881152698
Armijo: th(12.893302315591058)=-6.499763968862818; dx=0.21387052792391703 evalInputDelta=-3.998041023703318
Armijo: th(1.2893302315591058)=-8.189089403653668; dx=1.7212308941866978 evalInputDelta=-2.3087155889124684
New Minimum: -10.497804992566136 > -11.375761739084885
END: th(0.11721183923264598)=-11.375761739084885; dx=-4.378337780197057 evalInputDelta=0.8779567465187483
Fitness changed from -10.497804992566136 to -11.375761739084885
Iteration 8 complete. Error: -11.375761739084885 Total: 1.5151; Orientation: 0.0269; Line Search: 1.3783
Adding measurement 673fd5ca to history. Total: 4
Accepted: LBFGS Orientation magnitude: 3.285e+04, gradient 2.299e-03, dot 0.272; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
th(0)=-11.375761739084885;dx=-19.44766312840231
Armijo: th(0.25252525252525265)=-10.652764889130513; dx=0.33357319938002045 evalInputDelta=-0.722996849954372
Armijo: th(0.12626262626262633)=-11.09217524567928; dx=-0.36396692343869147 evalInputDelta=-0.28358649340560405
Armijo: th(0.04208754208754211)=-11.317735021645788; dx=-1.234118757198344 evalInputDelta=-0.05802671743909649
Armijo: th(0.010521885521885528)=-11.363153226283067; dx=-1.6090190394675807 evalInputDelta=-0.012608512801817184
Armijo: th(0.0021043771043771056)=-11.373336573369404; dx=-1.4425367060454597 evalInputDelta=-0.0024251657154810147
Armijo: th(3.5072951739618426E-4)=-11.375358320740316; dx=-1.5220343909069851 evalInputDelta=-4.0341834456825154E-4
Armijo: th(5.010421677088346E-5)=-11.375704652452304; dx=-1.7584294436305838 evalInputDelta=-5.708663258019442E-5
Armijo: th(6.263027096360433E-6)=-11.375754127550426; dx=-1.4507216347043297 evalInputDelta=-7.611534458717983E-6
Armijo: th(6.958918995956036E-7)=-11.375761739084885; dx=-1.6349207649140383 evalInputDelta=0.0
New Minimum: -11.375761739084885 > -11.375762373379194
END: th(6.958918995956036E-8)=-11.375762373379194; dx=-1.4758107325229748 evalInputDelta=6.342943095205555E-7
Fitness changed from -11.375761739084885 to -11.375762373379194
Iteration 9 complete. Error: -11.375762373379194 Total: 1.3904; Orientation: 0.0335; Line Search: 1.2452
Adding measurement 35061c5f to history. Total: 5
Rejected: LBFGS Orientation magnitude: 1.366e+07, gradient 2.254e-03, dot -0.071; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.375762373379194, -11.375761739084885, -10.497804992566136, -0.46127701092829815, -0.2764393200386627, -0.12461804430424184
Rejected: LBFGS Orientation magnitude: 2.373e+07, gradient 2.254e-03, dot -0.073; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.375762373379194, -11.375761739084885, -10.497804992566136, -0.46127701092829815, -0.2764393200386627
Accepted: LBFGS Orientation magnitude: 4.669e+07, gradient 2.254e-03, dot 0.068; [6b9446da-e7d6-496e-bbc7-213e4e20098f = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-11.375762373379194;dx=-7028.505548542989
Armijo: th(1.499253649000953E-7)=-11.375579060852212; dx=962.03123446878 evalInputDelta=-1.833125269818936E-4
Armijo: th(7.496268245004766E-8)=-11.375670400335245; dx=811.1067095478895 evalInputDelta=-9.1973043948812E-5
Armijo: th(2.4987560816682553E-8)=-11.375731927212499; dx=1162.2268576275483 evalInputDelta=-3.0446166695341503E-5
Armijo: th(6.246890204170638E-9)=-11.375753493255658; dx=741.3018621737369 evalInputDelta=-8.880123536059159E-6
Armijo: th(1.2493780408341277E-9)=-11.375760470496163; dx=776.8196047278201 evalInputDelta=-1.9028830315903633E-6
Armijo: th(2.082296734723546E-10)=-11.375762373379194; dx=989.6991440480493 evalInputDelta=0.0
MIN ALPHA (2.9747096210336376E-11): th(0.0)=-11.375762373379194
Fitness changed from -11.375762373379194 to -11.375762373379194
Static Iteration Total: 1.8214; Orientation: 0.1022; Line Search: 1.6095
Iteration 10 failed. Error: -11.375762373379194
Previous Error: 0.0 -> -11.375762373379194
Retrying iteration 10
Reset training subject: 141884300504000
Adding measurement 35bc79a4 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-11.375762373379194;dx=-4.707900221084247E-6
END: th(2.1544346900318838E-10)=-11.375762373379194; dx=-3.573023580775789E-6 evalInputDelta=0.0
Fitness changed from -11.375762373379194 to -11.375762373379194
Static Iteration Total: 0.4476; Orientation: 0.0015; Line Search: 0.2245
Iteration 11 failed. Error: -11.375762373379194
Previous Error: 0.0 -> -11.375762373379194
Optimization terminated 11
Final threshold in iteration 11: -11.375762373379194 (> -Infinity) after 9.001s (< 5400.000s)
Returns
-11.375762373379194