BasicOptimizer.scala:89 executed in 77.14 seconds (1.928 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2356295588075
Reset training subject: 2357227198193
Adding measurement 14196132 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.6056779315520587;dx=-1.661090584239865E-4
New Minimum: -0.6056779315520587 > -0.6060536912276168
WOLFE (weak): th(2.154434690031884)=-0.6060536912276168; dx=-1.7803898211472824E-4 evalInputDelta=3.7575967555802947E-4
New Minimum: -0.6060536912276168 > -0.6064291443566112
WOLFE (weak): th(4.308869380063768)=-0.6064291443566112; dx=-1.7836690912576792E-4 evalInputDelta=7.512128045524813E-4
New Minimum: -0.6064291443566112 > -0.6079326089289886
WOLFE (weak): th(12.926608140191302)=-0.6079326089289886; dx=-1.7892046739759142E-4 evalInputDelta=0.002254677376929859
New Minimum: -0.6079326089289886 > -0.614683067720818
WOLFE (weak): th(51.70643256076521)=-0.614683067720818; dx=-1.8047388927277746E-4 evalInputDelta=0.009005136168759309
New Minimum: -0.614683067720818 > -0.6499136473076098
WOLFE (weak): th(258.53216280382605)=-0.6499136473076098; dx=-1.7429568840071814E-4 evalInputDelta=0.044235715755551075
New Minimum: -0.6499136473076098 > -0.8591369299147033
WOLFE (weak): th(1551.1929768229563)=-0.8591369299147033; dx=-1.597802285484409E-4 evalInputDelta=0.2534589983626445
New Minimum: -0.8591369299147033 > -1.5559514720723824
END: th(10858.350837760694)=-1.5559514720723824; dx=-2.035719680744522E-5 evalInputDelta=0.9502735405203236
Fitness changed from -0.6056779315520587 to -1.5559514720723824
Iteration 1 complete. Error: -1.5559514720723824 Total: 11.1705; Orientation: 0.0198; Line Search: 8.4725
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 453f7dee to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.5559514720723824;dx=-4.16360475676186E-4
New Minimum: -1.5559514720723824 > -2.7003811708469954
END: th(23393.607721408407)=-2.7003811708469954; dx=-3.38799760073111E-5 evalInputDelta=1.144429698774613
Fitness changed from -1.5559514720723824 to -2.7003811708469954
Iteration 2 complete. Error: -2.7003811708469954 Total: 2.7782; Orientation: 0.0209; Line Search: 1.8465
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 31524fee to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.7003811708469954;dx=-7.44339969735431E-4
New Minimum: -2.7003811708469954 > -6.532694533641769
END: th(50400.00000000001)=-6.532694533641769; dx=-1.4731968912206293E-5 evalInputDelta=3.832313362794774
Fitness changed from -2.7003811708469954 to -6.532694533641769
Iteration 3 complete. Error: -6.532694533641769 Total: 2.8197; Orientation: 0.0207; Line Search: 1.8803
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 463cd0b8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.783e+03, gradient 3.227e-02, dot -0.145; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.532694533641769, -2.7003811708469954, -1.5559514720723824, -0.6056779315520587
LBFGS Accumulation History: 3 points
Removed measurement 463cd0b8 to history. Total: 3
th(0)=-6.532694533641769;dx=-0.001011874781378604
New Minimum: -6.532694533641769 > -19.398304564335078
END: th(108583.50837760697)=-19.398304564335078; dx=-9.35425603936964E-6 evalInputDelta=12.865610030693308
Fitness changed from -6.532694533641769 to -19.398304564335078
Iteration 4 complete. Error: -19.398304564335078 Total: 3.2269; Orientation: 0.4470; Line Search: 1.8651
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 78ef17a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.508e+03, gradient 4.562e-02, dot -0.101; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.398304564335078, -2.7003811708469954, -1.5559514720723824, -0.6056779315520587
LBFGS Accumulation History: 3 points
Removed measurement 78ef17a5 to history. Total: 3
th(0)=-19.398304564335078;dx=-0.0020429203683956136
New Minimum: -19.398304564335078 > -36.37051626145826
WOLF (strong): th(233936.07721408413)=-36.37051626145826; dx=1.7221640975148785E-5 evalInputDelta=16.972211697123186
New Minimum: -36.37051626145826 > -41.42254985776165
END: th(116968.03860704206)=-41.42254985776165; dx=-1.1654191936880046E-5 evalInputDelta=22.024245293426574
Fitness changed from -19.398304564335078 to -41.42254985776165
Iteration 5 complete. Error: -41.42254985776165 Total: 4.1722; Orientation: 0.4642; Line Search: 2.8065
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3e336662 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.324e+03, gradient 4.155e-02, dot -0.077; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.42254985776165, -2.7003811708469954, -1.5559514720723824, -0.6056779315520587
LBFGS Accumulation History: 3 points
Removed measurement 3e336662 to history. Total: 3
th(0)=-41.42254985776165;dx=-0.0017037508236694818
New Minimum: -41.42254985776165 > -41.73871245244036
WOLF (strong): th(252000.0000000001)=-41.73871245244036; dx=1.7212536686371126E-5 evalInputDelta=0.31616259467870833
New Minimum: -41.73871245244036 > -58.02555757622024
WOLF (strong): th(126000.00000000004)=-58.02555757622024; dx=5.239538263805392E-5 evalInputDelta=16.603007718458585
New Minimum: -58.02555757622024 > -70.44452188823841
END: th(42000.000000000015)=-70.44452188823841; dx=-1.3665336017231922E-4 evalInputDelta=29.02197203047676
Fitness changed from -41.42254985776165 to -70.44452188823841
Iteration 6 complete. Error: -70.44452188823841 Total: 5.1320; Orientation: 0.4591; Line Search: 3.7665
Adding measurement 4ba581f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.362e+04, gradient 4.449e-02, dot -0.094; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.44452188823841, -2.7003811708469954, -1.5559514720723824, -0.6056779315520587
LBFGS Accumulation History: 3 points
Removed measurement 4ba581f to history. Total: 3
th(0)=-70.44452188823841;dx=-0.0019565382066151545
New Minimum: -70.44452188823841 > -83.81078717489176
WOLF (strong): th(90486.25698133915)=-83.81078717489176; dx=5.257425729736268E-5 evalInputDelta=13.366265286653345
New Minimum: -83.81078717489176 > -92.49158006441722
END: th(45243.12849066957)=-92.49158006441722; dx=-7.583357882628359E-5 evalInputDelta=22.04705817617881
Fitness changed from -70.44452188823841 to -92.49158006441722
Iteration 7 complete. Error: -92.49158006441722 Total: 4.2526; Orientation: 0.5843; Line Search: 2.8129
Adding measurement 1512b134 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.918e+04, gradient 5.169e-02, dot -0.089; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.49158006441722, -2.7003811708469954, -1.5559514720723824, -0.6056779315520587
LBFGS Accumulation History: 3 points
Removed measurement 1512b134 to history. Total: 3
th(0)=-92.49158006441722;dx=-0.002647368066615806
New Minimum: -92.49158006441722 > -92.92693912760174
WOLF (strong): th(97473.3655058684)=-92.92693912760174; dx=9.268664345388604E-5 evalInputDelta=0.4353590631845208
New Minimum: -92.92693912760174 > -110.546370361039
WOLF (strong): th(48736.6827529342)=-110.546370361039; dx=2.1165142544031542E-4 evalInputDelta=18.054790296621775
New Minimum: -110.546370361039 > -115.9867062021549
END: th(16245.560917644732)=-115.9867062021549; dx=-6.317377344925065E-4 evalInputDelta=23.495126137737685
Fitness changed from -92.49158006441722 to -115.9867062021549
Iteration 8 complete. Error: -115.9867062021549 Total: 5.0806; Orientation: 0.4596; Line Search: 3.7594
Adding measurement 42ae185c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.706e+06, gradient 4.592e-02, dot -0.115; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -115.9867062021549, -2.7003811708469954, -1.5559514720723824, -0.6056779315520587
LBFGS Accumulation History: 3 points
Removed measurement 42ae185c to history. Total: 3
th(0)=-115.9867062021549;dx=-0.002082716930251564
New Minimum: -115.9867062021549 > -132.62634748372963
END: th(35000.000000000015)=-132.62634748372963; dx=-7.464925544855566E-5 evalInputDelta=16.639641281574725
Fitness changed from -115.9867062021549 to -132.62634748372963
Iteration 9 complete. Error: -132.62634748372963 Total: 3.2013; Orientation: 0.4631; Line Search: 1.8676
Adding measurement 569e5877 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.809e+04, gradient 6.173e-02, dot -0.065; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -132.62634748372963, -2.7003811708469954, -1.5559514720723824, -0.6056779315520587
LBFGS Accumulation History: 3 points
Removed measurement 569e5877 to history. Total: 3
th(0)=-132.62634748372963;dx=-0.003740752167685351
Armijo: th(75405.21415111597)=-109.18874766745655; dx=2.4233195061533591E-4 evalInputDelta=-23.437599816273078
New Minimum: -132.62634748372963 > -137.73593394608395
WOLF (strong): th(37702.607075557986)=-137.73593394608395; dx=3.9560477608969303E-4 evalInputDelta=5.109586462354315
New Minimum: -137.73593394608395 > -151.4769603974479
END: th(12567.535691852661)=-151.4769603974479; dx=-7.413683956850833E-4 evalInputDelta=18.850612913718265
Fitness changed from -132.62634748372963 to -151.4769603974479
Iteration 10 complete. Error: -151.4769603974479 Total: 5.0997; Orientation: 0.4552; Line Search: 3.7761
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 16d23134 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.568e+04, gradient 5.047e-02, dot 0.172; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-151.4769603974479;dx=-135.58251270342
Armijo: th(27075.934862741226)=-64.07928514860517; dx=8.84477679838826E-4 evalInputDelta=-87.39767524884273
Armijo: th(13537.967431370613)=-64.17862569495976; dx=0.0017963772591076087 evalInputDelta=-87.29833470248813
Armijo: th(4512.655810456871)=-65.23991555812476; dx=0.005260780641494382 evalInputDelta=-86.23704483932313
Armijo: th(1128.1639526142178)=-65.75322666155822; dx=0.020477407966644906 evalInputDelta=-85.72373373588968
Armijo: th(225.63279052284355)=-65.37495518640911; dx=0.10415688250775754 evalInputDelta=-86.10200521103879
Armijo: th(37.605465087140594)=-69.41167738234864; dx=0.5552748021731546 evalInputDelta=-82.06528301509925
Armijo: th(5.372209298162942)=-91.98665621640455; dx=4.927983678038046 evalInputDelta=-59.49030418104334
New Minimum: -151.4769603974479 > -162.10896460791426
WOLF (strong): th(0.6715261622703678)=-162.10896460791426; dx=10.773101998406435 evalInputDelta=10.632004210466363
END: th(0.07461401803004086)=-160.7007360359622; dx=-63.240942511573316 evalInputDelta=9.223775638514297
Fitness changed from -151.4769603974479 to -162.10896460791426
Iteration 11 complete. Error: -162.10896460791426 Total: 10.7603; Orientation: 0.4454; Line Search: 9.4027
Adding measurement 72ddddcc to history. Total: 4
Accepted: LBFGS Orientation magnitude: 6.555e+03, gradient 5.194e-02, dot 0.039; [d9fc741a-6147-453a-ac87-a7ef9c57d30b = 1.000/1.000e+00]
th(0)=-162.10896460791426;dx=-13.753129986132508
Armijo: th(0.16075102880658448)=-158.78414371994137; dx=7.916702997525135 evalInputDelta=-3.3248208879728907
Armijo: th(0.08037551440329224)=-160.4374756525108; dx=8.196029520334951 evalInputDelta=-1.671488955403447
Armijo: th(0.026791838134430746)=-161.54349501372687; dx=8.34227966303444 evalInputDelta=-0.5654695941873911
Armijo: th(0.0066979595336076865)=-161.96606391270672; dx=8.320827118965054 evalInputDelta=-0.142900695207544
Armijo: th(0.0013395919067215373)=-162.0804198538491; dx=8.041760309885088 evalInputDelta=-0.02854475406516599
Armijo: th(2.232653177869229E-4)=-162.10416331983888; dx=8.186107151985551 evalInputDelta=-0.004801288075384491
Armijo: th(3.189504539813184E-5)=-162.10825978532063; dx=8.361432740745835 evalInputDelta=-7.048225936330255E-4
Armijo: th(3.98688067476648E-6)=-162.1088621979841; dx=8.328240386563898 evalInputDelta=-1.0240993015031563E-4
Armijo: th(4.4298674164072E-7)=-162.10895858380252; dx=8.37924063836691 evalInputDelta=-6.0241117409987055E-6
Armijo: th(4.4298674164071996E-8)=-162.10896460791426; dx=8.341400593740758 evalInputDelta=0.0
Armijo: th(4.027152196733818E-9)=-162.10895858380252; dx=8.166948311932506 evalInputDelta=-6.0241117409987055E-6
Armijo: th(3.355960163944848E-10)=-162.10895858380252; dx=8.15585621128047 evalInputDelta=-6.0241117409987055E-6
MIN ALPHA (2.5815078184191137E-11): th(0.0)=-162.10896460791426
Fitness changed from -162.10896460791426 to -162.10896460791426
Static Iteration Total: 15.7607; Orientation: 0.6097; Line Search: 14.2856
Iteration 12 failed. Error: -162.10896460791426
Previous Error: 0.0 -> -162.10896460791426
Retrying iteration 12
Reset training subject: 2429750432980
Adding measurement 3bf048e9 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-162.10896460791426;dx=-0.0026777403088756218
END: th(2.1544346900318838E-10)=-162.10896460791426; dx=-0.0023166279294950234 evalInputDelta=0.0
Fitness changed from -162.10896460791426 to -162.10896460791426
Static Iteration Total: 3.6814; Orientation: 0.0203; Line Search: 1.9004
Iteration 13 failed. Error: -162.10896460791426
Previous Error: 0.0 -> -162.10896460791426
Optimization terminated 13
Final threshold in iteration 13: -162.10896460791426 (> -Infinity) after 77.136s (< 3600.000s)
Returns
-162.10896460791426