BasicOptimizer.scala:88 executed in 2100.37 seconds (29.442 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 2875447355647
Reset training subject: 2899670589809
Adding measurement 69dda6e6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=47.8076576590538;dx=-2.4094698702321332E-5
New Minimum: 47.8076576590538 > 47.80763837695122
WOLFE (weak): th(2.154434690031884)=47.80763837695122; dx=-2.4074216245556875E-5 evalInputDelta=1.9282102584838867E-5
New Minimum: 47.80763837695122 > 47.807618141174316
WOLFE (weak): th(4.308869380063768)=47.807618141174316; dx=-2.4074120816078184E-5 evalInputDelta=3.9517879486083984E-5
New Minimum: 47.807618141174316 > 47.80753952264786
WOLFE (weak): th(12.926608140191302)=47.80753952264786; dx=-2.4073982188378106E-5 evalInputDelta=1.1813640594482422E-4
New Minimum: 47.80753952264786 > 47.807186633348465
WOLFE (weak): th(51.70643256076521)=47.807186633348465; dx=-2.4074799303067985E-5 evalInputDelta=4.710257053375244E-4
New Minimum: 47.807186633348465 > 47.80530422925949
WOLFE (weak): th(258.53216280382605)=47.80530422925949; dx=-2.4070743846481804E-5 evalInputDelta=0.0023534297943115234
New Minimum: 47.80530422925949 > 47.79372811317444
WOLFE (weak): th(1551.1929768229563)=47.79372811317444; dx=-2.3531224863952046E-5 evalInputDelta=0.013929545879364014
New Minimum: 47.79372811317444 > 47.713349133729935
END: th(10858.350837760694)=47.713349133729935; dx=-2.135824706466716E-5 evalInputDelta=0.0943085253238678
Fitness changed from 47.8076576590538 to 47.713349133729935
Iteration 1 complete. Error: 47.713349133729935 Total: 258.1822; Orientation: 0.1250; Line Search: 188.9681
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2431b626 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=47.713349133729935;dx=-2.0147914747113203E-5
New Minimum: 47.713349133729935 > 47.54502984881401
WOLFE (weak): th(23393.607721408407)=47.54502984881401; dx=-1.8161667161309712E-5 evalInputDelta=0.16831928491592407
New Minimum: 47.54502984881401 > 47.39389368891716
END: th(46787.215442816814)=47.39389368891716; dx=-1.671238414203584E-5 evalInputDelta=0.31945544481277466
Fitness changed from 47.713349133729935 to 47.39389368891716
Iteration 2 complete. Error: 47.39389368891716 Total: 92.9314; Orientation: 0.1239; Line Search: 70.7123
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 52ae4d4a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=47.39389368891716;dx=-1.5816634533624868E-5
New Minimum: 47.39389368891716 > 46.862492710351944
END: th(100800.00000000001)=46.862492710351944; dx=-1.28401715214829E-5 evalInputDelta=0.5314009785652161
Fitness changed from 47.39389368891716 to 46.862492710351944
Iteration 3 complete. Error: 46.862492710351944 Total: 69.8650; Orientation: 0.1251; Line Search: 47.1304
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 344fbf6e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.693e+03, gradient 3.506e-03, dot -0.942; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 46.862492710351944, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 344fbf6e to history. Total: 3
th(0)=46.862492710351944;dx=-1.2292645482938259E-5
New Minimum: 46.862492710351944 > 46.063509076833725
END: th(217167.01675521393)=46.063509076833725; dx=-9.315290767856208E-6 evalInputDelta=0.798983633518219
Fitness changed from 46.862492710351944 to 46.063509076833725
Iteration 4 complete. Error: 46.063509076833725 Total: 72.3634; Orientation: 2.8089; Line Search: 47.0193
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 1d51202e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.227e+03, gradient 3.067e-03, dot -0.876; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 46.063509076833725, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 1d51202e to history. Total: 3
th(0)=46.063509076833725;dx=-9.406400366088689E-6
New Minimum: 46.063509076833725 > 44.95352464914322
END: th(467872.15442816826)=44.95352464914322; dx=-6.24142748862878E-6 evalInputDelta=1.109984427690506
Fitness changed from 46.063509076833725 to 44.95352464914322
Iteration 5 complete. Error: 44.95352464914322 Total: 73.1514; Orientation: 2.8444; Line Search: 47.4740
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 1d1a8dd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.927e+03, gradient 2.694e-03, dot -0.762; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 44.95352464914322, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 1d1a8dd to history. Total: 3
th(0)=44.95352464914322;dx=-7.259059356148246E-6
New Minimum: 44.95352464914322 > 43.6553989648819
END: th(1008000.0000000003)=43.6553989648819; dx=-3.585989262660557E-6 evalInputDelta=1.298125684261322
Fitness changed from 44.95352464914322 to 43.6553989648819
Iteration 6 complete. Error: 43.6553989648819 Total: 71.6196; Orientation: 2.8920; Line Search: 46.0897
Adding measurement 675d45bb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.574e+03, gradient 2.803e-03, dot -0.515; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 43.6553989648819, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 675d45bb to history. Total: 3
th(0)=43.6553989648819;dx=-7.858232134133628E-6
New Minimum: 43.6553989648819 > 43.20491769909859
END: th(2171670.1675521396)=43.20491769909859; dx=-1.0396993593700423E-6 evalInputDelta=0.45048126578330994
Fitness changed from 43.6553989648819 to 43.20491769909859
Iteration 7 complete. Error: 43.20491769909859 Total: 71.0408; Orientation: 2.8040; Line Search: 46.6233
Adding measurement 3f7e18be to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.305e+03, gradient 3.784e-03, dot -0.200; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 43.20491769909859, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 3f7e18be to history. Total: 3
th(0)=43.20491769909859;dx=-1.4316726182512679E-5
Armijo: th(4678721.544281683)=46.732086926698685; dx=1.4894882170186701E-6 evalInputDelta=-3.5271692276000977
Armijo: th(2339360.7721408415)=43.21034646034241; dx=1.3474891307715844E-7 evalInputDelta=-0.00542876124382019
New Minimum: 43.20491769909859 > 42.23713079094887
END: th(779786.9240469472)=42.23713079094887; dx=-2.0438486200096727E-6 evalInputDelta=0.9677869081497192
Fitness changed from 43.20491769909859 to 42.23713079094887
Iteration 8 complete. Error: 42.23713079094887 Total: 120.7890; Orientation: 2.7870; Line Search: 94.8225
Adding measurement 680c89c5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.847e+03, gradient 2.385e-03, dot -0.301; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 42.23713079094887, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 680c89c5 to history. Total: 3
th(0)=42.23713079094887;dx=-5.689310992604197E-6
New Minimum: 42.23713079094887 > 41.52213418483734
END: th(1680000.0000000007)=41.52213418483734; dx=-1.0429518051450537E-6 evalInputDelta=0.7149966061115265
Fitness changed from 42.23713079094887 to 41.52213418483734
Iteration 9 complete. Error: 41.52213418483734 Total: 73.4139; Orientation: 3.0663; Line Search: 46.5295
Adding measurement dbef8d0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.574e+03, gradient 2
...skipping 2802 bytes...
40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 40.34998494386673, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 5c516684 to history. Total: 3
th(0)=40.34998494386673;dx=-5.566375453794117E-6
Armijo: th(4524312.849066958)=42.55766245722771; dx=1.0882438495351289E-6 evalInputDelta=-2.207677513360977
New Minimum: 40.34998494386673 > 40.133141338825226
WOLF (strong): th(2262156.424533479)=40.133141338825226; dx=1.6863398238007063E-7 evalInputDelta=0.2168436050415039
New Minimum: 40.133141338825226 > 39.66326513886452
END: th(754052.1415111596)=39.66326513886452; dx=-1.6437221492099174E-6 evalInputDelta=0.6867198050022125
Fitness changed from 40.34998494386673 to 39.66326513886452
Iteration 13 complete. Error: 39.66326513886452 Total: 120.3090; Orientation: 3.0429; Line Search: 94.6309
Adding measurement 5dabd38f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.393e+03, gradient 1.724e-03, dot -0.258; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 39.66326513886452, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 5dabd38f to history. Total: 3
th(0)=39.66326513886452;dx=-2.97308532215655E-6
New Minimum: 39.66326513886452 > 39.198297411203384
END: th(1624556.0917644734)=39.198297411203384; dx=-7.517998573065663E-7 evalInputDelta=0.4649677276611328
Fitness changed from 39.66326513886452 to 39.198297411203384
Iteration 14 complete. Error: 39.198297411203384 Total: 69.9649; Orientation: 2.8103; Line Search: 45.1637
Adding measurement 175723d9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.824e+03, gradient 1.909e-03, dot -0.230; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 39.198297411203384, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 175723d9 to history. Total: 3
th(0)=39.198297411203384;dx=-3.6429425961208236E-6
Armijo: th(3500000.000000002)=39.93312257528305; dx=3.3097847948755593E-7 evalInputDelta=-0.7348251640796661
New Minimum: 39.198297411203384 > 38.96357065439224
END: th(1750000.000000001)=38.96357065439224; dx=-3.331086287858919E-7 evalInputDelta=0.23472675681114197
Fitness changed from 39.198297411203384 to 38.96357065439224
Iteration 15 complete. Error: 38.96357065439224 Total: 95.1796; Orientation: 3.2462; Line Search: 68.3656
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Adding measurement 2fccfa09 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.851e+03, gradient 1.853e-03, dot -0.220; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 38.96357065439224, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 2fccfa09 to history. Total: 3
th(0)=38.96357065439224;dx=-3.4333043189695914E-6
Armijo: th(3770260.707555799)=40.15396264195442; dx=6.878192934751071E-7 evalInputDelta=-1.1903919875621796
New Minimum: 38.96357065439224 > 38.795719504356384
END: th(1885130.3537778994)=38.795719504356384; dx=-2.762804906672336E-8 evalInputDelta=0.16785115003585815
Fitness changed from 38.96357065439224 to 38.795719504356384
Iteration 16 complete. Error: 38.795719504356384 Total: 96.2848; Orientation: 3.2390; Line Search: 69.3481
Adding measurement 673d85d0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.493e+03, gradient 2.150e-03, dot -0.169; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 38.795719504356384, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 673d85d0 to history. Total: 3
th(0)=38.795719504356384;dx=-4.620388158825608E-6
Armijo: th(4061390.2294111843)=40.53482976555824; dx=9.99948717159046E-7 evalInputDelta=-1.7391102612018585
New Minimum: 38.795719504356384 > 38.77304798364639
WOLF (strong): th(2030695.1147055922)=38.77304798364639; dx=2.947970236128269E-7 evalInputDelta=0.022671520709991455
New Minimum: 38.77304798364639 > 38.4044069647789
END: th(676898.3715685307)=38.4044069647789; dx=-7.47927730450369E-7 evalInputDelta=0.39131253957748413
Fitness changed from 38.795719504356384 to 38.4044069647789
Iteration 17 complete. Error: 38.4044069647789 Total: 120.6520; Orientation: 2.9964; Line Search: 94.1559
Adding measurement 17761d7b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.950e+04, gradient 5.487e-02, dot -0.029; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 38.4044069647789, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 17761d7b to history. Total: 3
th(0)=38.4044069647789;dx=-0.003010948487754292
Armijo: th(1458333.3333333344)=42.85553923249245; dx=4.819856035752004E-6 evalInputDelta=-4.451132267713547
Armijo: th(729166.6666666672)=41.509205758571625; dx=6.770351544659029E-6 evalInputDelta=-3.1047987937927246
Armijo: th(243055.55555555574)=39.62816071510315; dx=1.0925684557866865E-5 evalInputDelta=-1.2237537503242493
Armijo: th(60763.888888888934)=38.580708771944046; dx=5.947500910591799E-6 evalInputDelta=-0.17630180716514587
Armijo: th(12152.777777777786)=38.41083863377571; dx=-9.164486900855675E-8 evalInputDelta=-0.006431668996810913
New Minimum: 38.4044069647789 > 38.403570502996445
END: th(2025.4629629629644)=38.403570502996445; dx=-1.2687809231117437E-6 evalInputDelta=8.364617824554443E-4
Fitness changed from 38.4044069647789 to 38.403570502996445
Iteration 18 complete. Error: 38.403570502996445 Total: 190.3653; Orientation: 2.7694; Line Search: 164.8306
Adding measurement 48109f4f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.628e+03, gradient 1.132e-02, dot -0.067; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 38.403570502996445, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 48109f4f to history. Total: 3
th(0)=38.403570502996445;dx=-1.2813237730730678E-4
New Minimum: 38.403570502996445 > 38.40166598558426
END: th(4363.727670782175)=38.40166598558426; dx=-2.368605237345586E-6 evalInputDelta=0.001904517412185669
Fitness changed from 38.403570502996445 to 38.40166598558426
Iteration 19 complete. Error: 38.40166598558426 Total: 72.0015; Orientation: 2.7658; Line Search: 46.7303
Adding measurement 3f889956 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.278e+03, gradient 1.449e-03, dot -0.266; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
Orientation rejected. Popping history element from 38.40166598558426, 47.39389368891716, 47.713349133729935, 47.8076576590538
LBFGS Accumulation History: 3 points
Removed measurement 3f889956 to history. Total: 3
th(0)=38.40166598558426;dx=-2.100664421373774E-6
New Minimum: 38.40166598558426 > 38.39622324705124
WOLFE (weak): th(9401.36627178515)=38.39622324705124; dx=-2.0634480762969646E-6 evalInputDelta=0.0054427385330200195
New Minimum: 38.39622324705124 > 38.39081418514252
WOLFE (weak): th(18802.7325435703)=38.39081418514252; dx=-2.058140003989904E-6 evalInputDelta=0.010851800441741943
New Minimum: 38.39081418514252 > 38.368956327438354
WOLFE (weak): th(56408.1976307109)=38.368956327438354; dx=-1.0906287180747116E-5 evalInputDelta=0.03270965814590454
New Minimum: 38.368956327438354 > 38.28890618681908
END: th(225632.7905228436)=38.28890618681908; dx=-1.6569221738334265E-6 evalInputDelta=0.1127597987651825
Fitness changed from 38.40166598558426 to 38.28890618681908
Iteration 20 complete. Error: 38.28890618681908 Total: 142.6558; Orientation: 2.8535; Line Search: 116.1837
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Final threshold in iteration 21: 38.28890618681908 (> -Infinity) after 2100.366s (< 5400.000s)
Returns
38.28890618681908