Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

Iteration 20 Iteration 20

BasicOptimizer.scala:88 executed in 2100.37 seconds (29.442 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 2875447355647
Reset training subject: 2899670589809
Adding measurement 69dda6e6 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=47.8076576590538;dx=-2.4094698702321332E-5
New Minimum: 47.8076576590538 > 47.80763837695122
WOLFE (weak): th(2.154434690031884)=47.80763837695122; dx=-2.4074216245556875E-5 evalInputDelta=1.9282102584838867E-5
New Minimum: 47.80763837695122 > 47.807618141174316
WOLFE (weak): th(4.308869380063768)=47.807618141174316; dx=-2.4074120816078184E-5 evalInputDelta=3.9517879486083984E-5
New Minimum: 47.807618141174316 > 47.80753952264786
WOLFE (weak): th(12.926608140191302)=47.80753952264786; dx=-2.4073982188378106E-5 evalInputDelta=1.1813640594482422E-4
New Minimum: 47.80753952264786 > 47.807186633348465
WOLFE (weak): th(51.70643256076521)=47.807186633348465; dx=-2.4074799303067985E-5 evalInputDelta=4.710257053375244E-4
New Minimum: 47.807186633348465 > 47.80530422925949
WOLFE (weak): th(258.53216280382605)=47.80530422925949; dx=-2.4070743846481804E-5 evalInputDelta=0.0023534297943115234
New Minimum: 47.80530422925949 > 47.79372811317444
WOLFE (weak): th(1551.1929768229563)=47.79372811317444; dx=-2.3531224863952046E-5 evalInputDelta=0.013929545879364014
New Minimum: 47.79372811317444 > 47.713349133729935
END: th(10858.350837760694)=47.713349133729935; dx=-2.135824706466716E-5 evalInputDelta=0.0943085253238678
Fitness changed from 47.8076576590538 to 47.713349133729935
Iteration 1 complete. Error: 47.713349133729935 Total: 258.1822; Orientation: 0.1250; Line Search: 188.9681
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/9b286c08-0749-4755-b3ef-264d00f8ae6a.jpg)

    Adding measurement 2431b626 to history. Total: 1
    LBFGS Accumulation History: 2 points
    th(0)=47.713349133729935;dx=-2.0147914747113203E-5
    New Minimum: 47.713349133729935 > 47.54502984881401
    WOLFE (weak): th(23393.607721408407)=47.54502984881401; dx=-1.8161667161309712E-5 evalInputDelta=0.16831928491592407
    New Minimum: 47.54502984881401 > 47.39389368891716
    END: th(46787.215442816814)=47.39389368891716; dx=-1.671238414203584E-5 evalInputDelta=0.31945544481277466
    Fitness changed from 47.713349133729935 to 47.39389368891716
    Iteration 2 complete. Error: 47.39389368891716 Total: 92.9314; Orientation: 0.1239; Line Search: 70.7123
    <a id="p-5"></a>Iteration 2
    <a id="p-4"></a>![Iteration 2](etc/41a0cf6d-94d6-4166-848c-e43847520ca9.jpg)

    Adding measurement 52ae4d4a to history. Total: 2
    LBFGS Accumulation History: 3 points
    th(0)=47.39389368891716;dx=-1.5816634533624868E-5
    New Minimum: 47.39389368891716 > 46.862492710351944
    END: th(100800.00000000001)=46.862492710351944; dx=-1.28401715214829E-5 evalInputDelta=0.5314009785652161
    Fitness changed from 47.39389368891716 to 46.862492710351944
    Iteration 3 complete. Error: 46.862492710351944 Total: 69.8650; Orientation: 0.1251; Line Search: 47.1304
    <a id="p-7"></a>Iteration 3
    <a id="p-6"></a>![Iteration 3](etc/338c1611-7ddc-4333-89c7-0ccd27cb49a2.jpg)

    Adding measurement 344fbf6e to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 1.693e+03, gradient 3.506e-03, dot -0.942; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 46.862492710351944, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 344fbf6e to history. Total: 3
    th(0)=46.862492710351944;dx=-1.2292645482938259E-5
    New Minimum: 46.862492710351944 > 46.063509076833725
    END: th(217167.01675521393)=46.063509076833725; dx=-9.315290767856208E-6 evalInputDelta=0.798983633518219
    Fitness changed from 46.862492710351944 to 46.063509076833725
    Iteration 4 complete. Error: 46.063509076833725 Total: 72.3634; Orientation: 2.8089; Line Search: 47.0193
    <a id="p-9"></a>Iteration 4
    <a id="p-8"></a>![Iteration 4](etc/5ef60ac9-bd7a-432e-90a0-f4353a1bece7.jpg)

    Adding measurement 1d51202e to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.227e+03, gradient 3.067e-03, dot -0.876; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 46.063509076833725, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 1d51202e to history. Total: 3
    th(0)=46.063509076833725;dx=-9.406400366088689E-6
    New Minimum: 46.063509076833725 > 44.95352464914322
    END: th(467872.15442816826)=44.95352464914322; dx=-6.24142748862878E-6 evalInputDelta=1.109984427690506
    Fitness changed from 46.063509076833725 to 44.95352464914322
    Iteration 5 complete. Error: 44.95352464914322 Total: 73.1514; Orientation: 2.8444; Line Search: 47.4740
    <a id="p-11"></a>Iteration 5
    <a id="p-10"></a>![Iteration 5](etc/9c1bc26b-ad33-43f0-bd0e-27168341670f.jpg)

    Adding measurement 1d1a8dd to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 2.927e+03, gradient 2.694e-03, dot -0.762; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 44.95352464914322, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 1d1a8dd to history. Total: 3
    th(0)=44.95352464914322;dx=-7.259059356148246E-6
    New Minimum: 44.95352464914322 > 43.6553989648819
    END: th(1008000.0000000003)=43.6553989648819; dx=-3.585989262660557E-6 evalInputDelta=1.298125684261322
    Fitness changed from 44.95352464914322 to 43.6553989648819
    Iteration 6 complete. Error: 43.6553989648819 Total: 71.6196; Orientation: 2.8920; Line Search: 46.0897
    Adding measurement 675d45bb to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.574e+03, gradient 2.803e-03, dot -0.515; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 43.6553989648819, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 675d45bb to history. Total: 3
    th(0)=43.6553989648819;dx=-7.858232134133628E-6
    New Minimum: 43.6553989648819 > 43.20491769909859
    END: th(2171670.1675521396)=43.20491769909859; dx=-1.0396993593700423E-6 evalInputDelta=0.45048126578330994
    Fitness changed from 43.6553989648819 to 43.20491769909859
    Iteration 7 complete. Error: 43.20491769909859 Total: 71.0408; Orientation: 2.8040; Line Search: 46.6233
    Adding measurement 3f7e18be to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.305e+03, gradient 3.784e-03, dot -0.200; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 43.20491769909859, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 3f7e18be to history. Total: 3
    th(0)=43.20491769909859;dx=-1.4316726182512679E-5
    Armijo: th(4678721.544281683)=46.732086926698685; dx=1.4894882170186701E-6 evalInputDelta=-3.5271692276000977
    Armijo: th(2339360.7721408415)=43.21034646034241; dx=1.3474891307715844E-7 evalInputDelta=-0.00542876124382019
    New Minimum: 43.20491769909859 > 42.23713079094887
    END: th(779786.9240469472)=42.23713079094887; dx=-2.0438486200096727E-6 evalInputDelta=0.9677869081497192
    Fitness changed from 43.20491769909859 to 42.23713079094887
    Iteration 8 complete. Error: 42.23713079094887 Total: 120.7890; Orientation: 2.7870; Line Search: 94.8225
    Adding measurement 680c89c5 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.847e+03, gradient 2.385e-03, dot -0.301; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 42.23713079094887, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 680c89c5 to history. Total: 3
    th(0)=42.23713079094887;dx=-5.689310992604197E-6
    New Minimum: 42.23713079094887 > 41.52213418483734
    END: th(1680000.0000000007)=41.52213418483734; dx=-1.0429518051450537E-6 evalInputDelta=0.7149966061115265
    Fitness changed from 42.23713079094887 to 41.52213418483734
    Iteration 9 complete. Error: 41.52213418483734 Total: 73.4139; Orientation: 3.0663; Line Search: 46.5295
    Adding measurement dbef8d0 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 4.574e+03, gradient 2

...skipping 2802 bytes...

    40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 40.34998494386673, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 5c516684 to history. Total: 3
    th(0)=40.34998494386673;dx=-5.566375453794117E-6
    Armijo: th(4524312.849066958)=42.55766245722771; dx=1.0882438495351289E-6 evalInputDelta=-2.207677513360977
    New Minimum: 40.34998494386673 > 40.133141338825226
    WOLF (strong): th(2262156.424533479)=40.133141338825226; dx=1.6863398238007063E-7 evalInputDelta=0.2168436050415039
    New Minimum: 40.133141338825226 > 39.66326513886452
    END: th(754052.1415111596)=39.66326513886452; dx=-1.6437221492099174E-6 evalInputDelta=0.6867198050022125
    Fitness changed from 40.34998494386673 to 39.66326513886452
    Iteration 13 complete. Error: 39.66326513886452 Total: 120.3090; Orientation: 3.0429; Line Search: 94.6309
    Adding measurement 5dabd38f to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.393e+03, gradient 1.724e-03, dot -0.258; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 39.66326513886452, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 5dabd38f to history. Total: 3
    th(0)=39.66326513886452;dx=-2.97308532215655E-6
    New Minimum: 39.66326513886452 > 39.198297411203384
    END: th(1624556.0917644734)=39.198297411203384; dx=-7.517998573065663E-7 evalInputDelta=0.4649677276611328
    Fitness changed from 39.66326513886452 to 39.198297411203384
    Iteration 14 complete. Error: 39.198297411203384 Total: 69.9649; Orientation: 2.8103; Line Search: 45.1637
    Adding measurement 175723d9 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.824e+03, gradient 1.909e-03, dot -0.230; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 39.198297411203384, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 175723d9 to history. Total: 3
    th(0)=39.198297411203384;dx=-3.6429425961208236E-6
    Armijo: th(3500000.000000002)=39.93312257528305; dx=3.3097847948755593E-7 evalInputDelta=-0.7348251640796661
    New Minimum: 39.198297411203384 > 38.96357065439224
    END: th(1750000.000000001)=38.96357065439224; dx=-3.331086287858919E-7 evalInputDelta=0.23472675681114197
    Fitness changed from 39.198297411203384 to 38.96357065439224
    Iteration 15 complete. Error: 38.96357065439224 Total: 95.1796; Orientation: 3.2462; Line Search: 68.3656
    <a id="p-15"></a>Iteration 15
    <a id="p-14"></a>![Iteration 15](etc/10dc23bd-2a06-4e25-b600-f1095e2ad762.jpg)

    Adding measurement 2fccfa09 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.851e+03, gradient 1.853e-03, dot -0.220; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 38.96357065439224, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 2fccfa09 to history. Total: 3
    th(0)=38.96357065439224;dx=-3.4333043189695914E-6
    Armijo: th(3770260.707555799)=40.15396264195442; dx=6.878192934751071E-7 evalInputDelta=-1.1903919875621796
    New Minimum: 38.96357065439224 > 38.795719504356384
    END: th(1885130.3537778994)=38.795719504356384; dx=-2.762804906672336E-8 evalInputDelta=0.16785115003585815
    Fitness changed from 38.96357065439224 to 38.795719504356384
    Iteration 16 complete. Error: 38.795719504356384 Total: 96.2848; Orientation: 3.2390; Line Search: 69.3481
    Adding measurement 673d85d0 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 5.493e+03, gradient 2.150e-03, dot -0.169; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 38.795719504356384, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 673d85d0 to history. Total: 3
    th(0)=38.795719504356384;dx=-4.620388158825608E-6
    Armijo: th(4061390.2294111843)=40.53482976555824; dx=9.99948717159046E-7 evalInputDelta=-1.7391102612018585
    New Minimum: 38.795719504356384 > 38.77304798364639
    WOLF (strong): th(2030695.1147055922)=38.77304798364639; dx=2.947970236128269E-7 evalInputDelta=0.022671520709991455
    New Minimum: 38.77304798364639 > 38.4044069647789
    END: th(676898.3715685307)=38.4044069647789; dx=-7.47927730450369E-7 evalInputDelta=0.39131253957748413
    Fitness changed from 38.795719504356384 to 38.4044069647789
    Iteration 17 complete. Error: 38.4044069647789 Total: 120.6520; Orientation: 2.9964; Line Search: 94.1559
    Adding measurement 17761d7b to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 3.950e+04, gradient 5.487e-02, dot -0.029; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 38.4044069647789, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 17761d7b to history. Total: 3
    th(0)=38.4044069647789;dx=-0.003010948487754292
    Armijo: th(1458333.3333333344)=42.85553923249245; dx=4.819856035752004E-6 evalInputDelta=-4.451132267713547
    Armijo: th(729166.6666666672)=41.509205758571625; dx=6.770351544659029E-6 evalInputDelta=-3.1047987937927246
    Armijo: th(243055.55555555574)=39.62816071510315; dx=1.0925684557866865E-5 evalInputDelta=-1.2237537503242493
    Armijo: th(60763.888888888934)=38.580708771944046; dx=5.947500910591799E-6 evalInputDelta=-0.17630180716514587
    Armijo: th(12152.777777777786)=38.41083863377571; dx=-9.164486900855675E-8 evalInputDelta=-0.006431668996810913
    New Minimum: 38.4044069647789 > 38.403570502996445
    END: th(2025.4629629629644)=38.403570502996445; dx=-1.2687809231117437E-6 evalInputDelta=8.364617824554443E-4
    Fitness changed from 38.4044069647789 to 38.403570502996445
    Iteration 18 complete. Error: 38.403570502996445 Total: 190.3653; Orientation: 2.7694; Line Search: 164.8306
    Adding measurement 48109f4f to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 9.628e+03, gradient 1.132e-02, dot -0.067; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 38.403570502996445, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 48109f4f to history. Total: 3
    th(0)=38.403570502996445;dx=-1.2813237730730678E-4
    New Minimum: 38.403570502996445 > 38.40166598558426
    END: th(4363.727670782175)=38.40166598558426; dx=-2.368605237345586E-6 evalInputDelta=0.001904517412185669
    Fitness changed from 38.403570502996445 to 38.40166598558426
    Iteration 19 complete. Error: 38.40166598558426 Total: 72.0015; Orientation: 2.7658; Line Search: 46.7303
    Adding measurement 3f889956 to history. Total: 3
    Rejected: LBFGS Orientation magnitude: 6.278e+03, gradient 1.449e-03, dot -0.266; [b6defdda-1d5f-4414-a75b-be40d598c979 = 1.000/1.000e+00]
    Orientation rejected. Popping history element from 38.40166598558426, 47.39389368891716, 47.713349133729935, 47.8076576590538
    LBFGS Accumulation History: 3 points
    Removed measurement 3f889956 to history. Total: 3
    th(0)=38.40166598558426;dx=-2.100664421373774E-6
    New Minimum: 38.40166598558426 > 38.39622324705124
    WOLFE (weak): th(9401.36627178515)=38.39622324705124; dx=-2.0634480762969646E-6 evalInputDelta=0.0054427385330200195
    New Minimum: 38.39622324705124 > 38.39081418514252
    WOLFE (weak): th(18802.7325435703)=38.39081418514252; dx=-2.058140003989904E-6 evalInputDelta=0.010851800441741943
    New Minimum: 38.39081418514252 > 38.368956327438354
    WOLFE (weak): th(56408.1976307109)=38.368956327438354; dx=-1.0906287180747116E-5 evalInputDelta=0.03270965814590454
    New Minimum: 38.368956327438354 > 38.28890618681908
    END: th(225632.7905228436)=38.28890618681908; dx=-1.6569221738334265E-6 evalInputDelta=0.1127597987651825
    Fitness changed from 38.40166598558426 to 38.28890618681908
    Iteration 20 complete. Error: 38.28890618681908 Total: 142.6558; Orientation: 2.8535; Line Search: 116.1837
    <a id="p-17"></a>Iteration 20
    <a id="p-16"></a>![Iteration 20](etc/794a38db-31e2-45c9-8a1c-8df5c55ffcd1.jpg)

    Final threshold in iteration 21: 38.28890618681908 (> -Infinity) after 2100.366s (< 5400.000s)

Returns

    38.28890618681908