Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

BasicOptimizer.scala:89 executed in 60.59 seconds (0.782 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 148362405655700
Reset training subject: 148363323438000
Adding measurement 1cea6bbd to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-15.141570116941867;dx=-0.0015642790965752812
New Minimum: -15.141570116941867 > -15.145041350061096
WOLFE (weak): th(2.154434690031884)=-15.145041350061096; dx=-0.0015574241929374045 evalInputDelta=0.0034712331192281454
New Minimum: -15.145041350061096 > -15.148510485679436
WOLFE (weak): th(4.308869380063768)=-15.148510485679436; dx=-0.0015586853968737136 evalInputDelta=0.006940368737568647
New Minimum: -15.148510485679436 > -15.162394051015443
WOLFE (weak): th(12.926608140191302)=-15.162394051015443; dx=-0.0015647324457755789 evalInputDelta=0.020823934073575856
New Minimum: -15.162394051015443 > -15.224824515431118
WOLFE (weak): th(51.70643256076521)=-15.224824515431118; dx=-0.0015521238685275165 evalInputDelta=0.08325439848925065
New Minimum: -15.224824515431118 > -15.559368435869013
WOLFE (weak): th(258.53216280382605)=-15.559368435869013; dx=-0.0015674835714590651 evalInputDelta=0.4177983189271455
New Minimum: -15.559368435869013 > -17.64523062544915
WOLFE (weak): th(1551.1929768229563)=-17.64523062544915; dx=-0.0015534194631202218 evalInputDelta=2.5036605085072843
New Minimum: -17.64523062544915 > -30.586144358035394
END: th(10858.350837760694)=-30.586144358035394; dx=-0.0010452558303406549 evalInputDelta=15.444574241093527
Fitness changed from -15.141570116941867 to -30.586144358035394
Iteration 1 complete. Error: -30.586144358035394 Total: 9.6819; Orientation: 0.0153; Line Search: 7.1198
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/74242089-2476-4c90-a970-57f165cc6f68.jpg)
Adding measurement 46c9880b to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-30.586144358035394;dx=-0.0019371699467013149
New Minimum: -30.586144358035394 > -57.03539903355349
END: th(23393.607721408407)=-57.03539903355349; dx=-4.031085259246774E-4 evalInputDelta=26.449254675518095
Fitness changed from -30.586144358035394 to -57.03539903355349
Iteration 2 complete. Error: -57.03539903355349 Total: 2.6473; Orientation: 0.0154; Line Search: 1.6906
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>![Iteration 2](etc/947b0cef-d3a9-4022-9179-d4f501b9f297.jpg)
Adding measurement b65aaa4 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-57.03539903355349;dx=-0.002123582307570593
New Minimum: -57.03539903355349 > -75.055081841558
WOLF (strong): th(50400.00000000001)=-75.055081841558; dx=4.185472610370871E-5 evalInputDelta=18.019682808004518
New Minimum: -75.055081841558 > -82.48929400897764
END: th(25200.000000000004)=-82.48929400897764; dx=-3.980155034989924E-4 evalInputDelta=25.45389497542415
Fitness changed from -57.03539903355349 to -82.48929400897764
Iteration 3 complete. Error: -82.48929400897764 Total: 3.4585; Orientation: 0.0181; Line Search: 2.5292
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>![Iteration 3](etc/58b4972c-9cd4-4802-bf85-db5707e7dab3.jpg)
Adding measurement 4f90705c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.306e+03, gradient 4.723e-02, dot -0.239; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -82.48929400897764, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 4f90705c to history. Total: 3
th(0)=-82.48929400897764;dx=-0.0022232437610286366
New Minimum: -82.48929400897764 > -90.12993106751745
WOLF (strong): th(54291.75418880348)=-90.12993106751745; dx=1.7967300396989986E-4 evalInputDelta=7.640637058539809
New Minimum: -90.12993106751745 > -105.06109204130745
END: th(27145.87709440174)=-105.06109204130745; dx=-2.428026934737696E-4 evalInputDelta=22.57179803232981
Fitness changed from -82.48929400897764 to -105.06109204130745
Iteration 4 complete. Error: -105.06109204130745 Total: 3.8853; Orientation: 0.2728; Line Search: 2.6947
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>![Iteration 4](etc/7986a6a2-3c19-419f-86c1-6660b41f9d62.jpg)
Adding measurement 5bee40fc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.795e+03, gradient 4.577e-02, dot -0.221; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -105.06109204130745, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 5bee40fc to history. Total: 3
th(0)=-105.06109204130745;dx=-0.0020934173075143996
New Minimum: -105.06109204130745 > -108.25192195048278
WOLF (strong): th(58484.01930352103)=-108.25192195048278; dx=2.934272829876975E-4 evalInputDelta=3.190829909175335
New Minimum: -108.25192195048278 > -128.27184990689813
END: th(29242.009651760516)=-128.27184990689813; dx=-1.0430951058178497E-4 evalInputDelta=23.21075786559068
Fitness changed from -105.06109204130745 to -128.27184990689813
Iteration 5 complete. Error: -128.27184990689813 Total: 3.7980; Orientation: 0.2737; Line Search: 2.5969
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>![Iteration 5](etc/b4ea94a5-fd23-4542-959d-8defe7c0a98e.jpg)
Adding measurement 19d6eb52 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.881e+03, gradient 4.646e-02, dot -0.217; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -128.27184990689813, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 19d6eb52 to history. Total: 3
th(0)=-128.27184990689813;dx=-0.002149725818571072
Armijo: th(63000.00000000002)=-126.2553060865233; dx=2.9231571152462277E-4 evalInputDelta=-2.016543820374821
New Minimum: -128.27184990689813 > -150.65397160761336
WOLF (strong): th(31500.00000000001)=-150.65397160761336; dx=3.472969725078237E-5 evalInputDelta=22.38212170071523
END: th(10500.000000000004)=-147.737787179184; dx=-9.849290739721746E-4 evalInputDelta=19.465937272285885
Fitness changed from -128.27184990689813 to -150.65397160761336
Iteration 6 complete. Error: -150.65397160761336 Total: 4.7270; Orientation: 0.3011; Line Search: 3.4737
Adding measurement 23707801 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.194e+03, gradient 4.511e-02, dot -0.225; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -150.65397160761336, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 23707801 to history. Total: 3
th(0)=-150.65397160761336;dx=-0.0020218469351995427
New Minimum: -150.65397160761336 > -177.0992409977846
END: th(22621.564245334786)=-177.0992409977846; dx=-3.2713190856856845E-4 evalInputDelta=26.445269390171234
Fitness changed from -150.65397160761336 to -177.0992409977846
Iteration 7 complete. Error: -177.0992409977846 Total: 2.8244; Orientation: 0.2710; Line Search: 1.7344
Adding measurement 1c72c94 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.451e+04, gradient 3.885e-02, dot -0.286; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -177.0992409977846, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 1c72c94 to history. Total: 3
th(0)=-177.0992409977846;dx=-0.00150786735722568
New Minimum: -177.0992409977846 > -189.96577761153915
WOLF (strong): th(48736.6827529342)=-189.96577761153915; dx=8.067155484811885E-5 evalInputDelta=12.866536613754562
New Minimum: -189.96577761153915 > -195.905709445436
END: th(24368.3413764671)=-195.905709445436; dx=-3.497441045473184E-4 evalInputDelta=18.80646844765141
Fitness changed from -177.0992409977846 to -195.905709445436
Iteration 8 complete. Error: -195.905709445436 Total: 3.6669; Orientation: 0.2716; Line Search: 2.5723
Adding measurement 173cb476 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.910e+04, gradient 3.702e-02, dot -0.309; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -195.905709445436, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 173cb476 to history. Total: 3
th(0)=-195.905709445436;dx=-0.0013640660170591494
New Minimum: -195.905709445436 > -206.58850660889772
WOLF (strong): th(52500.00000000002)=-206.58850660889772; dx=4.956705637409777E-5 evalInputDelta=10.682797163461714
New Minimum: -206.58850660889772 > -213.6565020245301
END: th(26250.00000000001)=-213.6565020245301; dx=-2.985766673451759E-4 evalInputDelta=17.75079257909411
Fitness changed from -195.905709445436 to -213.6565020245301
Iteration 9 complete. Error: -213.6565020245301 Total: 3.8355; Orientation: 0.2746; Line Search: 2.7579
Adding measurement 45153ea4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.100e+04, gradient 3.814e-02, dot -0.323; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -213.6565020245301, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 45153ea4 to history. Total: 3
th(0)=-213.6565020245301;dx=-0.0014471217698274895
New Minimum: -213.6565020245301 > -219.07292460692204
WOLF (strong): th(56553.91061333697)=-219.07292460692204; dx=1.294026359914316E-4 evalInputDelta=5.416422582391931
New Minimum: -219.07292460692204 > -229.57927177495594
END: th(28276.955306668486)=-229.57927177495594; dx=-1.3951607506070457E-4 evalInputDelta=15.922769750425829
Fitness changed from -213.6565020245301 to -229.57927177495594
Iteration 10 complete. Error: -229.57927177495594 Total: 3.6235; Orientation: 0.2853; Line Search: 2.4997
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>![Iteration 10](etc/fd04de43-7bd9-4d4a-aa9e-837422a65bf0.jpg)
Adding measurement 3edd4634 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.445e+04, gradient 3.993e-02, dot -0.300; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -229.57927177495594, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 3edd4634 to history. Total: 3
th(0)=-229.57927177495594;dx=-0.0015786350649032587
Armijo: th(60920.853441167754)=-223.08824706658987; dx=2.5338697075192034E-4 evalInputDelta=-6.491024708366069
New Minimum: -229.57927177495594 > -244.7054879757931
WOLF (strong): th(30460.426720583877)=-244.7054879757931; dx=3.696263204545227E-5 evalInputDelta=15.126216200837149
END: th(10153.47557352796)=-242.66123875604924; dx=-8.289860623551349E-4 evalInputDelta=13.081966981093302
Fitness changed from -229.57927177495594 to -244.7054879757931
Iteration 11 complete. Error: -244.7054879757931 Total: 4.6711; Orientation: 0.2851; Line Search: 3.4353
Adding measurement 1a467ed0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.533e+04, gradient 4.255e-02, dot -0.307; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -244.7054879757931, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 1a467ed0 to history. Total: 3
th(0)=-244.7054879757931;dx=-0.0018094404615304417
New Minimum: -244.7054879757931 > -265.676952831175
END: th(21875.000000000015)=-265.676952831175; dx=-2.6107297515079035E-4 evalInputDelta=20.971464855381925
Fitness changed from -244.7054879757931 to -265.676952831175
Iteration 12 complete. Error: -265.676952831175 Total: 3.0060; Orientation: 0.4056; Line Search: 1.7925
Adding measurement 6e566cd0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.353e+05, gradient 3.347e-02, dot -0.387; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -265.676952831175, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 6e566cd0 to history. Total: 3
th(0)=-265.676952831175;dx=-0.001115602423330715
New Minimum: -265.676952831175 > -279.31327563875
END: th(47128.25884444749)=-279.31327563875; dx=-6.133249957917638E-5 evalInputDelta=13.63632280757497
Fitness changed from -265.676952831175 to -279.31327563875
Iteration 13 complete. Error: -279.31327563875 Total: 2.9644; Orientation: 0.3050; Line Search: 1.8124
Adding measurement 1f9a1324 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.053e+06, gradient 4.778e-02, dot -0.289; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -279.31327563875, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 1f9a1324 to history. Total: 3
th(0)=-279.31327563875;dx=-0.0022809032325075015
Armijo: th(101534.75573527961)=-224.27021205931106; dx=4.4928866510354033E-4 evalInputDelta=-55.04306357943892
New Minimum: -279.31327563875 > -279.88292697064327
WOLF (strong): th(50767.37786763981)=-279.88292697064327; dx=5.429359838284327E-4 evalInputDelta=0.569651331893283
New Minimum: -279.88292697064327 > -303.1228920842214
END: th(16922.459289213268)=-303.1228920842214; dx=-4.245082271524458E-4 evalInputDelta=23.80961644547142
Fitness changed from -279.31327563875 to -303.1228920842214
Iteration 14 complete. Error: -303.1228920842214 Total: 4.6536; Orientation: 0.2960; Line Search: 3.5278
Adding measurement 3d320f59 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.329e+05, gradient 3.147e-02, dot -0.407; [39af6616-479d-4b6b-bfa4-08d0271d28fb = 1.000/1.000e+00]
Orientation rejected. Popping history element from -303.1228920842214, -57.03539903355349, -30.586144358035394, -15.141570116941867
LBFGS Accumulation History: 3 points
Removed measurement 3d320f59 to history. Total: 3
th(0)=-303.1228920842214;dx=-9.836641680578075E-4
New Minimum: -303.1228920842214 > -320.96963071723115
END: th(36458.33333333336)=-320.96963071723115; dx=-4.025385906801495E-4 evalInputDelta=17.84673863300975
Fitness changed from -303.1228920842214 to -320.96963071723115
Iteration 15 complete. Error: -320.96963071723115 Total: 3.0475; Orientation: 0.2939; Line Search: 1.9288
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>![Iteration 15](etc/fac9ac3c-94bf-405e-8398-6c78ccf56435.jpg)
Final threshold in iteration 16: -320.96963071723115 (> -Infinity) after 60.592s (< 5400.000s)

Returns

    -320.96963071723115