Iteration 1 Iteration 1

Iteration 2 Iteration 2

Iteration 3 Iteration 3

Iteration 4 Iteration 4

Iteration 5 Iteration 5

Iteration 10 Iteration 10

Iteration 15 Iteration 15

BasicOptimizer.scala:89 executed in 63.62 seconds (0.952 gc):

    val lineSearchInstance: LineSearchStrategy = lineSearchFactory
    val trainer = new IterativeTrainer(trainable)
    trainer.setOrientation(orientation())
    trainer.setMonitor(new TrainingMonitor() {
      override def clear(): Unit = trainingMonitor.clear()
  
      override def log(msg: String): Unit = {
        trainingMonitor.log(msg)
        BasicOptimizer.this.log(msg)
      }
  
      override def onStepFail(currentPoint: Step): Boolean = {
        BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
      }
  
      override def onStepComplete(currentPoint: Step): Unit = {
        if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
          val image = currentImage()
          timelineAnimation += image
          val caption = "Iteration " + currentPoint.iteration
          out.p(caption + "\n" + out.jpg(image, caption))
        }
        BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
        trainingMonitor.onStepComplete(currentPoint)
        super.onStepComplete(currentPoint)
      }
    })
    trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
    trainer.setMaxIterations(trainingIterations)
    trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
    trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
    val result = trainer.run.asInstanceOf[lang.Double]
    trainer.freeRef()
    result
Logging
Reset training subject: 138708566836100
Reset training subject: 138709482391400
Adding measurement 48214788 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-2.372228979910108;dx=-1.3979703075829001E-5
New Minimum: -2.372228979910108 > -2.372257675221315
WOLFE (weak): th(2.154434690031884)=-2.372257675221315; dx=-1.3890829210998794E-5 evalInputDelta=2.8695311207194152E-5
New Minimum: -2.372257675221315 > -2.37228646304882
WOLFE (weak): th(4.308869380063768)=-2.37228646304882; dx=-1.3886757831004661E-5 evalInputDelta=5.7483138712122184E-5
New Minimum: -2.37228646304882 > -2.372400775135443
WOLFE (weak): th(12.926608140191302)=-2.372400775135443; dx=-1.3899149241617418E-5 evalInputDelta=1.717952253348365E-4
New Minimum: -2.372400775135443 > -2.3729172002501073
WOLFE (weak): th(51.70643256076521)=-2.3729172002501073; dx=-1.3883657313934066E-5 evalInputDelta=6.882203399993791E-4
New Minimum: -2.3729172002501073 > -2.3756679024052003
WOLFE (weak): th(258.53216280382605)=-2.3756679024052003; dx=-1.3890913885130836E-5 evalInputDelta=0.003438922495092367
New Minimum: -2.3756679024052003 > -2.39284472436939
WOLFE (weak): th(1551.1929768229563)=-2.39284472436939; dx=-1.3867142423000376E-5 evalInputDelta=0.020615744459282048
New Minimum: -2.39284472436939 > -2.5166242745874676
WOLFE (weak): th(10858.350837760694)=-2.5166242745874676; dx=-1.381458776064657E-5 evalInputDelta=0.14439529467735968
New Minimum: -2.5166242745874676 > -3.4517554875106478
END: th(86866.80670208555)=-3.4517554875106478; dx=-1.1836898187671689E-5 evalInputDelta=1.0795265076005398
Fitness changed from -2.372228979910108 to -3.4517554875106478
Iteration 1 complete. Error: -3.4517554875106478 Total: 10.8543; Orientation: 0.0185; Line Search: 8.1809
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>![Iteration 1](etc/690f6e21-2d6a-4060-bedf-b635ecc133b6.jpg)
Adding measurement 3411ba50 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.4517554875106478;dx=-1.5529647247584742E-5
New Minimum: -3.4517554875106478 > -5.4663161052573725
END: th(187148.86177126726)=-5.4663161052573725; dx=-8.522872971489161E-6 evalInputDelta=2.014560617746725
Fitness changed from -3.4517554875106478 to -5.4663161052573725
Iteration 2 complete. Error: -5.4663161052573725 Total: 3.0191; Orientation: 0.0158; Line Search: 2.0268
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>![Iteration 2](etc/f5c6c1b4-6d9f-4e27-8587-e4051d0157ed.jpg)
Adding measurement 3f55fd0e to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-5.4663161052573725;dx=-1.7572021620044046E-5
New Minimum: -5.4663161052573725 > -8.400122885731593
END: th(403200.00000000006)=-8.400122885731593; dx=-5.372269782700727E-6 evalInputDelta=2.93380678047422
Fitness changed from -5.4663161052573725 to -8.400122885731593
Iteration 3 complete. Error: -8.400122885731593 Total: 3.1887; Orientation: 0.0149; Line Search: 1.8352
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>![Iteration 3](etc/6d19c819-7ebd-4277-99b5-5235a9fc164d.jpg)
Adding measurement 3db75056 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.245e+04, gradient 4.403e-03, dot -0.322; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -8.400122885731593, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 3db75056 to history. Total: 3
th(0)=-8.400122885731593;dx=-1.9208794310628905E-5
New Minimum: -8.400122885731593 > -12.195116641517359
END: th(868668.0670208557)=-12.195116641517359; dx=-2.600100409849447E-6 evalInputDelta=3.7949937557857663
Fitness changed from -8.400122885731593 to -12.195116641517359
Iteration 4 complete. Error: -12.195116641517359 Total: 3.2504; Orientation: 0.3117; Line Search: 1.9637
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>![Iteration 4](etc/6a4811bd-e046-4cce-80da-6bc0765c1c5e.jpg)
Adding measurement a74d87b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.803e+04, gradient 4.417e-03, dot -0.207; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -12.195116641517359, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement a74d87b to history. Total: 3
th(0)=-12.195116641517359;dx=-1.9423734030787152E-5
New Minimum: -12.195116641517359 > -15.212196802028036
WOLF (strong): th(1871488.617712673)=-15.212196802028036; dx=1.455804285819965E-7 evalInputDelta=3.0170801605106767
New Minimum: -15.212196802028036 > -17.149750881751384
END: th(935744.3088563365)=-17.149750881751384; dx=-1.7601236594764643E-6 evalInputDelta=4.954634240234025
Fitness changed from -12.195116641517359 to -17.149750881751384
Iteration 5 complete. Error: -17.149750881751384 Total: 4.1199; Orientation: 0.2972; Line Search: 2.7628
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>![Iteration 5](etc/f6494d7c-f5eb-4002-b3cb-0d52112b09ae.jpg)
Adding measurement 745b8053 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.880e+04, gradient 4.232e-03, dot -0.172; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -17.149750881751384, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 745b8053 to history. Total: 3
th(0)=-17.149750881751384;dx=-1.779324777751982E-5
New Minimum: -17.149750881751384 > -17.159392430998757
WOLF (strong): th(2016000.0000000007)=-17.159392430998757; dx=7.828592850758215E-7 evalInputDelta=0.009641549247373149
New Minimum: -17.159392430998757 > -20.9887357087579
END: th(1008000.0000000003)=-20.9887357087579; dx=-7.158170703659605E-7 evalInputDelta=3.8389848270065166
Fitness changed from -17.149750881751384 to -20.9887357087579
Iteration 6 complete. Error: -20.9887357087579 Total: 3.9883; Orientation: 0.2852; Line Search: 2.7484
Adding measurement 467ba23 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.269e+04, gradient 4.275e-03, dot -0.198; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.9887357087579, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 467ba23 to history. Total: 3
th(0)=-20.9887357087579;dx=-1.8185340924193847E-5
Armijo: th(2171670.1675521396)=-20.063553980368724; dx=1.4360036905078502E-6 evalInputDelta=-0.9251817283891768
New Minimum: -20.9887357087579 > -24.845857366875503
WOLF (strong): th(1085835.0837760698)=-24.845857366875503; dx=4.5152154055506883E-7 evalInputDelta=3.8571216581176024
New Minimum: -24.845857366875503 > -25.24003759519242
END: th(361945.0279253566)=-25.24003759519242; dx=-7.67527916181239E-6 evalInputDelta=4.2513018864345185
Fitness changed from -20.9887357087579 to -25.24003759519242
Iteration 7 complete. Error: -25.24003759519242 Total: 4.8742; Orientation: 0.2843; Line Search: 3.7710
Adding measurement 35860edd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.730e+04, gradient 3.404e-03, dot -0.259; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -25.24003759519242, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 35860edd to history. Total: 3
th(0)=-25.24003759519242;dx=-1.1455925315669964E-5
New Minimum: -25.24003759519242 > -28.437504700362688
END: th(779786.9240469472)=-28.437504700362688; dx=-3.6154352595879224E-6 evalInputDelta=3.1974671051702686
Fitness changed from -25.24003759519242 to -28.437504700362688
Iteration 8 complete. Error: -28.437504700362688 Total: 2.8606; Orientation: 0.2796; Line Search: 1.7504
Adding measurement 4178ef21 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.105e+04, gradient 3.603e-03, dot -0.273; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -28.437504700362688, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 4178ef21 to history. Total: 3
th(0)=-28.437504700362688;dx=-1.2916892130410009E-5
Armijo: th(1680000.0000000007)=-28.21473414293807; dx=2.630659592603495E-7 evalInputDelta=-0.22277055742461727
New Minimum: -28.437504700362688 > -30.539527072241672
END: th(840000.0000000003)=-30.539527072241672; dx=-1.3019485239826524E-6 evalInputDelta=2.1020223718789843
Fitness changed from -28.437504700362688 to -30.539527072241672
Iteration 9 complete. Error: -30.539527072241672 Total: 3.9555; Orientation: 0.2794; Line Search: 2.8166
Adding measurement ebd6b18 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.360e+04, gradient 3.873e-03, dot -0.241; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -30.539527072241672, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement ebd6b18 to history. Total: 3
th(0)=-30.539527072241672;dx=-1.4941036654548911E-5
Armijo: th(1809725.139626783)=-27.483401794722088; dx=2.2065135330040135E-6 evalInputDelta=-3.056125277519584
New Minimum: -30.539527072241672 > -32.43533622490819
WOLF (strong): th(904862.5698133915)=-32.43533622490819; dx=8.463554028427363E-7 evalInputDelta=1.8958091526665193
New Minimum: -32.43533622490819 > -33.08549828380127
END: th(301620.8566044638)=-33.08549828380127; dx=-6.3815905546387775E-6 evalInputDelta=2.5459712115596
Fitness changed from -30.539527072241672 to -33.08549828380127
Iteration 10 complete. Error: -33.08549828380127 Total: 4.8801; Orientation: 0.2806; Line Search: 3.7231
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>![Iteration 10](etc/1563cdec-3c87-4146-9516-f3bae3c14589.jpg)
Adding measurement 7e97b443 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.824e+05, gradient 3.104e-03, dot -0.330; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.08549828380127, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 7e97b443 to history. Total: 3
th(0)=-33.08549828380127;dx=-9.52487591226207E-6
New Minimum: -33.08549828380127 > -35.217144918352965
END: th(649822.4367057893)=-35.217144918352965; dx=-3.7852174558588436E-6 evalInputDelta=2.131646634551693
Fitness changed from -33.08549828380127 to -35.217144918352965
Iteration 11 complete. Error: -35.217144918352965 Total: 2.9275; Orientation: 0.2790; Line Search: 1.7092
Adding measurement 649dfff8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.766e+05, gradient 3.175e-03, dot -0.324; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -35.217144918352965, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 649dfff8 to history. Total: 3
th(0)=-35.217144918352965;dx=-1.0036622614659331E-5
New Minimum: -35.217144918352965 > -36.22988857045814
END: th(1400000.0000000007)=-36.22988857045814; dx=-9.505793727858595E-7 evalInputDelta=1.012743652105172
Fitness changed from -35.217144918352965 to -36.22988857045814
Iteration 12 complete. Error: -36.22988857045814 Total: 3.0765; Orientation: 0.2817; Line Search: 1.9549
Adding measurement 18a83905 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.010e+05, gradient 3.993e-03, dot -0.266; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -36.22988857045814, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 18a83905 to history. Total: 3
th(0)=-36.22988857045814;dx=-1.5833761560614385E-5
Armijo: th(3016208.566044639)=-24.800496080038176; dx=2.4823916254904293E-6 evalInputDelta=-11.42939249041996
Armijo: th(1508104.2830223194)=-34.43610735008373; dx=3.395262994707296E-6 evalInputDelta=-1.793781220374406
New Minimum: -36.22988857045814 > -39.489165135562
END: th(502701.4276741065)=-39.489165135562; dx=-3.280713267752215E-6 evalInputDelta=3.2592765651038604
Fitness changed from -36.22988857045814 to -39.489165135562
Iteration 13 complete. Error: -39.489165135562 Total: 4.5970; Orientation: 0.2783; Line Search: 3.4806
Adding measurement 4964d305 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.930e+07, gradient 3.194e-03, dot -0.330; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -39.489165135562, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 4964d305 to history. Total: 3
th(0)=-39.489165135562;dx=-1.0151067839337607E-5
New Minimum: -39.489165135562 > -41.056942783157716
END: th(1083037.3945096491)=-41.056942783157716; dx=-9.147524835902797E-7 evalInputDelta=1.5677776475957188
Fitness changed from -39.489165135562 to -41.056942783157716
Iteration 14 complete. Error: -41.056942783157716 Total: 3.1868; Orientation: 0.4315; Line Search: 1.8756
Adding measurement 1f08627f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.791e+05, gradient 3.804e-03, dot -0.272; [a66c7c5a-893a-49ee-ae9f-fcb9f34974f9 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -41.056942783157716, -5.4663161052573725, -3.4517554875106478, -2.372228979910108
LBFGS Accumulation History: 3 points
Removed measurement 1f08627f to history. Total: 3
th(0)=-41.056942783157716;dx=-1.4337454024782445E-5
Armijo: th(2333333.333333335)=-33.20978853220619; dx=2.611476290566967E-6 evalInputDelta=-7.847154250951526
Armijo: th(1166666.6666666674)=-40.441880601194406; dx=2.4050117097643717E-6 evalInputDelta=-0.6150621819633102
New Minimum: -41.056942783157716 > -43.29493096981852
END: th(388888.88888888917)=-43.29493096981852; dx=-3.498743643690436E-6 evalInputDelta=2.2379881866608073
Fitness changed from -41.056942783157716 to -43.29493096981852
Iteration 15 complete. Error: -43.29493096981852 Total: 4.7253; Orientation: 0.2818; Line Search: 3.6307
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>![Iteration 15](etc/41495311-5fcb-4252-a918-10d8bf500982.jpg)
Final threshold in iteration 16: -43.29493096981852 (> -Infinity) after 63.618s (< 5400.000s)

Returns

    -43.29493096981852