BasicOptimizer.scala:89 executed in 62.64 seconds (0.659 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 150882117201800
Reset training subject: 150882991485200
Adding measurement 26e320b7 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-144.79822302302262;dx=-0.012531089762960513
New Minimum: -144.79822302302262 > -144.82604058404
WOLFE (weak): th(2.154434690031884)=-144.82604058404; dx=-0.012035429984972892 evalInputDelta=0.027817561017371872
New Minimum: -144.82604058404 > -144.85386628642328
WOLFE (weak): th(4.308869380063768)=-144.85386628642328; dx=-0.01201564114670478 evalInputDelta=0.055643263400668275
New Minimum: -144.85386628642328 > -144.9652840984696
WOLFE (weak): th(12.926608140191302)=-144.9652840984696; dx=-0.0120338938163355 evalInputDelta=0.16706107544698057
New Minimum: -144.9652840984696 > -145.467800077457
WOLFE (weak): th(51.70643256076521)=-145.467800077457; dx=-0.012074731829847082 evalInputDelta=0.6695770544343986
New Minimum: -145.467800077457 > -148.16461100115978
WOLFE (weak): th(258.53216280382605)=-148.16461100115978; dx=-0.012147490663392577 evalInputDelta=3.366387978137169
New Minimum: -148.16461100115978 > -165.49275380662442
WOLFE (weak): th(1551.1929768229563)=-165.49275380662442; dx=-0.012648404019790681 evalInputDelta=20.694530783601806
New Minimum: -165.49275380662442 > -294.17677774171773
END: th(10858.350837760694)=-294.17677774171773; dx=-0.011182685418773606 evalInputDelta=149.3785547186951
Fitness changed from -144.79822302302262 to -294.17677774171773
Iteration 1 complete. Error: -294.17677774171773 Total: 9.3088; Orientation: 0.0144; Line Search: 6.8007
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4d97707d to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-294.17677774171773;dx=-0.0213336395026543
New Minimum: -294.17677774171773 > -676.723655748785
END: th(23393.607721408407)=-676.723655748785; dx=-0.008271621164320671 evalInputDelta=382.5468780070672
Fitness changed from -294.17677774171773 to -676.723655748785
Iteration 2 complete. Error: -676.723655748785 Total: 2.6860; Orientation: 0.0146; Line Search: 1.7295
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 6ce61ce5 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-676.723655748785;dx=-0.02637755984794645
New Minimum: -676.723655748785 > -1201.744253158716
END: th(50400.00000000001)=-1201.744253158716; dx=-0.002521981017950353 evalInputDelta=525.020597409931
Fitness changed from -676.723655748785 to -1201.744253158716
Iteration 3 complete. Error: -1201.744253158716 Total: 2.5976; Orientation: 0.0151; Line Search: 1.6533
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 54d0b316 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.419e+05, gradient 2.052e-01, dot -0.178; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1201.744253158716, -676.723655748785, -294.17677774171773, -144.79822302302262
LBFGS Accumulation History: 3 points
Removed measurement 54d0b316 to history. Total: 3
th(0)=-1201.744253158716;dx=-0.04193699076282535
New Minimum: -1201.744253158716 > -1629.5400424659715
WOLF (strong): th(108583.50837760697)=-1629.5400424659715; dx=0.00214533416096057 evalInputDelta=427.7957893072555
New Minimum: -1629.5400424659715 > -1762.3741799061854
END: th(54291.75418880348)=-1762.3741799061854; dx=-3.8287294070709477E-4 evalInputDelta=560.6299267474694
Fitness changed from -1201.744253158716 to -1762.3741799061854
Iteration 4 complete. Error: -1762.3741799061854 Total: 3.9319; Orientation: 0.2771; Line Search: 2.7374
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 7648cb53 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.355e+05, gradient 2.012e-01, dot -0.090; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1762.3741799061854, -676.723655748785, -294.17677774171773, -144.79822302302262
LBFGS Accumulation History: 3 points
Removed measurement 7648cb53 to history. Total: 3
th(0)=-1762.3741799061854;dx=-0.04034925244478619
New Minimum: -1762.3741799061854 > -1866.965586185241
WOLF (strong): th(116968.03860704206)=-1866.965586185241; dx=0.004641152644978124 evalInputDelta=104.59140627905549
New Minimum: -1866.965586185241 > -2296.552851558178
WOLF (strong): th(58484.01930352103)=-2296.552851558178; dx=0.0024511501898881187 evalInputDelta=534.1786716519928
END: th(19494.673101173677)=-2265.0267106592805; dx=-0.010985395331579668 evalInputDelta=502.6525307530951
Fitness changed from -1762.3741799061854 to -2296.552851558178
Iteration 5 complete. Error: -2296.552851558178 Total: 4.6264; Orientation: 0.2794; Line Search: 3.3868
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 44647d28 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.355e+05, gradient 1.985e-01, dot -0.117; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2296.552851558178, -676.723655748785, -294.17677774171773, -144.79822302302262
LBFGS Accumulation History: 3 points
Removed measurement 44647d28 to history. Total: 3
th(0)=-2296.552851558178;dx=-0.03933414346713043
New Minimum: -2296.552851558178 > -2939.3012775147768
END: th(42000.000000000015)=-2939.3012775147768; dx=-0.0013333887113932462 evalInputDelta=642.7484259565986
Fitness changed from -2296.552851558178 to -2939.3012775147768
Iteration 6 complete. Error: -2939.3012775147768 Total: 2.8991; Orientation: 0.2760; Line Search: 1.7136
Adding measurement 74981010 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.787e+05, gradient 1.818e-01, dot -0.087; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -2939.3012775147768, -676.723655748785, -294.17677774171773, -144.79822302302262
LBFGS Accumulation History: 3 points
Removed measurement 74981010 to history. Total: 3
th(0)=-2939.3012775147768;dx=-0.03296532838667964
Armijo: th(90486.25698133915)=-2931.96487018518; dx=0.006501700136256739 evalInputDelta=-7.336407329596568
New Minimum: -2939.3012775147768 > -3368.9608190063595
WOLF (strong): th(45243.12849066957)=-3368.9608190063595; dx=4.558875614191312E-4 evalInputDelta=429.6595414915828
END: th(15081.04283022319)=-3278.435755051485; dx=-0.013604806400890814 evalInputDelta=339.13447753670835
Fitness changed from -2939.3012775147768 to -3368.9608190063595
Iteration 7 complete. Error: -3368.9608190063595 Total: 4.6333; Orientation: 0.2630; Line Search: 3.4337
Adding measurement 15477002 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.690e+06, gradient 1.927e-01, dot -0.127; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3368.9608190063595, -676.723655748785, -294.17677774171773, -144.79822302302262
LBFGS Accumulation History: 3 points
Removed measurement 15477002 to history. Total: 3
th(0)=-3368.9608190063595;dx=-0.03674579669029203
New Minimum: -3368.9608190063595 > -3758.375712990919
END: th(32491.12183528946)=-3758.375712990919; dx=-7.166944276040194E-4 evalInputDelta=389.4148939845595
Fitness changed from -3368.9608190063595 to -3758.375712990919
Iteration 8 complete. Error: -3758.375712990919 Total: 2.8083; Orientation: 0.2923; Line Search: 1.7088
Adding measurement 33da571 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.742e+06, gradient 1.768e-01, dot -0.101; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3758.375712990919, -676.723655748785, -294.17677774171773, -144.79822302302262
LBFGS Accumulation History: 3 points
Removed measurement 33da571 to history. Total: 3
th(0)=-3758.375712990919;dx=-0.031165249941974227
Armijo: th(70000.00000000003)=-3705.5285722822323; dx=0.005792084135281456 evalInputDelta=-52.847140708686766
New Minimum: -3758.375712990919 > -4034.59812125074
WOLF (strong): th(35000.000000000015)=-4034.59812125074; dx=1.0763959434565058E-4 evalInputDelta=276.2224082598209
END: th(11666.666666666672)=-3977.9973604817787; dx=-0.01207077586405984 evalInputDelta=219.6216474908597
Fitness changed from -3758.375712990919 to -4034.59812125074
Iteration 9 complete. Error: -4034.59812125074 Total: 4.7519; Orientation: 0.4119; Line Search: 3.5229
Adding measurement 6998de46 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.341e+06, gradient 1.826e-01, dot -0.134; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4034.59812125074, -676.723655748785, -294.17677774171773, -144.79822302302262
LBFGS Accumulation History: 3 points
Removed measurement 6998de46 to history. Total: 3
th(0)=-4034.59812125074;dx=-0.033157197126100234
New Minimum: -4034.59812125074 > -4301.971641003692
END: th(25135.071383705323)=-4301.971641003692; dx=-0.0023291498534272074 evalInputDelta=267.3735197529518
Fitness changed from -4034.59812125074 to -4301.971641003692
Iteration 10 complete. Error: -4301.971641003692 Total: 2.7528; Orientation: 0.2952; Line Search: 1.6468
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7e117d11 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 4.816e+06, gradient 1.632e-01, dot 0.123; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-4301.971641003692;dx=-96582.37502814727
Armijo: th(54151.86972548245)=-339.3211052970328; dx=0.005965645913229456 evalInputDelta=-3962.650535706659
Armijo: th(27075.934862741226)=-339.32111680898964; dx=0.012046091887414696 evalInputDelta=-3962.650524194702
Armijo: th(9025.311620913742)=-339.2014980067158; dx=0.03575903809452019 evalInputDelta=-3962.770142996976
Armijo: th(2256.3279052284356)=-339.071399652787; dx=0.14239047838033903 evalInputDelta=-3962.9002413509047
Armijo: th(451.2655810456871)=-339.00710953090646; dx=0.7092531283679275 evalInputDelta=-3962.9645314727854
Armijo: th(75.21093017428119)=-340.88141724799254; dx=4.28023405051851 evalInputDelta=-3961.0902237556993
Armijo: th(10.744418596325884)=-351.7351298384908; dx=30.70483085178054 evalInputDelta=-3950.2365111652007
Armijo: th(1.3430523245407355)=-374.6014548823856; dx=266.1885632140388 evalInputDelta=-3927.3701861213062
Armijo: th(0.14922803606008173)=-489.1615434853807; dx=3098.276503338673 evalInputDelta=-3812.810097518311
Armijo: th(0.014922803606008173)=-1640.8235736970626; dx=66733.12553826366 evalInputDelta=-2661.148067306629
Armijo: th(0.0013566185096371066)=-4122.699600989624; dx=169905.5360948081 evalInputDelta=-179.2720400140679
New Minimum: -4301.971641003692 > -4302.868345650376
WOLF (strong): th(1.1305154246975888E-4)=-4302.868345650376; dx=9803.178457766395 evalInputDelta=0.8967046466841566
END: th(8.69627249767376E-6)=-4302.178285473535; dx=-12158.695227909935 evalInputDelta=0.2066444698430132
Fitness changed from -4301.971641003692 to -4302.868345650376
Iteration 11 complete. Error: -4302.868345650376 Total: 13.3989; Orientation: 0.2726; Line Search: 12.2124
Adding measurement 4e7408b6 to history. Total: 4
Rejected: LBFGS Orientation magnitude: 2.225e+07, gradient 1.635e-01, dot -0.093; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4302.868345650376, -4301.971641003692, -676.723655748785, -294.17677774171773, -144.79822302302262
Rejected: LBFGS Orientation magnitude: 1.788e+07, gradient 1.635e-01, dot -0.371; [bba78cd0-2fe6-4be6-a14b-76a6f82aea0b = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4302.868345650376, -4301.971641003692, -676.723655748785, -294.17677774171773
LBFGS Accumulation History: 3 points
Removed measurement 4e7408b6 to history. Total: 4
Removed measurement 7e117d11 to history. Total: 3
th(0)=-4302.868345650376;dx=-0.026667434225255495
Armijo: th(1.873555114295856E-5)=-4302.868345650376; dx=-0.017879682951779503 evalInputDelta=0.0
END: th(9.36777557147928E-6)=-4302.868345650376; dx=-0.017848897543828304 evalInputDelta=0.0
Fitness changed from -4302.868345650376 to -4302.868345650376
Static Iteration Total: 4.1099; Orientation: 0.6021; Line Search: 2.6930
Iteration 12 failed. Error: -4302.868345650376
Previous Error: 0.0 -> -4302.868345650376
Retrying iteration 12
Reset training subject: 150940622262200
Adding measurement 7bbe6eca to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-4302.868345650376;dx=-0.02663254219058681
Armijo: th(2.0182260659628218E-5)=-4302.868345650376; dx=-0.01778285122219169 evalInputDelta=0.0
END: th(1.0091130329814109E-5)=-4302.868345650376; dx=-0.0178794640162327 evalInputDelta=0.0
Fitness changed from -4302.868345650376 to -4302.868345650376
Static Iteration Total: 4.1314; Orientation: 0.0143; Line Search: 2.5506
Iteration 13 failed. Error: -4302.868345650376
Previous Error: 0.0 -> -4302.868345650376
Optimization terminated 13
Final threshold in iteration 13: -4302.868345650376 (> -Infinity) after 62.636s (< 5400.000s)
Returns
-4302.868345650376