BasicOptimizer.scala:89 executed in 53.74 seconds (1.325 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 213125157575800
Reset training subject: 213125814146600
Adding measurement 202ed88d to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-21.783153849695815;dx=-0.002622091624648785
New Minimum: -21.783153849695815 > -21.787458962118592
WOLFE (weak): th(2.154434690031884)=-21.787458962118592; dx=-0.0026191626563842737 evalInputDelta=0.004305112422777313
New Minimum: -21.787458962118592 > -21.79175977600016
WOLFE (weak): th(4.308869380063768)=-21.79175977600016; dx=-0.0026216167069858807 evalInputDelta=0.008605926304344536
New Minimum: -21.79175977600016 > -21.80898928203549
WOLFE (weak): th(12.926608140191302)=-21.80898928203549; dx=-0.0026211757707126837 evalInputDelta=0.02583543233967589
New Minimum: -21.80898928203549 > -21.886843845815648
WOLFE (weak): th(51.70643256076521)=-21.886843845815648; dx=-0.0026265836266579703 evalInputDelta=0.10368999611983298
New Minimum: -21.886843845815648 > -22.30768774274503
WOLFE (weak): th(258.53216280382605)=-22.30768774274503; dx=-0.002668898673548476 evalInputDelta=0.5245338930492167
New Minimum: -22.30768774274503 > -25.104401609931923
WOLFE (weak): th(1551.1929768229563)=-25.104401609931923; dx=-0.0027415155940197756 evalInputDelta=3.3212477602361083
New Minimum: -25.104401609931923 > -39.04899089062924
END: th(10858.350837760694)=-39.04899089062924; dx=-0.0013952322008721038 evalInputDelta=17.265837040933427
Fitness changed from -21.783153849695815 to -39.04899089062924
Iteration 1 complete. Error: -39.04899089062924 Total: 7.5612; Orientation: 0.0198; Line Search: 5.6480
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 7fe1a856 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-39.04899089062924;dx=-0.004038283989195832
New Minimum: -39.04899089062924 > -57.17844468553779
END: th(23393.607721408407)=-57.17844468553779; dx=-6.881274375359166E-4 evalInputDelta=18.12945379490855
Fitness changed from -39.04899089062924 to -57.17844468553779
Iteration 2 complete. Error: -57.17844468553779 Total: 1.9568; Orientation: 0.0186; Line Search: 1.2938
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5b51a4c7 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-57.17844468553779;dx=-0.0037988982334810515
New Minimum: -57.17844468553779 > -61.54093760325475
END: th(50400.00000000001)=-61.54093760325475; dx=-2.0415175353043796E-4 evalInputDelta=4.362492917716956
Fitness changed from -57.17844468553779 to -61.54093760325475
Iteration 3 complete. Error: -61.54093760325475 Total: 1.9583; Orientation: 0.0173; Line Search: 1.3093
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 5409305f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.335e+03, gradient 6.193e-02, dot -0.152; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -61.54093760325475, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 5409305f to history. Total: 3
th(0)=-61.54093760325475;dx=-0.003828870823007982
Armijo: th(108583.50837760697)=-12.642285733524787; dx=7.405930209912686E-5 evalInputDelta=-48.89865186972996
New Minimum: -61.54093760325475 > -76.0992380103755
END: th(54291.75418880348)=-76.0992380103755; dx=-3.430820832223659E-4 evalInputDelta=14.558300407120754
Fitness changed from -61.54093760325475 to -76.0992380103755
Iteration 4 complete. Error: -76.0992380103755 Total: 2.9164; Orientation: 0.3177; Line Search: 1.9830
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 2dd25088 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.240e+03, gradient 6.261e-02, dot -0.151; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -76.0992380103755, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 2dd25088 to history. Total: 3
th(0)=-76.0992380103755;dx=-0.003909197879226818
Armijo: th(116968.03860704206)=-6.085464409005283; dx=1.256587312231534E-4 evalInputDelta=-70.01377360137022
New Minimum: -76.0992380103755 > -87.69581340924464
END: th(58484.01930352103)=-87.69581340924464; dx=-3.83163576103254E-4 evalInputDelta=11.596575398869135
Fitness changed from -76.0992380103755 to -87.69581340924464
Iteration 5 complete. Error: -87.69581340924464 Total: 3.4337; Orientation: 0.7120; Line Search: 2.0493
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 329cdafa to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.259e+03, gradient 6.325e-02, dot -0.149; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -87.69581340924464, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 329cdafa to history. Total: 3
th(0)=-87.69581340924464;dx=-0.003989822167896784
Armijo: th(126000.00000000004)=-13.971289937537136; dx=9.214308754607942E-5 evalInputDelta=-73.7245234717075
New Minimum: -87.69581340924464 > -105.52231993975667
END: th(63000.00000000002)=-105.52231993975667; dx=-5.258350106439287E-4 evalInputDelta=17.826506530512034
Fitness changed from -87.69581340924464 to -105.52231993975667
Iteration 6 complete. Error: -105.52231993975667 Total: 3.0794; Orientation: 0.3128; Line Search: 2.1060
Adding measurement 4b9fd79 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.164e+03, gradient 6.337e-02, dot -0.147; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -105.52231993975667, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 4b9fd79 to history. Total: 3
th(0)=-105.52231993975667;dx=-0.004004237648817361
Armijo: th(135729.38547200873)=-11.972451198508963; dx=4.9418424615372316E-5 evalInputDelta=-93.5498687412477
New Minimum: -105.52231993975667 > -117.34308962774372
END: th(67864.69273600436)=-117.34308962774372; dx=-6.330347010408043E-4 evalInputDelta=11.820769687987053
Fitness changed from -105.52231993975667 to -117.34308962774372
Iteration 7 complete. Error: -117.34308962774372 Total: 3.1320; Orientation: 0.3633; Line Search: 2.1041
Adding measurement 7ff167c4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 9.365e+03, gradient 6.228e-02, dot -0.142; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -117.34308962774372, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 7ff167c4 to history. Total: 3
th(0)=-117.34308962774372;dx=-0.0038722590232418376
Armijo: th(146210.0482588026)=-6.922146470816415; dx=5.985221287074952E-5 evalInputDelta=-110.42094315692731
New Minimum: -117.34308962774372 > -125.8861179298101
END: th(73105.0241294013)=-125.8861179298101; dx=-7.081884427056046E-4 evalInputDelta=8.543028302066375
Fitness changed from -117.34308962774372 to -125.8861179298101
Iteration 8 complete. Error: -125.8861179298101 Total: 3.1608; Orientation: 0.3678; Line Search: 2.1191
Adding measurement 1e4cf0e5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.336e+04, gradient 6.268e-02, dot -0.139; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -125.8861179298101, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 1e4cf0e5 to history. Total: 3
th(0)=-125.8861179298101;dx=-0.003912924455192653
Armijo: th(157500.00000000006)=1.4987388733613045; dx=1.1529692180254471E-4 evalInputDelta=-127.3848568031714
New Minimum: -125.8861179298101 > -132.66684676193483
END: th(78750.00000000003)=-132.66684676193483; dx=-6.427982881432924E-4 evalInputDelta=6.780728832124737
Fitness changed from -125.8861179298101 to -132.66684676193483
Iteration 9 complete. Error: -132.66684676193483 Total: 3.2257; Orientation: 0.3264; Line Search: 2.2784
Adding measurement 47828afb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.947e+04, gradient 6.356e-02, dot -0.133; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -132.66684676193483, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 47828afb to history. Total: 3
th(0)=-132.66684676193483;dx=-0.004030773960760143
Armijo: th(169661.7318400109)=18.248219331223055; dx=1.7409569650941504E-4 evalInputDelta=-150.9150660931579
New Minimum: -132.66684676193483 > -136.7864647167356
END: th(84830.86592000545)=-136.7864647167356; dx=-5.82746651358907E-4 evalInputDelta=4.119617954800759
Fitness changed from -132.66684676193483 to -136.7864647167356
Iteration 10 complete. Error: -136.7864647167356 Total: 2.9105; Orientation: 0.3191; Line Search: 1.9686
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 722b4f64 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.579e+04, gradient 6.493e-02, dot -0.134; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -136.7864647167356, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 722b4f64 to history. Total: 3
th(0)=-136.7864647167356;dx=-0.004213921285995091
Armijo: th(182762.56032350325)=41.83984087552591; dx=2.113213700985171E-4 evalInputDelta=-178.6263055922615
Armijo: th(91381.28016175162)=-131.73848007112133; dx=-5.508867723763892E-4 evalInputDelta=-5.047984645614264
New Minimum: -136.7864647167356 > -179.72091105053386
END: th(30460.426720583873)=-179.72091105053386; dx=-0.0023390394528523886 evalInputDelta=42.93444633379826
Fitness changed from -136.7864647167356 to -179.72091105053386
Iteration 11 complete. Error: -179.72091105053386 Total: 3.6816; Orientation: 0.3359; Line Search: 2.6518
Adding measurement 97f2dbb to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.558e+05, gradient 5.862e-02, dot -0.175; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -179.72091105053386, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 97f2dbb to history. Total: 3
th(0)=-179.72091105053386;dx=-0.0034328221785386087
New Minimum: -179.72091105053386 > -197.8413759771061
END: th(65625.00000000003)=-197.8413759771061; dx=-0.0012348817129582102 evalInputDelta=18.120464926572254
Fitness changed from -179.72091105053386 to -197.8413759771061
Iteration 12 complete. Error: -197.8413759771061 Total: 2.2722; Orientation: 0.3053; Line Search: 1.3917
Adding measurement 7d80f543 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.310e+06, gradient 6.142e-02, dot -0.169; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -197.8413759771061, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 7d80f543 to history. Total: 3
th(0)=-197.8413759771061;dx=-0.003762098330203344
Armijo: th(141384.77653334243)=-95.32801163277418; dx=-1.265428727084212E-4 evalInputDelta=-102.51336434433193
New Minimum: -197.8413759771061 > -207.3955501544354
END: th(70692.38826667122)=-207.3955501544354; dx=-0.001019304938047236 evalInputDelta=9.554174177329287
Fitness changed from -197.8413759771061 to -207.3955501544354
Iteration 13 complete. Error: -207.3955501544354 Total: 3.4166; Orientation: 0.5040; Line Search: 2.2797
Adding measurement 31daef26 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.316e+05, gradient 6.342e-02, dot -0.145; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Orientation rejected. Popping history element from -207.3955501544354, -57.17844468553779, -39.04899089062924, -21.783153849695815
LBFGS Accumulation History: 3 points
Removed measurement 31daef26 to history. Total: 3
th(0)=-207.3955501544354;dx=-0.004016476899949403
Armijo: th(152302.13360291938)=-77.09620586873268; dx=7.10379151918366E-5 evalInputDelta=-130.29934428570272
New Minimum: -207.3955501544354 > -211.96830311074024
END: th(76151.06680145969)=-211.96830311074024; dx=-7.250163381818694E-4 evalInputDelta=4.572752956304839
Fitness changed from -207.3955501544354 to -211.96830311074024
Iteration 14 complete. Error: -211.96830311074024 Total: 2.8953; Orientation: 0.3416; Line Search: 1.9378
Adding measurement 5882ff51 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 3.824e+03, gradient 6.459e-02, dot 0.441; [005080d5-2893-41b3-bb6c-f0b223cb6a2c = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-211.96830311074024;dx=-108.67345577945025
Armijo: th(164062.5000000001)=3716.6266724500438; dx=0.01011261868189156 evalInputDelta=-3928.594975560784
Armijo: th(82031.25000000004)=3689.318626576498; dx=0.020028659994641262 evalInputDelta=-3901.286929687238
Armijo: th(27343.750000000015)=3619.7115699425335; dx=0.058658084230048604 evalInputDelta=-3831.6798730532737
Armijo: th(6835.937500000004)=3477.918531475678; dx=0.22233112059183519 evalInputDelta=-3689.886834586418
Armijo: th(1367.1875000000007)=3221.1370567727586; dx=1.0027846554324387 evalInputDelta=-3433.1053598834987
Armijo: th(227.86458333333346)=2722.2040293915525; dx=5.084954408290445 evalInputDelta=-2934.1723325022926
Armijo: th(32.55208333333335)=1371.6279704680642; dx=18.522218717437603 evalInputDelta=-1583.5962735788044
Armijo: th(4.069010416666669)=24.818261254001868; dx=4.652238304869669 evalInputDelta=-236.7865643647421
Armijo: th(0.45211226851851877)=-209.29416880371275; dx=-57.09996270480727 evalInputDelta=-2.6741343070274866
New Minimum: -211.96830311074024 > -212.49589884173758
END: th(0.045211226851851874)=-212.49589884173758; dx=-79.56169931838208 evalInputDelta=0.5275957309973478
Fitness changed from -211.96830311074024 to -212.49589884173758
Iteration 15 complete. Error: -212.49589884173758 Total: 8.1140; Orientation: 0.3193; Line Search: 7.2126
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -212.49589884173758 (> -Infinity) after 53.736s (< 5400.000s)
Returns
-212.49589884173758