BasicOptimizer.scala:89 executed in 71.82 seconds (0.864 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 142540294196200
Reset training subject: 142541170029100
Adding measurement 2fe464b0 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-8.897662804267064;dx=-3.728850025765396E-5
New Minimum: -8.897662804267064 > -8.897776966070367
WOLFE (weak): th(2.154434690031884)=-8.897776966070367; dx=-3.662587256031951E-5 evalInputDelta=1.1416180330314774E-4
New Minimum: -8.897776966070367 > -8.897890150685765
WOLFE (weak): th(4.308869380063768)=-8.897890150685765; dx=-3.629844079552723E-5 evalInputDelta=2.2734641870059136E-4
New Minimum: -8.897890150685765 > -8.89834450087308
WOLFE (weak): th(12.926608140191302)=-8.89834450087308; dx=-3.6608540543458395E-5 evalInputDelta=6.816966060156915E-4
New Minimum: -8.89834450087308 > -8.900387976817898
WOLFE (weak): th(51.70643256076521)=-8.900387976817898; dx=-3.662892905801092E-5 evalInputDelta=0.002725172550833932
New Minimum: -8.900387976817898 > -8.911280008380029
WOLFE (weak): th(258.53216280382605)=-8.911280008380029; dx=-3.655885349633672E-5 evalInputDelta=0.013617204112964743
New Minimum: -8.911280008380029 > -8.979338919937849
WOLFE (weak): th(1551.1929768229563)=-8.979338919937849; dx=-3.643822428577476E-5 evalInputDelta=0.08167611567078481
New Minimum: -8.979338919937849 > -9.470466727211045
WOLFE (weak): th(10858.350837760694)=-9.470466727211045; dx=-3.632812950900451E-5 evalInputDelta=0.5728039229439812
New Minimum: -9.470466727211045 > -13.174281158272711
END: th(86866.80670208555)=-13.174281158272711; dx=-2.8521075198996634E-5 evalInputDelta=4.276618354005647
Fitness changed from -8.897662804267064 to -13.174281158272711
Iteration 1 complete. Error: -13.174281158272711 Total: 10.5508; Orientation: 0.0145; Line Search: 8.0421
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 40ce0b04 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-13.174281158272711;dx=-3.094244977806211E-5
New Minimum: -13.174281158272711 > -19.48115523449441
END: th(187148.86177126726)=-19.48115523449441; dx=-1.647412159992433E-5 evalInputDelta=6.306874076221698
Fitness changed from -13.174281158272711 to -19.48115523449441
Iteration 2 complete. Error: -19.48115523449441 Total: 2.6805; Orientation: 0.0150; Line Search: 1.7233
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 31e571a9 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-19.48115523449441;dx=-1.9290232364100048E-5
New Minimum: -19.48115523449441 > -27.57163188437827
END: th(403200.00000000006)=-27.57163188437827; dx=-9.971408043493769E-6 evalInputDelta=8.090476649883861
Fitness changed from -19.48115523449441 to -27.57163188437827
Iteration 3 complete. Error: -27.57163188437827 Total: 2.7277; Orientation: 0.0148; Line Search: 1.7758
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 3e183e80 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.813e+03, gradient 4.123e-03, dot -0.554; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.57163188437827, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 3e183e80 to history. Total: 3
th(0)=-27.57163188437827;dx=-1.6780807110784377E-5
New Minimum: -27.57163188437827 > -37.69333573400223
END: th(868668.0670208557)=-37.69333573400223; dx=-5.302579253859213E-6 evalInputDelta=10.121703849623962
Fitness changed from -27.57163188437827 to -37.69333573400223
Iteration 4 complete. Error: -37.69333573400223 Total: 2.8786; Orientation: 0.2950; Line Search: 1.6532
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 310ff3b8 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.284e+04, gradient 5.081e-03, dot -0.381; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -37.69333573400223, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 310ff3b8 to history. Total: 3
th(0)=-37.69333573400223;dx=-2.5637169540500974E-5
New Minimum: -37.69333573400223 > -46.5509949137828
WOLF (strong): th(1871488.617712673)=-46.5509949137828; dx=2.9640446529178033E-8 evalInputDelta=8.857659179780569
New Minimum: -46.5509949137828 > -47.43877991065633
END: th(935744.3088563365)=-47.43877991065633; dx=-2.4961554398515482E-6 evalInputDelta=9.745444176654097
Fitness changed from -37.69333573400223 to -47.43877991065633
Iteration 5 complete. Error: -47.43877991065633 Total: 3.8673; Orientation: 0.2790; Line Search: 2.5406
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7b105be9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.137e+04, gradient 4.937e-03, dot -0.287; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -47.43877991065633, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 7b105be9 to history. Total: 3
th(0)=-47.43877991065633;dx=-2.4081465696138274E-5
New Minimum: -47.43877991065633 > -53.192600192024415
WOLF (strong): th(2016000.0000000007)=-53.192600192024415; dx=1.1609493741544382E-6 evalInputDelta=5.753820281368085
New Minimum: -53.192600192024415 > -57.17873794365
END: th(1008000.0000000003)=-57.17873794365; dx=-1.8904602329758689E-6 evalInputDelta=9.73995803299367
Fitness changed from -47.43877991065633 to -57.17873794365
Iteration 6 complete. Error: -57.17873794365 Total: 3.8484; Orientation: 0.2848; Line Search: 2.6078
Adding measurement 5f5876e1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.420e+04, gradient 5.075e-03, dot -0.311; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -57.17873794365, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 5f5876e1 to history. Total: 3
th(0)=-57.17873794365;dx=-2.5418517351413045E-5
New Minimum: -57.17873794365 > -58.21646722826666
WOLF (strong): th(2171670.1675521396)=-58.21646722826666; dx=2.1587146206801872E-6 evalInputDelta=1.0377292846166597
New Minimum: -58.21646722826666 > -65.01429364788459
END: th(1085835.0837760698)=-65.01429364788459; dx=-1.9733485562697105E-7 evalInputDelta=7.8355557042345865
Fitness changed from -57.17873794365 to -65.01429364788459
Iteration 7 complete. Error: -65.01429364788459 Total: 3.7260; Orientation: 0.2885; Line Search: 2.6333
Adding measurement 61cab883 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.683e+04, gradient 5.182e-03, dot -0.267; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -65.01429364788459, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 61cab883 to history. Total: 3
th(0)=-65.01429364788459;dx=-2.6675105826182095E-5
Armijo: th(2339360.7721408415)=-60.690477792649425; dx=3.894510191130665E-6 evalInputDelta=-4.323815855235161
New Minimum: -65.01429364788459 > -72.36676160366395
WOLF (strong): th(1169680.3860704207)=-72.36676160366395; dx=1.1003870398461044E-6 evalInputDelta=7.35246795577936
New Minimum: -72.36676160366395 > -72.97001201001325
END: th(389893.4620234736)=-72.97001201001325; dx=-9.778659162842878E-6 evalInputDelta=7.955718362128664
Fitness changed from -65.01429364788459 to -72.97001201001325
Iteration 8 complete. Error: -72.97001201001325 Total: 4.6021; Orientation: 0.2772; Line Search: 3.5193
Adding measurement 9ef74dc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.337e+04, gradient 4.044e-03, dot -0.397; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -72.97001201001325, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 9ef74dc to history. Total: 3
th(0)=-72.97001201001325;dx=-1.6246636365924592E-5
New Minimum: -72.97001201001325 > -79.4969635862757
END: th(840000.0000000003)=-79.4969635862757; dx=-6.510375586489854E-6 evalInputDelta=6.526951576262448
Fitness changed from -72.97001201001325 to -79.4969635862757
Iteration 9 complete. Error: -79.4969635862757 Total: 2.8073; Orientation: 0.2845; Line Search: 1.7214
Adding measurement 1407c6f5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.678e+05, gradient 4.188e-03, dot -0.425; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -79.4969635862757, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 1407c6f5 to history. Total: 3
th(0)=-79.4969635862757;dx=-1.747234190547736E-5
New Minimum: -79.4969635862757 > -84.60737742200146
END: th(1809725.139626783)=-84.60737742200146; dx=-1.9666680618520303E-6 evalInputDelta=5.110413835725765
Fitness changed from -79.4969635862757 to -84.60737742200146
Iteration 10 complete. Error: -84.60737742200146 Total: 2.9375; Orientation: 0.2856; Line Search: 1.8149
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 39175850 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.330e+05, gradient 5.468e-03, dot -0.276; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -84.60737742200146, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement 39175850 to history. Total: 3
th(0)=-84.60737742200146;dx=-2.9742049150004983E-5
Armijo: th(3898934.6202347362)=-60.74737579390846; dx=3.996734783300961E-6 evalInputDelta=-23.860001628093002
Armijo: th(1949467.3101173681)=-81.40843072753053; dx=3.8030494981818517E-6 evalInputDelta=-3.1989466944709335
New Minimum: -84.60737742200146 > -92.80891353463024
END: th(649822.4367057894)=-92.80891353463024; dx=-3.1234160925738334E-6 evalInputDelta=8.201536112628773
Fitness changed from -84.60737742200146 to -92.80891353463024
Iteration 11 complete. Error: -92.80891353463024 Total: 4.8066; Orientation: 0.2989; Line Search: 3.5522
Adding measurement b06398c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.140e+07, gradient 4.583e-03, dot -0.347; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -92.80891353463024, -19.48115523449441, -13.174281158272711, -8.897662804267064
LBFGS Accumulation History: 3 points
Removed measurement b06398c to history. Total: 3
th(0)=-92.80891353463024;dx=-2.0746495975196264E-5
New Minimum: -92.80891353463024 > -94.78900889039447
WOLF (strong): th(1400000.000000001)=-94.78900889039447; dx=3.53885006680373E-7 evalInputDelta=1.9800953557642345
New Minimum: -94.78900889039447 > -97.54985464136769
END: th(700000.0000000005)=-97.54985464136769; dx=-3.8020947412278873E-6 evalInputDelta=4.740941106737452
Fitness changed from -92.80891353463024 to -97.54985464136769
Iteration 12 complete. Error: -97.54985464136769 Total: 3.6590; Orientation: 0.2732; Line Search: 2.5934
Adding measurement 2d628cce to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.879e+04, gradient 4.491e-03, dot 0.419; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-97.54985464136769;dx=-148.47827953693053
Armijo: th(1508104.2830223197)=-28.120414096490965; dx=9.435551752727868E-6 evalInputDelta=-69.42944054487673
Armijo: th(754052.1415111598)=-28.120759882429507; dx=1.8634644033127858E-5 evalInputDelta=-69.42909475893818
Armijo: th(251350.71383705328)=-28.12128019992424; dx=5.602945597389632E-5 evalInputDelta=-69.42857444144344
Armijo: th(62837.67845926332)=-28.13290908910405; dx=2.2370778177428343E-4 evalInputDelta=-69.41694555226364
Armijo: th(12567.535691852663)=-28.13093067357805; dx=0.0011187721282635696 evalInputDelta=-69.41892396778964
Armijo: th(2094.589281975444)=-28.13968929150295; dx=0.006710389303976461 evalInputDelta=-69.41016534986474
Armijo: th(299.2270402822063)=-28.27173873687945; dx=0.0467186476847852 evalInputDelta=-69.27811590448823
Armijo: th(37.40338003527579)=-29.522086796100954; dx=0.3869033602348657 evalInputDelta=-68.02776784526674
Armijo: th(4.1559311150306435)=-35.639155781829835; dx=3.8392115961014657 evalInputDelta=-61.91069885953785
Armijo: th(0.4155931115030643)=-79.95869715480826; dx=33.455157530406424 evalInputDelta=-17.591157486559425
New Minimum: -97.54985464136769 > -100.50911744511497
END: th(0.03778119195482403)=-100.50911744511497; dx=-62.19825613025496 evalInputDelta=2.9592628037472792
Fitness changed from -97.54985464136769 to -100.50911744511497
Iteration 13 complete. Error: -100.50911744511497 Total: 11.5819; Orientation: 0.2698; Line Search: 10.4960
Adding measurement 735b0dcc to history. Total: 4
Rejected: LBFGS Orientation magnitude: 1.548e+04, gradient 4.540e-03, dot -0.029; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -100.50911744511497, -97.54985464136769, -19.48115523449441, -13.174281158272711, -8.897662804267064
Rejected: LBFGS Orientation magnitude: 1.205e+05, gradient 4.540e-03, dot -0.270; [59154e4d-af39-4437-b4cc-67ae93726916 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -100.50911744511497, -97.54985464136769, -19.48115523449441, -13.174281158272711
LBFGS Accumulation History: 3 points
Removed measurement 735b0dcc to history. Total: 4
Removed measurement 2d628cce to history. Total: 3
th(0)=-100.50911744511497;dx=-2.0549974941959957E-5
Armijo: th(0.08139711057822642)=-100.50911744511497; dx=-1.3441232064959824E-5 evalInputDelta=0.0
Armijo: th(0.04069855528911321)=-100.50911744511497; dx=-1.337419885266258E-5 evalInputDelta=0.0
Armijo: th(0.01356618509637107)=-100.50911744511497; dx=-1.3445376775235744E-5 evalInputDelta=0.0
Armijo: th(0.0033915462740927676)=-100.50911744511497; dx=-1.3442629063048496E-5 evalInputDelta=0.0
Armijo: th(6.783092548185535E-4)=-100.50911744511497; dx=-1.3409069924592E-5 evalInputDelta=0.0
END: th(1.1305154246975892E-4)=-100.50911744511497; dx=-1.3363154781205069E-5 evalInputDelta=0.0
Fitness changed from -100.50911744511497 to -100.50911744511497
Static Iteration Total: 7.6447; Orientation: 0.6145; Line Search: 6.0136
Iteration 14 failed. Error: -100.50911744511497
Previous Error: 0.0 -> -100.50911744511497
Retrying iteration 14
Reset training subject: 142608612878200
Adding measurement 53e68f89 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-100.50911744511497;dx=-2.0527016527698225E-5
END: th(2.435621648584614E-4)=-100.50911744511497; dx=-1.3421622777026024E-5 evalInputDelta=0.0
Fitness changed from -100.50911744511497 to -100.50911744511497
Static Iteration Total: 3.5033; Orientation: 0.0146; Line Search: 1.8299
Iteration 15 failed. Error: -100.50911744511497
Previous Error: 0.0 -> -100.50911744511497
Optimization terminated 15
Final threshold in iteration 15: -100.50911744511497 (> -Infinity) after 71.822s (< 5400.000s)
Returns
-100.50911744511497