BasicOptimizer.scala:89 executed in 193.22 seconds (1.677 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 6768373299869
Reset training subject: 6769778636185
Adding measurement 6e0898fd to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.1610003790401224;dx=-1.4114505622544776E-4
New Minimum: -3.1610003790401224 > -3.1610163082949394
WOLFE (weak): th(2.154434690031884)=-3.1610163082949394; dx=-1.4134510548542988E-4 evalInputDelta=1.592925481697094E-5
New Minimum: -3.1610163082949394 > -3.1610321260769343
WOLFE (weak): th(4.308869380063768)=-3.1610321260769343; dx=-1.4234683978292433E-4 evalInputDelta=3.1747036811857754E-5
New Minimum: -3.1610321260769343 > -3.161099183682113
WOLFE (weak): th(12.926608140191302)=-3.161099183682113; dx=-1.42454348502308E-4 evalInputDelta=9.880464199074623E-5
New Minimum: -3.161099183682113 > -3.161397918053946
WOLFE (weak): th(51.70643256076521)=-3.161397918053946; dx=-1.43109036717086E-4 evalInputDelta=3.9753901382377776E-4
New Minimum: -3.161397918053946 > -3.162894283838895
WOLFE (weak): th(258.53216280382605)=-3.162894283838895; dx=-1.4165816359678842E-4 evalInputDelta=0.0018939047987727164
New Minimum: -3.162894283838895 > -3.172275255092595
WOLFE (weak): th(1551.1929768229563)=-3.172275255092595; dx=-1.3861551260493438E-4 evalInputDelta=0.011274876052472571
New Minimum: -3.172275255092595 > -3.253253713592788
END: th(10858.350837760694)=-3.253253713592788; dx=-1.1441556294226056E-4 evalInputDelta=0.09225333455266549
Fitness changed from -3.1610003790401224 to -3.253253713592788
Iteration 1 complete. Error: -3.253253713592788 Total: 41.0534; Orientation: 0.0178; Line Search: 35.8859
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 527c955a to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-3.253253713592788;dx=-1.3716292014161077E-4
New Minimum: -3.253253713592788 > -3.3593986952039554
END: th(23393.607721408407)=-3.3593986952039554; dx=-9.269347150913881E-5 evalInputDelta=0.10614498161116748
Fitness changed from -3.253253713592788 to -3.3593986952039554
Iteration 2 complete. Error: -3.3593986952039554 Total: 3.9016; Orientation: 0.0183; Line Search: 2.4755
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement ff0f9b9 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-3.3593986952039554;dx=-1.3933088707082279E-4
New Minimum: -3.3593986952039554 > -3.8089852532520116
END: th(50400.00000000001)=-3.8089852532520116; dx=-5.6990992225837085E-5 evalInputDelta=0.44958655804805625
Fitness changed from -3.3593986952039554 to -3.8089852532520116
Iteration 3 complete. Error: -3.8089852532520116 Total: 3.8171; Orientation: 0.0185; Line Search: 2.3458
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 1d4e7435 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.044e+03, gradient 1.186e-02, dot -0.364; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -3.8089852532520116, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 1d4e7435 to history. Total: 3
th(0)=-3.8089852532520116;dx=-1.3790174107512033E-4
New Minimum: -3.8089852532520116 > -4.66320056128876
END: th(108583.50837760697)=-4.66320056128876; dx=-2.6056951449447788E-5 evalInputDelta=0.8542153080367485
Fitness changed from -3.8089852532520116 to -4.66320056128876
Iteration 4 complete. Error: -4.66320056128876 Total: 4.3644; Orientation: 0.4988; Line Search: 2.4669
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 227e2a01 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.111e+03, gradient 1.091e-02, dot -0.197; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -4.66320056128876, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 227e2a01 to history. Total: 3
th(0)=-4.66320056128876;dx=-1.1726555834603358E-4
New Minimum: -4.66320056128876 > -5.425052190723932
END: th(233936.07721408413)=-5.425052190723932; dx=-1.1359480571939737E-5 evalInputDelta=0.7618516294351716
Fitness changed from -4.66320056128876 to -5.425052190723932
Iteration 5 complete. Error: -5.425052190723932 Total: 4.2437; Orientation: 0.4804; Line Search: 2.3466
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 53709344 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.188e+03, gradient 9.076e-03, dot -0.137; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.425052190723932, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 53709344 to history. Total: 3
th(0)=-5.425052190723932;dx=-8.174900190495444E-5
New Minimum: -5.425052190723932 > -5.560653281885076
END: th(504000.0000000002)=-5.560653281885076; dx=-4.473787504134002E-6 evalInputDelta=0.13560109116114383
Fitness changed from -5.425052190723932 to -5.560653281885076
Iteration 6 complete. Error: -5.560653281885076 Total: 4.3590; Orientation: 0.4912; Line Search: 2.3744
Adding measurement 8f9fade to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.674e+03, gradient 6.233e-03, dot -0.103; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.560653281885076, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 8f9fade to history. Total: 3
th(0)=-5.560653281885076;dx=-3.800902631308091E-5
New Minimum: -5.560653281885076 > -5.799387921605976
END: th(1085835.0837760698)=-5.799387921605976; dx=-1.7933448030369576E-6 evalInputDelta=0.23873463972090025
Fitness changed from -5.560653281885076 to -5.799387921605976
Iteration 7 complete. Error: -5.799387921605976 Total: 4.1384; Orientation: 0.5716; Line Search: 2.4497
Adding measurement 59e6208e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.757e+03, gradient 4.512e-03, dot -0.081; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -5.799387921605976, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 59e6208e to history. Total: 3
th(0)=-5.799387921605976;dx=-1.9780818908236138E-5
New Minimum: -5.799387921605976 > -6.2768738609592285
END: th(2339360.7721408415)=-6.2768738609592285; dx=-5.711286949761124E-7 evalInputDelta=0.4774859393532527
Fitness changed from -5.799387921605976 to -6.2768738609592285
Iteration 8 complete. Error: -6.2768738609592285 Total: 3.9623; Orientation: 0.4855; Line Search: 2.3624
Adding measurement 259ddb06 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.867e+03, gradient 3.219e-03, dot -0.061; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.2768738609592285, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 259ddb06 to history. Total: 3
th(0)=-6.2768738609592285;dx=-9.870354342836867E-6
Armijo: th(5040000.000000002)=-5.530894059233576; dx=-3.3022497529358193E-7 evalInputDelta=-0.7459798017256523
New Minimum: -6.2768738609592285 > -6.602058000750155
END: th(2520000.000000001)=-6.602058000750155; dx=-1.4137114088611235E-6 evalInputDelta=0.32518413979092653
Fitness changed from -6.2768738609592285 to -6.602058000750155
Iteration 9 complete. Error: -6.602058000750155 Total: 5.2786; Orientation: 0.4879; Line Search: 3.6425
Adding measurement 3d98b1a5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.516e+03, gradient 2.967e-03, dot -0.111; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.602058000750155, -3.3593986952039554, -3
...skipping 9740 bytes...
0](etc/24c023f6-914a-45da-8d65-feed2fc9ddc7.jpg)
Adding measurement 6ccbbeb9 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.648e+05, gradient 1.640e-03, dot -0.213; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.59363778000697, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 6ccbbeb9 to history. Total: 3
th(0)=-10.59363778000697;dx=-2.6525413399031008E-6
Armijo: th(1.6406250000000013E7)=-9.546153210254593; dx=-1.9601631268830876E-7 evalInputDelta=-1.0474845697523776
New Minimum: -10.59363778000697 > -10.668698026775928
END: th(8203125.0000000065)=-10.668698026775928; dx=-5.728868829080194E-7 evalInputDelta=0.07506024676895784
Fitness changed from -10.59363778000697 to -10.668698026775928
Iteration 21 complete. Error: -10.668698026775928 Total: 5.4418; Orientation: 0.5015; Line Search: 3.5268
Adding measurement 70b2fb01 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.292e+05, gradient 1.552e-03, dot -0.217; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.668698026775928, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 70b2fb01 to history. Total: 3
th(0)=-10.668698026775928;dx=-2.3536066766587005E-6
Armijo: th(1.767309706666781E7)=-10.513670725604216; dx=-1.9239897404317026E-7 evalInputDelta=-0.15502730117171204
New Minimum: -10.668698026775928 > -11.240043052518793
END: th(8836548.533333905)=-11.240043052518793; dx=-5.606792658414098E-7 evalInputDelta=0.5713450257428647
Fitness changed from -10.668698026775928 to -11.240043052518793
Iteration 22 complete. Error: -11.240043052518793 Total: 5.2529; Orientation: 0.4856; Line Search: 3.5682
Adding measurement 34d7915f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.994e+05, gradient 1.551e-03, dot -0.223; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.240043052518793, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 34d7915f to history. Total: 3
th(0)=-11.240043052518793;dx=-2.309554956020892E-6
Armijo: th(1.903776670036493E7)=-9.885940503202391; dx=-1.7838330351326417E-7 evalInputDelta=-1.3541025493164014
Armijo: th(9518883.350182464)=-11.17567473612996; dx=-5.118851910355446E-7 evalInputDelta=-0.06436831638883334
New Minimum: -11.240043052518793 > -11.52432107752105
END: th(3172961.116727488)=-11.52432107752105; dx=-1.1628467597347367E-6 evalInputDelta=0.28427802500225674
Fitness changed from -11.240043052518793 to -11.52432107752105
Iteration 23 complete. Error: -11.52432107752105 Total: 6.4064; Orientation: 0.4705; Line Search: 4.8262
Adding measurement f025544 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.395e+07, gradient 1.557e-03, dot -0.249; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.52432107752105, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement f025544 to history. Total: 3
th(0)=-11.52432107752105;dx=-2.375705451452409E-6
Armijo: th(6835937.500000006)=-11.499484862245048; dx=-7.218260842271259E-7 evalInputDelta=-0.024836215276001994
New Minimum: -11.52432107752105 > -11.673701179866931
END: th(3417968.750000003)=-11.673701179866931; dx=-1.1363517761173406E-6 evalInputDelta=0.14938010234588184
Fitness changed from -11.52432107752105 to -11.673701179866931
Iteration 24 complete. Error: -11.673701179866931 Total: 5.1684; Orientation: 0.4720; Line Search: 3.5569
Adding measurement 1cfa1a40 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.268e+05, gradient 1.551e-03, dot -0.246; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -11.673701179866931, -3.3593986952039554, -3.253253713592788, -3.1610003790401224
LBFGS Accumulation History: 3 points
Removed measurement 1cfa1a40 to history. Total: 3
th(0)=-11.673701179866931;dx=-2.3576798667815767E-6
Armijo: th(7363790.444444922)=-11.490049509056636; dx=-6.640301902456865E-7 evalInputDelta=-0.18365167081029554
Armijo: th(3681895.222222461)=-11.626743356697197; dx=-1.0678595734667151E-6 evalInputDelta=-0.04695782316973407
Armijo: th(1227298.407407487)=-11.663461492340764; dx=-1.4477341795074936E-6 evalInputDelta=-0.010239687526167884
New Minimum: -11.673701179866931 > -11.67790926475517
END: th(306824.60185187176)=-11.67790926475517; dx=-1.6827963397596149E-6 evalInputDelta=0.004208084888238872
Fitness changed from -11.673701179866931 to -11.67790926475517
Iteration 25 complete. Error: -11.67790926475517 Total: 7.6598; Orientation: 0.4827; Line Search: 5.9074
<a id="p-19"></a>Iteration 25
<a id="p-18"></a>
Adding measurement 14d9c8a7 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 7.305e+04, gradient 1.552e-03, dot 0.253; [5e003c01-c4bb-4e72-824c-c38c43970d71 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-11.67790926475517;dx=-28.755879330177187
Armijo: th(661033.5659848935)=-9.461702066496079; dx=1.9971815126297283E-6 evalInputDelta=-2.2162071982590916
Armijo: th(330516.78299244674)=-9.46077373475616; dx=3.8843892259144566E-6 evalInputDelta=-2.2171355299990108
Armijo: th(110172.26099748224)=-9.459282496816307; dx=1.2048642236115767E-5 evalInputDelta=-2.218626767938863
Armijo: th(27543.06524937056)=-9.45925033494206; dx=4.833305840282916E-5 evalInputDelta=-2.2186589298131096
Armijo: th(5508.613049874112)=-9.462666327494853; dx=2.3686248150062904E-4 evalInputDelta=-2.2152429372603173
Armijo: th(918.1021749790187)=-9.464347223100066; dx=0.001415681179881238 evalInputDelta=-2.213562041655104
Armijo: th(131.15745356843124)=-9.481618715161666; dx=0.009838618177639868 evalInputDelta=-2.1962905495935043
Armijo: th(16.394681696053905)=-9.70116733184685; dx=0.06469422133392697 evalInputDelta=-1.97674193290832
Armijo: th(1.821631299561545)=-10.460531926166123; dx=-0.21665980063243317 evalInputDelta=-1.2173773385890474
Armijo: th(0.1821631299561545)=-11.54899217233573; dx=-6.732462265562421 evalInputDelta=-0.1289170924194405
Armijo: th(0.01656028454146859)=-11.660024199837155; dx=-10.877676654979354 evalInputDelta=-0.01788506491801556
Armijo: th(0.0013800237117890491)=-11.676277068791526; dx=-11.02905891820198 evalInputDelta=-0.0016321959636442784
Armijo: th(1.0615567013761916E-4)=-11.677779488677603; dx=-11.486308599820887 evalInputDelta=-1.297760775678114E-4
Armijo: th(7.5825478669727975E-6)=-11.677898168715702; dx=-11.346600205617147 evalInputDelta=-1.1096039468583285E-5
Armijo: th(5.055031911315199E-7)=-11.677907817446274; dx=-11.41412904966764 evalInputDelta=-1.4473088967292824E-6
Armijo: th(3.159394944571999E-8)=-11.67790926475517; dx=-11.383716697699489 evalInputDelta=0.0
Armijo: th(1.8584676144541172E-9)=-11.67790926475517; dx=-11.22530414614469 evalInputDelta=0.0
Armijo: th(1.032482008030065E-10)=-11.677908782318891; dx=-11.371654880482126 evalInputDelta=-4.824362793698356E-7
MIN ALPHA (5.4341158317371844E-12): th(0.0)=-11.67790926475517
Fitness changed from -11.67790926475517 to -11.67790926475517
Static Iteration Total: 26.2120; Orientation: 0.4814; Line Search: 24.2205
Iteration 26 failed. Error: -11.67790926475517
Previous Error: 0.0 -> -11.67790926475517
Retrying iteration 26
Reset training subject: 6956983526944
Adding measurement 26d725d to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-11.67790926475517;dx=-2.3654408826238932E-6
END: th(2.1544346900318838E-10)=-11.67790926475517; dx=-1.8565656459312462E-6 evalInputDelta=0.0
Fitness changed from -11.67790926475517 to -11.67790926475517
Static Iteration Total: 4.6157; Orientation: 0.0171; Line Search: 2.3762
Iteration 27 failed. Error: -11.67790926475517
Previous Error: 0.0 -> -11.67790926475517
Optimization terminated 27
Final threshold in iteration 27: -11.67790926475517 (> -Infinity) after 193.215s (< 3600.000s)
Returns
-11.67790926475517