BasicOptimizer.scala:89 executed in 149.11 seconds (1.648 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 5135836641735
Reset training subject: 5137015184405
Adding measurement 2f6aa16 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-3.679379822719747;dx=-0.0013041341375188361
New Minimum: -3.679379822719747 > -3.6795125562934143
WOLFE (weak): th(2.154434690031884)=-3.6795125562934143; dx=-0.0013600148774326674 evalInputDelta=1.3273357366738736E-4
New Minimum: -3.6795125562934143 > -3.679648971921435
WOLFE (weak): th(4.308869380063768)=-3.679648971921435; dx=-0.0013720495026971822 evalInputDelta=2.6914920168819023E-4
New Minimum: -3.679648971921435 > -3.6801767052986833
WOLFE (weak): th(12.926608140191302)=-3.6801767052986833; dx=-0.0013642416593262429 evalInputDelta=7.968825789363976E-4
New Minimum: -3.6801767052986833 > -3.6824241163459854
WOLFE (weak): th(51.70643256076521)=-3.6824241163459854; dx=-0.0013579548054305107 evalInputDelta=0.003044293626238481
New Minimum: -3.6824241163459854 > -3.694852876924881
WOLFE (weak): th(258.53216280382605)=-3.694852876924881; dx=-0.0013213221619833822 evalInputDelta=0.015473054205134229
New Minimum: -3.694852876924881 > -3.7554582885907073
WOLFE (weak): th(1551.1929768229563)=-3.7554582885907073; dx=-0.0011932711783464917 evalInputDelta=0.07607846587096034
New Minimum: -3.7554582885907073 > -4.466215311198057
END: th(10858.350837760694)=-4.466215311198057; dx=-5.817258371973538E-4 evalInputDelta=0.7868354884783098
Fitness changed from -3.679379822719747 to -4.466215311198057
Iteration 1 complete. Error: -4.466215311198057 Total: 12.7452; Orientation: 0.0190; Line Search: 9.3608
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 2a25963b to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-4.466215311198057;dx=-0.0012525061766347182
New Minimum: -4.466215311198057 > -5.268104996123939
END: th(23393.607721408407)=-5.268104996123939; dx=-2.86917783008952E-4 evalInputDelta=0.8018896849258823
Fitness changed from -4.466215311198057 to -5.268104996123939
Iteration 2 complete. Error: -5.268104996123939 Total: 4.0711; Orientation: 0.0191; Line Search: 2.3324
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 39093a4a to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-5.268104996123939;dx=-0.0011934349033736807
New Minimum: -5.268104996123939 > -10.311675914848733
END: th(50400.00000000001)=-10.311675914848733; dx=-1.774953830087999E-4 evalInputDelta=5.043570918724794
Fitness changed from -5.268104996123939 to -10.311675914848733
Iteration 3 complete. Error: -10.311675914848733 Total: 3.6370; Orientation: 0.0190; Line Search: 2.2626
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 4f488f4c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.333e+03, gradient 4.457e-02, dot -0.228; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.311675914848733, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 4f488f4c to history. Total: 3
th(0)=-10.311675914848733;dx=-0.0019639052540962754
New Minimum: -10.311675914848733 > -16.083557080285804
END: th(108583.50837760697)=-16.083557080285804; dx=-5.204330351952705E-5 evalInputDelta=5.771881165437071
Fitness changed from -10.311675914848733 to -16.083557080285804
Iteration 4 complete. Error: -16.083557080285804 Total: 4.2661; Orientation: 0.4946; Line Search: 2.3037
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 27353d1c to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.372e+03, gradient 2.823e-02, dot -0.180; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.083557080285804, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 27353d1c to history. Total: 3
th(0)=-16.083557080285804;dx=-7.891317604672428E-4
Armijo: th(233936.07721408413)=-14.112276190616566; dx=-2.2356073580928616E-5 evalInputDelta=-1.971280889669238
Armijo: th(116968.03860704206)=-16.023956490410463; dx=-1.0402757032376453E-4 evalInputDelta=-0.059600589875341115
New Minimum: -16.083557080285804 > -16.91511882299738
END: th(38989.346202347355)=-16.91511882299738; dx=-3.8462182257880013E-4 evalInputDelta=0.8315617427115747
Fitness changed from -16.083557080285804 to -16.91511882299738
Iteration 5 complete. Error: -16.91511882299738 Total: 6.5474; Orientation: 0.4742; Line Search: 4.7013
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 6a2158a2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.648e+03, gradient 2.955e-02, dot -0.178; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.91511882299738, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 6a2158a2 to history. Total: 3
th(0)=-16.91511882299738;dx=-8.478130296283353E-4
New Minimum: -16.91511882299738 > -19.20791070245279
END: th(84000.00000000003)=-19.20791070245279; dx=-1.8866188992237338E-4 evalInputDelta=2.292791879455411
Fitness changed from -16.91511882299738 to -19.20791070245279
Iteration 6 complete. Error: -19.20791070245279 Total: 4.1175; Orientation: 0.4831; Line Search: 2.2704
Adding measurement 54eabbdf to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.102e+03, gradient 2.823e-02, dot -0.169; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -19.20791070245279, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 54eabbdf to history. Total: 3
th(0)=-19.20791070245279;dx=-7.795089249018935E-4
Armijo: th(180972.5139626783)=-17.937143787521784; dx=-6.349067508060095E-5 evalInputDelta=-1.2707669149310057
New Minimum: -19.20791070245279 > -20.464141553378738
END: th(90486.25698133915)=-20.464141553378738; dx=-1.8790526351316944E-4 evalInputDelta=1.2562308509259488
Fitness changed from -19.20791070245279 to -20.464141553378738
Iteration 7 complete. Error: -20.464141553378738 Total: 5.0983; Orientation: 0.4804; Line Search: 3.5172
Adding measurement 1dbd1049 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.974e+03, gradient 2.891e-02, dot -0.177; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.464141553378738, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 1dbd1049 to history. Total: 3
th(0)=-20.464141553378738;dx=-7.9931392347841E-4
New Minimum: -20.464141553378738 > -20.788578456833836
END: th(194946.7310117368)=-20.788578456833836; dx=-6.655634171807254E-5 evalInputDelta=0.3244369034550978
Fitness changed from -20.464141553378738 to -20.788578456833836
Iteration 8 complete. Error: -20.788578456833836 Total: 3.7979; Orientation: 0.4709; Line Search: 2.2720
Adding measurement 584b3bdc to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.363e+03, gradient 1.998e-02, dot -0.149; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -20.788578456833836, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 584b3bdc to history. Total: 3
th(0)=-20.788578456833836;dx=-3.839345601015604E-4
New Minimum: -20.788578456833836 > -21.098498585089793
END: th(420000.0000000002)=-21.098498585089793; dx=-3.122685753649662E-5 evalInputDelta=0.30992012825595694
Fitness changed from -20.788578456833836 to -21.098498585089793
Iteration 9 complete. Error: -21.098498585089793 Total: 3.9892; Orientation: 0.4967; Line Search: 2.3155
Adding measurement 6269b9cf to history. Total: 3
Rejected:
...skipping 7846 bytes...
2776864122673
Iteration 18 complete. Error: -42.92776864122673 Total: 5.5865; Orientation: 0.5034; Line Search: 3.7985
Adding measurement 34388183 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.205e+05, gradient 7.639e-03, dot -0.171; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -42.92776864122673, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 34388183 to history. Total: 3
th(0)=-42.92776864122673;dx=-5.7339556570878985E-5
Armijo: th(4712825.884444749)=-34.003039251069815; dx=-2.697759950901375E-6 evalInputDelta=-8.924729390156912
Armijo: th(2356412.9422223745)=-41.764318373401494; dx=-8.896351931628002E-6 evalInputDelta=-1.1634502678252332
New Minimum: -42.92776864122673 > -45.725027362006635
END: th(785470.9807407915)=-45.725027362006635; dx=-2.6894831866716963E-5 evalInputDelta=2.797258720779908
Fitness changed from -42.92776864122673 to -45.725027362006635
Iteration 19 complete. Error: -45.725027362006635 Total: 6.7089; Orientation: 0.5057; Line Search: 5.0520
Adding measurement 33b89217 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.519e+05, gradient 7.784e-03, dot -0.216; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.725027362006635, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 33b89217 to history. Total: 3
th(0)=-45.725027362006635;dx=-5.9689110588605646E-5
Armijo: th(1692245.928921327)=-43.17101693470004; dx=-1.4929143924741818E-5 evalInputDelta=-2.5540104273065936
New Minimum: -45.725027362006635 > -45.87778205835923
END: th(846122.9644606635)=-45.87778205835923; dx=-2.5879034764805225E-5 evalInputDelta=0.15275469635259498
Fitness changed from -45.725027362006635 to -45.87778205835923
Iteration 20 complete. Error: -45.87778205835923 Total: 5.3134; Orientation: 0.4939; Line Search: 3.6035
<a id="p-17"></a>Iteration 20
<a id="p-16"></a>
Adding measurement 788bebee to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.618e+07, gradient 7.703e-03, dot -0.237; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -45.87778205835923, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 788bebee to history. Total: 3
th(0)=-45.87778205835923;dx=-5.8368346954138983E-5
Armijo: th(1822916.6666666681)=-43.13313000885584; dx=-1.292529117200652E-5 evalInputDelta=-2.7446520495033937
Armijo: th(911458.3333333341)=-45.72640969144563; dx=-2.5084979233483775E-5 evalInputDelta=-0.15137236691359846
New Minimum: -45.87778205835923 > -46.24790995500748
END: th(303819.44444444467)=-46.24790995500748; dx=-3.672111802095321E-5 evalInputDelta=0.3701278966482491
Fitness changed from -45.87778205835923 to -46.24790995500748
Iteration 21 complete. Error: -46.24790995500748 Total: 6.7007; Orientation: 0.4808; Line Search: 4.7783
Adding measurement 4d697229 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.726e+04, gradient 7.762e-03, dot -0.236; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -46.24790995500748, -5.268104996123939, -4.466215311198057, -3.679379822719747
LBFGS Accumulation History: 3 points
Removed measurement 4d697229 to history. Total: 3
th(0)=-46.24790995500748;dx=-5.952677653077812E-5
Armijo: th(654559.1506173263)=-45.98175232598529; dx=-2.942453547946611E-5 evalInputDelta=-0.26615762902218876
New Minimum: -46.24790995500748 > -46.25461729205037
END: th(327279.57530866313)=-46.25461729205037; dx=-3.672961044079093E-5 evalInputDelta=0.00670733704289006
Fitness changed from -46.24790995500748 to -46.25461729205037
Iteration 22 complete. Error: -46.25461729205037 Total: 5.1650; Orientation: 0.4829; Line Search: 3.6066
Adding measurement 46f87131 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.090e+05, gradient 7.792e-03, dot 0.260; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-46.25461729205037;dx=-221.99245790051003
Armijo: th(705102.4703838862)=-31.986683477194223; dx=8.647506325813481E-6 evalInputDelta=-14.267933814856146
Armijo: th(352551.2351919431)=-31.98662511785583; dx=1.753001487489796E-5 evalInputDelta=-14.26799217419454
Armijo: th(117517.07839731437)=-31.98695491457812; dx=5.205912367394706E-5 evalInputDelta=-14.267662377472249
Armijo: th(29379.269599328592)=-31.99623668705653; dx=2.0932085146216017E-4 evalInputDelta=-14.258380604993839
Armijo: th(5875.853919865718)=-32.00043566871225; dx=0.0010294204839295476 evalInputDelta=-14.25418162333812
Armijo: th(979.3089866442864)=-31.986030660854436; dx=0.006047294134921148 evalInputDelta=-14.268586631195934
Armijo: th(139.90128380632663)=-32.07129297063401; dx=0.04276182618021139 evalInputDelta=-14.183324321416357
Armijo: th(17.48766047579083)=-32.638476563458944; dx=0.28208825827365325 evalInputDelta=-13.616140728591425
Armijo: th(1.943073386198981)=-37.14155058269804; dx=0.2645368955727409 evalInputDelta=-9.113066709352331
Armijo: th(0.1943073386198981)=-43.98856677112581; dx=-43.502141636513464 evalInputDelta=-2.2660505209245585
Armijo: th(0.01766430351089983)=-46.059021804938546; dx=-92.01378371379938 evalInputDelta=-0.19559548711182373
Armijo: th(0.0014720252925749858)=-46.23363790513137; dx=-101.84936167425425 evalInputDelta=-0.020979386919002252
Armijo: th(1.1323271481346045E-4)=-46.25313248822214; dx=-103.01444894303677 evalInputDelta=-0.0014848038282266884
Armijo: th(8.088051058104318E-6)=-46.25451217476302; dx=-103.99985754403006 evalInputDelta=-1.0511728734741155E-4
Armijo: th(5.392034038736212E-7)=-46.25460790658799; dx=-104.22595746389229 evalInputDelta=-9.385462377053955E-6
Armijo: th(3.3700212742101327E-8)=-46.25461353786565; dx=-104.46039343258309 evalInputDelta=-3.7541847177635645E-6
New Minimum: -46.25461729205037 > -46.254619169142615
END: th(1.982365455417725E-9)=-46.254619169142615; dx=-104.41160538550821 evalInputDelta=1.8770922451949446E-6
Fitness changed from -46.25461729205037 to -46.254619169142615
Iteration 23 complete. Error: -46.254619169142615 Total: 22.6299; Orientation: 0.4865; Line Search: 20.9563
Adding measurement 4a69f13f to history. Total: 4
Rejected: LBFGS Orientation magnitude: 5.452e+06, gradient 7.791e-03, dot -0.219; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -46.254619169142615, -46.25461729205037, -5.268104996123939, -4.466215311198057, -3.679379822719747
Rejected: LBFGS Orientation magnitude: 1.162e+07, gradient 7.791e-03, dot -0.221; [e2b7978e-6738-45dd-9325-6a5ac3685d34 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -46.254619169142615, -46.25461729205037, -5.268104996123939, -4.466215311198057
LBFGS Accumulation History: 3 points
Removed measurement 4a69f13f to history. Total: 4
Removed measurement 46f87131 to history. Total: 3
th(0)=-46.254619169142615;dx=-5.9526886943771444E-5
END: th(4.270876905472801E-9)=-46.254619169142615; dx=-4.514384405102661E-5 evalInputDelta=0.0
Fitness changed from -46.254619169142615 to -46.254619169142615
Static Iteration Total: 4.6914; Orientation: 1.1609; Line Search: 2.3133
Iteration 24 failed. Error: -46.254619169142615
Previous Error: 0.0 -> -46.254619169142615
Retrying iteration 24
Reset training subject: 5280225047842
Adding measurement f9bba41 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-46.254619169142615;dx=-5.984682806360805E-5
END: th(9.201325362006625E-9)=-46.254619169142615; dx=-4.5434736407538545E-5 evalInputDelta=0.0
Fitness changed from -46.254619169142615 to -46.254619169142615
Static Iteration Total: 4.7179; Orientation: 0.0178; Line Search: 2.4698
Iteration 25 failed. Error: -46.254619169142615
Previous Error: 0.0 -> -46.254619169142615
Optimization terminated 25
Final threshold in iteration 25: -46.254619169142615 (> -Infinity) after 149.106s (< 3600.000s)
Returns
-46.254619169142615