BasicOptimizer.scala:89 executed in 75.07 seconds (0.913 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 165007052455600
Reset training subject: 165007934762100
Adding measurement 62ba0971 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-12.821419342074126;dx=-0.005100214191810523
New Minimum: -12.821419342074126 > -12.83221794004225
WOLFE (weak): th(2.154434690031884)=-12.83221794004225; dx=-0.005109714599224404 evalInputDelta=0.010798597968124213
New Minimum: -12.83221794004225 > -12.843020527490067
WOLFE (weak): th(4.308869380063768)=-12.843020527490067; dx=-0.00509708042802505 evalInputDelta=0.021601185415940805
New Minimum: -12.843020527490067 > -12.88626642696626
WOLFE (weak): th(12.926608140191302)=-12.88626642696626; dx=-0.0050928873746823155 evalInputDelta=0.06484708489213453
New Minimum: -12.88626642696626 > -13.081431127060648
WOLFE (weak): th(51.70643256076521)=-13.081431127060648; dx=-0.005117855371666852 evalInputDelta=0.26001178498652244
New Minimum: -13.081431127060648 > -14.14503449032281
WOLFE (weak): th(258.53216280382605)=-14.14503449032281; dx=-0.005306029391260897 evalInputDelta=1.3236151482486846
New Minimum: -14.14503449032281 > -21.221398347016024
WOLFE (weak): th(1551.1929768229563)=-21.221398347016024; dx=-0.005549814711111959 evalInputDelta=8.399979004941898
New Minimum: -21.221398347016024 > -51.41081619157952
END: th(10858.350837760694)=-51.41081619157952; dx=-0.0018754457751145024 evalInputDelta=38.58939684950539
Fitness changed from -12.821419342074126 to -51.41081619157952
Iteration 1 complete. Error: -51.41081619157952 Total: 9.4167; Orientation: 0.0145; Line Search: 6.9108
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 55cdb7ec to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-51.41081619157952;dx=-0.010470067756121127
New Minimum: -51.41081619157952 > -103.01468531537142
WOLF (strong): th(23393.607721408407)=-103.01468531537142; dx=1.476829991065097E-4 evalInputDelta=51.6038691237919
New Minimum: -103.01468531537142 > -107.10711271316205
END: th(11696.803860704204)=-107.10711271316205; dx=-0.001597790859172282 evalInputDelta=55.69629652158253
Fitness changed from -51.41081619157952 to -107.10711271316205
Iteration 2 complete. Error: -107.10711271316205 Total: 3.4956; Orientation: 0.0144; Line Search: 2.5478
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 5b9f49d9 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-107.10711271316205;dx=-0.012916966057187106
New Minimum: -107.10711271316205 > -148.95690704445363
WOLF (strong): th(25200.000000000004)=-148.95690704445363; dx=4.5606075295650216E-4 evalInputDelta=41.84979433129158
New Minimum: -148.95690704445363 > -173.7164030017747
END: th(12600.000000000002)=-173.7164030017747; dx=-7.542153510449115E-4 evalInputDelta=66.60929028861264
Fitness changed from -107.10711271316205 to -173.7164030017747
Iteration 3 complete. Error: -173.7164030017747 Total: 3.4387; Orientation: 0.0148; Line Search: 2.5015
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 580262b1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.198e+03, gradient 1.115e-01, dot -0.247; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -173.7164030017747, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 580262b1 to history. Total: 3
th(0)=-173.7164030017747;dx=-0.012428008777043361
New Minimum: -173.7164030017747 > -233.91700250836834
WOLF (strong): th(27145.87709440174)=-233.91700250836834; dx=7.540350953116452E-4 evalInputDelta=60.20059950659365
New Minimum: -233.91700250836834 > -262.75521819271256
END: th(13572.93854720087)=-262.75521819271256; dx=-0.0015148445212727938 evalInputDelta=89.03881519093787
Fitness changed from -173.7164030017747 to -262.75521819271256
Iteration 4 complete. Error: -262.75521819271256 Total: 3.8666; Orientation: 0.2697; Line Search: 2.7192
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 67bca868 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.880e+03, gradient 1.081e-01, dot -0.238; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -262.75521819271256, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 67bca868 to history. Total: 3
th(0)=-262.75521819271256;dx=-0.011576802345439615
New Minimum: -262.75521819271256 > -309.38333322110293
WOLF (strong): th(29242.009651760516)=-309.38333322110293; dx=9.189959387553525E-4 evalInputDelta=46.62811502839037
New Minimum: -309.38333322110293 > -351.56652775470536
END: th(14621.004825880258)=-351.56652775470536; dx=-0.0015865401178677481 evalInputDelta=88.8113095619928
Fitness changed from -262.75521819271256 to -351.56652775470536
Iteration 5 complete. Error: -351.56652775470536 Total: 3.6624; Orientation: 0.2707; Line Search: 2.4588
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 3eb6fbd to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.281e+04, gradient 1.155e-01, dot -0.251; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -351.56652775470536, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 3eb6fbd to history. Total: 3
th(0)=-351.56652775470536;dx=-0.013276170038897753
New Minimum: -351.56652775470536 > -367.48928555809624
WOLF (strong): th(31500.00000000001)=-367.48928555809624; dx=0.0013190203796660928 evalInputDelta=15.92275780339088
New Minimum: -367.48928555809624 > -436.9954769502769
END: th(15750.000000000005)=-436.9954769502769; dx=-3.580598238725961E-4 evalInputDelta=85.42894919557153
Fitness changed from -351.56652775470536 to -436.9954769502769
Iteration 6 complete. Error: -436.9954769502769 Total: 3.6529; Orientation: 0.2814; Line Search: 2.4872
Adding measurement 343082b5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.729e+04, gradient 1.190e-01, dot -0.255; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -436.9954769502769, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 343082b5 to history. Total: 3
th(0)=-436.9954769502769;dx=-0.014131236499211792
Armijo: th(33932.34636800218)=-419.9990513382143; dx=0.001979575923661523 evalInputDelta=-16.99642561206258
New Minimum: -436.9954769502769 > -523.1661124824504
WOLF (strong): th(16966.17318400109)=-523.1661124824504; dx=6.675735847157328E-4 evalInputDelta=86.17063553217355
END: th(5655.391061333697)=-514.2103776665733; dx=-0.007513613172843729 evalInputDelta=77.21490071629643
Fitness changed from -436.9954769502769 to -523.1661124824504
Iteration 7 complete. Error: -523.1661124824504 Total: 4.6518; Orientation: 0.2571; Line Search: 3.4230
Adding measurement 5bf07c17 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.652e+04, gradient 1.224e-01, dot -0.271; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -523.1661124824504, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 5bf07c17 to history. Total: 3
th(0)=-523.1661124824504;dx=-0.014962690687630321
New Minimum: -523.1661124824504 > -628.7487326030964
END: th(12184.17068823355)=-628.7487326030964; dx=-0.001544267421712282 evalInputDelta=105.58262012064597
Fitness changed from -523.1661124824504 to -628.7487326030964
Iteration 8 complete. Error: -628.7487326030964 Total: 2.6995; Orientation: 0.2727; Line Search: 1.6422
Adding measurement 2617b41a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.655e+05, gradient 1.028e-01, dot -0.332; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -628.7487326030964, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 2617b41a to history. Total: 3
th(0)=-628.7487326030964;dx=-0.010530271593698057
New Minimum: -628.7487326030964 > -646.0966452474429
WOLF (strong): th(26250.00000000001)=-646.0966452474429; dx=0.0018005983372557433 evalInputDelta=17.34791264434648
New Minimum: -646.0966452474429 > -705.0121231227163
END: th(13125.000000000005)=-705.0121231227163; dx=-0.0017110457096557955 evalInputDelta=76.26339051961986
Fitness changed from -628.7487326030964 to -705.0121231227163
Iteration 9 complete. Error: -705.0121231227163 Total: 3.7881; Orientation: 0.2703; Line Search: 2.7193
Adding measurement 6c71e4e5 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.856e+06, gradient 1.105e-01, dot -0.345; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -705.0121231227163, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 6c71e4e5 to history. Total: 3
th(0)=-705.0121231227163;dx=-0.012164825076547443
Armijo: th(28276.955306668486)=-703.1942743651999; dx=5.68977228343609E-4 evalInputDelta=-1.8178487575163444
New Minimum: -705.0121231227163 > -753.4396127096053
END: th(14138.477653334243)=-753.4396127096053; dx=-4.7994953569373587E-4 evalInputDelta=48.42748958688901
Fitness changed from -705.0121231227163 to -753.4396127096053
Iteration 10 complete. Error: -753.4396127096053 Total: 3.5557; Orientation: 0.2628; Line Search: 2.5269
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 75fdee8d to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.667e+05, gradient 1.101e-01, dot -0.313; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -753.4396127096053, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 75fdee8d to history. Total: 3
th(0)=-753.4396127096053;dx=-0.012053846353850012
Armijo: th(30460.426720583877)=-724.9674130607527; dx=0.00247179185183019 evalInputDelta=-28.472199648852552
New Minimum: -753.4396127096053 > -828.9725945409775
WOLF (strong): th(15230.213360291938)=-828.9725945409775; dx=3.1515019348056484E-4 evalInputDelta=75.53298183137224
END: th(5076.73778676398)=-812.9157013491621; dx=-0.007626153336443819 evalInputDelta=59.47608863955679
Fitness changed from -753.4396127096053 to -828.9725945409775
Iteration 11 complete. Error: -828.9725945409775 Total: 4.5661; Orientation: 0.2677; Line Search: 3.3966
Adding measurement 6dd5aeb4 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.734e+04, gradient 1.198e-01, dot -0.237; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -828.9725945409775, -107.10711271316205, -51.41081619157952, -12.821419342074126
LBFGS Accumulation History: 3 points
Removed measurement 6dd5aeb4 to history. Total: 3
th(0)=-828.9725945409775;dx=-0.014337812443320268
New Minimum: -828.9725945409775 > -901.7847581324493
END: th(10937.500000000007)=-901.7847581324493; dx=-7.41139723437759E-4 evalInputDelta=72.81216359147174
Fitness changed from -828.9725945409775 to -901.7847581324493
Iteration 12 complete. Error: -901.7847581324493 Total: 2.9476; Orientation: 0.2618; Line Search: 1.7537
Adding measurement 1b6e6596 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.865e+04, gradient 1.013e-01, dot 0.522; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-901.7847581324493;dx=-988.7321274179744
Armijo: th(23564.129422223745)=-598.7371867104965; dx=0.01538640258737578 evalInputDelta=-303.04757142195274
Armijo: th(11782.064711111872)=-598.850487392304; dx=0.030736716081876704 evalInputDelta=-302.9342707401453
Armijo: th(3927.3549037039575)=-599.0421781894828; dx=0.0912806136354688 evalInputDelta=-302.7425799429665
Armijo: th(981.8387259259894)=-600.0865302187677; dx=0.3696945680417614 evalInputDelta=-301.69822791368154
Armijo: th(196.36774518519786)=-605.9574551897188; dx=1.8107260663649125 evalInputDelta=-295.8273029427305
Armijo: th(32.72795753086631)=-634.3818595688247; dx=10.276111494732572 evalInputDelta=-267.4028985636246
Armijo: th(4.6754225044094735)=-819.9083409137877; dx=44.458085193868115 evalInputDelta=-81.87641721866157
New Minimum: -901.7847581324493 > -1080.6634536246704
END: th(0.5844278130511842)=-1080.6634536246704; dx=-139.4607270913276 evalInputDelta=178.87869549222114
Fitness changed from -901.7847581324493 to -1080.6634536246704
Iteration 13 complete. Error: -1080.6634536246704 Total: 8.8110; Orientation: 0.2638; Line Search: 7.7703
Adding measurement 3b45b38 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 9.965e+03, gradient 1.260e-01, dot 0.172; [fd8dcae0-b960-4484-a7c5-62d4bd565c29 = 1.000/1.000e+00]
th(0)=-1080.6634536246704;dx=-215.76647589396822
Armijo: th(1.2591115542569398)=-1003.4318300213523; dx=16.13826660605726 evalInputDelta=-77.2316236033181
Armijo: th(0.6295557771284699)=-1047.9348739306274; dx=-4.659994147880017 evalInputDelta=-32.72857969404299
Armijo: th(0.20985192570948996)=-1072.2571986235391; dx=-30.085699055087304 evalInputDelta=-8.406255001131285
Armijo: th(0.05246298142737249)=-1078.9326322806257; dx=-42.911645895590965 evalInputDelta=-1.7308213440446707
Armijo: th(0.010492596285474498)=-1080.3296950468407; dx=-48.45436600602777 evalInputDelta=-0.3337585778297125
Armijo: th(0.001748766047579083)=-1080.6077572366396; dx=-49.179954979560904 evalInputDelta=-0.05569638803081034
Armijo: th(2.498237210827261E-4)=-1080.6554723870138; dx=-49.161220764142946 evalInputDelta=-0.007981237656622397
Armijo: th(3.1227965135340765E-5)=-1080.6624125970145; dx=-48.763044892348745 evalInputDelta=-0.0010410276559014164
Armijo: th(3.4697739039267516E-6)=-1080.6632801201306; dx=-48.593085713469875 evalInputDelta=-1.7350453981634928E-4
Armijo: th(3.4697739039267517E-7)=-1080.6634536246704; dx=-49.02498054036862 evalInputDelta=0.0
Armijo: th(3.154339912660683E-8)=-1080.6634536246704; dx=-48.51711364667562 evalInputDelta=0.0
Armijo: th(2.6286165938839028E-9)=-1080.6634536246704; dx=-48.92129801340717 evalInputDelta=0.0
END: th(2.022012764526079E-10)=-1080.6634536246704; dx=-49.40615843017968 evalInputDelta=0.0
Fitness changed from -1080.6634536246704 to -1080.6634536246704
Static Iteration Total: 13.1976; Orientation: 0.3324; Line Search: 12.0849
Iteration 14 failed. Error: -1080.6634536246704
Previous Error: 0.0 -> -1080.6634536246704
Retrying iteration 14
Reset training subject: 165078803056000
Adding measurement 7dd41648 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-1080.6634536246704;dx=-0.015787869329580255
END: th(4.3562944435822555E-10)=-1080.6634536246704; dx=-0.01182088053156851 evalInputDelta=0.0
Fitness changed from -1080.6634536246704 to -1080.6634536246704
Static Iteration Total: 3.3181; Orientation: 0.0167; Line Search: 1.6947
Iteration 15 failed. Error: -1080.6634536246704
Previous Error: 0.0 -> -1080.6634536246704
Optimization terminated 15
Final threshold in iteration 15: -1080.6634536246704 (> -Infinity) after 75.069s (< 5400.000s)
Returns
-1080.6634536246704