BasicOptimizer.scala:89 executed in 9.03 seconds (0.000 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 142725191821400
Reset training subject: 142725321522400
Adding measurement 63d835bc to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.07289430209946773;dx=-7.556705955469437E-8
New Minimum: -0.07289430209946773 > -0.07289452741839321
WOLFE (weak): th(2.154434690031884)=-0.07289452741839321; dx=-6.928473959472249E-8 evalInputDelta=2.2531892547583698E-7
New Minimum: -0.07289452741839321 > -0.07289477345550487
WOLFE (weak): th(4.308869380063768)=-0.07289477345550487; dx=-7.150773773230977E-8 evalInputDelta=4.713560371322201E-7
New Minimum: -0.07289477345550487 > -0.07289565918223094
WOLFE (weak): th(12.926608140191302)=-0.07289565918223094; dx=-7.220896300227961E-8 evalInputDelta=1.357082763206452E-6
New Minimum: -0.07289565918223094 > -0.07289982350750147
WOLFE (weak): th(51.70643256076521)=-0.07289982350750147; dx=-7.142689644103472E-8 evalInputDelta=5.521408033740127E-6
New Minimum: -0.07289982350750147 > -0.0729220136925737
WOLFE (weak): th(258.53216280382605)=-0.0729220136925737; dx=-6.931660529913345E-8 evalInputDelta=2.7711593105966825E-5
New Minimum: -0.0729220136925737 > -0.07306096438186298
WOLFE (weak): th(1551.1929768229563)=-0.07306096438186298; dx=-6.816682430813793E-8 evalInputDelta=1.6666228239524405E-4
New Minimum: -0.07306096438186298 > -0.07407298727196567
WOLFE (weak): th(10858.350837760694)=-0.07407298727196567; dx=-7.220847741003919E-8 evalInputDelta=0.0011786851724979314
New Minimum: -0.07407298727196567 > -0.0820123219835421
END: th(86866.80670208555)=-0.0820123219835421; dx=-6.641230152494571E-8 evalInputDelta=0.00911801988407436
Fitness changed from -0.07289430209946773 to -0.0820123219835421
Iteration 1 complete. Error: -0.0820123219835421 Total: 1.3616; Orientation: 0.0015; Line Search: 1.0139
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 377ffd04 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-0.0820123219835421;dx=-6.651733652277276E-8
New Minimum: -0.0820123219835421 > -0.10002098965843205
END: th(187148.86177126726)=-0.10002098965843205; dx=-5.225469591879949E-8 evalInputDelta=0.018008667674889955
Fitness changed from -0.0820123219835421 to -0.10002098965843205
Iteration 2 complete. Error: -0.10002098965843205 Total: 0.3489; Orientation: 0.0016; Line Search: 0.2272
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3a981dd6 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-0.10002098965843205;dx=-5.548762299737493E-8
New Minimum: -0.10002098965843205 > -0.12643914217634988
END: th(403200.00000000006)=-0.12643914217634988; dx=-3.2921219858645856E-8 evalInputDelta=0.026418152517917834
Fitness changed from -0.10002098965843205 to -0.12643914217634988
Iteration 3 complete. Error: -0.12643914217634988 Total: 0.3509; Orientation: 0.0015; Line Search: 0.2298
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 65544840 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.452e+02, gradient 2.438e-04, dot -0.531; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.12643914217634988, -0.10002098965843205, -0.0820123219835421, -0.07289430209946773
LBFGS Accumulation History: 3 points
Removed measurement 65544840 to history. Total: 3
th(0)=-0.12643914217634988;dx=-5.3437278305514355E-8
New Minimum: -0.12643914217634988 > -0.16238537310055826
END: th(868668.0670208557)=-0.16238537310055826; dx=-1.7603257050471168E-8 evalInputDelta=0.03594623092420837
Fitness changed from -0.12643914217634988 to -0.16238537310055826
Iteration 4 complete. Error: -0.16238537310055826 Total: 0.3721; Orientation: 0.0284; Line Search: 0.2252
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 470af4d1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.282e+02, gradient 2.826e-04, dot -0.357; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.16238537310055826, -0.10002098965843205, -0.0820123219835421, -0.07289430209946773
LBFGS Accumulation History: 3 points
Removed measurement 470af4d1 to history. Total: 3
th(0)=-0.16238537310055826;dx=-7.083117771061169E-8
New Minimum: -0.16238537310055826 > -0.21434334629209237
END: th(1871488.617712673)=-0.21434334629209237; dx=-7.680581792915643E-9 evalInputDelta=0.05195797319153411
Fitness changed from -0.16238537310055826 to -0.21434334629209237
Iteration 5 complete. Error: -0.21434334629209237 Total: 0.3767; Orientation: 0.0286; Line Search: 0.2256
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7b7e4917 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.190e+02, gradient 2.660e-04, dot -0.250; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.21434334629209237, -0.10002098965843205, -0.0820123219835421, -0.07289430209946773
LBFGS Accumulation History: 3 points
Removed measurement 7b7e4917 to history. Total: 3
th(0)=-0.21434334629209237;dx=-6.562500132966064E-8
New Minimum: -0.21434334629209237 > -0.3435508260897466
END: th(4032000.0000000014)=-0.3435508260897466; dx=-6.872147123603702E-9 evalInputDelta=0.1292074797976542
Fitness changed from -0.21434334629209237 to -0.3435508260897466
Iteration 6 complete. Error: -0.3435508260897466 Total: 0.3885; Orientation: 0.0293; Line Search: 0.2334
Adding measurement 339f5a5f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.983e+03, gradient 2.780e-04, dot -0.216; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.3435508260897466, -0.10002098965843205, -0.0820123219835421, -0.07289430209946773
LBFGS Accumulation History: 3 points
Removed measurement 339f5a5f to history. Total: 3
th(0)=-0.3435508260897466;dx=-7.355747719246141E-8
New Minimum: -0.3435508260897466 > -0.5639233242542242
END: th(8686680.670208558)=-0.5639233242542242; dx=-1.7482197080996693E-9 evalInputDelta=0.22037249816447763
Fitness changed from -0.3435508260897466 to -0.5639233242542242
Iteration 7 complete. Error: -0.5639233242542242 Total: 0.3688; Orientation: 0.0281; Line Search: 0.2263
Adding measurement 2ebcb449 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.234e+04, gradient 2.789e-04, dot -0.223; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.5639233242542242, -0.10002098965843205, -0.0820123219835421, -0.07289430209946773
LBFGS Accumulation History: 3 points
Removed measurement 2ebcb449 to history. Total: 3
th(0)=-0.5639233242542242;dx=-7.331329630097328E-8
New Minimum: -0.5639233242542242 > -0.6253150548837076
WOLF (strong): th(1.871488617712673E7)=-0.6253150548837076; dx=4.6935748433941206E-9 evalInputDelta=0.06139173062948344
New Minimum: -0.6253150548837076 > -0.8860858215230936
WOLF (strong): th(9357443.088563366)=-0.8860858215230936; dx=3.296360779310901E-9 evalInputDelta=0.32216249726886936
END: th(3119147.6961877886)=-0.8574607249497209; dx=-3.12432947685061E-8 evalInputDelta=0.2935374006954967
Fitness changed from -0.5639233242542242 to -0.8860858215230936
Iteration 8 complete. Error: -0.8860858215230936 Total: 0.5928; Orientation: 0.0269; Line Search: 0.4568
Adding measurement 71ca31b0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.497e+04, gradient 3.258e-04, dot -0.209; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -0.8860858215230936, -0.10002098965843205, -0.0820123219835421, -0.07289430209946773
LBFGS Accumulation History: 3 points
Removed measurement 71ca31b0 to history. Total: 3
th(0)=-0.8860858215230936;dx=-1.0438530196294187E-7
New Minimum: -0.8860858215230936 > -1.1359303464077974
END: th(6720000.000000003)=-1.1359303464077974; dx=-1.1927338240567426E-9 evalInputDelta=0.24984452488470388
Fitness changed from -0.8860858215230936 to -1.1359303464077974
Iteration 9 complete. Error: -1.1359303464077974 Total: 0.3651; Orientation: 0.0291; Line Search: 0.2264
Adding measurement 1c80e206 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.653e+04, gradient 3.053e-04, dot 0.324; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-1.1359303464077974;dx=-1.6588972426377468
Armijo: th(1.4477801117014265E7)=-0.7421991208776528; dx=3.029232754441431E-8 evalInputDelta=-0.3937312255301446
Armijo: th(7238900.558507132)=-0.7421991208776528; dx=6.140129945683471E-8 evalInputDelta=-0.3937312255301446
Armijo: th(2412966.8528357106)=-0.7421991208776528; dx=1.8000567368440687E-7 evalInputDelta=-0.3937312255301446
Armijo: th(603241.7132089277)=-0.7421991208776528; dx=7.462194218198209E-7 evalInputDelta=-0.3937312255301446
Armijo: th(120648.34264178554)=-0.7421991208776528; dx=3.6790164624282087E-6 evalInputDelta=-0.3937312255301446
Armijo: th(20108.057106964257)=-0.7422043627513067; dx=2.1105693691185334E-5 evalInputDelta=-0.3937259836564907
Armijo: th(2872.5795867091797)=-0.7422835398430279; dx=1.5390376899993143E-4 evalInputDelta=-0.3936468065647696
Armijo: th(359.07244833864746)=-0.7382239445166295; dx=0.0012570489168324808 evalInputDelta=-0.3977064018911679
Armijo: th(39.896938704294165)=-0.7331449658059935; dx=0.011241293646447296 evalInputDelta=-0.4027853806018039
Armijo: th(3.9896938704294165)=-0.7737665784765841; dx=0.07011118036267877 evalInputDelta=-0.3621637679312133
Armijo: th(0.3626994427663106)=-0.9988159450212724; dx=0.044451237918352204 evalInputDelta=-0.1371144013865251
New Minimum: -1.1359303464077974 > -1.1374708788111325
END: th(0.03022495356385922)=-1.1374708788111325; dx=-0.5320418151749666 evalInputDelta=0.001540532403335071
Fitness changed from -1.1359303464077974 to -1.1374708788111325
Iteration 10 complete. Error: -1.1374708788111325 Total: 1.6147; Orientation: 0.0266; Line Search: 1.4757
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 27e71178 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 7.056e+03, gradient 3.074e-04, dot 0.018; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
th(0)=-1.1374708788111325;dx=-0.04844959912092809
New Minimum: -1.1374708788111325 > -1.1386644794475995
WOLF (strong): th(0.06511768846258112)=-1.1386644794475995; dx=0.2927539238572178 evalInputDelta=0.001193600636467007
WOLF (strong): th(0.03255884423129056)=-1.1382391530282012; dx=0.2644377926545892 evalInputDelta=7.682742170687096E-4
WOLF (strong): th(0.010852948077096853)=-1.1377944259206962; dx=0.2532204872862266 evalInputDelta=3.2354710956372656E-4
WOLF (strong): th(0.002713237019274213)=-1.1375545782600978; dx=0.2552956934445908 evalInputDelta=8.36994489652465E-5
WOLF (strong): th(5.426474038548426E-4)=-1.1374892762231696; dx=0.24469074949080272 evalInputDelta=1.8397412037129968E-5
WOLF (strong): th(9.044123397580711E-5)=-1.137473683054821; dx=0.22663570264215505 evalInputDelta=2.804243688592223E-6
WOLF (strong): th(1.292017628225816E-5)=-1.1374711337426626; dx=0.23501853688464402 evalInputDelta=2.5493153010813785E-7
WOLF (strong): th(1.61502203528227E-6)=-1.1374709212997247; dx=0.2477743142234729 evalInputDelta=4.248859220012946E-8
WOLF (strong): th(1.794468928091411E-7)=-1.1374709637883156; dx=0.23894761542106824 evalInputDelta=8.497718306799129E-8
WOLF (strong): th(1.794468928091411E-8)=-1.1374709212997247; dx=0.2548899872965563 evalInputDelta=4.248859220012946E-8
WOLF (strong): th(1.63133538917401E-9)=-1.1374709212997247; dx=0.2354708544201277 evalInputDelta=4.248859220012946E-8
WOLF (strong): th(1.3594461576450083E-10)=-1.1374710062769047; dx=0.23660175574728498 evalInputDelta=1.2746577215949628E-7
MIN ALPHA (1.0457278135730833E-11): th(0.06511768846258112)=-1.1386644794475995
Fitness changed from -1.1374708788111325 to -1.1386644794475995
Iteration 11 complete. Error: -1.1386644794475995 Total: 1.7609; Orientation: 0.0336; Line Search: 1.6073
Adding measurement 19db7503 to history. Total: 5
Rejected: LBFGS Orientation magnitude: 9.455e+03, gradient 3.059e-04, dot -0.052; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.1386644794475995, -1.1374708788111325, -1.1359303464077974, -0.10002098965843205, -0.0820123219835421, -0.07289430209946773
Rejected: LBFGS Orientation magnitude: 1.881e+04, gradient 3.059e-04, dot -0.158; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Orientation rejected. Popping history element from -1.1386644794475995, -1.1374708788111325, -1.1359303464077974, -0.10002098965843205, -0.0820123219835421
Accepted: LBFGS Orientation magnitude: 1.079e+05, gradient 3.059e-04, dot 0.260; [caa308d1-c4b5-4f2c-9705-1965bf8dea0a = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-1.1386644794475995;dx=-8.689838973310373
Armijo: th(2.1544346900318838E-10)=-1.1386643521154285; dx=-4.370120737424839 evalInputDelta=-1.2733217102933736E-7
Armijo: th(1.0772173450159419E-10)=-1.1386644794475995; dx=-4.548987210923682 evalInputDelta=0.0
MIN ALPHA (3.590724483386473E-11): th(0.0)=-1.1386644794475995
Fitness changed from -1.1386644794475995 to -1.1386644794475995
Static Iteration Total: 0.6784; Orientation: 0.1020; Line Search: 0.4618
Iteration 12 failed. Error: -1.1386644794475995
Previous Error: 0.0 -> -1.1386644794475995
Retrying iteration 12
Reset training subject: 142733771366200
Adding measurement 2d83a157 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-1.1386644794475995;dx=-8.909278969272789E-8
END: th(2.1544346900318838E-10)=-1.1386644794475995; dx=-6.745030349243179E-8 evalInputDelta=0.0
Fitness changed from -1.1386644794475995 to -1.1386644794475995
Static Iteration Total: 0.4466; Orientation: 0.0015; Line Search: 0.2273
Iteration 13 failed. Error: -1.1386644794475995
Previous Error: 0.0 -> -1.1386644794475995
Optimization terminated 13
Final threshold in iteration 13: -1.1386644794475995 (> -Infinity) after 9.027s (< 5400.000s)
Returns
-1.1386644794475995