BasicOptimizer.scala:89 executed in 8.46 seconds (0.469 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 148157587126800
Reset training subject: 148157724029800
Adding measurement 62a724b3 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-1.4746451733664137;dx=-6.730709877286924E-4
New Minimum: -1.4746451733664137 > -1.4764419929600707
WOLFE (weak): th(2.154434690031884)=-1.4764419929600707; dx=-6.617730228076586E-4 evalInputDelta=0.001796819593657073
New Minimum: -1.4764419929600707 > -1.4782352942424415
WOLFE (weak): th(4.308869380063768)=-1.4782352942424415; dx=-6.77959789058824E-4 evalInputDelta=0.003590120876027836
New Minimum: -1.4782352942424415 > -1.4854408203795357
WOLFE (weak): th(12.926608140191302)=-1.4854408203795357; dx=-6.671104249615161E-4 evalInputDelta=0.010795647013122078
New Minimum: -1.4854408203795357 > -1.5180529152951667
WOLFE (weak): th(51.70643256076521)=-1.5180529152951667; dx=-6.716012842991678E-4 evalInputDelta=0.043407741928753074
New Minimum: -1.5180529152951667 > -1.6953343605194884
WOLFE (weak): th(258.53216280382605)=-1.6953343605194884; dx=-7.132544619800607E-4 evalInputDelta=0.22068918715307473
New Minimum: -1.6953343605194884 > -2.809797903825574
WOLFE (weak): th(1551.1929768229563)=-2.809797903825574; dx=-7.001734922953149E-4 evalInputDelta=1.3351527304591604
New Minimum: -2.809797903825574 > -5.5317528305512225
END: th(10858.350837760694)=-5.5317528305512225; dx=-1.2922983210494852E-4 evalInputDelta=4.057107657184809
Fitness changed from -1.4746451733664137 to -5.5317528305512225
Iteration 1 complete. Error: -5.5317528305512225 Total: 1.3146; Orientation: 0.0015; Line Search: 0.9443
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 5d350fd4 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-5.5317528305512225;dx=-0.0010715512491212714
Armijo: th(23393.607721408407)=-4.272588500201026; dx=-1.5498254942552254E-5 evalInputDelta=-1.2591643303501963
New Minimum: -5.5317528305512225 > -7.532073105519776
END: th(11696.803860704204)=-7.532073105519776; dx=-7.93354240777122E-6 evalInputDelta=2.000320274968553
Fitness changed from -5.5317528305512225 to -7.532073105519776
Iteration 2 complete. Error: -7.532073105519776 Total: 0.4720; Orientation: 0.0015; Line Search: 0.3472
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 1fc30deb to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-7.532073105519776;dx=-0.0015689356888405883
Armijo: th(25200.000000000004)=-7.48392678339526; dx=1.2382062345822238E-5 evalInputDelta=-0.04814632212451553
New Minimum: -7.532073105519776 > -10.689120943354009
END: th(12600.000000000002)=-10.689120943354009; dx=-1.373335908476885E-4 evalInputDelta=3.1570478378342335
Fitness changed from -7.532073105519776 to -10.689120943354009
Iteration 3 complete. Error: -10.689120943354009 Total: 0.4595; Orientation: 0.0014; Line Search: 0.3396
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 51a21f0 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.243e+03, gradient 4.388e-02, dot -0.300; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -10.689120943354009, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 51a21f0 to history. Total: 3
th(0)=-10.689120943354009;dx=-0.0018791129320596183
Armijo: th(27145.87709440174)=-7.918001240758789; dx=2.9067234231576258E-5 evalInputDelta=-2.77111970259522
New Minimum: -10.689120943354009 > -13.5279943282919
WOLF (strong): th(13572.93854720087)=-13.5279943282919; dx=4.86500506116523E-6 evalInputDelta=2.838873384937891
New Minimum: -13.5279943282919 > -16.249583760329024
END: th(4524.312849066957)=-16.249583760329024; dx=-4.0226825942364945E-4 evalInputDelta=5.560462816975015
Fitness changed from -10.689120943354009 to -16.249583760329024
Iteration 4 complete. Error: -16.249583760329024 Total: 0.6422; Orientation: 0.0276; Line Search: 0.4928
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 4154ca15 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.898e+03, gradient 3.670e-02, dot -0.389; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -16.249583760329024, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 4154ca15 to history. Total: 3
th(0)=-16.249583760329024;dx=-0.0013186954555396279
New Minimum: -16.249583760329024 > -22.5703674931757
END: th(9747.336550586839)=-22.5703674931757; dx=-3.039317607369783E-4 evalInputDelta=6.3207837328466745
Fitness changed from -16.249583760329024 to -22.5703674931757
Iteration 5 complete. Error: -22.5703674931757 Total: 0.3879; Orientation: 0.0293; Line Search: 0.2381
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 7542f938 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.860e+03, gradient 4.433e-02, dot -0.300; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -22.5703674931757, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 7542f938 to history. Total: 3
th(0)=-22.5703674931757;dx=-0.0018892609914802384
Armijo: th(21000.000000000007)=-18.64005443885746; dx=4.791363506467099E-5 evalInputDelta=-3.930313054318237
New Minimum: -22.5703674931757 > -24.219787082577994
WOLF (strong): th(10500.000000000004)=-24.219787082577994; dx=7.965696886757223E-5 evalInputDelta=1.6494195894022958
New Minimum: -24.219787082577994 > -27.556568491466756
END: th(3500.0000000000014)=-27.556568491466756; dx=-6.663009399774607E-4 evalInputDelta=4.986200998291057
Fitness changed from -22.5703674931757 to -27.556568491466756
Iteration 6 complete. Error: -27.556568491466756 Total: 0.6233; Orientation: 0.0303; Line Search: 0.4686
Adding measurement bbafd1a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.528e+03, gradient 3.631e-02, dot -0.384; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -27.556568491466756, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement bbafd1a to history. Total: 3
th(0)=-27.556568491466756;dx=-0.0012459830339216927
New Minimum: -27.556568491466756 > -33.30520314396671
END: th(7540.521415111596)=-33.30520314396671; dx=-3.70655124900472E-4 evalInputDelta=5.748634652499952
Fitness changed from -27.556568491466756 to -33.30520314396671
Iteration 7 complete. Error: -33.30520314396671 Total: 0.3749; Orientation: 0.0298; Line Search: 0.2337
Adding measurement 787d596 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.242e+03, gradient 4.219e-02, dot -0.352; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -33.30520314396671, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 787d596 to history. Total: 3
th(0)=-33.30520314396671;dx=-0.0016744670275813214
Armijo: th(16245.560917644734)=-32.006229754429214; dx=9.819522458326739E-5 evalInputDelta=-1.298973389537494
New Minimum: -33.30520314396671 > -38.257291579672916
WOLF (strong): th(8122.780458822367)=-38.257291579672916; dx=2.240773329376229E-5 evalInputDelta=4.952088435706209
New Minimum: -38.257291579672916 > -38.373816737719245
END: th(2707.5934862741224)=-38.373816737719245; dx=-8.252189709152984E-4 evalInputDelta=5.068613593752538
Fitness changed from -33.30520314396671 to -38.373816737719245
Iteration 8 complete. Error: -38.373816737719245 Total: 0.6010; Orientation: 0.0283; Line Search: 0.4595
Adding measurement 3ac70042 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 8.033e+03, gradient 3.255e-02, dot -0.481; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -38.373816737719245, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 3ac70042 to history. Total: 3
th(0)=-38.373816737719245;dx=-0.0010012445979107746
New Minimum: -38.373816737719245 > -44.06681344085818
END: th(5833.333333333337)=-44.06681344085818; dx=-6.803944709741289E-4 evalInputDelta=5.692996703138938
Fitness changed from -38.373816737719245 to -44.06681344085818
Iteration 9 complete. Error: -44.06681344085818 Total: 0.3648; Orientation: 0.0288; Line Search: 0.2257
Adding measurement 23887c58 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.370e+04, gradient 3.208e-02, dot -0.488; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -44.06681344085818, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 23887c58 to history. Total: 3
th(0)=-44.06681344085818;dx=-9.615438501204714E-4
New Minimum: -44.06681344085818 > -55.92858593118218
END: th(12567.535691852663)=-55.92858593118218; dx=-5.210094018572626E-4 evalInputDelta=11.861772490323993
Fitness changed from -44.06681344085818 to -55.92858593118218
Iteration 10 complete. Error: -55.92858593118218 Total: 0.8639; Orientation: 0.0288; Line Search: 0.2374
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 7d7e533e to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.010e+04, gradient 4.560e-02, dot -0.401; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -55.92858593118218, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 7d7e533e to history. Total: 3
th(0)=-55.92858593118218;dx=-0.001977182506672325
Armijo: th(27075.93486274123)=-48.98703691217844; dx=2.551104371293279E-5 evalInputDelta=-6.941549019003737
New Minimum: -55.92858593118218 > -57.16027636373474
WOLF (strong): th(13537.967431370615)=-57.16027636373474; dx=1.5029621506414012E-4 evalInputDelta=1.2316904325525613
New Minimum: -57.16027636373474 > -62.64308880474453
END: th(4512.655810456871)=-62.64308880474453; dx=-4.5640684335859685E-4 evalInputDelta=6.714502873562353
Fitness changed from -55.92858593118218 to -62.64308880474453
Iteration 11 complete. Error: -62.64308880474453 Total: 0.6074; Orientation: 0.0279; Line Search: 0.4590
Adding measurement 3bc9fd9a to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.246e+06, gradient 3.921e-02, dot -0.426; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -62.64308880474453, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 3bc9fd9a to history. Total: 3
th(0)=-62.64308880474453;dx=-0.0014565359910517885
New Minimum: -62.64308880474453 > -71.26497253341299
END: th(9722.222222222228)=-71.26497253341299; dx=-1.8160109155616612E-4 evalInputDelta=8.621883728668458
Fitness changed from -62.64308880474453 to -71.26497253341299
Iteration 12 complete. Error: -71.26497253341299 Total: 0.3667; Orientation: 0.0270; Line Search: 0.2294
Adding measurement 783ea2de to history. Total: 3
Rejected: LBFGS Orientation magnitude: 6.918e+05, gradient 5.361e-02, dot -0.335; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -71.26497253341299, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 783ea2de to history. Total: 3
th(0)=-71.26497253341299;dx=-0.0027932019149652897
Armijo: th(20945.892819754437)=-55.23182829753133; dx=2.9727661505822713E-4 evalInputDelta=-16.033144235881657
New Minimum: -71.26497253341299 > -71.84873320554071
WOLF (strong): th(10472.946409877219)=-71.84873320554071; dx=6.438137134712831E-4 evalInputDelta=0.583760672127724
New Minimum: -71.84873320554071 > -80.47104467446509
END: th(3490.9821366257397)=-80.47104467446509; dx=-6.097993022412666E-4 evalInputDelta=9.206072141052104
Fitness changed from -71.26497253341299 to -80.47104467446509
Iteration 13 complete. Error: -80.47104467446509 Total: 0.5978; Orientation: 0.0272; Line Search: 0.4631
Adding measurement 59d48158 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.194e+04, gradient 3.532e-02, dot -0.454; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -80.47104467446509, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 59d48158 to history. Total: 3
th(0)=-80.47104467446509;dx=-0.001212669795505989
New Minimum: -80.47104467446509 > -89.21912054416454
END: th(7521.093017428119)=-89.21912054416454; dx=-6.753470736128278E-4 evalInputDelta=8.748075869699449
Fitness changed from -80.47104467446509 to -89.21912054416454
Iteration 14 complete. Error: -89.21912054416454 Total: 0.3817; Orientation: 0.0317; Line Search: 0.2319
Adding measurement 54c18554 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.178e+04, gradient 3.624e-02, dot -0.450; [660d65ce-3de4-42a6-a0ae-24f510fd1ec8 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -89.21912054416454, -7.532073105519776, -5.5317528305512225, -1.4746451733664137
LBFGS Accumulation History: 3 points
Removed measurement 54c18554 to history. Total: 3
th(0)=-89.21912054416454;dx=-0.001248198735823621
New Minimum: -89.21912054416454 > -96.7012773234716
END: th(16203.703703703715)=-96.7012773234716; dx=-2.0803613090489772E-4 evalInputDelta=7.482156779307061
Fitness changed from -89.21912054416454 to -96.7012773234716
Iteration 15 complete. Error: -96.7012773234716 Total: 0.3891; Orientation: 0.0325; Line Search: 0.2436
<a id="p-15"></a>Iteration 15
<a id="p-14"></a>
Final threshold in iteration 16: -96.7012773234716 (> -Infinity) after 8.457s (< 5400.000s)
Returns
-96.7012773234716