BasicOptimizer.scala:89 executed in 10.66 seconds (0.806 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 140503083322600
Reset training subject: 140503215371700
Adding measurement 7da9f87f to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-6.786496974624742;dx=-0.008625214914508149
New Minimum: -6.786496974624742 > -6.818639659790593
WOLFE (weak): th(2.154434690031884)=-6.818639659790593; dx=-0.008532925825388704 evalInputDelta=0.032142685165850615
New Minimum: -6.818639659790593 > -6.8510837489450465
WOLFE (weak): th(4.308869380063768)=-6.8510837489450465; dx=-0.008592004547824867 evalInputDelta=0.06458677432030413
New Minimum: -6.8510837489450465 > -6.983069889524578
WOLFE (weak): th(12.926608140191302)=-6.983069889524578; dx=-0.008751832463160606 evalInputDelta=0.19657291489983564
New Minimum: -6.983069889524578 > -7.593847827737051
WOLFE (weak): th(51.70643256076521)=-7.593847827737051; dx=-0.009277581904302796 evalInputDelta=0.8073508531123084
New Minimum: -7.593847827737051 > -10.93821381978776
WOLFE (weak): th(258.53216280382605)=-10.93821381978776; dx=-0.008625962075855174 evalInputDelta=4.151716845163018
New Minimum: -10.93821381978776 > -23.57924709029324
END: th(1551.1929768229563)=-23.57924709029324; dx=-0.002882798485723699 evalInputDelta=16.792750115668497
Fitness changed from -6.786496974624742 to -23.57924709029324
Iteration 1 complete. Error: -23.57924709029324 Total: 1.1576; Orientation: 0.0014; Line Search: 0.7978
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 4f1280a2 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-23.57924709029324;dx=-0.0071901080478346145
New Minimum: -23.57924709029324 > -44.15899842303787
END: th(3341.943960201201)=-44.15899842303787; dx=-0.0016508915836058971 evalInputDelta=20.57975133274463
Fitness changed from -23.57924709029324 to -44.15899842303787
Iteration 2 complete. Error: -44.15899842303787 Total: 0.3543; Orientation: 0.0016; Line Search: 0.2300
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 2d60d5ac to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-44.15899842303787;dx=-0.011445629019457781
New Minimum: -44.15899842303787 > -76.75898879202502
END: th(7200.000000000001)=-76.75898879202502; dx=-4.1329533999739015E-4 evalInputDelta=32.59999036898715
Fitness changed from -44.15899842303787 to -76.75898879202502
Iteration 3 complete. Error: -76.75898879202502 Total: 0.3539; Orientation: 0.0016; Line Search: 0.2296
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 441b55f6 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.680e+03, gradient 1.318e-01, dot -0.349; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -76.75898879202502, -44.15899842303787, -23.57924709029324, -6.786496974624742
LBFGS Accumulation History: 3 points
Removed measurement 441b55f6 to history. Total: 3
th(0)=-76.75898879202502;dx=-0.016741123368054965
New Minimum: -76.75898879202502 > -96.45868077386655
WOLF (strong): th(15511.929768229566)=-96.45868077386655; dx=4.438485201297686E-4 evalInputDelta=19.699691981841525
New Minimum: -96.45868077386655 > -130.99934363736503
END: th(7755.964884114783)=-130.99934363736503; dx=-0.001049936285946919 evalInputDelta=54.24035484534001
Fitness changed from -76.75898879202502 to -130.99934363736503
Iteration 4 complete. Error: -130.99934363736503 Total: 0.4906; Orientation: 0.0278; Line Search: 0.3398
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement 6fc768b1 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.003e+03, gradient 1.186e-01, dot -0.270; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -130.99934363736503, -44.15899842303787, -23.57924709029324, -6.786496974624742
LBFGS Accumulation History: 3 points
Removed measurement 6fc768b1 to history. Total: 3
th(0)=-130.99934363736503;dx=-0.013531933581966677
New Minimum: -130.99934363736503 > -133.6373227320379
WOLF (strong): th(16709.71980100601)=-133.6373227320379; dx=2.1326423907074292E-4 evalInputDelta=2.6379790946728576
New Minimum: -133.6373227320379 > -179.9196002366057
END: th(8354.859900503005)=-179.9196002366057; dx=-0.0017477156888841055 evalInputDelta=48.92025659924067
Fitness changed from -130.99934363736503 to -179.9196002366057
Iteration 5 complete. Error: -179.9196002366057 Total: 0.4939; Orientation: 0.0280; Line Search: 0.3422
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 52c9dce to history. Total: 3
Rejected: LBFGS Orientation magnitude: 7.055e+03, gradient 1.175e-01, dot -0.316; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -179.9196002366057, -44.15899842303787, -23.57924709029324, -6.786496974624742
LBFGS Accumulation History: 3 points
Removed measurement 52c9dce to history. Total: 3
th(0)=-179.9196002366057;dx=-0.013275603917682462
New Minimum: -179.9196002366057 > -227.77785266988124
WOLF (strong): th(18000.000000000007)=-227.77785266988124; dx=4.538706220622507E-4 evalInputDelta=47.85825243327554
New Minimum: -227.77785266988124 > -256.47174039386914
END: th(9000.000000000004)=-256.47174039386914; dx=-0.0019573456449048033 evalInputDelta=76.55214015726344
Fitness changed from -179.9196002366057 to -256.47174039386914
Iteration 6 complete. Error: -256.47174039386914 Total: 0.4988; Orientation: 0.0279; Line Search: 0.3469
Adding measurement 3842a6a7 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.772e+04, gradient 1.127e-01, dot -0.352; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -256.47174039386914, -44.15899842303787, -23.57924709029324, -6.786496974624742
LBFGS Accumulation History: 3 points
Removed measurement 3842a6a7 to history. Total: 3
th(0)=-256.47174039386914;dx=-0.012282685670360508
New Minimum: -256.47174039386914 > -262.7050741356489
WOLF (strong): th(19389.912210286962)=-262.7050741356489; dx=9.927324303792937E-4 evalInputDelta=6.233333741779745
New Minimum: -262.7050741356489 > -313.0029908044388
END: th(9694.956105143481)=-313.0029908044388; dx=-0.001978167199598738 evalInputDelta=56.53125041056967
Fitness changed from -256.47174039386914 to -313.0029908044388
Iteration 7 complete. Error: -313.0029908044388 Total: 0.4920; Orientation: 0.0272; Line Search: 0.3524
Adding measurement 54711083 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.194e+05, gradient 1.222e-01, dot -0.382; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -313.0029908044388, -44.15899842303787, -23.57924709029324, -6.786496974624742
LBFGS Accumulation History: 3 points
Removed measurement 54711083 to history. Total: 3
th(0)=-313.0029908044388;dx=-0.014279630692823652
New Minimum: -313.0029908044388 > -376.3732761028661
WOLF (strong): th(20887.149751257515)=-376.3732761028661; dx=0.0010049831371637564 evalInputDelta=63.37028529842729
New Minimum: -376.3732761028661 > -403.3465419068875
END: th(10443.574875628758)=-403.3465419068875; dx=-6.677483795309819E-4 evalInputDelta=90.34355110244871
Fitness changed from -313.0029908044388 to -403.3465419068875
Iteration 8 complete. Error: -403.3465419068875 Total: 0.5020; Orientation: 0.0270; Line Search: 0.3569
Adding measurement 4f957495 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.282e+05, gradient 1.213e-01, dot -0.346; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -403.3465419068875, -44.15899842303787, -23.57924709029324, -6.786496974624742
LBFGS Accumulation History: 3 points
Removed measurement 4f957495 to history. Total: 3
th(0)=-403.3465419068875;dx=-0.014181018699794648
Armijo: th(22500.00000000001)=-321.39054629722403; dx=0.0029760951154020974 evalInputDelta=-81.95599560966349
New Minimum: -403.3465419068875 > -445.429370695668
WOLF (strong): th(11250.000000000005)=-445.429370695668; dx=4.5160112515740456E-4 evalInputDelta=42.08282878878049
New Minimum: -445.429370695668 > -449.1020416253953
END: th(3750.000000000002)=-449.1020416253953; dx=-0.007759070705817233 evalInputDelta=45.75549971850779
Fitness changed from -403.3465419068875 to -449.1020416253953
Iteration 9 complete. Error: -449.1020416253953 Total: 0.6220; Orientation: 0.0272; Line Search: 0.4824
Adding measurement fb65bf2 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 3.492e+04, gradient 9.808e-02, dot -0.414; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -449.1020416253953, -44.15899842303787, -23.57924709029324, -6.786496974624742
LBFGS Accumulation History: 3 points
Removed measurement fb65bf2 to history. Total: 3
th(0)=-449.1020416253953;dx=-0.009194174474890546
New Minimum: -449.1020416253953 > -492.47443689461153
END: th(8079.130087619568)=-492.47443689461153; dx=-0.00470839264203259 evalInputDelta=43.37239526921621
Fitness changed from -449.1020416253953 to -492.47443689461153
Iteration 10 complete. Error: -492.47443689461153 Total: 0.3983; Orientation: 0.0322; Line Search: 0.2527
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 295e6b0b to history. Total: 3
Accepted: LBFGS Orientation magnitude: 2.319e+04, gradient 1.036e-01, dot 0.538; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-492.47443689461153;dx=-1287.81276219739
Armijo: th(17405.95812604793)=-370.42273904613745; dx=0.010625464796566599 evalInputDelta=-122.05169784847408
Armijo: th(8702.979063023966)=-370.4001694624701; dx=0.021208075916407013 evalInputDelta=-122.07426743214143
Armijo: th(2900.993021007989)=-370.56949280561406; dx=0.06347171229153328 evalInputDelta=-121.90494408899747
Armijo: th(725.2482552519972)=-370.5646493930821; dx=0.2608053448642631 evalInputDelta=-121.90978750152942
Armijo: th(145.04965105039943)=-371.71987952368846; dx=1.2757010628635488 evalInputDelta=-120.75455737092307
Armijo: th(24.17494184173324)=-376.75896626788693; dx=7.400751261314385 evalInputDelta=-115.7154706267246
Armijo: th(3.4535631202476056)=-426.26421067839914; dx=51.837381661234396 evalInputDelta=-66.21022621621239
New Minimum: -492.47443689461153 > -661.3784475693967
WOLF (strong): th(0.4316953900309507)=-661.3784475693967; dx=7.960052499856062 evalInputDelta=168.90401067478518
END: th(0.04796615444788341)=-566.06169039887; dx=-818.1982887358881 evalInputDelta=73.58725350425851
Fitness changed from -492.47443689461153 to -661.3784475693967
Iteration 11 complete. Error: -661.3784475693967 Total: 1.5225; Orientation: 0.2281; Line Search: 1.1717
Adding measurement 1a41c90 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.306e+04, gradient 1.254e-01, dot 0.234; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
th(0)=-661.3784475693967;dx=-383.4790513788614
Armijo: th(0.10333994708994715)=-648.8758594753032; dx=21.437442403908193 evalInputDelta=-12.50258809409354
Armijo: th(0.051669973544973574)=-655.3715139108003; dx=11.110022276582107 evalInputDelta=-6.006933658596381
Armijo: th(0.01722332451499119)=-659.4146883454874; dx=1.620671204949229 evalInputDelta=-1.9637592239092783
Armijo: th(0.0043058311287477976)=-660.9026101313945; dx=-1.297422441159057 evalInputDelta=-0.47583743800225875
Armijo: th(8.611662257495595E-4)=-661.2829836824517; dx=8.494811060472717 evalInputDelta=-0.09546388694502639
Armijo: th(1.4352770429159324E-4)=-661.362545813597; dx=2.8460398118939523 evalInputDelta=-0.015901755799745843
Armijo: th(2.050395775594189E-5)=-661.3761147010098; dx=2.2750986141012692 evalInputDelta=-0.0023328683869294764
Armijo: th(2.5629947194927363E-6)=-661.3781619124853; dx=5.115811192427103 evalInputDelta=-2.856569113873775E-4
Armijo: th(2.8477719105474847E-7)=-661.3784237646588; dx=4.965658844750449 evalInputDelta=-2.3804737907084927E-5
Armijo: th(2.8477719105474847E-8)=-661.3784475693967; dx=2.319756900197264 evalInputDelta=0.0
Armijo: th(2.5888835550431677E-9)=-661.3784475693967; dx=0.3643857009294713 evalInputDelta=0.0
New Minimum: -661.3784475693967 > -661.37849517887
WOLF (strong): th(2.157402962535973E-10)=-661.37849517887; dx=8.579425032652189 evalInputDelta=4.7609473313059425E-5
MIN ALPHA (1.659540740412287E-11): th(2.157402962535973E-10)=-661.37849517887
Fitness changed from -661.3784475693967 to -661.37849517887
Iteration 12 complete. Error: -661.37849517887 Total: 1.7596; Orientation: 0.0338; Line Search: 1.6097
Adding measurement 71e996e9 to history. Total: 5
Rejected: LBFGS Orientation magnitude: 3.107e+05, gradient 1.265e-01, dot -0.181; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -661.37849517887, -661.3784475693967, -492.47443689461153, -44.15899842303787, -23.57924709029324, -6.786496974624742
Rejected: LBFGS Orientation magnitude: 4.259e+05, gradient 1.265e-01, dot -0.187; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -661.37849517887, -661.3784475693967, -492.47443689461153, -44.15899842303787, -23.57924709029324
Accepted: LBFGS Orientation magnitude: 9.980e+05, gradient 1.265e-01, dot 0.203; [2a25e050-4e2f-4e77-9930-dc340ab91856 = 1.000/1.000e+00]
Overwriting history with 4 points
th(0)=-661.37849517887;dx=-26143.706111689542
Armijo: th(2.1544346900318838E-10)=-661.37849517887; dx=-973.2386131675537 evalInputDelta=0.0
Armijo: th(1.0772173450159419E-10)=-661.3784237646588; dx=-891.4732417561114 evalInputDelta=-7.141421122014435E-5
MIN ALPHA (3.590724483386473E-11): th(0.0)=-661.37849517887
Fitness changed from -661.37849517887 to -661.37849517887
Static Iteration Total: 0.7110; Orientation: 0.1136; Line Search: 0.4833
Iteration 13 failed. Error: -661.37849517887
Previous Error: 0.0 -> -661.37849517887
Retrying iteration 13
Reset training subject: 140512439938000
Adding measurement 1550b6f2 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-661.37849517887;dx=-0.015478889171844272
END: th(2.1544346900318838E-10)=-661.37849517887; dx=-0.010231510672420584 evalInputDelta=0.0
Fitness changed from -661.37849517887 to -661.37849517887
Static Iteration Total: 1.3071; Orientation: 0.0020; Line Search: 0.2385
Iteration 14 failed. Error: -661.37849517887
Previous Error: 0.0 -> -661.37849517887
Optimization terminated 14
Final threshold in iteration 14: -661.37849517887 (> -Infinity) after 10.664s (< 5400.000s)
Returns
-661.37849517887