BasicOptimizer.scala:89 executed in 127.79 seconds (1.446 gc):
val lineSearchInstance: LineSearchStrategy = lineSearchFactory
val trainer = new IterativeTrainer(trainable)
trainer.setOrientation(orientation())
trainer.setMonitor(new TrainingMonitor() {
override def clear(): Unit = trainingMonitor.clear()
override def log(msg: String): Unit = {
trainingMonitor.log(msg)
BasicOptimizer.this.log(msg)
}
override def onStepFail(currentPoint: Step): Boolean = {
BasicOptimizer.this.onStepFail(trainable.addRef().asInstanceOf[Trainable], currentPoint)
}
override def onStepComplete(currentPoint: Step): Unit = {
if (0 < logEvery && (0 == currentPoint.iteration % logEvery || currentPoint.iteration < logEvery)) {
val image = currentImage()
timelineAnimation += image
val caption = "Iteration " + currentPoint.iteration
out.p(caption + "\n" + out.jpg(image, caption))
}
BasicOptimizer.this.onStepComplete(trainable.addRef().asInstanceOf[Trainable], currentPoint)
trainingMonitor.onStepComplete(currentPoint)
super.onStepComplete(currentPoint)
}
})
trainer.setTimeout(trainingMinutes, TimeUnit.MINUTES)
trainer.setMaxIterations(trainingIterations)
trainer.setLineSearchFactory((_: CharSequence) => lineSearchInstance)
trainer.setTerminateThreshold(java.lang.Double.NEGATIVE_INFINITY)
val result = trainer.run.asInstanceOf[lang.Double]
trainer.freeRef()
result
Reset training subject: 11421754888730
Reset training subject: 11422924959085
Adding measurement 3e057336 to history. Total: 0
LBFGS Accumulation History: 1 points
Constructing line search parameters: GD+Trust
th(0)=-0.1473493785632601;dx=-1.3530830423264735E-5
New Minimum: -0.1473493785632601 > -0.1473843331338149
WOLFE (weak): th(2.154434690031884)=-0.1473843331338149; dx=-1.3823074533409395E-5 evalInputDelta=3.4954570554801556E-5
New Minimum: -0.1473843331338149 > -0.14741941840142028
WOLFE (weak): th(4.308869380063768)=-0.14741941840142028; dx=-1.4590901049347393E-5 evalInputDelta=7.003983816017412E-5
New Minimum: -0.14741941840142028 > -0.1475588618641506
WOLFE (weak): th(12.926608140191302)=-0.1475588618641506; dx=-1.3059906993404243E-5 evalInputDelta=2.094833008904895E-4
New Minimum: -0.1475588618641506 > -0.1481906570112763
WOLFE (weak): th(51.70643256076521)=-0.1481906570112763; dx=-1.3422093085940733E-5 evalInputDelta=8.412784480161806E-4
New Minimum: -0.1481906570112763 > -0.15157820236822234
WOLFE (weak): th(258.53216280382605)=-0.15157820236822234; dx=-1.3479213382343661E-5 evalInputDelta=0.004228823804962228
New Minimum: -0.15157820236822234 > -0.17319327591839467
WOLFE (weak): th(1551.1929768229563)=-0.17319327591839467; dx=-1.4952381954610533E-5 evalInputDelta=0.025843897355134565
New Minimum: -0.17319327591839467 > -0.34182551672684464
WOLFE (weak): th(10858.350837760694)=-0.34182551672684464; dx=-1.2550942106758922E-5 evalInputDelta=0.19447613816358453
New Minimum: -0.34182551672684464 > -1.3769362861198882
END: th(86866.80670208555)=-1.3769362861198882; dx=-7.174863268021667E-6 evalInputDelta=1.2295869075566281
Fitness changed from -0.1473493785632601 to -1.3769362861198882
Iteration 1 complete. Error: -1.3769362861198882 Total: 14.9816; Orientation: 0.0185; Line Search: 11.4606
<a id="p-3"></a>Iteration 1
<a id="p-2"></a>
Adding measurement 162cf775 to history. Total: 1
LBFGS Accumulation History: 2 points
th(0)=-1.3769362861198882;dx=-5.4392322217393E-5
New Minimum: -1.3769362861198882 > -2.5679161704691857
END: th(187148.86177126726)=-2.5679161704691857; dx=-1.2085023391018164E-6 evalInputDelta=1.1909798843492976
Fitness changed from -1.3769362861198882 to -2.5679161704691857
Iteration 2 complete. Error: -2.5679161704691857 Total: 3.7132; Orientation: 0.0174; Line Search: 2.3953
<a id="p-5"></a>Iteration 2
<a id="p-4"></a>
Adding measurement 3ade3f0 to history. Total: 2
LBFGS Accumulation History: 3 points
th(0)=-2.5679161704691857;dx=-5.48834891935696E-5
New Minimum: -2.5679161704691857 > -6.225221266893037
END: th(403200.00000000006)=-6.225221266893037; dx=-2.3551543543096788E-6 evalInputDelta=3.657305096423851
Fitness changed from -2.5679161704691857 to -6.225221266893037
Iteration 3 complete. Error: -6.225221266893037 Total: 3.7322; Orientation: 0.0172; Line Search: 2.4222
<a id="p-7"></a>Iteration 3
<a id="p-6"></a>
Adding measurement 32ced97f to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.257e+03, gradient 1.328e-02, dot -0.122; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -6.225221266893037, -2.5679161704691857, -1.3769362861198882, -0.1473493785632601
LBFGS Accumulation History: 3 points
Removed measurement 32ced97f to history. Total: 3
th(0)=-6.225221266893037;dx=-1.6474387939363521E-4
New Minimum: -6.225221266893037 > -9.070177496169064
WOLF (strong): th(868668.0670208557)=-9.070177496169064; dx=3.2887399370282103E-6 evalInputDelta=2.8449562292760273
New Minimum: -9.070177496169064 > -11.608730950338359
WOLF (strong): th(434334.03351042786)=-11.608730950338359; dx=2.0108001593281653E-6 evalInputDelta=5.383509683445322
New Minimum: -11.608730950338359 > -15.955862679059626
END: th(144778.01117014262)=-15.955862679059626; dx=-1.604712271572816E-5 evalInputDelta=9.73064141216659
Fitness changed from -6.225221266893037 to -15.955862679059626
Iteration 4 complete. Error: -15.955862679059626 Total: 6.6234; Orientation: 0.4643; Line Search: 4.8494
<a id="p-9"></a>Iteration 4
<a id="p-8"></a>
Adding measurement f291663 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.223e+04, gradient 2.077e-02, dot -0.075; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -15.955862679059626, -2.5679161704691857, -1.3769362861198882, -0.1473493785632601
LBFGS Accumulation History: 3 points
Removed measurement f291663 to history. Total: 3
th(0)=-15.955862679059626;dx=-4.024644545202301E-4
New Minimum: -15.955862679059626 > -16.105404798411715
WOLF (strong): th(311914.76961877884)=-16.105404798411715; dx=4.41845862614946E-6 evalInputDelta=0.1495421193520894
New Minimum: -16.105404798411715 > -21.14138140738443
END: th(155957.38480938942)=-21.14138140738443; dx=-5.5815308249574E-6 evalInputDelta=5.185518728324803
Fitness changed from -15.955862679059626 to -21.14138140738443
Iteration 5 complete. Error: -21.14138140738443 Total: 5.4024; Orientation: 0.4594; Line Search: 3.6429
<a id="p-11"></a>Iteration 5
<a id="p-10"></a>
Adding measurement 35ef753b to history. Total: 3
Rejected: LBFGS Orientation magnitude: 1.026e+04, gradient 2.431e-02, dot -0.058; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -21.14138140738443, -2.5679161704691857, -1.3769362861198882, -0.1473493785632601
LBFGS Accumulation History: 3 points
Removed measurement 35ef753b to history. Total: 3
th(0)=-21.14138140738443;dx=-5.695464488139482E-4
Armijo: th(336000.0000000001)=-16.966458610253277; dx=5.12123021301799E-6 evalInputDelta=-4.174922797131153
New Minimum: -21.14138140738443 > -29.839384292918457
END: th(168000.00000000006)=-29.839384292918457; dx=-1.4398510049502657E-5 evalInputDelta=8.698002885534027
Fitness changed from -21.14138140738443 to -29.839384292918457
Iteration 6 complete. Error: -29.839384292918457 Total: 5.8883; Orientation: 0.4607; Line Search: 4.1302
Adding measurement 3a176724 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 2.813e+04, gradient 2.114e-02, dot -0.073; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -29.839384292918457, -2.5679161704691857, -1.3769362861198882, -0.1473493785632601
LBFGS Accumulation History: 3 points
Removed measurement 3a176724 to history. Total: 3
th(0)=-29.839384292918457;dx=-4.1675847376060283E-4
New Minimum: -29.839384292918457 > -32.03720219087101
WOLF (strong): th(361945.0279253566)=-32.03720219087101; dx=4.042388799857044E-6 evalInputDelta=2.1978178979525502
New Minimum: -32.03720219087101 > -49.20881100045118
END: th(180972.5139626783)=-49.20881100045118; dx=-1.6196079949717015E-5 evalInputDelta=19.36942670753272
Fitness changed from -29.839384292918457 to -49.20881100045118
Iteration 7 complete. Error: -49.20881100045118 Total: 5.2670; Orientation: 0.4703; Line Search: 3.6588
Adding measurement 342174ce to history. Total: 3
Rejected: LBFGS Orientation magnitude: 4.987e+05, gradient 2.638e-02, dot -0.064; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -49.20881100045118, -2.5679161704691857, -1.3769362861198882, -0.1473493785632601
LBFGS Accumulation History: 3 points
Removed measurement 342174ce to history. Total: 3
th(0)=-49.20881100045118;dx=-6.46979153456976E-4
Armijo: th(389893.4620234736)=-24.380991351005655; dx=2.9585265341481774E-5 evalInputDelta=-24.827819649445523
New Minimum: -49.20881100045118 > -56.3665979049206
WOLF (strong): th(194946.7310117368)=-56.3665979049206; dx=5.16714306481389E-6 evalInputDelta=7.157786904469425
New Minimum: -56.3665979049206 > -70.64495016896112
END: th(64982.24367057893)=-70.64495016896112; dx=-1.2562281815919364E-4 evalInputDelta=21.43613916850994
Fitness changed from -49.20881100045118 to -70.64495016896112
Iteration 8 complete. Error: -70.64495016896112 Total: 6.6016; Orientation: 0.4565; Line Search: 4.9830
Adding measurement 58921873 to history. Total: 3
Rejected: LBFGS Orientation magnitude: 5.648e+05, gradient 2.524e-02, dot -0.077; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
Orientation rejected. Popping history element from -70.64495016896112, -2.5679161704691857, -1.3769362861198882, -0.1473493785632601
LBFGS Accumulation History: 3 points
Removed measurement 58921873 to history. Total: 3
th(0)=-70.64495016896112;dx=-6.013155877753768E-4
New Minimum: -70.64495016896112 > -82.19394283206792
END: th(140000.00000000006)=-82.19394283206792; dx=-2.803772656014968E-5 evalInputDelta=11.548992663106802
Fitness changed from -70.64495016896112 to -82.19394283206792
Iteration 9 complete. Error: -82.19394283206792 Total: 4.0995; Orientation: 0.4623; Line Search: 2.4779
Adding measurement 63fb9c89 to history. Total: 3
Accepted: LBFGS Orientation magnitude: 1.639e+04, gradient 3.084e-02, dot 0.106; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
Constructing line search parameters: LBFGS+Trust
th(0)=-82.19394283206792;dx=-53.50500846211085
Armijo: th(301620.8566044639)=-17.968433251284303; dx=1.719995476169632E-5 evalInputDelta=-64.22550958078361
Armijo: th(150810.42830223194)=-17.959554372581707; dx=3.9720586176059114E-5 evalInputDelta=-64.23438845948621
Armijo: th(50270.142767410645)=-17.90033084840674; dx=1.0071161478873723E-4 evalInputDelta=-64.29361198366118
Armijo: th(12567.535691852661)=-17.916356189018927; dx=4.7316053201305175E-4 evalInputDelta=-64.27758664304899
Armijo: th(2513.507138370532)=-18.110301825937704; dx=0.002560967510442622 evalInputDelta=-64.08364100613022
Armijo: th(418.9178563950887)=-18.65651172345731; dx=0.01645917480698289 evalInputDelta=-63.537431108610605
Armijo: th(59.84540805644124)=-21.087577945404625; dx=0.10298295679516697 evalInputDelta=-61.10636488666329
Armijo: th(7.480676007055155)=-37.41567667874156; dx=1.4269903769989079 evalInputDelta=-44.778266153326356
Armijo: th(0.8311862230061283)=-69.867375758021; dx=-4.172127611698638 evalInputDelta=-12.32656707404692
New Minimum: -82.19394283206792 > -85.11620904956206
END: th(0.08311862230061283)=-85.11620904956206; dx=-26.008675681371557 evalInputDelta=2.9222662174941405
Fitness changed from -82.19394283206792 to -85.11620904956206
Iteration 10 complete. Error: -85.11620904956206 Total: 15.2223; Orientation: 0.4589; Line Search: 13.5978
<a id="p-13"></a>Iteration 10
<a id="p-12"></a>
Adding measurement 4dbb6a22 to history. Total: 4
Accepted: LBFGS Orientation magnitude: 1.260e+04, gradient 2.962e-02, dot 0.009; [d8aa0abf-3e56-4b5d-ba16-8f218e6f64e2 = 1.000/1.000e+00]
th(0)=-85.11620904956206;dx=-5.5914842249996015
Armijo: th(0.17907364327209802)=-78.23560919987139; dx=23.244978022847995 evalInputDelta=-6.880599849690668
Armijo: th(0.08953682163604901)=-82.05900423342493; dx=20.379772308445794 evalInputDelta=-3.05720481613713
Armijo: th(0.029845607212016335)=-84.2024972722009; dx=18.436247411753307 evalInputDelta=-0.9137117773611578
Armijo: th(0.007461401803004084)=-84.89915318151; dx=17.149603913658215 evalInputDelta=-0.21705586805205712
Armijo: th(0.0014922803606008167)=-85.0729552492668; dx=18.334923602451916 evalInputDelta=-0.04325380029526116
Armijo: th(2.4871339343346947E-4)=-85.10917563543751; dx=17.48417872153619 evalInputDelta=-0.0070334141245496085
Armijo: th(3.5530484776209925E-5)=-85.11520226287429; dx=15.889415396186635 evalInputDelta=-0.001006786687767658
Armijo: th(4.441310597026241E-6)=-85.11608284333667; dx=17.463138794947636 evalInputDelta=-1.2620622538861426E-4
Armijo: th(4.934789552251379E-7)=-85.11619183963325; dx=16.792298544824327 evalInputDelta=-1.7209928813599618E-5
Armijo: th(4.934789552251379E-8)=-85.11620044459809; dx=18.21267292256196 evalInputDelta=-8.604963966263313E-6
Armijo: th(4.4861723202285265E-9)=-85.11620904956206; dx=18.78707930044898 evalInputDelta=0.0
Armijo: th(3.738476933523772E-10)=-85.11620044459809; dx=18.51188373680859 evalInputDelta=-8.604963966263313E-6
MIN ALPHA (2.8757514873259787E-11): th(0.0)=-85.11620904956206
Fitness changed from -85.11620904956206 to -85.11620904956206
Static Iteration Total: 19.3974; Orientation: 0.9666; Line Search: 17.0429
Iteration 11 failed. Error: -85.11620904956206
Previous Error: 0.0 -> -85.11620904956206
Retrying iteration 11
Reset training subject: 11512683982929
Adding measurement 3fd58432 to history. Total: 0
LBFGS Accumulation History: 1 points
th(0)=-85.11620904956206;dx=-8.600153998981432E-4
WOLFE (weak): th(2.1544346900318838E-10)=-85.11620904956206; dx=-7.85677885399896E-4 evalInputDelta=0.0
WOLFE (weak): th(4.3088693800637676E-10)=-85.11620904956206; dx=-8.002471487280746E-4 evalInputDelta=0.0
WOLFE (weak): th(1.2926608140191302E-9)=-85.11620904956206; dx=-8.014399750361622E-4 evalInputDelta=0.0
WOLFE (weak): th(5.170643256076521E-9)=-85.11620904956206; dx=-8.018956381368053E-4 evalInputDelta=0.0
WOLFE (weak): th(2.5853216280382606E-8)=-85.11620904956206; dx=-8.037484933283393E-4 evalInputDelta=0.0
WOLFE (weak): th(1.5511929768229565E-7)=-85.11620904956206; dx=-7.897502682621446E-4 evalInputDelta=0.0
WOLFE (weak): th(1.0858350837760695E-6)=-85.11620904956206; dx=-7.985804383996376E-4 evalInputDelta=0.0
Armijo: th(8.686680670208556E-6)=-85.11620618124083; dx=-7.88621142705133E-4 evalInputDelta=-2.868321232085691E-6
WOLFE (weak): th(4.886257876992313E-6)=-85.11620904956206; dx=-7.941117460461604E-4 evalInputDelta=0.0
Armijo: th(6.7864692736004345E-6)=-85.11620618124083; dx=-8.008555799942456E-4 evalInputDelta=-2.868321232085691E-6
Armijo: th(5.836363575296374E-6)=-85.11620618124083; dx=-8.005412998492939E-4 evalInputDelta=-2.868321232085691E-6
Armijo: th(5.361310726144344E-6)=-85.11620618124083; dx=-7.864186555352245E-4 evalInputDelta=-2.868321232085691E-6
WOLFE (weak): th(5.123784301568328E-6)=-85.11620904956206; dx=-7.962996978348954E-4 evalInputDelta=0.0
WOLFE (weak): th(5.242547513856336E-6)=-85.11620904956206; dx=-7.921712899411134E-4 evalInputDelta=0.0
Armijo: th(5.30192912000034E-6)=-85.11620618124083; dx=-7.904757472029228E-4 evalInputDelta=-2.868321232085691E-6
WOLFE (weak): th(5.2722383169283386E-6)=-85.11620904956206; dx=-7.879042356546317E-4 evalInputDelta=0.0
Armijo: th(5.287083718464339E-6)=-85.11620618124083; dx=-7.966217804806134E-4 evalInputDelta=-2.868321232085691E-6
Armijo: th(5.279661017696339E-6)=-85.11620618124083; dx=-8.006642051497459E-4 evalInputDelta=-2.868321232085691E-6
WOLFE (weak): th(5.275949667312339E-6)=-85.11620904956206; dx=-7.800266064595563E-4 evalInputDelta=0.0
WOLFE (weak): th(5.277805342504339E-6)=-85.11620904956206; dx=-7.990475161550356E-4 evalInputDelta=0.0
WOLFE (weak): th(5.278733180100339E-6)=-85.11620904956206; dx=-7.988753355838461E-4 evalInputDelta=0.0
Armijo: th(5.279197098898339E-6)=-85.11620618124083; dx=-7.870019285988292E-4 evalInputDelta=-2.868321232085691E-6
WOLFE (weak): th(5.278965139499339E-6)=-85.11620904956206; dx=-7.770702698572589E-4 evalInputDelta=0.0
WOLFE (weak): th(5.279081119198839E-6)=-85.11620904956206; dx=-8.005143512596808E-4 evalInputDelta=0.0
WOLFE (weak): th(5.279139109048589E-6)=-85.11620904956206; dx=-7.987836999619568E-4 evalInputDelta=0.0
WOLFE (weak): th(5.279168103973464E-6)=-85.11620904956206; dx=-7.923676733845003E-4 evalInputDelta=0.0
mu ~= nu (5.279168103973464E-6): th(0.0)=-85.11620904956206
Fitness changed from -85.11620904956206 to -85.11620904956206
Static Iteration Total: 36.8606; Orientation: 0.0172; Line Search: 34.4753
Iteration 12 failed. Error: -85.11620904956206
Previous Error: 0.0 -> -85.11620904956206
Optimization terminated 12
Final threshold in iteration 12: -85.11620904956206 (> -Infinity) after 127.790s (< 3600.000s)
Returns
-85.11620904956206